phy: xgene: Remove duplicate code to set ctx->dev

Set it once is enough and it's done after devm_kzalloc().

Signed-off-by: Axel Lin <axel.lin@ingics.com>
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>