usb: gadget: Fix code quality issues when accessing port

Fix code quality issues found by the automatic checking tool.
Verify that port and port_usb are valid before accessing them.

CRs-Fixed: 566927
Change-Id: Ie1971426ef3756c2444797c5f4fd5717bea1f8b6
Signed-off-by: Danny Segal <dsegal@codeaurora.org>
Signed-off-by: Maya Erez <merez@codeaurora.org>