USB: gadget: Fix race condition to over come memory corruption

With dynamic composition change, it has been observed that port
structure is being accessed after it is being freed. Hence it
creates memory corruption and device is crashing with the same.
This change avoid port access after it is being freed to ovecome
this issue.

CRs-Fixed: 636780
Change-Id: Ibc2792791cecbcea63edeba864df7b7da73b5f40
Signed-off-by: Mayank Rana <mrana@codeaurora.org>