usb: gadget: f_midi: remove useless midi reference from port struct

remove a field which is unnecessary. No functional changes.

Signed-off-by: Felipe F. Tonello <eu@felipetonello.com>
Signed-off-by: Michal Nazarewicz <mina86@mina86.com>
Signed-off-by: Felipe Balbi <balbi@kernel.org>