usb: dwc3: gadget: avoid memcpy()ing event buffer

We're only using the 4 byte events and memcpy()
will make us slower. We can easily avoid that.

Signed-off-by: Felipe Balbi <balbi@ti.com>