usb: don't enter usb subdirectories directly

Instead, make we enter usb/ directory on all
needed cases and enter the subdirectories from
drivers/usb/Makefile.

Signed-off-by: Felipe Balbi <balbi@ti.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>