Revert "usb: gadget: f_qdss: Fix unclocked access during connect in qdss mode"

This reverts commit 0d4e77a2
With this QDSS unclocked access fix, SMMU fault issues are
reported. Revert this change so that this fault can be
avoided.

CRs-Fixed: 714562
Change-Id: Ied0e681803adc6b2b969cfd43ce850ee8cedbbcc
Signed-off-by: Sujeet Kumar <ksujeet@codeaurora.org>