Skip the VDM call if running on the default device.

The call is redundant and is causing a performance regression.

We still need to call VDM if the device id association is not
the defult device but in the vast majority of cases it will be
skipped.

Test: presubmit
Fix: 316962296
Change-Id: I9e95d53dcdaf5fe0992d5fa2ec18933d0c005a1c