PatchPanel: fix deadlock when releasing a patch

Commit 1e28aaa6 was incomplete in that it only addressed a potential deadlock
when creating an audio patch but the same problem exists when releasing
an audio patch. This change fixes the release audio patch sequence.

Bug: 276386807
Test: repro steps in the bug
Change-Id: I3615f22d4f0a4060b1397d6639625acbc18fbe86