leaudio: Remove reduntant check

This check is wrong and will always pass it thru

Bug: 331775328
Test: atest bluetooth_le_audio_client_test
Flag: Exempt, trivial fix, regression tested with unit tests
Change-Id: I3976c9983c9e1612ffa5b4a6dda7527a8c0d81c9