leaudio: Improve cleanup function

Make sure device is removed from background connect when cleanup
function is called

Bug: 331775328
Flag: Exempt, regression tested with unit tests
Test: atest bluetooth_le_audio_client_test
Change-Id: Ifaaf02c6e35f536ffbdfbe4d692934d658be4a69