LeAudio: Improve audio stream context mixing logic

Bidirectional scenarios should have the audio context aligned
in both directions even if the metadata did come from AF only
on one direction.

Additionally UNSPECIFIED should not be mixed with other contexts.

Bug: 269372008
Test: atest --host bluetooth_le_audio_client_test bluetooth_le_audio_test --no-bazel-mode
Test: manual testing
Change-Id: I8613df78c9ec612fa6eabe04770a834f1c41d226