Ensure we only remove messages for the same app info

Bug: 130151718
Bug: 132179354
Test: Switch between gesture nav modes, ensure that system and sysui are
      in the right state

Change-Id: I364d155292fc378af552b6ce9e6acca879224693