Don't remove controls prematurely

The test for whether controls can resume can happen while the session is
active, so we don't want to remove them yet if that's the case

Bug: 151103474
Test: manual - play app that can't resume (e.g. GPM) verify controls show,
then pause and swipe from recents, verify controls are removed

Change-Id: I1630feddf7d367299fc7488ddeb376d5fc96748f