Remove portalToDisplay functionality

There should not be any more users of the portal to display
functionality. We can now delete this unused code.

Bug: 167946720
Test: atest inputflinger_tests
Change-Id: I9e9a2e7d6c3e5d5fe969ae6207f7c1f2db169f04