Revert "Instantiate InputMethodManager for each display"

This reverts commit c53d78e9.

Reason for revert:
Caused performance regression in
LatencyTests#testExpandNotificationsLatency.

Fix: 117434607
Bug: 111364446
Bug: 115893206
Test: atest google/perf/app-transition/sysui-latency-test-trace
Change-Id: If0d7a1b8f6d126d5a7c384ec4c2ff44260b8c35f