Fix for 2155255: Revert "Cache inflated view to avoid inflation of layouts in preferences"

This introduces problems when scrolling through preference screens, due
to a clash in this cache versus the ListView's cache.

This reverts commit 01dbc2ed.