Always remove IconState in onViewRemoved, even if animations are off

Recent changes to avoid animating added a case where we can remove views
but not remove the actual IconState for that view, causing a leak

Fixes: 135180058
Test: heap dump?
Change-Id: If3e3924348a606484ac0968f644b4895871bd406