Optimize onAttachedTo/onDetachedFromWindow

Apps detaching/attaching large subtrees would waste a few milliseconds
dealing with dirty display lists. This change removes the need to do
ArrayList.remove() on every detachedFromWindow().

Change-Id: Icee72516c40d48ff0fd9d6f3128589f99bf61428