Don't crash when we get a drag-ended after being detached

Removing view groups during a drag would cause an NPE here.  The
child views will still get the drag-ended event as expected.

Bug 8298439

Change-Id: I665ba1d139e8aaa820f5f515186bc4ce8bcdb1ea