Don't remove Activities and Tasks until animation done

Just like stacks and displays, activities and tasks need to stick
around until animations have completed.

Change-Id: I54fe8f6855d60cbc3a25cbc6e762defd5ac50bf5