Revert "Remove activity from task if not set properly."

This reverts commit 0f9bde84.

Reason for revert: causing b/35879503

Change-Id: I06402356ecdb559dcb8342f3a6cca755aea6533a
Fixes: 35879503
Bug: 35034729
Bug: 35035258
Bug: 34179495
Test: manual with repro steps