Skip config-at-end sync if activity removed

If an Activity is removed before we commit config-at-end,
it won't have a surface. In this case we can skip sync but
still need to clean-up config-at-end state.

Bug: 385917347
Test: remove activity before cfg-at-end finishes. This mostly
      happens during automated tests, so monitor dashboards.
Flag: EXEMPT bugfix
Change-Id: I16610b69cb0c9a85134458377689205a9cb843c9