Fix summary lifecycle

Once more and probably not for all

Don't let multiple listening state changes get queued, this can cause
rapid changes like listening->not listening->listening.  Also store
listening state on the worker thread now that we drop some state
messages and only notify summaries when there is a state change.

Change-Id: I93a5f364e9b35929f9088e044ebefd9be69740e6
Fixes: 28319383