Prevent ConcurrentModificationExceptions

Use sets backed by ConcurrentHashMaps instead of HashSets, and
CopyOnWriteArrayLists instead of ArrayLists, to prevent concurrent
exceptions if listeners try to remove themselves in callbacks while
iterating over the listeners.

Bug:16325026
Change-Id: I55e081eda6ba19fa466bbf019c648bbdaf833c33