Remove conference participants from cached list

- when a conference call is disconnected, remove the
participants from the cached list of participants and
their connect time

Change-Id: I37792e6d093a6ee5c66ea7ef86f6755eb1e513d1
Bug: 143468041