clean-up client management.

simplified things a lot, the biggest change is that the concept
of "ClientID" is now gone, instead we simply use references.

Change-Id: Icbc57f80865884aa5f35ad0d0a0db26f19f9f7ce