Fixing the issue where empty contacts could not be joined

Basically if you did not have any _displayable_ items on a contact,
you could not merge it with another contact.

Change-Id: I58807cc25390bce4edb5fb0870641920281571f9