Fix crash when separating multiple empty raw contacts

This changes the back reference logic a little bit to prevent
adding back references where there is nothing to back reference.
This fixes weird cases where a user could hit "new contact" twice
without saving data and then try to separate the two dummy
contacts.

Change-Id: I1a6b4e1a6bda99f5d8ee07ea57cace75a6bed4ab