Don't show "set default" for directory contacts

Directory contacts don't allow setting (or clearing) of defaults
so don't populate those actions on the context menu.

Bug: 6563335
Change-Id: Ib96af2dd875c5fefe771360b05df39c01bec7c38