Fix the bug that quick contact cannot be removed.

The "removed" button doesn't received drag and drop event since it's not
visible at the time the drag event starts. The workaround is to change
the visibility of its child view instead.

Change-Id: Iada912fcc36983ad758296f33092cc54ad6f9f5e
Fix: 28479593
(cherry-picked from b20b76437ae6aae8fe4da64097351cc80f030919)
(cherry picked from commit 22a193b7babae99256149a0536748c6ad9c6b943)