Properly set the tap down time when entering non-tap modes

This prevents us from erroneously generating tap gestures on things like quick
button presses.

Change-Id: I708e7df9341df589927fd2c0e15c202ccb30ba9b