Revert "mm: allow vmas with vm_ops to be speculatively handled"

This reverts commit b37bae60.

Reason for revert: remove SPF non upstream code
Bug: 140544941
Test: boot
Change-Id: I466435dabfed767085934109a43bf7ca3da855a3
Signed-off-by: Martin Liu <liumartin@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>