drm/vmwgfx: Use modeset display memory validation for layout ioctl

Call the same display memory validation function which is used by
modeset_check. This ensure consistency that kernel change preferred
mode/topology only if supported.

Also change the internal function to use drm_rect instead of
drm_vmw_rect.

Signed-off-by: Deepak Rawat <drawat@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>