msm: mdss: add support to validate pixel extension calculations

If a layer needs a scaling than its pixel extension values are calculated
in the user-space library and passed down to driver. Currently driver
blindly programs these settings to HW without validating for correctness.
This leads to HW hangs and stability issues. Add a proper sanitization
checks for pixel extension and report error back to user-space if any.

Change-Id: I086cf0b23fbccd37a23bb509f067bffdf396c247
Signed-off-by: Ujwal Patel <ujwalp@codeaurora.org>
Signed-off-by: Jeevan Shriram <jshriram@codeaurora.org>