msm: sde: update rotator OT to use total pixels

Currently, OT calculation compares resolution and frame
rate separately, which may select inappropriate OT
according to overall pixels. Update rotator OT
calculation to combine resolution and frame rate into
total pixel before comparison. Also, apply the same
same OT calculation for RGB formats as well.

Change-Id: Id7e4db53c55b5af61ca45ade88ecd3318f6eef15
Signed-off-by: Alan Kwong <akwong@codeaurora.org>