Fix issue where surface corner rounding is ignored

It's not correct to only check the diagonal of the transform matrix
for scaling. The scale will be distributed across four components
when the layer is rotated.

Test: manual
Test: atest LayerTypeAndRenderTypeTransaction
Fixes: 147415720
Change-Id: I140b373efd7fad705d0cd54aa6e86b4142e190e5