Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 0f580386 authored by Thomas Hellstrom's avatar Thomas Hellstrom Committed by Sinclair Yeh
Browse files

drm/vmwgfx: Fix a potential integer overflow 



Found by coverity.

Signed-off-by: default avatarThomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: default avatarSinclair Yeh <syeh@vmware.com>
Reviewed-by: default avatarCharmaine Lee <charmainel@vmware.com>
parent 0fd67c1c

drivers/gpu/drm/vmwgfx/vmwgfx_kms.c

+1 −1
Original line number Diff line number Diff line
@@ -1671,7 +1671,7 @@ int vmw_kms_update_layout_ioctl(struct drm_device *dev, void *data, 
		 *	1. Bounding box (assuming 32bpp) must be < prim_bb_mem

		 *      2. Total pixels (assuming 32bpp) must be < prim_bb_mem

		 */

		u64 bb_mem    = bounding_box.w * bounding_box.h * 4;

		u64 bb_mem    = (u64) bounding_box.w * bounding_box.h * 4;

		u64 pixel_mem = total_pixels * 4;



		if (bb_mem > dev_priv->prim_bb_mem) {