drm/radeon: add a check for allocation failure (v2)

We can easily return -ENOMEM here if kzalloc() fails.

v2: agd5f: drop the vm mutex

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>