RDMA/mlx5: Refactor QP type check to be as early as possible

Perform QP type check in one place and fail as early as possible.

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>