Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 986ef95e authored by Sagi Grimberg's avatar Sagi Grimberg Committed by Doug Ledford
Browse files

IB/mlx5: Expose correct max_sge_rd limit 



mlx5 devices (Connect-IB, ConnectX-4, ConnectX-4-LX) has a limitation
where rdma read work queue entries cannot exceed 512 bytes.
A rdma_read wqe needs to fit in 512 bytes:
- wqe control segment (16 bytes)
- rdma segment (16 bytes)
- scatter elements (16 bytes each)

So max_sge_rd should be: (512 - 16 - 16) / 16 = 30.

Cc: linux-stable@vger.kernel.org
Reported-by: default avatarChristoph Hellwig <hch@lst.de>
Tested-by: default avatarChristoph Hellwig <hch@lst.de>
Signed-off-by: default avatarSagi Grimberg <sagig@grimberg.me>
Signed-off-by: default avatarLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
parent 32cc92c7

drivers/infiniband/hw/mlx5/main.c

+1 −1
Original line number Diff line number Diff line
@@ -530,7 +530,7 @@ static int mlx5_ib_query_device(struct ib_device *ibdev, 
		     sizeof(struct mlx5_wqe_ctrl_seg)) /

		     sizeof(struct mlx5_wqe_data_seg);

	props->max_sge = min(max_rq_sg, max_sq_sg);

	props->max_sge_rd = props->max_sge;

	props->max_sge_rd	   = MLX5_MAX_SGE_RD;

	props->max_cq		   = 1 << MLX5_CAP_GEN(mdev, log_max_cq);

	props->max_cqe = (1 << MLX5_CAP_GEN(mdev, log_max_cq_sz)) - 1;

	props->max_mr		   = 1 << MLX5_CAP_GEN(mdev, log_max_mkey);

include/linux/mlx5/device.h

+11 −0
Original line number Diff line number Diff line
@@ -392,6 +392,17 @@ enum { 
	MLX5_CAP_OFF_CMDIF_CSUM		= 46,

};



enum {

	/*

	 * Max wqe size for rdma read is 512 bytes, so this

	 * limits our max_sge_rd as the wqe needs to fit:

	 * - ctrl segment (16 bytes)

	 * - rdma segment (16 bytes)

	 * - scatter elements (16 bytes each)

	 */

	MLX5_MAX_SGE_RD	= (512 - 16 - 16) / 16

};



struct mlx5_inbox_hdr {

	__be16		opcode;

	u8		rsvd[4];