Commit 838b6fd2 authored Jan 23, 2019 by Kaike Wan Committed by Doug Ledford Feb 05, 2019

IB/hfi1: TID RDMA RcvArray programming and TID allocation

TID entries are used by hfi1 hardware to receive data payload from
incoming packets directly into a user buffer and thus avoid data copying
by software. This patch implements the functions for TID allocation,
freeing, and programming TID RcvArray entries in hardware for kernel
clients. TID entries are managed via lists of TID groups similar to PSM.
Furthermore, to track TID resource allocation for each request, software
flows are also allocated and freed as needed. Since software flows
consume large amount of memory for tracking TID allocation and freeing,
it is generally desirable to allocate them dynamically in the send queue
and only for TID RDMA requests, but pre-allocate them for receive queue
because the send queue could have thousands of entries while the receive
queue has only a limited number of entries.

Signed-off-by: Mitko Haralanov <mitko.haralanov@intel.com>
Signed-off-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Kaike Wan <kaike.wan@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@intel.com>
Signed-off-by: Doug Ledford <dledford@redhat.com>

parent 37356e78

drivers/infiniband/hw/hfi1/hfi.h

+2 −0

Original line number	Diff line number	Diff line
		@@ -303,6 +303,8 @@ struct hfi1_ctxtdata {
		spinlock_t exp_lock;
		/* Queue for QP's waiting for HW TID flows */
		struct tid_queue flow_queue;
		/* Queue for QP's waiting for HW receive array entries */
		struct tid_queue rarr_queue;
		/* when waiting for rcv or pioavail */
		wait_queue_head_t wait;
		/* uuid from PSM */

drivers/infiniband/hw/hfi1/init.c

+2 −1

Original line number	Diff line number	Diff line
		@@ -372,6 +372,7 @@ int hfi1_create_ctxtdata(struct hfi1_pportdata *ppd, int numa,
		mutex_init(&rcd->exp_mutex);
		spin_lock_init(&rcd->exp_lock);
		INIT_LIST_HEAD(&rcd->flow_queue.queue_head);
		INIT_LIST_HEAD(&rcd->rarr_queue.queue_head);

		hfi1_cdbg(PROC, "setting up context %u\n", rcd->ctxt);

		@@ -1596,7 +1597,7 @@ static void cleanup_device_data(struct hfi1_devdata *dd)
		struct hfi1_ctxtdata *rcd = dd->rcd[ctxt];

		if (rcd) {
		hfi1_clear_tids(rcd);
		hfi1_free_ctxt_rcv_groups(rcd);
		hfi1_free_ctxt(rcd);
		}
		}