Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit a3a0e43f authored by Jens Axboe's avatar Jens Axboe
Browse files

io_uring: don't enter poll loop if we have CQEs pending 



We need to check if we have CQEs pending before starting a poll loop,
as those could be the events we will be spinning for (and hence we'll
find none). This can happen if a CQE triggers an error, or if it is
found by eg an IRQ before we get a chance to find it through polling.

Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
parent cb32de1b

fs/io_uring.c

+15 −7
Original line number Diff line number Diff line
@@ -679,6 +679,13 @@ static void io_put_req(struct io_kiocb *req) 
		io_free_req(req);

}



static unsigned io_cqring_events(struct io_cq_ring *ring)

{

	/* See comment at the top of this file */

	smp_rmb();

	return READ_ONCE(ring->r.tail) - READ_ONCE(ring->r.head);

}



/*

 * Find and free completed poll iocbs

 */
@@ -818,6 +825,14 @@ static int io_iopoll_check(struct io_ring_ctx *ctx, unsigned *nr_events, 
	do {

		int tmin = 0;



		/*

		 * Don't enter poll loop if we already have events pending.

		 * If we do, we can potentially be spinning for commands that

		 * already triggered a CQE (eg in error).

		 */

		if (io_cqring_events(ctx->cq_ring))

			break;



		/*

		 * If a submit got punted to a workqueue, we can have the

		 * application entering polling for a command before it gets
@@ -2449,13 +2464,6 @@ static int io_ring_submit(struct io_ring_ctx *ctx, unsigned int to_submit) 
	return submit;

}



static unsigned io_cqring_events(struct io_cq_ring *ring)

{

	/* See comment at the top of this file */

	smp_rmb();

	return READ_ONCE(ring->r.tail) - READ_ONCE(ring->r.head);

}



/*

 * Wait until events become available, if we don't already have some. The

 * application must reap them itself, as they reside on the shared cq ring.