Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit a9b204d1 authored by Eric Dumazet's avatar Eric Dumazet Committed by David S. Miller
Browse files

tcp: tsq: avoid one atomic in tcp_wfree() 



Under high load, tcp_wfree() has an atomic operation trying
to schedule a tasklet over and over.

We can schedule it only if our per cpu list was empty.

Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parent b223feb9

net/ipv4/tcp_output.c

+4 −1
Original line number Diff line number Diff line
@@ -880,6 +880,7 @@ void tcp_wfree(struct sk_buff *skb) 


	for (oval = READ_ONCE(tp->tsq_flags);; oval = nval) {

		struct tsq_tasklet *tsq;

		bool empty;



		if (!(oval & TSQF_THROTTLED) || (oval & TSQF_QUEUED))

			goto out;
@@ -892,7 +893,9 @@ void tcp_wfree(struct sk_buff *skb) 
		/* queue this socket to tasklet queue */

		local_irq_save(flags);

		tsq = this_cpu_ptr(&tsq_tasklet);

		empty = list_empty(&tsq->head);

		list_add(&tp->tsq_node, &tsq->head);

		if (empty)

			tasklet_schedule(&tsq->tasklet);

		local_irq_restore(flags);

		return;