Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 1a991488 authored by Eric Dumazet's avatar Eric Dumazet Committed by David S. Miller
Browse files

tcp: batch calls to sk_flush_backlog() 



Starting from commit d41a69f1 ("tcp: make tcp_sendmsg() aware of socket backlog")
loopback flows got hurt, because for each skb sent, the socket receives an
immediate ACK and sk_flush_backlog() causes extra work.

Intent was to not let the backlog grow too much, but we went a bit too far.

We can check the backlog every 16 skbs (about 1MB chunks)
to increase TCP over loopback performance by about 15 %

Note that the call to sk_flush_backlog() handles a single ACK,
thanks to coalescing done on backlog, but cleans the 16 skbs
found in rtx rb-tree.

Reported-by: default avatarSoheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
Acked-by: default avatarSoheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parent fcc32a21

net/ipv4/tcp.c

+6 −5
Original line number Diff line number Diff line
@@ -1162,7 +1162,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) 
	struct sockcm_cookie sockc;

	int flags, err, copied = 0;

	int mss_now = 0, size_goal, copied_syn = 0;

	bool process_backlog = false;

	int process_backlog = 0;

	bool zc = false;

	long timeo;
@@ -1254,8 +1254,9 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) 
			if (!sk_stream_memory_free(sk))

				goto wait_for_sndbuf;



			if (process_backlog && sk_flush_backlog(sk)) {

				process_backlog = false;

			if (unlikely(process_backlog >= 16)) {

				process_backlog = 0;

				if (sk_flush_backlog(sk))

					goto restart;

			}

			first_skb = tcp_rtx_and_write_queues_empty(sk);
@@ -1264,7 +1265,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) 
			if (!skb)

				goto wait_for_memory;



			process_backlog = true;

			process_backlog++;

			skb->ip_summed = CHECKSUM_PARTIAL;



			skb_entail(sk, skb);