Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 501e7ef5 authored by Eric Dumazet's avatar Eric Dumazet Committed by David S. Miller
Browse files

net-rfs: fix false sharing accessing sd->input_queue_head 



sd->input_queue_head is incremented for each processed packet
in process_backlog(), and read from other cpus performing
Out Of Order avoidance in get_rps_cpu()

Moving this field in a separate cache line keeps it mostly
hot for the cpu in process_backlog(), as other cpus will
only read it.

In a stress test, process_backlog() was consuming 6.80 % of cpu cycles,
and the patch reduced the cost to 0.65 %

Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
Acked-by: default avatarTom Herbert <tom@herbertland.com>
Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parent 35ef7d68

include/linux/netdevice.h

+6 −2
Original line number Diff line number Diff line
@@ -2747,11 +2747,15 @@ struct softnet_data { 
	struct sk_buff		*completion_queue;



#ifdef CONFIG_RPS

	/* Elements below can be accessed between CPUs for RPS */

	/* input_queue_head should be written by cpu owning this struct,

	 * and only read by other cpus. Worth using a cache line.

	 */

	unsigned int		input_queue_head ____cacheline_aligned_in_smp;



	/* Elements below can be accessed between CPUs for RPS/RFS */

	struct call_single_data	csd ____cacheline_aligned_in_smp;

	struct softnet_data	*rps_ipi_next;

	unsigned int		cpu;

	unsigned int		input_queue_head;

	unsigned int		input_queue_tail;

#endif

	unsigned int		dropped;