Commit b402b73b authored Jun 29, 2012 by Dimitri Sivanich Committed by Paul E. McKenney Sep 23, 2012

rcu: Segregate rcu_state fields to improve cache locality



The fields in the rcu_state structure that are protected by the
root rcu_node structure's ->lock can share a cache line with the
fields protected by ->onofflock.  This can result in excessive
memory contention on large systems, so this commit applies
____cacheline_internodealigned_in_smp to the ->onofflock field in
order to segregate them.

Signed-off-by: Dimitri Sivanich <sivanich@sgi.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Dimitri Sivanich <sivanich@sgi.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>

parent b626c1b6

kernel/rcutree.h

+2 −1

Original line number	Diff line number	Diff line
		@@ -394,7 +394,8 @@ struct rcu_state {

		/* End of fields guarded by root rcu_node's lock. */

		raw_spinlock_t onofflock; /* exclude on/offline and */
		raw_spinlock_t onofflock ____cacheline_internodealigned_in_smp;
		/* exclude on/offline and */
		/* starting new GP. */
		struct rcu_head orphan_nxtlist; / Orphaned callbacks that */
		/* need a grace period. */