Commit c4bfa3f5 authored Jun 17, 2015 by Peter Zijlstra Committed by Thomas Gleixner Jun 19, 2015

seqcount: Introduce raw_write_seqcount_barrier()



Introduce raw_write_seqcount_barrier(), a new construct that can be
used to provide write barrier semantics in seqcount read loops instead
of the usual consistency guarantee.

raw_write_seqcount_barier() is equivalent to:

	raw_write_seqcount_begin();
	raw_write_seqcount_end();

But avoids issueing two back-to-back smp_wmb() instructions.

This construct works because the read side will 'stall' when observing
odd values. This means that -- referring to the example in the comment
below -- even though there is no (matching) read barrier between the
loads of X and Y, we cannot observe !x && !y, because:

 - if we observe Y == false we must observe the first sequence
   increment, which makes us loop, until

 - we observe !(seq & 1) -- the second sequence increment -- at which
   time we must also observe T == true.

Suggested-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: umgwanakikbuti@gmail.com
Cc: ktkhai@parallels.com
Cc: rostedt@goodmis.org
Cc: juri.lelli@gmail.com
Cc: pang.xunlei@linaro.org
Cc: oleg@redhat.com
Cc: wanpeng.li@linux.intel.com
Cc: Al Viro <viro@ZenIV.linux.org.uk>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/20150617122924.GP3644@twins.programming.kicks-ass.net


Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

parent a7c6f571

include/linux/seqlock.h

+41 −0

Original line number	Diff line number	Diff line
		@@ -233,6 +233,47 @@ static inline void raw_write_seqcount_end(seqcount_t *s)
		s->sequence++;
		}

		/**
		* raw_write_seqcount_barrier - do a seq write barrier
		* @s: pointer to seqcount_t
		*
		* This can be used to provide an ordering guarantee instead of the
		* usual consistency guarantee. It is one wmb cheaper, because we can
		* collapse the two back-to-back wmb()s.
		*
		* seqcount_t seq;
		* bool X = true, Y = false;
		*
		* void read(void)
		* {
		* bool x, y;
		*
		* do {
		* int s = read_seqcount_begin(&seq);
		*
		* x = X; y = Y;
		*
		* } while (read_seqcount_retry(&seq, s));
		*
		* BUG_ON(!x && !y);
		* }
		*
		* void write(void)
		* {
		* Y = true;
		*
		* raw_write_seqcount_barrier(seq);
		*
		* X = false;
		* }
		*/
		static inline void raw_write_seqcount_barrier(seqcount_t *s)
		{
		s->sequence++;
		smp_wmb();
		s->sequence++;
		}

		/*
		* raw_write_seqcount_latch - redirect readers to even/odd copy
		* @s: pointer to seqcount_t