Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 85341c61 authored by Linus Torvalds's avatar Linus Torvalds
Browse files

Merge branch 'core-fixes-for-linus' of...

Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip

* 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip:
  rcu: Make RCU lockdep check the lockdep_recursion variable
  rcu: Update docs for rcu_access_pointer and rcu_dereference_protected
  rcu: Better explain the condition parameter of rcu_dereference_check()
  rcu: Add rcu_access_pointer and rcu_dereference_protected
parents 375db481 bc293d62
Loading
Loading
Loading
Loading
+22 −17
Original line number Diff line number Diff line
@@ -34,7 +34,7 @@ NMI handler.
		cpu = smp_processor_id();
		++nmi_count(cpu);

		if (!rcu_dereference(nmi_callback)(regs, cpu))
		if (!rcu_dereference_sched(nmi_callback)(regs, cpu))
			default_do_nmi(regs);

		nmi_exit();
@@ -47,12 +47,13 @@ function pointer. If this handler returns zero, do_nmi() invokes the
default_do_nmi() function to handle a machine-specific NMI.  Finally,
preemption is restored.

Strictly speaking, rcu_dereference() is not needed, since this code runs
only on i386, which does not need rcu_dereference() anyway.  However,
it is a good documentation aid, particularly for anyone attempting to
do something similar on Alpha.
In theory, rcu_dereference_sched() is not needed, since this code runs
only on i386, which in theory does not need rcu_dereference_sched()
anyway.  However, in practice it is a good documentation aid, particularly
for anyone attempting to do something similar on Alpha or on systems
with aggressive optimizing compilers.

Quick Quiz:  Why might the rcu_dereference() be necessary on Alpha,
Quick Quiz:  Why might the rcu_dereference_sched() be necessary on Alpha,
	     given that the code referenced by the pointer is read-only?


@@ -99,17 +100,21 @@ invoke irq_enter() and irq_exit() on NMI entry and exit, respectively.

Answer to Quick Quiz

	Why might the rcu_dereference() be necessary on Alpha, given
	Why might the rcu_dereference_sched() be necessary on Alpha, given
	that the code referenced by the pointer is read-only?

	Answer: The caller to set_nmi_callback() might well have
		initialized some data that is to be used by the
		new NMI handler.  In this case, the rcu_dereference()
		would be needed, because otherwise a CPU that received
		an NMI just after the new handler was set might see
		the pointer to the new NMI handler, but the old
		pre-initialized version of the handler's data.

		More important, the rcu_dereference() makes it clear
		to someone reading the code that the pointer is being
		protected by RCU.
		initialized some data that is to be used by the new NMI
		handler.  In this case, the rcu_dereference_sched() would
		be needed, because otherwise a CPU that received an NMI
		just after the new handler was set might see the pointer
		to the new NMI handler, but the old pre-initialized
		version of the handler's data.

		This same sad story can happen on other CPUs when using
		a compiler with aggressive pointer-value speculation
		optimizations.

		More important, the rcu_dereference_sched() makes it
		clear to someone reading the code that the pointer is
		being protected by RCU-sched.
+4 −3
Original line number Diff line number Diff line
@@ -260,7 +260,8 @@ over a rather long period of time, but improvements are always welcome!
	The reason that it is permissible to use RCU list-traversal
	primitives when the update-side lock is held is that doing so
	can be quite helpful in reducing code bloat when common code is
	shared between readers and updaters.
	shared between readers and updaters.  Additional primitives
	are provided for this case, as discussed in lockdep.txt.

10.	Conversely, if you are in an RCU read-side critical section,
	and you don't hold the appropriate update-side lock, you -must-
@@ -344,8 +345,8 @@ over a rather long period of time, but improvements are always welcome!
	requiring SRCU's read-side deadlock immunity or low read-side
	realtime latency.

	Note that, rcu_assign_pointer() and rcu_dereference() relate to
	SRCU just as they do to other forms of RCU.
	Note that, rcu_assign_pointer() relates to SRCU just as they do
	to other forms of RCU.

15.	The whole point of call_rcu(), synchronize_rcu(), and friends
	is to wait until all pre-existing readers have finished before
+26 −2
Original line number Diff line number Diff line
@@ -32,9 +32,20 @@ checking of rcu_dereference() primitives:
	srcu_dereference(p, sp):
		Check for SRCU read-side critical section.
	rcu_dereference_check(p, c):
		Use explicit check expression "c".
		Use explicit check expression "c".  This is useful in
		code that is invoked by both readers and updaters.
	rcu_dereference_raw(p)
		Don't check.  (Use sparingly, if at all.)
	rcu_dereference_protected(p, c):
		Use explicit check expression "c", and omit all barriers
		and compiler constraints.  This is useful when the data
		structure cannot change, for example, in code that is
		invoked only by updaters.
	rcu_access_pointer(p):
		Return the value of the pointer and omit all barriers,
		but retain the compiler constraints that prevent duplicating
		or coalescsing.  This is useful when when testing the
		value of the pointer itself, for example, against NULL.

The rcu_dereference_check() check expression can be any boolean
expression, but would normally include one of the rcu_read_lock_held()
@@ -59,7 +70,20 @@ In case (1), the pointer is picked up in an RCU-safe manner for vanilla
RCU read-side critical sections, in case (2) the ->file_lock prevents
any change from taking place, and finally, in case (3) the current task
is the only task accessing the file_struct, again preventing any change
from taking place.
from taking place.  If the above statement was invoked only from updater
code, it could instead be written as follows:

	file = rcu_dereference_protected(fdt->fd[fd],
					 lockdep_is_held(&files->file_lock) ||
					 atomic_read(&files->count) == 1);

This would verify cases #2 and #3 above, and furthermore lockdep would
complain if this was used in an RCU read-side critical section unless one
of these two cases held.  Because rcu_dereference_protected() omits all
barriers and compiler constraints, it generates better code than do the
other flavors of rcu_dereference().  On the other hand, it is illegal
to use rcu_dereference_protected() if either the RCU-protected pointer
or the RCU-protected data that it points to can change concurrently.

There are currently only "universal" versions of the rcu_assign_pointer()
and RCU list-/tree-traversal primitives, which do not (yet) check for
+6 −0
Original line number Diff line number Diff line
@@ -840,6 +840,12 @@ SRCU: Initialization/cleanup
	init_srcu_struct
	cleanup_srcu_struct

All:  lockdep-checked RCU-protected pointer access

	rcu_dereference_check
	rcu_dereference_protected
	rcu_access_pointer

See the comment headers in the source code (or the docbook generated
from them) for more information.

+56 −9
Original line number Diff line number Diff line
@@ -101,10 +101,7 @@ extern struct lockdep_map rcu_sched_lock_map;
# define rcu_read_release_sched() \
		lock_release(&rcu_sched_lock_map, 1, _THIS_IP_)

static inline int debug_lockdep_rcu_enabled(void)
{
	return likely(rcu_scheduler_active && debug_locks);
}
extern int debug_lockdep_rcu_enabled(void);

/**
 * rcu_read_lock_held - might we be in RCU read-side critical section?
@@ -195,12 +192,30 @@ static inline int rcu_read_lock_sched_held(void)

/**
 * rcu_dereference_check - rcu_dereference with debug checking
 * @p: The pointer to read, prior to dereferencing
 * @c: The conditions under which the dereference will take place
 *
 * Do an rcu_dereference(), but check that the conditions under which the
 * dereference will take place are correct.  Typically the conditions indicate
 * the various locking conditions that should be held at that point.  The check
 * should return true if the conditions are satisfied.
 *
 * For example:
 *
 *	bar = rcu_dereference_check(foo->bar, rcu_read_lock_held() ||
 *					      lockdep_is_held(&foo->lock));
 *
 * Do an rcu_dereference(), but check that the context is correct.
 * For example, rcu_dereference_check(gp, rcu_read_lock_held()) to
 * ensure that the rcu_dereference_check() executes within an RCU
 * read-side critical section.  It is also possible to check for
 * locks being held, for example, by using lockdep_is_held().
 * could be used to indicate to lockdep that foo->bar may only be dereferenced
 * if either the RCU read lock is held, or that the lock required to replace
 * the bar struct at foo->bar is held.
 *
 * Note that the list of conditions may also include indications of when a lock
 * need not be held, for example during initialisation or destruction of the
 * target struct:
 *
 *	bar = rcu_dereference_check(foo->bar, rcu_read_lock_held() ||
 *					      lockdep_is_held(&foo->lock) ||
 *					      atomic_read(&foo->usage) == 0);
 */
#define rcu_dereference_check(p, c) \
	({ \
@@ -209,12 +224,44 @@ static inline int rcu_read_lock_sched_held(void)
		rcu_dereference_raw(p); \
	})

/**
 * rcu_dereference_protected - fetch RCU pointer when updates prevented
 *
 * Return the value of the specified RCU-protected pointer, but omit
 * both the smp_read_barrier_depends() and the ACCESS_ONCE().  This
 * is useful in cases where update-side locks prevent the value of the
 * pointer from changing.  Please note that this primitive does -not-
 * prevent the compiler from repeating this reference or combining it
 * with other references, so it should not be used without protection
 * of appropriate locks.
 */
#define rcu_dereference_protected(p, c) \
	({ \
		if (debug_lockdep_rcu_enabled() && !(c)) \
			lockdep_rcu_dereference(__FILE__, __LINE__); \
		(p); \
	})

#else /* #ifdef CONFIG_PROVE_RCU */

#define rcu_dereference_check(p, c)	rcu_dereference_raw(p)
#define rcu_dereference_protected(p, c) (p)

#endif /* #else #ifdef CONFIG_PROVE_RCU */

/**
 * rcu_access_pointer - fetch RCU pointer with no dereferencing
 *
 * Return the value of the specified RCU-protected pointer, but omit the
 * smp_read_barrier_depends() and keep the ACCESS_ONCE().  This is useful
 * when the value of this pointer is accessed, but the pointer is not
 * dereferenced, for example, when testing an RCU-protected pointer against
 * NULL.  This may also be used in cases where update-side locks prevent
 * the value of the pointer from changing, but rcu_dereference_protected()
 * is a lighter-weight primitive for this use case.
 */
#define rcu_access_pointer(p)	ACCESS_ONCE(p)

/**
 * rcu_read_lock - mark the beginning of an RCU read-side critical section.
 *
Loading