Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 51816e9e authored by Waiman Long's avatar Waiman Long Committed by Linus Torvalds
Browse files

locking/lock_events: Use this_cpu_add() when necessary 



The kernel test robot has reported that the use of __this_cpu_add()
causes bug messages like:

  BUG: using __this_cpu_add() in preemptible [00000000] code: ...

Given the imprecise nature of the count and the possibility of resetting
the count and doing the measurement again, this is not really a big
problem to use the unprotected __this_cpu_*() functions.

To make the preemption checking code happy, the this_cpu_*() functions
will be used if CONFIG_DEBUG_PREEMPT is defined.

The imprecise nature of the locking counts are also documented with
the suggestion that we should run the measurement a few times with the
counts reset in between to get a better picture of what is going on
under the hood.

Fixes: a8654596 ("locking/rwsem: Enable lock event counting")
Suggested-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: default avatarWaiman Long <longman@redhat.com>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent 0a72ef89

kernel/locking/lock_events.h

+40 −2
Original line number Diff line number Diff line
@@ -30,13 +30,51 @@ enum lock_events { 
 */

DECLARE_PER_CPU(unsigned long, lockevents[lockevent_num]);



/*

 * The purpose of the lock event counting subsystem is to provide a low

 * overhead way to record the number of specific locking events by using

 * percpu counters. It is the percpu sum that matters, not specifically

 * how many of them happens in each cpu.

 *

 * It is possible that the same percpu counter may be modified in both

 * the process and interrupt contexts. For architectures that perform

 * percpu operation with multiple instructions, it is possible to lose

 * count if a process context percpu update is interrupted in the middle

 * and the same counter is updated in the interrupt context. Therefore,

 * the generated percpu sum may not be precise. The error, if any, should

 * be small and insignificant.

 *

 * For those architectures that do multi-instruction percpu operation,

 * preemption in the middle and moving the task to another cpu may cause

 * a larger error in the count. Again, this will be few and far between.

 * Given the imprecise nature of the count and the possibility of resetting

 * the count and doing the measurement again, this is not really a big

 * problem.

 *

 * To get a better picture of what is happening under the hood, it is

 * suggested that a few measurements should be taken with the counts

 * reset in between to stamp out outliner because of these possible

 * error conditions.

 *

 * To minimize overhead, we use __this_cpu_*() in all cases except when

 * CONFIG_DEBUG_PREEMPT is defined. In this particular case, this_cpu_*()

 * will be used to avoid the appearance of unwanted BUG messages.

 */

#ifdef CONFIG_DEBUG_PREEMPT

#define lockevent_percpu_inc(x)		this_cpu_inc(x)

#define lockevent_percpu_add(x, v)	this_cpu_add(x, v)

#else

#define lockevent_percpu_inc(x)		__this_cpu_inc(x)

#define lockevent_percpu_add(x, v)	__this_cpu_add(x, v)

#endif



/*

 * Increment the PV qspinlock statistical counters

 */

static inline void __lockevent_inc(enum lock_events event, bool cond)

{

	if (cond)

		__this_cpu_inc(lockevents[event]);

		lockevent_percpu_inc(lockevents[event]);

}



#define lockevent_inc(ev)	  __lockevent_inc(LOCKEVENT_ ##ev, true)
@@ -44,7 +82,7 @@ static inline void __lockevent_inc(enum lock_events event, bool cond) 


static inline void __lockevent_add(enum lock_events event, int inc)

{

	__this_cpu_add(lockevents[event], inc);

	lockevent_percpu_add(lockevents[event], inc);

}



#define lockevent_add(ev, c)	__lockevent_add(LOCKEVENT_ ##ev, c)