Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 44c65ff2 authored by Paul E. McKenney's avatar Paul E. McKenney
Browse files

rcu: Eliminate NOCBs CPU-state Kconfig options 



The CONFIG_RCU_NOCB_CPU_ALL, CONFIG_RCU_NOCB_CPU_NONE, and
CONFIG_RCU_NOCB_CPU_ZERO Kconfig options are used only in testing and
are redundant with the rcu_nocbs= boot parameter.  This commit therefore
removes these three Kconfig options and adjusts the rcutorture scripts
to use the boot parameter instead.

Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
parent ae91aa0a

Documentation/kernel-per-CPU-kthreads.txt

+14 −17
Original line number Diff line number Diff line
@@ -109,13 +109,12 @@ SCHED_SOFTIRQ: Do all of the following: 
	on that CPU.  If a thread that expects to run on the de-jittered

	CPU awakens, the scheduler will send an IPI that can result in

	a subsequent SCHED_SOFTIRQ.

2.	Build with CONFIG_RCU_NOCB_CPU=y, CONFIG_RCU_NOCB_CPU_ALL=y,

	CONFIG_NO_HZ_FULL=y, and, in addition, ensure that the CPU

	to be de-jittered is marked as an adaptive-ticks CPU using the

	"nohz_full=" boot parameter.  This reduces the number of

	scheduler-clock interrupts that the de-jittered CPU receives,

	minimizing its chances of being selected to do the load balancing

	work that runs in SCHED_SOFTIRQ context.

2.	CONFIG_NO_HZ_FULL=y and ensure that the CPU to be de-jittered

	is marked as an adaptive-ticks CPU using the "nohz_full="

	boot parameter.  This reduces the number of scheduler-clock

	interrupts that the de-jittered CPU receives, minimizing its

	chances of being selected to do the load balancing work that

	runs in SCHED_SOFTIRQ context.

3.	To the extent possible, keep the CPU out of the kernel when it

	is non-idle, for example, by avoiding system calls and by

	forcing both kernel threads and interrupts to execute elsewhere.
@@ -135,11 +134,10 @@ HRTIMER_SOFTIRQ: Do all of the following: 
RCU_SOFTIRQ:  Do at least one of the following:

1.	Offload callbacks and keep the CPU in either dyntick-idle or

	adaptive-ticks state by doing all of the following:

	a.	Build with CONFIG_RCU_NOCB_CPU=y, CONFIG_RCU_NOCB_CPU_ALL=y,

		CONFIG_NO_HZ_FULL=y, and, in addition ensure that the CPU

		to be de-jittered is marked as an adaptive-ticks CPU using

		the "nohz_full=" boot parameter.  Bind the rcuo kthreads

		to housekeeping CPUs, which can tolerate OS jitter.

	a.	CONFIG_NO_HZ_FULL=y and ensure that the CPU to be

		de-jittered is marked as an adaptive-ticks CPU using the

		"nohz_full=" boot parameter.  Bind the rcuo kthreads to

		housekeeping CPUs, which can tolerate OS jitter.

	b.	To the extent possible, keep the CPU out of the kernel

		when it is non-idle, for example, by avoiding system

		calls and by forcing both kernel threads and interrupts
@@ -236,11 +234,10 @@ To reduce its OS jitter, do at least one of the following: 
	is feasible only if your workload never requires RCU priority

	boosting, for example, if you ensure frequent idle time on all

	CPUs that might execute within the kernel.

3.	Build with CONFIG_RCU_NOCB_CPU=y and CONFIG_RCU_NOCB_CPU_ALL=y,

	which offloads all RCU callbacks to kthreads that can be moved

	off of CPUs susceptible to OS jitter.  This approach prevents the

	rcuc/%u kthreads from having any work to do, so that they are

	never awakened.

3.	Build with CONFIG_RCU_NOCB_CPU=y and boot with the rcu_nocbs=

	boot parameter offloading RCU callbacks from all CPUs susceptible

	to OS jitter.  This approach prevents the rcuc/%u kthreads from

	having any work to do, so that they are never awakened.

4.	Ensure that the CPU never enters the kernel, and, in particular,

	avoid initiating any CPU hotplug operations on this CPU.  This is

	another way of preventing any callbacks from being queued on the

Documentation/timers/NO_HZ.txt

+3 −26
Original line number Diff line number Diff line
@@ -194,32 +194,9 @@ that the RCU callbacks are processed in a timely fashion. 


Another approach is to offload RCU callback processing to "rcuo" kthreads

using the CONFIG_RCU_NOCB_CPU=y Kconfig option.  The specific CPUs to

offload may be selected via several methods:



1.	One of three mutually exclusive Kconfig options specify a

	build-time default for the CPUs to offload:



	a.	The CONFIG_RCU_NOCB_CPU_NONE=y Kconfig option results in

		no CPUs being offloaded.



	b.	The CONFIG_RCU_NOCB_CPU_ZERO=y Kconfig option causes

		CPU 0 to be offloaded.



	c.	The CONFIG_RCU_NOCB_CPU_ALL=y Kconfig option causes all

		CPUs to be offloaded.  Note that the callbacks will be

		offloaded to "rcuo" kthreads, and that those kthreads

		will in fact run on some CPU.  However, this approach

		gives fine-grained control on exactly which CPUs the

		callbacks run on, along with their scheduling priority

		(including the default of SCHED_OTHER), and it further

		allows this control to be varied dynamically at runtime.



2.	The "rcu_nocbs=" kernel boot parameter, which takes a comma-separated

	list of CPUs and CPU ranges, for example, "1,3-5" selects CPUs 1,

	3, 4, and 5.  The specified CPUs will be offloaded in addition to

	any CPUs specified as offloaded by CONFIG_RCU_NOCB_CPU_ZERO=y or

	CONFIG_RCU_NOCB_CPU_ALL=y.  This means that the "rcu_nocbs=" boot

	parameter has no effect for kernels built with RCU_NOCB_CPU_ALL=y.

offload may be selected using The "rcu_nocbs=" kernel boot parameter,

which takes a comma-separated list of CPUs and CPU ranges, for example,

"1,3-5" selects CPUs 1, 3, 4, and 5.



The offloaded CPUs will never queue RCU callbacks, and therefore RCU

never prevents offloaded CPUs from entering either dyntick-idle mode

init/Kconfig

+0 −53
Original line number Diff line number Diff line
@@ -709,59 +709,6 @@ config RCU_NOCB_CPU 
	  Say Y here if you want to help to debug reduced OS jitter.

	  Say N here if you are unsure.



choice

	prompt "Build-forced no-CBs CPUs"

	default RCU_NOCB_CPU_NONE

	depends on RCU_NOCB_CPU

	help

	  This option allows no-CBs CPUs (whose RCU callbacks are invoked

	  from kthreads rather than from softirq context) to be specified

	  at build time.  Additional no-CBs CPUs may be specified by

	  the rcu_nocbs= boot parameter.



config RCU_NOCB_CPU_NONE

	bool "No build_forced no-CBs CPUs"

	help

	  This option does not force any of the CPUs to be no-CBs CPUs.

	  Only CPUs designated by the rcu_nocbs= boot parameter will be

	  no-CBs CPUs, whose RCU callbacks will be invoked by per-CPU

	  kthreads whose names begin with "rcuo".  All other CPUs will

	  invoke their own RCU callbacks in softirq context.



	  Select this option if you want to choose no-CBs CPUs at

	  boot time, for example, to allow testing of different no-CBs

	  configurations without having to rebuild the kernel each time.



config RCU_NOCB_CPU_ZERO

	bool "CPU 0 is a build_forced no-CBs CPU"

	help

	  This option forces CPU 0 to be a no-CBs CPU, so that its RCU

	  callbacks are invoked by a per-CPU kthread whose name begins

	  with "rcuo".	Additional CPUs may be designated as no-CBs

	  CPUs using the rcu_nocbs= boot parameter will be no-CBs CPUs.

	  All other CPUs will invoke their own RCU callbacks in softirq

	  context.



	  Select this if CPU 0 needs to be a no-CBs CPU for real-time

	  or energy-efficiency reasons, but the real reason it exists

	  is to ensure that randconfig testing covers mixed systems.



config RCU_NOCB_CPU_ALL

	bool "All CPUs are build_forced no-CBs CPUs"

	help

	  This option forces all CPUs to be no-CBs CPUs.  The rcu_nocbs=

	  boot parameter will be ignored.  All CPUs' RCU callbacks will

	  be executed in the context of per-CPU rcuo kthreads created for

	  this purpose.  Assuming that the kthreads whose names start with

	  "rcuo" are bound to "housekeeping" CPUs, this reduces OS jitter

	  on the remaining CPUs, but might decrease memory locality during

	  RCU-callback invocation, thus potentially degrading throughput.



	  Select this if all CPUs need to be no-CBs CPUs for real-time

	  or energy-efficiency reasons.



endchoice



endmenu # "RCU Subsystem"



config BUILD_BIN2C

kernel/rcu/rcu.h

+1 −3
Original line number Diff line number Diff line
@@ -564,9 +564,7 @@ void rcu_bh_force_quiescent_state(void); 
void rcu_sched_force_quiescent_state(void);

#endif /* #else #ifdef CONFIG_TINY_RCU */



#if defined(CONFIG_RCU_NOCB_CPU_ALL)

static inline bool rcu_is_nocb_cpu(int cpu) { return true; }

#elif defined(CONFIG_RCU_NOCB_CPU)

#ifdef CONFIG_RCU_NOCB_CPU

bool rcu_is_nocb_cpu(int cpu);

#else

static inline bool rcu_is_nocb_cpu(int cpu) { return false; }

kernel/rcu/tree_plugin.h

+3 −24
Original line number Diff line number Diff line
@@ -1296,8 +1296,7 @@ static void rcu_prepare_kthreads(int cpu) 
int rcu_needs_cpu(u64 basemono, u64 *nextevt)

{

	*nextevt = KTIME_MAX;

	return IS_ENABLED(CONFIG_RCU_NOCB_CPU_ALL)

	       ? 0 : rcu_cpu_has_callbacks(NULL);

	return rcu_cpu_has_callbacks(NULL);

}



/*
@@ -1409,10 +1408,6 @@ int rcu_needs_cpu(u64 basemono, u64 *nextevt) 
	unsigned long dj;



	RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_needs_cpu() invoked with irqs enabled!!!");

	if (IS_ENABLED(CONFIG_RCU_NOCB_CPU_ALL)) {

		*nextevt = KTIME_MAX;

		return 0;

	}



	/* Snapshot to detect later posting of non-lazy callback. */

	rdtp->nonlazy_posted_snap = rdtp->nonlazy_posted;
@@ -1462,8 +1457,7 @@ static void rcu_prepare_for_idle(void) 
	int tne;



	RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_prepare_for_idle() invoked with irqs enabled!!!");

	if (IS_ENABLED(CONFIG_RCU_NOCB_CPU_ALL) ||

	    rcu_is_nocb_cpu(smp_processor_id()))

	if (rcu_is_nocb_cpu(smp_processor_id()))

		return;



	/* Handle nohz enablement switches conservatively. */
@@ -1518,8 +1512,7 @@ static void rcu_prepare_for_idle(void) 
static void rcu_cleanup_after_idle(void)

{

	RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_cleanup_after_idle() invoked with irqs enabled!!!");

	if (IS_ENABLED(CONFIG_RCU_NOCB_CPU_ALL) ||

	    rcu_is_nocb_cpu(smp_processor_id()))

	if (rcu_is_nocb_cpu(smp_processor_id()))

		return;

	if (rcu_try_advance_all_cbs())

		invoke_rcu_core();
@@ -1786,7 +1779,6 @@ static void rcu_init_one_nocb(struct rcu_node *rnp) 
	init_swait_queue_head(&rnp->nocb_gp_wq[1]);

}



#ifndef CONFIG_RCU_NOCB_CPU_ALL

/* Is the specified CPU a no-CBs CPU? */

bool rcu_is_nocb_cpu(int cpu)

{
@@ -1794,7 +1786,6 @@ bool rcu_is_nocb_cpu(int cpu) 
		return cpumask_test_cpu(cpu, rcu_nocb_mask);

	return false;

}

#endif /* #ifndef CONFIG_RCU_NOCB_CPU_ALL */



/*

 * Kick the leader kthread for this NOCB group.
@@ -2253,10 +2244,6 @@ void __init rcu_init_nohz(void) 
	bool need_rcu_nocb_mask = true;

	struct rcu_state *rsp;



#ifdef CONFIG_RCU_NOCB_CPU_NONE

	need_rcu_nocb_mask = false;

#endif /* #ifndef CONFIG_RCU_NOCB_CPU_NONE */



#if defined(CONFIG_NO_HZ_FULL)

	if (tick_nohz_full_running && cpumask_weight(tick_nohz_full_mask))

		need_rcu_nocb_mask = true;
@@ -2272,14 +2259,6 @@ void __init rcu_init_nohz(void) 
	if (!have_rcu_nocb_mask)

		return;



#ifdef CONFIG_RCU_NOCB_CPU_ZERO

	pr_info("\tOffload RCU callbacks from CPU 0\n");

	cpumask_set_cpu(0, rcu_nocb_mask);

#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ZERO */

#ifdef CONFIG_RCU_NOCB_CPU_ALL

	pr_info("\tOffload RCU callbacks from all CPUs\n");

	cpumask_copy(rcu_nocb_mask, cpu_possible_mask);

#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ALL */

#if defined(CONFIG_NO_HZ_FULL)

	if (tick_nohz_full_running)

		cpumask_or(rcu_nocb_mask, rcu_nocb_mask, tick_nohz_full_mask);