Commit c4575f83 authored May 02, 2014 by Kaushal Kumar

sched: Remove get_online_cpus() usage



Remove get_online_cpus() usage from the scheduler; there's 4 sites that
use it:

 - sched_init_smp(); where its completely superfluous since we're in
   'early' boot and there simply cannot be any hotplugging.

 - sched_getaffinity(); we already take a raw spinlock to protect the
   task cpus_allowed mask, this disables preemption and therefore
   also stabilizes cpu_online_mask as that's modified using
   stop_machine. However switch to active mask for symmetry with
   sched_setaffinity()/set_cpus_allowed_ptr(). We guarantee active
   mask stability by inserting sync_rcu/sched() into _cpu_down.

 - sched_setaffinity(); we don't appear to need get_online_cpus()
   either, there's two sites where hotplug appears relevant:
    * cpuset_cpus_allowed(); for the !cpuset case we use possible_mask,
      for the cpuset case we hold task_lock, which is a spinlock and
      thus for mainline disables preemption (might cause pain on RT).
    * set_cpus_allowed_ptr(); Holds all scheduler locks and thus has
      preemption properly disabled; also it already deals with hotplug
      races explicitly where it releases them.

 - migrate_swap(); we can make stop_two_cpus() do the heavy lifting for
   us with a little trickery. By adding a sync_sched/rcu() after the
   CPU_DOWN_PREPARE notifier we can provide preempt/rcu guarantees for
   cpu_active_mask. Use these to validate that both our cpus are active
   when queueing the stop work before we queue the stop_machine works
   for take_cpu_down().

CRs-fixed: 647141
Change-Id: Id41e66659574f716de0e7c29f477e56a86db9404
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Cc: "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Rik van Riel <riel@redhat.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Link: http://lkml.kernel.org/r/20131011123820.GV3081@twins.programming.kicks-ass.net


Signed-off-by: Ingo Molnar <mingo@kernel.org>
Git-commit: 6acce3ef84520537f8a09a12c9ddbe814a584dd2
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git


[kaushalk@codeaurora.org: get_online_cpus has only 3 sites of usage in
 kernel/sched/core.c of msm-3.10 so migrate_swap changes are not
 applicable here. stop_two_cpus related change is not applicable to
 msm-3.10, so skip it.]
Signed-off-by: Kaushal Kumar <kaushalk@codeaurora.org>

parent b748d2da

kernel/cpu.c

+17 −0

Original line number	Diff line number	Diff line
		@@ -313,6 +313,23 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
		}
		smpboot_park_threads(cpu);

		/*
		* By now we've cleared cpu_active_mask, wait for all preempt-disabled
		* and RCU users of this state to go away such that all new such users
		* will observe it.
		*
		* For CONFIG_PREEMPT we have preemptible RCU and its sync_rcu() might
		* not imply sync_sched(), so explicitly call both.
		*/
		#ifdef CONFIG_PREEMPT
		synchronize_sched();
		#endif
		synchronize_rcu();

		/*
		* So now all preempt/rcu users must observe !cpu_active().
		*/

		err = __stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
		if (err) {
		/* CPU didn't die: tell everyone. Can't complain. */

kernel/sched/core.c

+6 −7

Original line number	Diff line number	Diff line
		@@ -4553,7 +4553,6 @@ long sched_setaffinity(pid_t pid, const struct cpumask *in_mask)
		struct task_struct *p;
		int retval;

		get_online_cpus();
		rcu_read_lock();

		p = find_process_by_pid(pid);
		@@ -4616,7 +4615,6 @@ out_free_cpus_allowed:
		free_cpumask_var(cpus_allowed);
		out_put_task:
		put_task_struct(p);
		put_online_cpus();
		return retval;
		}

		@@ -4659,7 +4657,6 @@ long sched_getaffinity(pid_t pid, struct cpumask *mask)
		unsigned long flags;
		int retval;

		get_online_cpus();
		rcu_read_lock();

		retval = -ESRCH;
		@@ -4672,12 +4669,11 @@ long sched_getaffinity(pid_t pid, struct cpumask *mask)
		goto out_unlock;

		raw_spin_lock_irqsave(&p->pi_lock, flags);
		cpumask_and(mask, &p->cpus_allowed, cpu_online_mask);
		cpumask_and(mask, &p->cpus_allowed, cpu_active_mask);
		raw_spin_unlock_irqrestore(&p->pi_lock, flags);

		out_unlock:
		rcu_read_unlock();
		put_online_cpus();

		return retval;
		}
		@@ -7287,14 +7283,17 @@ void __init sched_init_smp(void)

		sched_init_numa();

		get_online_cpus();
		/*
		* There's no userspace yet to cause hotplug operations; hence all the
		* cpu masks are stable and all blatant races in the below code cannot
		* happen.
		*/
		mutex_lock(&sched_domains_mutex);
		init_sched_domains(cpu_active_mask);
		cpumask_andnot(non_isolated_cpus, cpu_possible_mask, cpu_isolated_map);
		if (cpumask_empty(non_isolated_cpus))
		cpumask_set_cpu(smp_processor_id(), non_isolated_cpus);
		mutex_unlock(&sched_domains_mutex);
		put_online_cpus();

		hotcpu_notifier(sched_domains_numa_masks_update, CPU_PRI_SCHED_ACTIVE);
		hotcpu_notifier(cpuset_cpu_active, CPU_PRI_CPUSET_ACTIVE);