Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 6dc1a672 authored by Rik van Riel's avatar Rik van Riel Committed by Ingo Molnar
Browse files

sched/numa: Use effective_load() to balance NUMA loads 



When CONFIG_FAIR_GROUP_SCHED is enabled, the load that a task places
on a CPU is determined by the group the task is in. The active groups
on the source and destination CPU can be different, resulting in a
different load contribution by the same task at its source and at its
destination. As a result, the load needs to be calculated separately
for each CPU, instead of estimated once with task_h_load().

Getting this calculation right allows some workloads to converge,
where previously the last thread could get stuck on another node,
without being able to migrate to its final destination.

Signed-off-by: default avatarRik van Riel <riel@redhat.com>
Cc: mgorman@suse.de
Cc: chegu_vinod@hp.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: default avatarPeter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1403538378-31571-3-git-send-email-riel@redhat.com


Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
parent 28a21745

kernel/sched/fair.c

+14 −6
Original line number Original line Diff line number Diff line
@@ -1151,6 +1151,7 @@ static void task_numa_compare(struct task_numa_env *env, 
	struct rq *src_rq = cpu_rq(env->src_cpu);

	struct rq *src_rq = cpu_rq(env->src_cpu);

	struct rq *dst_rq = cpu_rq(env->dst_cpu);

	struct rq *dst_rq = cpu_rq(env->dst_cpu);

	struct task_struct *cur;

	struct task_struct *cur;

	struct task_group *tg;

	long src_load, dst_load;

	long src_load, dst_load;

	long load;

	long load;

	long imp = (groupimp > 0) ? groupimp : taskimp;

	long imp = (groupimp > 0) ? groupimp : taskimp;
@@ -1225,14 +1226,21 @@ static void task_numa_compare(struct task_numa_env *env, 
	 * In the overloaded case, try and keep the load balanced.

	 * In the overloaded case, try and keep the load balanced.

	 */

	 */

balance:

balance:

	load = task_h_load(env->p);

	src_load = env->src_stats.load;

	dst_load = env->dst_stats.load + load;

	dst_load = env->dst_stats.load;

	src_load = env->src_stats.load - load;



	/* Calculate the effect of moving env->p from src to dst. */

	load = env->p->se.load.weight;

	tg = task_group(env->p);

	src_load += effective_load(tg, env->src_cpu, -load, -load);

	dst_load += effective_load(tg, env->dst_cpu, load, load);





	if (cur) {

	if (cur) {

		load = task_h_load(cur);

		/* Cur moves in the opposite direction. */

		dst_load -= load;

		load = cur->se.load.weight;

		src_load += load;

		tg = task_group(cur);

		src_load += effective_load(tg, env->src_cpu, load, load);

		dst_load += effective_load(tg, env->dst_cpu, -load, -load);

	}

	}





	if (load_too_imbalanced(src_load, dst_load, env))

	if (load_too_imbalanced(src_load, dst_load, env))