Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit a426f99c authored by Waiman Long's avatar Waiman Long Committed by Ingo Molnar
Browse files

sched/fair: Avoid redundant idle_cpu() call in update_sg_lb_stats() 



Part of the responsibility of the update_sg_lb_stats() function is to
update the idle_cpus statistical counter in struct sg_lb_stats. This
check is done by calling idle_cpu(). The idle_cpu() function, in
turn, checks a number of fields within the run queue structure such
as rq->curr and rq->nr_running.

With the current layout of the run queue structure, rq->curr and
rq->nr_running are in separate cachelines. The rq->curr variable is
checked first followed by nr_running. As nr_running is also accessed
by update_sg_lb_stats() earlier, it makes no sense to load another
cacheline when nr_running is not 0 as idle_cpu() will always return
false in this case.

This patch eliminates this redundant cacheline load by checking the
cached nr_running before calling idle_cpu().

Signed-off-by: default avatarWaiman Long <Waiman.Long@hpe.com>
Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
Cc: Douglas Hatch <doug.hatch@hpe.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Scott J Norton <scott.norton@hpe.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1448478580-26467-2-git-send-email-Waiman.Long@hpe.com


Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
parent ed82b8a1

kernel/sched/fair.c

+7 −3
Original line number Diff line number Diff line
@@ -6398,7 +6398,7 @@ static inline void update_sg_lb_stats(struct lb_env *env, 
			bool *overload)

{

	unsigned long load;

	int i;

	int i, nr_running;



	memset(sgs, 0, sizeof(*sgs));
@@ -6415,7 +6415,8 @@ static inline void update_sg_lb_stats(struct lb_env *env, 
		sgs->group_util += cpu_util(i);

		sgs->sum_nr_running += rq->cfs.h_nr_running;



		if (rq->nr_running > 1)

		nr_running = rq->nr_running;

		if (nr_running > 1)

			*overload = true;



#ifdef CONFIG_NUMA_BALANCING
@@ -6423,7 +6424,10 @@ static inline void update_sg_lb_stats(struct lb_env *env, 
		sgs->nr_preferred_running += rq->nr_preferred_running;

#endif

		sgs->sum_weighted_load += weighted_cpuload(i);

		if (idle_cpu(i))

		/*

		 * No need to call idle_cpu() if nr_running is not 0

		 */

		if (!nr_running && idle_cpu(i))

			sgs->idle_cpus++;

	}