Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit df1c99d4 authored by Mike Galbraith's avatar Mike Galbraith Committed by Ingo Molnar
Browse files

sched: add avg_overlap decay 



Impact: more precise avg_overlap metric - better load-balancing

avg_overlap is used to measure the runtime overlap of the waker and
wakee.

However, when a process changes behaviour, eg a pipe becomes
un-congested and we don't need to go to sleep after a wakeup
for a while, the avg_overlap value grows stale.

When running we use the avg runtime between preemption as a
measure for avg_overlap since the amount of runtime can be
correlated to cache footprint.

The longer we run, the less likely we'll be wanting to be
migrated to another CPU.

Signed-off-by: default avatarMike Galbraith <efault@gmx.de>
Signed-off-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <1236709131.25234.576.camel@laptop>
Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
parent 57310a98

kernel/sched.c

+23 −1
Original line number Diff line number Diff line
@@ -4620,6 +4620,28 @@ static inline void schedule_debug(struct task_struct *prev) 
#endif

}



static void put_prev_task(struct rq *rq, struct task_struct *prev)

{

	if (prev->state == TASK_RUNNING) {

		u64 runtime = prev->se.sum_exec_runtime;



		runtime -= prev->se.prev_sum_exec_runtime;

		runtime = min_t(u64, runtime, 2*sysctl_sched_migration_cost);



		/*

		 * In order to avoid avg_overlap growing stale when we are

		 * indeed overlapping and hence not getting put to sleep, grow

		 * the avg_overlap on preemption.

		 *

		 * We use the average preemption runtime because that

		 * correlates to the amount of cache footprint a task can

		 * build up.

		 */

		update_avg(&prev->se.avg_overlap, runtime);

	}

	prev->sched_class->put_prev_task(rq, prev);

}



/*

 * Pick up the highest-prio task:

 */
@@ -4698,7 +4720,7 @@ need_resched_nonpreemptible: 
	if (unlikely(!rq->nr_running))

		idle_balance(cpu, rq);



	prev->sched_class->put_prev_task(rq, prev);

	put_prev_task(rq, prev);

	next = pick_next_task(rq);



	if (likely(prev != next)) {