Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 61d44b22 authored by Patrick Bellasi's avatar Patrick Bellasi Committed by Quentin Perret
Browse files

BACKPORT: sched/cpufreq, sched/uclamp: Add clamps for FAIR and RT tasks 



Each time a frequency update is required via schedutil, a frequency is
selected to (possibly) satisfy the utilization reported by each
scheduling class and irqs. However, when utilization clamping is in use,
the frequency selection should consider userspace utilization clamping
hints.  This will allow, for example, to:

 - boost tasks which are directly affecting the user experience
   by running them at least at a minimum "requested" frequency

 - cap low priority tasks not directly affecting the user experience
   by running them only up to a maximum "allowed" frequency

These constraints are meant to support a per-task based tuning of the
frequency selection thus supporting a fine grained definition of
performance boosting vs energy saving strategies in kernel space.

Add support to clamp the utilization of RUNNABLE FAIR and RT tasks
within the boundaries defined by their aggregated utilization clamp
constraints.

Do that by considering the max(min_util, max_util) to give boosted tasks
the performance they need even when they happen to be co-scheduled with
other capped tasks.

Signed-off-by: default avatarPatrick Bellasi <patrick.bellasi@arm.com>
Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
Cc: Alessio Balsini <balsini@android.com>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Joel Fernandes <joelaf@google.com>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Morten Rasmussen <morten.rasmussen@arm.com>
Cc: Paul Turner <pjt@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Quentin Perret <quentin.perret@arm.com>
Cc: Rafael J . Wysocki <rafael.j.wysocki@intel.com>
Cc: Steve Muckle <smuckle@google.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Todd Kjos <tkjos@google.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Link: https://lkml.kernel.org/r/20190621084217.8167-10-patrick.bellasi@arm.com


Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
(cherry picked from commit 982d9cdc22c9f6df5ad790caa229ff74fb1d95e7)

Conflicts:
	kernel/sched/cpufreq_schedutil.c

	1. Merged the if condition to include the non-upstream
	   sched_feat(SUGOV_RT_MAX_FREQ) check

	2. Change the function signature to pass util_cfs and define
	   util as an automatic variable.

Bug: 120440300
Signed-off-by: default avatarQais Yousef <qais.yousef@arm.com>
Change-Id: Ie222c9ad84776fc2948e30c116eee876df697a17
Signed-off-by: default avatarQuentin Perret <qperret@google.com>
parent fa167475

kernel/sched/cpufreq_schedutil.c

+19 −8
Original line number Diff line number Diff line
@@ -237,15 +237,16 @@ static unsigned int get_next_freq(struct sugov_policy *sg_policy, 
 * based on the task model parameters and gives the minimal utilization

 * required to meet deadlines.

 */

unsigned long schedutil_freq_util(int cpu, unsigned long util,

unsigned long schedutil_freq_util(int cpu, unsigned long util_cfs,

				  unsigned long max, enum schedutil_type type)

{

	unsigned long dl_util, irq;

	unsigned long dl_util, util, irq;

	struct rq *rq = cpu_rq(cpu);



	if (sched_feat(SUGOV_RT_MAX_FREQ) && type == FREQUENCY_UTIL &&

						rt_rq_is_runnable(&rq->rt))

	if ((sched_feat(SUGOV_RT_MAX_FREQ) || !IS_BUILTIN(CONFIG_UCLAMP_TASK)) &&

	    type == FREQUENCY_UTIL && rt_rq_is_runnable(&rq->rt)) {

		return max;

	}



	/*

	 * Early check to see if IRQ/steal time saturates the CPU, can be
@@ -257,11 +258,21 @@ unsigned long schedutil_freq_util(int cpu, unsigned long util, 
		return max;



	/*

	 * The function is called with @util defined as the aggregation (the

	 * sum) of RT and CFS signals, hence leaving the special case of DL

	 * to be delt with. The exact way of doing things depend on the calling

	 * context.

	 * Because the time spend on RT/DL tasks is visible as 'lost' time to

	 * CFS tasks and we use the same metric to track the effective

	 * utilization (PELT windows are synchronized) we can directly add them

	 * to obtain the CPU's actual utilization.

	 *

	 * CFS and RT utilization can be boosted or capped, depending on

	 * utilization clamp constraints requested by currently RUNNABLE

	 * tasks.

	 * When there are no CFS RUNNABLE tasks, clamps are released and

	 * frequency will be gracefully reduced with the utilization decay.

	 */

	util = util_cfs + cpu_util_rt(rq);

	if (type == FREQUENCY_UTIL)

		util = uclamp_util(rq, util);



	dl_util = cpu_util_dl(rq);



	/*

kernel/sched/fair.c

+4 −0
Original line number Diff line number Diff line
@@ -11219,6 +11219,10 @@ const struct sched_class fair_sched_class = { 
#ifdef CONFIG_FAIR_GROUP_SCHED

	.task_change_group	= task_change_group_fair,

#endif



#ifdef CONFIG_UCLAMP_TASK

	.uclamp_enabled		= 1,

#endif

};



#ifdef CONFIG_SCHED_DEBUG

kernel/sched/rt.c

+4 −0
Original line number Diff line number Diff line
@@ -2409,6 +2409,10 @@ const struct sched_class rt_sched_class = { 
	.switched_to		= switched_to_rt,



	.update_curr		= update_curr_rt,



#ifdef CONFIG_UCLAMP_TASK

	.uclamp_enabled		= 1,

#endif

};



#ifdef CONFIG_RT_GROUP_SCHED

kernel/sched/sched.h

+23 −0
Original line number Diff line number Diff line
@@ -2308,6 +2308,29 @@ static inline void cpufreq_update_util(struct rq *rq, unsigned int flags) 
static inline void cpufreq_update_util(struct rq *rq, unsigned int flags) {}

#endif /* CONFIG_CPU_FREQ */



#ifdef CONFIG_UCLAMP_TASK

static inline unsigned int uclamp_util(struct rq *rq, unsigned int util)

{

	unsigned int min_util = READ_ONCE(rq->uclamp[UCLAMP_MIN].value);

	unsigned int max_util = READ_ONCE(rq->uclamp[UCLAMP_MAX].value);



	/*

	 * Since CPU's {min,max}_util clamps are MAX aggregated considering

	 * RUNNABLE tasks with _different_ clamps, we can end up with an

	 * inversion. Fix it now when the clamps are applied.

	 */

	if (unlikely(min_util >= max_util))

		return min_util;



	return clamp(util, min_util, max_util);

}

#else /* CONFIG_UCLAMP_TASK */

static inline unsigned int uclamp_util(struct rq *rq, unsigned int util)

{

	return util;

}

#endif /* CONFIG_UCLAMP_TASK */



#ifdef arch_scale_freq_capacity

# ifndef arch_scale_freq_invariant

#  define arch_scale_freq_invariant()	true