Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit d6aa8f85 authored by Peter Zijlstra's avatar Peter Zijlstra Committed by Ingo Molnar
Browse files

sched: Fix ttwu() for __ARCH_WANT_INTERRUPTS_ON_CTXSW 



Marc reported that e4a52bcb (sched: Remove rq->lock from the first
half of ttwu()) broke his ARM-SMP machine. Now ARM is one of the few
__ARCH_WANT_INTERRUPTS_ON_CTXSW users, so that exception in the ttwu()
code was suspect.

Yong found that the interrupt could hit after context_switch() changes
current but before it clears p->on_cpu, if that interrupt were to
attempt a wake-up of p we would indeed find ourselves spinning in IRQ
context.

Fix this by reverting to the old behaviour for this situation and
perform a full remote wake-up.

Cc: Frank Rowand <frank.rowand@am.sony.com>
Cc: Yong Zhang <yong.zhang0@gmail.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Reported-by: default avatarMarc Zyngier <Marc.Zyngier@arm.com>
Tested-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
parent cd4ae6ad

kernel/sched.c

+28 −9
Original line number Diff line number Diff line
@@ -2573,7 +2573,26 @@ static void ttwu_queue_remote(struct task_struct *p, int cpu) 
	if (!next)

		smp_send_reschedule(cpu);

}

#endif



#ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW

static int ttwu_activate_remote(struct task_struct *p, int wake_flags)

{

	struct rq *rq;

	int ret = 0;



	rq = __task_rq_lock(p);

	if (p->on_cpu) {

		ttwu_activate(rq, p, ENQUEUE_WAKEUP);

		ttwu_do_wakeup(rq, p, wake_flags);

		ret = 1;

	}

	__task_rq_unlock(rq);



	return ret;



}

#endif /* __ARCH_WANT_INTERRUPTS_ON_CTXSW */

#endif /* CONFIG_SMP */



static void ttwu_queue(struct task_struct *p, int cpu)

{
@@ -2631,17 +2650,17 @@ try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags) 
	while (p->on_cpu) {

#ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW

		/*

		 * If called from interrupt context we could have landed in the

		 * middle of schedule(), in this case we should take care not

		 * to spin on ->on_cpu if p is current, since that would

		 * deadlock.

		 * In case the architecture enables interrupts in

		 * context_switch(), we cannot busy wait, since that

		 * would lead to deadlocks when an interrupt hits and

		 * tries to wake up @prev. So bail and do a complete

		 * remote wakeup.

		 */

		if (p == current) {

			ttwu_queue(p, cpu);

		if (ttwu_activate_remote(p, wake_flags))

			goto stat;

		}

#endif

#else

		cpu_relax();

#endif

	}

	/*

	 * Pairs with the smp_wmb() in finish_lock_switch().