Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit e4c0fc5f authored Jun 09, 2017 by Nicholas Piggin Committed by Michael Ellerman Jun 15, 2017

powerpc/64s: Leave interrupts hard enabled in context switch for radix



Commit 4387e9ff25 ("[POWERPC] Fix PMU + soft interrupt disable bug")
hard disabled interrupts over the low level context switch, because
the SLB management can't cope with a PMU interrupt accesing the stack
in that window.

Radix based kernel mapping does not use the SLB so it does not require
interrupts hard disabled here.

This is worth 1-2% in context switch performance on POWER9.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>

parent bc4f65e4

arch/powerpc/kernel/entry_64.S

+8 −0

Original line number	Diff line number	Diff line
		@@ -607,6 +607,14 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_1T_SEGMENT)
		top of the kernel stack. */
		addi r7,r7,THREAD_SIZE-SWITCH_FRAME_SIZE

		/*
		* PMU interrupts in radix may come in here. They will use r1, not
		* PACAKSAVE, so this stack switch will not cause a problem. They
		* will store to the process stack, which may then be migrated to
		* another CPU. However the rq lock release on this CPU paired with
		* the rq lock acquire on the new CPU before the stack becomes
		* active on the new CPU, will order those stores.
		*/
		mr r1,r8 /* start using new stack pointer */
		std r7,PACAKSAVE(r13)

arch/powerpc/kernel/process.c

+8 −6

Original line number	Diff line number	Diff line
		@@ -1199,12 +1199,14 @@ struct task_struct __switch_to(struct task_struct prev,

		__switch_to_tm(prev, new);

		if (!radix_enabled()) {
		/*
		* We can't take a PMU exception inside _switch() since there is a
		* window where the kernel stack SLB and the kernel stack are out
		* of sync. Hard disable here.
		* We can't take a PMU exception inside _switch() since there
		* is a window where the kernel stack SLB and the kernel stack
		* are out of sync. Hard disable here.
		*/
		hard_irq_disable();
		}

		/*
		* Call restore_sprs() before calling _switch(). If we move it after