Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 0f561fce authored by Dave Hansen's avatar Dave Hansen Committed by Ingo Molnar
Browse files

x86/pti: Enable global pages for shared areas 



The entry/exit text and cpu_entry_area are mapped into userspace and
the kernel.  But, they are not _PAGE_GLOBAL.  This creates unnecessary
TLB misses.

Add the _PAGE_GLOBAL flag for these areas.

Signed-off-by: default avatarDave Hansen <dave.hansen@linux.intel.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hugh Dickins <hughd@google.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Kees Cook <keescook@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Nadav Amit <namit@vmware.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20180406205515.2977EE7D@viggo.jf.intel.com


Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
parent 639d6aaf

arch/x86/mm/cpu_entry_area.c

+13 −1
Original line number Diff line number Diff line
@@ -27,8 +27,20 @@ EXPORT_SYMBOL(get_cpu_entry_area); 
void cea_set_pte(void *cea_vaddr, phys_addr_t pa, pgprot_t flags)

{

	unsigned long va = (unsigned long) cea_vaddr;

	pte_t pte = pfn_pte(pa >> PAGE_SHIFT, flags);



	set_pte_vaddr(va, pfn_pte(pa >> PAGE_SHIFT, flags));

	/*

	 * The cpu_entry_area is shared between the user and kernel

	 * page tables.  All of its ptes can safely be global.

	 * _PAGE_GLOBAL gets reused to help indicate PROT_NONE for

	 * non-present PTEs, so be careful not to set it in that

	 * case to avoid confusion.

	 */

	if (boot_cpu_has(X86_FEATURE_PGE) &&

	    (pgprot_val(flags) & _PAGE_PRESENT))

		pte = pte_set_flags(pte, _PAGE_GLOBAL);



	set_pte_vaddr(va, pte);

}



static void __init

arch/x86/mm/pti.c

+22 −1
Original line number Diff line number Diff line
@@ -299,6 +299,27 @@ pti_clone_pmds(unsigned long start, unsigned long end, pmdval_t clear) 
		if (WARN_ON(!target_pmd))

			return;



		/*

		 * Only clone present PMDs.  This ensures only setting

		 * _PAGE_GLOBAL on present PMDs.  This should only be

		 * called on well-known addresses anyway, so a non-

		 * present PMD would be a surprise.

		 */

		if (WARN_ON(!(pmd_flags(*pmd) & _PAGE_PRESENT)))

			return;



		/*

		 * Setting 'target_pmd' below creates a mapping in both

		 * the user and kernel page tables.  It is effectively

		 * global, so set it as global in both copies.  Note:

		 * the X86_FEATURE_PGE check is not _required_ because

		 * the CPU ignores _PAGE_GLOBAL when PGE is not

		 * supported.  The check keeps consistentency with

		 * code that only set this bit when supported.

		 */

		if (boot_cpu_has(X86_FEATURE_PGE))

			*pmd = pmd_set_flags(*pmd, _PAGE_GLOBAL);



		/*

		 * Copy the PMD.  That is, the kernelmode and usermode

		 * tables will share the last-level page tables of this
@@ -348,7 +369,7 @@ static void __init pti_clone_entry_text(void) 
{

	pti_clone_pmds((unsigned long) __entry_text_start,

			(unsigned long) __irqentry_text_end,

		       _PAGE_RW | _PAGE_GLOBAL);

		       _PAGE_RW);

}



/*