Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit ff867910 authored by Joerg Roedel's avatar Joerg Roedel Committed by Greg Kroah-Hartman
Browse files

x86/mm: Flush global TLB when switching to trampoline page-table 



[ Upstream commit 71d5049b053876afbde6c3273250b76935494ab2 ]

Move the switching code into a function so that it can be re-used and
add a global TLB flush. This makes sure that usage of memory which is
not mapped in the trampoline page-table is reliably caught.

Also move the clearing of CR4.PCIDE before the CR3 switch because the
cr4_clear_bits() function will access data not mapped into the
trampoline page-table.

Signed-off-by: default avatarJoerg Roedel <jroedel@suse.de>
Signed-off-by: default avatarBorislav Petkov <bp@suse.de>
Link: https://lore.kernel.org/r/20211202153226.22946-4-joro@8bytes.org


Signed-off-by: default avatarSasha Levin <sashal@kernel.org>
parent 16f2ef98

arch/x86/include/asm/realmode.h

+1 −0
Original line number Diff line number Diff line
@@ -82,6 +82,7 @@ static inline void set_real_mode_mem(phys_addr_t mem) 
}



void reserve_real_mode(void);

void load_trampoline_pgtable(void);



#endif /* __ASSEMBLY__ */

arch/x86/kernel/reboot.c

+2 −10
Original line number Diff line number Diff line
@@ -113,17 +113,9 @@ void __noreturn machine_real_restart(unsigned int type) 
	spin_unlock(&rtc_lock);



	/*

	 * Switch back to the initial page table.

	 * Switch to the trampoline page table.

	 */

#ifdef CONFIG_X86_32

	load_cr3(initial_page_table);

#else

	write_cr3(real_mode_header->trampoline_pgd);



	/* Exiting long mode will fail if CR4.PCIDE is set. */

	if (boot_cpu_has(X86_FEATURE_PCID))

		cr4_clear_bits(X86_CR4_PCIDE);

#endif

	load_trampoline_pgtable();



	/* Jump to the identity-mapped low memory code */

#ifdef CONFIG_X86_32

arch/x86/realmode/init.c

+26 −0
Original line number Diff line number Diff line
@@ -16,6 +16,32 @@ u32 *trampoline_cr4_features; 
/* Hold the pgd entry used on booting additional CPUs */

pgd_t trampoline_pgd_entry;



void load_trampoline_pgtable(void)

{

#ifdef CONFIG_X86_32

	load_cr3(initial_page_table);

#else

	/*

	 * This function is called before exiting to real-mode and that will

	 * fail with CR4.PCIDE still set.

	 */

	if (boot_cpu_has(X86_FEATURE_PCID))

		cr4_clear_bits(X86_CR4_PCIDE);



	write_cr3(real_mode_header->trampoline_pgd);

#endif



	/*

	 * The CR3 write above will not flush global TLB entries.

	 * Stale, global entries from previous page tables may still be

	 * present.  Flush those stale entries.

	 *

	 * This ensures that memory accessed while running with

	 * trampoline_pgd is *actually* mapped into trampoline_pgd.

	 */

	__flush_tlb_all();

}



void __init reserve_real_mode(void)

{

	phys_addr_t mem;