Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 3d55cc8a authored by Masami Hiramatsu's avatar Masami Hiramatsu Committed by Ingo Molnar
Browse files

x86: Add text_poke_smp for SMP cross modifying code 



Add generic text_poke_smp for SMP which uses stop_machine()
to synchronize modifying code.
This stop_machine() method is officially described at "7.1.3
Handling Self- and Cross-Modifying Code" on the intel's
software developer's manual 3A.

Since stop_machine() can't protect code against NMI/MCE, this
function can not modify those handlers. And also, this function
is basically for modifying multibyte-single-instruction. For
modifying multibyte-multi-instructions, we need another special
trap & detour code.

This code originaly comes from immediate values with
stop_machine() version. Thanks Jason and Mathieu!

Signed-off-by: default avatarMasami Hiramatsu <mhiramat@redhat.com>
Cc: systemtap <systemtap@sources.redhat.com>
Cc: DLE <dle-develop@lists.sourceforge.net>
Cc: Mathieu Desnoyers <compudj@krystal.dyndns.org>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Jim Keniston <jkenisto@us.ibm.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Anders Kaseorg <andersk@ksplice.com>
Cc: Tim Abbott <tabbott@ksplice.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Jason Baron <jbaron@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
LKML-Reference: <20100225133438.6725.80273.stgit@localhost6.localdomain6>
Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
parent f007ea26

arch/x86/include/asm/alternative.h

+3 −1
Original line number Diff line number Diff line
@@ -160,10 +160,12 @@ static inline void apply_paravirt(struct paravirt_patch_site *start, 
 * invalid instruction possible) or if the instructions are changed from a

 * consistent state to another consistent state atomically.

 * More care must be taken when modifying code in the SMP case because of

 * Intel's errata.

 * Intel's errata. text_poke_smp() takes care that errata, but still

 * doesn't support NMI/MCE handler code modifying.

 * On the local CPU you need to be protected again NMI or MCE handlers seeing an

 * inconsistent instruction while you patch.

 */

extern void *text_poke(void *addr, const void *opcode, size_t len);

extern void *text_poke_smp(void *addr, const void *opcode, size_t len);



#endif /* _ASM_X86_ALTERNATIVE_H */

arch/x86/kernel/alternative.c

+60 −0
Original line number Diff line number Diff line
@@ -7,6 +7,7 @@ 
#include <linux/mm.h>

#include <linux/vmalloc.h>

#include <linux/memory.h>

#include <linux/stop_machine.h>

#include <asm/alternative.h>

#include <asm/sections.h>

#include <asm/pgtable.h>
@@ -570,3 +571,62 @@ void *__kprobes text_poke(void *addr, const void *opcode, size_t len) 
	local_irq_restore(flags);

	return addr;

}



/*

 * Cross-modifying kernel text with stop_machine().

 * This code originally comes from immediate value.

 */

static atomic_t stop_machine_first;

static int wrote_text;



struct text_poke_params {

	void *addr;

	const void *opcode;

	size_t len;

};



static int __kprobes stop_machine_text_poke(void *data)

{

	struct text_poke_params *tpp = data;



	if (atomic_dec_and_test(&stop_machine_first)) {

		text_poke(tpp->addr, tpp->opcode, tpp->len);

		smp_wmb();	/* Make sure other cpus see that this has run */

		wrote_text = 1;

	} else {

		while (!wrote_text)

			smp_rmb();

		sync_core();

	}



	flush_icache_range((unsigned long)tpp->addr,

			   (unsigned long)tpp->addr + tpp->len);

	return 0;

}



/**

 * text_poke_smp - Update instructions on a live kernel on SMP

 * @addr: address to modify

 * @opcode: source of the copy

 * @len: length to copy

 *

 * Modify multi-byte instruction by using stop_machine() on SMP. This allows

 * user to poke/set multi-byte text on SMP. Only non-NMI/MCE code modifying

 * should be allowed, since stop_machine() does _not_ protect code against

 * NMI and MCE.

 *

 * Note: Must be called under get_online_cpus() and text_mutex.

 */

void *__kprobes text_poke_smp(void *addr, const void *opcode, size_t len)

{

	struct text_poke_params tpp;



	tpp.addr = addr;

	tpp.opcode = opcode;

	tpp.len = len;

	atomic_set(&stop_machine_first, 1);

	wrote_text = 0;

	stop_machine(stop_machine_text_poke, (void *)&tpp, NULL);

	return addr;

}