Loading Documentation/arm/kernel_user_helpers.txt 0 → 100644 +267 −0 Original line number Diff line number Diff line Kernel-provided User Helpers ============================ These are segment of kernel provided user code reachable from user space at a fixed address in kernel memory. This is used to provide user space with some operations which require kernel help because of unimplemented native feature and/or instructions in many ARM CPUs. The idea is for this code to be executed directly in user mode for best efficiency but which is too intimate with the kernel counter part to be left to user libraries. In fact this code might even differ from one CPU to another depending on the available instruction set, or whether it is a SMP systems. In other words, the kernel reserves the right to change this code as needed without warning. Only the entry points and their results as documented here are guaranteed to be stable. This is different from (but doesn't preclude) a full blown VDSO implementation, however a VDSO would prevent some assembly tricks with constants that allows for efficient branching to those code segments. And since those code segments only use a few cycles before returning to user code, the overhead of a VDSO indirect far call would add a measurable overhead to such minimalistic operations. User space is expected to bypass those helpers and implement those things inline (either in the code emitted directly by the compiler, or part of the implementation of a library call) when optimizing for a recent enough processor that has the necessary native support, but only if resulting binaries are already to be incompatible with earlier ARM processors due to useage of similar native instructions for other things. In other words don't make binaries unable to run on earlier processors just for the sake of not using these kernel helpers if your compiled code is not going to use new instructions for other purpose. New helpers may be added over time, so an older kernel may be missing some helpers present in a newer kernel. For this reason, programs must check the value of __kuser_helper_version (see below) before assuming that it is safe to call any particular helper. This check should ideally be performed only once at process startup time, and execution aborted early if the required helpers are not provided by the kernel version that process is running on. kuser_helper_version -------------------- Location: 0xffff0ffc Reference declaration: extern int32_t __kuser_helper_version; Definition: This field contains the number of helpers being implemented by the running kernel. User space may read this to determine the availability of a particular helper. Usage example: #define __kuser_helper_version (*(int32_t *)0xffff0ffc) void check_kuser_version(void) { if (__kuser_helper_version < 2) { fprintf(stderr, "can't do atomic operations, kernel too old\n"); abort(); } } Notes: User space may assume that the value of this field never changes during the lifetime of any single process. This means that this field can be read once during the initialisation of a library or startup phase of a program. kuser_get_tls ------------- Location: 0xffff0fe0 Reference prototype: void * __kuser_get_tls(void); Input: lr = return address Output: r0 = TLS value Clobbered registers: none Definition: Get the TLS value as previously set via the __ARM_NR_set_tls syscall. Usage example: typedef void * (__kuser_get_tls_t)(void); #define __kuser_get_tls (*(__kuser_get_tls_t *)0xffff0fe0) void foo() { void *tls = __kuser_get_tls(); printf("TLS = %p\n", tls); } Notes: - Valid only if __kuser_helper_version >= 1 (from kernel version 2.6.12). kuser_cmpxchg ------------- Location: 0xffff0fc0 Reference prototype: int __kuser_cmpxchg(int32_t oldval, int32_t newval, volatile int32_t *ptr); Input: r0 = oldval r1 = newval r2 = ptr lr = return address Output: r0 = success code (zero or non-zero) C flag = set if r0 == 0, clear if r0 != 0 Clobbered registers: r3, ip, flags Definition: Atomically store newval in *ptr only if *ptr is equal to oldval. Return zero if *ptr was changed or non-zero if no exchange happened. The C flag is also set if *ptr was changed to allow for assembly optimization in the calling code. Usage example: typedef int (__kuser_cmpxchg_t)(int oldval, int newval, volatile int *ptr); #define __kuser_cmpxchg (*(__kuser_cmpxchg_t *)0xffff0fc0) int atomic_add(volatile int *ptr, int val) { int old, new; do { old = *ptr; new = old + val; } while(__kuser_cmpxchg(old, new, ptr)); return new; } Notes: - This routine already includes memory barriers as needed. - Valid only if __kuser_helper_version >= 2 (from kernel version 2.6.12). kuser_memory_barrier -------------------- Location: 0xffff0fa0 Reference prototype: void __kuser_memory_barrier(void); Input: lr = return address Output: none Clobbered registers: none Definition: Apply any needed memory barrier to preserve consistency with data modified manually and __kuser_cmpxchg usage. Usage example: typedef void (__kuser_dmb_t)(void); #define __kuser_dmb (*(__kuser_dmb_t *)0xffff0fa0) Notes: - Valid only if __kuser_helper_version >= 3 (from kernel version 2.6.15). kuser_cmpxchg64 --------------- Location: 0xffff0f60 Reference prototype: int __kuser_cmpxchg64(const int64_t *oldval, const int64_t *newval, volatile int64_t *ptr); Input: r0 = pointer to oldval r1 = pointer to newval r2 = pointer to target value lr = return address Output: r0 = success code (zero or non-zero) C flag = set if r0 == 0, clear if r0 != 0 Clobbered registers: r3, lr, flags Definition: Atomically store the 64-bit value pointed by *newval in *ptr only if *ptr is equal to the 64-bit value pointed by *oldval. Return zero if *ptr was changed or non-zero if no exchange happened. The C flag is also set if *ptr was changed to allow for assembly optimization in the calling code. Usage example: typedef int (__kuser_cmpxchg64_t)(const int64_t *oldval, const int64_t *newval, volatile int64_t *ptr); #define __kuser_cmpxchg64 (*(__kuser_cmpxchg64_t *)0xffff0f60) int64_t atomic_add64(volatile int64_t *ptr, int64_t val) { int64_t old, new; do { old = *ptr; new = old + val; } while(__kuser_cmpxchg64(&old, &new, ptr)); return new; } Notes: - This routine already includes memory barriers as needed. - Due to the length of this sequence, this spans 2 conventional kuser "slots", therefore 0xffff0f80 is not used as a valid entry point. - Valid only if __kuser_helper_version >= 5 (from kernel version 3.1). arch/arm/Kconfig +1 −1 Original line number Diff line number Diff line Loading @@ -10,7 +10,7 @@ config ARM select GENERIC_ATOMIC64 if (CPU_V6 || !CPU_32v6K || !AEABI) select HAVE_OPROFILE if (HAVE_PERF_EVENTS) select HAVE_ARCH_KGDB select HAVE_KPROBES if (!XIP_KERNEL && !THUMB2_KERNEL) select HAVE_KPROBES if !XIP_KERNEL select HAVE_KRETPROBES if (HAVE_KPROBES) select HAVE_FUNCTION_TRACER if (!XIP_KERNEL) select HAVE_FTRACE_MCOUNT_RECORD if (!XIP_KERNEL) Loading arch/arm/include/asm/assembler.h +9 −0 Original line number Diff line number Diff line Loading @@ -293,4 +293,13 @@ .macro ldrusr, reg, ptr, inc, cond=al, rept=1, abort=9001f usracc ldr, \reg, \ptr, \inc, \cond, \rept, \abort .endm /* Utility macro for declaring string literals */ .macro string name:req, string .type \name , #object \name: .asciz "\string" .size \name , . - \name .endm #endif /* __ASM_ASSEMBLER_H__ */ arch/arm/include/asm/dma.h +6 −5 Original line number Diff line number Diff line #ifndef __ASM_ARM_DMA_H #define __ASM_ARM_DMA_H #include <asm/memory.h> /* * This is the maximum virtual address which can be DMA'd from. */ #ifndef ARM_DMA_ZONE_SIZE #define MAX_DMA_ADDRESS 0xffffffff #ifndef CONFIG_ZONE_DMA #define MAX_DMA_ADDRESS 0xffffffffUL #else #define MAX_DMA_ADDRESS (PAGE_OFFSET + ARM_DMA_ZONE_SIZE) #define MAX_DMA_ADDRESS ({ \ extern unsigned long arm_dma_zone_size; \ arm_dma_zone_size ? \ (PAGE_OFFSET + arm_dma_zone_size) : 0xffffffffUL; }) #endif #ifdef CONFIG_ISA_DMA_API Loading arch/arm/include/asm/hwcap.h +20 −16 Original line number Diff line number Diff line Loading @@ -4,22 +4,26 @@ /* * HWCAP flags - for elf_hwcap (in kernel) and AT_HWCAP */ #define HWCAP_SWP 1 #define HWCAP_HALF 2 #define HWCAP_THUMB 4 #define HWCAP_26BIT 8 /* Play it safe */ #define HWCAP_FAST_MULT 16 #define HWCAP_FPA 32 #define HWCAP_VFP 64 #define HWCAP_EDSP 128 #define HWCAP_JAVA 256 #define HWCAP_IWMMXT 512 #define HWCAP_CRUNCH 1024 #define HWCAP_THUMBEE 2048 #define HWCAP_NEON 4096 #define HWCAP_VFPv3 8192 #define HWCAP_VFPv3D16 16384 #define HWCAP_TLS 32768 #define HWCAP_SWP (1 << 0) #define HWCAP_HALF (1 << 1) #define HWCAP_THUMB (1 << 2) #define HWCAP_26BIT (1 << 3) /* Play it safe */ #define HWCAP_FAST_MULT (1 << 4) #define HWCAP_FPA (1 << 5) #define HWCAP_VFP (1 << 6) #define HWCAP_EDSP (1 << 7) #define HWCAP_JAVA (1 << 8) #define HWCAP_IWMMXT (1 << 9) #define HWCAP_CRUNCH (1 << 10) #define HWCAP_THUMBEE (1 << 11) #define HWCAP_NEON (1 << 12) #define HWCAP_VFPv3 (1 << 13) #define HWCAP_VFPv3D16 (1 << 14) #define HWCAP_TLS (1 << 15) #define HWCAP_VFPv4 (1 << 16) #define HWCAP_IDIVA (1 << 17) #define HWCAP_IDIVT (1 << 18) #define HWCAP_IDIV (HWCAP_IDIVA | HWCAP_IDIVT) #if defined(__KERNEL__) && !defined(__ASSEMBLY__) /* Loading Loading
Documentation/arm/kernel_user_helpers.txt 0 → 100644 +267 −0 Original line number Diff line number Diff line Kernel-provided User Helpers ============================ These are segment of kernel provided user code reachable from user space at a fixed address in kernel memory. This is used to provide user space with some operations which require kernel help because of unimplemented native feature and/or instructions in many ARM CPUs. The idea is for this code to be executed directly in user mode for best efficiency but which is too intimate with the kernel counter part to be left to user libraries. In fact this code might even differ from one CPU to another depending on the available instruction set, or whether it is a SMP systems. In other words, the kernel reserves the right to change this code as needed without warning. Only the entry points and their results as documented here are guaranteed to be stable. This is different from (but doesn't preclude) a full blown VDSO implementation, however a VDSO would prevent some assembly tricks with constants that allows for efficient branching to those code segments. And since those code segments only use a few cycles before returning to user code, the overhead of a VDSO indirect far call would add a measurable overhead to such minimalistic operations. User space is expected to bypass those helpers and implement those things inline (either in the code emitted directly by the compiler, or part of the implementation of a library call) when optimizing for a recent enough processor that has the necessary native support, but only if resulting binaries are already to be incompatible with earlier ARM processors due to useage of similar native instructions for other things. In other words don't make binaries unable to run on earlier processors just for the sake of not using these kernel helpers if your compiled code is not going to use new instructions for other purpose. New helpers may be added over time, so an older kernel may be missing some helpers present in a newer kernel. For this reason, programs must check the value of __kuser_helper_version (see below) before assuming that it is safe to call any particular helper. This check should ideally be performed only once at process startup time, and execution aborted early if the required helpers are not provided by the kernel version that process is running on. kuser_helper_version -------------------- Location: 0xffff0ffc Reference declaration: extern int32_t __kuser_helper_version; Definition: This field contains the number of helpers being implemented by the running kernel. User space may read this to determine the availability of a particular helper. Usage example: #define __kuser_helper_version (*(int32_t *)0xffff0ffc) void check_kuser_version(void) { if (__kuser_helper_version < 2) { fprintf(stderr, "can't do atomic operations, kernel too old\n"); abort(); } } Notes: User space may assume that the value of this field never changes during the lifetime of any single process. This means that this field can be read once during the initialisation of a library or startup phase of a program. kuser_get_tls ------------- Location: 0xffff0fe0 Reference prototype: void * __kuser_get_tls(void); Input: lr = return address Output: r0 = TLS value Clobbered registers: none Definition: Get the TLS value as previously set via the __ARM_NR_set_tls syscall. Usage example: typedef void * (__kuser_get_tls_t)(void); #define __kuser_get_tls (*(__kuser_get_tls_t *)0xffff0fe0) void foo() { void *tls = __kuser_get_tls(); printf("TLS = %p\n", tls); } Notes: - Valid only if __kuser_helper_version >= 1 (from kernel version 2.6.12). kuser_cmpxchg ------------- Location: 0xffff0fc0 Reference prototype: int __kuser_cmpxchg(int32_t oldval, int32_t newval, volatile int32_t *ptr); Input: r0 = oldval r1 = newval r2 = ptr lr = return address Output: r0 = success code (zero or non-zero) C flag = set if r0 == 0, clear if r0 != 0 Clobbered registers: r3, ip, flags Definition: Atomically store newval in *ptr only if *ptr is equal to oldval. Return zero if *ptr was changed or non-zero if no exchange happened. The C flag is also set if *ptr was changed to allow for assembly optimization in the calling code. Usage example: typedef int (__kuser_cmpxchg_t)(int oldval, int newval, volatile int *ptr); #define __kuser_cmpxchg (*(__kuser_cmpxchg_t *)0xffff0fc0) int atomic_add(volatile int *ptr, int val) { int old, new; do { old = *ptr; new = old + val; } while(__kuser_cmpxchg(old, new, ptr)); return new; } Notes: - This routine already includes memory barriers as needed. - Valid only if __kuser_helper_version >= 2 (from kernel version 2.6.12). kuser_memory_barrier -------------------- Location: 0xffff0fa0 Reference prototype: void __kuser_memory_barrier(void); Input: lr = return address Output: none Clobbered registers: none Definition: Apply any needed memory barrier to preserve consistency with data modified manually and __kuser_cmpxchg usage. Usage example: typedef void (__kuser_dmb_t)(void); #define __kuser_dmb (*(__kuser_dmb_t *)0xffff0fa0) Notes: - Valid only if __kuser_helper_version >= 3 (from kernel version 2.6.15). kuser_cmpxchg64 --------------- Location: 0xffff0f60 Reference prototype: int __kuser_cmpxchg64(const int64_t *oldval, const int64_t *newval, volatile int64_t *ptr); Input: r0 = pointer to oldval r1 = pointer to newval r2 = pointer to target value lr = return address Output: r0 = success code (zero or non-zero) C flag = set if r0 == 0, clear if r0 != 0 Clobbered registers: r3, lr, flags Definition: Atomically store the 64-bit value pointed by *newval in *ptr only if *ptr is equal to the 64-bit value pointed by *oldval. Return zero if *ptr was changed or non-zero if no exchange happened. The C flag is also set if *ptr was changed to allow for assembly optimization in the calling code. Usage example: typedef int (__kuser_cmpxchg64_t)(const int64_t *oldval, const int64_t *newval, volatile int64_t *ptr); #define __kuser_cmpxchg64 (*(__kuser_cmpxchg64_t *)0xffff0f60) int64_t atomic_add64(volatile int64_t *ptr, int64_t val) { int64_t old, new; do { old = *ptr; new = old + val; } while(__kuser_cmpxchg64(&old, &new, ptr)); return new; } Notes: - This routine already includes memory barriers as needed. - Due to the length of this sequence, this spans 2 conventional kuser "slots", therefore 0xffff0f80 is not used as a valid entry point. - Valid only if __kuser_helper_version >= 5 (from kernel version 3.1).
arch/arm/Kconfig +1 −1 Original line number Diff line number Diff line Loading @@ -10,7 +10,7 @@ config ARM select GENERIC_ATOMIC64 if (CPU_V6 || !CPU_32v6K || !AEABI) select HAVE_OPROFILE if (HAVE_PERF_EVENTS) select HAVE_ARCH_KGDB select HAVE_KPROBES if (!XIP_KERNEL && !THUMB2_KERNEL) select HAVE_KPROBES if !XIP_KERNEL select HAVE_KRETPROBES if (HAVE_KPROBES) select HAVE_FUNCTION_TRACER if (!XIP_KERNEL) select HAVE_FTRACE_MCOUNT_RECORD if (!XIP_KERNEL) Loading
arch/arm/include/asm/assembler.h +9 −0 Original line number Diff line number Diff line Loading @@ -293,4 +293,13 @@ .macro ldrusr, reg, ptr, inc, cond=al, rept=1, abort=9001f usracc ldr, \reg, \ptr, \inc, \cond, \rept, \abort .endm /* Utility macro for declaring string literals */ .macro string name:req, string .type \name , #object \name: .asciz "\string" .size \name , . - \name .endm #endif /* __ASM_ASSEMBLER_H__ */
arch/arm/include/asm/dma.h +6 −5 Original line number Diff line number Diff line #ifndef __ASM_ARM_DMA_H #define __ASM_ARM_DMA_H #include <asm/memory.h> /* * This is the maximum virtual address which can be DMA'd from. */ #ifndef ARM_DMA_ZONE_SIZE #define MAX_DMA_ADDRESS 0xffffffff #ifndef CONFIG_ZONE_DMA #define MAX_DMA_ADDRESS 0xffffffffUL #else #define MAX_DMA_ADDRESS (PAGE_OFFSET + ARM_DMA_ZONE_SIZE) #define MAX_DMA_ADDRESS ({ \ extern unsigned long arm_dma_zone_size; \ arm_dma_zone_size ? \ (PAGE_OFFSET + arm_dma_zone_size) : 0xffffffffUL; }) #endif #ifdef CONFIG_ISA_DMA_API Loading
arch/arm/include/asm/hwcap.h +20 −16 Original line number Diff line number Diff line Loading @@ -4,22 +4,26 @@ /* * HWCAP flags - for elf_hwcap (in kernel) and AT_HWCAP */ #define HWCAP_SWP 1 #define HWCAP_HALF 2 #define HWCAP_THUMB 4 #define HWCAP_26BIT 8 /* Play it safe */ #define HWCAP_FAST_MULT 16 #define HWCAP_FPA 32 #define HWCAP_VFP 64 #define HWCAP_EDSP 128 #define HWCAP_JAVA 256 #define HWCAP_IWMMXT 512 #define HWCAP_CRUNCH 1024 #define HWCAP_THUMBEE 2048 #define HWCAP_NEON 4096 #define HWCAP_VFPv3 8192 #define HWCAP_VFPv3D16 16384 #define HWCAP_TLS 32768 #define HWCAP_SWP (1 << 0) #define HWCAP_HALF (1 << 1) #define HWCAP_THUMB (1 << 2) #define HWCAP_26BIT (1 << 3) /* Play it safe */ #define HWCAP_FAST_MULT (1 << 4) #define HWCAP_FPA (1 << 5) #define HWCAP_VFP (1 << 6) #define HWCAP_EDSP (1 << 7) #define HWCAP_JAVA (1 << 8) #define HWCAP_IWMMXT (1 << 9) #define HWCAP_CRUNCH (1 << 10) #define HWCAP_THUMBEE (1 << 11) #define HWCAP_NEON (1 << 12) #define HWCAP_VFPv3 (1 << 13) #define HWCAP_VFPv3D16 (1 << 14) #define HWCAP_TLS (1 << 15) #define HWCAP_VFPv4 (1 << 16) #define HWCAP_IDIVA (1 << 17) #define HWCAP_IDIVT (1 << 18) #define HWCAP_IDIV (HWCAP_IDIVA | HWCAP_IDIVT) #if defined(__KERNEL__) && !defined(__ASSEMBLY__) /* Loading