Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit e80c3533 authored by Daniel Borkmann's avatar Daniel Borkmann Committed by Greg Kroah-Hartman
Browse files

bpf: Introduce BPF nospec instruction for mitigating Spectre v4 



commit f5e81d1117501546b7be050c5fbafa6efd2c722c upstream.

In case of JITs, each of the JIT backends compiles the BPF nospec instruction
/either/ to a machine instruction which emits a speculation barrier /or/ to
/no/ machine instruction in case the underlying architecture is not affected
by Speculative Store Bypass or has different mitigations in place already.

This covers both x86 and (implicitly) arm64: In case of x86, we use 'lfence'
instruction for mitigation. In case of arm64, we rely on the firmware mitigation
as controlled via the ssbd kernel parameter. Whenever the mitigation is enabled,
it works for all of the kernel code with no need to provide any additional
instructions here (hence only comment in arm64 JIT). Other archs can follow
as needed. The BPF nospec instruction is specifically targeting Spectre v4
since i) we don't use a serialization barrier for the Spectre v1 case, and
ii) mitigation instructions for v1 and v4 might be different on some archs.

The BPF nospec is required for a future commit, where the BPF verifier does
annotate intermediate BPF programs with speculation barriers.

Co-developed-by: default avatarPiotr Krysiuk <piotras@gmail.com>
Co-developed-by: default avatarBenedict Schlueter <benedict.schlueter@rub.de>
Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
Signed-off-by: default avatarPiotr Krysiuk <piotras@gmail.com>
Signed-off-by: default avatarBenedict Schlueter <benedict.schlueter@rub.de>
Acked-by: default avatarAlexei Starovoitov <ast@kernel.org>
Signed-off-by: default avatarSasha Levin <sashal@kernel.org>
[OP: - adjusted context for 5.4
     - apply riscv changes to /arch/riscv/net/bpf_jit_comp.c]
Signed-off-by: default avatarOvidiu Panait <ovidiu.panait@windriver.com>
Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
parent 1c9424a7

arch/arm/net/bpf_jit_32.c

+3 −0
Original line number Diff line number Diff line
@@ -1602,6 +1602,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx) 
		rn = arm_bpf_get_reg32(src_lo, tmp2[1], ctx);

		emit_ldx_r(dst, rn, off, ctx, BPF_SIZE(code));

		break;

	/* speculation barrier */

	case BPF_ST | BPF_NOSPEC:

		break;

	/* ST: *(size *)(dst + off) = imm */

	case BPF_ST | BPF_MEM | BPF_W:

	case BPF_ST | BPF_MEM | BPF_H:

arch/arm64/net/bpf_jit_comp.c

+13 −0
Original line number Diff line number Diff line
@@ -701,6 +701,19 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, 
		}

		break;



	/* speculation barrier */

	case BPF_ST | BPF_NOSPEC:

		/*

		 * Nothing required here.

		 *

		 * In case of arm64, we rely on the firmware mitigation of

		 * Speculative Store Bypass as controlled via the ssbd kernel

		 * parameter. Whenever the mitigation is enabled, it works

		 * for all of the kernel code with no need to provide any

		 * additional instructions.

		 */

		break;



	/* ST: *(size *)(dst + off) = imm */

	case BPF_ST | BPF_MEM | BPF_W:

	case BPF_ST | BPF_MEM | BPF_H:

arch/mips/net/ebpf_jit.c

+3 −0
Original line number Diff line number Diff line
@@ -1355,6 +1355,9 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, 
		}

		break;



	case BPF_ST | BPF_NOSPEC: /* speculation barrier */

		break;



	case BPF_ST | BPF_B | BPF_MEM:

	case BPF_ST | BPF_H | BPF_MEM:

	case BPF_ST | BPF_W | BPF_MEM:

arch/powerpc/net/bpf_jit_comp64.c

+6 −0
Original line number Diff line number Diff line
@@ -644,6 +644,12 @@ static int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, 
			}

			break;



		/*

		 * BPF_ST NOSPEC (speculation barrier)

		 */

		case BPF_ST | BPF_NOSPEC:

			break;



		/*

		 * BPF_ST(X)

		 */

arch/riscv/net/bpf_jit_comp.c

+4 −0
Original line number Diff line number Diff line
@@ -1313,6 +1313,10 @@ static int emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, 
		emit(rv_ld(rd, 0, RV_REG_T1), ctx);

		break;



	/* speculation barrier */

	case BPF_ST | BPF_NOSPEC:

		break;



	/* ST: *(size *)(dst + off) = imm */

	case BPF_ST | BPF_MEM | BPF_B:

		emit_imm(RV_REG_T1, imm, ctx);