Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 47397118 authored by Arnd Bergmann's avatar Arnd Bergmann Committed by Herbert Xu
Browse files

crypto: serpent - mark __serpent_setkey_sbox noinline 



The same bug that gcc hit in the past is apparently now showing
up with clang, which decides to inline __serpent_setkey_sbox:

crypto/serpent_generic.c:268:5: error: stack frame size of 2112 bytes in function '__serpent_setkey' [-Werror,-Wframe-larger-than=]

Marking it 'noinline' reduces the stack usage from 2112 bytes to
192 and 96 bytes, respectively, and seems to generate more
useful object code.

Fixes: c871c10e ("crypto: serpent - improve __serpent_setkey with UBSAN")
Signed-off-by: default avatarArnd Bergmann <arnd@arndb.de>
Reviewed-by: default avatarEric Biggers <ebiggers@kernel.org>
Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
parent 149c4e6e

crypto/serpent_generic.c

+7 −1
Original line number Diff line number Diff line
@@ -229,7 +229,13 @@ 
	x4 ^= x2;					\

	})



static void __serpent_setkey_sbox(u32 r0, u32 r1, u32 r2, u32 r3, u32 r4, u32 *k)

/*

 * both gcc and clang have misoptimized this function in the past,

 * producing horrible object code from spilling temporary variables

 * on the stack. Forcing this part out of line avoids that.

 */

static noinline void __serpent_setkey_sbox(u32 r0, u32 r1, u32 r2,

					   u32 r3, u32 r4, u32 *k)

{

	k += 100;

	S3(r3, r4, r0, r1, r2); store_and_load_keys(r1, r2, r4, r3, 28, 24);