Commit c5ce7c69 authored May 26, 2016 by Nicolai Stange Committed by Herbert Xu May 31, 2016

lib/digsig: digsig_verify_rsa(): return -EINVAL if modulo length is zero



Currently, if digsig_verify_rsa() detects that the modulo's length is zero,
i.e. mlen == 0, it returns -ENOMEM which doesn't really fit here.

Make digsig_verify_rsa() return -EINVAL upon mlen == 0.

Signed-off-by: Nicolai Stange <nicstange@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

parent 03cdfaad

lib/digsig.c

+5 −3

Original line number	Diff line number	Diff line
		@@ -114,13 +114,15 @@ static int digsig_verify_rsa(struct key *key,
		datap += remaining;
		}

		err = -ENOMEM;

		mblen = mpi_get_nbits(pkey[0]);
		mlen = DIV_ROUND_UP(mblen, 8);

		if (mlen == 0)
		if (mlen == 0) {
		err = -EINVAL;
		goto err;
		}

		err = -ENOMEM;

		out1 = kzalloc(mlen, GFP_KERNEL);
		if (!out1)