Merge remote-tracking branch 'aosp/upstream-f2fs-stable-linux-4.19.y' into android-4.19

entropy from the master key.  HKDF is also standardized and widely

used by other software, whereas the AES-128-ECB based KDF is ad-hoc.

Per-file keys

-------------

Per-file encryption keys

------------------------

Since each master key can protect many files, it is necessary to

"tweak" the encryption of each file so that the same plaintext in two

Therefore, to improve performance and save memory, for Adiantum a

"direct key" configuration is supported.  When the user has enabled

this by setting FSCRYPT_POLICY_FLAG_DIRECT_KEY in the fscrypt policy,

per-file keys are not used.  Instead, whenever any data (contents or

filenames) is encrypted, the file's 16-byte nonce is included in the

IV.  Moreover:

per-file encryption keys are not used.  Instead, whenever any data

(contents or filenames) is encrypted, the file's 16-byte nonce is

included in the IV.  Moreover:

- For v1 encryption policies, the encryption is done directly with the

  master key.  Because of this, users **must not** use the same master

identifier" is also derived using the KDF.  This value is stored in

the clear, since it is needed to reliably identify the key itself.

Dirhash keys

------------

For directories that are indexed using a secret-keyed dirhash over the

plaintext filenames, the KDF is also used to derive a 128-bit

SipHash-2-4 key per directory in order to hash filenames.  This works

just like deriving a per-file encryption key, except that a different

KDF context is used.  Currently, only casefolded ("case-insensitive")

encrypted directories use this style of hashing.

Encryption modes and usage

==========================

Adiantum is a (primarily) stream cipher-based mode that is fast even

on CPUs without dedicated crypto instructions.  It's also a true

wide-block mode, unlike XTS.  It can also eliminate the need to derive

per-file keys.  However, it depends on the security of two primitives,

XChaCha12 and AES-256, rather than just one.  See the paper

"Adiantum: length-preserving encryption for entry-level processors"

(https://eprint.iacr.org/2018/720.pdf) for more details.  To use

Adiantum, CONFIG_CRYPTO_ADIANTUM must be enabled.  Also, fast

per-file encryption keys.  However, it depends on the security of two

primitives, XChaCha12 and AES-256, rather than just one.  See the

paper "Adiantum: length-preserving encryption for entry-level

processors" (https://eprint.iacr.org/2018/720.pdf) for more details.

To use Adiantum, CONFIG_CRYPTO_ADIANTUM must be enabled.  Also, fast

implementations of ChaCha and NHPoly1305 should be enabled, e.g.

CONFIG_CRYPTO_CHACHA20_NEON and CONFIG_CRYPTO_NHPOLY1305_NEON for ARM.

- ``EEXIST``: the file is already encrypted with an encryption policy

  different from the one specified

- ``EINVAL``: an invalid encryption policy was specified (invalid

  version, mode(s), or flags; or reserved bits were set)

  version, mode(s), or flags; or reserved bits were set); or a v1

  encryption policy was specified but the directory has the casefold

  flag enabled (casefolding is incompatible with v1 policies).

- ``ENOKEY``: a v2 encryption policy was specified, but the key with

  the specified ``master_key_identifier`` has not been added, nor does

  the process have the CAP_FOWNER capability in the initial user

    struct fscrypt_add_key_arg {

            struct fscrypt_key_specifier key_spec;

            __u32 raw_size;

            __u32 __reserved[9];

            __u32 key_id;

            __u32 __reserved[8];

            __u8 raw[];

    };

            } u;

    };

    struct fscrypt_provisioning_key_payload {

            __u32 type;

            __u32 __reserved;

            __u8 raw[];

    };

:c:type:`struct fscrypt_add_key_arg` must be zeroed, then initialized

as follows:

  ``Documentation/security/keys/core.rst``).

- ``raw_size`` must be the size of the ``raw`` key provided, in bytes.

  Alternatively, if ``key_id`` is nonzero, this field must be 0, since

  in that case the size is implied by the specified Linux keyring key.

- ``key_id`` is 0 if the raw key is given directly in the ``raw``

  field.  Otherwise ``key_id`` is the ID of a Linux keyring key of

  type "fscrypt-provisioning" whose payload is a :c:type:`struct

  fscrypt_provisioning_key_payload` whose ``raw`` field contains the

  raw key and whose ``type`` field matches ``key_spec.type``.  Since

  ``raw`` is variable-length, the total size of this key's payload

  must be ``sizeof(struct fscrypt_provisioning_key_payload)`` plus the

  raw key size.  The process must have Search permission on this key.

  Most users should leave this 0 and specify the raw key directly.

  The support for specifying a Linux keyring key is intended mainly to

  allow re-adding keys after a filesystem is unmounted and re-mounted,

  without having to store the raw keys in userspace memory.

- ``raw`` is a variable-length field which must contain the actual

  key, ``raw_size`` bytes long.

  key, ``raw_size`` bytes long.  Alternatively, if ``key_id`` is

  nonzero, then this field is unused.

For v2 policy keys, the kernel keeps track of which user (identified

by effective user ID) added the key, and only allows the key to be

- ``EACCES``: FSCRYPT_KEY_SPEC_TYPE_DESCRIPTOR was specified, but the

  caller does not have the CAP_SYS_ADMIN capability in the initial

  user namespace

  user namespace; or the raw key was specified by Linux key ID but the

  process lacks Search permission on the key.

- ``EDQUOT``: the key quota for this user would be exceeded by adding

  the key

- ``EINVAL``: invalid key size or key specifier type, or reserved bits

  were set

- ``EKEYREJECTED``: the raw key was specified by Linux key ID, but the

  key has the wrong type

- ``ENOKEY``: the raw key was specified by Linux key ID, but no key

  exists with that ID

- ``ENOTTY``: this type of filesystem does not implement encryption

- ``EOPNOTSUPP``: the kernel was not configured with encryption

  support for this filesystem, or the filesystem superblock has not

policy structs (see `Setting an encryption policy`_), except that the

context structs also contain a nonce.  The nonce is randomly generated

by the kernel and is used as KDF input or as a tweak to cause

different files to be encrypted differently; see `Per-file keys`_ and

`DIRECT_KEY policies`_.

different files to be encrypted differently; see `Per-file encryption

keys`_ and `DIRECT_KEY policies`_.

Data path changes

-----------------

allows the filesystem to still, with a high degree of confidence, map

the filename given in ->lookup() back to a particular directory entry

that was previously listed by readdir().  See :c:type:`struct

fscrypt_digested_name` in the source for more details.

fscrypt_nokey_name` in the source for more details.

Note that the precise way that filenames are presented to userspace

without the key is subject to change in the future.  It is only meant

config FS_ENCRYPTION

	bool "FS Encryption (Per-file encryption)"

	select CRYPTO

	select CRYPTO_AES

	select CRYPTO_CBC

	select CRYPTO_ECB

	select CRYPTO_XTS

	select CRYPTO_CTS

	select CRYPTO_SHA512

	select CRYPTO_HMAC

	select CRYPTO_HASH

	select CRYPTO_BLKCIPHER

	select KEYS

	help

	  Enable encryption of files and directories.  This

	  decrypted pages in the page cache.  Currently Ext4,

	  F2FS and UBIFS make use of this feature.

# Filesystems supporting encryption must select this if FS_ENCRYPTION.  This

# allows the algorithms to be built as modules when all the filesystems are.

config FS_ENCRYPTION_ALGS

	tristate

	select CRYPTO_AES

	select CRYPTO_CBC

	select CRYPTO_CTS

	select CRYPTO_ECB

	select CRYPTO_HMAC

	select CRYPTO_SHA256

	select CRYPTO_SHA512

	select CRYPTO_XTS

config FS_ENCRYPTION_INLINE_CRYPT

	bool "Enable fscrypt to use inline crypto"

	depends on FS_ENCRYPTION && BLK_INLINE_ENCRYPTION

}

EXPORT_SYMBOL(fscrypt_decrypt_bio);

int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk,

static int fscrypt_zeroout_range_inlinecrypt(const struct inode *inode,

					     pgoff_t lblk,

					     sector_t pblk, unsigned int len)

{

	const unsigned int blockbits = inode->i_blkbits;

	const unsigned int blocksize = 1 << blockbits;

	const bool inlinecrypt = fscrypt_inode_uses_inline_crypto(inode);

	struct page *ciphertext_page;

	const unsigned int blocks_per_page_bits = PAGE_SHIFT - blockbits;

	const unsigned int blocks_per_page = 1 << blocks_per_page_bits;

	unsigned int i;

	struct bio *bio;

	int ret, err = 0;

	if (inlinecrypt) {

		ciphertext_page = ZERO_PAGE(0);

	} else {

		ciphertext_page = fscrypt_alloc_bounce_page(GFP_NOWAIT);

		if (!ciphertext_page)

			return -ENOMEM;

	int ret, err;

	/* This always succeeds since __GFP_DIRECT_RECLAIM is set. */

	bio = bio_alloc(GFP_NOFS, BIO_MAX_PAGES);

	do {

		bio_set_dev(bio, inode->i_sb->s_bdev);

		bio->bi_iter.bi_sector = pblk << (blockbits - 9);

		bio_set_op_attrs(bio, REQ_OP_WRITE, 0);

		fscrypt_set_bio_crypt_ctx(bio, inode, lblk, GFP_NOFS);

		i = 0;

		do {

			unsigned int blocks_this_page =

				min(len, blocks_per_page);

			unsigned int bytes_this_page =

				blocks_this_page << blockbits;

			ret = bio_add_page(bio, ZERO_PAGE(0),

					   bytes_this_page, 0);

			if (WARN_ON(ret != bytes_this_page)) {

				err = -EIO;

				goto out;

			}

			lblk += blocks_this_page;

			pblk += blocks_this_page;

			len -= blocks_this_page;

		} while (++i != BIO_MAX_PAGES && len != 0);

	while (len--) {

		if (!inlinecrypt) {

			err = fscrypt_crypt_block(inode, FS_ENCRYPT, lblk,

						  ZERO_PAGE(0), ciphertext_page,

						  blocksize, 0, GFP_NOFS);

		err = submit_bio_wait(bio);

		if (err)

				goto errout;

			goto out;

		bio_reset(bio);

	} while (len != 0);

	err = 0;

out:

	bio_put(bio);

	return err;

}

		bio = bio_alloc(GFP_NOWAIT, 1);

		if (!bio) {

			err = -ENOMEM;

			goto errout;

/**

 * fscrypt_zeroout_range() - zero out a range of blocks in an encrypted file

 * @inode: the file's inode

 * @lblk: the first file logical block to zero out

 * @pblk: the first filesystem physical block to zero out

 * @len: number of blocks to zero out

 *

 * Zero out filesystem blocks in an encrypted regular file on-disk, i.e. write

 * ciphertext blocks which decrypt to the all-zeroes block.  The blocks must be

 * both logically and physically contiguous.  It's also assumed that the

 * filesystem only uses a single block device, ->s_bdev.

 *

 * Note that since each block uses a different IV, this involves writing a

 * different ciphertext to each block; we can't simply reuse the same one.

 *

 * Return: 0 on success; -errno on failure.

 */

int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk,

			  sector_t pblk, unsigned int len)

{

	const unsigned int blockbits = inode->i_blkbits;

	const unsigned int blocksize = 1 << blockbits;

	const unsigned int blocks_per_page_bits = PAGE_SHIFT - blockbits;

	const unsigned int blocks_per_page = 1 << blocks_per_page_bits;

	struct page *pages[16]; /* write up to 16 pages at a time */

	unsigned int nr_pages;

	unsigned int i;

	unsigned int offset;

	struct bio *bio;

	int ret, err;

	if (len == 0)

		return 0;

	if (fscrypt_inode_uses_inline_crypto(inode))

		return fscrypt_zeroout_range_inlinecrypt(inode, lblk, pblk,

							 len);

	BUILD_BUG_ON(ARRAY_SIZE(pages) > BIO_MAX_PAGES);

	nr_pages = min_t(unsigned int, ARRAY_SIZE(pages),

			 (len + blocks_per_page - 1) >> blocks_per_page_bits);

	/*

	 * We need at least one page for ciphertext.  Allocate the first one

	 * from a mempool, with __GFP_DIRECT_RECLAIM set so that it can't fail.

	 *

	 * Any additional page allocations are allowed to fail, as they only

	 * help performance, and waiting on the mempool for them could deadlock.

	 */

	for (i = 0; i < nr_pages; i++) {

		pages[i] = fscrypt_alloc_bounce_page(i == 0 ? GFP_NOFS :

						     GFP_NOWAIT | __GFP_NOWARN);

		if (!pages[i])

			break;

	}

		fscrypt_set_bio_crypt_ctx(bio, inode, lblk, GFP_NOIO);

	nr_pages = i;

	if (WARN_ON(nr_pages <= 0))

		return -EINVAL;

	/* This always succeeds since __GFP_DIRECT_RECLAIM is set. */

	bio = bio_alloc(GFP_NOFS, nr_pages);

	do {

		bio_set_dev(bio, inode->i_sb->s_bdev);

		bio->bi_iter.bi_sector = pblk << (blockbits - 9);

		bio_set_op_attrs(bio, REQ_OP_WRITE, 0);

		ret = bio_add_page(bio, ciphertext_page, blocksize, 0);

		if (WARN_ON(ret != blocksize)) {

			/* should never happen! */

			bio_put(bio);

			err = -EIO;

			goto errout;

		}

		err = submit_bio_wait(bio);

		if (err == 0 && bio->bi_status)

			err = -EIO;

		bio_put(bio);

		i = 0;

		offset = 0;

		do {

			err = fscrypt_crypt_block(inode, FS_ENCRYPT, lblk,

						  ZERO_PAGE(0), pages[i],

						  blocksize, offset, GFP_NOFS);

			if (err)

			goto errout;

				goto out;

			lblk++;

			pblk++;

			len--;

			offset += blocksize;

			if (offset == PAGE_SIZE || len == 0) {

				ret = bio_add_page(bio, pages[i++], offset, 0);

				if (WARN_ON(ret != offset)) {

					err = -EIO;

					goto out;

				}

				offset = 0;

			}

		} while (i != nr_pages && len != 0);

		err = submit_bio_wait(bio);

		if (err)

			goto out;

		bio_reset(bio);

	} while (len != 0);

	err = 0;

errout:

	if (!inlinecrypt)

		fscrypt_free_bounce_page(ciphertext_page);

out:

	bio_put(bio);

	for (i = 0; i < nr_pages; i++)

		fscrypt_free_bounce_page(pages[i]);

	return err;

}

EXPORT_SYMBOL(fscrypt_zeroout_range);

#include <linux/module.h>

#include <linux/scatterlist.h>

#include <linux/ratelimit.h>

#include <linux/dcache.h>

#include <linux/namei.h>

#include <crypto/skcipher.h>

#include "fscrypt_private.h"

 *		multiple of the filesystem's block size.

 * @offs:      Byte offset within @page of the first block to encrypt.  Must be

 *		a multiple of the filesystem's block size.

 * @gfp_flags: Memory allocation flags

 * @gfp_flags: Memory allocation flags.  See details below.

 *

 * A new bounce page is allocated, and the specified block(s) are encrypted into

 * it.  In the bounce page, the ciphertext block(s) will be located at the same

 *

 * This is for use by the filesystem's ->writepages() method.

 *

 * The bounce page allocation is mempool-backed, so it will always succeed when

 * @gfp_flags includes __GFP_DIRECT_RECLAIM, e.g. when it's GFP_NOFS.  However,

 * only the first page of each bio can be allocated this way.  To prevent

 * deadlocks, for any additional pages a mask like GFP_NOWAIT must be used.

 *

 * Return: the new encrypted bounce page on success; an ERR_PTR() on failure

 */

struct page *fscrypt_encrypt_pagecache_blocks(struct page *page,

}

EXPORT_SYMBOL(fscrypt_decrypt_block_inplace);

/*

 * Validate dentries in encrypted directories to make sure we aren't potentially

 * caching stale dentries after a key has been added.

 */

static int fscrypt_d_revalidate(struct dentry *dentry, unsigned int flags)

{

	struct dentry *dir;

	int err;

	int valid;

	/*

	 * Plaintext names are always valid, since fscrypt doesn't support

	 * reverting to ciphertext names without evicting the directory's inode

	 * -- which implies eviction of the dentries in the directory.

	 */

	if (!(dentry->d_flags & DCACHE_ENCRYPTED_NAME))

		return 1;

	/*

	 * Ciphertext name; valid if the directory's key is still unavailable.

	 *

	 * Although fscrypt forbids rename() on ciphertext names, we still must

	 * use dget_parent() here rather than use ->d_parent directly.  That's

	 * because a corrupted fs image may contain directory hard links, which

	 * the VFS handles by moving the directory's dentry tree in the dcache

	 * each time ->lookup() finds the directory and it already has a dentry

	 * elsewhere.  Thus ->d_parent can be changing, and we must safely grab

	 * a reference to some ->d_parent to prevent it from being freed.

	 */

	if (flags & LOOKUP_RCU)

		return -ECHILD;

	dir = dget_parent(dentry);

	err = fscrypt_get_encryption_info(d_inode(dir));

	valid = !fscrypt_has_encryption_key(d_inode(dir));

	dput(dir);

	if (err < 0)

		return err;

	return valid;

}

const struct dentry_operations fscrypt_d_ops = {

	.d_revalidate = fscrypt_d_revalidate,

};

/**

 * fscrypt_initialize() - allocate major buffers for fs encryption.

 * @cop_flags:  fscrypt operations flags