crypto: msm: Add QTI crypto drivers

Introduction:

=============

Storage encryption has been one of the most required feature from security

point of view. QTI based storage encryption solution uses general purpose

crypto engine. While this kind of solution provide a decent amount of

performance, it falls short as storage speed is improving significantly

continuously. To overcome performance degradation, newer chips are going to

have Inline Crypto Engine (ICE) embedded into storage device. ICE is supposed

to meet the line speed of storage devices.

Hardware Description

====================

ICE is a HW block that is embedded into storage device such as UFS/eMMC. By

default, ICE works in bypass mode i.e. ICE HW does not perform any crypto

operation on data to be processed by storage device. If required, ICE can be

configured to perform crypto operation in one direction (i.e. either encryption

or decryption) or in both direction(both encryption & decryption).

When a switch between the operation modes(plain to crypto or crypto to plain)

is desired for a particular partition, SW must complete all transactions for

that particular partition before switching the crypto mode i.e. no crypto, one

direction crypto or both direction crypto operation. Requests for other

partitions are not impacted due to crypto mode switch.

ICE HW currently supports AES128/256 bit ECB & XTS mode encryption algorithms.

Keys for crypto operations are loaded from SW. Keys are stored in a lookup

table(LUT) located inside ICE HW. Maximum of 32 keys can be loaded in ICE key

LUT. A Key inside the LUT can be referred using a key index.

SW Description

==============

ICE HW has catagorized ICE registers in 2 groups: those which can be accessed by

only secure side i.e. TZ and those which can be accessed by non-secure side such

as HLOS as well. This requires that ICE driver to be split in two pieces: one

running from TZ space and another from HLOS space.

ICE driver from TZ would configure keys as requested by HLOS side.

ICE driver on HLOS side is responsible for initialization of ICE HW.

SW Architecture Diagram

=======================

Following are all the components involved in the ICE driver for control path:

+++++++++++++++++++++++++++++++++++++++++

+               App layer               +

+++++++++++++++++++++++++++++++++++++++++

+             System layer              +

+   ++++++++         +++++++            +

+   + VOLD +         + PFM +            +

+   ++++++++         +++++++            +

+         ||         ||                 +

+         ||         ||                 +

+         \/         \/                 +

+        ++++++++++++++                 +

+        + LibQSEECom +                 +

+        ++++++++++++++                 +

+++++++++++++++++++++++++++++++++++++++++

+             Kernel                    +       +++++++++++++++++

+                                       +       +     KMS       +

+  +++++++  +++++++++++  +++++++++++    +       +++++++++++++++++

+  + ICE +  + Storage +  + QSEECom +    +       +   ICE Driver  +

+++++++++++++++++++++++++++++++++++++++++ <===> +++++++++++++++++

               ||                                    ||

               ||                                    ||

               \/                                    \/

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

+                      Storage Device                           +

+                      ++++++++++++++                           +

+                      +   ICE HW   +                           +

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Use Cases:

----------

a) Device bootup

ICE HW is detected during bootup time and corresponding probe function is

called. ICE driver parses its data from device tree node. ICE HW and storage

HW are tightly coupled. Storage device probing is dependent upon ICE device

probing. ICE driver configures all the required registers to put the ICE HW

in bypass mode.

b) Configuring keys

Currently, there are couple of use cases to configure the keys.

1) Full Disk Encryption(FDE)

System layer(VOLD) at invocation of apps layer would call libqseecom to create

the encryption key. Libqseecom calls qseecom driver to communicate with KMS

module on the secure side i.e. TZ. KMS would call ICE driver on the TZ side to

create and set the keys in ICE HW. At the end of transaction, VOLD would have

key index of key LUT where encryption key is present.

2) Per File Encryption (PFE)

Per File Manager(PFM) calls QSEECom api to create the key. PFM has a peer comp-

onent(PFT) at kernel layer which gets the corresponding key index from PFM.

Following are all the components involved in the ICE driver for data path:

+++++++++++++++++++++++++++++++++++++++++

+               App layer               +

+++++++++++++++++++++++++++++++++++++++++

+              VFS                      +

+---------------------------------------+

+         File System (EXT4)            +

+---------------------------------------+

+             Block Layer               +

+ --------------------------------------+

+                              +++++++  +

+              dm-req-crypt => + PFT +  +

+                              +++++++  +

+                                       +

+---------------------------------------+

+    +++++++++++           +++++++      +

+    + Storage +           + ICE +      +

+++++++++++++++++++++++++++++++++++++++++

+                  ||                   +

+                  || (Storage Req with +

+                  \/  ICE parameters ) +

+++++++++++++++++++++++++++++++++++++++++

+          Storage Device               +

+          ++++++++++++++               +

+          +   ICE HW   +               +

+++++++++++++++++++++++++++++++++++++++++

c) Data transaction

Once the crypto key has been configured, VOLD/PFM creates device mapping for

data partition. As part of device mapping VOLD passes key index, crypto

algorithm, mode and key length to dm layer. In case of PFE, keys are provided

by PFT as and when request is processed by dm-req-crypt. When any application

needs to read/write data, it would go through DM layer which would add crypto

information, provided by VOLD/PFT, to Request. For each Request, Storage driver

would ask ICE driver to configure crypto part of request. ICE driver extracts

crypto data from Request structure and provide it to storage driver which would

finally dispatch request to storage device.

d) Error Handling

Due to issue # 1 mentioned in "Known Issues", ICE driver does not register for

any interrupt. However, it enables sources of interrupt for ICE HW. After each

data transaction, Storage driver receives transaction completion event. As part

of event handling, storage driver calls  ICE driver to check if any of ICE

interrupt status is set. If yes, storage driver returns error to upper layer.

Error handling would be changed in future chips.

Interfaces

==========

ICE driver exposes interfaces for storage driver to :

1. Get the global instance of ICE driver

2. Get the implemented interfaces of the particular ice instance

3. Initialize the ICE HW

4. Reset the ICE HW

5. Resume/Suspend the ICE HW

6. Get the Crypto configuration for the data request for storage

7. Check if current data transaction has generated any interrupt

Driver Parameters

=================

This driver is built and statically linked into the kernel; therefore,

there are no module parameters supported by this driver.

There are no kernel command line parameters supported by this driver.

Power Management

================

ICE driver does not do power management on its own as it is part of storage

hardware. Whenever storage driver receives request for power collapse/suspend

resume, it would call ICE driver which exposes APIs for Storage HW. ICE HW

during power collapse or reset, wipes crypto configuration data. When ICE

driver receives request to resume, it would ask ICE driver on TZ side to

restore the configuration. ICE driver does not do anything as part of power

collapse or suspend event.

Interface:

==========

ICE driver exposes following APIs for storage driver to use:

int (*init)(struct platform_device *, void *, ice_success_cb, ice_error_cb);

	-- This function is invoked by storage controller during initialization of

	storage controller. Storage controller would provide success and error call

	backs which would be invoked asynchronously once ICE HW init is done.

int (*reset)(struct platform_device *);

	-- ICE HW reset as part of storage controller reset. When storage controller

	received reset command, it would call reset on ICE HW. As of now, ICE HW

	does not need to do anything as part of reset.

int (*resume)(struct platform_device *);

	-- ICE HW while going to reset, wipes all crypto keys and other data from ICE

	HW. ICE driver would reconfigure those data as part of resume operation.

int (*suspend)(struct platform_device *);

	-- This API would be called by storage driver when storage device is going to

	suspend mode. As of today, ICE driver does not do anything to handle suspend.

int (*config)(struct platform_device *, struct request* , struct ice_data_setting*);

	-- Storage driver would call this interface to get all crypto data required to

	perform crypto operation.

int (*status)(struct platform_device *);

	-- Storage driver would call this interface to check if previous data transfer

	generated any error.

Config options

==============

This driver is enabled by the kernel config option CONFIG_CRYPTO_DEV_MSM_ICE.

Dependencies

============

ICE driver depends upon corresponding ICE driver on TZ side to function

appropriately.

Known Issues

============

1. ICE HW emits 0s even if it has generated an interrupt

This issue has significant impact on how ICE interrupts are handled. Currently,

ICE driver does not register for any of the ICE interrupts but enables the

sources of interrupt. Once storage driver asks to check the status of interrupt,

it reads and clears the clear status and provide read status to storage driver.

This mechanism though not optimal but prevents filesystem curruption.

This issue has been fixed in newer chips.

2. ICE HW wipes all crypto data during power collapse

This issue necessiate that ICE driver on TZ side store the crypto material

which is not required in the case of general purpose crypto engine.

This issue has been fixed in newer chips.

Further Improvements

====================

Currently, Due to PFE use case, ICE driver is dependent upon dm-req-crypt to

provide the keys as part of request structure. This couples ICE driver with

dm-req-crypt based solution. It is under discussion to expose an IOCTL based

and registeration based interface APIs from ICE driver. ICE driver would use

these two interfaces to find out if any key exists for current request. If

yes, choose the right key index received from IOCTL or registeration based

APIs. If not, dont set any crypto parameter in the request.

Introduction:

=============

The QTI crypto engine (qce) driver is a module that

provides common services for accessing the QTI crypto device.

Currently, the two main clients of qce are

-qcrypto driver (module provided for accessing CE HW by kernel space apps)

-qcedev driver (module provided for accessing CE HW by user space apps)

The crypto engine (qce) driver is a client to the DMA driver for the QTI

DMA device - Application Data Mover (ADM). ADM is used to provide the DMA

transfer capability between QTI crypto device hardware and DDR memory

for crypto operations.

  Figure 1.

  ---------

  Linux kernel

  (ex:IPSec)<-----*  QTI crypto driver----+

			(qcrypto)	  |

		   (for kernel space app) |

					  |

					  +-->|

					      |

					      | *qce   <----> QTI

					      | driver        ADM driver <---> ADM HW

					  +-->|			|		|

					  |			|		|

					  |			|		|

					  |			|		|

   Linux kernel				  |			|		|

   misc device  <--- *QCEDEV Driver-------+			|		|

   interface             (qcedev) 			(Reg interface)	 (DMA interface)

			(for user space app)			\		/

								 \	       /

								  \	      /

								   \	     /

								    \	    /

								     \	   /

								      \	  /

								QTI crypto CE3 HW

 The entities marked with (*) in the Figure 1, are the software components of

 the Linux QTI crypto modules.

===============

IMPORTANT NOTE:

===============

(1) The CE hardware can be accessed either from user space OR kernel space,

    at one time. Both user space and kernel space clients cannot access the

    qce driver (and the CE hardware) at the same time.

	- If your device has user space apps that needs to access the crypto

	  hardware, make sure to have the qcrypto module disabled/unloaded.

	  This will result in the kernel space apps to use the registered

	  software implementation of the crypto algorithms.

	- If your device has kernel space apps that needs to access the

	  crypto hardware, make sure to have qcedev module disabled/unloaded

	  and implement your user space application to use the software

	  implementation (ex: openssl/crypto) of the crypto algorithms.

(2) If your device has Playready(Windows Media DRM) application enabled and

    uses the qcedev module to access the crypto hardware accelerator,

    please be informed that for performance reasons, the CE hardware will need

    to be dedicated to playready application.  Any other user space application

    should be implemented to use the SW implementation (ex: openssl/crypto)

    of the crypto algorithms.

Hardware description:

=====================

QTI Crypto HW device family provides a series of algorithms implemented

in the device hardware.

Crypto 2 hardware provides hashing - SHA-1, SHA-256, ciphering - DES, 3DES, AES

algorithms, and concurrent operations of hashing, and ciphering.

In addition to those functions provided by Crypto 2 HW, Crypto 3 HW provides

fast AES algorithms.

In addition to those functions provided by Crypto 3 HW, Crypto 3E provides

HMAC-SHA1 hashing algorithm, and Over The Air (OTA) f8/f9 algorithms as

defined by the 3GPP forum.

Software description

====================

The crypto device is defined as a platform device. The driver is

independent of the platform. The driver supports multiple instances of

crypto HW.

All the platform specific parameters are defined in the board init

file, eg. arch/arm/mach-msm/board-msm7x30.c for MSM7x30.

The qce driver provide the common services of HW crypto

access to the two drivers as listed above (qcedev, qcrypto. It sets up

the crypto HW device for the operation, then it requests ADM driver for

the DMA of the crypto operation.

Two ADM channels and two command lists (one command list for each

channel) are involved in an operation.

The setting up of the command lists and the procedure of the operation

of the crypto device are described in the following sections.

The command list for the first DMA channel is set up as follows:

  1st command of the list is for the DMA transfer from DDR memory to the

  crypto device to input data to crypto device. The dst crci of the command

  is set for crci-in for this crypto device.

  2nd command is for the DMA transfer is from crypto device to DDR memory for

  the authentication result. The src crci is set as crci-hash-done of the

  crypto device. If authentication is not required in the operation,

  the 2nd command is not used.

The command list for the second DMA channel is set up as follows:

  One command to DMA data from crypto device to DDR memory for encryption or

  decryption output from crypto device.

To accomplish ciphering and authentication concurrent operations, the driver

performs the following steps:

    (a). set up HW crypto device

    (b). hit the crypto go register.

    (c). issue the DMA command of first channel to the ADM driver,

    (d). issue the DMA command of 2nd channel to the ADM driver.

SHA1/SHA256 is an authentication/integrity hash algorithm. To accomplish

hash operation (or any authentication only algorithm), 2nd DMA channel is

not required. Only steps (a) to (c) are performed.

At the completion of the DMA operation (for (c) and (d)) ADM driver

invokes the callback registered to the DMA driver. This signifies the end of

the DMA operation(s). The driver reads the status and other information from

the CE hardware register and then invokes the callback to the qce driver client.

This signal the completion and the results of the DMA along with the status of

the CE hardware to the qce driver client. This completes a crypto operation.

In the qce driver initialization, memory for the two command lists, descriptor

lists for each crypto device are allocated out of coherent memory, using Linux

DMA API. The driver pre-configures most of the two ADM command lists

in the initialization. During each crypto operation, minimal set up is required.

src_dscr or/and dst_dscr descriptor list of the ADM command are populated

from the information obtained from the corresponding data structure. eg: for

AEAD request, the following data structure provides the information:

    struct aead_request *req

      ......

    req->assoc

    req->src

    req->dst

The DMA address of a scatter list will be retrieved and set up in the

descriptor list of an ADM command.

Power Management

================

  none

Interface:

==========

The interface is defined in qce.h

The clients qcrypto, qcedev drivers are the clients using

the interfaces.

The following services are provided by the qce driver -

     qce_open(), qce_close(), qce_ablk_cipher_req(),

     qce_hw_support(), qce_process_sha_req()

  qce_open() is the first request from the client, ex. QTI crypto

  driver (qcedev, qcrypto), to open a crypto engine. It is normally

  called at the probe function of the client for a device. During the

  probe,

  - ADM command list structure will be set up

  - Crypto device will be initialized.

  - Resource associated with the crypto engine is retrieved by doing

    platform_get_resource() or platform_get_resource_byname().

 The resources for a device are

    - crci-in, crci-out, crci-hash-done

    - two DMA channel IDs, one for encryption and decryption input, one for

      output.

    - base address of the HW crypto device.

  qce_close() is the last request from the client. Normally, it is

  called from the remove function of the client.

  qce_hw_support() allows the client to query what is supported

  by the crypto engine hardware.

  qce_ablk_cipher_req() provides ciphering service to the client.

  qce_process_sha_req() provide hashing service to the client.

  qce_aead_req() provide aead service to the client.

Module parameters:

==================

The following module parameters are defined in the board init file.

-CE hardware base register address

-Data mover channel used for transfer to/from CE hardware

These parameters differ in each platform.

Dependencies:

=============

Existing DMA driver.

The transfers are DMA'ed between the crypto hardware and DDR memory via the

data mover, ADM. The data transfers are set up to use the existing dma driver.

User space utilities:

=====================

  n/a

Known issues:

=============

  n/a

To do:

======

  n/a

Introduction:

=============

QTI Crypto (qcrypto) driver is a Linux crypto driver which interfaces

with the Linux kernel crypto API layer to provide the HW crypto functions.

This driver is accessed by kernel space apps via the kernel crypto API layer.

At present there is no means for user space apps to access this module.

Hardware description:

=====================

QTI Crypto HW device family provides a series of algorithms implemented

in the device.

Crypto 2 hardware provides hashing - SHA-1, SHA-256, ciphering - DES, 3DES, AES

algorithms, and concurrent operations of hashing, and ciphering.

In addition to those functions provided by Crypto 2 HW, Crypto 3 provides fast

AES algorithms.

In addition to those functions provided by Crypto 3 HW, Crypto 3E provides

HMAC-SHA1 hashing algorithm.

In addition to those functions provided by Crypto 3 HW, Crypto 4.0 provides

HMAC-SHA1/SHA256, AES CBC-MAC hashing algorithm and AES XTS/CCM cipher

algorithms.

Software description

====================

The module init function (_qcrypto_init()), does a platform_register(),

to register the driver. As the result, the driver probe function,

_qcrypto_probe(), will be invoked for each registered device.

In the probe function, driver opens the low level CE (qce_open), and

registers the supported algorithms to the kernel crypto API layer.

Currently, qcrypto supports the following algorithms.

      ablkcipher -

          cbc(aes),ecb(aes),ctr(aes)

      ahash -

          sha1, sha256

      aead -

          authenc(hmac(sha1),cbc(aes))

      The hmac(sha1), hmac(sha256, authenc(hmac(sha1),cbc(aes)), ccm(aes)

      and xts(aes) algorithms are registered for some platforms that

      support these in the CE hardware

The HW device  can support various algorithms. However, the most important

algorithms to gain the performance using a HW crypto accelerator are

AEAD, and ABLKCIPHER.

AEAD stands for "authentication encryption with association data".

ABLKCIPHER stands of "asynchronous block cipher".

The AEAD structure is described in the following header file aead.h

The design of the driver is to allow multiple requests

issued from kernel client SW (eg IPSec).

Therefore, the driver will have to internally queue the requests, and

serialize the requests to the low level qce driver.

When a request is received from the client, if there is no outstanding

request, a qce (or qce40) request is issued, otherwise, the request is

queued in the driver queue.

On completion of a request, the qce (or qce40) invokes the registered

callback from the qcrypto.  The internal tasklet (done_tasklet) is scheduled

in this callback function. The sole purpose of done_tasklet is

to call the completion of the current active request, and

issue more requests to the qce (or qce40), if any exists.

A spin lock is used to protect the critical section of internal queue to

be accessed from multiple tasks, SMP, and completion callback

from qce.

The driver maintains a set of statistics using debug fs. The files are

in /debug/qcrypto/stats1, /debug/qcrypto/stats2, /debug/qcrypto/stats3;

one for each instance of device. Reading the file associated with

a device will retrieve the driver statistics for that device.

Any write to the file will clear the statistics.

Test vectors for  authenc(hmac(sha1),cbc(aes)) algorithm are

developed offline, and imported to crypto/testmgr.c, and crypto/testmgr.h.

Power Management

================

  none

Interface:

==========

The kernel interface is defined in crypto.h.

Module parameters:

==================

All the platform specific parameters are defined in the board init

file, eg. arch/arm/mach-msm/board-mssm7x30.c for msm7x30.

Dependencies:

=============

qce driver.

User space utilities:

=====================

  n/a

Known issues:

=============

  n/a

To do:

======

  Add Hashing algorithms.

Limitations:

===============

(1) Each packet transfer size (for cipher and hash) is limited to maximum of

    32KB.  This is a limitation in the crypto engine hardware. Client will

    have to break packets larger than 32KB into multiple requests of smaller

    size data packets.

(2) Do not load this driver if your device has user space apps that needs to

    access the crypto hardware. Please make sure to have the qcrypto module

    disabled/unloaded.

    Not having the driver loaded, will result in the kernel space apps to use

    the registered software implementation of the crypto algorithms.

(3) If your device has Playready application enabled and uses the qcedev module

    to access the crypto hardware accelerator, please be informed that for

    performance reasons, the CE hardware will need to be dedicated to playready

    application.  Any other user space or kernel application should be implemented

    to use the software implementation of the crypto algorithms.

    (NOTE:  Please refer to details on the limitations listed in qce/40.txt)