Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6

ARM TCM (Tightly-Coupled Memory) handling in Linux

----

Written by Linus Walleij <linus.walleij@stericsson.com>

Some ARM SoC:s have a so-called TCM (Tightly-Coupled Memory).

This is usually just a few (4-64) KiB of RAM inside the ARM

processor.

Due to being embedded inside the CPU The TCM has a

Harvard-architecture, so there is an ITCM (instruction TCM)

and a DTCM (data TCM). The DTCM can not contain any

instructions, but the ITCM can actually contain data.

The size of DTCM or ITCM is minimum 4KiB so the typical

minimum configuration is 4KiB ITCM and 4KiB DTCM.

ARM CPU:s have special registers to read out status, physical

location and size of TCM memories. arch/arm/include/asm/cputype.h

defines a CPUID_TCM register that you can read out from the

system control coprocessor. Documentation from ARM can be found

at http://infocenter.arm.com, search for "TCM Status Register"

to see documents for all CPUs. Reading this register you can

determine if ITCM (bit 0) and/or DTCM (bit 16) is present in the

machine.

There is further a TCM region register (search for "TCM Region

Registers" at the ARM site) that can report and modify the location

size of TCM memories at runtime. This is used to read out and modify

TCM location and size. Notice that this is not a MMU table: you

actually move the physical location of the TCM around. At the

place you put it, it will mask any underlying RAM from the

CPU so it is usually wise not to overlap any physical RAM with

the TCM. The TCM memory exists totally outside the MMU and will

override any MMU mappings.

Code executing inside the ITCM does not "see" any MMU mappings

and e.g. register accesses must be made to physical addresses.

TCM is used for a few things:

- FIQ and other interrupt handlers that need deterministic

  timing and cannot wait for cache misses.

- Idle loops where all external RAM is set to self-refresh

  retention mode, so only on-chip RAM is accessible by

  the CPU and then we hang inside ITCM waiting for an

  interrupt.

- Other operations which implies shutting off or reconfiguring

  the external RAM controller.

There is an interface for using TCM on the ARM architecture

in <asm/tcm.h>. Using this interface it is possible to:

- Define the physical address and size of ITCM and DTCM.

- Tag functions to be compiled into ITCM.

- Tag data and constants to be allocated to DTCM and ITCM.

- Have the remaining TCM RAM added to a special

  allocation pool with gen_pool_create() and gen_pool_add()

  and provice tcm_alloc() and tcm_free() for this

  memory. Such a heap is great for things like saving

  device state when shutting off device power domains.

A machine that has TCM memory shall select HAVE_TCM in

arch/arm/Kconfig for itself, and then the

rest of the functionality will depend on the physical

location and size of ITCM and DTCM to be defined in

mach/memory.h for the machine. Code that needs to use

TCM shall #include <asm/tcm.h> If the TCM is not located

at the place given in memory.h it will be moved using

the TCM Region registers.

Functions to go into itcm can be tagged like this:

int __tcmfunc foo(int bar);

Variables to go into dtcm can be tagged like this:

int __tcmdata foo;

Constants can be tagged like this:

int __tcmconst foo;

To put assembler into TCM just use

.section ".tcm.text" or .section ".tcm.data"

respectively.

Example code:

#include <asm/tcm.h>

/* Uninitialized data */

static u32 __tcmdata tcmvar;

/* Initialized data */

static u32 __tcmdata tcmassigned = 0x2BADBABEU;

/* Constant */

static const u32 __tcmconst tcmconst = 0xCAFEBABEU;

static void __tcmlocalfunc tcm_to_tcm(void)

{

	int i;

	for (i = 0; i < 100; i++)

		tcmvar ++;

}

static void __tcmfunc hello_tcm(void)

{

	/* Some abstract code that runs in ITCM */

	int i;

	for (i = 0; i < 100; i++) {

		tcmvar ++;

	}

	tcm_to_tcm();

}

static void __init test_tcm(void)

{

	u32 *tcmem;

	int i;

	hello_tcm();

	printk("Hello TCM executed from ITCM RAM\n");

	printk("TCM variable from testrun: %u @ %p\n", tcmvar, &tcmvar);

	tcmvar = 0xDEADBEEFU;

	printk("TCM variable: 0x%x @ %p\n", tcmvar, &tcmvar);

	printk("TCM assigned variable: 0x%x @ %p\n", tcmassigned, &tcmassigned);

	printk("TCM constant: 0x%x @ %p\n", tcmconst, &tcmconst);

	/* Allocate some TCM memory from the pool */

	tcmem = tcm_alloc(20);

	if (tcmem) {

		printk("TCM Allocated 20 bytes of TCM @ %p\n", tcmem);

		tcmem[0] = 0xDEADBEEFU;

		tcmem[1] = 0x2BADBABEU;

		tcmem[2] = 0xCAFEBABEU;

		tcmem[3] = 0xDEADBEEFU;

		tcmem[4] = 0x2BADBABEU;

		for (i = 0; i < 5; i++)

			printk("TCM tcmem[%d] = %08x\n", i, tcmem[i]);

		tcm_free(tcmem, 20);

	}

}

 */

#include <stdio.h>

#include <string.h>

#define EXAMPLES	6

# echo 0 > tasks

2.3 Mounting hierarchies by name

--------------------------------

Passing the name=<x> option when mounting a cgroups hierarchy

associates the given name with the hierarchy.  This can be used when

mounting a pre-existing hierarchy, in order to refer to it by name

rather than by its set of active subsystems.  Each hierarchy is either

nameless, or has a unique name.

The name should match [\w.-]+

When passing a name=<x> option for a new hierarchy, you need to

specify subsystems manually; the legacy behaviour of mounting all

subsystems when none are explicitly specified is not supported when

you give a subsystem a name.

The name of the subsystem appears as part of the hierarchy description

in /proc/mounts and /proc/<pid>/cgroups.

3. Kernel API

=============

called multiple times against a cgroup.

int can_attach(struct cgroup_subsys *ss, struct cgroup *cgrp,

	       struct task_struct *task)

	       struct task_struct *task, bool threadgroup)

(cgroup_mutex held by caller)

Called prior to moving a task into a cgroup; if the subsystem

task is passed, then a successful result indicates that *any*

unspecified task can be moved into the cgroup. Note that this isn't

called on a fork. If this method returns 0 (success) then this should

remain valid while the caller holds cgroup_mutex.

remain valid while the caller holds cgroup_mutex. If threadgroup is

true, then a successful result indicates that all threads in the given

thread's threadgroup can be moved together.

void attach(struct cgroup_subsys *ss, struct cgroup *cgrp,

	    struct cgroup *old_cgrp, struct task_struct *task)

	    struct cgroup *old_cgrp, struct task_struct *task,

	    bool threadgroup)

(cgroup_mutex held by caller)

Called after the task has been attached to the cgroup, to allow any

post-attachment activity that requires memory allocations or blocking.

If threadgroup is true, the subsystem should take care of all threads

in the specified thread's threadgroup. Currently does not support any

subsystem that might need the old_cgrp for every thread in the group.

void fork(struct cgroup_subsy *ss, struct task_struct *task)

pages that are selected for reclaiming come from the per cgroup LRU

list.

NOTE: Reclaim does not work for the root cgroup, since we cannot set any

limits on the root cgroup.

2. Locking

The memory controller uses the following hierarchy

NOTE: We can use a suffix (k, K, m, M, g or G) to indicate values in kilo,

mega or gigabytes.

NOTE: We can write "-1" to reset the *.limit_in_bytes(unlimited).

NOTE: We cannot set limits on the root cgroup any more.

# cat /cgroups/0/memory.limit_in_bytes

4194304

NOTE2: This feature can be enabled/disabled per subtree.

7. TODO

7. Soft limits

Soft limits allow for greater sharing of memory. The idea behind soft limits

is to allow control groups to use as much of the memory as needed, provided

a. There is no memory contention

b. They do not exceed their hard limit

When the system detects memory contention or low memory control groups

are pushed back to their soft limits. If the soft limit of each control

group is very high, they are pushed back as much as possible to make

sure that one control group does not starve the others of memory.

Please note that soft limits is a best effort feature, it comes with

no guarantees, but it does its best to make sure that when memory is

heavily contended for, memory is allocated based on the soft limit

hints/setup. Currently soft limit based reclaim is setup such that

it gets invoked from balance_pgdat (kswapd).

7.1 Interface

Soft limits can be setup by using the following commands (in this example we

assume a soft limit of 256 megabytes)

# echo 256M > memory.soft_limit_in_bytes

If we want to change this to 1G, we can at any time use

# echo 1G > memory.soft_limit_in_bytes

NOTE1: Soft limits take effect over a long period of time, since they involve

       reclaiming memory for balancing between memory cgroups

NOTE2: It is recommended to set the soft limit always below the hard limit,

       otherwise the hard limit will take precedence.

8. TODO

1. Add support for accounting huge pages (as a separate controller)

2. Make per-cgroup scanner reclaim not-shared pages first

3.1 General format of the API:

struct dma_async_tx_descriptor *

async_<operation>(<op specific parameters>,

		  enum async_tx_flags flags,

        	  struct dma_async_tx_descriptor *dependency,

        	  dma_async_tx_callback callback_routine,

		  void *callback_parameter);

async_<operation>(<op specific parameters>, struct async_submit ctl *submit)

3.2 Supported operations:

memcpy  - memory copy between a source and a destination buffer

memset  - fill a destination buffer with a byte value

xor     - xor a series of source buffers and write the result to a

	  destination buffer

xor_zero_sum - xor a series of source buffers and set a flag if the

xor_val - xor a series of source buffers and set a flag if the

	  result is zero.  The implementation attempts to prevent

	  writes to memory

pq	- generate the p+q (raid6 syndrome) from a series of source buffers

pq_val  - validate that a p and or q buffer are in sync with a given series of

	  sources

datap	- (raid6_datap_recov) recover a raid6 data block and the p block

	  from the given sources

2data	- (raid6_2data_recov) recover 2 raid6 data blocks from the given

	  sources

3.3 Descriptor management:

The return value is non-NULL and points to a 'descriptor' when the operation

recycle (or free) the descriptor.  A descriptor can be acked by one of the

following methods:

1/ setting the ASYNC_TX_ACK flag if no child operations are to be submitted

2/ setting the ASYNC_TX_DEP_ACK flag to acknowledge the parent

   descriptor of a new operation.

2/ submitting an unacknowledged descriptor as a dependency to another

   async_tx call will implicitly set the acknowledged state.

3/ calling async_tx_ack() on the descriptor.

3.4 When does the operation execute?

Perform a xor->copy->xor operation where each operation depends on the

result from the previous operation:

void complete_xor_copy_xor(void *param)

void callback(void *param)

{

	printk("complete\n");

	struct completion *cmp = param;

	complete(cmp);

}

int run_xor_copy_xor(struct page **xor_srcs,

void run_xor_copy_xor(struct page **xor_srcs,

		      int xor_src_cnt,

		      struct page *xor_dest,

		      size_t xor_len,

		      size_t copy_len)

{

	struct dma_async_tx_descriptor *tx;

	addr_conv_t addr_conv[xor_src_cnt];

	struct async_submit_ctl submit;

	addr_conv_t addr_conv[NDISKS];

	struct completion cmp;

	init_async_submit(&submit, ASYNC_TX_XOR_DROP_DST, NULL, NULL, NULL,

			  addr_conv);

	tx = async_xor(xor_dest, xor_srcs, 0, xor_src_cnt, xor_len, &submit)

	tx = async_xor(xor_dest, xor_srcs, 0, xor_src_cnt, xor_len,

		       ASYNC_TX_XOR_DROP_DST, NULL, NULL, NULL);

	tx = async_memcpy(copy_dest, copy_src, 0, 0, copy_len,

			  ASYNC_TX_DEP_ACK, tx, NULL, NULL);

	tx = async_xor(xor_dest, xor_srcs, 0, xor_src_cnt, xor_len,

		       ASYNC_TX_XOR_DROP_DST | ASYNC_TX_DEP_ACK | ASYNC_TX_ACK,

		       tx, complete_xor_copy_xor, NULL);

	submit->depend_tx = tx;

	tx = async_memcpy(copy_dest, copy_src, 0, 0, copy_len, &submit);

	init_completion(&cmp);

	init_async_submit(&submit, ASYNC_TX_XOR_DROP_DST | ASYNC_TX_ACK, tx,

			  callback, &cmp, addr_conv);

	tx = async_xor(xor_dest, xor_srcs, 0, xor_src_cnt, xor_len, &submit);

	async_tx_issue_pending_all();

	wait_for_completion(&cmp);

}

See include/linux/async_tx.h for more information on the flags.  See the