Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 7f2dc5c4 authored by Linus Torvalds's avatar Linus Torvalds
Browse files

Merge tag 'dm-3.13-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm 

Pull device mapper changes from Mike Snitzer:
 "A set of device-mapper changes for 3.13.

  Improve reliability of buffer allocations for dm messages with a small
  number of arguments, a couple path group initialization fixes for dm
  multipath, a fix for resizing a dm array, various fixes and
  optimizations for dm cache, a fix for device mapper's Kconfig menu
  indentation.

  Features added include:
   - dm crypt support for activating legacy CBC TrueCrypt containers
     (useful for forensics of these old TCRYPT containers)
   - reduced dm-cache memory requirements for each block in the cache
   - basic support for shrinking a dm-cache's cache (fast) device
   - most notably, dm-cache support for managing cache coherency when
     deploying dm-cache with sophisticated origin volumes (that support
     hardware snapshots and/or clustering): these changes come in the
     form of a new passthrough operation mode and a cache block
     invalidation interface"

* tag 'dm-3.13-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm: (32 commits)
  dm cache: resolve small nits and improve Documentation
  dm cache: add cache block invalidation support
  dm cache: add remove_cblock method to policy interface
  dm cache policy mq: reduce memory requirements
  dm cache metadata: check the metadata version when reading the superblock
  dm cache: add passthrough mode
  dm cache: cache shrinking support
  dm cache: promotion optimisation for writes
  dm cache: be much more aggressive about promoting writes to discarded blocks
  dm cache policy mq: implement writeback_work() and mq_{set,clear}_dirty()
  dm cache: optimize commit_if_needed
  dm space map disk: optimise sm_disk_dec_block
  MAINTAINERS: add reference to device-mapper's linux-dm.git tree
  dm: fix Kconfig menu indentation
  dm: allow remove to be deferred
  dm table: print error on preresume failure
  dm crypt: add TCW IV mode for old CBC TCRYPT containers
  dm crypt: properly handle extra key string in initialization
  dm cache: log error message if dm_kcopyd_copy() fails
  dm cache: use cell_defer() boolean argument consistently
  ...
parents 82cb6ace 7b6b2bc9

Documentation/device-mapper/cache-policies.txt

+4 −2
Original line number Diff line number Diff line
@@ -30,8 +30,10 @@ multiqueue 


This policy is the default.



The multiqueue policy has two sets of 16 queues: one set for entries

waiting for the cache and another one for those in the cache.

The multiqueue policy has three sets of 16 queues: one set for entries

waiting for the cache and another two for those in the cache (a set for

clean entries and a set for dirty entries).



Cache entries in the queues are aged based on logical time. Entry into

the cache is based on variable thresholds and queue selection is based

on hit count on entry. The policy aims to take different cache miss

Documentation/device-mapper/cache.txt

+51 −6
Original line number Diff line number Diff line
@@ -68,10 +68,11 @@ So large block sizes are bad because they waste cache space. And small 
block sizes are bad because they increase the amount of metadata (both

in core and on disk).



Writeback/writethrough

----------------------

Cache operating modes

---------------------



The cache has two modes, writeback and writethrough.

The cache has three operating modes: writeback, writethrough and

passthrough.



If writeback, the default, is selected then a write to a block that is

cached will go only to the cache and the block will be marked dirty in
@@ -81,8 +82,31 @@ If writethrough is selected then a write to a cached block will not 
complete until it has hit both the origin and cache devices.  Clean

blocks should remain clean.



If passthrough is selected, useful when the cache contents are not known

to be coherent with the origin device, then all reads are served from

the origin device (all reads miss the cache) and all writes are

forwarded to the origin device; additionally, write hits cause cache

block invalidates.  To enable passthrough mode the cache must be clean.

Passthrough mode allows a cache device to be activated without having to

worry about coherency.  Coherency that exists is maintained, although

the cache will gradually cool as writes take place.  If the coherency of

the cache can later be verified, or established through use of the

"invalidate_cblocks" message, the cache device can be transitioned to

writethrough or writeback mode while still warm.  Otherwise, the cache

contents can be discarded prior to transitioning to the desired

operating mode.



A simple cleaner policy is provided, which will clean (write back) all

dirty blocks in a cache.  Useful for decommissioning a cache.

dirty blocks in a cache.  Useful for decommissioning a cache or when

shrinking a cache.  Shrinking the cache's fast device requires all cache

blocks, in the area of the cache being removed, to be clean.  If the

area being removed from the cache still contains dirty blocks the resize

will fail.  Care must be taken to never reduce the volume used for the

cache's fast device until the cache is clean.  This is of particular

importance if writeback mode is used.  Writethrough and passthrough

modes already maintain a clean cache.  Future support to partially clean

the cache, above a specified threshold, will allow for keeping the cache

warm and in writeback mode during resize.



Migration throttling

--------------------
@@ -161,7 +185,7 @@ Constructor 
 block size      : cache unit size in sectors



 #feature args   : number of feature arguments passed

 feature args    : writethrough.  (The default is writeback.)

 feature args    : writethrough or passthrough (The default is writeback.)



 policy          : the replacement policy to use

 #policy args    : an even number of arguments corresponding to
@@ -177,6 +201,13 @@ Optional feature arguments are: 
		   back cache block contents later for performance reasons,

		   so they may differ from the corresponding origin blocks.



   passthrough	 : a degraded mode useful for various cache coherency

		   situations (e.g., rolling back snapshots of

		   underlying storage).	 Reads and writes always go to

		   the origin.	If a write goes to a cached origin

		   block, then the cache block is invalidated.

		   To enable passthrough mode the cache must be clean.



A policy called 'default' is always registered.  This is an alias for

the policy we currently think is giving best all round performance.
@@ -231,12 +262,26 @@ The message format is: 
E.g.

   dmsetup message my_cache 0 sequential_threshold 1024





Invalidation is removing an entry from the cache without writing it

back.  Cache blocks can be invalidated via the invalidate_cblocks

message, which takes an arbitrary number of cblock ranges.  Each cblock

must be expressed as a decimal value, in the future a variant message

that takes cblock ranges expressed in hexidecimal may be needed to

better support efficient invalidation of larger caches.  The cache must

be in passthrough mode when invalidate_cblocks is used.



   invalidate_cblocks [<cblock>|<cblock begin>-<cblock end>]*



E.g.

   dmsetup message my_cache 0 invalidate_cblocks 2345 3456-4567 5678-6789



Examples

========



The test suite can be found here:



https://github.com/jthornber/thinp-test-suite

https://github.com/jthornber/device-mapper-test-suite



dmsetup create my_cache --table '0 41943040 cache /dev/mapper/metadata \

	/dev/mapper/ssd /dev/mapper/origin 512 1 writeback default 0'

Documentation/device-mapper/dm-crypt.txt

+9 −2
Original line number Diff line number Diff line
@@ -4,12 +4,15 @@ dm-crypt 
Device-Mapper's "crypt" target provides transparent encryption of block devices

using the kernel crypto API.



For a more detailed description of supported parameters see:

http://code.google.com/p/cryptsetup/wiki/DMCrypt



Parameters: <cipher> <key> <iv_offset> <device path> \

	      <offset> [<#opt_params> <opt_params>]



<cipher>

    Encryption cipher and an optional IV generation mode.

    (In format cipher[:keycount]-chainmode-ivopts:ivmode).

    (In format cipher[:keycount]-chainmode-ivmode[:ivopts]).

    Examples:

       des

       aes-cbc-essiv:sha256
@@ -19,7 +22,11 @@ Parameters: <cipher> <key> <iv_offset> <device path> \ 


<key>

    Key used for encryption. It is encoded as a hexadecimal number.

    You can only use key sizes that are valid for the selected cipher.

    You can only use key sizes that are valid for the selected cipher

    in combination with the selected iv mode.

    Note that for some iv modes the key string can contain additional

    keys (for example IV seed) so the key contains more parts concatenated

    into a single string.



<keycount>

    Multi-key compatibility mode. You can define <keycount> keys and

MAINTAINERS

+1 −0
Original line number Diff line number Diff line
@@ -2647,6 +2647,7 @@ M: dm-devel@redhat.com 
L:	dm-devel@redhat.com

W:	http://sources.redhat.com/dm

Q:	http://patchwork.kernel.org/project/dm-devel/list/

T:	git git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git

T:	quilt http://people.redhat.com/agk/patches/linux/editing/

S:	Maintained

F:	Documentation/device-mapper/

drivers/md/Kconfig

+11 −11
Original line number Diff line number Diff line
@@ -297,6 +297,17 @@ config DM_MIRROR 
         Allow volume managers to mirror logical volumes, also

         needed for live data migration tools such as 'pvmove'.



config DM_LOG_USERSPACE

	tristate "Mirror userspace logging"

	depends on DM_MIRROR && NET

	select CONNECTOR

	---help---

	  The userspace logging module provides a mechanism for

	  relaying the dm-dirty-log API to userspace.  Log designs

	  which are more suited to userspace implementation (e.g.

	  shared storage logs) or experimental logs can be implemented

	  by leveraging this framework.



config DM_RAID

       tristate "RAID 1/4/5/6/10 target"

       depends on BLK_DEV_DM
@@ -323,17 +334,6 @@ config DM_RAID 
	 RAID-5, RAID-6 distributes the syndromes across the drives

	 in one of the available parity distribution methods.



config DM_LOG_USERSPACE

	tristate "Mirror userspace logging"

	depends on DM_MIRROR && NET

	select CONNECTOR

	---help---

	  The userspace logging module provides a mechanism for

	  relaying the dm-dirty-log API to userspace.  Log designs

	  which are more suited to userspace implementation (e.g.

	  shared storage logs) or experimental logs can be implemented

	  by leveraging this framework.



config DM_ZERO

	tristate "Zero target"

	depends on BLK_DEV_DM