Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 4ca9bc22 authored by Mauro Carvalho Chehab's avatar Mauro Carvalho Chehab Committed by Jonathan Corbet
Browse files

docs: target: convert docs to ReST and rename to *.rst



Convert the TCM docs to ReST format and add them to the
bookset.

This has a mix of userspace-faced and Kernelspace faced
docs. Still, it sounds a better candidate to be added at
the kernel API set of docs.

The conversion is actually:
  - add blank lines and identation in order to identify paragraphs;
  - fix tables markups;
  - add some lists markups;
  - mark literal blocks;
  - adjust title markups.

At its new index.rst, let's add a :orphan: while this is not linked to
the main index.rst file, in order to avoid build warnings.

Signed-off-by: default avatarMauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: default avatarJonathan Corbet <corbet@lwn.net>
parent bdf3a950
Loading
Loading
Loading
Loading
+19 −0
Original line number Diff line number Diff line
:orphan:

==================
TCM Virtual Device
==================

.. toctree::
    :maxdepth: 1

    tcmu-design
    tcm_mod_builder
    scripts

.. only::  subproject and html

   Indices
   =======

   * :ref:`genindex`
+11 −0
Original line number Diff line number Diff line
TCM mod builder script
----------------------

.. literalinclude:: tcm_mod_builder.py
    :language: perl

Target export device script
---------------------------

.. literalinclude:: target-export-device
    :language: shell
+149 −0
Original line number Diff line number Diff line
=========================================
The TCM v4 fabric module script generator
=========================================

Greetings all,

This document is intended to be a mini-HOWTO for using the tcm_mod_builder.py
script to generate a brand new functional TCM v4 fabric .ko module of your very own,
that once built can be immediately be loaded to start access the new TCM/ConfigFS
fabric skeleton, by simply using::

	modprobe $TCM_NEW_MOD
	mkdir -p /sys/kernel/config/target/$TCM_NEW_MOD

This script will create a new drivers/target/$TCM_NEW_MOD/, and will do the following

	1) Generate new API callers for drivers/target/target_core_fabric_configs.c logic
	   ->make_tpg(), ->drop_tpg(), ->make_wwn(), ->drop_wwn().  These are created
	   into $TCM_NEW_MOD/$TCM_NEW_MOD_configfs.c
	2) Generate basic infrastructure for loading/unloading LKMs and TCM/ConfigFS fabric module
	   using a skeleton struct target_core_fabric_ops API template.
	3) Based on user defined T10 Proto_Ident for the new fabric module being built,
	   the TransportID / Initiator and Target WWPN related handlers for
	   SPC-3 persistent reservation are automatically generated in $TCM_NEW_MOD/$TCM_NEW_MOD_fabric.c
	   using drivers/target/target_core_fabric_lib.c logic.
	4) NOP API calls for all other Data I/O path and fabric dependent attribute logic
	   in $TCM_NEW_MOD/$TCM_NEW_MOD_fabric.c

tcm_mod_builder.py depends upon the mandatory '-p $PROTO_IDENT' and '-m
$FABRIC_MOD_name' parameters, and actually running the script looks like::

  target:/mnt/sdb/lio-core-2.6.git/Documentation/target# python tcm_mod_builder.py -p iSCSI -m tcm_nab5000
  tcm_dir: /mnt/sdb/lio-core-2.6.git/Documentation/target/../../
  Set fabric_mod_name: tcm_nab5000
  Set fabric_mod_dir:
  /mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000
  Using proto_ident: iSCSI
  Creating fabric_mod_dir:
  /mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000
  Writing file:
  /mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000/tcm_nab5000_base.h
  Using tcm_mod_scan_fabric_ops:
  /mnt/sdb/lio-core-2.6.git/Documentation/target/../../include/target/target_core_fabric_ops.h
  Writing file:
  /mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000/tcm_nab5000_fabric.c
  Writing file:
  /mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000/tcm_nab5000_fabric.h
  Writing file:
  /mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000/tcm_nab5000_configfs.c
  Writing file:
  /mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000/Kbuild
  Writing file:
  /mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000/Kconfig
  Would you like to add tcm_nab5000to drivers/target/Kbuild..? [yes,no]: yes
  Would you like to add tcm_nab5000to drivers/target/Kconfig..? [yes,no]: yes

At the end of tcm_mod_builder.py. the script will ask to add the following
line to drivers/target/Kbuild::

	obj-$(CONFIG_TCM_NAB5000)       += tcm_nab5000/

and the same for drivers/target/Kconfig::

	source "drivers/target/tcm_nab5000/Kconfig"

#) Run 'make menuconfig' and select the new CONFIG_TCM_NAB5000 item::

	<M>   TCM_NAB5000 fabric module

#) Build using 'make modules', once completed you will have::

    target:/mnt/sdb/lio-core-2.6.git# ls -la drivers/target/tcm_nab5000/
    total 1348
    drwxr-xr-x 2 root root   4096 2010-10-05 03:23 .
    drwxr-xr-x 9 root root   4096 2010-10-05 03:22 ..
    -rw-r--r-- 1 root root    282 2010-10-05 03:22 Kbuild
    -rw-r--r-- 1 root root    171 2010-10-05 03:22 Kconfig
    -rw-r--r-- 1 root root     49 2010-10-05 03:23 modules.order
    -rw-r--r-- 1 root root    738 2010-10-05 03:22 tcm_nab5000_base.h
    -rw-r--r-- 1 root root   9096 2010-10-05 03:22 tcm_nab5000_configfs.c
    -rw-r--r-- 1 root root 191200 2010-10-05 03:23 tcm_nab5000_configfs.o
    -rw-r--r-- 1 root root  40504 2010-10-05 03:23 .tcm_nab5000_configfs.o.cmd
    -rw-r--r-- 1 root root   5414 2010-10-05 03:22 tcm_nab5000_fabric.c
    -rw-r--r-- 1 root root   2016 2010-10-05 03:22 tcm_nab5000_fabric.h
    -rw-r--r-- 1 root root 190932 2010-10-05 03:23 tcm_nab5000_fabric.o
    -rw-r--r-- 1 root root  40713 2010-10-05 03:23 .tcm_nab5000_fabric.o.cmd
    -rw-r--r-- 1 root root 401861 2010-10-05 03:23 tcm_nab5000.ko
    -rw-r--r-- 1 root root    265 2010-10-05 03:23 .tcm_nab5000.ko.cmd
    -rw-r--r-- 1 root root    459 2010-10-05 03:23 tcm_nab5000.mod.c
    -rw-r--r-- 1 root root  23896 2010-10-05 03:23 tcm_nab5000.mod.o
    -rw-r--r-- 1 root root  22655 2010-10-05 03:23 .tcm_nab5000.mod.o.cmd
    -rw-r--r-- 1 root root 379022 2010-10-05 03:23 tcm_nab5000.o
    -rw-r--r-- 1 root root    211 2010-10-05 03:23 .tcm_nab5000.o.cmd

#) Load the new module, create a lun_0 configfs group, and add new TCM Core
   IBLOCK backstore symlink to port::

    target:/mnt/sdb/lio-core-2.6.git# insmod drivers/target/tcm_nab5000.ko
    target:/mnt/sdb/lio-core-2.6.git# mkdir -p /sys/kernel/config/target/nab5000/iqn.foo/tpgt_1/lun/lun_0
    target:/mnt/sdb/lio-core-2.6.git# cd /sys/kernel/config/target/nab5000/iqn.foo/tpgt_1/lun/lun_0/
    target:/sys/kernel/config/target/nab5000/iqn.foo/tpgt_1/lun/lun_0# ln -s /sys/kernel/config/target/core/iblock_0/lvm_test0 nab5000_port

    target:/sys/kernel/config/target/nab5000/iqn.foo/tpgt_1/lun/lun_0# cd -
    target:/mnt/sdb/lio-core-2.6.git# tree /sys/kernel/config/target/nab5000/
    /sys/kernel/config/target/nab5000/
    |-- discovery_auth
    |-- iqn.foo
    |   `-- tpgt_1
    |       |-- acls
    |       |-- attrib
    |       |-- lun
    |       |   `-- lun_0
    |       |       |-- alua_tg_pt_gp
    |       |       |-- alua_tg_pt_offline
    |       |       |-- alua_tg_pt_status
    |       |       |-- alua_tg_pt_write_md
    |	|	`-- nab5000_port -> ../../../../../../target/core/iblock_0/lvm_test0
    |       |-- np
    |       `-- param
    `-- version

    target:/mnt/sdb/lio-core-2.6.git# lsmod
    Module                  Size  Used by
    tcm_nab5000             3935  4
    iscsi_target_mod      193211  0
    target_core_stgt        8090  0
    target_core_pscsi      11122  1
    target_core_file        9172  2
    target_core_iblock      9280  1
    target_core_mod       228575  31
    tcm_nab5000,iscsi_target_mod,target_core_stgt,target_core_pscsi,target_core_file,target_core_iblock
    libfc                  73681  0
    scsi_debug             56265  0
    scsi_tgt                8666  1 target_core_stgt
    configfs               20644  2 target_core_mod

----------------------------------------------------------------------

Future TODO items
=================

	1) Add more T10 proto_idents
	2) Make tcm_mod_dump_fabric_ops() smarter and generate function pointer
	   defs directly from include/target/target_core_fabric_ops.h:struct target_core_fabric_ops
	   structure members.

October 5th, 2010

Nicholas A. Bellinger <nab@linux-iscsi.org>
+0 −145
Original line number Diff line number Diff line
>>>>>>>>>> The TCM v4 fabric module script generator <<<<<<<<<<

Greetings all,

This document is intended to be a mini-HOWTO for using the tcm_mod_builder.py
script to generate a brand new functional TCM v4 fabric .ko module of your very own,
that once built can be immediately be loaded to start access the new TCM/ConfigFS
fabric skeleton, by simply using:

	modprobe $TCM_NEW_MOD
	mkdir -p /sys/kernel/config/target/$TCM_NEW_MOD

This script will create a new drivers/target/$TCM_NEW_MOD/, and will do the following

	*) Generate new API callers for drivers/target/target_core_fabric_configs.c logic
	   ->make_tpg(), ->drop_tpg(), ->make_wwn(), ->drop_wwn().  These are created
	   into $TCM_NEW_MOD/$TCM_NEW_MOD_configfs.c
	*) Generate basic infrastructure for loading/unloading LKMs and TCM/ConfigFS fabric module
	   using a skeleton struct target_core_fabric_ops API template.
	*) Based on user defined T10 Proto_Ident for the new fabric module being built,
	   the TransportID / Initiator and Target WWPN related handlers for
	   SPC-3 persistent reservation are automatically generated in $TCM_NEW_MOD/$TCM_NEW_MOD_fabric.c
	   using drivers/target/target_core_fabric_lib.c logic.
	*) NOP API calls for all other Data I/O path and fabric dependent attribute logic
	   in $TCM_NEW_MOD/$TCM_NEW_MOD_fabric.c

tcm_mod_builder.py depends upon the mandatory '-p $PROTO_IDENT' and '-m
$FABRIC_MOD_name' parameters, and actually running the script looks like:

target:/mnt/sdb/lio-core-2.6.git/Documentation/target# python tcm_mod_builder.py -p iSCSI -m tcm_nab5000
tcm_dir: /mnt/sdb/lio-core-2.6.git/Documentation/target/../../
Set fabric_mod_name: tcm_nab5000
Set fabric_mod_dir:
/mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000
Using proto_ident: iSCSI
Creating fabric_mod_dir:
/mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000
Writing file:
/mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000/tcm_nab5000_base.h
Using tcm_mod_scan_fabric_ops:
/mnt/sdb/lio-core-2.6.git/Documentation/target/../../include/target/target_core_fabric_ops.h
Writing file:
/mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000/tcm_nab5000_fabric.c
Writing file:
/mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000/tcm_nab5000_fabric.h
Writing file:
/mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000/tcm_nab5000_configfs.c
Writing file:
/mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000/Kbuild
Writing file:
/mnt/sdb/lio-core-2.6.git/Documentation/target/../../drivers/target/tcm_nab5000/Kconfig
Would you like to add tcm_nab5000to drivers/target/Kbuild..? [yes,no]: yes
Would you like to add tcm_nab5000to drivers/target/Kconfig..? [yes,no]: yes

At the end of tcm_mod_builder.py. the script will ask to add the following
line to drivers/target/Kbuild:

	obj-$(CONFIG_TCM_NAB5000)       += tcm_nab5000/

and the same for drivers/target/Kconfig:

	source "drivers/target/tcm_nab5000/Kconfig"

*) Run 'make menuconfig' and select the new CONFIG_TCM_NAB5000 item:

	<M>   TCM_NAB5000 fabric module

*) Build using 'make modules', once completed you will have:

target:/mnt/sdb/lio-core-2.6.git# ls -la drivers/target/tcm_nab5000/
total 1348
drwxr-xr-x 2 root root   4096 2010-10-05 03:23 .
drwxr-xr-x 9 root root   4096 2010-10-05 03:22 ..
-rw-r--r-- 1 root root    282 2010-10-05 03:22 Kbuild
-rw-r--r-- 1 root root    171 2010-10-05 03:22 Kconfig
-rw-r--r-- 1 root root     49 2010-10-05 03:23 modules.order
-rw-r--r-- 1 root root    738 2010-10-05 03:22 tcm_nab5000_base.h
-rw-r--r-- 1 root root   9096 2010-10-05 03:22 tcm_nab5000_configfs.c
-rw-r--r-- 1 root root 191200 2010-10-05 03:23 tcm_nab5000_configfs.o
-rw-r--r-- 1 root root  40504 2010-10-05 03:23 .tcm_nab5000_configfs.o.cmd
-rw-r--r-- 1 root root   5414 2010-10-05 03:22 tcm_nab5000_fabric.c
-rw-r--r-- 1 root root   2016 2010-10-05 03:22 tcm_nab5000_fabric.h
-rw-r--r-- 1 root root 190932 2010-10-05 03:23 tcm_nab5000_fabric.o
-rw-r--r-- 1 root root  40713 2010-10-05 03:23 .tcm_nab5000_fabric.o.cmd
-rw-r--r-- 1 root root 401861 2010-10-05 03:23 tcm_nab5000.ko
-rw-r--r-- 1 root root    265 2010-10-05 03:23 .tcm_nab5000.ko.cmd
-rw-r--r-- 1 root root    459 2010-10-05 03:23 tcm_nab5000.mod.c
-rw-r--r-- 1 root root  23896 2010-10-05 03:23 tcm_nab5000.mod.o
-rw-r--r-- 1 root root  22655 2010-10-05 03:23 .tcm_nab5000.mod.o.cmd
-rw-r--r-- 1 root root 379022 2010-10-05 03:23 tcm_nab5000.o
-rw-r--r-- 1 root root    211 2010-10-05 03:23 .tcm_nab5000.o.cmd

*) Load the new module, create a lun_0 configfs group, and add new TCM Core
   IBLOCK backstore symlink to port:

target:/mnt/sdb/lio-core-2.6.git# insmod drivers/target/tcm_nab5000.ko
target:/mnt/sdb/lio-core-2.6.git# mkdir -p /sys/kernel/config/target/nab5000/iqn.foo/tpgt_1/lun/lun_0
target:/mnt/sdb/lio-core-2.6.git# cd /sys/kernel/config/target/nab5000/iqn.foo/tpgt_1/lun/lun_0/
target:/sys/kernel/config/target/nab5000/iqn.foo/tpgt_1/lun/lun_0# ln -s /sys/kernel/config/target/core/iblock_0/lvm_test0 nab5000_port

target:/sys/kernel/config/target/nab5000/iqn.foo/tpgt_1/lun/lun_0# cd -
target:/mnt/sdb/lio-core-2.6.git# tree /sys/kernel/config/target/nab5000/
/sys/kernel/config/target/nab5000/
|-- discovery_auth
|-- iqn.foo
|   `-- tpgt_1
|       |-- acls
|       |-- attrib
|       |-- lun
|       |   `-- lun_0
|       |       |-- alua_tg_pt_gp
|       |       |-- alua_tg_pt_offline
|       |       |-- alua_tg_pt_status
|       |       |-- alua_tg_pt_write_md
|	|	`-- nab5000_port -> ../../../../../../target/core/iblock_0/lvm_test0
|       |-- np
|       `-- param
`-- version

target:/mnt/sdb/lio-core-2.6.git# lsmod
Module                  Size  Used by
tcm_nab5000             3935  4
iscsi_target_mod      193211  0
target_core_stgt        8090  0
target_core_pscsi      11122  1
target_core_file        9172  2
target_core_iblock      9280  1
target_core_mod       228575  31
tcm_nab5000,iscsi_target_mod,target_core_stgt,target_core_pscsi,target_core_file,target_core_iblock
libfc                  73681  0
scsi_debug             56265  0
scsi_tgt                8666  1 target_core_stgt
configfs               20644  2 target_core_mod

----------------------------------------------------------------------

Future TODO items:

	*) Add more T10 proto_idents
	*) Make tcm_mod_dump_fabric_ops() smarter and generate function pointer
	   defs directly from include/target/target_core_fabric_ops.h:struct target_core_fabric_ops
	   structure members.

October 5th, 2010
Nicholas A. Bellinger <nab@linux-iscsi.org>
+148 −124
Original line number Diff line number Diff line
Contents:
====================
TCM Userspace Design
====================


.. Contents:

   1) TCM Userspace Design
     a) Background
@@ -19,7 +24,7 @@ Contents:


TCM Userspace Design
--------------------
====================

TCM is another name for LIO, an in-kernel iSCSI target (server).
Existing TCM targets run in the kernel.  TCMU (TCM in Userspace)
@@ -32,7 +37,8 @@ modules for file, block device, RAM or using another SCSI device as
storage.  These are called "backstores" or "storage engines".  These
built-in modules are implemented entirely as kernel code.

Background:
Background
----------

In addition to modularizing the transport protocol used for carrying
SCSI commands ("fabrics"), the Linux kernel target, LIO, also modularizes
@@ -60,7 +66,8 @@ kernel, another approach is to create a userspace pass-through
backstore for LIO, "TCMU".


Benefits:
Benefits
--------

In addition to allowing relatively easy support for RBD and GLFS, TCMU
will also allow easier development of new backstores. TCMU combines
@@ -72,21 +79,25 @@ The disadvantage is there are more distinct components to configure, and
potentially to malfunction. This is unavoidable, but hopefully not
fatal if we're careful to keep things as simple as possible.

Design constraints:
Design constraints
------------------

- Good performance: high throughput, low latency
- Cleanly handle if userspace:

   1) never attaches
   2) hangs
   3) dies
   4) misbehaves

- Allow future flexibility in user & kernel implementations
- Be reasonably memory-efficient
- Simple to configure & run
- Simple to write a userspace backend


Implementation overview:
Implementation overview
-----------------------

The core of the TCMU interface is a memory region that is shared
between kernel and userspace. Within this region is: a control area
@@ -108,7 +119,8 @@ the region mapped at a different virtual address.

See target_core_user.h for the struct definitions.

The Mailbox:
The Mailbox
-----------

The mailbox is always at the start of the shared memory region, and
contains a version, details about the starting offset and size of the
@@ -117,19 +129,27 @@ userspace (respectively) to put commands on the ring, and indicate
when the commands are completed.

version - 1 (userspace should abort if otherwise)

flags:
- TCMU_MAILBOX_FLAG_CAP_OOOC: indicates out-of-order completion is
  supported.  See "The Command Ring" for details.
cmdr_off - The offset of the start of the command ring from the start
    - TCMU_MAILBOX_FLAG_CAP_OOOC:
	indicates out-of-order completion is supported.
	See "The Command Ring" for details.

cmdr_off
	The offset of the start of the command ring from the start
	of the memory region, to account for the mailbox size.
cmdr_size - The size of the command ring. This does *not* need to be a
cmdr_size
	The size of the command ring. This does *not* need to be a
	power of two.
cmd_head - Modified by the kernel to indicate when a command has been
cmd_head
	Modified by the kernel to indicate when a command has been
	placed on the ring.
cmd_tail - Modified by userspace to indicate when it has completed
cmd_tail
	Modified by userspace to indicate when it has completed
	processing of a command.

The Command Ring:
The Command Ring
----------------

Commands are placed on the ring by the kernel incrementing
mailbox.cmd_head by the size of the command, modulo cmdr_size, and
@@ -180,27 +200,29 @@ opcode it does not handle, it must set UNKNOWN_OP bit (bit 0) in
hdr.uflags, update cmd_tail, and proceed with processing additional
commands, if any.

The Data Area:
The Data Area
-------------

This is shared-memory space after the command ring. The organization
of this area is not defined in the TCMU interface, and userspace
should access only the parts referenced by pending iovs.


Device Discovery:
Device Discovery
----------------

Other devices may be using UIO besides TCMU. Unrelated user processes
may also be handling different sets of TCMU devices. TCMU userspace
processes must find their devices by scanning sysfs
class/uio/uio*/name. For TCMU devices, these names will be of the
format:
format::

	tcm-user/<hba_num>/<device_name>/<subtype>/<path>

where "tcm-user" is common for all TCMU-backed UIO devices. <hba_num>
and <device_name> allow userspace to find the device's path in the
kernel target's configfs tree. Assuming the usual mount point, it is
found at:
found at::

	/sys/kernel/config/target/core/user_<hba_num>/<device_name>

@@ -214,7 +236,7 @@ configure the device, if needed. The name cannot contain ':', due to
LIO limitations.

For all devices so discovered, the user handler opens /dev/uioX and
calls mmap():
calls mmap()::

	mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0)

@@ -222,7 +244,8 @@ where size must be equal to the value read from
/sys/class/uio/uioX/maps/map0/size.


Device Events:
Device Events
-------------

If a new device is added or removed, a notification will be broadcast
over netlink, using a generic netlink family name of "TCM-USER" and a
@@ -233,7 +256,8 @@ the LIO device, so that after determining the device is supported
(based on subtype) it can take the appropriate action.


Other contingencies:
Other contingencies
-------------------

Userspace handler process never attaches:

@@ -258,7 +282,7 @@ Userspace handler process is malicious:


Writing a user pass-through handler (with example code)
-------------------------------------------------------
=======================================================

A user process handing a TCMU device must support the following:

@@ -277,9 +301,9 @@ TCMU is designed so that multiple unrelated processes can manage TCMU
devices separately. All handlers should make sure to only open their
devices, based opon a known subtype string.

a) Discovering and configuring TCMU UIO devices:
a) Discovering and configuring TCMU UIO devices::

(error checking omitted for brevity)
      /* error checking omitted for brevity */

      int fd, dev_fd;
      char buf[256];
@@ -319,7 +343,7 @@ while (1) {
      }


c) Managing the command ring
c) Managing the command ring::

      #include <linux/target_core_user.h>

@@ -373,7 +397,7 @@ int handle_device_events(int fd, void *map)


A final note
------------
============

Please be careful to return codes as defined by the SCSI
specifications. These are different than some values defined in the
Loading