Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 2cb79266 authored by Matthew R. Ochs's avatar Matthew R. Ochs Committed by James Bottomley
Browse files

cxlflash: Virtual LUN support



Add support for physical LUN segmentation (virtual LUNs) to device
driver supporting the IBM CXL Flash adapter. This patch allows user
space applications to virtually segment a physical LUN into N virtual
LUNs, taking advantage of the translation features provided by this
adapter.

Signed-off-by: default avatarMatthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: default avatarManoj N. Kumar <manoj@linux.vnet.ibm.com>
Reviewed-by: default avatarMichael Neuling <mikey@neuling.org>
Reviewed-by: default avatarWen Xiong <wenxiong@linux.vnet.ibm.com>
Signed-off-by: default avatarJames Bottomley <JBottomley@Odin.com>
parent 65be2c79
Loading
Loading
Loading
Loading
+62 −1
Original line number Diff line number Diff line
@@ -163,7 +163,8 @@ DK_CXLFLASH_ATTACH

        - These tokens are only valid for the process under which they
          were created. The child of a forked process cannot continue
          to use the context id or file descriptor created by its parent.
          to use the context id or file descriptor created by its parent
          (see DK_CXLFLASH_VLUN_CLONE for further details).

        - These tokens are only valid for the lifetime of the context and
          the process under which they were created. Once either is
@@ -193,6 +194,45 @@ DK_CXLFLASH_USER_DIRECT
    treated as a resource handle that is returned to the user. The user
    is then able to use the handle to reference the LUN during I/O.

DK_CXLFLASH_USER_VIRTUAL
------------------------
    This ioctl is responsible for transitioning the LUN to virtual mode
    of access and configuring the AFU for virtual access from user space
    on a per-context basis. Additionally, the block size and last logical
    block address (LBA) are returned to the user.

    As mentioned previously, when operating in user space access mode,
    LUNs may be accessed in whole or in part. Only one mode is allowed
    at a time and if one mode is active (outstanding references exist),
    requests to use the LUN in a different mode are denied.

    The AFU is configured for virtual access from user space by adding
    an entry to the AFU's resource handle table. The index of the entry
    is treated as a resource handle that is returned to the user. The
    user is then able to use the handle to reference the LUN during I/O.

    By default, the virtual LUN is created with a size of 0. The user
    would need to use the DK_CXLFLASH_VLUN_RESIZE ioctl to adjust the grow
    the virtual LUN to a desired size. To avoid having to perform this
    resize for the initial creation of the virtual LUN, the user has the
    option of specifying a size as part of the DK_CXLFLASH_USER_VIRTUAL
    ioctl, such that when success is returned to the user, the
    resource handle that is provided is already referencing provisioned
    storage. This is reflected by the last LBA being a non-zero value.

DK_CXLFLASH_VLUN_RESIZE
-----------------------
    This ioctl is responsible for resizing a previously created virtual
    LUN and will fail if invoked upon a LUN that is not in virtual
    mode. Upon success, an updated last LBA is returned to the user
    indicating the new size of the virtual LUN associated with the
    resource handle.

    The partitioning of virtual LUNs is jointly mediated by the cxlflash
    driver and the AFU. An allocation table is kept for each LUN that is
    operating in the virtual mode and used to program a LUN translation
    table that the AFU references when provided with a resource handle.

DK_CXLFLASH_RELEASE
-------------------
    This ioctl is responsible for releasing a previously obtained
@@ -214,6 +254,27 @@ DK_CXLFLASH_DETACH
    success, all "tokens" which had been provided to the user from the
    DK_CXLFLASH_ATTACH onward are no longer valid.

DK_CXLFLASH_VLUN_CLONE
----------------------
    This ioctl is responsible for cloning a previously created
    context to a more recently created context. It exists solely to
    support maintaining user space access to storage after a process
    forks. Upon success, the child process (which invoked the ioctl)
    will have access to the same LUNs via the same resource handle(s)
    and fd2 as the parent, but under a different context.

    Context sharing across processes is not supported with CXL and
    therefore each fork must be met with establishing a new context
    for the child process. This ioctl simplifies the state management
    and playback required by a user in such a scenario. When a process
    forks, child process can clone the parents context by first creating
    a context (via DK_CXLFLASH_ATTACH) and then using this ioctl to
    perform the clone from the parent to the child.

    The clone itself is fairly simple. The resource handle and lun
    translation tables are copied from the parent context to the child's
    and then synced with the AFU.

DK_CXLFLASH_VERIFY
------------------
    This ioctl is used to detect various changes such as the capacity of
+1 −1
Original line number Diff line number Diff line
obj-$(CONFIG_CXLFLASH) += cxlflash.o
cxlflash-y += main.o superpipe.o lunmgt.o
cxlflash-y += main.o superpipe.o lunmgt.o vlun.o
+4 −0
Original line number Diff line number Diff line
@@ -116,6 +116,9 @@ struct cxlflash_cfg {

	atomic_t num_user_contexts;

	/* Parameters that are LUN table related */
	int last_lun_index[CXLFLASH_NUM_FC_PORTS];
	int promote_lun_index;
	struct list_head lluns; /* list of llun_info structs */

	wait_queue_head_t tmf_waitq;
@@ -200,5 +203,6 @@ int cxlflash_ioctl(struct scsi_device *, int, void __user *);
void cxlflash_stop_term_user_contexts(struct cxlflash_cfg *);
int cxlflash_mark_contexts_error(struct cxlflash_cfg *);
void cxlflash_term_local_luns(struct cxlflash_cfg *);
void cxlflash_restore_luntable(struct cxlflash_cfg *);

#endif /* ifndef _CXLFLASH_COMMON_H */
+3 −0
Original line number Diff line number Diff line
@@ -20,6 +20,7 @@

#include "sislite.h"
#include "common.h"
#include "vlun.h"
#include "superpipe.h"

/**
@@ -42,6 +43,7 @@ static struct llun_info *create_local(struct scsi_device *sdev, u8 *wwid)
	lli->sdev = sdev;
	lli->newly_created = true;
	lli->host_no = sdev->host->host_no;
	lli->in_table = false;

	memcpy(lli->wwid, wwid, DK_CXLFLASH_MANAGE_LUN_WWID_LEN);
out:
@@ -208,6 +210,7 @@ void cxlflash_term_global_luns(void)
	mutex_lock(&global.mutex);
	list_for_each_entry_safe(gli, temp, &global.gluns, list) {
		list_del(&gli->list);
		cxlflash_ba_terminate(&gli->blka.ba_lun);
		kfree(gli);
	}
	mutex_unlock(&global.mutex);
+13 −0
Original line number Diff line number Diff line
@@ -1989,6 +1989,8 @@ static int init_afu(struct cxlflash_cfg *cfg)
	afu_err_intr_init(cfg->afu);
	atomic64_set(&afu->room, readq_be(&afu->host_map->cmd_room));

	/* Restore the LUN mappings */
	cxlflash_restore_luntable(cfg);
err1:
	pr_debug("%s: returning rc=%d\n", __func__, rc);
	return rc;
@@ -2286,6 +2288,17 @@ static int cxlflash_probe(struct pci_dev *pdev,

	cfg->init_state = INIT_STATE_NONE;
	cfg->dev = pdev;

	/*
	 * The promoted LUNs move to the top of the LUN table. The rest stay
	 * on the bottom half. The bottom half grows from the end
	 * (index = 255), whereas the top half grows from the beginning
	 * (index = 0).
	 */
	cfg->promote_lun_index  = 0;
	cfg->last_lun_index[0] = CXLFLASH_NUM_VLUNS/2 - 1;
	cfg->last_lun_index[1] = CXLFLASH_NUM_VLUNS/2 - 1;

	cfg->dev_id = (struct pci_device_id *)dev_id;
	cfg->mcctx = NULL;

Loading