Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 074be7fd authored by David S. Miller's avatar David S. Miller
Browse files

Merge branch 'nfp-implement-firmware-loading-policy' 



Simon Horman says:

====================
nfp: implement firmware loading policy

Dirk says:

This series adds configuration capabilities to the firmware loading policy of
the NFP driver.

NFP firmware loading is controlled via three HWinfo keys which can be set per
device: 'abi_drv_reset', 'abi_drv_load_ifc' and 'app_fw_from_flash'.
Refer to patch #11 for more detail on how these control the firmware loading.

In order to configure the full extend of FW loading policy, a new devlink
parameter has been introduced, 'reset_dev_on_drv_probe', which controls if the
driver should reset the device when it's probed. This, in conjunction with the
existing 'fw_load_policy' (extended to include a 'disk' option) provides the
means to tweak the NFP HWinfo keys as required by users.

Patches 1 and 2 adds the devlink modifications and patches 3 through 9 adds the
support into the NFP driver. Furthermore, the last 2 patches are documentation
only.

v2:
  Renamed all 'reset_dev_on_drv_probe' defines the same as the devlink parameter
  name (Jiri)
====================

Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parents 4bb2f84a 40a962be

Documentation/networking/device_drivers/netronome/nfp.rst

0 → 100644
+133 −0
Original line number Diff line number Diff line 
.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)



=============================================

Netronome Flow Processor (NFP) Kernel Drivers

=============================================



Copyright (c) 2019, Netronome Systems, Inc.



Contents

========



- `Overview`_

- `Acquiring Firmware`_



Overview

========



This driver supports Netronome's line of Flow Processor devices,

including the NFP4000, NFP5000, and NFP6000 models, which are also

incorporated in the company's family of Agilio SmartNICs. The SR-IOV

physical and virtual functions for these devices are supported by

the driver.



Acquiring Firmware

==================



The NFP4000 and NFP6000 devices require application specific firmware

to function.  Application firmware can be located either on the host file system

or in the device flash (if supported by management firmware).



Firmware files on the host filesystem contain card type (`AMDA-*` string), media

config etc.  They should be placed in `/lib/firmware/netronome` directory to

load firmware from the host file system.



Firmware for basic NIC operation is available in the upstream

`linux-firmware.git` repository.



Firmware in NVRAM

-----------------



Recent versions of management firmware supports loading application

firmware from flash when the host driver gets probed.  The firmware loading

policy configuration may be used to configure this feature appropriately.



Devlink or ethtool can be used to update the application firmware on the device

flash by providing the appropriate `nic_AMDA*.nffw` file to the respective

command.  Users need to take care to write the correct firmware image for the

card and media configuration to flash.



Available storage space in flash depends on the card being used.



Dealing with multiple projects

------------------------------



NFP hardware is fully programmable therefore there can be different

firmware images targeting different applications.



When using application firmware from host, we recommend placing

actual firmware files in application-named subdirectories in

`/lib/firmware/netronome` and linking the desired files, e.g.::



    $ tree /lib/firmware/netronome/

    /lib/firmware/netronome/

    ├── bpf

    │   ├── nic_AMDA0081-0001_1x40.nffw

    │   └── nic_AMDA0081-0001_4x10.nffw

    ├── flower

    │   ├── nic_AMDA0081-0001_1x40.nffw

    │   └── nic_AMDA0081-0001_4x10.nffw

    ├── nic

    │   ├── nic_AMDA0081-0001_1x40.nffw

    │   └── nic_AMDA0081-0001_4x10.nffw

    ├── nic_AMDA0081-0001_1x40.nffw -> bpf/nic_AMDA0081-0001_1x40.nffw

    └── nic_AMDA0081-0001_4x10.nffw -> bpf/nic_AMDA0081-0001_4x10.nffw



    3 directories, 8 files



You may need to use hard instead of symbolic links on distributions

which use old `mkinitrd` command instead of `dracut` (e.g. Ubuntu).



After changing firmware files you may need to regenerate the initramfs

image.  Initramfs contains drivers and firmware files your system may

need to boot.  Refer to the documentation of your distribution to find

out how to update initramfs.  Good indication of stale initramfs

is system loading wrong driver or firmware on boot, but when driver is

later reloaded manually everything works correctly.



Selecting firmware per device

-----------------------------



Most commonly all cards on the system use the same type of firmware.

If you want to load specific firmware image for a specific card, you

can use either the PCI bus address or serial number.  Driver will print

which files it's looking for when it recognizes a NFP device::



    nfp: Looking for firmware file in order of priority:

    nfp:  netronome/serial-00-12-34-aa-bb-cc-10-ff.nffw: not found

    nfp:  netronome/pci-0000:02:00.0.nffw: not found

    nfp:  netronome/nic_AMDA0081-0001_1x40.nffw: found, loading...



In this case if file (or link) called *serial-00-12-34-aa-bb-5d-10-ff.nffw*

or *pci-0000:02:00.0.nffw* is present in `/lib/firmware/netronome` this

firmware file will take precedence over `nic_AMDA*` files.



Note that `serial-*` and `pci-*` files are **not** automatically included

in initramfs, you will have to refer to documentation of appropriate tools

to find out how to include them.



Firmware loading policy

-----------------------



Firmware loading policy is controlled via three HWinfo parameters

stored as key value pairs in the device flash:



app_fw_from_flash

    Defines which firmware should take precedence, 'Disk' (0), 'Flash' (1) or

    the 'Preferred' (2) firmware. When 'Preferred' is selected, the management

    firmware makes the decision over which firmware will be loaded by comparing

    versions of the flash firmware and the host supplied firmware.

    This variable is configurable using the 'fw_load_policy'

    devlink parameter.



abi_drv_reset

    Defines if the driver should reset the firmware when

    the driver is probed, either 'Disk' (0) if firmware was found on disk,

    'Always' (1) reset or 'Never' (2) reset. Note that the device is always

    reset on driver unload if firmware was loaded when the driver was probed.

    This variable is configurable using the 'reset_dev_on_drv_probe'

    devlink parameter.



abi_drv_load_ifc

    Defines a list of PF devices allowed to load FW on the device.

    This variable is not currently user configurable.

Documentation/networking/devlink-params-nfp.txt

0 → 100644
+5 −0
Original line number Diff line number Diff line 
fw_load_policy		[DEVICE, GENERIC]

			Configuration mode: permanent



reset_dev_on_drv_probe	[DEVICE, GENERIC]

			Configuration mode: permanent

Documentation/networking/devlink-params.txt

+16 −0
Original line number Diff line number Diff line
@@ -48,4 +48,20 @@ fw_load_policy [DEVICE, GENERIC] 
			  Load firmware version preferred by the driver.

			* DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_FLASH (1)

			  Load firmware currently stored in flash.

			* DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DISK (2)

			  Load firmware currently available on host's disk.

			Type: u8



reset_dev_on_drv_probe	[DEVICE, GENERIC]

			Controls the device's reset policy on driver probe.

			Valid values:

			* DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_UNKNOWN (0)

			  Unknown or invalid value.

			* DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_ALWAYS (1)

			  Always reset device on driver probe.

			* DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_NEVER (2)

			  Never reset device on driver probe.

			* DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_DISK (3)

			  Reset only if device firmware can be found in the

			  filesystem.

			Type: u8

drivers/net/ethernet/netronome/nfp/Makefile

+1 −0
Original line number Diff line number Diff line
@@ -17,6 +17,7 @@ nfp-objs := \ 
	    nfpcore/nfp_target.o \

	    ccm.o \

	    ccm_mbox.o \

	    devlink_param.o \

	    nfp_asm.o \

	    nfp_app.o \

	    nfp_app_nic.o \

drivers/net/ethernet/netronome/nfp/devlink_param.c

0 → 100644
+254 −0
Original line number Diff line number Diff line 
// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)

/* Copyright (C) 2019 Netronome Systems, Inc. */



#include <net/devlink.h>



#include "nfpcore/nfp.h"

#include "nfpcore/nfp_nsp.h"

#include "nfp_main.h"



/**

 * struct nfp_devlink_param_u8_arg - Devlink u8 parameter get/set arguments

 * @hwinfo_name:	HWinfo key name

 * @default_hi_val:	Default HWinfo value if HWinfo doesn't exist

 * @invalid_dl_val:	Devlink value to use if HWinfo is unknown/invalid.

 *			-errno if there is no unknown/invalid value available

 * @hi_to_dl:	HWinfo to devlink value mapping

 * @dl_to_hi:	Devlink to hwinfo value mapping

 * @max_dl_val:	Maximum devlink value supported, for validation only

 * @max_hi_val:	Maximum HWinfo value supported, for validation only

 */

struct nfp_devlink_param_u8_arg {

	const char *hwinfo_name;

	const char *default_hi_val;

	int invalid_dl_val;

	u8 hi_to_dl[4];

	u8 dl_to_hi[4];

	u8 max_dl_val;

	u8 max_hi_val;

};



static const struct nfp_devlink_param_u8_arg nfp_devlink_u8_args[] = {

	[DEVLINK_PARAM_GENERIC_ID_FW_LOAD_POLICY] = {

		.hwinfo_name = "app_fw_from_flash",

		.default_hi_val = NFP_NSP_APP_FW_LOAD_DEFAULT,

		.invalid_dl_val = -EINVAL,

		.hi_to_dl = {

			[NFP_NSP_APP_FW_LOAD_DISK] =

				DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DISK,

			[NFP_NSP_APP_FW_LOAD_FLASH] =

				DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_FLASH,

			[NFP_NSP_APP_FW_LOAD_PREF] =

				DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DRIVER,

		},

		.dl_to_hi = {

			[DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DRIVER] =

				NFP_NSP_APP_FW_LOAD_PREF,

			[DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_FLASH] =

				NFP_NSP_APP_FW_LOAD_FLASH,

			[DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DISK] =

				NFP_NSP_APP_FW_LOAD_DISK,

		},

		.max_dl_val = DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DISK,

		.max_hi_val = NFP_NSP_APP_FW_LOAD_PREF,

	},

	[DEVLINK_PARAM_GENERIC_ID_RESET_DEV_ON_DRV_PROBE] = {

		.hwinfo_name = "abi_drv_reset",

		.default_hi_val = NFP_NSP_DRV_RESET_DEFAULT,

		.invalid_dl_val =

			DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_UNKNOWN,

		.hi_to_dl = {

			[NFP_NSP_DRV_RESET_ALWAYS] =

				DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_ALWAYS,

			[NFP_NSP_DRV_RESET_NEVER] =

				DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_NEVER,

			[NFP_NSP_DRV_RESET_DISK] =

				DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_DISK,

		},

		.dl_to_hi = {

			[DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_ALWAYS] =

				NFP_NSP_DRV_RESET_ALWAYS,

			[DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_NEVER] =

				NFP_NSP_DRV_RESET_NEVER,

			[DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_DISK] =

				NFP_NSP_DRV_RESET_DISK,

		},

		.max_dl_val = DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_DISK,

		.max_hi_val = NFP_NSP_DRV_RESET_NEVER,

	}

};



static int

nfp_devlink_param_u8_get(struct devlink *devlink, u32 id,

			 struct devlink_param_gset_ctx *ctx)

{

	const struct nfp_devlink_param_u8_arg *arg;

	struct nfp_pf *pf = devlink_priv(devlink);

	struct nfp_nsp *nsp;

	char hwinfo[32];

	long value;

	int err;



	if (id >= ARRAY_SIZE(nfp_devlink_u8_args))

		return -EOPNOTSUPP;



	arg = &nfp_devlink_u8_args[id];



	nsp = nfp_nsp_open(pf->cpp);

	if (IS_ERR(nsp)) {

		err = PTR_ERR(nsp);

		nfp_warn(pf->cpp, "can't access NSP: %d\n", err);

		return err;

	}



	snprintf(hwinfo, sizeof(hwinfo), arg->hwinfo_name);

	err = nfp_nsp_hwinfo_lookup_optional(nsp, hwinfo, sizeof(hwinfo),

					     arg->default_hi_val);

	if (err) {

		nfp_warn(pf->cpp, "HWinfo lookup failed: %d\n", err);

		goto exit_close_nsp;

	}



	err = kstrtol(hwinfo, 0, &value);

	if (err || value < 0 || value > arg->max_hi_val) {

		nfp_warn(pf->cpp, "HWinfo '%s' value %li invalid\n",

			 arg->hwinfo_name, value);



		if (arg->invalid_dl_val >= 0)

			ctx->val.vu8 = arg->invalid_dl_val;

		else

			err = arg->invalid_dl_val;



		goto exit_close_nsp;

	}



	ctx->val.vu8 = arg->hi_to_dl[value];



exit_close_nsp:

	nfp_nsp_close(nsp);

	return err;

}



static int

nfp_devlink_param_u8_set(struct devlink *devlink, u32 id,

			 struct devlink_param_gset_ctx *ctx)

{

	const struct nfp_devlink_param_u8_arg *arg;

	struct nfp_pf *pf = devlink_priv(devlink);

	struct nfp_nsp *nsp;

	char hwinfo[32];

	int err;



	if (id >= ARRAY_SIZE(nfp_devlink_u8_args))

		return -EOPNOTSUPP;



	arg = &nfp_devlink_u8_args[id];



	nsp = nfp_nsp_open(pf->cpp);

	if (IS_ERR(nsp)) {

		err = PTR_ERR(nsp);

		nfp_warn(pf->cpp, "can't access NSP: %d\n", err);

		return err;

	}



	/* Note the value has already been validated. */

	snprintf(hwinfo, sizeof(hwinfo), "%s=%u",

		 arg->hwinfo_name, arg->dl_to_hi[ctx->val.vu8]);

	err = nfp_nsp_hwinfo_set(nsp, hwinfo, sizeof(hwinfo));

	if (err) {

		nfp_warn(pf->cpp, "HWinfo set failed: %d\n", err);

		goto exit_close_nsp;

	}



exit_close_nsp:

	nfp_nsp_close(nsp);

	return err;

}



static int

nfp_devlink_param_u8_validate(struct devlink *devlink, u32 id,

			      union devlink_param_value val,

			      struct netlink_ext_ack *extack)

{

	const struct nfp_devlink_param_u8_arg *arg;



	if (id >= ARRAY_SIZE(nfp_devlink_u8_args))

		return -EOPNOTSUPP;



	arg = &nfp_devlink_u8_args[id];



	if (val.vu8 > arg->max_dl_val) {

		NL_SET_ERR_MSG_MOD(extack, "parameter out of range");

		return -EINVAL;

	}



	if (val.vu8 == arg->invalid_dl_val) {

		NL_SET_ERR_MSG_MOD(extack, "unknown/invalid value specified");

		return -EINVAL;

	}



	return 0;

}



static const struct devlink_param nfp_devlink_params[] = {

	DEVLINK_PARAM_GENERIC(FW_LOAD_POLICY,

			      BIT(DEVLINK_PARAM_CMODE_PERMANENT),

			      nfp_devlink_param_u8_get,

			      nfp_devlink_param_u8_set,

			      nfp_devlink_param_u8_validate),

	DEVLINK_PARAM_GENERIC(RESET_DEV_ON_DRV_PROBE,

			      BIT(DEVLINK_PARAM_CMODE_PERMANENT),

			      nfp_devlink_param_u8_get,

			      nfp_devlink_param_u8_set,

			      nfp_devlink_param_u8_validate),

};



static int nfp_devlink_supports_params(struct nfp_pf *pf)

{

	struct nfp_nsp *nsp;

	bool supported;

	int err;



	nsp = nfp_nsp_open(pf->cpp);

	if (IS_ERR(nsp)) {

		err = PTR_ERR(nsp);

		dev_err(&pf->pdev->dev, "Failed to access the NSP: %d\n", err);

		return err;

	}



	supported = nfp_nsp_has_hwinfo_lookup(nsp) &&

		    nfp_nsp_has_hwinfo_set(nsp);



	nfp_nsp_close(nsp);

	return supported;

}



int nfp_devlink_params_register(struct nfp_pf *pf)

{

	struct devlink *devlink = priv_to_devlink(pf);

	int err;



	err = nfp_devlink_supports_params(pf);

	if (err <= 0)

		return err;



	err = devlink_params_register(devlink, nfp_devlink_params,

				      ARRAY_SIZE(nfp_devlink_params));

	if (err)

		return err;



	devlink_params_publish(devlink);

	return 0;

}



void nfp_devlink_params_unregister(struct nfp_pf *pf)

{

	int err;



	err = nfp_devlink_supports_params(pf);

	if (err <= 0)

		return;



	devlink_params_unregister(priv_to_devlink(pf), nfp_devlink_params,

				  ARRAY_SIZE(nfp_devlink_params));

}