Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 7937963a authored by David S. Miller's avatar David S. Miller
Browse files

Merge branch 'mlx5-enhanced-flow-steering' 

Or Gerlitz says:

====================
net/mlx5_core: Enhance flow steering support

v0 --> v1 changes:
  - fixed improperly formatted comments.
  - compare value of ib_spec->eth.mask.ether_type in network byte order
     in ('IB/mlx5: Add flow steering utilities').

v1 --> v2 changes:
  - made sure that service functions added in the IB driver are only static-fied
    on the last commit, to make sure bisection with -Werror works fine.

v2 --> v3 changes:
   - squashed patches 11 and 12 into one patch, s.t Dave's comment
     on unused static functions gcc complaints during bisection is
     correctly addressed.

v3 has been generated against net-next commit c9c99311 "Merge tag
'batman-adv-for-davem' of git://git.open-mesh.org/linux-merge

"

The series is signed by Matan who was revently assigned to a maintainer for
the mlx5_core and IB drivers (this is a 4.5-rc1 change to the maintainers file coming
from the rdma tree) -- as such I didn't see a neeed to add my signature (Or).

This series adds three new functionalists to the driver flow-steering
infrastructure: auto-grouped flow tables, chaining of flow tables and
updates for the root flow table.

1. Auto-grouped flow tables - Flow table with auto grouping management.
When a flow table is created, hints regarding the number of rule types
and the number of rules are given in advance. Thus, a flow table is
divided into #NUM_TYPES+1 groups each contains
(#NUM_RULES)/(#NUM_TYPES+1) rules. The first #NUM_TYPES parts are groups
which are filled if the added rule matches the group specification or
the group is empty. The last part is filled by rules that can't fit
any of the former groups.

2. Chaining flow tables - Flow tables from different priorities are chained
together, if there is no match in flow table of priority i we continue
searching for a match in priority i+1. This is both true if priorities
i and i+1 belongs to the same namespace or not.

3. Updating the root flow table - the root flow table is the flow table
with the lowest level. The hardware start searching for a match in the
root flow table and continue according to the matches it find along
the way.

The first usage for the new functionality is flow steering for user-space
ConnectX-4 offloaded HW Eth RX queues done through the mlx5 IB driver.

When the mlx5 core driver is loaded, it opens three flow namespaces:
1. By-pass namespace (used by mlx5 IB driver).
2. Kernel namespace (used in order to get packets to the networking stack
through mlx5 EN driver).
3. Leftovers namespace (used by mlx5 IB and future sniffer)

The series is built as follows:

Patch #1 introduces auto-grouped flow tables support.

Patch #2 add utility functions for finding the next and the previous
flow tables in different priorities. This is used in order to chain
the flow tables in a downstream patch.

Patch #3 introduces a firmware command for updating the root flow table.

Patch #4 introduces modify flow table firmware command, this command is used
when we want to change the next flow table of an existing flow table.
This is used for chaining flow tables as well.

Patch #5 connect/disconnect flow tables. This is actually the chaining
process when we want to link flow tables. This means that if we couldn't
find a match in the first flow table, we'll continue in the chained
flow table.

Patch #6 updates priority's attributes that is required for flow table
level allocation. We update both the max_fts (the number of allowed FTs
in the sub-tree of this priority) and the start_level (which is the first
level we'll assign to the flow-tables created inside the priority).

Patch #7 adds checking of required device capabilities. Some namespaces
could be only created if the hardware supports certain attributes.
This is especially true for the Bypass and leftovers namespaces. This
adds a generic mechanism to check these required attributes.

Patch #8 creates two additional namespaces:
	a. Bypass flow rules(has nine priorities)
	b. Leftovers packets(have one priority) - for unmatched packets.

Patch #9 re-factors ipv4/ipv6 match fields in the mlx5 firmware interface
header to be more clear.

Patch #10 exports the flow steering API for mlx5_ib usage

Patch #11 implements the required support in mlx5_ib in order
to support the RDMA flow steering verbs.
====================

Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parents 23c09c26 038d2ef8

drivers/infiniband/hw/mlx5/main.c

+463 −0
Original line number Diff line number Diff line
@@ -43,6 +43,9 @@ 
#include <linux/mlx5/vport.h>

#include <rdma/ib_smi.h>

#include <rdma/ib_umem.h>

#include <linux/in.h>

#include <linux/etherdevice.h>

#include <linux/mlx5/fs.h>

#include "user.h"

#include "mlx5_ib.h"
@@ -835,6 +838,457 @@ static int mlx5_ib_dealloc_pd(struct ib_pd *pd) 
	return 0;

}



static bool outer_header_zero(u32 *match_criteria)

{

	int size = MLX5_ST_SZ_BYTES(fte_match_param);

	char *outer_headers_c = MLX5_ADDR_OF(fte_match_param, match_criteria,

					     outer_headers);



	return outer_headers_c[0] == 0 && !memcmp(outer_headers_c,

						  outer_headers_c + 1,

						  size - 1);

}



static int parse_flow_attr(u32 *match_c, u32 *match_v,

			   union ib_flow_spec *ib_spec)

{

	void *outer_headers_c = MLX5_ADDR_OF(fte_match_param, match_c,

					     outer_headers);

	void *outer_headers_v = MLX5_ADDR_OF(fte_match_param, match_v,

					     outer_headers);

	switch (ib_spec->type) {

	case IB_FLOW_SPEC_ETH:

		if (ib_spec->size != sizeof(ib_spec->eth))

			return -EINVAL;



		ether_addr_copy(MLX5_ADDR_OF(fte_match_set_lyr_2_4, outer_headers_c,

					     dmac_47_16),

				ib_spec->eth.mask.dst_mac);

		ether_addr_copy(MLX5_ADDR_OF(fte_match_set_lyr_2_4, outer_headers_v,

					     dmac_47_16),

				ib_spec->eth.val.dst_mac);



		if (ib_spec->eth.mask.vlan_tag) {

			MLX5_SET(fte_match_set_lyr_2_4, outer_headers_c,

				 vlan_tag, 1);

			MLX5_SET(fte_match_set_lyr_2_4, outer_headers_v,

				 vlan_tag, 1);



			MLX5_SET(fte_match_set_lyr_2_4, outer_headers_c,

				 first_vid, ntohs(ib_spec->eth.mask.vlan_tag));

			MLX5_SET(fte_match_set_lyr_2_4, outer_headers_v,

				 first_vid, ntohs(ib_spec->eth.val.vlan_tag));



			MLX5_SET(fte_match_set_lyr_2_4, outer_headers_c,

				 first_cfi,

				 ntohs(ib_spec->eth.mask.vlan_tag) >> 12);

			MLX5_SET(fte_match_set_lyr_2_4, outer_headers_v,

				 first_cfi,

				 ntohs(ib_spec->eth.val.vlan_tag) >> 12);



			MLX5_SET(fte_match_set_lyr_2_4, outer_headers_c,

				 first_prio,

				 ntohs(ib_spec->eth.mask.vlan_tag) >> 13);

			MLX5_SET(fte_match_set_lyr_2_4, outer_headers_v,

				 first_prio,

				 ntohs(ib_spec->eth.val.vlan_tag) >> 13);

		}

		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_c,

			 ethertype, ntohs(ib_spec->eth.mask.ether_type));

		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_v,

			 ethertype, ntohs(ib_spec->eth.val.ether_type));

		break;

	case IB_FLOW_SPEC_IPV4:

		if (ib_spec->size != sizeof(ib_spec->ipv4))

			return -EINVAL;



		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_c,

			 ethertype, 0xffff);

		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_v,

			 ethertype, ETH_P_IP);



		memcpy(MLX5_ADDR_OF(fte_match_set_lyr_2_4, outer_headers_c,

				    src_ipv4_src_ipv6.ipv4_layout.ipv4),

		       &ib_spec->ipv4.mask.src_ip,

		       sizeof(ib_spec->ipv4.mask.src_ip));

		memcpy(MLX5_ADDR_OF(fte_match_set_lyr_2_4, outer_headers_v,

				    src_ipv4_src_ipv6.ipv4_layout.ipv4),

		       &ib_spec->ipv4.val.src_ip,

		       sizeof(ib_spec->ipv4.val.src_ip));

		memcpy(MLX5_ADDR_OF(fte_match_set_lyr_2_4, outer_headers_c,

				    dst_ipv4_dst_ipv6.ipv4_layout.ipv4),

		       &ib_spec->ipv4.mask.dst_ip,

		       sizeof(ib_spec->ipv4.mask.dst_ip));

		memcpy(MLX5_ADDR_OF(fte_match_set_lyr_2_4, outer_headers_v,

				    dst_ipv4_dst_ipv6.ipv4_layout.ipv4),

		       &ib_spec->ipv4.val.dst_ip,

		       sizeof(ib_spec->ipv4.val.dst_ip));

		break;

	case IB_FLOW_SPEC_TCP:

		if (ib_spec->size != sizeof(ib_spec->tcp_udp))

			return -EINVAL;



		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_c, ip_protocol,

			 0xff);

		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_v, ip_protocol,

			 IPPROTO_TCP);



		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_c, tcp_sport,

			 ntohs(ib_spec->tcp_udp.mask.src_port));

		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_v, tcp_sport,

			 ntohs(ib_spec->tcp_udp.val.src_port));



		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_c, tcp_dport,

			 ntohs(ib_spec->tcp_udp.mask.dst_port));

		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_v, tcp_dport,

			 ntohs(ib_spec->tcp_udp.val.dst_port));

		break;

	case IB_FLOW_SPEC_UDP:

		if (ib_spec->size != sizeof(ib_spec->tcp_udp))

			return -EINVAL;



		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_c, ip_protocol,

			 0xff);

		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_v, ip_protocol,

			 IPPROTO_UDP);



		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_c, udp_sport,

			 ntohs(ib_spec->tcp_udp.mask.src_port));

		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_v, udp_sport,

			 ntohs(ib_spec->tcp_udp.val.src_port));



		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_c, udp_dport,

			 ntohs(ib_spec->tcp_udp.mask.dst_port));

		MLX5_SET(fte_match_set_lyr_2_4, outer_headers_v, udp_dport,

			 ntohs(ib_spec->tcp_udp.val.dst_port));

		break;

	default:

		return -EINVAL;

	}



	return 0;

}



/* If a flow could catch both multicast and unicast packets,

 * it won't fall into the multicast flow steering table and this rule

 * could steal other multicast packets.

 */

static bool flow_is_multicast_only(struct ib_flow_attr *ib_attr)

{

	struct ib_flow_spec_eth *eth_spec;



	if (ib_attr->type != IB_FLOW_ATTR_NORMAL ||

	    ib_attr->size < sizeof(struct ib_flow_attr) +

	    sizeof(struct ib_flow_spec_eth) ||

	    ib_attr->num_of_specs < 1)

		return false;



	eth_spec = (struct ib_flow_spec_eth *)(ib_attr + 1);

	if (eth_spec->type != IB_FLOW_SPEC_ETH ||

	    eth_spec->size != sizeof(*eth_spec))

		return false;



	return is_multicast_ether_addr(eth_spec->mask.dst_mac) &&

	       is_multicast_ether_addr(eth_spec->val.dst_mac);

}



static bool is_valid_attr(struct ib_flow_attr *flow_attr)

{

	union ib_flow_spec *ib_spec = (union ib_flow_spec *)(flow_attr + 1);

	bool has_ipv4_spec = false;

	bool eth_type_ipv4 = true;

	unsigned int spec_index;



	/* Validate that ethertype is correct */

	for (spec_index = 0; spec_index < flow_attr->num_of_specs; spec_index++) {

		if (ib_spec->type == IB_FLOW_SPEC_ETH &&

		    ib_spec->eth.mask.ether_type) {

			if (!((ib_spec->eth.mask.ether_type == htons(0xffff)) &&

			      ib_spec->eth.val.ether_type == htons(ETH_P_IP)))

				eth_type_ipv4 = false;

		} else if (ib_spec->type == IB_FLOW_SPEC_IPV4) {

			has_ipv4_spec = true;

		}

		ib_spec = (void *)ib_spec + ib_spec->size;

	}

	return !has_ipv4_spec || eth_type_ipv4;

}



static void put_flow_table(struct mlx5_ib_dev *dev,

			   struct mlx5_ib_flow_prio *prio, bool ft_added)

{

	prio->refcount -= !!ft_added;

	if (!prio->refcount) {

		mlx5_destroy_flow_table(prio->flow_table);

		prio->flow_table = NULL;

	}

}



static int mlx5_ib_destroy_flow(struct ib_flow *flow_id)

{

	struct mlx5_ib_dev *dev = to_mdev(flow_id->qp->device);

	struct mlx5_ib_flow_handler *handler = container_of(flow_id,

							  struct mlx5_ib_flow_handler,

							  ibflow);

	struct mlx5_ib_flow_handler *iter, *tmp;



	mutex_lock(&dev->flow_db.lock);



	list_for_each_entry_safe(iter, tmp, &handler->list, list) {

		mlx5_del_flow_rule(iter->rule);

		list_del(&iter->list);

		kfree(iter);

	}



	mlx5_del_flow_rule(handler->rule);

	put_flow_table(dev, &dev->flow_db.prios[handler->prio], true);

	mutex_unlock(&dev->flow_db.lock);



	kfree(handler);



	return 0;

}



#define MLX5_FS_MAX_TYPES	 10

#define MLX5_FS_MAX_ENTRIES	 32000UL

static struct mlx5_ib_flow_prio *get_flow_table(struct mlx5_ib_dev *dev,

						struct ib_flow_attr *flow_attr)

{

	struct mlx5_flow_namespace *ns = NULL;

	struct mlx5_ib_flow_prio *prio;

	struct mlx5_flow_table *ft;

	int num_entries;

	int num_groups;

	int priority;

	int err = 0;



	if (flow_attr->type == IB_FLOW_ATTR_NORMAL) {

		if (flow_is_multicast_only(flow_attr))

			priority = MLX5_IB_FLOW_MCAST_PRIO;

		else

			priority = flow_attr->priority;

		ns = mlx5_get_flow_namespace(dev->mdev,

					     MLX5_FLOW_NAMESPACE_BYPASS);

		num_entries = MLX5_FS_MAX_ENTRIES;

		num_groups = MLX5_FS_MAX_TYPES;

		prio = &dev->flow_db.prios[priority];

	} else if (flow_attr->type == IB_FLOW_ATTR_ALL_DEFAULT ||

		   flow_attr->type == IB_FLOW_ATTR_MC_DEFAULT) {

		ns = mlx5_get_flow_namespace(dev->mdev,

					     MLX5_FLOW_NAMESPACE_LEFTOVERS);

		build_leftovers_ft_param(&priority,

					 &num_entries,

					 &num_groups);

		prio = &dev->flow_db.prios[MLX5_IB_FLOW_LEFTOVERS_PRIO];

	}



	if (!ns)

		return ERR_PTR(-ENOTSUPP);



	ft = prio->flow_table;

	if (!ft) {

		ft = mlx5_create_auto_grouped_flow_table(ns, priority,

							 num_entries,

							 num_groups);



		if (!IS_ERR(ft)) {

			prio->refcount = 0;

			prio->flow_table = ft;

		} else {

			err = PTR_ERR(ft);

		}

	}



	return err ? ERR_PTR(err) : prio;

}



static struct mlx5_ib_flow_handler *create_flow_rule(struct mlx5_ib_dev *dev,

						     struct mlx5_ib_flow_prio *ft_prio,

						     struct ib_flow_attr *flow_attr,

						     struct mlx5_flow_destination *dst)

{

	struct mlx5_flow_table	*ft = ft_prio->flow_table;

	struct mlx5_ib_flow_handler *handler;

	void *ib_flow = flow_attr + 1;

	u8 match_criteria_enable = 0;

	unsigned int spec_index;

	u32 *match_c;

	u32 *match_v;

	int err = 0;



	if (!is_valid_attr(flow_attr))

		return ERR_PTR(-EINVAL);



	match_c = kzalloc(MLX5_ST_SZ_BYTES(fte_match_param), GFP_KERNEL);

	match_v = kzalloc(MLX5_ST_SZ_BYTES(fte_match_param), GFP_KERNEL);

	handler = kzalloc(sizeof(*handler), GFP_KERNEL);

	if (!handler || !match_c || !match_v) {

		err = -ENOMEM;

		goto free;

	}



	INIT_LIST_HEAD(&handler->list);



	for (spec_index = 0; spec_index < flow_attr->num_of_specs; spec_index++) {

		err = parse_flow_attr(match_c, match_v, ib_flow);

		if (err < 0)

			goto free;



		ib_flow += ((union ib_flow_spec *)ib_flow)->size;

	}



	/* Outer header support only */

	match_criteria_enable = (!outer_header_zero(match_c)) << 0;

	handler->rule = mlx5_add_flow_rule(ft, match_criteria_enable,

					   match_c, match_v,

					   MLX5_FLOW_CONTEXT_ACTION_FWD_DEST,

					   MLX5_FS_DEFAULT_FLOW_TAG,

					   dst);



	if (IS_ERR(handler->rule)) {

		err = PTR_ERR(handler->rule);

		goto free;

	}



	handler->prio = ft_prio - dev->flow_db.prios;



	ft_prio->flow_table = ft;

free:

	if (err)

		kfree(handler);

	kfree(match_c);

	kfree(match_v);

	return err ? ERR_PTR(err) : handler;

}



enum {

	LEFTOVERS_MC,

	LEFTOVERS_UC,

};



static struct mlx5_ib_flow_handler *create_leftovers_rule(struct mlx5_ib_dev *dev,

							  struct mlx5_ib_flow_prio *ft_prio,

							  struct ib_flow_attr *flow_attr,

							  struct mlx5_flow_destination *dst)

{

	struct mlx5_ib_flow_handler *handler_ucast = NULL;

	struct mlx5_ib_flow_handler *handler = NULL;



	static struct {

		struct ib_flow_attr	flow_attr;

		struct ib_flow_spec_eth eth_flow;

	} leftovers_specs[] = {

		[LEFTOVERS_MC] = {

			.flow_attr = {

				.num_of_specs = 1,

				.size = sizeof(leftovers_specs[0])

			},

			.eth_flow = {

				.type = IB_FLOW_SPEC_ETH,

				.size = sizeof(struct ib_flow_spec_eth),

				.mask = {.dst_mac = {0x1} },

				.val =  {.dst_mac = {0x1} }

			}

		},

		[LEFTOVERS_UC] = {

			.flow_attr = {

				.num_of_specs = 1,

				.size = sizeof(leftovers_specs[0])

			},

			.eth_flow = {

				.type = IB_FLOW_SPEC_ETH,

				.size = sizeof(struct ib_flow_spec_eth),

				.mask = {.dst_mac = {0x1} },

				.val = {.dst_mac = {} }

			}

		}

	};



	handler = create_flow_rule(dev, ft_prio,

				   &leftovers_specs[LEFTOVERS_MC].flow_attr,

				   dst);

	if (!IS_ERR(handler) &&

	    flow_attr->type == IB_FLOW_ATTR_ALL_DEFAULT) {

		handler_ucast = create_flow_rule(dev, ft_prio,

						 &leftovers_specs[LEFTOVERS_UC].flow_attr,

						 dst);

		if (IS_ERR(handler_ucast)) {

			kfree(handler);

			handler = handler_ucast;

		} else {

			list_add(&handler_ucast->list, &handler->list);

		}

	}



	return handler;

}



static struct ib_flow *mlx5_ib_create_flow(struct ib_qp *qp,

					   struct ib_flow_attr *flow_attr,

					   int domain)

{

	struct mlx5_ib_dev *dev = to_mdev(qp->device);

	struct mlx5_ib_flow_handler *handler = NULL;

	struct mlx5_flow_destination *dst = NULL;

	struct mlx5_ib_flow_prio *ft_prio;

	int err;



	if (flow_attr->priority > MLX5_IB_FLOW_LAST_PRIO)

		return ERR_PTR(-ENOSPC);



	if (domain != IB_FLOW_DOMAIN_USER ||

	    flow_attr->port > MLX5_CAP_GEN(dev->mdev, num_ports) ||

	    flow_attr->flags)

		return ERR_PTR(-EINVAL);



	dst = kzalloc(sizeof(*dst), GFP_KERNEL);

	if (!dst)

		return ERR_PTR(-ENOMEM);



	mutex_lock(&dev->flow_db.lock);



	ft_prio = get_flow_table(dev, flow_attr);

	if (IS_ERR(ft_prio)) {

		err = PTR_ERR(ft_prio);

		goto unlock;

	}



	dst->type = MLX5_FLOW_DESTINATION_TYPE_TIR;

	dst->tir_num = to_mqp(qp)->raw_packet_qp.rq.tirn;



	if (flow_attr->type == IB_FLOW_ATTR_NORMAL) {

		handler = create_flow_rule(dev, ft_prio, flow_attr,

					   dst);

	} else if (flow_attr->type == IB_FLOW_ATTR_ALL_DEFAULT ||

		   flow_attr->type == IB_FLOW_ATTR_MC_DEFAULT) {

		handler = create_leftovers_rule(dev, ft_prio, flow_attr,

						dst);

	} else {

		err = -EINVAL;

		goto destroy_ft;

	}



	if (IS_ERR(handler)) {

		err = PTR_ERR(handler);

		handler = NULL;

		goto destroy_ft;

	}



	ft_prio->refcount++;

	mutex_unlock(&dev->flow_db.lock);

	kfree(dst);



	return &handler->ibflow;



destroy_ft:

	put_flow_table(dev, ft_prio, false);

unlock:

	mutex_unlock(&dev->flow_db.lock);

	kfree(dst);

	kfree(handler);

	return ERR_PTR(err);

}



static int mlx5_ib_mcg_attach(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)

{

	struct mlx5_ib_dev *dev = to_mdev(ibqp->device);
@@ -1439,10 +1893,19 @@ static void *mlx5_ib_add(struct mlx5_core_dev *mdev) 
			(1ull << IB_USER_VERBS_CMD_CLOSE_XRCD);

	}



	if (mlx5_ib_port_link_layer(&dev->ib_dev) ==

	    IB_LINK_LAYER_ETHERNET) {

		dev->ib_dev.create_flow	= mlx5_ib_create_flow;

		dev->ib_dev.destroy_flow = mlx5_ib_destroy_flow;

		dev->ib_dev.uverbs_ex_cmd_mask |=

			(1ull << IB_USER_VERBS_EX_CMD_CREATE_FLOW) |

			(1ull << IB_USER_VERBS_EX_CMD_DESTROY_FLOW);

	}

	err = init_node_data(dev);

	if (err)

		goto err_dealloc;



	mutex_init(&dev->flow_db.lock);

	mutex_init(&dev->cap_mask_mutex);



	err = create_dev_resources(&dev->devr);

drivers/infiniband/hw/mlx5/mlx5_ib.h

+44 −1
Original line number Diff line number Diff line
@@ -105,6 +105,36 @@ struct mlx5_ib_pd { 
	u32			pdn;

};



#define MLX5_IB_FLOW_MCAST_PRIO		(MLX5_BY_PASS_NUM_PRIOS - 1)

#define MLX5_IB_FLOW_LAST_PRIO		(MLX5_IB_FLOW_MCAST_PRIO - 1)

#if (MLX5_IB_FLOW_LAST_PRIO <= 0)

#error "Invalid number of bypass priorities"

#endif

#define MLX5_IB_FLOW_LEFTOVERS_PRIO	(MLX5_IB_FLOW_MCAST_PRIO + 1)



#define MLX5_IB_NUM_FLOW_FT		(MLX5_IB_FLOW_LEFTOVERS_PRIO + 1)

struct mlx5_ib_flow_prio {

	struct mlx5_flow_table		*flow_table;

	unsigned int			refcount;

};



struct mlx5_ib_flow_handler {

	struct list_head		list;

	struct ib_flow			ibflow;

	unsigned int			prio;

	struct mlx5_flow_rule	*rule;

};



struct mlx5_ib_flow_db {

	struct mlx5_ib_flow_prio	prios[MLX5_IB_NUM_FLOW_FT];

	/* Protect flow steering bypass flow tables

	 * when add/del flow rules.

	 * only single add/removal of flow steering rule could be done

	 * simultaneously.

	 */

	struct mutex			lock;

};



/* Use macros here so that don't have to duplicate

 * enum ib_send_flags and enum ib_qp_type for low-level driver

 */
@@ -171,9 +201,21 @@ struct mlx5_ib_pfault { 
	struct mlx5_pagefault	mpfault;

};



struct mlx5_ib_rq {

	u32			tirn;

};



struct mlx5_ib_raw_packet_qp {

	struct mlx5_ib_rq rq;

};



struct mlx5_ib_qp {

	struct ib_qp		ibqp;

	union {

		struct mlx5_core_qp		mqp;

		struct mlx5_ib_raw_packet_qp	raw_packet_qp;

	};



	struct mlx5_buf		buf;



	struct mlx5_db		db;
@@ -431,6 +473,7 @@ struct mlx5_ib_dev { 
	 */

	struct srcu_struct      mr_srcu;

#endif

	struct mlx5_ib_flow_db	flow_db;

};



static inline struct mlx5_ib_cq *to_mibcq(struct mlx5_core_cq *mcq)

drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c

+51 −1
Original line number Diff line number Diff line
@@ -38,9 +38,28 @@ 
#include "fs_cmd.h"

#include "mlx5_core.h"



int mlx5_cmd_update_root_ft(struct mlx5_core_dev *dev,

			    struct mlx5_flow_table *ft)

{

	u32 in[MLX5_ST_SZ_DW(set_flow_table_root_in)];

	u32 out[MLX5_ST_SZ_DW(set_flow_table_root_out)];



	memset(in, 0, sizeof(in));



	MLX5_SET(set_flow_table_root_in, in, opcode,

		 MLX5_CMD_OP_SET_FLOW_TABLE_ROOT);

	MLX5_SET(set_flow_table_root_in, in, table_type, ft->type);

	MLX5_SET(set_flow_table_root_in, in, table_id, ft->id);



	memset(out, 0, sizeof(out));

	return mlx5_cmd_exec_check_status(dev, in, sizeof(in), out,

					  sizeof(out));

}



int mlx5_cmd_create_flow_table(struct mlx5_core_dev *dev,

			       enum fs_flow_table_type type, unsigned int level,

			       unsigned int log_size, unsigned int *table_id)

			       unsigned int log_size, struct mlx5_flow_table

			       *next_ft, unsigned int *table_id)

{

	u32 out[MLX5_ST_SZ_DW(create_flow_table_out)];

	u32 in[MLX5_ST_SZ_DW(create_flow_table_in)];
@@ -51,6 +70,10 @@ int mlx5_cmd_create_flow_table(struct mlx5_core_dev *dev, 
	MLX5_SET(create_flow_table_in, in, opcode,

		 MLX5_CMD_OP_CREATE_FLOW_TABLE);



	if (next_ft) {

		MLX5_SET(create_flow_table_in, in, table_miss_mode, 1);

		MLX5_SET(create_flow_table_in, in, table_miss_id, next_ft->id);

	}

	MLX5_SET(create_flow_table_in, in, table_type, type);

	MLX5_SET(create_flow_table_in, in, level, level);

	MLX5_SET(create_flow_table_in, in, log_size, log_size);
@@ -83,6 +106,33 @@ int mlx5_cmd_destroy_flow_table(struct mlx5_core_dev *dev, 
					  sizeof(out));

}



int mlx5_cmd_modify_flow_table(struct mlx5_core_dev *dev,

			       struct mlx5_flow_table *ft,

			       struct mlx5_flow_table *next_ft)

{

	u32 in[MLX5_ST_SZ_DW(modify_flow_table_in)];

	u32 out[MLX5_ST_SZ_DW(modify_flow_table_out)];



	memset(in, 0, sizeof(in));

	memset(out, 0, sizeof(out));



	MLX5_SET(modify_flow_table_in, in, opcode,

		 MLX5_CMD_OP_MODIFY_FLOW_TABLE);

	MLX5_SET(modify_flow_table_in, in, table_type, ft->type);

	MLX5_SET(modify_flow_table_in, in, table_id, ft->id);

	MLX5_SET(modify_flow_table_in, in, modify_field_select,

		 MLX5_MODIFY_FLOW_TABLE_MISS_TABLE_ID);

	if (next_ft) {

		MLX5_SET(modify_flow_table_in, in, table_miss_mode, 1);

		MLX5_SET(modify_flow_table_in, in, table_miss_id, next_ft->id);

	} else {

		MLX5_SET(modify_flow_table_in, in, table_miss_mode, 0);

	}



	return mlx5_cmd_exec_check_status(dev, in, sizeof(in), out,

					  sizeof(out));

}



int mlx5_cmd_create_flow_group(struct mlx5_core_dev *dev,

			       struct mlx5_flow_table *ft,

			       u32 *in,

drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.h

+8 −1
Original line number Diff line number Diff line
@@ -35,11 +35,16 @@ 


int mlx5_cmd_create_flow_table(struct mlx5_core_dev *dev,

			       enum fs_flow_table_type type, unsigned int level,

			       unsigned int log_size, unsigned int *table_id);

			       unsigned int log_size, struct mlx5_flow_table

			       *next_ft, unsigned int *table_id);



int mlx5_cmd_destroy_flow_table(struct mlx5_core_dev *dev,

				struct mlx5_flow_table *ft);



int mlx5_cmd_modify_flow_table(struct mlx5_core_dev *dev,

			       struct mlx5_flow_table *ft,

			       struct mlx5_flow_table *next_ft);



int mlx5_cmd_create_flow_group(struct mlx5_core_dev *dev,

			       struct mlx5_flow_table *ft,

			       u32 *in, unsigned int *group_id);
@@ -62,4 +67,6 @@ int mlx5_cmd_delete_fte(struct mlx5_core_dev *dev, 
			struct mlx5_flow_table *ft,

			unsigned int index);



int mlx5_cmd_update_root_ft(struct mlx5_core_dev *dev,

			    struct mlx5_flow_table *ft);

#endif

drivers/net/ethernet/mellanox/mlx5/core/fs_core.c

+534 −67

File changed.

Preview size limit exceeded, changes collapsed.