net: ipc_router: Add snapshot of IPC Router

Introduction

============

Inter Process Communication(IPC) Message Router

IPC Router provides a connectionless message routing service between

multiple processes in a MSM setup. The communicating processes can

run either in the same processor or in a different processor within

the MSM setup. The IPC Router has been designed to

	1) Route messages of any types

	2) Support a broader network of processors

The IPC Router follows the same protocol as the existing RPC Router

in the kernel while communicating with its peer IPC Routers.

Hardware description

====================

The IPC Router doesn't implement any specific hardware driver.

The IPC Router uses the existing hardware drivers to transport messages

across to the peer router. IPC Router contains a XPRT interface layer to

handle the different types of transports/links. This interface layer

abstracts the underlying transport complexities from the router and

provides a packet/message interface with the transports.

Software description

====================

The IPC Router is designed to support a client-server model. The

clients and servers communicate with one another by exchanging data

units known as messages. A message is a byte string from 1 to 64K bytes

long. The IPC Router provides a connectionless message routing service

between the clients and servers i.e. any client/server can communicate

with any other client/server in the network of processors.

Network Topology Overview:

--------------------------

The system is organized as a network of nodes. Each processor in the

network is the most fundamental element called node. The complete

network is hierarchically structured i.e. the network is divided into

tiers and each tier is fully-meshed. The following figure shows an

example network topology.

	---N1---	---N4---

	|      |	|      |

	|      |	|      |

        N2----N3-------N5-----N6

	       |	|

	       |	|

	       ---N7----

	       |	|

	       |	|

	       N8------N9

Each node in the complete network is identified using a unique node id

(Nx in the example network). In the example network, nodes N1, N2 & N3

are fully-meshed to form a tier 1 network. Similarly nodes N4, N5 & N6

form another tier 1 network and nodes N7, N8 & N9 form third tier 1

network. These three tier 1 networks are fully-meshed to form a tier 2

network.

Each transport/link in the network is identified using a unique name/id

called XPRT id. This XPRT id is used by the nodes to identify the

link to be used while sending message to a specific destination.

In addition, each transport/link in the network is assigned a link id.

This link id is used to identify the tier to which the link belongs to.

This link marking is used to avoid the routing loops while forwarding

the broadcast messages. The incoming messages are only forwarded onto an

egress link which has a link id different from that of an ingress link.

IPC Router Addressing Overview:

-------------------------------

Each client/server in the network is identified using a unique

<Node_id:Port_id> combination. Node_id identifies the processor on which

a client/server is running. Port_id is a unique id within a node. This

Port_id is assigned by the IPC Router in that node when a client/server

comes up. The Node_id & Port_id are 32 bits each.

Port_id 0xFFFFFFFE is reserved for Router &

0xFFFFFFFF is reserved for broadcast messages.

Each server in the network can also be addressed using a service name.

The service name is of the form <service(32 bits):instance(32 bits)>.

When a server comes up, it binds itself with a service name. This name

information along with the <Node_id:Port_id> is broadcast onto the

entire network.

Control Path:

-------------

IPC Router uses the control messages to communicate and propagate the

system wide events to the other routers in the system. Some of the

events include:

	1) Node Status

	2) Server Status

	3) Client Status

	4) Flow Control Request/Confirmation

Message Header:

---------------

IPC Router prepends a header to every message that it communicates with

other IPC Router. The receiving IPC Routers use the header to identify

the source and destination of the message, size of the message, type

of the message and handle any flow control requests. The IPC Router

header format is as follows:

	0                                              31

	-------------------------------------------------

	|                     Version                   |

	-------------------------------------------------

	|                  Message Type                 |

	-------------------------------------------------

	|                 Source Node ID                |

	-------------------------------------------------

	|                 Source Port ID                |

	-------------------------------------------------

	|                   Confirm RX                  |

	-------------------------------------------------

	|                 Payload Length                |

	-------------------------------------------------

	|               Destination Node ID             |

	-------------------------------------------------

	|               Destination Port ID             |

	-------------------------------------------------

Message Header v2(Optimized):

-----------------------------

The following optimization has been done to the IPC Router header to

make it concise, align with the system requirement and enable future

expansion:

	0               8               16              24             31

	-----------------------------------------------------------------

	|    Version    | Message Type  |         Control Flag          |

	-----------------------------------------------------------------

	|                         Payload Length                        |

	-----------------------------------------------------------------

	|        Source Node ID         |        Source Port ID         |

	-----------------------------------------------------------------

	|     Destination Node ID       |     Destination Port ID       |

	-----------------------------------------------------------------

Control Flag:

	0                                       14          15

	------------------------------------------------------------------

	|               Reserved                | Opt. Hdr. | Confirm RX |

	------------------------------------------------------------------

IPC Router identifies and processes the header depending on the version

field. The Confirm RX field in message header v1 becomes part of the

control flag. All the other fields are reduced in size to align with the

system requirement.

Optional Header:

An optional header bit field is introduced in the control flag to handle

any unforeseen future requirement that this header cannot handle. When

that bit is set, an optional header follows the current header. The

optional header format is as follows:

	0               8               16                             31

	-----------------------------------------------------------------

	| Length(words) |      Type     |           Control Flag        |

	-----------------------------------------------------------------

	|                                                               |

	|                     Optional Header Contents                  |

	|                                                               |

	-----------------------------------------------------------------

Design

======

The IPC Router is organized into 2 layers:

	1) Router Core layer

	2) Router - XPRT Interface layer

This organization allows the router to abstract the XPRT's complexities

from that of the core router functionalities. The Router Core layer

performs the following core functions:

	1) Message Routing

	2) Distributed name service

	3) Flow control between ports

The Router core layer contains the following important data structures

to perform the core functions in their respective order:

	1) Routing Table

	2) Table of Active Servers

	3) Table of Active Remote ports

All these data structures get updated based on the events passed through

the control path.

The Router - XPRT Interface layer hides the underlying transport

complexities and provides and abstracted packet interface to the

Router Core layer. The Router - XPRT Interface layer registers itself

with the Router Core layer upon complete initialization of the XPRT.

The Router - XPRT Interface layer upon registration exports the

following functionalities to the Router Core:

	1) Read from the XPRT

	2) # of bytes of data available to read

	3) Write to the XPRT

	4) Space available to write to the XPRT

	5) Close the XPRT

The user behavioral model of the IPC Router should be

	1) Create a port

	2) If server, register a name to the port

	3) If remote port not known, lookup through the name

	4) Send messages over the port to the remote port

	5) Receive messages along with the source info from the port

	6) Repeat steps 3, 4 & 5 as required

	7) If server, unregister the name from the port

	8) Close the port

Power Management

================

IPC Message Router uses wakelocks to ensure that the system does not go

into suspend mode while there are pending messages to be handled. Once all

the messages are handled, IPC Message Router releases the wakelocks to

allow the system to go into suspend mode and comply with the system power

management requirement.

SMP/multi-core

==============

The IPC Router uses mutexes & spinlocks to protect the shared data

structures to be SMP/multi-core safe.

Security

========

None

Performance

===========

None

Interface

=========

Kernel-space APIs:

------------------

/*

 * msm_ipc_router_create_port - Create a IPC Router port

 *

 * @msm_ipc_port_notify: notification function which will notify events

 *			 like READ_DATA_AVAIL, WRITE_DONE etc.

 * @priv: caller private context pointer, passed to the notify callback.

 *

 * @return: a valid port pointer on success, NULL on failure

 *

 */

struct msm_ipc_port * msm_ipc_router_create_port(

	void (*msm_ipc_port_notify)(unsigned event, void *data,

				    void *addr, void *priv),

	void *priv)

/*

 * msm_ipc_router_close_port - Close down the port

 *

 * @port: Port to be closed

 *

 * @return: 0 on success, -ve value on failure

 *

 */

int msm_ipc_router_close_port(struct msm_ipc_port *port)

/*

 * msm_ipc_router_send_to - Send data to a remote port

 *

 * @from_port: Source port of the message

 * @data: Data to be sent

 * @to_addr: Destination port name or address

 *

 * @return: number of bytes sent on success, -ve value on failure

 *

 */

int msm_ipc_router_send_to(struct msm_ipc_port *from_port,

			   struct sk_buff_head *data,

			   struct msm_ipc_addr *to_addr)

/*

 * msm_ipc_router_recv_from - Receive data over a port

 *

 * @port: Port from which the data has to be read

 * @data: Pointer to the data

 * @src_addr: If valid, filled with the source address of the data

 * @timeout: Time to wait for the data, if already not present

 *

 * @return: number of bytes received on success, -ve value on failure

 *

 */

int msm_ipc_router_recv_from(struct msm_ipc_port *port,

			     struct sk_buff_head **data,

			     struct msm_ipc_addr *src_addr,

			     unsigned long timeout)

/*

 * msm_ipc_router_register_server - Bind a local port with a service

 *				    name

 *

 * @server_port: Port to be bound with a service name

 * @name: Name to bind with the port

 *

 * @return: 0 on success, -ve value on failure

 *

 */

int msm_ipc_router_register_server(struct msm_ipc_port *server_port,

				   struct msm_ipc_addr *name)

/*

 * msm_ipc_router_unregister_server - Unbind the local port from its

 *				      service name

 *

 * @server_port: Port to be unbound from its service name

 *

 * @return: 0 on success, -ve value on failure

 *

 */

int msm_ipc_router_unregister_server(struct msm_ipc_port *server_port)

/*

 * msm_ipc_router_lookup_server - Lookup port address for the port name

 *

 * @name: Name to be looked up for

 *

 * @return: Port address corresponding to the service name on success,

 *	    NULL on failure

 *

 */

struct msm_ipc_addr * msm_ipc_router_lookup_server(

				struct msm_ipc_addr *name)

User-space APIs:

----------------

User-space applications/utilities can use the socket APIs to interface

with the IPC Router. IPC Router, in order to support the socket APIs,

will register a new socket Address/Protocol Family with the kernel

Socket layer. The identity of the new Address/Protocol Family will be

defined using the macro AF_MSM_IPC/PF_MSM_IPC (hardcoded to 38) in

include/linux/socket.h file. Since IPC Router supports only message

oriented transfer, only SOCK_DGRAM type of sockets will be supported

by the IPC Router.

Driver parameters

=================

debug_mask - This module parameter is used to enable/disable Router

log messages in the kernel logs. This parameter can take any value

from 0 to 255.

Dependencies

============

Drivers in this project:

-----------------------

The following drivers are present in this project, listed in the

bottom - up order of the stack.

1a) Router - SMD XPRT Interface driver. This driver is used to interface

the Router with the SMD transport.

1b) Router - HSIC XPRT Interface driver. This driver is used to interface

the Router with the HSIC_IPC Bridge transport for off-chip communication.

2) Core Router driver. This driver performs the core functionalities

3) Socket - Router Interface driver. This driver enables the socket

interface to be used with the IPC Router.

In the write/send direction, these drivers interact by invoking the

exported APIs from the underlying drivers. In the read/receive

directions, these drivers interact by passing messages/events.

Drivers Needed:

---------------

	1) SMD

	2) Kernel Socket Layer

	3) Platform Driver

	4) HSIC IPC Bridge Driver

To do

=====

Improvements:

-------------

The IPC Router is designed to route any messages, as long as the system

follows the network architecture and addressing schemes. But the

implementation in progress will route only QMI messages. With few

additional enhancements, it can route existing RPC messages too.

Qualcomm Technologies, Inc. IPC Router

Required properties:

-compatible:	should be "qcom,ipc_router"

-qcom,node-id:	unique ID to identify the node in network

Optional properties:

-qcom,default-peripheral: String property to indicate the default peripheral

			  to communicate

Example:

	qcom,ipc_router {

		compatible = "qcom,ipc_router";

		qcom,node-id = <1>;

		qcom,default-peripheral = "modem";

	};

/* Copyright (c) 2012-2016, The Linux Foundation. All rights reserved.

 *

 * This program is free software; you can redistribute it and/or modify

 * it under the terms of the GNU General Public License version 2 and

 * only version 2 as published by the Free Software Foundation.

 *

 * This program is distributed in the hope that it will be useful,

 * but WITHOUT ANY WARRANTY; without even the implied warranty of

 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

 * GNU General Public License for more details.

 */

#ifndef _IPC_ROUTER_H

#define _IPC_ROUTER_H

#include <linux/types.h>

#include <linux/socket.h>

#include <linux/errno.h>

#include <linux/mm.h>

#include <linux/list.h>

#include <linux/pm.h>

#include <linux/msm_ipc.h>

#include <linux/device.h>

#include <linux/kref.h>

/* Maximum Wakeup Source Name Size */

#define MAX_WS_NAME_SZ 32

#define IPC_RTR_ERR(buf, ...) \

	pr_err("IPC_RTR: " buf, __VA_ARGS__)

/**

 * enum msm_ipc_router_event - Events that will be generated by IPC Router

 */

enum msm_ipc_router_event {

	IPC_ROUTER_CTRL_CMD_DATA = 1,

	IPC_ROUTER_CTRL_CMD_HELLO,

	IPC_ROUTER_CTRL_CMD_BYE,

	IPC_ROUTER_CTRL_CMD_NEW_SERVER,

	IPC_ROUTER_CTRL_CMD_REMOVE_SERVER,

	IPC_ROUTER_CTRL_CMD_REMOVE_CLIENT,

	IPC_ROUTER_CTRL_CMD_RESUME_TX,

};

/**

 * rr_control_msg - Control message structure

 * @cmd: Command identifier for HELLO message in Version 1.

 * @hello: Message structure for HELLO message in Version 2.

 * @srv: Message structure for NEW_SERVER/REMOVE_SERVER events.

 * @cli: Message structure for REMOVE_CLIENT event.

 */

union rr_control_msg {

	uint32_t cmd;

	struct {

		uint32_t cmd;

		uint32_t checksum;

		uint32_t versions;

		uint32_t capability;

		uint32_t reserved;

	} hello;

	struct {

		uint32_t cmd;

		uint32_t service;

		uint32_t instance;

		uint32_t node_id;

		uint32_t port_id;

	} srv;

	struct {

		uint32_t cmd;

		uint32_t node_id;

		uint32_t port_id;

	} cli;

};

struct comm_mode_info {

	int mode;

	void *xprt_info;

};

enum ipc_rtr_af_event_type {

	IPCRTR_AF_INIT = 1,

	IPCRTR_AF_DEINIT,

};

/**

 * msm_ipc_port - Definition of IPC Router port

 * @list: List(local/control ports) in which this port is present.

 * @ref: Reference count for this port.

 * @this_port: Contains port's node_id and port_id information.

 * @port_name: Contains service & instance info if the port hosts a service.

 * @type: Type of the port - Client, Service, Control or Security Config.

 * @flags: Flags to identify the port state.

 * @port_lock_lhc3: Lock to protect access to the port information.

 * @mode_info: Communication mode of the port owner.

 * @port_rx_q: Receive queue where incoming messages are queued.

 * @port_rx_q_lock_lhc3: Lock to protect access to the port's rx_q.

 * @rx_ws_name: Name of the receive wakeup source.

 * @port_rx_ws: Wakeup source to prevent suspend until the rx_q is empty.

 * @port_rx_wait_q: Wait queue to wait for the incoming messages.

 * @restart_state: Flag to hold the restart state information.

 * @restart_lock: Lock to protect access to the restart_state.

 * @restart_wait: Wait Queue to wait for any restart events.

 * @endpoint: Contains the information related to user-space interface.

 * @notify: Function to notify the incoming events on the port.

 * @check_send_permissions: Function to check access control from this port.

 * @num_tx: Number of packets transmitted.

 * @num_rx: Number of packets received.

 * @num_tx_bytes: Number of bytes transmitted.

 * @num_rx_bytes: Number of bytes received.

 * @priv: Private information registered by the port owner.

 */

struct msm_ipc_port {

	struct list_head list;

	struct kref ref;

	struct msm_ipc_port_addr this_port;

	struct msm_ipc_port_name port_name;

	uint32_t type;

	unsigned int flags;

	struct mutex port_lock_lhc3;

	struct comm_mode_info mode_info;

	struct msm_ipc_port_addr dest_addr;

	int conn_status;

	struct list_head port_rx_q;

	struct mutex port_rx_q_lock_lhc3;

	char rx_ws_name[MAX_WS_NAME_SZ];

	struct wakeup_source *port_rx_ws;

	wait_queue_head_t port_rx_wait_q;

	wait_queue_head_t port_tx_wait_q;

	int restart_state;

	spinlock_t restart_lock;

	wait_queue_head_t restart_wait;

	void *rport_info;

	void *endpoint;

	void (*notify)(unsigned int event, void *oob_data,

		       size_t oob_data_len, void *priv);

	int (*check_send_permissions)(void *data);

	uint32_t num_tx;

	uint32_t num_rx;

	unsigned long num_tx_bytes;

	unsigned long num_rx_bytes;

	void *priv;

};

#ifdef CONFIG_IPC_ROUTER

/**

 * msm_ipc_router_create_port() - Create a IPC Router port/endpoint

 * @notify: Callback function to notify any event on the port.

 *   @event: Event ID to be handled.

 *   @oob_data: Any out-of-band data associated with the event.

 *   @oob_data_len: Size of the out-of-band data, if valid.

 *   @priv: Private data registered during the port creation.

 * @priv: Private info to be passed while the notification is generated.

 *

 * @return: Pointer to the port on success, NULL on error.

 */

struct msm_ipc_port *msm_ipc_router_create_port(

	void (*notify)(unsigned int event, void *oob_data,

		       size_t oob_data_len, void *priv),

	void *priv);

/**

 * msm_ipc_router_bind_control_port() - Bind a port as a control port

 * @port_ptr: Port which needs to be marked as a control port.

 *

 * @return: 0 on success, standard Linux error codes on error.

 */

int msm_ipc_router_bind_control_port(struct msm_ipc_port *port_ptr);

/**

 * msm_ipc_router_lookup_server_name() - Resolve server address

 * @srv_name: Name<service:instance> of the server to be resolved.

 * @srv_info: Buffer to hold the resolved address.

 * @num_entries_in_array: Number of server info the buffer can hold.

 * @lookup_mask: Mask to specify the range of instances to be resolved.

 *

 * @return: Number of server addresses resolved on success, < 0 on error.

 */

int msm_ipc_router_lookup_server_name(struct msm_ipc_port_name *srv_name,

				      struct msm_ipc_server_info *srv_info,

				      int num_entries_in_array,

				      uint32_t lookup_mask);

/**

 * msm_ipc_router_send_msg() - Send a message/packet

 * @src: Sender's address/port.

 * @dest: Destination address.

 * @data: Pointer to the data to be sent.

 * @data_len: Length of the data to be sent.

 *

 * @return: 0 on success, < 0 on error.

 */

int msm_ipc_router_send_msg(struct msm_ipc_port *src,

			    struct msm_ipc_addr *dest,

			    void *data, unsigned int data_len);

/**

 * msm_ipc_router_get_curr_pkt_size() - Get the packet size of the first

 *                                      packet in the rx queue

 * @port_ptr: Port which owns the rx queue.

 *

 * @return: Returns the size of the first packet, if available.

 *          0 if no packets available, < 0 on error.

 */

int msm_ipc_router_get_curr_pkt_size(struct msm_ipc_port *port_ptr);

/**

 * msm_ipc_router_read_msg() - Read a message/packet

 * @port_ptr: Receiver's port/address.

 * @data: Pointer containing the address of the received data.

 * @src: Address of the sender/source.

 * @len: Length of the data being read.

 *

 * @return: 0 on success, < 0 on error.

 */

int msm_ipc_router_read_msg(struct msm_ipc_port *port_ptr,

			    struct msm_ipc_addr *src,

			    unsigned char **data,

			    unsigned int *len);

/**

 * msm_ipc_router_close_port() - Close the port

 * @port_ptr: Pointer to the port to be closed.

 *

 * @return: 0 on success, < 0 on error.

 */

int msm_ipc_router_close_port(struct msm_ipc_port *port_ptr);

/**

 * msm_ipc_router_register_server() - Register a service on a port

 * @server_port: IPC Router port with which a service is registered.

 * @name: Service name <service_id:instance_id> that gets registered.

 *

 * @return: 0 on success, standard Linux error codes on error.

 */

int msm_ipc_router_register_server(struct msm_ipc_port *server_port,

				   struct msm_ipc_addr *name);

/**

 * msm_ipc_router_unregister_server() - Unregister a service from a port

 * @server_port: Port with with a service is already registered.

 *

 * @return: 0 on success, standard Linux error codes on error.

 */

int msm_ipc_router_unregister_server(struct msm_ipc_port *server_port);

/**

 * register_ipcrtr_af_init_notifier() - Register for ipc router socket

 *				address family initialization callback

 * @nb: Notifier block which will be notified once address family is

 *	initialized.

 *

 * Return: 0 on success, standard error code otherwise.

 */

int register_ipcrtr_af_init_notifier(struct notifier_block *nb);

/**

 * unregister_ipcrtr_af_init_notifier() - Unregister for ipc router socket

 *					address family initialization callback

 * @nb: Notifier block which will be notified once address family is

 *	initialized.

 *

 * Return: 0 on success, standard error code otherwise.

 */

int unregister_ipcrtr_af_init_notifier(struct notifier_block *nb);

#else

struct msm_ipc_port *msm_ipc_router_create_port(

	void (*notify)(unsigned int event, void *oob_data,

		       size_t oob_data_len, void *priv),

	void *priv)

{

	return NULL;

}

static inline int msm_ipc_router_bind_control_port(

		struct msm_ipc_port *port_ptr)

{

	return -ENODEV;

}

int msm_ipc_router_lookup_server_name(struct msm_ipc_port_name *srv_name,

				      struct msm_ipc_server_info *srv_info,

				      int num_entries_in_array,

				      uint32_t lookup_mask)

{

	return -ENODEV;

}

int msm_ipc_router_send_msg(struct msm_ipc_port *src,

			    struct msm_ipc_addr *dest,

			    void *data, unsigned int data_len)

{

	return -ENODEV;

}

int msm_ipc_router_get_curr_pkt_size(struct msm_ipc_port *port_ptr)

{

	return -ENODEV;

}

int msm_ipc_router_read_msg(struct msm_ipc_port *port_ptr,

			    struct msm_ipc_addr *src,

			    unsigned char **data,

			    unsigned int *len)

{

	return -ENODEV;

}

int msm_ipc_router_close_port(struct msm_ipc_port *port_ptr)

{

	return -ENODEV;

}

static inline int msm_ipc_router_register_server(

			struct msm_ipc_port *server_port,

			struct msm_ipc_addr *name)

{

	return -ENODEV;

}

static inline int msm_ipc_router_unregister_server(

			struct msm_ipc_port *server_port)

{

	return -ENODEV;

}

int register_ipcrtr_af_init_notifier(struct notifier_block *nb)

{

	return -ENODEV;

}

int unregister_ipcrtr_af_init_notifier(struct notifier_block *nb)

{

	return -ENODEV;

}

#endif

#endif

header-y += msdos_fs.h

header-y += msg.h

header-y += msm_ion.h

header-y += msm_ipc.h

header-y += msm_rmnet.h

header-y += mtio.h

header-y += nbd.h