Merge branch 'pktgen-new-scripts'

 # ethtool -C ethX rx-usecs 30

Viewing threads

===============

/proc/net/pktgen/kpktgend_0 

Name: kpktgend_0  max_before_softirq: 10000

Kernel threads

==============

Pktgen creates a thread for each CPU with affinity to that CPU.

Which is controlled through procfile /proc/net/pktgen/kpktgend_X.

Example: /proc/net/pktgen/kpktgend_0

 Running:

Stopped: eth1 

Result: OK: max_before_softirq=10000

 Stopped: eth4@0

 Result: OK: add_device=eth4@0

Most important are the devices assigned to the thread.

The two basic thread commands are:

 * add_device DEVICE@NAME -- adds a single device

 * rem_device_all         -- remove all associated devices

Most important are the devices assigned to the thread.  Note that a

device can only belong to one thread.

When adding a device to a thread, a corrosponding procfile is created

which is used for configuring this device. Thus, device names need to

be unique.

To support adding the same device to multiple threads, which is useful

with multi queue NICs, a the device naming scheme is extended with "@":

 device@something

The part after "@" can be anything, but it is custom to use the thread

number.

Viewing devices

===============

holds running statistics.  The Result is printed after a run or after

interruption.  Example:

/proc/net/pktgen/eth1       

/proc/net/pktgen/eth4@0

Params: count 10000000  min_pkt_size: 60  max_pkt_size: 60

     frags: 0  delay: 0  clone_skb: 1000000  ifname: eth1

 Params: count 100000  min_pkt_size: 60  max_pkt_size: 60

     frags: 0  delay: 0  clone_skb: 64  ifname: eth4@0

     flows: 0 flowlen: 0

     dst_min: 10.10.11.2  dst_max: 

     queue_map_min: 0  queue_map_max: 0

     dst_min: 192.168.81.2  dst_max:

     src_min:   src_max:

     src_mac: 00:00:00:00:00:00  dst_mac: 00:04:23:AC:FD:82

     udp_src_min: 9  udp_src_max: 9  udp_dst_min: 9  udp_dst_max: 9

     src_mac: 90:e2:ba:0a:56:b4 dst_mac: 00:1b:21:3c:9d:f8

     udp_src_min: 9  udp_src_max: 109  udp_dst_min: 9  udp_dst_max: 9

     src_mac_count: 0  dst_mac_count: 0

     Flags: 

     Flags: UDPSRC_RND  NO_TIMESTAMP  QUEUE_MAP_CPU

 Current:

     pkts-sofar: 10000000  errors: 39664

     started: 1103053986245187us  stopped: 1103053999346329us idle: 880401us

     seq_num: 10000011  cur_dst_mac_offset: 0  cur_src_mac_offset: 0

     cur_saddr: 0x10a0a0a  cur_daddr: 0x20b0a0a

     cur_udp_dst: 9  cur_udp_src: 9

     pkts-sofar: 100000  errors: 0

     started: 623913381008us  stopped: 623913396439us idle: 25us

     seq_num: 100001  cur_dst_mac_offset: 0  cur_src_mac_offset: 0

     cur_saddr: 192.168.8.3  cur_daddr: 192.168.81.2

     cur_udp_dst: 9  cur_udp_src: 42

     cur_queue_map: 0

     flows: 0

Result: OK: 13101142(c12220741+d880401) usec, 10000000 (60byte,0frags)

  763292pps 390Mb/sec (390805504bps) errors: 39664

 Result: OK: 15430(c15405+d25) usec, 100000 (60byte,0frags)

  6480562pps 3110Mb/sec (3110669760bps) errors: 0

Configuring threads and devices

================================

Configuring devices

===================

This is done via the /proc interface, and most easily done via pgset

as defined in the sample scripts.

Sample scripts

==============

A collection of small tutorial scripts for pktgen is in the

samples/pktgen directory:

A collection of tutorial scripts and helpers for pktgen is in the

samples/pktgen directory. The helper parameters.sh file support easy

and consistant parameter parsing across the sample scripts.

Usage example and help:

 ./pktgen_sample01_simple.sh -i eth4 -m 00:1B:21:3C:9D:F8 -d 192.168.8.2

Usage: ./pktgen_sample01_simple.sh [-vx] -i ethX

  -i : ($DEV)       output interface/device (required)

  -s : ($PKT_SIZE)  packet size

  -d : ($DEST_IP)   destination IP

  -m : ($DST_MAC)   destination MAC-addr

  -t : ($THREADS)   threads to start

  -c : ($SKB_CLONE) SKB clones send before alloc new SKB

  -b : ($BURST)     HW level bursting of SKBs

  -v : ($VERBOSE)   verbose

  -x : ($DEBUG)     debug

The global variables being set are also listed.  E.g. the required

interface/device parameter "-i" sets variable $DEV.  Copy the

pktgen_sampleXX scripts and modify them to fit your own needs.

The old scripts:

pktgen.conf-1-1                  # 1 CPU 1 dev 

pktgen.conf-1-2                  # 1 CPU 2 dev

pktgen.conf-2-1                  # 2 CPU's 1 dev 

pktgen.conf-2-2                  # 2 CPU's 2 dev

pktgen.conf-1-1-rdos             # 1 CPU 1 dev w. route DoS 

pktgen.conf-1-1-ip6              # 1 CPU 1 dev ipv6

pktgen.conf-1-1-ip6-rdos         # 1 CPU 1 dev ipv6  w. route DoS

pktgen.conf-1-1-flows            # 1 CPU 1 dev multiple flows.

Run in shell: ./pktgen.conf-X-Y

This does all the setup including sending.

Interrupt affinity

===================

also assign /proc/irq/XX/smp_affinity so that the TX interrupts are bound

to the same CPU.  This reduces cache bouncing when freeing skbs.

Plus using the device flag QUEUE_MAP_CPU, which maps the SKBs TX queue

to the running threads CPU (directly from smp_processor_id()).

Enable IPsec

============

Default IPsec transformation with ESP encapsulation plus transport mode

start

stop

reset

** Thread commands:

add_device

rem_device_all

max_before_softirq

** Device commands:

count

clone_skb

burst

debug

frags

min_pkt_size

max_pkt_size

queue_map_min

queue_map_max

skb_priority

tos           (ipv4)

traffic_class (ipv6)

mpls

udp_src_min

udp_dst_min

udp_dst_max

node

flag

  IPSRC_RND

  IPDST_RND

  NODE_ALLOC

  NO_TIMESTAMP

spi (ipsec)

dst_min

dst_max

clear_counters

dst6

src6

dst6

dst6_max

dst6_min

flows

flowlen

xmit_mode <start_xmit|netif_receive>

vlan_cfi

vlan_id

vlan_p

svlan_cfi

svlan_id

svlan_p

References:

ftp://robur.slu.se/pub/Linux/net-development/pktgen-testing/

#include <asm/dma.h>

#include <asm/div64.h>		/* do_div */

#define VERSION	"2.74"

#define VERSION	"2.75"

#define IP_NAME_SZ 32

#define MAX_MPLS_LABELS 16 /* This is the max label stack depth */

#define MPLS_STACK_BOTTOM htonl(0x00000100)

		pktgen_reset_all_threads(pn);

	else

		pr_warn("Unknown command: %s\n", data);

		return -EINVAL;

	return count;

}

Sample and benchmark scripts for pktgen (packet generator)

==========================================================

This directory contains some pktgen sample and benchmark scripts, that

can easily be copied and adjusted for your own use-case.

General doc is located in kernel: Documentation/networking/pktgen.txt

Helper include files

====================

This directory contains two helper shell files, that can be "included"

by shell source'ing.  Namely "functions.sh" and "parameters.sh".

Common parameters

-----------------

The parameters.sh file support easy and consistant parameter parsing

across the sample scripts.  Usage example is printed on errors::

 Usage: ./pktgen_sample01_simple.sh [-vx] -i ethX

  -i : ($DEV)       output interface/device (required)

  -s : ($PKT_SIZE)  packet size

  -d : ($DEST_IP)   destination IP

  -m : ($DST_MAC)   destination MAC-addr

  -t : ($THREADS)   threads to start

  -c : ($SKB_CLONE) SKB clones send before alloc new SKB

  -b : ($BURST)     HW level bursting of SKBs

  -v : ($VERBOSE)   verbose

  -x : ($DEBUG)     debug

The global variable being set is also listed.  E.g. the required

interface/device parameter "-i" sets variable $DEV.

Common functions

----------------

The functions.sh file provides; Three different shell functions for

configuring the different components of pktgen: pg_ctrl(), pg_thread()

and pg_set().

These functions correspond to pktgens different components.

 * pg_ctrl()   control "pgctrl" (/proc/net/pktgen/pgctrl)

 * pg_thread() control the kernel threads and binding to devices

 * pg_set()    control setup of individual devices

See sample scripts for usage examples.

#

# Common functions used by pktgen scripts

#  - Depending on bash 3 (or higher) syntax

#

# Author: Jesper Dangaaard Brouer

# License: GPL

## -- General shell logging cmds --

function err() {

    local exitcode=$1

    shift

    echo "ERROR: $@" >&2

    exit $exitcode

}

function warn() {

    echo "WARN : $@" >&2

}

function info() {

    if [[ -n "$VERBOSE" ]]; then

	echo "INFO : $@" >&2

    fi

}

## -- Pktgen proc config commands -- ##

export PROC_DIR=/proc/net/pktgen

#

# Three different shell functions for configuring the different

# components of pktgen:

#   pg_ctrl(), pg_thread() and pg_set().

#

# These functions correspond to pktgens different components.

# * pg_ctrl()   control "pgctrl" (/proc/net/pktgen/pgctrl)

# * pg_thread() control the kernel threads and binding to devices

# * pg_set()    control setup of individual devices

function pg_ctrl() {

    local proc_file="pgctrl"

    proc_cmd ${proc_file} "$@"

}

function pg_thread() {

    local thread=$1

    local proc_file="kpktgend_${thread}"

    shift

    proc_cmd ${proc_file} "$@"

}

function pg_set() {

    local dev=$1

    local proc_file="$dev"

    shift

    proc_cmd ${proc_file} "$@"

}

# More generic replacement for pgset(), that does not depend on global

# variable for proc file.

function proc_cmd() {

    local result

    local proc_file=$1

    # after shift, the remaining args are contained in $@

    shift

    local proc_ctrl=${PROC_DIR}/$proc_file

    if [[ ! -e "$proc_ctrl" ]]; then

	err 3 "proc file:$proc_ctrl does not exists (dev added to thread?)"

    else

	if [[ ! -w "$proc_ctrl" ]]; then

	    err 4 "proc file:$proc_ctrl not writable, not root?!"

	fi

    fi

    if [[ "$DEBUG" == "yes" ]]; then

	echo "cmd: $@ > $proc_ctrl"

    fi

    # Quoting of "$@" is important for space expansion

    echo "$@" > "$proc_ctrl"

    local status=$?

    result=$(grep "Result: OK:" $proc_ctrl)

    # Due to pgctrl, cannot use exit code $? from grep

    if [[ "$result" == "" ]]; then

	grep "Result:" $proc_ctrl >&2

    fi

    if (( $status != 0 )); then

	err 5 "Write error($status) occurred cmd: \"$@ > $proc_ctrl\""

    fi

}

# Old obsolete "pgset" function, with slightly improved err handling

function pgset() {

    local result

    if [[ "$DEBUG" == "yes" ]]; then

	echo "cmd: $1 > $PGDEV"

    fi

    echo $1 > $PGDEV

    local status=$?

    result=`cat $PGDEV | fgrep "Result: OK:"`

    if [[ "$result" == "" ]]; then

         cat $PGDEV | fgrep Result:

    fi

    if (( $status != 0 )); then

	err 5 "Write error($status) occurred cmd: \"$1 > $PGDEV\""

    fi

}

## -- General shell tricks --

function root_check_run_with_sudo() {

    # Trick so, program can be run as normal user, will just use "sudo"

    #  call as root_check_run_as_sudo "$@"

    if [ "$EUID" -ne 0 ]; then

	if [ -x $0 ]; then # Directly executable use sudo

	    info "Not root, running with sudo"

            sudo "$0" "$@"

            exit $?

	fi

	err 4 "cannot perform sudo run of $0"

    fi

}

#

# Common parameter parsing for pktgen scripts

#

function usage() {

    echo ""

    echo "Usage: $0 [-vx] -i ethX"

    echo "  -i : (\$DEV)       output interface/device (required)"

    echo "  -s : (\$PKT_SIZE)  packet size"

    echo "  -d : (\$DEST_IP)   destination IP"

    echo "  -m : (\$DST_MAC)   destination MAC-addr"

    echo "  -t : (\$THREADS)   threads to start"

    echo "  -c : (\$SKB_CLONE) SKB clones send before alloc new SKB"

    echo "  -b : (\$BURST)     HW level bursting of SKBs"

    echo "  -v : (\$VERBOSE)   verbose"

    echo "  -x : (\$DEBUG)     debug"

    echo ""

}

##  --- Parse command line arguments / parameters ---

## echo "Commandline options:"

while getopts "s:i:d:m:t:c:b:vxh" option; do

    case $option in

        i) # interface

          export DEV=$OPTARG

	  info "Output device set to: DEV=$DEV"

          ;;

        s)

          export PKT_SIZE=$OPTARG

	  info "Packet size set to: PKT_SIZE=$PKT_SIZE bytes"

          ;;

        d) # destination IP

          export DEST_IP=$OPTARG

	  info "Destination IP set to: DEST_IP=$DEST_IP"

          ;;

        m) # MAC

          export DST_MAC=$OPTARG

	  info "Destination MAC set to: DST_MAC=$DST_MAC"

          ;;

        t)

	  export THREADS=$OPTARG

          export CPU_THREADS=$OPTARG

	  let "CPU_THREADS -= 1"

	  info "Number of threads to start: $THREADS (0 to $CPU_THREADS)"

          ;;

        c)

	  export CLONE_SKB=$OPTARG

	  info "CLONE_SKB=$CLONE_SKB"

          ;;

        b)

	  export BURST=$OPTARG

	  info "SKB bursting: BURST=$BURST"

          ;;

        v)

          export VERBOSE=yes

          info "Verbose mode: VERBOSE=$VERBOSE"

          ;;

        x)

          export DEBUG=yes

          info "Debug mode: DEBUG=$DEBUG"

          ;;

        h|?|*)

          usage;

          err 2 "[ERROR] Unknown parameters!!!"

    esac

done

shift $(( $OPTIND - 1 ))

if [ -z "$PKT_SIZE" ]; then

    # NIC adds 4 bytes CRC

    export PKT_SIZE=60

    info "Default packet size set to: set to: $PKT_SIZE bytes"

fi

if [ -z "$THREADS" ]; then

    # Zero CPU threads means one thread, because CPU numbers are zero indexed

    export CPU_THREADS=0

    export THREADS=1

fi

if [ -z "$DEV" ]; then

    usage

    err 2 "Please specify output device"

fi

if [ -z "$DST_MAC" ]; then

    warn "Missing destination MAC address"

fi

if [ -z "$DEST_IP" ]; then

    warn "Missing destination IP address"

fi

if [ ! -d /proc/net/pktgen ]; then

    info "Loading kernel module: pktgen"

    modprobe pktgen

fi