3 This plugin is optional and only has a specific use case: monitoring DPDK applications
4 that don't expose stats in any other way than the DPDK xstats API.
6 **Data Plane Development Kit** (DPDK) is a set of drivers and libraries for fast
7 packet processing. Please note that this plugin is a polling based plugin rather
8 than an events based plugin (using it will drive up core utilization on a system).
10 **PLEASE DO NOT USE THIS PLUGIN FOR OVS-DPDK**. dpdkstat is really for DPDK
11 applications that have no other way of exposing stats. For OVS or OVS-with-DPDK the
12 Open vSwitch plugins available in collectd Version 5.8.0 should be used for
13 collecting stats and events. In addition the OVS plugin is events based rather
14 than polling based and will have a smaller footprint on the system.
18 The *dpdkstat plugin* has the following requirements:
23 You can also build with GCC 4.8 (e.g. Ubuntu 14.04) if you specify the SSSE3
24 instruction set manually:
26 make -j CFLAGS+='-mssse3'
30 * Setup the build environment:
32 Ensure that you have GCC 4.9 or later. Ubuntu 14.04, for example, has GCC
33 4.8 by default and requires an upgrade:
35 add-apt-repository ppa:ubuntu-toolchain-r/test
37 apt-get install gcc-4.9
39 If you know that the platform that you wish to run collectd on supports the
40 SSSE3 instruction set, GCC 4.8 also works if you enable SSSE3 manually:
42 make -j CFLAGS+='-mssse3'
46 git clone git://dpdk.org/dpdk
48 * Checkout the [DPDK system
49 requirements](http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html) and make
50 sure you have the required tools and hugepage setup as specified there.
52 **Note:** It's recommended to use the 1GB hugepage setup for best
53 performance, please follow the instruction for "Reserving Hugepages for DPDK
54 Use" in the link above.
56 However if you plan on configuring 2MB hugepages on the fly please ensure to
57 add appropriate commands to reserve hugepages in a system startup script if
58 collectd is booted at system startup time. These commands include:
61 mount -t hugetlbfs nodev /mnt/huge
62 echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
64 * To configure the DPDK build for the combined shared library and enable autoload
65 of pmd drivers modify `config/common_base` in your DPDK as follows
68 # Compile to share library
70 -CONFIG_RTE_BUILD_SHARED_LIB=n
71 +CONFIG_RTE_BUILD_SHARED_LIB=y
73 # Default driver path (or "" to disable)
74 -CONFIG_RTE_EAL_PMD_PATH=""
75 +CONFIG_RTE_EAL_PMD_PATH="/usr/lib/dpdk-pmd/"
77 * Prepare the configuration for the appropriate target as specified at:
78 http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html.
82 make config T=x86_64-native-linuxapp-gcc
88 * Install DPDK to `/usr`
90 sudo make install prefix=/usr
92 * Create dpdk-pmd folder
94 mkdir -p /usr/lib/dpdk-pmd
96 * Create symlinks to pmd drivers
98 find /usr/lib -type f -name 'librte_pmd*' | while read path ; do ln -s $path /usr/lib/dpdk-pmd/`echo $path | grep -o 'librte_.*so'` ; done
100 **Note 1:** You must run make install as the configuration of collectd with
101 DPDK expects DPDK to be installed somewhere.
103 **Note 2:** If you don't specify a prefix then DPDK will be installed in
106 **Note 3:** If you are not root then use sudo to make install DPDK to the
107 appropriate location.
109 **Note 4:** You **MUST** create symlink to a NIC driver lib. This way collectd
110 will be able to work with device bound to dpdk.
112 * Check that the DPDK library has been installed in `/usr/lib` or `/lib`:
114 ls /usr/lib | grep dpdk
116 * Bind the interfaces to use with dpdkstat to DPDK:
118 DPDK devices can be setup with either the VFIO (for DPDK 1.7+) or UIO
121 **Note:** UIO requires inserting an out of tree driver `igb_uio.ko` that is
130 * Insert `igb_uio.ko`:
132 sudo insmod $DPDK_BUILD/kmod/igb_uio.ko
134 * Bind network device to `igb_uio`:
136 sudo $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio eth1
140 * VFIO needs to be supported in the kernel and the BIOS. More information
141 can be found at: http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html.
142 * Insert the `vfio-pci.ko` module:
146 * Set the correct permissions for the VFIO device:
148 sudo /usr/bin/chmod a+x /dev/vfio
149 sudo /usr/bin/chmod 0666 /dev/vfio/*
151 * Bind the network device to `vfio-pci`:
153 sudo $DPDK_DIR/tools/dpdk_nic_bind.py --bind=vfio-pci eth1
155 **Note:** Please ensure to add appropriate commands to bind the network
156 interfaces to DPDK in a system startup script if collectd is booted at
159 * Run `ldconfig` to update the shared library cache.
163 To build static DPDK library for use with collectd:
165 * To configure DPDK to build the combined static library `libdpdk.a` ensure
166 that `CONFIG_RTE_BUILD_SHARED_LIB` is set to ānā in `config/common_base` in
167 your DPDK as follows:
170 # Compile to share library
172 CONFIG_RTE_BUILD_SHARED_LIB=n
174 * Prepare the configuration for the appropriate target as specified at:
175 http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html.
179 make config T=x86_64-native-linuxapp-gcc
181 * Build the target using `-fPIC`:
183 make EXTRA_CFLAGS=-fPIC -j
185 * Install DPDK to `/usr`:
187 sudo make install prefix=/usr
189 ## Build collectd with DPDK
191 **Note:** DPDK 16.04 is the minimum version and currently supported version of
192 DPDK required for the dpdkstat plugin. This is to allow the plugin to take
193 advantage of functions added to detect if the DPDK primary process is alive.
196 **Note:** The *Address-Space Layout Randomization* (ASLR) security feature in
197 Linux should be disabled, in order for the same hugepage memory mappings to be
198 present in all DPDK multi-process applications. Note that this has security
203 echo 0 > /proc/sys/kernel/randomize_va_space
205 * To fully enable ASLR:
207 echo 2 > /proc/sys/kernel/randomize_va_space
209 See also: http://dpdk.org/doc/guides/prog_guide/multi_proc_support.html
211 * Generate the build script as specified below. (i.e. run `build.sh`).
212 * Configure collectd with the DPDK shared library:
214 ./configure --with-libdpdk=/usr
216 ### Build with the static DPDK library
218 To configure collectd with the DPDK static library:
220 * Run *configure* with the following CFLAGS:
222 ./configure --with-libdpdk=/usr CFLAGS=" -lpthread -Wl,--whole-archive -Wl,-ldpdk -Wl,-lm -Wl,-lrt -Wl,-lpcap -Wl,-ldl -Wl,--no-whole-archive"
224 * Make sure that dpdk and dpdkstat are enabled in the *configure* output.
230 libdpdk . . . . . . . . yes
234 dpdkstat . . . . . . .yes
238 make -j && make -j install.
240 **Note:** As mentioned above, if you are building on Ubuntu 14.04 with
241 GCC <= 4.8.X, you need to use:
243 make -j CFLAGS+='-mssse3' && make -j install
247 * The same PCI device configuration should be passed to the primary process as
248 the secondary process uses the same port indexes as the primary.
249 * A blacklist / whitelist of NICs isn't supported yet.
253 The *dpdkstat plugin* is copyright (c) 2016 *Intel Corporation* and licensed
254 under the *MIT license*. Full licensing terms can be found in the file