How to compile a DPDK application as a library - dpdk

Question

I have a program using DPDK and I am compiling it using the Makefile provided in the examples.
If i compile the program as an APP (as describe here), all goes
well.
However, my code is part of a larger project, for which the use of a
separate makefile causes a lot of troubles.
So I bundled my code in a library, as described in the same page.
The program that calls the functions in the library (to initialize the
EAL) is getting this error:
MBUF: error setting mempool handler
Cannot init mbuf pool
It seems that, when the application is compiled as a library, the EAL cannot be initialized correctly.
I report here the steps to reproduce the problem using the l2fwd example.
Background
I built DPDK from source as described here, and I have one ethernet interface bound to DPDK drivers:
$ $RTE_SDK/usertools/dpdk-devbind.py --status |head -n4
Network devices using DPDK-compatible driver
============================================
0000:01:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' drv=igb_uio unused=ixgbe
Run the l2fwd example
Copy the example folder first and then run the example:
$ cp -r $RTE_SDK/examples/l2fwd $RTE_SDK/examples/l2fwd-lib/
$ cd $RTE_SDK/examples/l2fwd
$ make
CC main.o
LD l2fwd
INSTALL-APP l2fwd
INSTALL-MAP l2fwd.map
$ sudo ./build/l2fwd -l 0-3 -- -p 0x1
Port statistics ====================================
Statistics for port 0 ------------------------------
Packets sent: 0
Packets received: 0
Packets dropped: 0
Aggregate statistics ===============================
Total packets sent: 0
Total packets received: 0
Total packets dropped: 0
====================================================
It works!
Build the same example as a library
cd ../l2fwd-lib/
mv main.c l2fwd.c
Modify l2fwd.c, adding #include "l2fwd.h" on top and replacing
int main(int argc, char **argv)
with
int start(int argc, char **argv)
Create the header file l2fwd.h with the library interface:
int start(int argc, char **argv);
Modify the Makefile as described in the docs:
APP = l2fwd ---> LIB = libl2fwd.a
SRCS-y := main.c ---> SRCS-y := l2fwd.c
include $(RTE_SDK)/mk/rte.extapp.mk ---> include $(RTE_SDK)/mk/rte.extlib.mk
Compile the library:
$ make
CC l2fwd.o
AR libl2fwd.a
INSTALL-LIB libl2fwd.a
Write a program that uses the library. Create the main.c file with just these 2 lines:
#include "l2fwd.h"
int main (int argc, char **argv) { start(argc, argv); }
compile it (with all the needed libraries):
gcc -L build/lib/ -L $RTE_SDK/build/lib/ main.c -o main.o -l l2fwd -l dpdk -l numa -pthread -l dl
Finally, run it with the same parameters used earlier:
$ sudo ./main.o -l 0-3 -- -p 0x1
EAL: Detected 40 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Probing VFIO support...
MAC updating enabled
EAL: Error - exiting with code: 1
Cause: No Ethernet ports - bye
In this case, the EAL cannot be initialized properly (the ethernet port is still bound to the DPDK drivers).
Edit 1
According to #Andriy Berestovskyy, the linker --whole-archive option is needed when linking DPDK libraries. This solves the problem with the examples.
However, my program is facing a different problem now. I need to use a custom build system, so I am linking the DPDK application as a library. During execution I get the error:
MBUF: error setting mempool handler
mempool/dpaa2: Not a valid dpaa2 buffer pool
Looks like it is using the dpaa2 driver, that is the wrong driver (my NIC is using igb_uio). Any hints on why this is happening? The same code worked when compiled as a DPDK app, so it is probably related to the linking process.
Edit 2
This error is due to the fact that I compiled DPDK with the
CONFIG_RTE_BUILD_SHARED_LIB=y option. When DPDK is built as a shared library, the driver must be explicitly set using the -d EAL command line option.
I re-compiled DPDK with CONFIG_RTE_BUILD_SHARED_LIB=n and the problem was solved.

Answer 1

compile it (with all the needed libraries):
gcc -L build/lib/ -L $RTE_SDK/build/lib/ main.c -o main.o -l l2fwd -l dpdk -l numa -pthread -l dl
I guess the issue is in this step. DPDK uses link-level constructors (i.e. __attribute__((constructor))). See the definition of RTE_INIT() for example. Also there are callbacks etc. etc.
So to properly link with DPDK we have to:
either use an rte.app.mk in the Makefile (see Development Kit Build System)
or in case we need to use a custom build system, we need to link DPDK libraries after the --whole-archive option.

Answer 2

I got the similar issue and as mentioned in here, I used -d option to link the libray "-d /usr/lib64/librte_mempool_ring.so". It worked.