DPDK KNI example - dpdk

I want to work with DPDK for packet processing I can't run kni example
I tried every possible config and I get :
EAL: Error - exiting with code: 1
Cause: Invalid option specified
./build/kni -c 0x0f -n 4 -- -P -p 0x3 -config="(0,2,3),(1,4,5)"
./build/kni -c 0x0f -n 4 -- -P -p 0x3 -config="(0,4,6),(1,5,7)"


I should put a "--" before config, but in all samples it has one dash


Your command has en error:
./build/kni -c 0x0f -n 4 -- -P -p 0x3 -config="(0,4,6),(1,5,7)"
The -P option should put before -config, like this:
-c 0x0f -n 4 -- -p 0x3 -P --config="(0,4,6),(1,5,7)"


From DPDK sample application user guide:
to run the application with two ports served by six lcores, one lcore of RX, one
lcore of TX, and one lcore of kernel thread for each port:
./build/kni -c 0xf0 -n 4 -- -P -p 0x3 -config="(0,4,6,8),(1,5,7,9)"
you need to check your "config" inputs(should be 4 elements for each port).


the following command will work
./build/kni -l 4-5 -n 4 -- -p1 -P --config="(0, 4, 5)"

Related

rte_eth_dev_count_avail() from the DPDK API returns 0 intel x550 ethernet card on linux 5.4.0 dpdk version 19.11.12

built dpdk 19.11.12 from source and i have written an ext app built using these libraries. I use dpdk-devbind to bind a device to igb_uio .
The device i have is an Intel X550. When i run testpmd it seems to recognize the device but int my app rte_eth_dev_count_avail always returns 0
I would belive this has to either do with the way i have built my dpdk library or my app
this is extract of the Makefile i am using to build my app
RTE_SDK ?= /usr/local/share/dpdk
RTE_TARGET ?= x86_64-default-linuxapp-gcc
include $(RTE_SDK)/mk/rte.vars.mk
# binary name
APP = myapp
# all source are stored in SRCS-y
SRCS-y := main.c
WERROR_FLAGS += -Werror
CFLAGS += -O3 -g
CFLAGS += $(WERROR_FLAGS) -Wno-address-of-packed-member
CFLAGS += -Wall
CFLAGS += -pie -fPIE -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -fstack-protector-strong
CFLAGS += -DSTATSD
LDFLAGS += -z relro -z now
LDFLAGS += -ljansson -lrte_pmd_ixgbe
LDLIBS += -lrte_pmd_ixgbe
# disable since libxenstore.so isn't linked properly
#CONFIG_RTE_LIBRTE_PMD_XENVIRT = n
CONFIG_RTE_LIBRTE_PMD_IXGBE = y
include $(RTE_SDK)/mk/rte.extapp.mk
I build my dpdk libraries using this
RUN make -C /dpdk/dpdk-stable-${DPDK_VERSION}/ config T=x86_64-native-linuxapp-gcc
RUN cd /dpdk/dpdk-stable-${DPDK_VERSION}/ && cat build/.config | sed -e s/CONFIG_RTE_BUILD_SHARED_LIB=n/CONFIG_RTE_BUILD_SHARED_LIB=y/g > /tmp/out && mv /tmp/out build/.config
RUN cd /dpdk/dpdk-stable-${DPDK_VERSION}/ && cat build/.config | sed -e s/CONFIG_RTE_LIBRTE_ETHDEV_DEBUG=n/CONFIG_RTE_LIBRTE_ETHDEV_DEBUG=y/g > /tmp/out && mv /tmp/out build/.config
RUN cd /dpdk/dpdk-stable-${DPDK_VERSION}/ && cat build/.config | sed -e s/CONFIG_RTE_LIBRTE_E1000_DEBUG_RX=n/CONFIG_RTE_LIBRTE_E1000_DEBUG_RX=y/g > /tmp/out && mv /tmp/out build/.config
RUN cd /dpdk/dpdk-stable-${DPDK_VERSION}/ && cat build/.config | sed -e s/CONFIG_RTE_LIBRTE_E1000_DEBUG_TX=n/CONFIG_RTE_LIBRTE_E1000_DEBUG_TX=y/g > /tmp/out && mv /tmp/out build/.config
RUN cd /dpdk/dpdk-stable-${DPDK_VERSION}/ && cat build/.config | sed -e s/CONFIG_RTE_LIBRTE_IXGBE_DEBUG_RX=n/CONFIG_RTE_LIBRTE_IXGBE_DEBUG_RX=y/g > /tmp/out && mv /tmp/out build/.config
RUN cd /dpdk/dpdk-stable-${DPDK_VERSION}/ && cat build/.config | sed -e s/CONFIG_RTE_LIBRTE_IXGBE_DEBUG_TX=n/CONFIG_RTE_LIBRTE_IXGBE_DEBUG_TX=y/g > /tmp/out && mv /tmp/out build/.config
RUN cd /dpdk/dpdk-stable-${DPDK_VERSION}/ && cat build/.config | sed -e s/CONFIG_RTE_LOG_DP_LEVEL=RTE_LOG_INFO/CONFIG_RTE_LOG_DP_LEVEL=RTE_LOG_DEBUG/g > /tmp/out && mv /tmp/out build/.config
RUN cd /dpdk/dpdk-stable-${DPDK_VERSION}/ && cat build/.config | sed -e s/CONFIG_RTE_LOG_HISTORY=256/CONFIG_RTE_LOG_HISTORY=2048/g > /tmp/out && mv /tmp/out build/.config
i can bind to the interfaces using dpdk-devbind and testpmd also seems to identify and be able to bind to these devices
but my app is not able to recognize the devices.
What am i missing here ?
these are the logs --log-level=8 for EAL logs and I get this output
Sep 09 04:56:50 {"timestamp": 1662717410.8821511, "level": "info", "message": "Initialising EAL ..."}
Sep 09 04:56:50 EAL: Detected 64 lcore(s)
Sep 09 04:56:50 EAL: Detected 2 NUMA nodes
Sep 09 04:56:50 EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
Sep 09 04:56:50 EAL: Selected IOVA mode 'PA'
Sep 09 04:56:50 EAL: No available hugepages reported in hugepages-1048576kB
Sep 09 04:56:50 EAL: Debug dataplane logs available - lower performance
Sep 09 04:56:50 EAL: Debug dataplane logs available - lower performance
Sep 09 04:56:50 EAL: Probing VFIO support...
Sep 09 04:56:50 EAL: VFIO support initialized
Sep 09 04:56:50 EAL: Probing VFIO support...
Sep 09 04:56:50 EAL: VFIO support initialized
also adding the -d parameter with the directory where the plugins are located /usr/local/lib/ gives me an error where the plugin is not found
the eal param initialized is
--master-lcore 4 -l 4,5,6,7 --log-level 8 -d /usr/local/lib/
Sep 09 04:47:29 EAL: Detected 64 lcore(s)
Sep 09 04:47:29 EAL: Detected 2 NUMA nodes
Sep 09 04:47:29 EAL: librte_compressdev.so.0.200: cannot open shared object file: No such file or directory
Sep 09 04:47:29 EAL: FATAL: Cannot init plugins
Sep 09 04:47:29 EAL: Cannot init plugins
the librte_compressdev.so.0.200 is present in the path
ls -l /usr/local/lib/librte_compressdev.so.0.200
-rwxr-xr-x 1 root root 38024 Sep 8 23:51 /usr/local/lib/librte_compressdev.so.0.200

[based on the live debug]
DPDK 17.11 LTS use libdpdk.so and a complete shared library, which makes all the necessary ethdev PMD available. hence using the current makefile, it pans out properly without any issues.
But moving to DPDK 19.11 LTS, libdpdk.so is no longer availble. hence one have to manually link either in shared or static libraries properly (as pointed out in comments). For example
DPDK_CFLAGS=" -DHAVE_DPDK -I${RTE_SDK}/${RTE_TARGET}/include/ -include ${RTE_SDK}/${RTE_TARGET}/include/rte_config.h "
DPDK_LDFLAGS="-I$RTE_SDK/lib/librte_eal/linux/eal/include \
-DRTE_USE_FUNCTION_VERSIONING \
-I$RTE_SDK/$RTE_TARGET/include -include $RTE_SDK/$RTE_TARGET/include/rte_config.h \
-D_GNU_SOURCE -W -Wall -DALLOW_EXPERIMENTAL_API \
-L$RTE_SDK/$RTE_TARGET/lib -Wl,-lrte_flow_classify -Wl,--whole-archive -Wl,-lrte_pipeline -Wl,--no-whole-archive \
-Wl,--whole-archive -Wl,-lrte_table -Wl,--no-whole-archive -Wl,--whole-archive \
-Wl,-lrte_port -Wl,--no-whole-archive -Wl,-lrte_pdump -Wl,-lrte_ip_frag -Wl,-lrte_meter \
-Wl,-lrte_fib -Wl,-lrte_rib -Wl,-lrte_lpm -Wl,-lrte_acl -Wl,-lrte_power -Wl,-lrte_kni \
-Wl,-lrte_efd -Wl,-lrte_bpf -Wl,-lrte_ipsec -Wl,--whole-archive -Wl,-lrte_cfgfile -Wl,-lrte_gro \
-Wl,-lrte_gso -Wl,-lrte_hash -Wl,-lrte_member -Wl,-lrte_vhost -Wl,-lrte_kvargs -Wl,-lrte_mbuf \
-Wl,-lrte_net -Wl,-lrte_ethdev -Wl,-lrte_cryptodev -Wl,-lrte_security -Wl,-lrte_compressdev \
-Wl,-lrte_eventdev -Wl,-lrte_timer -Wl,-lrte_mempool -Wl,-lrte_stack -Wl,-lrte_mempool_ring \
-Wl,-lrte_ring -Wl,-lrte_pci -Wl,-lrte_eal -Wl,-lrte_cmdline -Wl,-lrte_reorder -Wl,-lrte_sched \
-Wl,-lrte_rcu -Wl,-lrte_bus_pci -Wl,-lrte_bus_vdev -Wl,-lrte_mempool_bucket -Wl,-lrte_mempool_stack \
-Wl,-lrte_pmd_af_packet \
-Wl,-lrte_pmd_ixgbe \
-Wl,-lrte_pmd_memif \
-Wl,-lrte_pmd_null \
-Wl,-lpcap -Wl,-lrte_pmd_ring -Wl,-lrte_pmd_tap -Wl,-lrte_pmd_virtio -Wl,-lrte_bus_vmbus \
-Wl,-lcrypto -Wl,-lrte_pmd_null_crypto -Wl,-lrte_pmd_crypto_scheduler -Wl,-lrte_pmd_virtio_crypto \
-Wl,-lrte_pmd_skeleton_event -Wl,-lrte_pmd_sw_event \
-Wl,--no-whole-archive -Wl,-lrt -Wl,-lm -Wl,-lnuma -Wl,-ldl -Wl,-export-dynamic -Wl,-export-dynamic \
-L$RTE_SDK/examples/l2fwd/build/lib -L$RTE_SDK/$RTE_TARGET/lib -Wl,--as-needed"
Since DPDk 19.11 LTS makes uses on meson-ninja, my suggestion and recommendation is to make use of pkg-config to built the right CFLAGS and LDFLAGS. Please refer StackOverflow query on using dpdk cmake too
Note: recommended to build a custom target folder rather than make install, since the built and application run is done in docker.

DL4006 warning: Set the SHELL option -o pipefail before RUN with a pipe in it

I have a Dockerfile
FROM strimzi/kafka:0.20.1-kafka-2.6.0
USER root:root
RUN mkdir -p /opt/kafka/plugins/debezium
# Download, unpack, and place the debezium-connector-postgres folder into the /opt/kafka/plugins/debezium directory
RUN curl -s https://repo1.maven.org/maven2/io/debezium/debezium-connector-postgres/1.7.0.Final/debezium-connector-postgres-1.7.0.Final-plugin.tar.gz | tar xvz --transform 's/debezium-connector-postgres/debezium/' --directory /opt/kafka/plugins/
USER 1001
When I use hadolint on it by
hadolint Dockerfile
I got warning
Dockerfile:6 DL4006 warning: Set the SHELL option -o pipefail before RUN with a pipe in it. If you are using /bin/sh in an alpine image or if your shell is symlinked to busybox then consider explicitly setting your SHELL to /bin/ash, or disable this check
I know I have a pipe | in the line started with RUN.
However, I still really don't know how to fix based on this warning.

Oh, just found the solution in the wiki page at https://github.com/hadolint/hadolint/wiki/DL4006
Here is my fixed version:
FROM strimzi/kafka:0.20.1-kafka-2.6.0
USER root:root
RUN mkdir -p /opt/kafka/plugins/debezium
# Download, unpack, and place the debezium-connector-postgres folder into the /opt/kafka/plugins/debezium directory
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN curl -s https://repo1.maven.org/maven2/io/debezium/debezium-connector-postgres/1.7.0.Final/debezium-connector-postgres-1.7.0.Final-plugin.tar.gz | tar xvz --transform 's/debezium-connector-postgres/debezium/' --directory /opt/kafka/plugins/
USER 1001
The reason adding SHELL ["/bin/bash", "-o", "pipefail", "-c"] is at https://github.com/docker/docker.github.io/blob/master/develop/develop-images/dockerfile_best-practices.md#using-pipes
Below is a copy:
Some RUN commands depend on the ability to pipe the output of one command into another, using the pipe character (|), as in the following example:
RUN wget -O - https://some.site | wc -l > /number
Docker executes these commands using the /bin/sh -c interpreter, which only
evaluates the exit code of the last operation in the pipe to determine success.
In the example above this build step succeeds and produces a new image so long
as the wc -l command succeeds, even if the wget command fails.
If you want the command to fail due to an error at any stage in the pipe,
prepend set -o pipefail && to ensure that an unexpected error prevents the
build from inadvertently succeeding. For example:
RUN set -o pipefail && wget -O - https://some.site | wc -l > /number
Not all shells support the -o pipefail option.
In cases such as the dash shell on
Debian-based images, consider using the exec form of RUN to explicitly
choose a shell that does support the pipefail option. For example:
RUN ["/bin/bash", "-c", "set -o pipefail && wget -O - https://some.site | wc -l > /number"]

mpirun with socwatch fails

On trying to measure energy consumption on my system with an OpenMPI program.
For OpenMPI, we can run with
mpirun -n 4 ./hello
This program will print hello from 4 processes.
Now, If it is a C++ program, we can run SocWatch with it as
sudo socwatch -m -f sys -f wakelock -t 35 -p ./hello
With OpenMPI, I try
sudo socwatch -m -f sys -f wakelock -t 35 -p mpirun -n 4 ./hello
But the program is stropped because socwatch did not recognize to mpi-run.
How to run SoCWatch with MPI

The most likely reason would be mpirun is in your $PATH as a user, but is no more in your $PATH when sudo is used. Try using the absolute path to mpirun instead. Note you will need to pass the --allow-run-as-root option to mpirun.

How to get latest files based on lastmodified using unix

currently I'm using below command to get latest 4 files.
But this command also fetches directories...but i want to exclude directory. How to do?
ls -tr /home/user | tail -4

If you want just the files to show you can issue:
for i in `ls -tr /home/user`; do if test -f $i; then echo $i; fi done | tail -4
-f flag on test checks if argument is a file
-d flag on test checks if argument is a directory

Using a FilteredClassifier with a Bayesian Network in WEKA

I'm running WEKA from the command line to create a Bayesian network model for training, and then using this model to test on a separate data set. The Bayesian network uses the TAN search option with a simple estimator. My command line call for training looks like this:
java -cp weka.jar:mysql-connector-java-5.0.8-bin.jar
weka.classifiers.bayes.BayesNet -d BN_model -p 0 -distribution -t tr2.arff -D
-Q weka.classifiers.bayes.net.search.local.TAN -- -S BAYES
-E weka.classifiers.bayes.net.estimate.SimpleEstimator -- -A 1 > tr_preds
Now, I just realized I need to use a a FilteredClassifier since I want to add instance IDs to my test data. This is where I'm stuck. I run the following:
java -cp weka.jar:mysql-connector-java-5.0.8-bin.jar
weka.classifiers.meta.FilteredClassifier
-F weka.filters.unsupervised.attribute.Remove -R 1
-W weka.classifiers.bayes.BayesNet -d BN_model -p 1 -distribution -t tr2.arff -D
-Q weka.classifiers.bayes.net.search.local.TAN -- -S BAYES
-E weka.classifiers.bayes.net.estimate.SimpleEstimator -- -A 1 > tr_preds
and I get the following error:
Weka exception: Illegal options: -S BAYES
To trouble shoot, I removed all the fancy search options and estimators, and just ran the Bayesian Network classifier with default settings. This worked fine, but it's obviously not what I need. It's clear that I'm having issues constructing this command line statement properly.
Could you please advise what I'm doing wrong? Thanks so much!
Best,
Francisco

The problem is that your command line arguments are in a wrong order. Because -- -S BAYES is the first -- after -W, it thinks this option belongs to weka.classifiers.bayes.BayesNet, which is an illegal option. You forgot the -- after -W ... to specifiy the options of the classifier.
Also, general options like -t tr2.arff -d BN_model -p 1 -distribution were placed somewhere in the middle, it is better to place them after the class you're calling:
java -cp weka.jar:mysql-connector-java-5.0.8-bin.jar
weka.classifiers.meta.FilteredClassifier
-t tr2.arff -d BN_model -p 1 -distribution
-F "weka.filters.unsupervised.attribute.Remove -R 1"
-W weka.classifiers.bayes.BayesNet
-- -D -Q weka.classifiers.bayes.net.search.local.TAN
-- -S BAYES
-E weka.classifiers.bayes.net.estimate.SimpleEstimator
-- -A 1