DPDK l2fwd - How to forward ethernet interface to my PMD - dpdk

I have a board with one ethernet interface (eth0) running Linux.
I'm trying to forward all incoming traffic from eth0 to my PMD driver, using dpdk-l2fwd example application.
Here is what I've tried:
./dpdk-l2fwd -c 0x3 --vdev={my_pmd}0 -- -p 0x3 -T 0
I can see that my rx_pkt_burst callback is polled by the application, but that's it.
How can I forward all incoming eth0 packets to my PMD?
I tried to use net_tap, using the following command:
./dpdk-l2fwd -c 0xff --vdev=net_tap0 --vdev={my_pmd}0 -- -p 0x7 -T 0 --portmap="(1,2)"
And my tx_pkt_burst callback is called occasionally, but not when I think it should be called.
For example, if I ping this board from another one, the ping is successful, but the tx_pkt_burst callback is not been called.
I tried to use devbind tool, but no devices are detected:
./usertools/dpdk-devbind.py --status
No 'Network' devices detected
=============================
No 'Baseband' devices detected
==============================
No 'Crypto' devices detected
============================
No 'Eventdev' devices detected
==============================
No 'Mempool' devices detected
=============================
No 'Compress' devices detected
==============================
No 'Misc (rawdev)' devices detected
===================================
No 'Regex' devices detected
===========================
Update
DPDK version - 20.11.
My HW is a embedded device based on NXP's Layerscape.
$ lshw -class network
*-network
description: Ethernet interface
physical id: 3
logical name: eth0
serial: 00:11:22:44:11:44
size: 1Gbit/s
capacity: 1Gbit/s
capabilities: ethernet physical tp mii 10bt-fd 100bt-fd 1000bt-fd autonegotiation
configuration: autonegotiation=on broadcast=yes driver=fsl_dpaa2_eth driverversion=5.10.35-00002-g3434eea0e1e7-dir duplex=full firmware=7.17 ip=192.168.15.157 link=yes multicast=yes port=twisted pair speed=1Gbit/s
I'm trying to bypass all traffic to the PMD I'm currently developing.
Thanks.

[EDIT-1] clarification of using same interface for DPDK and Kernel routing
Answer> as discussed over comments please refer to DPDKD + kernel on same interface
Based on the information shared there are multiple questions to the single query I'm trying to bypass all traffic to the PMD I'm currently developing. Addressing each one separately below
question 1: using dpdk-l2fwd example application
Answer> DPDK application l2fwd application makes use of basic APi with almost no HW offloads. Based on your environment (I have a board with one ethernet interface (eth0)), the right set of parameters should be -p 0x1 --no-mac-updating -T 1. This will configure the application to receive and transmit packet using single DPDK interface (that is eth0 on your board).
Note: DPDK Application can work with DPDK PMD both physical and virtual
question 2: I tried to use net_tap, using the following command:
Answer> If the intend is to intercept the traffic from physical and then forward to tap interface, then one needs modify the eal arguments as ./build/l2fwd --vdev=net_tap0,iface="my_eth0" -- -p 0x3 -T 1 --no-mac-updating. This will allow the application to probe physical NXP interface (eth0) and make use of Linux TAP interface as secondary interface. Thus any traffic from NXP and TAP will be cross connected such as NXP (eth0) <==> TAP (my_eth0)
question 3: ./usertools/dpdk-devbind.py --status returns empty
Answer> Form the dpdk site supported NIC list NXP dpaa, dpaa2, enetc, enetfec, pfe. Cross checking the kernel driver fsl_dpaa2_eth I think it is safe to assume dpaa2 PMD is supported. As you have mentioned the NIC is not enumerated, it looks like there are certain caveats to such model revision, supported board, BSP package, vendor-sub vendor ID check etc. More details can be found Board Support Package, and DPAA2 NIC guide
Debug & Alternative solutions:
To start with use the Kernel Driver to bring in packets
Use extra logging and debug to identify why the NIC is shown in the application
Approach 1:
Make sure the NIC is bind with kernel driver fsl_dpaa2_eth.
ensure NIC is connected and link is up with ethtool eth0
set to promiscous mode with ifconfig eth0 promisc up
start DPDK application with PCAP PMD, ./build/l2fwd --vdev=net_pcap0,iface=eth0 -- -p 1 --no-mac-updating -T 1
Check packet are received and redirected to PCAP eth0 PMD by checking the statistics.
Approach 2:
Ideally the NIC should be categorized under network device to be probed by debind.py.
Check the device details using lshw -c net -businfo for network.
try checking with lspci -Dvmmnnk [PCIe BUS:Slot:Function id] for network details.
If above details does not show up as network device this might be reason for not getting listed.
Suggestions or workaround: You can try to forcefully bind with igb_uio or vfio-pci (I am not much famialr with NXP SoC) by dpdk-devbind -b vfio-pci [PCIe S:B:F]. Then cross check with lspci -ks [PCIe S:B:F]. Once successfully done, one can start dpdk l2fwd in PMD debug mode with ./build/l2fwd -a [PCIe S:B:F] --log-level=pmd,8 -- -p 1 --no-mac-updating | more. Thus by intercepting and interpreting the logs one can identify what is going
Note:
It is assumed the application is build with static libraries and not dynamic. To build with static libraries use make static for l2fwd.
For the described use case recommended application is basicfwd/skeleton rather than l2fwd.

Found the problem.
I had to unbind eth0 from Linux kernel.
Now I can simply run:
./dpdk-l2fwd -c 0x3 --vdev={MY_PMD}0 -- -p 0x3 -T 1
And all traffic in the physical port is forwarded to my PMD.

Related

Unable to bind intel NIC X710

I am trying to bind this nic I40E:
Ethernet Controller X710 for 10GbE backplane 1581
my OS is ubuntu 18.04
kernel: 4.15.0-74-generic
I used dpdk-setup.sh to Insert VFIO module.
i also add iommu=on to grub file.
running devbind command:
sudo ./dpdk-devbind.py -b vfio-pci 02:00.1
i Got this erros:
Error: bind failed for 0000:02:00.1 - Cannot bind to driver vfio-pci
dmesg output:
[ 5091.393436] vfio-pci: probe of 0000:02:00.1 failed with error -22
There is no issue in binding vfio-pci on Ethernet Controller X710. I have followed the following steps with success
DPDK version: dpdk-20.11
NIC: driverversion=2.1.14-k, firmware=6.01
modprobe vfio-pci
confirm via lsmod
bind with sudo ./usertools/dpdk-devbind.py -b vfio-pci [PCIe B:D:F]
[EDIT-1] as per DPDK documentation workaround for VT-d or iommu is suggested.
note:
If VT-D is not enabled in BIOS and kernel command line (grub option), the work around is to use echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode, then follow the above steps.
if the interface is already used in Kernel - Warning: routing table indicates that interface [PCIe B:D:F] is active. Not modifying. Simply disable the interface with ifconfig [interface name] down and bind again.

"Incompatible hardware version" error when running DPDK on VMWare with VMXNET3 interface

We're trying to run DPDK example apps in a guest machine running Centos 7.5. The host is ESXi version 6.5.
I'm building dpdk on the guest machine where I'm trying to run it. I've tried both DPDK versions 18.05 and 18.08.
We have created five interfaces on esxi for connection to our guest. One management port and four data ports. We're binding theses four data ports to DPDK. The ports are all VMXNET3 interfaces. They are basically setup like the VMXNET3 interfaces in [https://doc.dpdk.org/guides/nics/vmxnet3.html], using a vswitch to connect to a physical interface. However note that we do not have any VF interfaces as shown in this document, only VMXNET3 interfaces. Unfortunately this document does not show any details on how to do the setup.
This document from vmware also shows a very similar setup. But again no details on how to setup.
Fundamentally, the roadblock we are hitting is that the VMXNET3 interfaces are failing initialization when starting the DPDK example app. Here is what we see:
[root#rg-vm ~]# ./dpdk-18.08/examples/packet_ordering/build/packet_ordering -c 0x0e0 -- -p 0xf
EAL: Detected 24 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Probing VFIO support...
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL: Invalid NUMA socket, default to 0
EAL: probe driver: 15ad:7b0 net_vmxnet3
eth_vmxnet3_dev_init(): Incompatible hardware version: 0
EAL: Requested device 0000:04:00.0 cannot be used
We see this for all four interfaces that we are trying to bind to DPDK. However, strangely, sometimes after a reboot, the first two interfaces initialize correctly. But after that first attempt, all four interfaces then fail the same way.
Here are the commands we're using to setup DPDK.
modprobe uio
insmod ./dpdk-18.08/build/build/kernel/linux/igb_uio/igb_uio.ko
./dpdk-18.08/usertools/dpdk-devbind.py --bind=igb_uio 04:00.0
./dpdk-18.08/usertools/dpdk-devbind.py --bind=igb_uio 0c:00.0
./dpdk-18.08/usertools/dpdk-devbind.py --bind=igb_uio 13:00.0
./dpdk-18.08/usertools/dpdk-devbind.py --bind=igb_uio 1b:00.0
Note that we have also tried using the uio_pci_generic with the same results. We have not been able to get the vfio-pci driver to bind to the VMXNET3 interfaces.
I'm not sure it matters, but the physical interfaces on the other side of the vswitch that we're connecting to are:
17:00.0 Ethernet controller: Intel Corporation I350 Gigabit Fiber Network Connection (rev 01)
We have also tried using a Ethernet cards based on the intel 82576 chipset (this is the chipset DPDK shows being used in their documentation), and one based on the Intel X710. We see the same error using either of these cards as we did with the i350. So I think that eliminates the ethernet hardware, which makes sense, as using the vswitch between us and the ethernet controller should make us agnostic to what it actually is.
We are running on a Dell R540. Also note that when we run Centos 7.5 with DPDK on this hardware without VMWare, everything works fine. Also if we run in VMWare, but "passthrough" the i350 interfaces to the VM (instead of using vswitch and vmxnet) everything also works fine in that case.
I've tried updating the kernel (3.10) to the latest (4.18) but still get the same error.
If I try to read the version register (VRRS) (the one that causes this error) in the vmxnet3 pci bar registers (before I bind to DPDK) using ethtool, it looks fine (0xf). I've googled around a lot but can't seem to find much help on this. It is very possible the issue is with how I'm setting things up but I can't find any info that gives details on how else to do it.
Any help would be greatly appreciated. Thanks!
Try these steps:
cd /etc/default
vi grub
Edit GRUB-CMDLINE and Add “nopku”
GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet nopku transparent_hugepage=never log_buf_len=8M"
Recompile grub: sudo grub2-mkconfig -o /boot/grub2/grub.cfg
Reboot the VM and try DPDK.

KGDB remote debugging connection issue via USB and Serial connection

I am having issues with serial and usb connection between host and target. Below is my setup. Both host and target do NOT have any serial (DB9) ports.
Host : Running windows + VMshare + Ubuntu
Target: Running linux kernel 3.19 . Has a MINI usb port which acts as a serial port, i think its ( CP210x uart to usb )
Connection 1 : Host ( USB to DB9 male-PL2303) + DB9 female to female + (DB9 male to USB) target.
Connection 2 : Host ( USB ) --cable-- (USB mini) Target
Host ( ubuntu VM ), can recognize the USB device (both connections types ) as /dev/ttyUSB0. The device does not show up on the windows device manager since VM takes over the device control.
Target boots into UEFI shell. I modify the syslinux.cfg file to append "kgdbwait kgdboc =ttyS0, 115200" to APPEND flag. Save the change ( press F2) then exit ( press F3 ). Boot into the image. Target now enters the kdb prompt with the following message
kgdb: Waiting for connection from remote gdb...
Entering kdb ( current= <64bit address>, pid 1) on processor 0 due to Keyboard Entry
Kgdb > _
on the host side, i do the following commands and below is error
root#ubuntu: cd /images
root#ubuntu: sudo gdb ./vmlinux
Reading symbols from ./vmlinux done.
(gdb)
(gdb) target remote /dev/ttyUSB0
Remote debugging using /dev/ttyUSB0
Ignoring packet error, continuing...
warning: unrecognized item "timeout" in "qSupported" response
Ignoring packet error, continuing...
Ignoring packet error, continuing...
Bogus trace status reply from target: timeout
Experiments i tried
on Host i used " target remote /dev/ttyS0 " , still same issue
Tried different cables in each connection ( 1 and 2 ) mentioned above
on Target removed the edit in the syslinux.cfg file in UEFI shell, booted the image and entered kgdb using "echo g > /proc/sysrq-trigger"
All kernel configuration related to KGDB* , KGDB_SERIAL*, KGDB_USB* are enabled
all available baud rates
Questions
If i use "kgdbwait kgdboc=ttyUSB0, 115200" instead of " kgdbwait kgdboc=ttyS0, 115200" the target does NOT halt. When the target boots up fully to login prompt, i can see the device is recognized as ttyUSB0 when using connection 1. However as it does not stop does that mean, KGDB using USB does not work ? or for USB debug , i need to use direct USB--USB wire ( connection 3 ) ?
does syslinux.cfg support USB debug? becuase there is a SERIAL flag which has value" 0, 115200 " where 0 refers to ttyS0. syslinux documentation does not have any values for USB type device.
using connection 2, why am i seeing the timeout and packet error issues
occasionally with connection 2, when i execute " target remote /dev/ttyUSB0 " on the host, i notice junk characters on the target. So there is some communication happening, so tried different baud rates still same issue. Does this indicate anything inherently wrong with my setup?
In many online forums/ documents i do not see the "entering kdb due to keyboard entry" when the kernel enters kdb prompt. is this unusual?
The setup of remote kgdb debug is a bit tedious. There are several prerequisites/limitations for the kgdb to work. I will try to break it down.
You have to prepare two machines in this setup.
HOST: Where the agent-proxy & GDB installed.
TARGET: The Linux system being debugged.
Connection Setup
[Host /dev/ttyUSB0] USB to Serial --------- COM port [Target /dev/ttyS0]
On the TARGET side, it's not possible to use the USB interface with kgdb. This is because all the USB-Serial driver (CP210x, PL2303, ...etc) did not implement the polling hook. You have to connect the COM port with a serial cable directly. It is ok to use the USB interface on the HOST side. Since it is a serial connection, you have to use a USB-to-Serial converter and install the proper driver on HOST.
Set the proper baud rate on both sides:
[Target] stty -F /dev/ttyS0 115200
[Host] stty -F /dev/ttyUSB0 115200
Make sure the serial connection works on both direction. You can use:
[Host] cat /dev/ttyUSB0
[Target] echo 'from TARGET to HOST' > /dev/ttyS0
[Target] cat /dev/ttyS0
[Host] echo 'from HOST to TARGET' > /dev/ttyUSB0
You should see the messages on both side of machine. If not, there might be some problems on the cable or driver.
Compile Kernel
Enable KGDB* , KGDB_SERIAL*, KGDB_USB*, DEBUG_INFO, DEBUG_INFO_DWARF4, MAGIC_SYSRQ in the kernel config. Compile and install on the TARGET.
The main purpose here is to enable KGDB feature & preserve debug information in vmlinux.
agent-proxy Setup
agent-proxy acts as a proxy for the TARGET's serial port. It splits up the serial port for multiplexing. One for primary console I/O, the other for GDB session. Thus, we can work on both simultaneously. You should run the agent-proxy on HOST machine.
git clone http://git.kernel.org/pub/scm/utils/kernel/kgdb/agent-proxy.git
cd agent-proxy ; make
./agent-proxy 5550^5551 0 /dev/ttyUSB0,115200
This will redirect:
TARGET's console to HOST:5550
TARGET's kgdb listening port to HOST:5551
Start To Debug
First, open the primary console:
[Host] telnet localhost 5550
Entering the kdb mode, either by:
[Target] echo ttyS0,115200 > /sys/module/kgdboc/parameters/kgdboc
[Target] dmesg | tail
(you should see KGDB: Registered I/O driver kgdboc, otherwise it failed)
[Target] echo g >/proc/sysrq-trigger
Or, by adding the following kernel parameters in TARGET's bootloader (for early kernel debug):
console=tty0 console=ttyS0,115200 kgdbwait kgdboc=ttyS0,115200
The TARGET machine will halt immediately once it breaks into the kdb.
At the same time, you will see a kdb prompt on the primary console:
....
Entering kdb (current=0xcb846c80, pid 2301) on processor 3 due to Keyboard Entry
[3]kdb>
Type kgdb then enter. The TARGET is now pending for remote GDB's connection. We will connect it from the HOST.
Host> gdb vmlinux
(gdb) target remote localhost:5551
Remote debugging using localhost:5551
kgdb_breakpoint () at kernel/debug/debug_core.c:1072
1072 wmb(); /* Sync point after breakpoint */
(gdb)
Enjoy your kernel debugging!

Setting up Bluetooth (BLE) communication between Arduino UNO and Laptop running Linux

I have a bluno (arduino uno with a built-in BLE[TI CC2540]) and laptop (ubuntu) with a built-in Bluetooth module.
I want to make a Bleutooth BLE communication program between arduino and linux.
I want to send data from linux to arduino.
Any help would be awesome!
(reference) below operation confirm
$sudo hciconfig
hci0: Type: BR/EDR Bus: USB
BD Address: 6C:71:D9:B1:A5:1A ACL MTU: 1022:8 SCO MTU: 183:5
UP RUNNING PSCAN ISCAN
RX bytes:1786 acl:16 sco:0 events:99 errors:0
TX bytes:1407 acl:12 sco:0 commands:65 errors:0
$ hciconfig hci0 up
$ sudo hcitool lescan
LE Scan ...
D0:39:72:C4:CA:72 (unknown)
D0:39:72:C4:CA:72 Bluno
And where exactly is your problem respectively at which stage is your development currently?
In general:
You have somehow to talk with the bluetooth module on your linux system (Raspberry?). For this you need e.g. API-calls to give and read command from you bluetooth module. If there are no read-to-use APIs then you need to write it yourself, reading the manual of the module and communicating with it using SPI, I2C,...
After a successful hardware-communication with your bluetooth module on the linux system, you have to communicate with the BLE module. For this developing a protocol with cyclic redundancy checks,... would be a good idea, insted of sending plain "chars".

Two-machine GDB debugging between Macs over Ethernet - transaction timed out

I am trying to debug a device driver which is crashing the kernel on a Mac using a remote machine running gdb (trying to follow the instructions here). Both machines are connected to the same network by Ethernet (same router even, and both can access the network). I have also set nvram boot-args="debug=0x144" on the target and restarted.
I then load the kernel extension on the target as usual. On the host machine I start gdb like this:
$ gdb -arch i386 /Volumes/KernelDebugKit/mach_kernel
Once in gdb, I load the kernel macros and set up for remote attachment
(gdb) source /Volumes/KernelDebugKit/kgmacros
(gdb) target remote-kdp
(gdb) kdp-reattach 11.22.33.44
However, the last command then does not make a connection and I get an endless spool of
kdp_reply_wait: error from kdp_receive: receive timeout exceeded
kdp_transaction (remote_connect): transaction timed out
kdp_transaction (remote_connect): re-sending transaction
What is the correct way to get gdb connected to the target machine?
There are a number of ways to break into the target, including:
Kernel panic, as stated in your answer above.
Non-maskable interrupt, which is triggered by the cmd-option-ctrl-shift-esc key combination.
Code a break in your kernel extension using PE_enter_debugger(), which is declared in pexpert/pexpert.h
Halt at boot by setting DB_HALT (0x01) in the NVRAM boot-args value.
Additionally, you may need to set a persistent ARP table entry, as the target is unable to respond to ARP requests while stopped in the debugger. I use the following in my debugger-launch shell script to set the ARP entry if it doesn't already exist:
if !(arp -a -n -i en0 | grep '10\.211\.55\.10[)] at 0:1c:42:d7:29:47 on en0 permanent' > /dev/null) ; then
echo "Adding arp entry"
sudo arp -s 10.211.55.10 00:1c:42:d7:29:47
fi
Someone more expert could probably improve on my bit of shell script.
All of the above is documented in http://developer.apple.com/library/mac/documentation/Darwin/Conceptual/KernelProgramming/KernelProgramming.pdf.
The answer is simply to make sure the target has a kernel panic before you try to attach gdb from the host.