Is it possible to assign priority on multiple rx queue on a single core - DPDK? - dpdk

Question

Hi I'm relatively new to DPDK. I want to know if its possible to assign more than 1 Rx-queues per core on DPDK.
But here is more about the question
1'st queue always takes priority and will be used for main packet processing (Rx -> decode -> do_some_oper -> Tx)
2nd queue should only be used if there are no more mbuf's left in the 1'st queue. (Rx -> save_5_tuple_info ->drop)
Packet must be first tried on 1st queue if it fails put in the 2nd queue.
So basically my end goal is to capture few important details (5 tuples and few other) which would rather be dropped since the queue is full.

Answer 1

#RoninGoda there are multiple questions and sub-sections. So let me try explain step by step
[Question-1] if its possible to assign more than 1 Rx-queues per core on DPDK.
[Answer] Yes, If a DPDK port supports multiple queues and one has configured and started multiple queues; a single DPDK logical core can query all the queues.
[Question-2] But here is more about the question
1'st queue always takes priority and will be used for main packet processing (Rx -> decode -> do_some_oper -> Tx)
2nd queue should only be used if there are no more mbuf's left in the 1'st queue. (Rx -> save_5_tuple_info ->drop)
Packet must be first tried on 1st queue if it fails put in the 2nd queue.
[Answer]
First part: here is where some amount of clarity is required. Whether it is a physical NIC (PF or VF) or vdev NIC supporting multiple queues, one needs to either enable RSS or RTE_FLOW or use external flow director to program the NIC to distribute packets on Queue-0 and Queue-1.
If the above mechanism is not configured all packets will fall on default queue-0. hence the rx_burst will always return n packets as long the traffic flows.
Second part: Assuming RSS or RTE_FLOW or Flow Distributor is enabled for queue-0 and queue-1, there can be the following 4 scenarios
queue-0 has packets, queue-1 has no packets
queue-0 has no packets, queue-1 has no packets
queue-0 has packets, queue-1 has packets
queue-0 has no packets, queue-1 has no packets
For scenario 1 your current approach always processes the packets and tx_burst as you have shared. For scenario 2 since packets are not there in queue-0, queue-1 packets will be fetched, accounted and drop.
Problem scenario is 3 because when queue-0 & queue-1 has continuous packets and your logic always processes and tx packets from queue-0 this will run continuously. While packets on queue-1 start accumulating the RX descriptors (that packets DMA to host memory) run out. Once there are no more RX descriptors for the queue, packets destined for queue -1 will get dropped in the NIC. Hence you will start losing packets in the NIC itself.
Scenario 4 is not an issue since there are no packets in both queues.
note: packets will also get accumulated in queue-0 if the processing thread is consuming more cycles to process and tx_burst too. There can be also scenarios in which mbuf can run out leading to starvation on both queue-0 and queue-1 too.
[Recommended solution to avoid drops]
check if DPDK NIC supports multiple mempool for each queue. If yes, set queue-0 with a dedicated mbuf with a larger element and queue-1 with a mempool with smaller elements.
set the RX descriptor for queue-0 to a larger value like 2048 or 4096 allowing more accumulation, while set queue-2 RX descriptor as 512 or 1024.
if the processing of queue-0 is heavy, try using tx_buffer rather than tx_burst to amortize the cost to PCIe writes.
If NIC PMD parses and stores the l2, l3, l4 header in rte_mbuf make use of the same rather than parsing the 5 tuples from queue-1 to save the CPU cycles.
Important change: rather than having a fixed loop
example:
burst1 = q0 rx_burst
burst2 = q0 rx_burst
burst3 = q0 rx_burst
if (burst1 ==0 && burst2 ==0 && burst3 ==0); then
burst = q1 rx_burst
have packet counter driven approach
portqueue_to_rxburst[4] = {0,0,0,0} /* global array*/
for (index =0; index < 4; index)
{
burst = q-portqueue_to_rxburst[index] rx_burst
if (unlikely(portqueue_to_rxburst[index] == 1) /* drop logic*/
{
....
continue
}
/* process queue-0 packets*/
}
use an external timer thread or service thread to get_xstats | get_Stats from the port and check the queue stats*. if packets are arriving on queue-1 update global array index 3 to queue-1*.