Working on parsing Arp packets and I found this nice problem.
when receiving an Arp packet I was parsing the target's IP address.
I have c0 a8 in my hex dumb but after that it ends. I am missing data! I see the data in Wireshark but I am not getting the data through WinPCap.
I have yet to run into this issue before. Any ideas SO? So far no memory access errors though. Probably just luck. :x
EDIT:
My main look for processing packets is from the example pktdump_ex.
Here is the while line
while((res = pcap_next_ex( fp, &header, &pkt_data)) >= 0)
After that is executed, the snalen is 2b.
As noted in he comment, this smells like a faulty snaplen configuration. If you look at the winpcap api docs pcap_open() apidoc, it states:
snaplen,: length of the packet that has to be retained. For each packet received by the filter, only the first 'snaplen' bytes are stored in the buffer and passed to the user application. For instance, snaplen equal to 100 means that only the first 100 bytes of each packet are stored.
As explanation for the second parameter of pcap_open. Unless you provide some more detailed code snippets to work with, this is the closest to an answer we will get.
Related
I am trying to programmatically track the amount of data in my receive buffer. I am receiving UDP data. After doing some research it seems that the only way to do this in Linux is to look at /proc/net/udp. This seems like a good solution until I realized that two applications could be listening to the same multicast group and I need to tell them apart. It seems that I am supposed to do this by determining what my inode is.
I spend some time looking into this and there are suggestions that sockfd_lookup or sock_from_file is the way to go but on my CentOS Linux machine, these functions do not seem to be available.
Can someone please help me to figure out which line in /proc/net/udp belongs to my application?
I started using the ioctl (handle, FIONREAD, &bytesInBuffer) call only to discover that in Linux this only returns the size of the first datagram packet in the buffer.
Google seems to suggest that the sockfd_lookup call can be used to get the inode but a grep in my /usr/local/include/ does not return these functions.
My linux/net.h seems pretty bare-bone compared to some I can find on google which includes structs like "socket" which has the sock member which I believe has the inode information. My linux/net.h on CentOs only is 58 lines long and has only a few #defines and an enum.
after a bit of fiddling I noticed that readlink("/proc/self/fd/$fd") (under Linux 5.3) gives me something like:
socket:[3753088]
back. I can parse this and use the resulting digits to look up the relevant line in /proc/net/udp:
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
2867: 00000000:0BB8 00000000:0000 07 00000000:00000000 00:00000000 00000000 1000 0 3753088 2 000000003ae8e911 0
that said, I don't understand why you'd want to do this, but never mind!
I'm also not sure why you don't just look up by sock&peer name, which might be easier
first of all a little background on my situation:
- Qt/C++ UI desktop application
- embedded device (Stm32l4xx family) +ATWINC1500 wifi module
I'm developing the gui application in order to send commands and files to the emdedded device via sockets.
For simple commands I've done all successfully, but for sending files (text files in GCODE format) I am stuck with some issues.
The embedded device has already a socket management(not written by me, so I have not the possibility to modify the way sockets are managed, coming from third party company), and the reception of that type of files is managed in a way that the API waits for every single line of the file being sent, and then wrotes it into a reserved portion of the flash.
My problem is that when I send file from qt Application(by reading each line and and calling write() on the line, in reality my socket sends an entire chunk of the file, like 50 lines, resulting in my device not managing the file reception.
My sending code is this:
void sendGCODE(const QString fileName)
{
QFile *file = new QFile(fileName,this);
bool result = true;
if (file->open(QIODevice::ReadOnly))
{
while (!file->atEnd())
{
QByteArray bytes(file->readLine());
result = communicationSocket->write(bytes);
communicationSocket->flush();
if(result)
{
console->append("-> GCODE line sent:"+ QString(bytes));
}
else
{
console->append("-> Error sending GCODE line!");
}
}
file->close();
}
}
Have anyone of you guys any hints on what I am doing wrong?
I've already searched and someone suggests on other topic that for this purpose it should be better to use UDP instead of TCP sockets, but unfortunately I cannot touch the embedded-device-side code.
thank you all!
EDIT
After suggestions from comments, I've sniffed tcp packets and the packets are sent correctly(i.e. each packet contains a single line). BUT... at the receiver(device), I understood that there is something regarding memory which is not well managed. an example:
sender sends the line "G1 X470.492 Y599.623 F1000" ; receiver receives correctly the string "G1 X470.492 Y599.623 F1000"
next, if the line length is less than the previous sent, i.e. sending "G1 Z5", the receiver receives: "G1 Z5\n\n.492 Y599.623 F1000", so it is clear that the buffer used to store the data packet is not re-initialized from previous packet content, and the new part overwrites the previous values where the remaining part is from the previous packet
I'm trying to figure out how I could reset that part of memory.
This is all wrong. TCP is not a message-oriented protocol. There is no way to ensure that the TCP packets contain any particular amount of data. The receiver code on the device mustn't expect that either - you perhaps misunderstood the receiver's code, or are otherwise doing something wrong (or the vendor is). What the receiver must do is wait for a packet, add the packet's data to a buffer, then extract and process as many complete lines as it can, then move the remaining data to the beginning of the buffer. And repeat that on every packet.
Thus you're looking for the wrong problem at the wrong place, unless your device never ever had a chance of working. If that device works OK with other software, then your "packetized" TCP assumption doesn't hold any water.
Here's how to proceed:
If the device is commercially available and has been tested to work, then you're looking in the wrong place.
If the device is a new product and still in development, then someone somewhere did something particularly stupid and you either need to fix that stupidity, or have the vendor fix it, or hire a consultant to fix it. But just to be completely clear: that's not how TCP works, and you cannot just accept that "it's how it is".
I'm trying to write some code to read from an Isochronous pipe using LibUsbK in Win32. I have successfully initialised the device into the correct state to send and receive Isochronous data and I can see data being sent over the USB in my hardware USB analyser, but the buffers I am receiving are always unfilled even though the analyser shows that there was data in the packets sent to the PC.
I'm new to LibUsbK and using Isochronous transfers though I'm not new to USB in general but I've been really struggling with this.
The code I'm using to read from the device is something like this...
UsbK_SelectInterface(usbHandle,1,0);
UsbK_SetAltInterface(usbHandle,1,0,1);
IsoK_Init(&isoCtx, ISO_PACKETS_PER_XFER, 0);
IsoK_SetPackets(isoCtx, ISO_PACKET_SIZE); // Size of each individual packet
OvlK_Init(&ovlPool, usbHandle, 4, 0);
OvlK_ResetPipe(usbHandle, 0x83);
OclK_Acquire(&ovlkHandle, ovlPool);
UsbK_IsoReadPipe(usbHandle, 0x83, inBuffer, sizeof(inBuffer), ovlkHandle, isoCtx);
while(!finished)
{
if(OvlK_IsComplete(ovlkHandle)
{
fwrite(inBuffer, sizeof(inBuffer), 1, outFile);
memset(inBuffer,0xcc,sizeof(inBuffer));
OvlK_ReUse(ovlkHandle);
UsbK_IsoReadPipe(usbHandle, 0x83, inBuffer, sizeof(inBuffer), ovlkHandle, isoCtx);
{
}
If I put a breakpoint at the fwrite line then the inBuffer is always full of 0xCC - ie, not having been filled by the iso read.
I've checked all the error return values from the UsbK/OvlK function calls and they are all as they should be. I've checked my buffers are sufficiently big to receive the data.
I use very similar code to write to the ISO out pipe on endpoint 0x02 and that works perfectly, the only difference really between the code above and my write code is that the fwrite/memset commands are replaced with a call to a "fillbuffer" function that populates my outBuffer before calling UsbK_IsoWritePipe function.
I tried looking through any examples I could find in the samples and also online but struggled to understand/get them to work with my particular device.
Any suggestions or help greatly appreciated.
So it appears that the above code did work and I was being mislead by the fact that the debugger was interrupting the flow of things - I keep forgetting that trying to debug real time stuff can introduce it's own issues.
The first issue was that stepping through the code in the debugger was causing issues with the low level libusbk code capturing the usb packets and filling my buffers correctly - once I let it run full speed and found other ways to test the buffers I did actually find there was some data in there.
The second problem I had was that quite often the buffer was starting to be filled part way through only (and not always right from the start) so when I examined the data I was only printing the first part of the buffer to the console and so all I saw was 0xCC and I was therefore assuming it hadn't worked.
Once I realised that there was actually some data later in the buffer I just started looking through the buffer in packet sized chuncks, if the packet was completely contained of 0xCC I would skip it and move on, but if any of it was not 0xCC then I would treat it as a valid packet - this worked perfectly and I was successfully receiving all the data. I'm sure there's a more "proper" way of doing this, but it works for me now.
Our server is seemingly packet based. It is an adaptation from an old serial based system. It has been added, modified, re-built, etc over the years. Since TCP is a stream protocol and not a packet protocol, sometimes the packets get broken up. The ServerSocket is designed in such a way that when the Client sends data, part of the data contains the size of our message such as 55. Sometimes these packets are split into multiple pieces. They arrive in order but since we do not know how the messages will be split, our server sometimes does not know how to identify the split message.
So, having given you the background information. What is the best method to rebuild the packets as they come in if they are split? We are using C++ Builder 5 (yes I know, old IDE but this is all we can work with at the moment. ALOT of work to re-design in .NET or newer technology).
TCP guarantees that the data will arrive in the same order it was sent.
That beeing said, you can just append all the incoming data to a buffer. Then check if your buffer contains one or more packets, and remove them from the buffer, keeping all the remaining data into the buffer for future check.
This, of course, suppose that your packets have some header that indicates the size of the following data.
Lets consider packets have the following structure:
[LEN] X X X...
Where LEN is the size of the data and each X is an byte.
If you receive:
4 X X X
[--1--]
The packet is not complete, you can leave it in the buffer. Then, other data arrives, you just append it to the buffer:
4 X X X X 3 X X X
[---2---]
You then have 2 complete messages that you can easily parse.
If you do it, don't forget to send any length in a host-independant form (ntohs and ntohl can help).
This is often accomplished by prefixing messages with a one or two-byte length value which, like you said, gives the length of the remaining data. If I've understood you correctly, you're sending this as plain text (i.e., '5', '5') and this might get split up. Since you don't know the length of a decimal number, it's somewhat ambiguous. If you absolutely need to go with plain text, perhaps you could encode the length as a 16-bit hex value, i.e.:
00ff <255 bytes data>
000a <10 bytes data>
This way, the length of the size header is fixed to 4 bytes and can be used as a minimum read length when receiving on the socket.
Edit: Perhaps I misunderstood -- if reading the length value isn't a problem, deal with splits by concatenating incoming data to a string, byte buffer, or whatever until its length is equal to the value you read in the beginning. TCP will take care of the rest.
Take extra precautions to make sure that you can't get stuck in a blocking read state should the client not send a complete message. For example, say you receive the length header, and start a loop that keeps reading through blocking recv() calls until the buffer is filled. If a malicious client intentionally stops sending data, your server might be locked until the client either disconnects, or starts sending.
I would have a function called readBytes or something that takes a buffer and a length parameter and reads until that many bytes have been read. You'll need to capture the number of bytes actually read and if it's less than the number you're expecting, advance your buffer pointer and read the rest. Keep looping until you've read them all.
Then call this function once for the header (containing the length), assuming that the header is a fixed length. Once you have the length of the actual data, call this function again.
I want to hold onto packets that fail the crc check. To do this I have commented out a section of the CC2420RecieveP readDone function that checks the msb bit of the LQI byte in the received buffer.
I think this is working,
However, once I receive the packet in my own receive function I send it through the serial component (not just the payload, I copy the whole received radio packet into the payload area of the serial packet).
When I use the program Listen, it seems that the crc bool value is not there (only the LQI and RSSI) even though the crc is clearly copied into the bufPTR in the function receiveDone_task.
:(
Help!
Mike.
i was only copying the first 28 bytes (not the header plus a 28 byte payload plus the metadata)
:P