I need to write (c++, linux) rtp analyzer like:http://mmlab.disi.unitn.it/wiki/index.php/Analyzer. But I can't understand some technical moments, such as how server, wich receives udp-datagrams, determines that one udp-datagram had finished and another udp-datagram started and how it identifies udp-datagrams, containing rtp. It would be great to see the code of analyzer, but I haven't find it.
Could you help me to find the answers to my questions (I wonder to know how it can be implemented via sockets). Thank you very much, I will really appreciate it!
If I do understand your question correctly, you can follow the steps stated below:
Write a server program in c++ using one of the socket libraries, and create a UDP socket.
Listen to the port to which your client is sending the RTP packets.
While you listen to the specified UDP port, you will get UDP packets one by one.
When you get a UDP packet parse the length field which is a 2 byte field, and starts at 4th byte.
Length - 8 (UDP header size in bytes) is what you looking for, and actually the size of the RTP packet.
Get the RTP packet by getting the bits between 8th byte of the UDP packet to Length'the byte.
Then you have your RTP packet, you can also parse it in the same way by looking at how it looks like in RFC3550 by IETF.
(Also good to specify that UDP packets might arrive out of order, and you can use the sequence number field in the RTP header, to reorder them)
Related
I have relatively limited network programming experience and was hoping someone could help me with perhaps a n00bish question, thanks in advance for any help! :)
I have two C++ programs running separately. One is a simple packet sniffer written with the libpcap library that is filtering such that it only listens to packets received on a specific port, and then parses the packet such that it writes out the payload length and payload content to a binary file.
The other program is a simple bsd socket server program that is set to receives messages on that specific port with the recv() function, and then similarly writes the received message length and message content out to a binary file.
Everything appears to be running fine, so when I check the binary output files I expect the content to be the same... but unfortunately, I am finding they are only somewhat similar. It seems that the packet sniffer is catching the same data that the server is catching, but it is also recording extra packets with 6-byte long payloads, that apparently aren't being received by the server program.
So, either the packet sniffer is picking up packets that the server isn't supposed to know about, or the server program is missing packets that it's supposed to be receiving. However, I feel somewhat confident that the server program is functioning properly and is receiving the right messages, so these extra packets being detected by the packet sniffer baffle me. Does anyone know what these extra packets may be, and how I might filter them out?
Notes:
The client that is sending data to the ports I'm listening on is an old Windows NT machine in a small network, it's basically passing binary data to the server program for processing.
Unfortunately, I can't upload the code thanks to workplace policies. However, examples of what my code is like may be found here:
http://www.tcpdump.org/pcap.html
http://beej.us/guide/bgnet/output/html/singlepage/bgnet.html
Thanks again!
One is a simple packet sniffer written with the libpcap library that is filtering such that it only listens to packets received on a specific port, and then parses the packet such that it writes out the payload length and payload content to a binary file.
TCP port or UDP port?
If it's a TCP port, your program might receive the 3-way handshake packets and ACK-only packets.
How are you calculating the payload length? You should do it by taking the IP packet's total length, subtracting from it the IP header length (not 20 bytes, but the length from the version/header length field, so that you handle IP options; if that length is < 5, i.e. < 20 bytes, just discard the packet, it's bogus), to get the total TCP packet length, and then subtract from that the TCP header length (not 20 bytes, but the "Data Offset" field, so you handle TCP options; if that length is < 5, i.e. < 20 bytes, just discard the packet, it's bogus) to get the TCP payload length.
That should handle the options to which mfontanini refers, as well as the IP options.
If it's a UDP port, you should still handle IP options and get the total UDP packet length, and then:
if that's less than the length in the UDP header, discard the packet, it's bogus;
if the length in the UDP header is < 8, discard the packet, it's bogus;
otherwise, use the length in the UDP header, minus 8, as the payload length.
The other program is a simple bsd socket server program that is set to receives messages on that specific port with the recv() function, and then similarly writes the received message length and message content out to a binary file.
That won't see the initial handshake or ACK-only packets.
My magic ball indicates those 6 bytes long payloads might be just TCP options. You should skip those options if you only want to work with the payload. Note that some packets might contain both a payload and some options.
The data offset field in the TCP packet indicates at which point the payload starts. You can have a look at that here in section 3.1.
I'd anyway recommend you using some higher level library which deals with that stuff internally. Using libtins you could do that task using some short snippet such as:
#include <tins/tins.h>
using namespace Tins;
bool callback(const PDU &pdu) {
const RawPDU &raw = pdu.rfind_pdu<RawPDU>();
// raw.payload() returns a std::vector<uint8_t>
process_payload(raw.payload());
return true;
}
int main() {
Sniffer some_sniffer(
"eth0",
Sniffer::NON_PROMISC,
"some pcap filter"
);
some_sniffer.sniff_loop(callback);
}
I have a C++ application with GUI that runs (on PC 1) just like a network game, and receives data packets from another computer (2) via WiFi (ad-hoc, so it's quite reliable) at fairly regular intervals (like 40ms), once per loop on program (2). I use send/read.
Here is the problem:
- Packets are not always fully sent (but apparently you can simply keep send()ing the remaining data until all is sent, and thats works well)
- More importantly, packets are stacked in the socket during (1)'s loop until the read() occurs, and then there is no way to distinguish packets in the big stream of data, or know if you were already in the middle of a packet.
I tried to fix this with ID headers (you find an ID as first bytes and you know the length of the packet), but I often get lost (unknown ID : we are not at the beginning of the packet) and am forced to ignore all the remaining data.
So my question is:
Why do packets stack? (generally I have 400B of data whereas my packets are <100B long and fps (1) and (2) are not very different)
How can I have a more reliable way to receive actual packets, say, 80% of packets (discarding packet loss, it's not a question of UDP/TCP)?
Would a separate thread for receiving packets work? (on (1), the server)
How do real-time network games to that (including multiple client management)?
Thanks in advance.
(Sorry I do not have the code here, but I tried to be as clear as I could)
Well:
1) UDP transfers MESSAGES, but is unreliable.
2) TCP transfers BYTE STREAMS, and is reliable.
UDP cannot reliably transfer messages. Anything more reliable requires a protocol on top of UDP.
TCP cannot transfer messages unless they are one byte long. Anything more complex requires a protocol on top of TCP.
Why do packets stack? (generally I have 400B of data whereas my packets are <100B long and fps (1) and (2) are not very different)
Because the time to send packets across the net varies, it typically does not make sense to send packets at a high rate, so most networking libraries (e.g. RakNet) will queue up packets and do a send every 10 ms.
In the case of TCP, there is Nagle's algorithm which is a more principled way of doing the same thing. You can turn Nagle's off by setting the NO_DELAY TCP flag.
How can I have a more reliable way to receive actual packets, say, 80% of packets (discarding packet loss, it's not a question of UDP/TCP)?
If you use TCP, you will receive all of the packets and in the right order. The penalty for using TCP is if a packet is dropped, the packets after it wait until that packet can be resent before they are processed. This results in a noticeable delay, so any games that use TCP have sophisticated prediction techniques to hide this delay and other techniques to smoothly "catch up" once the missing packet arrives.
If you use UDP, you can implement a layer on top that gives you reliability but without the ordering if the order of the packets doesn't matter by sending a counter with each packet and having the receiver repeatedly notify the sender of gaps in the counts. You can also implement ordering by doing something similar. Of course, if you enforce both, then you are creating your own TCP layer. See http://www.jenkinssoftware.com/raknet/manual/reliabilitytypes.html for more details.
What you describe is what would happen if you are using TCP without a protocol on top of it to structure your transmitted data. Your idea of using an ID header and packet length is one such protocol. If you send a 4-byte ID followed by a 4-byte length followed by X number of bytes, then the receiver knows that it has to read 4 bytes followed by 4 bytes followed by X bytes to receive a complete packet. It doesn't get much simplier than that. The fact that you are still having problems reading packets with such a simple protocol suggests that your underlying socket reading code is flawed to begin with. Without seeing your actual code, it is difficult to tell you what you are doing wrong.
I have a tcp socket on which I receive video stream. I want to receive data as packet by packet from socket so that I could remove the packet header and keep the only stream data. How can I do this??
any help will be appreciated.
You can't. TCP doesn't work with packets / messages etc. TCP works with bytes. You get a stream of bytes. The problem is that there's no guarantee reagarding the number of bytes you'll get each time you read from a socket. The usual way to handle this:
When you want to send a "packet" include as the first thing a length
When you read stuff from a socket make sure you read at least that length
Your message could be:
|Message Length:4bytes|Additional header Information:whatever1|Message Data:whatever2|
What you'll then have to do is read 4 bytes and then read as much as those 4 bytes tell you. Then you'll be able to strip the header and get the data.
As others have mentioned, TCP is a streaming protocol. This means from an API point of view, there is no concept of "packet". As a user, all you can expect is a stream of data.
Internally, TCP will break the stream into segments that can be placed into IP packets. These packets will be sent along with control data, over IP, to the remote end. The remote end will receive these IP packets. It may discard certain IP packets (in the case of duplicates), reorder the packets or withhold data until earlier packets have arrived. All this is internal to TCP meaning the concept of a "TCP packet" is meaningless.
You might be able to use raw sockets to receive the raw IP packets but this will mean you will have to reimplement much of the TCP stack (like sending ACKs and adjusting window size) to get the remote end to perform correctly. You do not want to do this.
UDP, on the other hand, is a datagram protocol. This means that the user is made aware of how the data is sent over the network. If the concept of packets or datagrams are important to you, you will need to build your own protocol on top of UDP.
TCP is a stream protocol and it doesn't guaranty that when you call socket read function you will receive one, complete packet. UDP or SCTP are packet oriented protocols and guaranty this. For TCP you can get part of the packet or few packet at once. You have to build your own application protocol on top of TCP and fragment/defragment messages manually.
TCP is a streaming protocol. You get bytes with no message boundaries. The solution is to buffer all your reads and extract/process full video packets from the buffer.
Algorithm:
Initialize an empty buffer.
Examine buffer for a complete packet.
If found, remove complete packet from beginning of buffer and process it.
If not found, append data from a recv() to the buffer and go to #2.
What a "complete packet" contains should be defined by the video streaming protocol.
Are you pretty sure about this approach? In my opinion these "preprocessing" will introduce an additional overhead to the system. And of course this is handled by a lower layer (Read about OSI model) so it is not easy to change. Note that most of the existing streaming protocols are already optimized for the best performance.
I am using SDL and Net2 lib for a client-server application.
The problem I am facing is that I am not receiving all of my TCP packets from my client unless I place a delay before sending each packet from client.
Removing the delay I get only one packet.
A TCP connection is a stream of bytes. Your client could send 20 packets of 5 bytes each, and the server read it as one 100-byte sequence. You'll need to split the data up yourself.
Well you're not guaranteed (in regular sockets) to receive all packets at one time, you may have to call your receive function more than once, to receive all data. This is of course depends on your definition of a "packet" are you receiving all of your data?
+1 erik
Although it is not guaranteed to be reliable, you most likely want to use UDP, not TCP. Net2 handles UDP very well. UDP is actually very reliable. UDP is message oriented. UDP messages tend to get sent quickly and get special treatment by routers (not always a good thing :-). UDP is often used in games.
BTW, if you had asked this question on the SDL mailing list, or sent it to me directly, you would have gotten this advice many months ago.
I wrote Net2 and I hang out on the SDL list. I do not hang out here because this place is an infinite time sink.
Bob Pendleton
So I'm almost done an assignment involving Win32 programming and sockets, but I have to generate and analyze some statistics about the transfers. The only part I'm having trouble with is how to figure out the number of packets that were sent to the server from the client.
The data sent can be variable-length, so I can't just divide the total bytes received by a #define'd value.
We have to use asynchronous calls to do everything, so I've been trying to increment a counter with every FD_READ message I get for the server's socket. However, because I have to be able to accept a potentially large file size, I have to call recv/recvfrom with a buffer size around 64k. If I send a small packet (a-z), there are no problems. But if I send a string of 1024 characters 10x, the server reports 2 or 3 packets received, but 0% data loss in terms of bytes sent/received.
Any idea how to get the number of packets?
Thanks in advance :)
This really boils down to what you mean by 'packet.'
As you are probably aware, when a TCP/UDP message is sent on the wire, the data being sent is 'wrapped,' or prepended, with a corresponding TCP/UDP header. This is then 'wrapped' in an IP header, which is in turn 'wrapped' in an Ethernet frame. You can see this breakout if you use a sniffing package like Wireshark.
The point is this. When I hear the term 'packet,' I think of data at the IP level. IP data is truly packetized on the wire, so packet counts make sense when talking about IP. However, if you're using regular sockets to send and receive your data, the IP headers, as well as the TCP/UDP headers, are stripped off, i.e., you don't get this information from the socket. And without that information, it is impossible to determine the number of 'packets' (again, I'm thinking IP) that were transmitted.
You could do what others are suggesting by adding your own header with a length and a counter. This information will help you accurately size your receive buffers, but it won't help you determine the number of packets (again, IP...), especially if you're doing TCP.
If you want to accurately determine the number of packets using Winsock sockets, I would suggest creating a 'raw' socket as suggested here. This socket will collect all IP traffic seen by your local NIC. Use the IP and TCP/UDP headers to filter the data based on your client and server sockets, i.e., IP addresses and port numbers. This will give an accurate picture of how many IP packets were actually used to transmit your data.
Not a direct answer to your question but rather a suggestion for a different solution.
What if you send a length-descriptor in front of the data you want to transfer? That way you can already allocate the correct buffer size (not too much, not too little) on the client and also check if there were any losses when the transfer is over.
With TCP you should have no problem at all because the protocol itself handles the error-free transmission or otherwise you should get a meaningful error.
Maybe with UDP you could also split up your transfer into fixed-size chunks with a propper sequence-id. You'd have to accumulate all incoming packages before you sort them (UDP makes no guarantee on the receive-order) and paste the data together.
On the other hand you should think about it if it is really necessary to support UDP as there is quite some manual overhead if you want to get that protocol error-safe... (see the Wikipedia Article on TCP for a list of the problems to get around)
Do your packets have a fixed header, or are you allowed to define your own. If you can define your own, include a packet counter in the header, along with the length. You'll have to keep a running total that accounts for rollover in your counter, but this will ensure you're counting packets sent, rather than packets received. For an simple assignment, you probably won't be encountering loss (with UDP, obviously) but if you were, a packet counter would make sure your statistics reflected the sent message accurately.