I am developing a client program which will display the media captured from IP camera. So I want to whether the RTP packets using RTSP protocol contain both audio and video if contains both how should I extract it?
RTSP stream does not carry video/audio itself, it provides a method to control independent RTP video and audio streams (they are in turn independent one from another).
One of the options though is when RTP streams are tunnelled through RTSP connection, in which case all communication might be taking place through single TCP connection.
You can read the SDP returned in the SETUP request to the RtspServer.
There should be a MediaInformation for each stream available.
That will tell you if there is audio or video etc...
http://en.wikipedia.org/wiki/Session_Description_Protocol
Related
I'm trying to figure out how RTSP works, when the handshake is complete.
Does the client have to query the server for each new piece of data? Or the server sends data all the time and doesn't care how the client receives the data?
The reason why I ask this is my Gstreamer pipeline receiving RTSP stream from IP camera:
uridecodebin -> nvstreammux -> queue1 -> nvinfer -> queue2 -> fakesink
The IP camera has 30FPS but the nvinfer element can process only 10FPS. So I assumed that pending frames are stored in queue1 element waiting to be processed. However the current number of buffers of the queue1 is 1 all the time.
So one possible answer is that frames or packets are queued in one of the elements of uridecodebin element, but I did not find any such element there. Next it can mean that the IP camera was informed by uridecodebin to decrease FPS. Or if the uridecodebin element has to ask for each new piece of data, it just means that it asks for new data only when all frames are processed in the pipeline.
Do you have any idea?
First it is worth mentioning that RTSP is a control protocol - the actual video media is usually sent over RTP protocol in most 'RTSP' video streaming cases.
RTSP is a streaming control protocol, to control streaming servers (whoami's remote control analogy here is nice: https://stackoverflow.com/a/43045354/334402) It defines how to package data to stream it and how both ends of the connection should behave to support the protocol.
So RTSP does not actually transport the media data itself - as mentioned above it is usually RTP (Real Time Transport) that does this.
Back to your question - RTCP and RTP can be transferred over UDP or TCP in IP Networks. In the former case, a socket is opened a stream established and the client can sit back and wait for the packets to arrive. In the latter case, the client requests each packet as you outline.
In practice you will likely find TCP is more commonly used, even if it is not necessarily the best match for streamed video, because it is more suited to traversing the many firewalls and NAT's on the internet.
In fact it gets more complicated sometimes with the RTSP data interleaved with RTP and RTCP (Real Tine Control Protocol that collects statistics about the flow) packets over TCP - you can see most detail by looking at the IEFT RFC: https://datatracker.ietf.org/doc/html/rfc2326
Established: I am currently streaming data packets(.mp3 files) from source(server) to sink(client A) using gstreamer over RTP. This thing is pretty easy and I am successfully able to stream music over a network from server to device.
Requirement: Now, I want to retransmit the data packets in real time(or atleast as close to real time as possible) from the client A to say any other client B . Hence the control would still remain with client A and only the music would now be actually streaming with client B.
What is the most optimum way to do such a thing.
Assuming you want to quickly try this. The ideal way is to setup a rtsp server on client A which can forward client B data. If you want to do it your way here is one way to do it:
If you have a player in client B which can play a rtp stream given a sdp file this is what you can do:
Create a copy of the sdp you get in client A. Give it to client B via some path. [Say tcp socket that both agree to communicate upon]
Stream a copy of what you get in client A to client B as well.
You need to change port numbers in sdp to be the one available at client B [This what RTSP negotiation does]. If the client B can tell client A the port number before it gets the SDP, great, you can then set the port number in the sdp correctly, give it to client B and then send a copy of he stream to client B. you are done.
you also uset RTP over TCP.
So it would be easy for you, as no changes require at RTP level.
You need to pass packet to RTSP instead of sending directly over UDP socket.
everything will be handle by RTSP.
If you are using RTP as a stand alone ,accoeding to rfc 3550 , its not a part of RTP
Still you wan't to do so, then you are not following standard way of RTP implementation.
I would like to develop a very tiny and small RTSP client to get the video stream from network cameras. Does anybody know where can I find a simple explanation of the protocol and some good examples?
Best regards,
You connect to the camera via RTSP protocol to query its capabilities, identify streams and prepare/start transmission.
RFC 2326 - Real Time Streaming Protocol (RTSP)
As a part of initialization and handshaking, you will discover available streams.
RFC 4566 - SDP Session Description Protocol
Then you will set up RTP session(s) to receive data, over UDP or sharing the same TCP connection.
RFC 3550 - RTP A Transport Protocol for Real-Time Applications
RFC 4571 - Framing Real-time Transport Protocol (RTP) and RTP Control Protocol (RTCP) Packets over Connection-Oriented Trans
To decode media streams you will convert the payload into pure data you need for further processing. With IP cameras your primary interest is perhaps MPEG-4 AVC (H.264):
RFC 3984 - RTP Payload Format for H.264 Video
RFC 6184 - RTP Payload Format for H.264 Video
This looks like some (introductory) reading.
Try GStreammer library. It is modular, wery flexible library, which can be used for streamming (both client and server). Just check the docs and pick right plugins.
GStreammer could be used in two ways: as a commandline tool or as a library in your project, depending on your requirements.
I got a pcap file with voip conversation, how i can separate RTP packets from the other packets?
I can obtain sip packets, but I can't differenciate the RTP packets from the rest.
Search for RTP headers as defined in RFC3550 within your file. Or better use pcap-filter, for instance with this wiki (look for "Q: What is a good filter for just capturing SIP and RTP packets?").
If you want to see the RTP traffic in wireshark then:
Select Analyze->Display Filters...
Select "UDP", OK
Right click on any UDP packet and select "Decode as..."
Select "RTP" from the list, OK
Now you can see all RTP packets.
Hope that helps. :)
p.s. edited to note that this is for Wireshark. Thanks to a commentor for pointing that out!
Check #macs recommendation about PCap filter. If this cannot satisfy your needs (e.g. you need to filter out RTP packets of specific SIP session) there's no simple way. You need to parse SIP messages, retrieve RTP port numbers, takes packets going to/from these ports in particular time period and (optionally) check if these packets are RTP by checking their headers (magic number in headers)
An open source software that extract the RTP/RTCP packets from a pcap file are:
rtpbreak: http://dallachiesa.com/code/rtpbreak/doc/rtpbreak_en.html
xplico: http://www.xplico.org
From the source code you can view and understand the methodologies used.
I can obtain sip packets, but I can't differenciate the RTP packets
from the rest.
If you are able to decode the SIP, then you can find (inside INVITE message) the SDP message. If you decode it you can find the IP and PORT of RTP "stream" (and RTCP => port + 1). With these informations you can identify uniquely the RTP and RTCP packets. Keep in mind that there are often packages (with the same IP-PORT) with the STUN protocol which must be separate from RTP.
You have to consider where is the packet capture (network context and constraints), you may take into account NAT.
I've compiled with VS the live555 source code, and it works just fine if I try to stream locally a file
e.g.
Command Line:
live555.exe myfile.mp3
VLC Connection String
rtsp://169.254.1.231:8554/myfile.mp3
but if I try to stream it over the internet, VLC communicates with live555, but live555 won't send data to him
Command Line
live555.exe myfile.mp3
VLC Connection String
rtsp://80.223.43.123:8554/myfile.mp3
I've already forwarded the 8554 port (both tcp/udp) and tried to disable my firewall but this doesn't solve.
How is that?
To troubleshoot:
Are you streaming RTP over RTSP: have you checked the "Use RTP over RTSP (TCP)" option in VLC? You can check this in VLC under the preferences: input/codecs->Demuxers->RTP/RTSP. You can try to see if this solves the problem in which case it could be that UDP is blocked.
You speak of forwarding. Do you mean port forwarding from one machine to the RTSP server? if so-> if you are not doing RTP over RTSP, then you would also need to forward the ports for the media which is not the same as the RTSP port (554 or 8554). These ports are exchanged during the RTSP SETUP. If you do RTP over RTSP the media is interleaved over 554 or 8554 and you don't have to worry about this.
Also, another good debugging tool is the live555 openRTSP application. You can run it from the command line and specify "-t" for RTP over RTSP, which is basically what the VLC option does. You can specify "-T" for HTTP tunneling, etc and it allows you to write captured media packets to file, etc.