Okay I got the following problem:
I have an IP Camera which is able to stream MPEG4 data over RTP
I am able to connect to this camera via RTSP
I can receive the raw RTP data.
So what problems do I have now?
1. Extract Data
What is the data I actually want? I know that I have to trunkate the RTP Header - but is there anything else I need to cut from the RTP packets?
2. Packetization Mode
I read that I should expect a field Packetization Mode in my SDP- well it's not there. Does that mean I have to assume some kind of standard packetization mode?
3. Depacketization
If I got it right I need to buffer all incoming frames with the Marker Bit = false until I get a frame with Marker Bit = true to get a complete MPEG4 Frame. What exactly do I have to understand by MPEG4 Frame? Keyframe + data until next keyframe?
4. Decode
Do I have the decode the data any further then? In other threads I saw that people used another decoder - but what is there left to decode? I mean the camera should send the data already MPEG4 coded?
5. Libraries
If I really need to decode the data, are there any open libraries I could use for that? Or maybe there is even a library which has some functions where I can just dump my RTP data and then magic happens and I get my mp4. ( But I assume there will be nothing like that .. )
Note: Everything I want to do should be part of my own application, meaning for example, I can't use an external software to parse the data.
Well long story short - I'd really need some kind of step by step explanation for this to do. I know this is a broad question but I don't know any further. I also looked into the RFCs, but I couldnt extract much information out of them.
Also I already looked up these two Questions:
How to process raw UDP packets so that they can be decoded by a decoder filter in a directshow source filter
MPEG4 extract from RTP payload
But also the long answer from the first question could not make everything clear to me.
UPDATE: Well I informed a bit further and now I don't know where to look anymore. It seems that all the packetization stuff etc. is actually not needed for my purpose. I also recorded a stream with openRTSP. When I open those files in a Hex-Editor I see that there are 16 Bytes which I can't identify, followed by the config part of the SDP. Then the frame starts with the usual 00 00 01 B6. Also oprenRTSP adds some kind of tail to the MP4 - well I actually don't know what I need and whats just some "extra" stuff which isn't mandatory.
I know that I have to trunkate the RTP Header - but is there anything
else I need to cut from the RTP packets?
RTP packet might have stuff data from a file format (such as MP4) or it could have directly based on RFC 3640 or something similar. You need to find that out.
What exactly do I have to understand by MPEG4 Frame? Keyframe + data
until next keyframe? Do I have the decode the data any further then?
In other threads I saw that people used another decoder - but what is
there left to decode? I mean the camera should send the data already
MPEG4 coded?
You should explore basics of MPEG compression to appreciate this fully. The depacketization only give you a string of bits. This is compressed data. You need to uncompress it (decode it) to see it on the screen.
are there any open libraries I could use for that?
try ffmpeg or MPEG4IP
Related
I’m in search for a solution that makes it possible to read the Timing Reference Signal (TRS) and Ancillary data (HANC and VANC) of a serial digital video. The TRS gives information about the start and the end of active video (SAV/EAV), the Ancillary data gives, for example, information about embedded audio. With this I want to code an application that analyzes the data that is transported in the non-picture area of serial video.
I read much about GStreamer and found with GstVideo Ancillary a collection that makes it possible to handle the Ancillary Data of a video.
Unfortunately, it’s not clear to me, how this collection works. For me, it looks like that this collection can only construct Ancillary data for a video and it’s not possible to read ancillary data from a detected videostream.
Another idea is to read the whole video stream and display, in a first step, the data words of the stream. TRS and ANC packets have to start with a special sequence of identifiers that makes it possible to localize them. Is GStreamer for this the right choice? Are there better recommended libraries for this task?
I want to be able to break a song into packets and have access to these individual packets.
The reason for that is that I want to send each individual packet over the network using an experimental network protocol called Named Data Network.
As the packets arrive at the destination I want to play them. So I want to implement a streaming functionality. The only difference is the network layer that I will use. This network layer is not based on IP.
Does anyone know any C/C++ implementation of breaking a song file into pieces and then playing these packets individually? I looked over Gstreamer, but it seems complicated to get individual packets from its pipeline structure.
I found this reference which was the closest to what I wanted, however it was not so clear for me: how can I parse audio raw data recorder with gstreamer?
Summarizing the points I need:
Break a song into packets
Play the audio content of a single packet (or a small set of packets).
Thank you very much for the help!
An MP3 file is just a succession of MP3 frames. Each frame is made of a header and a data block.
Splitting the MP3 file as MP3 frames will involve parsing the MP3 file. You can refer to this documentation for a good description of the format.
Note that in the case of mpeg layer 3 codec, frames are not independant. In the worst case, 9 input frames may be needed before beeing able to decode one single frame.
What I would do instead of this
I guess you could probably ignore most of these details and focus on the streaming problem itself. Here is what I would try to build first:
on the sender side, split a file into packets, and send them one by one using your system. Command example: send_stream test.mp3
on the receiver side, receive the packets and rebuild the original file. Command example: receive_stream test.mp3
Once you have this working fine, modify the receiver program so that it writes the packets in-order on the standard output. This will allow you to redirect stdout to a file
# sender side did not change
send_stream test.mp3
# receiver side
receive_stream > test.mp3
Then, you can use madplay to play the mp3 while it is received simply by redirecting receive_stream output to madplay:
# madplay - tells madplay to read its input from standard input.
receive_stream | madplay -
For a good mp3 decoder, take a look at MAD.
I am currently trying to implement an algorithm which can quickly discard unwanted frames from an MP4 video when encoding to another MP4 (using Media Foundation).
The encoding part doesn't seem to bad - the "Source Reader plus Sink Writer" approach is nice and fast. You basically just have to create an IMFSourceReader and an IMFSinkWriter, set the source native media type on the writer, yada, yada, yada, and just loop: source.ReadSample(&sample) --> writer.WriteSample(&sample). The WriteSample() calls can be conditioned on whether they're "! 2 b discarded".
That naive approach is no good if you consider that the samples read will be "predicted-frames", a.k.a., P-frames in the H.264 encoded streams. Dropping any preceding "intra-coded picture frame" (I-frame or key-frame) before that will result in garbled video.
So, my question is, is it possible to introduce an I-frame (somehow) into the sink writer before resuming the sample writing in the sink writer?
Doing something with the MFSampleExtension_CleanPoint attribute doesn't seem to help. I could manually create an IMFSample (via MFCreateSample), but getting it in the right H.264 format might be tricky.
Any ideas? Or thoughts on other approaches to dropping frames during encoding?
I think that this is not possible without reenconding the video! The Reference between P and I Frames are in the h264 Bitstream and not in the container (MP4). You can only safely skip frames, which are not referenced from other frames:
last P-Frames of a GOP (before the next I-Frame)
B-Frames
Normaly these Frames are not referrenced, but they can be! This dependes on the encoder-settings used to create the h264 stream
I have written an application which triggers an IP Camera to stream it's data (MPEG4) over RTP. This works fine so far - I start to setup and start the stream with the corresponding RTSP commands ( DESCRIBE, SETUP and PLAY ).
While streaming I receive the usual Sender Reports and send my own Receiver Reports - Everything is working fine here.
Now with the application mentioned above, I do NOT read the stream. I have a seperate hardware , which just logs all the stuff going over the Ethernet ( a little bit like Wireshark ). Now when the whole streaming is finished I can download those logs from my hardware and extract data from them.
So what I have then is a logfile with all the data from the RTP stream as raw data.
My question would now is: How do I write this appropriately into a MPEG4 file? I know this is a very broad question and I don't expect to get a step-by-step tutorial. But actually I am a bit overwhelmed and don't know where to start.If I just memcpy all the Payload from the RTP messages sequentially into a MPEG4 file it doesn't work. Now I am also a bit confused by SDP and stuff.
Well maybe someone has a link or some help for me..?
You should first read RFC3016, which describes the RTP format of MPEG-4 stream, then you'll know how to extract MPEG-4 frames from the RTP stream.
I actually changed from MPEG4 to H.264 - it actually was a little bit easier to write a video file like this. For H.264 this answer covers it pretty much:
How to process raw UDP packets so that they can be decoded by a decoder filter in a directshow source filter
Totally new to this! As the title says, I'm trying to serve a stream from OpenCV through Live555 using H.264 that is captured from a webcam.
I've tried something like:
#define LOCALADDRESS "rtsp://localhost:8081" // Address media is served
#define FOURCCCODEC CV_FOURCC('H','2','6','4') // H.264 codec
#define FPS 25 // Frame rate things run at
m_writer = cvCreateVideoWriter(LOCALADDRESS, FOURCCCODEC, FPS, cvSize(VIDEOWIDTH, VIDEOHEIGHT));
as reading a rtsp stream, is done similarly:
CvCapture *capture = cvCreateFileCapture(LOCALADDRESS);
which doesn't work so I'm turning to Live555. How do I feed a CvCapture encoded in H.264 to be served by Live555? There doesn't seem to be a straitforward way to serve a bytestream from one to another or perhaps I'm missing something.
There really isn't a straight-forward way I know of; certainly nothing that will happen in anything less than a few hundred lines of code.
I'm assuming you want to use an on-demand RTSP server (this is where the server's just sitting there, waiting for a client to connect, and then it starts streaming when the client establishes a connection and makes a request)? If so, this item in the Live555 FAQ applies.
However, Live555 is a weird (possibly misguided?) library, so it's unfortunately a bit more complicated than that. Live555 uses a single thread of operation with an event loop, so what you'll have to do is shove your raw bytestream into a buffer or queue, and then in your subsession class for streaming H.264, you'll check and see if there's available data in the queue and if so, pass it along. If not, schedule another check in a few milliseconds. You'll also need to strip off any NALU identifiers before you pass them along to live555.