I need to implement VP9 decoding over RTP protocol in existing project which is written in C++ for VS09 compiler. As result i need bytes of decoded picture. I tried using libvpx. I compiled libvpx 1.8.0 for VS9 compiler and add it to my project. I already have part where i receive packets on some port, then i parse raw packet like RTP and payload pass to vpx_codec_decode but libvpx cannot parse payload.
Related
overall procedure is like below.
client record the voice for some duration(ex, 5 sec) in some format(webm or wav)
then it send it to server using websocket.
server received the several packets(each packet size is 4096bytes), and each packet is sent to opus decoder.
but opus decoder return invalid packet error.
server is coded with c++(using libwebsocket and libopus library) in ubuntu.
could anyone help me how to do ?
general procedure or some example code is ok
its difficult to find info or community.
thanks
The opus decoder may be expecting an Ogg Opus container file. If you're using WebM, you could extract the encoded audio pages and pass them to a raw Opus decoder that is not dependent on the Ogg container. You could also see how the ffmpeg project is decoding WebM Opus files to PCM
Could the client send an Ogg Opus file instead of a WebM Opus file? There's a chunk-based decoder written in C that can decode Ogg Opus files very quickly. It's intended for WebAssembly, but the C code is not dependent on WASM. See opus_chunk_decoder.c:
https://github.com/AnthumChris/opus-stream-decoder/tree/master/src
We have the captured pcap file which includes RTP opus payload per rfc6716, now we can cut off the RTP header and extract the opus payload, we want to encapsulate the payload to ogg opus per spec https://datatracker.ietf.org/doc/html/draft-ietf-codec-oggopus-07 (Ogg Encapsulation for the Opus Audio Codec) and send out, so that VLC can playback the captured opus, we don't want to save to an ogg file then let VLC to playback, we will send the ogg opus out to VLC directly once one packet is encapsulated, anyone who have the referenced implementation of the encapsulation, or 3rd party library I can refer?
The packets can be read using the libpcap library and then encapsulated in Ogg using the libogg library. There is an example program called opusrtp in the opus-tools package that can sniff for Opus RTP packets on the loopback interface using libpcap and write them to Ogg. You would want to do something similar, but change the pcap_open_live() to something like pcap_open_offline() if you want to read from a pcap save file, and write the Ogg pages from libogg to a socket instead of a file. Also define OPUS_PAYLOAD_TYPE to be the RTP payload type you want to look for.
I had a similar need, and following the advice from this answer I wrote an integration of opusrtp that can receive as input a pcap file and then generates the .opus from it.
The gist was in fact using pcap_open_offline() instead of pcap_open_live(), set the correct payload type, and a few other details to adapt to the input file format.
I have the modified opusrtp in a fork on github.
You can use it with something like
./opusrtp --extract PCAPFILE
It generates rtpdump.opus, which you can then transform as needed.
I currently working on a project simulate webcam video transmission in C++, at sender side, I capture the raw webcame video with v4l2, encoded with FFmpeg, video file are put into an array and transmitted. And at decoder side, video data received to an array, decoded and play. The program works fine with codec_id AV_CODEC_ID_MPEG1VIDEO, but when I try replace it with AV_CODEC_ID_H264, some problem happen in decoding, please refer to FFmpeg c++ H264 decoding error. Some people suggest me to use parser but I have no idea how is a parse in ffmpeg looks like. Any simple example of how to build a parser for H.264 in FFmepg? I cannot find such tutorial in google.....
From this WebRTC Code and API page:
StartRTPDump
This function enables capturing of RTP packets to a binary file on a
specific channel and for a given direction. The file can later be
replayed using e.g. RTP Tools’ rtpplay since the binary file format
is compatible with the rtpdump format.
NOTE: It is recommended that you use this API for debugging purposes only since the created files can become very large.
What is a recommended way to save RTP packets into a file for later processing?
I need to access and read an http stream which is sending live MJPEG footage from a network camera, in order to do some opencv image processing on the image.
I can access the camera's footage through VLC, or simply by going to the URL in chrome or firefox. But how can I programmatically access the http server and separate each frame, when the server is just sending a continuous feed?
The data seems to be simply formatted, looping between the HTTP Header and JPEG data. The only way I can think of approaching this is somehow sending a request to the server, parsing the data as it comes in, and separating the header from the actual jpeg data, and, in turn, passing that to opencv.
However, that sounds awfully convoluted and I'm not quite sure where I'd start. Do you guys know if there are any libraries out there, or just a simpler approach I'm overlooking, that could make all this easier?
Thanks alot
For HTTP download, you can use Libcurl library.
AFAIK MJPEG format is not a standardized format. Its actual byte format vary by implementations. But basically just concatenation of jpeg file with delimiters. If you look at bytes with a hex editor you could easily distinguish each jpeg file.
For example, ffmpeg's mjpeg output is structured like below:
0xff 0xd8 // start of jpeg
{ ... } // jpeg body
0xff 0xd9 // end of jpeg
...
0xff 0xd8 // start of jpeg
{ ... } // jpeg body
0xff 0xd9 // end of jpeg
...
In this page:
http://thistleshrub.net/www/index.php?controller=posts&action=show&id=2012-05-13DisplayingStreamedMJPEGinJava.txt
Parse a MJPEG Stream with Java, I implemented this with flawlessly results in Java.
If you try to use with C++ you find some things missed: socket conection and render canvas, libcurl seems to be a good option to http request, but still missing the canvas, you can use something like GLUT or Qt.
I read in some forums that OpenCV can read input stream of type MJPEG Streamer, but seems they need to be the recent version of OpenCV (compile OpenCV from scratch it's hard).
I hope this help.