I'm developing app which sends mpeg2ts stream using FFMPEG API.(avio_open, avformat_new_stream etc..)
The problem is that the app already has AAC-LC audio so audio frame does not need to be encoded because my app just bypass data received from socket buffer.
To open and send mpegts using FFMPEG, I must have AVFormattContext data which is created from FFMPEG API for encoder as far as I know.
Can I create AVFormatContext manually with encoded AAC-LC data? or I should decode and encode the data? The information I know is samplerate, codec, bitrate..
Any help will be greatly appreciated. Thanks in advance.
Yes, you can use the encoded data as-is if your container supports it. There are two steps involved here - encoding and muxing. Encoding compress the data, muxing mixes it together in the output file, so the packets are properly interleaved. Muxing example in FFMpeg distribution helped me with this.
You might also take a look at the following class: https://sourceforge.net/p/karlyriceditor/code/HEAD/tree/src/ffmpegvideoencoder.cpp - this file is from one of my projects, and contains video encoder. Starting from the line 402 you'll see the setup for non-converted audio - it is kind of a hackish way, but it worked. Unfortunately I still end up reencoding audio because for my formats it was not possible to achieve frame-perfect synchronization which I needed
Related
I can record audio using MediaFoundation, but it only gives me a PCM Wave buffer. I want to grab the full buffer, encode it to MP3, and then use the new buffer for networking stuff.
What is the right way to encode the audio after receiving the samples? I have gotten lost reading through MediaSession, MediaSinks, SinkWriter, Transcode API, Transform API, Source Resolver, etc.
I see there is an MP3 encoder object, but I cant find the documentation on how to use it. I also found an MP3 MediaSink but im not sure how the MediaSink fits in with the SourceReader / SinkWriter schema or how to create/use the IMFByteStream it requires.
Is the MediaFoundation the right WinAPI for the task?
I'm interested in webRTC's ability to P2P livestream an mp3 audio from user's machine. Only example, that I found is this: https://webrtc-mp3-stream.herokuapp.com/ from this article http://servicelab.org/2013/07/24/streaming-audio-between-browsers-with-webrtc-and-webaudio/
But, as you can see, the audio quality on receiving side is pretty poor (45kb\sec), is there any way to get a full quality MP3 streaming + ability to manipulate this stream's data (like adjusting frequencies with equalizer) on the each user's sides?
If impossible through webRTC, is there any other flash-plugin or pluginless options for this?
Edit: also I stumbled upon this 'shoutcast kinda' guys http://unltd.fm/ , declaring, that they are using webRTC to deliver top quality radio broadcasting including streaming mp3. If they are, then how?
WebRTC supports 2 audio codecs: OPUS (max bitrate 510kbit/s) and G711. You stick with OPUS, it is modern and more promising, introduced in 2012.
Main files in webrtc-mp3-stream are outdated by 2 years (Jul 18, 2013). I couldn't find OPUS preference in the code, so possibly demo runs via G711.
The webrtc-mp3-stream demo does the encoding job (MP3 as a media source), then it transmits the data over UPD/TCP via WebRTC. I do not think you need to decode it to MP3 on receiver side, this would be an overkill. Just try to enable OPUS to make the code of webrtc-mp3-stream more up-to-date.
Please refer to Is there a way to choose codecs in WebRTC PeerConnection? to enable OPUS to see the difference.
I'm the founder of unltd.fm.
igorpavlov is right but I can't comment answer. We also use OPUS (Stereo / 48Khz) codec over WebRTC.
Decoding mp3 ( or any other audio format ) using webaudio then encoding it in OPUS is the way to go. You "just" need to force SDP negotiations to use OPUS.
You should have send us an email you would have saved your 50 points ;)
You can increase the quality of a stream by setting the SDP to be stereo and increase the maxaveragebitrate:
let answer = await peer.conn.createAnswer(offerOptions);
answer.sdp = answer.sdp.replace('useinbandfec=1', 'useinbandfec=1; stereo=1; maxaveragebitrate=510000');
await peer.conn.setLocalDescription(answer);
This should output a SDP string which looks like this:
a=fmtp:111 minptime=10;useinbandfec=1; stereo=1; maxaveragebitrate=510000
This gives a potential maximum bitrate of 520kb/s for stereo, which is 260kps per channel. Actual bitrate depends on the speed of your network and strength of your signal.
You can read more about the other available SDP attributes at: https://www.rfc-editor.org/rfc/rfc7587
I am using FFmpeg to write a mp4 file, I grab bitmap images from remote IP camera and encode it by h.264, the media container is mp4 file, no problem to generate the MP4 file if I only record several minutes, the problem is FFmpeg never flushs buffer data to disk when I call method av_interleaved_write_frame(all encoded data in memory, never free them), only when I call method avio_close(oc->pb);, it will flush all encoded data to disk, I tried to call method avcodec_flush_buffers every time after calling av_interleaved_write_frame, but no effect. I am newbie to FFmpeg, if you are familiar with FFmpeg, please help me.
thanks in advance.
Sean
I got the problem, that was caused by I never write audio frame, so if just want to encode several bmps to a video file, please note:
1) don't add audio stream(add_stream).
2) don't open audio stream(open_audio).
hope this also helps others.
Sean
Another suggestion (it worked just well in my case) is to call with NULL AVPacket pointer:
av_interleaved_write_frame(AVFormatContext*, NULL);
then it flushes whatever stream it has in the buffer.
I need to access and read an http stream which is sending live MJPEG footage from a network camera, in order to do some opencv image processing on the image.
I can access the camera's footage through VLC, or simply by going to the URL in chrome or firefox. But how can I programmatically access the http server and separate each frame, when the server is just sending a continuous feed?
The data seems to be simply formatted, looping between the HTTP Header and JPEG data. The only way I can think of approaching this is somehow sending a request to the server, parsing the data as it comes in, and separating the header from the actual jpeg data, and, in turn, passing that to opencv.
However, that sounds awfully convoluted and I'm not quite sure where I'd start. Do you guys know if there are any libraries out there, or just a simpler approach I'm overlooking, that could make all this easier?
Thanks alot
For HTTP download, you can use Libcurl library.
AFAIK MJPEG format is not a standardized format. Its actual byte format vary by implementations. But basically just concatenation of jpeg file with delimiters. If you look at bytes with a hex editor you could easily distinguish each jpeg file.
For example, ffmpeg's mjpeg output is structured like below:
0xff 0xd8 // start of jpeg
{ ... } // jpeg body
0xff 0xd9 // end of jpeg
...
0xff 0xd8 // start of jpeg
{ ... } // jpeg body
0xff 0xd9 // end of jpeg
...
In this page:
http://thistleshrub.net/www/index.php?controller=posts&action=show&id=2012-05-13DisplayingStreamedMJPEGinJava.txt
Parse a MJPEG Stream with Java, I implemented this with flawlessly results in Java.
If you try to use with C++ you find some things missed: socket conection and render canvas, libcurl seems to be a good option to http request, but still missing the canvas, you can use something like GLUT or Qt.
I read in some forums that OpenCV can read input stream of type MJPEG Streamer, but seems they need to be the recent version of OpenCV (compile OpenCV from scratch it's hard).
I hope this help.
I'm looking for a way to extract the audio part of a FLV file.
I'm recording from the user's microphone and the audio is encoded using the Nellymoser Asao Codec. This is the default codec and there's no way to change this.
ffMpeg is the way to go !
It worked for me with SVN Rev 14277.
The command I used is : ffmpeg -i source.flv -nv -f mp3 destination.mp3
GOTCHA :
If you get this error message : Unsupported audio codec (n),
check the FLV Spec in the Audio Tags section.
ffMpeg can decode n=6 (Nellymoser).
But for n=4 (Nellymoser 8-kHz mono) and n=5 (Nellymoser 16-kHz mono) it doesn't work.
To fix this use the default microphone rate when recording your streams, overwise ffMpeg is unable to decode them.
Hope this helps !
This isn't an exact answer, but some relevant notes I've made from investigating FLV files for a business requirement.
Most FLV audio is encoded in the MP3 format, meaning you can extract it directly from the FLV container. If the FLV was created from someone recording from their microphone, the audio is encoded with the Nellymoser Asao codec, which is proprietary (IIRC).
I'd check out libavcodec, which handles FLV/MP3/Nellymoser natively, and should let you get to the audio.
I'm currently using FFmpeg version SVN-r12665 for this, with no problems (the console version, without any wrapper library). There are some caveats to using console applications from non-console .NET environments, but it's all fairly straightforward. Using the libavcodec DLL directly is much more cumbersome.
I was going to recommend this: http://code.google.com/hosting/takenDown?project=nelly2pcm¬ice=7281.
But its been taken down. Glad I got a copy first :-)