I'm finding out panasonic camera. I want to get audio data from this and try to play it. I sent a command to camera and receiced a stream data as following:
HTTP/1.0 200 OK
Content-type: multipart/x-mixed-replace;boundary=--myboundary
--myboundary
Content-type: audio/g726
...binary data...
I have searched some open sources on internet to decode the stream data but actually i don't know the data encoded by this camera using u/a/l law, 2/3/4 bits, rigt/left packed these parameter neccessary to decode exactly. Does anyone know please tell me? Is there any format for g726?
Assume that I decoded successfully. Could you give me a piece of code written by C/C++ how to play the data after decoding. I mean that I want to hear the audio data which get from camera on my PC.
ITU makes available source code for the G.726 codec here, though giving you code to do placyback seems a bit out of scope for an answer, and we'd have to at least know your operatating system..
Related
I'm trying to get and store the data from an IP camera and I'd appreciate some high level advice as to the best way to do this.
So far I've successfully initiated an RTSP conversation with the camera and have it sending me UDP packets with RTP payloads. But I'm unsure where to go from here.
I'm very happy to do the work, I'd just appreciate some pointers / a high level overview of the steps so I can deconstruct the project!
There is no direct answer to the OPs question here for his question is a bit broad, and without further information that pertains to what the OP intends to do with that information it is difficult to give a precise answer. What I can do here is to suggest to the OP steps that maybe taken and what problems to consider.
OP had stated:
So far I've successfully initiated an RTSP conversation with the camera and have it sending me UDP packets with RTP payloads. But I'm unsure where to go from here.
Now that you have an established communication with the camera and are able to receive data packets via video stream it is now a matter of being able to understand what the RTP payloads are, or how to interpret that data. So at this point you will have to do your research on the RTP protocol which appears to me to be a type of a Network Protocol. Once you have written your structure and functions to work successfully with this protocol it is a matter of breaking down the UPD packets into useful bytes of information. Normally in many cases when it comes to processing either graphic, video or audio data either from a file directly or a stream object, they are usually accompanied with some type of header information. Next, it is a matter of understanding this Header information which is normal in a form of a structure that gives information about the type of content this file or stream holds, so that you know how many bytes of information to extract from it.
I know it's not going to just be a case of saving the RTP payload directly to a file, but what other steps are involved?
The steps involved may vary depending on your needs and what you intend to do with the information: Are you trying to write the properties or the general information about the video content to a file such as: its compression type, its audio - video codec type, its resolution and frame rate information, its byte rate etc.? Or are you trying to write the actual video content itself to a file that your application will use either for play back or for editing purposes? This all depends on your intentions.
Is the data compressed, in which case I have to decompress it?
At this point once you have successfully been able to interpret the RTP Protocol and parsed the data packets by understanding their header information and saving it to a proper structure, it is then a matter of using that header information to determine what is actually within that stream object. For example and according to the PDF about the properties of the Video Camera that you have supplied the Video Compression can be saved in 2 types, H.264 or MJPEG, this you will have to determine by the information that was provided in the header, from here you would have to branch your code and be able to read and parse each type of compression or, accept the one that you are willing to work with and disregard the other. Next is the Audio Compression if you are concerned about the audio and the types available are AAC(Encoding only), G.711 A-Law, & G.711 U-Law and the same mechanisms apply here. The once are you able to get past the audio and video compression you will then need vital information about the video information itself such as what Resolution and Frame Rate (buffer sizes) were stored from the header information so you know how many bytes to read from the stream and how far to move your pointer through the stream. If you notice the Resolution And Frame Rate there are different acceptable formats available from each type of Compression that is being used:
H.26
1920 x 180 (2.1MP) # 30 fps (1080p)
1280 x 720 # 60 fps (720p)*
720 x 480/576 # 30/25 fps (D1)
704 x 480/576 # 30/20 fps (4CIF)
352 x 240/288 # 30/25 fps (CIF)
MJPEG
720 x 480/576 # 30/25 fps (D1)
740 x 480/578 # 30/25 fps (4CIF)
352 x 240/288 # 30/25 fps (CIF)
Now this is for the resolution & frame rate but the next thing to consider is
you are working with video stream so the above may not apply here in your case and according to the properties about Video-Stream capabilities from the Video Camera These are the types available that you will have to take into account for:
Single-stream H.264 up to 1080p (1920 x 1080) # 30 fps
Dual-stream H.264 and MJPEG
H.264: Primary stream programmable up to 1280 x 720 # 25/20 fps
MJPEG: Secondary stream programmable up to 720 x 576 # 25/20 fps
With these different types available for your Video Camera to use you have to take all these into consideration. Now this also depends on your intentions of your application and what you intend to do with the information. You can write your program to accept all of these types or you can program it to accept only one type with a specific format of that type. This depends on you.
Do I have to do any other modifications?
I don't think you would have any modifications to do unless if your intentions within your actual application is to modify the video - audio information itself. If your intentions within your application are to just read the file for simple playback then the answer would be no as long as all the appropriate information was saved properly and your file parser for reading your custom file structure is able to read in your file's contents and is able to parse the data appropriately for general playback.
Where can I learn about what I'll need to do specific to this camera?
I don't think you need to much more information about the camera itself, for the PDF that you provided in the link within your question has already given you enough information to go on with. What you would need from here is information and documentation about the specific Protocols, Packet Types, Compression & Stream types which a general search of these should suffice.
UDP
Do a Google search for c++ programming UDP Sockets for either Linux or Winsock.
RTP
Do a Google search for c++ programming RTP Packets
Video Compression
Do a Goggle search for both H.26 & MJPEG compression and structure information on stream objects.
Audio Compression
Do a Google search for each of AAC(encoding only), G.711 A-Law, G.711 U-Law if you are interested in the audio as well.
From there once you have the valid specifications for these data structures as a stream object and have required the appropriate header information to determine which type and format this video content is saved as then you should be able to easily parse the Data Packets appropriately. Now as to how you save them or write them to a file all depends on your intentions.
I have provided this as a guideline to follow in order to help lead you in the right direction in a similar manner that a chemist, physicist, scientist, or engineer would approach any typical problem.
The general steps are by following a scientific approach about the current problem. These typically are:
Assessing the Situation
Create either a Hypothesis or a Thesis about the Situation.
Gather the Known Facts
Determine the Unknowns
Draft a Model that Shows a Relationship Between the Known and Unknowns.
Perform both Research and Experimentation
Record or Log Events and Data
Analyze the Data
Draw a Conclusion
Now in the case of writing software application the concept is similar but the approaches may be different or vary as not all of the steps above may be needed and or some additional steps may be required. One such step in the Application Development Cycle that is not found in the Scientific approach would be the process of Debugging an Application. But the general guideline still applies. If you can keep to this type of strategy I am sure that you will be able to have the confidence in gathering what you will need and how to use it from a step by step process to achieve your goals.
I'm trying to get and store the data from a Cisco IPC camera, and I'd appreciate some high level advice as to the best way to do this.
You can probably use openRTSP to do this, which can also output to file. For this approach you would have to write NO code. Implementing RTP, RTSP and RTCP correctly is complex and a lot of work. Should you have requirements that openRTSP doesn't meet, you can use the live555 libraries to for RTSP/RTP/RTCP and write some minimal code to do something with the received video. The mailing list is very responsive provided that you ask "good" questions, and make sure you read the FAQ first.
I know it's not going to just be a case of saving the RTP payload directly to a file, but what other steps are involved?
You don't need to know this if you use openRTSP. If you use the live555 libraries directly, you'll be passed entire video frames that you would then either have to decode and write to file yourself depending on what you want to achieve. If you DO need/want to know about RTP and RTP payload formats, read the corresponding RFCs, e.g. RFC2326, RFC3550, RFC6184.
Is the data compressed, in which case I have to decompress it?
Generally you want to store compressed media in a file, and use media player software to decode it on playback (Otherwise you end up with huge files).
Where can I learn about what I'll need to do specific to this camera?
If you just want to save the video you ideally don't need to know anything about the camera, other than what standards it implements (which you already do).
I'm developing app which sends mpeg2ts stream using FFMPEG API.(avio_open, avformat_new_stream etc..)
The problem is that the app already has AAC-LC audio so audio frame does not need to be encoded because my app just bypass data received from socket buffer.
To open and send mpegts using FFMPEG, I must have AVFormattContext data which is created from FFMPEG API for encoder as far as I know.
Can I create AVFormatContext manually with encoded AAC-LC data? or I should decode and encode the data? The information I know is samplerate, codec, bitrate..
Any help will be greatly appreciated. Thanks in advance.
Yes, you can use the encoded data as-is if your container supports it. There are two steps involved here - encoding and muxing. Encoding compress the data, muxing mixes it together in the output file, so the packets are properly interleaved. Muxing example in FFMpeg distribution helped me with this.
You might also take a look at the following class: https://sourceforge.net/p/karlyriceditor/code/HEAD/tree/src/ffmpegvideoencoder.cpp - this file is from one of my projects, and contains video encoder. Starting from the line 402 you'll see the setup for non-converted audio - it is kind of a hackish way, but it worked. Unfortunately I still end up reencoding audio because for my formats it was not possible to achieve frame-perfect synchronization which I needed
I'm interested in webRTC's ability to P2P livestream an mp3 audio from user's machine. Only example, that I found is this: https://webrtc-mp3-stream.herokuapp.com/ from this article http://servicelab.org/2013/07/24/streaming-audio-between-browsers-with-webrtc-and-webaudio/
But, as you can see, the audio quality on receiving side is pretty poor (45kb\sec), is there any way to get a full quality MP3 streaming + ability to manipulate this stream's data (like adjusting frequencies with equalizer) on the each user's sides?
If impossible through webRTC, is there any other flash-plugin or pluginless options for this?
Edit: also I stumbled upon this 'shoutcast kinda' guys http://unltd.fm/ , declaring, that they are using webRTC to deliver top quality radio broadcasting including streaming mp3. If they are, then how?
WebRTC supports 2 audio codecs: OPUS (max bitrate 510kbit/s) and G711. You stick with OPUS, it is modern and more promising, introduced in 2012.
Main files in webrtc-mp3-stream are outdated by 2 years (Jul 18, 2013). I couldn't find OPUS preference in the code, so possibly demo runs via G711.
The webrtc-mp3-stream demo does the encoding job (MP3 as a media source), then it transmits the data over UPD/TCP via WebRTC. I do not think you need to decode it to MP3 on receiver side, this would be an overkill. Just try to enable OPUS to make the code of webrtc-mp3-stream more up-to-date.
Please refer to Is there a way to choose codecs in WebRTC PeerConnection? to enable OPUS to see the difference.
I'm the founder of unltd.fm.
igorpavlov is right but I can't comment answer. We also use OPUS (Stereo / 48Khz) codec over WebRTC.
Decoding mp3 ( or any other audio format ) using webaudio then encoding it in OPUS is the way to go. You "just" need to force SDP negotiations to use OPUS.
You should have send us an email you would have saved your 50 points ;)
You can increase the quality of a stream by setting the SDP to be stereo and increase the maxaveragebitrate:
let answer = await peer.conn.createAnswer(offerOptions);
answer.sdp = answer.sdp.replace('useinbandfec=1', 'useinbandfec=1; stereo=1; maxaveragebitrate=510000');
await peer.conn.setLocalDescription(answer);
This should output a SDP string which looks like this:
a=fmtp:111 minptime=10;useinbandfec=1; stereo=1; maxaveragebitrate=510000
This gives a potential maximum bitrate of 520kb/s for stereo, which is 260kps per channel. Actual bitrate depends on the speed of your network and strength of your signal.
You can read more about the other available SDP attributes at: https://www.rfc-editor.org/rfc/rfc7587
I have an audio streamer based on ffplay. It works great using ffmpeg 0.11 but when I use ffmpeg 1.0 or the latest 1.2 release the audio seems to be decoded or played weirdly.
Essentially it sounds like chipmunks with mp3 streams and with aac streams I hear tons of static barely hearing the actual stream and the actual stream sounds slow.
Any ideas the possible changes in ffmpeg that could have caused these types of issues?
Similar issue was posted here but no actual answer about what is going on. Supposedly this code reproduces the same issue.
UPDATE 1:
I have done a step by step copy from ffplay and still no luck! :/ The channel and sampling rate look correct so there must be something internally that is returning a weird decoded format?
pCodecCtx->request_sample_fmt = AV_SAMPLE_FMT_S16
doesn't change the behavior! I didn't find any better solution than using swr_convert to convert audio sample from AV_SAMPLE_FMT_FLT to AV_SAMPLE_FMT_S16.
Found a fix... So initially it sounded like something wasn't matching correctly with the decoded raw PCM data and my PCM player. I took a deeper look at what was being returned by the decoder and any potential differences. Turns out that the default decoded format has changed from AV_SAMPLE_FMT_S16 to AV_SAMPLE_FMT_S16P. The fix was to simply specify pCodecCtx->request_sample_fmt = AV_SAMPLE_FMT_S16; before opening the decoder.
Any feedback if this is a bad idea? I'm concerned if there might be issues with other formats and potential performance issues...
I need to access and read an http stream which is sending live MJPEG footage from a network camera, in order to do some opencv image processing on the image.
I can access the camera's footage through VLC, or simply by going to the URL in chrome or firefox. But how can I programmatically access the http server and separate each frame, when the server is just sending a continuous feed?
The data seems to be simply formatted, looping between the HTTP Header and JPEG data. The only way I can think of approaching this is somehow sending a request to the server, parsing the data as it comes in, and separating the header from the actual jpeg data, and, in turn, passing that to opencv.
However, that sounds awfully convoluted and I'm not quite sure where I'd start. Do you guys know if there are any libraries out there, or just a simpler approach I'm overlooking, that could make all this easier?
Thanks alot
For HTTP download, you can use Libcurl library.
AFAIK MJPEG format is not a standardized format. Its actual byte format vary by implementations. But basically just concatenation of jpeg file with delimiters. If you look at bytes with a hex editor you could easily distinguish each jpeg file.
For example, ffmpeg's mjpeg output is structured like below:
0xff 0xd8 // start of jpeg
{ ... } // jpeg body
0xff 0xd9 // end of jpeg
...
0xff 0xd8 // start of jpeg
{ ... } // jpeg body
0xff 0xd9 // end of jpeg
...
In this page:
http://thistleshrub.net/www/index.php?controller=posts&action=show&id=2012-05-13DisplayingStreamedMJPEGinJava.txt
Parse a MJPEG Stream with Java, I implemented this with flawlessly results in Java.
If you try to use with C++ you find some things missed: socket conection and render canvas, libcurl seems to be a good option to http request, but still missing the canvas, you can use something like GLUT or Qt.
I read in some forums that OpenCV can read input stream of type MJPEG Streamer, but seems they need to be the recent version of OpenCV (compile OpenCV from scratch it's hard).
I hope this help.