I am converting a .wav file to .pcm file using gstream, i need to verify the sampling rate of .pcm file after converting. Which tool can i use for this?Or is there any way to check it?Sorry if its a wrong question.
Already i tried with sox tool, but it doesn't support .pcm format. Please help me with this. Below is an example
gst-launch-1.0 filesrc location=file.wav ! wavparse ! audioresample ! audioconvert ! audio/x-raw,format=S16BE,channels=1,rate=8000 ! filesink location=file.pcm
UPDATE : PCM files sampling rate cannot be checked using any tools, "Audacity" software can be used to import the raw samples and set the sampling rate. We can check by listening to the actual audio with the converted audio and verify whether the sampling rate converted is ok.
PCM files are plain raw files without any header so it is not possible to detect the sampling rate. You need sampling rate, channels, etc to be set in header which is what WAV format does.
Equivalent of PCM in Audio,is YUV in video or RGB in image. Neither of them have any header. It is upto the user to parse them with desired sample rate, resolution, format etc and make meaningful buffers out of them
Related
Hey,
I am new to Gstreamer and want to send a video that is captured from a camera and manipulated with OpenCV over a network to the receiving part. The receiving part then read it and displays it. This shall be done in real-time. It basically works with the code/gstreamer settings below however as soon a frame is dropped (at least I think this is the reason) the video get corrupted in form of grey parts (attached picture).
OpenCV Sending Part:
cv::VideoWriter videoTransmitter("appsrc ! videoconvert ! videoscale ! x264enc ! rtph264pay config-interval=1 pt=96 ! udpsink host=192.168.168.99 port=5000", cv::VideoWriter::fourcc('H', '2', '6', '4'), 10, videoTransmitter_imageSize, true);
OpenCV Receiving part:
cv::VideoCapture videoReceiver("udpsrc port=5000 ! application/x-rtp ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! videoconvert ! appsink", cv::CAP_GSTREAMER);
It basically works but I often get grey parts in the video which then stay for a bit until the video is displayed correctly. I guessed it happens always when a frame is dropped due to the transmission. However, how can I get rid of these grey/corrupted frames? Any Hints? Any Gstreamer parameters I need to set to tune result? Is there a better way to stream a video with opencv over network?
Any help is appreciated!
No, there isn't any mechanism in Gstreamer to detect corrupted frames, because this doesn't make sense.
In most modern video codec, frame aren't sent in full anymore, but split in slices (meaning only a small part of the frame). It can takes multiple intra packets (each containing multiple slices) to build a complete frame, and this is a good thing, because it makes your stream more resilient to errors, and allow multithreaded decoding of the slices (for example).
In order to achieve what you want, you have multiple solutions:
Use RTP/RTCP instead of RTP over UDP only. At least RTP contains a sequence number and "end of frame" markers so it possible to detect some packet drops. Gstreamer doesn't care about those by default unless you have started a RTP/RTCP session. If you set up a session with RTCP, you can have reports when some packets were dropped. I'm not sure there is a pipeline way to be informed when a packet is dropped, so you might still have to write an appsink in your gstreamer pipeline to add some code for detecting this event. However, this will tell you something is wrong, but not when it's ok to resume or how much wrong it is. In Gstreamer speak, it's called RTPSession, and you're interested in the stats::XXX_nack_count properties,
Add some additional protocol to compute the checksum of the encoder's output frame/NAL/packet and transmit out of band. Make sure the decoder also compute the checksum of incoming frame/NAL/packet and if doesn't match, you'll know it'll fail decoding. Beware of packet/frame reordering (typically B frames will be re-ordered after their dependencies) that could disturb your algorithm. Again, you have no way to know when to resume upon an error. Using TCP instead of UDP might be enough to fix it if you only have partial packet drop, but it'll fail to resume if it's a bandwidth issue (if the video bandwidth > network bandwidth, it'll collapse, since TCP can't drop packets to adapt)
Use intra only video codec (like APNG, or JPEG). JPEG can also partially decode, but gstreamer's default software jpeg decoder doesn't output a partial JPEG frame.
Set a closed and shorter GOP in your encoder. Many encoder have a pseudo "gop = group of picture" parameter and count the frames in your decoder when decoding after an error. A GOP ensure that whatever the state of the encoding, after GOP frames, the encoder will emit an non-dependent group of frames (likely enough intra frame/slices to rebuild the complete frame). This will allow resuming after an error by dropping GOP - 1 frames (you must decode them, but you can't use them, they might be corrupted), you'll need a way to detect the error, see point 1 or 2 above. For x264enc the parameter is called key-int-max. You might want to try also intra-refresh=true so the broken frame effect upon error will be shorter. The downside is an increase in bandwidth for the same video quality.
Use a video codec with scalable video coding (SVC instead of AVC for exemple). In that case, in case of decoding error, you'll get a lower quality instead of corrupted frame. There isn't any free SVC encoder I'm aware of in Gstreamer.
Deal with it. Compute a saturation map of the picture with OpenCV and compute its deviation & mean. If it's very different from the previous picture, stop computation until the GOP has elapsed and the saturation is back to expected levels.
I'm using GStreamer to process videos in my project. The input videos can have various formats (both resolutions and aspect-ratio). It can be for example 400x300, 1080p, 4K, 2000x1000, etc.
I would like to downscale videos automatically which are larger than 1080p. So if video is in 4K it should be downscaled to 1080p but if it is in 400x300 format it should be kept in original format.
I've found videoscale pipeline plugin but it does not work as I would like. It is changing scale up and down without any distinctions. Also it is not changing sizes proportionally when only width or height is provided.
Do you know any straightforward way in GStreamer to downscale resolutions automatically to desired size?
GStreamer's caps allow ranges. So I believe you are looking for something like this:
video/x-raw,width=[1,1920],height=[1,1080],pixel-aspect-ratio=1/1
This will keep the same aspect ratio but scales down in case it is required to fit into 1920x1080.
E.g.
gst-launch-1.0 videotestsrc ! video/x-raw,width=4000,height=2000 ! videoscale ! video/x-raw,width=[1,1920],height=[1,1080],pixel-aspect-ratio=1/1 ! autovideosink
Will be scaled down to 1920x960.
And:
gst-launch-1.0 videotestsrc ! video/x-raw,width=400,height=200 ! videoscale ! video/x-raw,width=[1,1920],height=[1,1080],pixel-aspect-ratio=1/1 ! autovideosink
Will stay at 400x200.
I am trying to build a simple transcoder that can take MP3 and WAV files, and segment them using the segment formatting option, while also possibly changing the sample rate, bit rate and channel layout.
For this, I followed the code in the transcoding.c example. The issue is that when trying to transcode from a 32K HZ MP3 to 48K HZ MP3. The problem is that the MP3 encoder expects 1152 frame size, but libavfilter provides me with frames that contain 1254 number of samples. So when I try to do the encoding, I get this message: more samples than frame size. This problem can also be reproduced using the example code, just set the sample rate of the encoder to 48K.
One option is to use the asetnsamples filter, and set it to 1152, that will fix upsampling to 48K, but then downsampling to 24K won't work, because the encoder expects frame sizes of 576.
I wouldn't want to set this filter's value depending on the input information, it may become messy later if I support more file types, such as AAC.
Is there any way of making the libavfilter libraries know about this flow, and trigger proper filtering and transcoding without having to use lower level APIs, like libswresample or doing frame buffering?
Anybody, has tried upsampling audio stream from 8K to 44.1K?
I need to resample input audio stream 8KHz to 44.1K since Mac OSX default audio output device support minimum 44.1K audio sampling rate.
I tried to up-sampling using FFMPEG swr_convert() API, it converts with lots of noise. Which is not good.
If anybody has tried successfully upscale 8K to 44.1 or 48K then please share it.
Solution with C/C++ library code is preferable. Didn't tried Core-audio samples.
I Tried swr_convert() code from following link https://www.ffmpeg.org/doxygen/2.1/group__lswr.html#details
Thanks,
Ramanand
I am using panda board and i have installed opencv and wrote a code for sticking 3 different images from 3 different cams.now this stitched image is stored in a matrix location(pointer).i for that 3 cams images will be continuously captured and stitched.so it becomes a video.so i need to stream that stitched image to iPhone .can any one help me with this.i am really stuck here and need help.its very important for me.
I would suggest you look at constructing either mjpeg stream or better a RTSP (encapsulating mpeg4 - saving bandwidth) stream based on RTP protocol. Say you decide to go with mjpeg stream, then each of your opencv IplImage* can be converted to JPEG Frames using libjpeg compression. See my answer here Compressing IplImage to JPEG using libjpeg in OpenCV. You would compress each frame and then create mjpeg stream. See creating my own MJPEG stream. You would need a webserver to run mjpeg cgi that streams your image stream. You could look at lighttpd web server running on Panda Board. Gstreamer is the package that may be helpful in your situation. On the decoding side (iphone) you can construct gstreamer decoding pipeline as follows - say you are streaming mjpeg gst-launch -v souphttpsrc location="http://<ip>:<port>/cgi_bin/<mjpegcginame>.cgi" do-timestamp=true is_live=true ! multipartdemux ! jpegdec ! ffmpegcolorspace ! autovideosink