I have a java app that uses gstreamer to process some audio. My pipeline is:
udpsrc name=side1 address="loadtest3.pslab.corp" retrieve-sender-address=false reuse=false port=17840 caps="application/x-rtp, media=(string)audio, payload=0, clock-rate=(int)8000" timeout=10000000000 ! rtppcmudepay ! rtppcmupay min-ptime=1000000000 max-ptime=1000000000 mtu=10012 ! rtppcmudepay ! mulawdec ! interleave name=mix ! mulawenc ! appsink name=appsink wait-on-eos=false emit-signals=true sync=false async=false udpsrc name=side2 address="loadtest3.pslab.corp" retrieve-sender-address=false reuse=false port=17842 caps="application/x-rtp, media=(string)audio, payload=0, clock-rate=(int)8000" ! rtppcmudepay ! rtppcmupay min-ptime=1000000000 max-ptime=1000000000 mtu=10012 ! rtppcmudepay ! mulawdec ! mix.
I do have over 200 pipelines running in parrallel and when the code sens an EOS in PLAYING state some of the pipelines are getting deadlocked into the sendEOS request.
For example, from 1693 pipelines executed I do get 67 that end-up in that deadlock.
See image. I would like to understand why this happens & how can I avoid it. I have added wait-on-eos on the appsink to false, but the behaviour remains the same.
Any ideas ?!? Thx.
enter image description here
Related
everyone
The version of GStreamer I use is 1.x. I've spent a lot of time in searching a way to delete a tee branch.
In an active pipeline, a recording bin is created as below and inserted into this pipeline by branching the tee element.
"queue ! video/x-h264, width=800, height=600, framerate=10/1, stream-format=(string)byte-stream ! h264parse ! mp4mux ! filesink location=/xxxx"
It works perfectly except that I want to dynamically delete the recording bin and get a playable mp4 file. According to some discussion and tutorial, to get a correct mp4 file , we need to handle something about EOS. After trying some methods, I always got broken mp4 files.
Does anyone have sample code written in C to show me ? I'd appreciate your help.
Your best bet for cases like this may be to create two processes. The first process would run the video, and half of the tee it has would deliver h264 data to the second process through whatever means.
Here are two pipelines demonstrating the concept using UDP sockets.
gst-launch-1.0 videotestsrc ! x264enc ! tee name=t ! h264parse ! avdec_h264 ! videoconvert ! ximagesink t. ! queue ! h264parse ! rtph264pay ! udpsink host=localhost port=8888
gst-launch-1.0 udpsrc port=8888 num-buffers=300 ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! mp4mux ! filesink location=/tmp/264.mp4
The trick to getting that clean mp4 is to make sure an EOS event is delivered reliably.
Instead of dynamically adding it you just have it in the pipeline by default, and add a probe callback at the source pad of the queue in the probe callback you have to do the trick either to pass the buffer or not (GST_PAD_PROBE_DROP drops the buffer and GST_PAD_PROBE_OK passes on the buffer to next element) so when you get an event to start/stop recoding you just need to return appropriate values. And filesink you can use multifilesink instead so as to write to different files everytime you start/stop.
Note the queue which drops the buffers needs before the mux element otherwise the file would be corrupt.
Hope that helps!
Finally, I came up with a solution.
Let's say that there is an active pipeline including a recording bin.
"udpsrc port=4444 caps=\"application/x-rtp, media=(string)video,
clock-rate=(int)90000, encoding-name=(string)H264 ! rtph264depay !
tee name=tp tp. ! queue ! video/x-h264, width=800, height=600,
framerate=10/1 ! decodebin ! videoconvert ! video/x-raw, format=RGBA !
autovideosink"
recording bin:
"queue ! video/x-h264, width=800, height=600, framerate=10/1,
stream-format=(string)byte-stream ! h264parse ! mp4mux ! filesink
location=/xxxx"
After a period of time, we want to stop recording and save as a mp4 file, and video media is still streaming.
First, I use a blocking probe to block the src pad of tee. In this blocking probe callback, I use an event probe to catch EOS in the sink pad of filesink and do a busy waiting.
*if EOS is catched in the event probe callback
self->isGotEOS = YES;
*busy waiting in the blocking probe callback
while (self->isGotEOS == NO) {
usleep(100000);
}
Before entering the busy waiting while loop, an EOS event is created and sent to the sink pad of recording bin.
After the busy waiting is done:
usleep(200000);
[self destory_record_elements];
I think usleep(200000) is a trick. Without it, a non-playable mp4 file is usually the result. It would seem that 200ms is long enough handling the EOS.
I had similar problem previously, my pipeline
videotestsrc do-timestamp="TRUE" ! videoflip method=0 ! tee name=t
t. ! queue ! videoconvert ! glupload ! glshader ! autovideosink async="FALSE"
t. ! queue ! identity drop-probability=1 ! videoconvert name=conv2 ! openh264enc ! h264parse ! avimux ! multifilesink async="FALSE" post-messages=true next-file=4
Then I just change drop-probability property on identity element
drop-probability = 1 + gst_pad_send_event(conv2_sinkpad, gst_event_new_eos()); - stop recording
drop-probability = 0 - resume recording
I'm using the following pipeline (SIMPLIFIED) in Gstreamer OSS Build 0.10.7 on Win 7 x64:
udpsrc ! application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96 !
gstrtpjitterbuffer latency=200 ! rtph264depay ! tee name=h264Tee
h264Tee. ! queue ! h264parse ! mux.
matroskamux name=mux ! filesink location=rec.mkv sync=false // same for avimux/mp4/qt
h264Tee. ! queue ! ffdec_h264 ! tee name=videoTee
//.videoTee ! queue ! dx9videosink
//.videoTee ! queue ! appsink
//udpsrc ! queue ! directsoundsink
audiotestsrc ! mux. //only for testing, should be connected to udpsrc
The pipeline is launched via Gstreamer-Sharp.
Here's the console output of the pipeline:
WARN default xoverlay.c:354:gst_x_overlay_set_xwindow_id:<videoSink> Using deprecated gst_x_overlay_set_xwindow_id()
ERROR d3dvideosink d3dvideosink.c:2204:gst_d3dvideosink_release_swap_chain: Direct3D device has not been initialized
WARN bin gstbin.c:2378:gst_bin_do_latency_func:<pipeline0> failed to query latency
WARN matroskamux matroska-mux.c:970:gst_matroska_mux_video_pad_setcaps:<mux> pad video_0 refused caps 05370C40
Both video and audio are playing just fine as long as I leave out the muxer. When include the muxer in the pipeline, the video freezes immedeately and no sound can be heard. What's wrong why does the muxer refuse the caps?
Ok solved it my self:
The video caps above don't contain sprop-parameter-sets which aren't needed for playback. For encoding however they are needed since various properties of the stream are encoded within these:
udpsrc !
application/x-rtp, media=(string)video, clock-rate=(int)90000,
encoding-name=(string)H264,
sprop-parameter-sets= (string)\"Z0LADdkBQfsBEAAAAwAQAAADAyjxQqSA\\,aMuMTIA\\=\",
payload=(int)96,
ssrc (uint)2332354585,
clock-base=(uint)1158355497,
seqnum-base=(uint)10049 !
gstrtpjitterbuffer latency=200 ! rtph264depay ! tee name=h264Tee
...
i have created a web server and hosted a AAC stream for HLS streaming, i am able to play the file on another machine using
gst-launch-0.10 souphttpsrc location=http://xx.xx.xx.xx/prog_index.m3u8 ! hlsdemux ! decodebin2 ! alsasink
but when i do this
souphttpsrc location=http://xx.xx.xx.xx/prog_index.m3u8 ! hlsdemux ! aacparse ! faad ! alsasink
i get this error in hlsdemux log and no audio output as aacparse doesn't receive any data
0:00:00.066165787 8139 0xb07098f0 INFO hlsdemux gsthlsdemux.c:734:gst_hls_demux_loop:<hlsdemux0> First fragments cached successfully
0:00:00.066190861 8139 0xb07098f0 DEBUG hlsdemux gsthlsdemux.c:680:switch_pads: Switching pads (oldpad:(nil))
0:00:00.066450610 8139 0xb07098f0 DEBUG hlsdemux gsthlsdemux.c:757:gst_hls_demux_loop:<hlsdemux0> Sending new-segment. segment start:0:00:00.000000000
0:00:00.066510536 8139 0xb07098f0 DEBUG hlsdemux gsthlsdemux.c:796:gst_hls_demux_loop:<hlsdemux0> error, stopping task
Pipeline is PREROLLED ...
0:00:00.066541057 8139 0xb07098f0 DEBUG hlsdemux gsthlsdemux.c:989:gst_hls_demux_stop_update:<hlsdemux0> Stopping updates thread
i am able to play the individual segment file using
gst-launch-0.10 filesrc location=fileSequence0.aac ! aacparse ! faad ! alsasink
try adding a caps: ... ! hlsdemux ! audio/mpeg ! aacparse ! ...
Try adding mpegtsdemux between hlsdemux and aacparse...
because hlsdemux fetches uri till it get .ts streams n u will need mpegtsdemux to demux it.
This is the solution for the HLS audio stream:
... m3u8 ! hlsdemux ! decodebin ! audioconvert ! autoaudiosink
or if you need to stream it again as an UDP:
... m3u8 ! hlsdemux ! decodebin ! audioconvert ! faac ! audio/mpeg, stream-format=raw ! aacparse ! mpegtsmux ! udpsink host=230.0.0.1 port=5000
I'm having a problem trying to record audio+video from my webcam to a file. If I use videotestsrc and autoaudiosrc I get everything right (read as in I get a file with audio recorded from the webcam's mic, and test-video image), but as soon as I replace videotestsrc with v4l2src (or autovideosrc) I get Error starting streaming on device '/dev/video0'.
The command I'm using:
gst-launch-0.10 videotestsrc ! queue ! ffmpegcolorspace! theoraenc ! queue ! oggmux name=mux autoaudiosrc ! queue ! audioconvert ! vorbisenc ! queue ! mux. mux. ! queue ! filesink location = test.ogg
Why is that happening? What am I doing wrong?
EDIT:
In fact, something as simple as
gst-launch-0.10 autovideosrc ! autovideosink autoaudiosrc ! autoaudiosink
is failing with the same error (Error starting streaming on device '/dev/video0')
Replacing autovideosrc with videotestsrc gives me test image + real audio.
Replacing autoauidosrc with audiotestsrc gives me real image + test audio.
I'm starting to think that this is some kind of limitation of my webcam. Is that possible?
EDIT:
GST_DEBUG=2 log here: http://pastie.org/4755009
EDIT 2:
GST_DEBUG="v4l2*:5" (gstreamer 0.10): http://pastie.org/4810519
GST_DEBUG="v4l2*:5" (gstreamer 1.0): http://pastie.org/4810502
Please do a
gst-launch-1.0 v4l2src ! videoscale ! videoconvert ! autovideosink
Does that run? If not repeat as
GST_DEBUG="v4l2*:5" GST_DEBUG_NO_COLOR=1 gst-launch 2>debug.log ...
and check the log for errors. You also might want to run v4l-info (install v4l-conf under debian/ubuntu) and report what formats your camera supports.
hi I am trying to visualize a music file in gstreamer using the following command:
gst-launch filesrc location=file.mp3 ! decodebin ! audioconvert !
tee name=myT myT. ! queue ! autoaudiosink myT. ! queue ! goom !
colorspace ! autovideosink
But I get this error : "There may be a timestamping problem, or this computer is too slow."
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstAudioSinkClock
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstDshowVideoSink:autovideosink0-actual-sink-dshowvideo: A lot of buffers are being dropped.
Additional debug info:
..\Source\gstreamer\libs\gst\base\gstbasesink.c(2572): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstDshowVideoSink:autovideosink0-actual-sink-dshowvideo:
There may be a timestamping problem, or this computer is too slow.
ERROR: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0
Assuming this is something to do with the thread, I tried the following command:
gst-launch filesrc location=file.mp3 ! decodebin ! audioconvert ! tee name=myT
{ ! queue ! autoaudiosink } { tee. ! queue ! goom ! colorspace ! autovideosink }
But then it gives the folloiwng link error:
** (gst-launch-0.10:5308): WARNING **: Trying to connect elements that don't share a common ancestor: tee and queue1
0:00:00.125000000 5308 003342F0 ERROR GST_PIPELINE grammar.tab.c:656:gst_parse_perform_link: could not link tee to queue1
WARNING: erroneous pipeline: could not link tee to queue1
Can anyone tell what is wrong? Thanks
I cannot give you an exact answer because i don't have windows installed.
For debugging this use your first pipeline (in linux works). Use parameter -v with gst-launch and put element identity just before autovideosink. This will print buffer information that passes through element identity, look for anything strange.
Also you could try to use directdrawsink instead of autovideosink. Another test that i will do is to generate the audio with audiotestsrc.
Remember that if you find a bug you can open a bug report in gnome bugzilla so GStreamer developers are aware that there is a problem. Even you could fix it yourself and send a patch.
For There may be a timestamping problem, or this computer is too slow. Error Try sync=false like
`gst-launch filesrc location=file.mp3 ! decodebin ! audioconvert ! tee name=myT myT. ! queue ! autoaudiosink myT. ! queue ! goom ! colorspace ! autovideosink sync=false`
or you may have to try at both sink ends of the Tee like
`gst-launch filesrc location=file.mp3 ! decodebin ! audioconvert ! tee name=myT myT. ! queue ! autoaudiosink sync=false myT. ! queue ! goom ! colorspace ! autovideosink sync=false`
I also observed that if you replace autovideosink with xvimagesink or ximagesink the timestamping problem apparently seems to be solved.