I need to compose a pipeline for "picture-in-picture" effect to combine media from two files:
1) video content from the first file is showed on the full window
2) video from the second file is resized and is showed in the top-left corner of a window,
3) audio from both files mixed
4) the content from both files should be played simultaneously
So far I got the following pipeline:
gst-launch-1.0 -e \
filesrc name="src0" location=$FILE0 \
! decodebin name="decodebin0" ! queue ! videoscale ! capsfilter caps="video/x-raw,width=120" ! videoconvert ! videomixer.sink_0 decodebin0. ! queue ! audioconvert ! audiomixer.sink_0 \
filesrc name="src1" location=$FILE1 \
! decodebin name="decodebin1" ! queue ! videoscale ! capsfilter caps="video/x-raw" ! videoconvert ! videomixer.sink_1 decodebin1. ! queue ! audioconvert ! audiomixer.sink_1 \
videomixer name="videomixer" ! autovideosink \
audiomixer name="audiomixer" ! autoaudiosink
However, it plays streams one by one, not in parallel. Does anyone know what should be changed here in order to play streams simultaneously ?
Ps: attaching the diagram of this pipeline visualized:
Surprisingly - the order of the sources in the pipeline does matter - after slight modification of the pipeline and placing the source with "larger" frame on the first place I was able to get the result as expected:
gst-launch-1.0 -ev \
filesrc name="src1" location=$FILE1 \
! decodebin name="decodebin1" ! queue ! videoscale ! capsfilter caps="video/x-raw,framerate=15/1" ! videoconvert ! videomixer.sink_1 decodebin1. ! queue ! audioconvert name="ac1" \
filesrc name="src0" location=$FILE0 \
! decodebin name="decodebin0" ! queue ! videoscale ! capsfilter caps="video/x-raw,width=120,framerate=15/1" ! videoconvert ! videomixer.sink_0 decodebin0. ! queue ! audioconvert name="ac0"\
ac0. ! audiomixer.sink_0 \
ac1. ! audiomixer.sink_1 \
videomixer name="videomixer" ! autovideosink \
audiomixer name="audiomixer" ! autoaudiosink \
Related
I'm using gstreamer to make a picture-in-picture composition of two rtmp inputs into an rtmp output.
I've managed to create a pipeline that works very well when both streams are offline
However, when one of the rtmp streams is not live when starting the pipeline - the pipeline does not start.
Does anyone know how to overcome this issue, and make sure the pipeline is not blocked if one rtmp source is offline?
You may have to insert identity into compositor input sub-pipelines.
Simulating 2 sources on localhost with:
gst-launch-1.0 videotestsrc ! x264enc insert-vui=1 ! h264parse config-interval=1 ! mpegtsmux ! rtpmp2tpay ! udpsink port=5004
(and second source the same to port 5005), the following runs fine for 0, 1 or 2 sources active at launch time:
gst-launch-1.0 -v \
udpsrc port=5004 ! application/x-rtp,media=video,encoding-name=MP2T,clock-rate=90000,payload=33 ! rtpjitterbuffer latency=300 ! rtpmp2tdepay ! tsdemux ! h264parse ! avdec_h264 ! videoconvert ! identity ! queue ! comp.sink_0 \
udpsrc port=5005 ! application/x-rtp,media=video,encoding-name=MP2T,clock-rate=90000,payload=33 ! rtpjitterbuffer latency=300 ! rtpmp2tdepay ! tsdemux ! h264parse ! avdec_h264 ! videoconvert ! identity ! queue ! comp.sink_1 \
compositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=240 sink_1::xpos=0 sink_1::ypos=240 sink_1::width=320 sink_1::height=240 ! video/x-raw,width=320,height=480 ! videoconvert ! xvimagesink
For rtmp, with mpeg audio from first source, it would be something like:
gst-launch-1.0 -v \
rtmpsrc <your source1 and options> ! flvdemux name=demux0 ! queue ! h264parse ! avdec_h264 ! videoconvert ! identity ! queue ! comp.sink_0 \
rtmpsrc <your source2 and options> ! flvdemux name=demux1 ! queue ! h264parse ! avdec_h264 ! videoconvert ! identity ! queue ! comp.sink_1 \
compositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=240 sink_1::xpos=0 sink_1::ypos=240 sink_1::width=320 sink_1::height=240 ! video/x-raw,width=320,height=480 ! videoconvert ! autovideosink \
demux0. ! queue ! audio/mpeg ! decodebin ! audioconvert ! audioresample ! autoaudiosink
I am trying to get a stream from the webcam and then using tee to get two sinks(filesink and autovideosink) so I can visualise the video in a window and in the same time save it into a folder. When I run this command I get only a frozen image in the window and not a video stream. It works with two autovideosinks(I get two windows with two videostreams) so I guess the problem is in the filesink part. The filesink works perfectly alone.
gst-launch-1.0 -v v4l2src device=/dev/video0 ! tee name=t \
t. ! queue ! videoscale ! video/x-raw,framerate=30/1,width=320,height=240 ! \
videoconvert ! autovideosink \
t. ! queue ! video/x-raw,framerate=30/1,width=320,height=240 ! \
x264enc ! mpegtsmux ! filesink location=~/Videos/test1.mp4
Try to add async=0 property to filesink.
gst-launch-1.0 -v v4l2src device=/dev/video0 ! tee name=t \
t. ! queue ! videoscale ! video/x-raw,framerate=30/1,width=320,height=240 ! \
videoconvert ! autovideosink \
t. ! queue ! video/x-raw,framerate=30/1,width=320,height=240 ! \
x264enc ! mpegtsmux ! filesink **async=0** location=~/Videos/test1.mp4
I am using the following pipeline to convert an flv file to mp4.
gst-launch-1.0 -vvv -e filesrc location="c.flv" ! flvdemux name=demux \
demux.audio ! queue ! decodebin ! audioconvert ! faac bitrate=32000 ! mux. \
demux.video ! queue ! decodebin ! videoconvert ! video/x-raw,format=I420 ! x264enc speed-preset=superfast tune=zerolatency psy-tune=grain sync-lookahead=5 bitrate=480 key-int-max=50 ref=2 ! mux. \
mp4mux name=mux ! filesink location="c.mp4"
The problem is when (for example) audio is missing, the pipeline gets stuck. (Same thing happens if just hooking a fakesink to demux.audio).
I need a way for the filters to ignore missing tracks, or produce empty tracks.
Hi to all i try to play and record mp3 souphttpsrc in the same time but i don't have a good result someone can help please?
gst-launch-1.0 -e filesrc location=/dev/fd/0 ! h264parse ! tee name=myvid \! queue ! decodebin ! xvimagesink sync=false \ myvid. ! queue ! mux.video_0 \ alsasrc device="plughw:2,0" ! "audio/x-raw,rate=44100,channels=1,depth=24" ! audioconvert ! queue ! filesink location=/tmp/out.mp4
thank you
Hi your pipeline is slightly wrong.
There is no encoding happening with the audio so you're saving raw audio into the container.
There is no muxer and mux.video_0 therefore does not resolve to any pad on any element.
Here is a pipeline without these issues:
gst-launch-1.0 -e mp4mux name=mux ! filesink location=/tmp/out.mp4 filesrc location=/dev/fd/0 ! h264parse ! tee name=myvid ! queue ! decodebin ! xvimagesink sync=false myvid. ! queue ! mux.video_0 \ alsasrc device="plughw:2,0" ! "audio/x-raw,rate=44100,channels=1,depth=24" ! audioconvert ! queue ! lame ! mux.audio_0
I am trying to use gstreamer to save images into a video format in real time as they are
being captured. I have the command to save the images. This line is:
gst-launch -e v4l2src device=/dev/video0 ! 'image/jpeg,width=640,height=480,framerate=30/1' ! jpegdec ! timeoverlay halign=right valign=bottom ! clockoverlay halign=left valign=bottom time-format="%Y/%m/%d %H:%M:%S" ! tee name=t ! queue ! sdlvideosink t. ! queue ! videorate ! capsfilter caps="video/x-raw-yuv,framerate=1/1" ! ffmpegcolorspace ! jpegenc ! multifilesink location="./Desktop/frames/frame%06d.jpg"
This command saves the images to a folder. I wrote another command that takes those pictures and saves them to a video. This command is:
gst-launch -e multifilesrc location=./Desktop/frames/frame%06d.jpg ! image/jpeg,framerate=30/1 ! decodebin ! videoscale ! video/x-raw-yuv ! progressreport name=progress ! avimux ! filesink location=test.avi
I need a way of combining these two commands so that the video can be saved in real time. I cannot seem to figure out how this is done.
Thanks!
I took away the multifilesink element from your first line and added avimux and filesink to your second line (and formatted it better for this forum) to produce this:
gst-launch -e v4l2src device=/dev/video0 ! \
'image/jpeg,width=640,height=480,framerate=30/1' ! \
jpegdec ! timeoverlay halign=right valign=bottom ! \
clockoverlay halign=left valign=bottom time-format="%Y/%m/%d %H:%M:%S" ! \
tee name=t ! \
queue ! \
sdlvideosink t. ! \
queue ! \
videorate ! \
capsfilter caps="video/x-raw-yuv,framerate=1/1" ! \
ffmpegcolorspace ! \
jpegenc ! \
avimux ! \
filesink location=test.avi
Not sure if it will work, and it also discards the progressreport component (not sure how it works). If the command line fails, please post the gst-launch console error messages.