I try to write a GStreamer pipeline to capture the screen, put a box on the corner capturing the webcam and record audio (all at the same time).
If I hit Ctrl+C to stop after ten seconds, for example, I realize I only record about 2 seconds of video (and audio). Actually, I don't care that the recording were done in real time, but I just want that GStreamer records the full lenght it should be.
This is the pipeline I have so far:
gst-launch-1.0 --gst-debug=3 ximagesrc use-damage=0 \
! video/x-raw,width=1366,height=768,framerate=30/1 ! videoconvert \
! videomixer name=mix sink_0::alpha=1 sink_1::alpha=1 sink_1::xpos=1046 sink_1::ypos=528 \
! videoconvert ! queue max-size-buffers=0 max-size-time=0 max-size-bytes=0 \
! vp8enc ! webmmux name=mux ! filesink location="out.webm" \
pulsesrc ! audioconvert ! queue max-size-buffers=0 max-size-time=0 max-size-bytes=0 ! vorbisenc ! mux. \
v4l2src do-timestamp=true ! video/x-raw,width=320,height=240,framerate=30/1 ! mix.
I hope to have a solution, thank you.
Related
I am using something similar to the following pipeline to ingest rtsp stream from a camera and provide original and 360p(transcoded) variants in a manifest. I am dynamically generating this pipeline using gstreamer rust.
The video works fine in web, VLC and ffplay. However it fails with AVPlayer(quicktime).
I found that the issue seems to be the audio/video alignment in the ts segments generated by gstreamer.
How can I ensure that the audio and video are aligned in the ts segments? Can audiobuffersplit be helpful? I am not sure how to use it in a pipeline like mine where hlssink2 internally is muxing it.
Appreciate any help in this! Thanks!
gst-launch-1.0 hlssink2 name=ingest1 playlist-length=5 max-files=10
target-duration=2 send-keyframe-requests=true
playlist-location=/live/stream.m3u8 location=/live/%d.ts \
rtspsrc location=rtsp://admin:password#10.10.10.20:554/ protocols=4
name=rtspsrc0 rtspsrc0. ! rtph264depay ! tee name=t t.! queue !
ingest1.video \
t.! queue ! decodebin name=video_decoder ! tee name=one_decode \
one_decode. ! queue ! videorate ! video/x-raw,framerate=15/1 !
videoscale ! video/x-raw,width=640,height=360 ! vaapih264enc ! hlssink2
name=ingest2 target-duration=2 playlist-location=/live/360/stream.m3u8
location=/live/360/%d.ts \
rtspsrc0. ! decodebin name=audio_decoder ! fdkaacenc ! tee name=audio_t \
audio_t. ! queue ! ingest1.audio \
audio_t. ! queue ! ingest2.audio
gst-launch-1.0 -v -e \
videotestsrc ! tee name=t0 \
t0. ! queue ! x264enc ! matroskamux ! filesink location="test.mkv" \
t0. ! queue ! queue ! autovideosink
Works, with both the file, and the on-screen display working
gst-launch-1.0 -v -e \
videotestsrc ! tee name=t0 \
t0. ! queue ! x264enc ! matroskamux ! filesink location="test.mkv" \
t0. ! queue ! autovideosink
Does not work.
Here's another set of examples.
gst-launch-1.0 -v -e \
videotestsrc ! tee name=t0 \
t0. ! queue ! autovideosink \
t0. ! queue ! autovideosink
Works.
gst-launch-1.0 -v -e \
videotestsrc ! tee name=t0 \
t0. ! queue ! autovideosink \
t0. ! autovideosink
Doesn't. Why not? Why do both outputs from the tee need to be queues? At worst, I'd expect one autovideosink to work and the other to be blank, but instead one displays a single frame and the other is black.
But the following DOES work. What's going on?
gst-launch-1.0 -v -e \
videotestsrc ! tee name=t0 \
t0. ! queue ! autovideosink \
t0. ! queue ! autovideosink \
t0. ! autovideosink
Why does adding a third output negate the need for a queue on all of them?
gst-launch-1.0 --version
gst-launch-1.0 version 1.12.4
GStreamer 1.12.4
https://packages.gentoo.org/package/media-libs/gstreamer
Does anyone know why queue behaves like this?
This is the pipeline that I'm trying to make. The above are just minified examples.
(Note: the weird caps in the first line of the pipeline are to make sure my Logitech c920 camera outputs h264 instead of raw, and that my Logitech BRIO outputs video jpeg at 1080p, instead of raw at 720p. This has been tested, and works much better than simply "decodebin")
gst-launch-1.0 -e \
v4l2src device=/dev/video0 ! 'video/x-h264;image/jpeg;video/x-raw' ! decodebin ! 'video/x-raw' ! tee name=t0 \
v4l2src device=/dev/video1 ! 'video/x-h264;image/jpeg;video/x-raw' ! decodebin ! 'video/x-raw' ! tee name=t1 \
v4l2src device=/dev/video2 ! 'video/x-h264;image/jpeg;video/x-raw' ! decodebin ! 'video/x-raw' ! tee name=t2 \
v4l2src device=/dev/video3 ! 'video/x-h264;image/jpeg;video/x-raw' ! decodebin ! 'video/x-raw' ! tee name=t3 \
matroskamux name=mux \
t0. ! queue ! autovideoconvert ! x264enc ! mux. \
t1. ! queue ! autovideoconvert ! x264enc ! mux. \
t2. ! queue ! autovideoconvert ! x264enc ! mux. \
t3. ! queue ! autovideoconvert ! x264enc ! mux. \
mux. ! queue ! filesink location="test.mkv" \
videomixer name=mix \
sink_0::zorder=1 sink_0::alpha=1.0 sink_0::ypos=0 sink_0::xpos=0 \
sink_1::zorder=1 sink_1::alpha=1.0 sink_1::ypos=0 sink_1::xpos=960 \
sink_2::zorder=1 sink_2::alpha=1.0 sink_2::ypos=540 sink_2::xpos=0 \
sink_3::zorder=1 sink_3::alpha=1.0 sink_3::ypos=540 sink_3::xpos=960 \
t0. ! queue ! autovideoconvert ! video/x-raw, width=960, height=540 ! mix.sink_0 \
t1. ! queue ! autovideoconvert ! video/x-raw, width=960, height=540 ! mix.sink_1 \
t2. ! queue ! autovideoconvert ! video/x-raw, width=960, height=540 ! mix.sink_2 \
t3. ! queue ! autovideoconvert ! video/x-raw, width=960, height=540 ! mix.sink_3 \
mix. ! queue ! autovideosink sync=false
This question was solved by adding max-size-bytes=0 max-size-buffers=0 max-size-time=10000000000 to the queue.
For anyone not initiated into the gstreamer low level bits, this is incredibly counter-intuitive. But if it works it works, I guess.
Read about the concept of PREROLLING in GStreamer:
https://gstreamer.freedesktop.org/documentation/design/preroll.html
A sink element can only complete the state change to PAUSED after a
buffer has been queued on the input pad or pads.
What is not emphasized in the documentation is that the pipeline will only transition from PAUSED to PLAYING after all sinks have PREROLLED.
Also note that a tee is not threaded, so it is sequentially pushing samples downstream.
Here is what happens: sink 1 receives a sample, but will not start playing because it waits until all other sinks in the pipeline have received a sample so audio/video sync can be respected.
So now that sink 1 is waiting it is effectively blocking the tee preventing it from sending more data - in that case so sink 2. Since no data will ever reach sink 2 you are in a deadlock.
A queue will automatically add a thread in the pipeline path as a side effect - preventing the deadlock.
If you have only one queue it may actually work - depending in which order you connect your sinks to the tee. If the path with the queue is delivered first it won't deadlock and the tee can deliver data to the other one and the state change will be successful. (Same as the example with three sinks, if all paths have a queue but not the last you may get away with it)
It is good practice to use queues for all tee outputs.
The x264enc example is especially tricky. The problem you face here is that the encoder consumes too much data but not producing anything (yet) effectively stalling the pipeline.
Two ways to fix it:
use tune=zerolatency for the x264enc element
increase the buffer sizes in the queue of the non-encoder path to compensate for the encoder latency.
With queue ! queue you are actually doing case 2. by doubling the buffer sizes.
I need to compose a pipeline for "picture-in-picture" effect to combine media from two files:
1) video content from the first file is showed on the full window
2) video from the second file is resized and is showed in the top-left corner of a window,
3) audio from both files mixed
4) the content from both files should be played simultaneously
So far I got the following pipeline:
gst-launch-1.0 -e \
filesrc name="src0" location=$FILE0 \
! decodebin name="decodebin0" ! queue ! videoscale ! capsfilter caps="video/x-raw,width=120" ! videoconvert ! videomixer.sink_0 decodebin0. ! queue ! audioconvert ! audiomixer.sink_0 \
filesrc name="src1" location=$FILE1 \
! decodebin name="decodebin1" ! queue ! videoscale ! capsfilter caps="video/x-raw" ! videoconvert ! videomixer.sink_1 decodebin1. ! queue ! audioconvert ! audiomixer.sink_1 \
videomixer name="videomixer" ! autovideosink \
audiomixer name="audiomixer" ! autoaudiosink
However, it plays streams one by one, not in parallel. Does anyone know what should be changed here in order to play streams simultaneously ?
Ps: attaching the diagram of this pipeline visualized:
Surprisingly - the order of the sources in the pipeline does matter - after slight modification of the pipeline and placing the source with "larger" frame on the first place I was able to get the result as expected:
gst-launch-1.0 -ev \
filesrc name="src1" location=$FILE1 \
! decodebin name="decodebin1" ! queue ! videoscale ! capsfilter caps="video/x-raw,framerate=15/1" ! videoconvert ! videomixer.sink_1 decodebin1. ! queue ! audioconvert name="ac1" \
filesrc name="src0" location=$FILE0 \
! decodebin name="decodebin0" ! queue ! videoscale ! capsfilter caps="video/x-raw,width=120,framerate=15/1" ! videoconvert ! videomixer.sink_0 decodebin0. ! queue ! audioconvert name="ac0"\
ac0. ! audiomixer.sink_0 \
ac1. ! audiomixer.sink_1 \
videomixer name="videomixer" ! autovideosink \
audiomixer name="audiomixer" ! autoaudiosink \
I am trying to use gstreamer to save images into a video format in real time as they are
being captured. I have the command to save the images. This line is:
gst-launch -e v4l2src device=/dev/video0 ! 'image/jpeg,width=640,height=480,framerate=30/1' ! jpegdec ! timeoverlay halign=right valign=bottom ! clockoverlay halign=left valign=bottom time-format="%Y/%m/%d %H:%M:%S" ! tee name=t ! queue ! sdlvideosink t. ! queue ! videorate ! capsfilter caps="video/x-raw-yuv,framerate=1/1" ! ffmpegcolorspace ! jpegenc ! multifilesink location="./Desktop/frames/frame%06d.jpg"
This command saves the images to a folder. I wrote another command that takes those pictures and saves them to a video. This command is:
gst-launch -e multifilesrc location=./Desktop/frames/frame%06d.jpg ! image/jpeg,framerate=30/1 ! decodebin ! videoscale ! video/x-raw-yuv ! progressreport name=progress ! avimux ! filesink location=test.avi
I need a way of combining these two commands so that the video can be saved in real time. I cannot seem to figure out how this is done.
Thanks!
I took away the multifilesink element from your first line and added avimux and filesink to your second line (and formatted it better for this forum) to produce this:
gst-launch -e v4l2src device=/dev/video0 ! \
'image/jpeg,width=640,height=480,framerate=30/1' ! \
jpegdec ! timeoverlay halign=right valign=bottom ! \
clockoverlay halign=left valign=bottom time-format="%Y/%m/%d %H:%M:%S" ! \
tee name=t ! \
queue ! \
sdlvideosink t. ! \
queue ! \
videorate ! \
capsfilter caps="video/x-raw-yuv,framerate=1/1" ! \
ffmpegcolorspace ! \
jpegenc ! \
avimux ! \
filesink location=test.avi
Not sure if it will work, and it also discards the progressreport component (not sure how it works). If the command line fails, please post the gst-launch console error messages.
I'm using Gstreamer to take 4 videos (MKV container, Mjpeg codec, 25 frames per second, 5 minutes long) to generate a "wall" of videos (basically a 2x2 matrix). I'm using the following pipeline:
#!/bin/sh
gst-launch -e videomixer name=mix ! ffmpegcolorspace ! jpegenc ! avimux ! filesink location=$1.avi \
uridecodebin uri="file://${PWD}/$1/1.mkv" ! videoscale ! videorate ! video/x-raw-yuv,width=300,height=200, framerate=25/1 ! videobox border-alpha=0 top=0 left=0 ! mix. \
uridecodebin uri="file://${PWD}/$1/2.mkv" ! videoscale ! videorate ! video/x-raw-yuv,width=300,height=200,framerate=25/1 ! videobox border-alpha=0 top=0 left=-300 ! mix. \
uridecodebin uri="file://${PWD}/$1/3.mkv" ! videoscale ! videorate ! video/x-raw-yuv,width=300,height=200,framerate=25/1 ! videobox border-alpha=0 top=-200 left=0 ! mix. \
uridecodebin uri="file://${PWD}/$1/4.mkv" ! videoscale ! videorate ! video/x-raw-yuv,width=300,height=200,framerate=25/1 ! videobox border-alpha=0 top=-200 left=-300 ! mix. \
The code works, but the end result is only 17 seconds long instead of 5 minutes like the source videos and it doesn't seem like I'm using the videorate element properly -- the output video seems to randomly "speed up", reading frames as they become available instead of maintaining the speed of the original videos.
Interestingly enough, when the source files are .wmv (Windows Media 9 codec) everything appears to be working just fine. Any ideas?
Try putting your capsfilter in quotes ... videorate ! "video/x-raw-yuv,width=300,height=200, framerate=25/1" ! videobox ...
Also try videomixer2