I am super new to OpenVINO and GStreamer. I am trying to apply a pipeline in GStreamer plugin using OpenVINO. I downloaded the model as well it's proc files. I am trying to apply the model and it's proc to a video. For some reason I am not getting any output in the video.
I don't know how to debug the GStreamer pipeline. I added one text overlay for debugging and found out every step is working but the final output is not having any results. I get the same input video as output (with text overlay). Could anyone help me out here?
# Model is downloaded from model downloader's downloader.py
# Model-proc is downloaded from
# https://github.com/openvinotoolkit/dlstreamer_gst/blob/master/samples/model_proc/intel/human_pose_estimation/human-pose-estimation-0001.json
VIDEO_FILE = 'HumanPoseEstimation/Sample_Video/face-demographics-walking.mp4'
ESTIMATION_MODEL = "HumanPoseEstimation/Models/intel/human-pose-estimation-0001/FP32/human-pose-estimation-0001.xml"
ESTIMATION_MODEL_PROC = "HumanPoseEstimation/Models/intel/human-pose-estimation-0001/human-post-estimation-0001-proc.json"
SINK_LOCATION = "HumanPoseEstimation/output.mp4"
# GStreamer pipeline
GST_PIPELINE = f'''
filesrc location={VIDEO_FILE}
! decodebin
! videoconvert
! queue
! gvaclassify model={ESTIMATION_MODEL} model-proc={ESTIMATION_MODEL_PROC} device=CPU
! queue
! textoverlay text="Human Pose Estimation"
! gvawatermark
! x264enc
! filesink location={SINK_LOCATION}
'''
You may be missing some part at the end of the pipeline for muxing into a mp4-compliant file container. You may try:
...
! x264enc
! h264parse
! qtmux
! filesink location={SINK_LOCATION}
You can use the Human Pose Estimation Sample to run with human-pose-estimation-0001 model.
Refer to human_pose_estimation.sh for the pipeline elements used in the Human Pose Estimation Sample.
Looks like you need to add "inference-region=full-frame" to your gvaclassify, like:
! gvaclassify model={ESTIMATION_MODEL} model-proc={ESTIMATION_MODEL_PROC} device=CPU inference-region=full-frame
The use of classification (gvaclassify) without a prior "detection" (resulting in at least one region-of-interest) is unusual...
That's why you would need to specify to use the full-frame for doing the classification (instead of typically only to classify a previously detected "object").
Related
I have this working but have been unable to get video from my magwell to intergrate and could use help with the correct pipline.
gst-launch-1.0 videotestsrc ! video/x-raw,width=848,height=480,framerate=25/1 ! x264enc bitrate=700 ! video/x-h264,width=848,height=480,framerate=25/1,stream-format=byte-stream,profile=baseline ! tee name=t\
t. ! queue ! tcpclientsink host=172.18.0.3 port=8000 \
t. ! queue ! tcpclientsink host=172.18.0.4 port=8000
I do not see the receiver side pipeline in the question description. This is required to verify that there are no issues at the receiver side. Based on your current pipeline I have the following suggestions:
You don't need set the caps again after the element x264enc, because the output is anyhow of type video/x-h264. What you need is to add h264parse after x264enc. You need to also add h264parse, before passing the data to decoder you are using at the receiver side.
The bitrate set for x264enc is also very less. The units are in kbits/sec, and for a video this might be very less. It's best to leave this to default setting if you do not have any strict resource constraints. Otherwise try for a higher value.
Also is there any reason why you are using TCP. Using UDP might be a better idea for video, in case video data/packet loss is not an issue.
I'm looking to decode and demux an mp4 file with gst-launch-1.0. Instead of using a bin - decodebin - I'd rather work with the seperate elements. Unfortunately, I did not find this.
My question is simple: what basic elements are contained in the decodebin?
If you can direct me to a place where I can find the composition of other bins or autopluggers that whould also be nice.
The decodebin will use all available elements in your gstreamer installation. Remember that you can launch the pipeline with decodebin and using verbose -v and guess what elements is the decodebin creating. For example, in the next pipeline that plays succesfully a mp4 file (video and audio):
gst-launch-1.0 -v filesrc location=/home/usuario/GST_/BigBuckBunny_320x180.mp4 ! queue ! qtdemux name=demuxer demuxer.video_0 ! queue ! decodebin ! videoconvert ! autovideosink demuxer.audio_0 ! queue ! decodebin ! audioconvert ! autoaudiosink
Watching the output I can conclude that the resulting pipeline is:
gst-launch-1.0 -v filesrc location=/home/usuario/GST_/BigBuckBunny_320x180.mp4 ! queue ! qtdemux name=demuxer demuxer.video_0 ! queue ! h264parse ! avdec_h264 ! videoconvert ! autovideosink demuxer.audio_0 ! queue ! aacparse ! avdec_aac ! audioconvert ! autoaudiosink
The playback components from gstreamer are available here. The playbin element will give you the full pipeline (video, audio, etc...) from the uri input.
For example, if you even don't know what kind of source you have, you can use playbin element:
gst-launch-1.0 playbin uri=file:///home/usuario/GST_/BigBuckBunny_320x180.mp4 -v
This will play automatically the file (if it is possible), and verbose output will show you the used plugins and status information.
gst-launch-1.0 can create .dot file with pipeline diagram every time pipeline changes state. To enable this functionality, set GST_DEBUG_DUMP_DOT_DIR variable to path where generated files should be saved. In this dir gst-launch-1.0 will create files like 0.00.00.069441527-gst-launch.READY_PAUSED.dot. You can then convert them to .png files using dot from ghraphviz package. To convert one file, use following command:
dot -Tpng 0.00.00.069441527-gst-launch.READY_PAUSED.dot -o0.00.00.069441527-gst-launch.READY_PAUSED.png
You also can convert them all, using following command in bash shell:
ls -1 *.dot | xargs -I{} dot -Tpng {} -o{}.png
You can find more details here:
How to generate a Gstreamer pipeline diagram (graph)
I'm trying to use a jpg-File as a virtual webcam for Skype (or similar). The image file is reloading every few seconds and the Pipeline should also transmit always the newest image.
I started creating a Pipeline like this
gst-launch filesrc location=~/image.jpg ! jpegdec ! ffmpegcolorspace ! freeze ! v4l2sink device=/dev/video2
but it only streams the first image and ignores the newer versions of the image file. I read something about concat and dynamically changing the Pipeline but I couldn't get this working for me.
Could you give me any hints on how to get this working?
Dynamic refresh the input file is NOT possible (at least with filesrc).
Besides, your sample use freeze, which will prevent the image change.
One possible method is using multifilesrc and videorate instead.
multifilesrc can read many files (with a provided pattern similar to scanf/printf), and videorate can control the speed.
For example, you create 100 images with format image0000.jpg, image0001.jpg, ..., image0100.jpg. Then play them continuously, with each image in 1 second:
gst-launch multifilesrc location=~/image%04d.jpg start-index=0 stop-index=100 loop=true caps="image/jpeg,framerate=\(fraction\)1/1" ! jpegdec ! ffmpegcolorspace ! videorate ! v4l2sink device=/dev/video2
Changing the number of image at stop-index=100, and change speed at caps="image/jpeg,framerate=\(fraction\)1/1"
For more information about these elements, refer to their documents at gstreamer.freedesktop.org/documentation/plugins.html
EDIT: Look like you use GStreamer 0.10, not 1.x
In this case, please refer to old documents multifilesrc and videorate
You can use a general file name with multifilesrc if you add some parameter adjustments and pair it with an identity on a delay. It's a bit fragile but it'll do fine for a temporary one-off program as long as you keep your input images the same dimensions and format.
gst-launch-1.0 multifilesrc loop=true start-index=0 stop-index=0 location=/tmp/whatever ! decodebin ! identity sleep-time=1000000 ! videoconvert ! v4l2sink
I need to capture a video using a webcam and output a single image for each video frame captured.
I have tried using gstreamer with a multifilesink, e.g.:
gst-launch v4l2src device=/dev/video1 ! video/x-raw-yuv,framerate=30/1 ! ffmpegcolorspace ! pngenc ! multifilesink location="frame%d.png"
However, this does not actually output every frame, meaning that if I record for 2 seconds at 30 fps, I don't get 60 images. I'm assuming this is because the encoding can't go that fast, so I need another method.
I figured it might work if I have one pipeline capture a video, and a separate pipeline convert that video to frames, but I don't know enough about codecs. Do I need to encode the video to a file like h264 or mp4 just to then decode it again?
Does anyone have any thoughts or suggestions? Keep in mind that I need to be able to do this in code, not using an application like Adobe Premiere, for example.
Thanks!
You could simply add a queue in there like this:
gst-launch v4l2src device=/dev/video1 ! video/x-raw-yuv,framerate=30/1 ! queue ! ffmpegcolorspace ! pngenc ! multifilesink location="frame%d.png"
This should make sure the video-capture is allowed to run at 30 fps, and then writing it to
disk can happen in its own tempo. Just be aware that the queue will grow to quite a large size
if you leave this setup for too long.
the solution I have to offer doesn't use gstreamer but ffmpeg. I hope that's fine for you too.
As described in this forum post, you can use something like this:
ffmpeg -i movie.avi frame%d.png
to get a png/jpg image for each frame of the video.
But depending on the input file you use, you might have to convert it to an MPEG vid before running ffmpeg.
Note:
If you want leading zeroes in your image file names, use %05d instead (for 5-digit numbers, like in C's printf()):
ffmpeg -i movie.avi frame%05d.png
The output file format depends on the file extension, so you might use .jpg, .bmp, ... instead of .png.
I ended up doing this in two parts.
Write video to file.
gst-launch v4l2src device=/dev/video2 ! video/x-raw-yuv,framerate=30/1 ! xvidenc ! queue ! avimux ! filesink location=test.avi
Post process.
gst-launch-1.0 --gst-debug-level=3 filesrc location=test.avi ! decodebin ! queue ! autovideoconvert ! pngenc ! multifilesink location="frame%d.png"
I am trying to run certain pipelines on the Command prompt for playing a video and I am often getting these errors/messages/warnings :
WARNING: erroneous pipeline: no element "qtdemux"
WARNING: erroneous pipeline: no element "playbin2"
WARNING: erroneous pipeline: no element "decodebin2"
ERROR: pipeline could not be constructed: no element "playbin".
Following are the pipelines :
gst-launch filesrc location=path to the mp4 file ! playbin2 ! queue ! ffmpegcolorspace ! autovideosink
or
gst-launch -v filesrc location=path to the mp4 file ! qtdemux name=demuxer ! { queue ! decodebin ! sdlvideosink } { demuxer. ! queue ! decodebin ! alsasink }
or
gst-launch -v playbin uri=path to the mp4 file
or
gst-launch -v playbin2 uri=path to the mp4 file
Questions
I wanted to know, if I am I missing the plugins to execute this.
How do I know which plugin is responsible for which or found where?
What is the benefit of implementing the pipeline via c code.Are the missing plugins still required.
Is it good to install the missing plugins form the Synaptic manager or form the Gstreamer site(base,good,bad,ugly)
When we do gst-inspect we get output like this:
postproc: postproc_hdeblock: LibPostProc hdeblock filter
libvisual: libvisual_oinksie: libvisual oinksie plugin plugin v.0.1
flump3dec: flump3dec: Fluendo MP3 Decoder (liboil build)
vorbis: vorbistag: VorbisTag
vorbis: vorbisparse: VorbisParse
vorbis: vorbisdec: Vorbis audio decoder
vorbis: vorbisenc: Vorbis audio encoder
coreindexers: fileindex: A index that stores entries in file
coreindexers: memindex: A index that stores entries in memory
amrnb: amrnbenc: AMR-NB audio encoder
amrnb: amrnbdec: AMR-NB audio decoder
audioresample: audioresample: Audio resampler
flv: flvmux: FLV muxer
flv: flvdemux: FLV Demuxer
What does the x : y ( x and y mean ) ?
Answers,
It looks like gstreamer at your ends was not installed correctly. playbin2, decodebin2 are basic and part of the base plugins
1 Yes you may be missing some plugins
2 Use gst-inspect command to check if it is available
3 From C code you can manage states, register callback, learn more
Yes missing plugins are still required
4 I guess gstreamer site would be better
5 Not sure about this one, would help if you arrange the result in a proper way
Most probably the GST_PLUGIN_PATH is incorrect. Please set the correct path to where the gstremer has been installed.