Did anybody encounter the above? I was doing dynamic pipeline involving attach/removing a bin containing a queue and a multifilesink plugin. Was trying to record and split the video/audio file in 60 seconds segment. video/audio stream was of H264 video and G711 audio and muxed by matroskamux, which has the property streamable set to be true
Understand that multifilesink could only take streaming stream and there was no timing indexes. But that was fine with me and I could open the file to play them using VLC. However there is the problem of playing the first file produced
It seem to missing header or metadata. Have used mediaInfo to query them (as below)
first file video0.mp4 (cannot be played)
The 2nd-last file (can be played)
The properties i have set for multifilesink
g_object_set(G_OBJECT(multifilesink),
"aggregate-gops", TRUE,
"location", "video%d",
"max-file-duration", 60000000000,
"next-file", 5,
"index", 0,
"post-messages", TRUE,
NULL);
Have tried
adjusting "next-file" and "aggregate-gops", it does not seem to change the matters much
changing file format(cross-finger, though I do not believe it will help). It doesnt help
Have tried with a static pipeline... It seems that all the file can be played properly
Just wonder what did i do wrongly?
Related
I have a gstreamer pipeline created from the python gst bindings, which is set up to play a headset's microphone back to the headset's speaker. This works fine and is playing in a pipeline like this:
JackAudioSrc -> GstAudioMixer -> Queue -> GstJackAudioSink
Then many seconds later I want to play a short 10 second .wav file into the pipeline so the wav file is mixed with the microphone and heard on the headset. To do this, a GstFileSrc is dynamically added to the GstAudioMixer to mix in a short 10 second wav file to the headset's speaker, which gives pipeline like this:
GstJackAudioSrc -> GstAudioMixer -> Queue -> GstJackAudioSink
/
Gstfilesrc -> Gstwavparse ->/
When the Gstfilesrc and Gstwavparse file is dynamically added to a sink pad of the mixer, at a time say 6 seconds since the start of the pipeline, only the last 4 seconds of the wav is heard.
The problem seems to be that the wav file seeks to the time relative to when the pipeline started PLAYING.
I have tried changing "do-timestamp" in a multifilesrc, and GstIndentity "sync"=True, and can't find a way to set "live" on a filesrc, and many others but to no avail.
However, the whole 10 second wav file will play nicely if the pipeline is set to Gst.State.NULL then back to Gst.State.PLAYING when the filesrc is added at 6 seconds. This works as the pipeline time gets set back to zero, but this produces a click on the headset, which is unacceptable.
How can I ensure that the wav file starts playing from the start of the wav file, so that the whole 10 seconds is heard on the headset, if added to the pipeline at any random time?
An Update:
I can now get the timing of the wave file correct by adding a clocksync and setting its timestamp offset, before the wavparse:
nanosecs = pipeline.query_position(Gst.Format.TIME)[1]
clocksync.set_property("ts-offset", nanosecs)
Although the start/stop times are now correct, the wav audio is corrupted and heard as nothing but clicks and blips, but at least it starts playing at the correct time and finishes at the correct time. Note that without the clocksync the wav file audio is perfectly clear, it just starts and stops at the wrong time. So the ts-offset is somehow corrupting the audio.
Why is the audio being corrupted?
So I got this working and the answer is not to use the clocksync, but instead request a mixer sink pad, then call set_offset(nanosecs) on the mixer sink pad, before linking the wavparse to the mixer:
sink_pad = audio_mixer.get_request_pad("sink_%u")
nanosecs = pipeline.query_position(Gst.Format.TIME)[1]
sink_pad.set_offset(nanosecs)
sink_pad.add_probe(GstPadProbeType.IDLE, wav_callback)
def wav_callback(pad, pad_probe_info, userdata):
wavparse.link(audio_mixer)
wav_bin.set_state(Gst.State.PLAYING)
return Gst.PadProbeReturn.REMOVE
Then if the wav file needs to be rewound/replayed:
def replay_wav():
global wav_bin
global sink_pad
wav_bin.seek_simple(Gst.Format.TIME, Gst.SeekFlags.FLUSH, 0)
nanosecs = pipeline.query_position(Gst.Format.TIME)[1]
sink_pad.set_offset(nanosecs)
I have to get a processed video from gstreamer pipe, compress it based on h264 or h265 alg. and then write it to storage. There are some problems in this project that must be handled:
Saved video must be playable by any standard video players such as vlcplaye, windows media player, kmplayer and ...
If for any reason the destination file does not close properly (such as a power outage), the entire file should not be lost and the saved video should be playable until the problem occurs.
My solution to this project with these constraints, is an opencv writer with a gstreamer pipe as follow:
...
std::string gstPipe("appsrc ! videoconvert ! omxh264enc ! "
"splitmuxsink muxer=matroskamux "
"max-size-time=50000000000 location="
"/file/path/save%d.mkv");
cv::Size frameSize(frameWidth, frameHeight);
bool result = videoWriter.open(gstPipe, cv::CAP_GSTREAMER, 0,
fps, frameSize);
This solution splits a video stream into multiple files, but it is needed to save whole video in one file.
Does anyone have a better solution to offer?
Thank you very much in advance for your helps.
I'm using ReSpeaker Mic Array v2.0 on my robot, I used the following git repo: https://github.com/furushchev/respeaker_ros.git to capture the audio received by the speaker. I subscribed to it's raw audio ros topic /audio which is just byte array data(http://docs.ros.org/noetic/api/audio_common_msgs/html/msg/AudioData.html)
How can I write the AudioData message's uint8[] data into a wav file in C++? I would like to play the wav file by other means afterwards.
I saw that in ros audio_common library example it uses gstreamer to do the writing, but I'm quite confused after reading the code(https://github.com/ros-drivers/audio_common/blob/master/audio_capture/src/audio_capture.cpp)
Example that you saw is using Gstremaer's alsasrc to capture audio from mic in this line
_source = gst_element_factory_make("alsasrc", "source");
So Gstreamer's pipeline is internally handling/capturing audio byte array and, in case of input parameters dst_type=="filesink" and format=="wave", encoding it with
_filter = gst_element_factory_make("wavenc", "filter");
and creating .wav file with
_sink = gst_element_factory_make("filesink", "sink");
On the other hand, running that code with input parameters dst_type=="appsink" and format=="wave" actually captures audio bytes again but, instead of writing to file, publishes them on ros topic /audio.
If you cannot (from any reason) use this code with input parameters dst_type=="filesink" and format=="wave", I suppose you will need to use Gstreamer's appsrc element and feed it with bytes from your AudioData message. In that case, the rest of Gstreamer pipeline for encoding and writing to file should remain the same as in the example.
I wrote code to create H.264 stream, which has a loop to generate H.264 encoded frame.
while(true) {
...
x264_encoder_encode(encoder, &buffer, &i_buffer, &pic_in, &pic_out);
...
/*TODO: Write one frame in the buffer to a streamable mp4 file*/
}
Every single time, an H.264 encoded frame is generated and stored in the buffer. How can I write it into a streamable mp4 file directly through the buffer?
I spent lots of time searching for the solution. All I can find is to read stream from a file using
avformat_open_input(&fmtCtx, in_filename, 0, 0)
Is there any way to read directly from buffer without a file?
MP4 is actually not streamable. So in other words, you can't do it at all. I ran in that very problem.
The reason why it won't work is because when you open an mp4 file, you have to have all sorts of parameters, which by default get saved at the end of the file. When you create an MP4, you can always forcibly save that info at the start. However, to know what those parameters are, you need all the data. And without those parameters, the software trying to load the mp4 fails very early on. This is true for some other formats such as webm videos and .m4a or .wav for audio.
What you have to do is stream the actual H.264, possibly using RTSP or a format of your own if you're in control of both sides.
I have problem with recording streams using gstreamer.
I have to write audio and video separately and cut in when signal arrived. I have correctly working video, but still have problems with wav files.
Even simple pipeline in gst-launch don't work correctly. I have wave file and I am trying to split it using multifilesink:
gst-launch filesrc location=test.wav ! multifilesink location=test2%d.wav next-file=4 max-file-size=512000
But final wav files are corrupted while the same pipeline with ts files is working ok:
gst-launch-1.0 filesrc location=test.ts ! multifilesink location=test2%d.ts next-file=4 max-file-size=2000000
multifilesink doesn't know anything about the data it splits up, so it won't take care of adding headers to each of the files it writes.
The reason why your .ts files work is because it was designed to be a streaming format where each separate packet will be treated independently. Therefore, one can just 'tune in' to the stream whenever one likes. The decoder will simply look for the next packet header it finds and start decoding there (for Details have a look at MPEG TS' wiki page.
The WAV file format however was designed as pure file (and not as a streaming) format. Therefore, there's only one header at the start of the file. When you split that file up into multiple files, these headers are missing (the file contains only raw PCM data then).
To work around that issue, you can...
manually copy the .wav header from the first file to all the other ones
use programs that support PCM files and either work directly with them or convert the files (you'll have to set the channel count, sample rate and bitrate manually when opening those files though).
use another, stream oriented file format like .mp3 which comes from the same family of codecs as .ts and also uses a separate 4-byte header for each frame (Keep in mind though that MP3 is a lossy file format).
An example pipeline would be:
gst-launch filesrc location=test.wav ! wavparse ! lame ! multifilesink location=test%d.mp3 next-file=4 max-file-size=100000
If you're willing to use some scripting as well and split the task up into different gst-launch calls, I can offer you another possible way to solve your little problem:
The following script is a Linux bash script. You should be able to translate that to Windows batch script (or a C or python app if you want):
#!/bin/bash -e
# First write the buffer stream to .buff files (annotated using GStreamer's GDP format)
gst-launch -e filesrc location=test.wav ! wavparse ! gdppay ! multifilesink next-file=4 max-file-size=1000000 location=foo%05d.buff
# use the following instead for any other source (e.g. internet radio streams)
#gst-launch -e uridecodebin uri=http://url.to/stream ! gdppay ! multifilesink next-file=4 max-file-size=1000000 location=foo%05d.buff
# After we're done, convert each of the resulting files to proper .wav files with headers
for file in *.buff; do
tgtFile="$(echo "$file"|sed 's/.buff$/.wav/')"
gst-launch-0.10 filesrc "location=$file" ! gdpdepay ! wavenc ! filesink "location=$tgtFile"
done
# Uncomment the following line to remove the .buff files here, but to avoid accidentally
# deleting stuff we haven't properly converted if something went wrong, I'm not gonna do that now.
#rm *.buff
Now to what the script does:
First we're gonna use multifilesink to create a set of .buff files, each under 1MB of size (gdppay will annotate each buffer with its caps; the -e flag of gst-launch will cause it to trigger an EOS if the process gets killed prematurely which is useful if you're reading and decoding an internet stream)
The second gst-launch invocation within the for loop takes one of the .buff files, parses the GDP headers using gdpdepay (and strips them), adds a WAV header and writes the result to a .wav file.
Hope this is a solution you can live with, because I doubt there's a way to do it with one single gst-launch run.