I am using cImg to generate images in a CImgList.
when I save that stack to video by calling the save_video method on the stack, it ignores the fps and the output seems to be always 25fps. I opened the file in different players (VLC, windows movie,...) and its always 25fps.
cImg is using ffmpeg to create the video. I'm not specifying any codec so i assume the default mpeg2 is used (based on what VLC tells me).
I also have no specfic settings for cImg.
I have a fixed amount of images of 500 and it always produces around 20 seconds which is 25fps.
What do I need to do to output it to for example 60fps?
I fixed it but not sure if it's a bug in cImg or just me not knowing how to use it properly but from what I found is that cImg doesn't pass the --framerate parameter to ffmpeg, only the -r parameter.
I updated the code to include the framerate parameter and it does seem to work. This is the updated cImg code in the save_ffmpeg_external function:
cimg_snprintf(command,command._width,
"\"%s\" -framerate %u -v -8 -y -i \"%s_%%6d.ppm\" -pix_fmt yuv420p -vcodec %s -b %uk -r %u \"%s\"",
cimg::ffmpeg_path(),
fps,
CImg<charT>::string(filename_tmp)._system_strescape().data(),
_codec,bitrate,fps,
CImg<charT>::string(filename)._system_strescape().data());
Related
I've made a C++ program that lives in gke and takes some videos as input using ffmpeg, then does something with that input using opengl(not relevant), then finally encodes those edited videos as a single output. Normally the program works perfectly fine on my local machine, it encodes just as I want it to with no warnings or valgrind errors whatsoever. Then, after encoding the said video, I want my program to upload that video to the google cloud storage. This is where the problem comes, I have tried 2 methods for this: First, I tried using curl to upload to the cloud using a signed url. Second, I tried mounting the google storage using gcsfuse(I was already mounting the bucket to access the inputs in question). Both of those methods yielded undefined, weird behaviour's ranging from: Outputing a 0byte or 44byte file, (This is the most common one:) encoding in the correct file size ~500mb but the video is 0 seconds long, outputing a 0.4 second video or just encoding the desired output normally (really rare).
From the logs I can't see anything unusual, everything seems to work fine and ffmpeg does not give any errors or warnings, so does valgrind. Everything seems to work normally, even when I use curl to upload the video to the cloud the output is perfectly fine when it first encodes it (before sending it with curl) but the video gets messed up when curl uploads it to the cloud.
I'm using the muxing.c example of ffmpeg to encode my video with the only difference being:
void video_encoder::fill_yuv_image(AVFrame *frame, struct SwsContext *sws_context) {
const int in_linesize[1] = { 4 * width };
//uint8_t* dest[4] = { rgb_data, NULL, NULL, NULL };
sws_context = sws_getContext(
width, height, AV_PIX_FMT_RGBA,
width, height, AV_PIX_FMT_YUV420P,
SWS_BICUBIC, 0, 0, 0);
sws_scale(sws_context, (const uint8_t * const *)&rgb_data, in_linesize, 0,
height, frame->data, frame->linesize);
}
rgb_data is the data I got after editing the inputs. Again, this works fine and I don't think there are any errors here.
I'm not sure where the error is and since the code is huge I can't provide a replicable example. I'm just looking for someone to point me to the right direction.
Running the cloud's output in mplayer wields this result (This is when the video is the right size but is 0 seconds long, the most common one.):
MPlayer 1.4 (Debian), built with gcc-11 (C) 2000-2019 MPlayer Team
do_connect: could not connect to socket
connect: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.
Playing /media/c36c2633-d4ee-4d37-825f-88ae54b86100.
libavformat version 58.76.100 (external)
libavformat file format detected.
[mov,mp4,m4a,3gp,3g2,mj2 # 0x7f2cba1168e0]moov atom not found
LAVF_header: av_open_input_stream() failed
libavformat file format detected.
[mov,mp4,m4a,3gp,3g2,mj2 # 0x7f2cba1168e0]moov atom not found
LAVF_header: av_open_input_stream() failed
RAWDV file format detected.
VIDEO: [DVSD] 720x480 24bpp 29.970 fps 0.0 kbps ( 0.0 kbyte/s)
X11 error: BadMatch (invalid parameter attributes)
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory
[vdpau] Error when calling vdp_device_create_x11: 1
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
libavcodec version 58.134.100 (external)
[dvvideo # 0x7f2cb987a380]Requested frame threading with a custom get_buffer2() implementation which is not marked as thread safe. This is not supported anymore, make your callback thread-safe.
Selected video codec: [ffdv] vfm: ffmpeg (FFmpeg DV)
==========================================================================
Load subtitles in /media/
==========================================================================
Opening audio decoder: [libdv] Raw DV Audio Decoder
Unknown/missing audio format -> no sound
ADecoder init failed :(
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
[dvaudio # 0x7f2cb987a380]Decoder requires channel count but channels not set
Could not open codec.
ADecoder init failed :(
ADecoder init failed :(
Cannot find codec for audio format 0x56444152.
Audio: no sound
Starting playback...
[dvvideo # 0x7f2cb987a380]could not find dv frame profile
Error while decoding frame!
[dvvideo # 0x7f2cb987a380]could not find dv frame profile
Error while decoding frame!
V: 0.0 2/ 2 ??% ??% ??,?% 0 0
Exiting... (End of file)
Edit: Since the code runs on a VM, I'm using xvfb-run ro start my application, but again even when using xvfb-run it works completely fine on when not encoding to the cloud.
Apparently, I'm assuming for security reasons, the google cloud storage does not allow us to do multiple continuous operations on a file, just a singular read/write operation. So I found a workaround by encoding my video to a local file inside the pod and then doing a copy operation to the cloud.
Hi stackoverflow community,
I have a tricky problem and I need your help to understand what is going on here.
My program captures frames from a video grabber card (Blackmagic) which just works fine so far, at the same time I display the captured images with opencv (cv::imshow) which works good as well (But pretty cpu wasting).
The captured images are supposed to be stored on the disk as well, for this I put the captured Frames (cv::Mat) on a stack, to finally write them async with opencv:
cv::VideoWriter videoWriter(path, cv::CAP_FFMPEG, fourcc, fps, *size);
videoWriter.set(cv::VIDEOWRITER_PROP_QUALITY, 100);
int id = metaDataWriter.insertNow(path);
while (this->isRunning) {
while (!this->stackFrames.empty()) {
cv:Mat m = this->stackFrames.pop();
videoWriter << m;
}
}
videoWriter.release();
This code is running in an additional thread and will be stopped from outside.
The code is working so far, but it is sometimes pretty slow, which means my stack size increases and my system runs out of ram and get killed by the OS.
Currently it is running on my developing system:
Ubuntu 18.04.05
OpenCV 4.4.0 compiled with Cuda
Intel i7 10. generation 32GB RAM, GPU Nvidia p620, M.2 SSD
Depending on the codec (fourcc) this produces a high CPU load. So far I used mainly "MJPG", "x264". Sometimes even MJPG turns one core of the CPU to 100% load, and my stack raises until the programs run out of run. After a restart, sometimes, this problem is fixed, and it seems the load is distributed over all cores.
Regarding to the Intel Doc. for my CPU, it has integrated hardware encoding/decoding for several codecs. But I guess opencv is not using them. Opencv even uses its own ffmpeg and not the one of my system. Here is my build command of opencv:
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D WITH_TBB=ON \
-D WITH_CUDA=ON \
-D BUILD_opencv_cudacodec=OFF \
-D ENABLE_FAST_MATH=1 \
-D CUDA_FAST_MATH=1 \
-D WITH_CUBLAS=1 \
-D WITH_V4L=ON \
-D WITH_QT=OFF \
-D WITH_OPENGL=ON \
-D WITH_GSTREAMER=ON \
-D OPENCV_GENERATE_PKGCONFIG=ON \
-D OPENCV_ENABLE_NONFREE=ON \
-D WITH_FFMPEG=1 \
-D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules \
-D WITH_CUDNN=ON \
-D OPENCV_DNN_CUDA=ON \
-D CUDA_ARCH_BIN=6.1 ..
I just started development with linux and C++, before I was working with Java/Maven, so the use of cmake is still a work in progress, pls go easy on me.
Basically my question is, how can I make the video encoding/writing faster, use the hardware acceleration at best?
Or if you think there is something else fishy, pls let me know.
BR Michael
-------- old - look up answer on bottom --------
Thank #Micka for the many advises, I found the right thing on the way.
Using cudacodec::VideoWriter is not that easy, after compiling I was not able to use it because of this error, and even if I can make it run, the deployment PC does not have a nvidia GPU.
Since I am going to use PCs with AMD CPUs as well, I can't use the cv::CAP_INTEL_MFX for the api-reference parameter of the cv::VideoWriter.
But there is also the cv::CAP_OPENCV_MJPEG, which works fine for the MJPG codec (not all video container are supported, I use .avi, sadly .mkv was not working with this configuration). If the user does not use MJPG as a codec I use cv::CAP_ANY, and opencv decides what is to use.
So,
cv::VideoWriter videoWriter(path, cv::CAP_OPENCV_MJPEG, fourcc, fps, *size);
works pretty well, even on my old system.
Unfortunately I never changed the api-reference parameter before, only from ffmpeg to gstreamer, I read in the doc of opencv only the last line "cv::CAP_FFMPEG or cv::CAP_GSTREAMER." and I did not see that there is an "e.g." before...
Thank you #Micka to make me read again.
P.S. for my performance problem with cv::imshow I changed from
cv::namedWindow(WINDOW_NAME, cv::WINDOW_NORMAL);
to
cv::namedWindow(WINDOW_NAME, cv::WINDOW_OPENGL);
Which obviously uses OpenGL, and does a better job. Also changing from cv::Mat to cv::UMat can speed up the performance, see here
-------------- EDIT better solution ----------------
Since I still had problems with the OpenCV VideoWriter for some systems, I was looking for another solution. Now I write the frames with FFMPEG.
For FFMPEG I can use the GPU or CPU depending on the codec I use.
If FFMPEG is installed via snapd (Ubuntu 18.04) it comes with cuda enabled by default:
sudo snap install ffmpeg --devmode
(--devmode is optional, but I had problems writing files on specific location, this was the only way for me to fix it)
And here is my code:
//this string is automatically created in my program, depending on user input and the parameters of the input frames
string ffmpegCommand = "ffmpeg -y -f rawvideo -vcodec rawvideo -framerate 50 -pix_fmt bgr24 -s 1920x1080 -i - -c:v h264_nvenc -crf 14 -maxrate:v 10M -r 50 myVideoFile.mkv";
FILE *pipeout = popen(ffmpegCommand.data(), "w");
int id = metaDataWriter.insertNow(path);
//loop will be stopped from another thread
while (this->isRunning) {
//this->frames is a stack with cv::Mat elements in the right order
//it is filled by another thread
while (!this->frames.empty()) {
cv::Mat mat = frames.front();
frames.pop();
fwrite(mat.data, 1, s, pipeout);
}
}
fflush(pipeout);
pclose(pipeout);
So a file (pipeout) is used to write the mat.data to ffmpeg, ffmpeg itself is doing the encoding and file writing. To the parameters:
-y = Overwrite output files without asking
-f = format, in this case used for input rawvideo
-vcodec = codec for input which is rawvideo as well, because the used cv::Mat.data has no compression/codec
-framerate = the input framerate I receive from my grabber card/OpenCv
-pix_fmt = the format of my raw data, in this case bgr24, so 8 bit each channel, because I use a regular OpenCV bgr cv::Mat
-s = size of each frame, in my case 1920x1080
-i = input, in this case we read from the stdinput you can see it here "-", so the file (pipeout) is captured by ffmpeg
-c:v = output codec, so this is to encode the video, here h264_nvenc is used, which is a GPU codec
-r = frame output rate, also 50 in this case myVideoFile.mkv = this is just the name of the file which is produced by ffmpeg, you can change this file and path
Additional parameters for higher quality: -crf 14 -maxrate:v 10M
This works very good for me and uses my hardware acceleration of the GPU or with another codec in charge the CPU.
I hope this helps other developers as well.
Working hard for 4 days now to fix the google cloud speech to text api to work, but still see no light at the end of the tunnel. Searched on the net a lot, read the documentations a lot but see no result.
Our site is bbsradio.com, we are trying to auto extract transcript from our mp3 files using google speech-to-text api. Code is written on PHP and almost exact copy of this: https://github.com/GoogleCloudPlatform/php-docs-samples/blob/master/speech/src/transcribe_async.php
I see process is completed and its reached out here "$operation->pollUntilComplete();" but its not showing it was successful at "if ($operation->operationSucceeded()) {" and its not returning any error either at $operation->getError().
I am converting the mp3 to raw file like this: ffmpeg -y -loglevel panic -i /public_html/sites/default/files/show-archives/audio-clips-9-23-2020/911freefall2020-05-24.mp3 -f s16le -acodec pcm_s16le -vn -ac 1 -ar 16000 -map_metadata -1 /home/mp3_to_raw/911freefall2020-05-24.raw
While tried with FLAC format as well, not worked. I tested converted FLAC file using windows media player, I can listen conversation clearly. I checked the files its Hz 16000, channel = 1 and its 16 bit. I see file is uploaded in cloud storage. Checked this:
https://cloud.google.com/speech-to-text/docs/troubleshooting and
https://cloud.google.com/speech-to-text/docs/best-practices
There are lot of discussion and documentation, seems nothing is helpful at this moment. If some one can really help me out to find out the issue, it will be really really really great!
TLDR; convert from MP3 to a 1-channel FLAC file with the same sample rate as your MP3 file.
Long explanation:
Since you're using MP3 files as your process input, probably you MP3 compression artifacts might be hurting you when you resample to to 16KHz (you cannot hear this, but the algoritm will).
To confirm this theory:
Execute ffprobe -hide_banner filename.mp3 it will output something like this:
Metadata:
...
Duration: 00:02:12.21, start: 0.025057, bitrate: 320 kb/s
Stream #0:0: Audio: mp3, 44100 Hz, stereo, s16p, 320 kb/s
Metadata:
encoder : LAME3.99r
In this case, the sample rate is OK for Google-Spech-Api. Just transcode the file without changing the sample rate (remove the -ar 16000 from your ffmpeg command)
You might get into trouble if the original MP3 bitrate is low. 320kb/s seems safe (unless the recording has a lot of noise).
Take into account that voice recoded under 64kb/s (ISDN line quality) can be understood only by humans if there is some noise.
At last I found the solution and reason of the issue. Actually getting empty results is a bug of the php api code. What you need to do:
Replace this:
$operation->pollUntilComplete();
by this:
while(!$operation->isDone()){
$operation->pollUntilComplete();
}
Read this: enter link description here
I use the library of ffmpeg to decode stream from [TTQ HD Camera] and encode it to a rtmp stream.
but I receive a lot of warnings like the picture below.
i try to set qmin and qmax , it seems a little better. but still not totally resolve the problem.
encoder_context->qmin = 10;
encoder_context->qmax = 51;
who knows this is why ?
[dshow # 04bfc640] real-time buffer [TTQ HD Camera] [video input] too full or near too full (101% of size: 3041280 [rtbufsize parameter])! frame dropped!
Have you tried increasing the -rtbufsize parameter to something larger than 3041280? If you have the RAM for it, try something like 2000M. It should be defined before the -i of the camera.
So something like:
ffmpeg -f dshow -video_size 1920x1080 -rtbufsize 2147.48M -framerate 30 -pixel_format bgr0 -i video=...
Note that the resolution and frame rate are just examples and you would have to fill in your values that you have used in the ffmpeg command already.
I'm working with IP camera, and I have got Jpeg frames and audio data (PCM) from camera.
Now, I want to create video file (both audio and video) under .avi or .mp4 format from above data.
I searched and I knew that ffmpeg library can do it. But I don't know how to using ffmpeg to do this.
Can you suggest me some sample code or the function of ffmpeg to do it?
If your objective is to write a c++ app to do this for you please disregard this answer, I'll just leave it here for future reference. If not, here's how you can do it in bash:
First, make sure your images are in a nice format, easy to handle by ffmpeg. You can copy the images to a different directory:
mkdir tmp
x=1; for i in *jpg; do counter=$(printf %03d $x); cp "$i" tmp/img"$counter".jpg; x=$(($x+1)); done
Copy your audio data to the tmp directory and encode the video. Let's say your camera took a picture every ten seconds:
cd tmp
ffmpeg -i audio.wav -f image2 -i img%03d.jpg -vcodec msmpeg4v2 -r 0.1 -intra out.avi
Where -r 0.1 indicates a framerate of 0.1 which is one frame every 10 seconds.
The possible issues here are:
Your audio/video might go slightly out of sync unless you calculate your desired framerate carefully in advance. You should be able to get the length of the audio (or video) using ffmpeg and some grep magic. Even so the sync might be an issue with longer clips.
if you have more than 999 images the %03d format will not be enough, make sure to change the 3 to the desired length of the index
The video will inherit its length from the longer of the streams, you can restrict it using the -t switch:
-t duration - Restrict the transcoded/captured video sequence to the duration specified in seconds. "hh:mm:ss[.xxx]" syntax is also supported.