I am working with a high resolution camera: 4008x2672. I a writing a simple program which grabs frame from the camera and sends the frame to a avi file. For working with such a high resolution, I found only x264 codec that could do the trick (Suggestions welcome). I am using opencv for most of the image handling stuff. As mentioned in this post http://doom10.org/index.php?topic=1019.0 , I modified the AVCodecContext members as per ffmpeg presets for libx264 (Had to do this to avoid broken ffmpeg defaults settings error). This is output I am getting when I try to run the program
libx264 # 0x992d040]non-strictly-monotonic PTS
1294846981.526675 1 0 //Timestamp camera_no frame_no
1294846981.621101 1 1
1294846981.715521 1 2
1294846981.809939 1 3
1294846981.904360 1 4
1294846981.998782 1 5
1294846982.093203 1 6
Last message repeated 7 times
[avi # 0x992beb0]st:0 error, non monotone timestamps
-614891469123651720 >= -614891469123651720
OpenCV Error: Unspecified error (Error while writing video frame) in
icv_av_write_frame_FFMPEG, file
/home/ajoshi/ext/OpenCV-2.2.0/modules/highgui/src/cap_ffmpeg.cpp, line 1034
terminate called after throwing an instance of 'cv::Exception'
what(): /home/ajoshi/ext/OpenCV-2.2.0/modules/highgui/src/cap_ffmpeg.cpp:1034:
error: (-2) Error while writing video frame in function icv_av_write_frame_FFMPEG
Aborted
Modifications to the AVCodecContext are:
if(codec_id == CODEC_ID_H264)
{
//fprintf(stderr, "Trying to parse a preset file for libx264\n");
//Setting Values manually from medium preset
c->me_method = 7;
c->qcompress=0.6;
c->qmin = 10;
c->qmax = 51;
c->max_qdiff = 4;
c->i_quant_factor=0.71;
c->max_b_frames=3;
c->b_frame_strategy = 1;
c->me_range = 16;<br>
c->me_subpel_quality=7;
c->coder_type = 1;
c->scenechange_threshold=40;
c->partitions = X264_PART_I8X8 | X264_PART_I4X4 | X264_PART_P8X8 | X264_PART_B8X8;
c->flags = CODEC_FLAG_LOOP_FILTER;
c->flags2 = CODEC_FLAG2_BPYRAMID | CODEC_FLAG2_MIXED_REFS | CODEC_FLAG2_WPRED | CODEC_FLAG2_8X8DCT | CODEC_FLAG2_FASTPSKIP;
c->keyint_min = 25;
c->refs = 3;
c->trellis=1;
c->directpred = 1;
c->weighted_p_pred=2;
}
I am probably not setting the dts and pts values which I believed ffmpeg should be setting it for me.
Any sugggestions welcome.
Thanks in advance
I would probably run the x264 executable in another process and pipe either rgb or yuv pixels to it. Then you can use all the normal x264 (or ffmpeg) flags and it handles multi threading for you.
And since x264 is GPL licensed it also gives you more freedom on licensing your app.
ps. Here is some sample code using ffmpeg from Qt you can ignore the Qt specific bits but it gives a good starting point for using ffmpeg from a c++ app.
Actual error is "non monotone timestamps". I seems that you didn't properly initialized video frame properties. If its possible use libx264 directly. It'll be more easy to handle.
PS. you can work around ffmpeg x264 setting problem by specify 264 preset file with -fvpre option.
The pts value of the AVFrame you send as the last argument to avcodec_encode_video needs to be set by you. Once you set this, the codec context's coded_from->pts field will have the correct value which you can av_rescale_q() and set in the AVPacket for your av_interleaved_write_frame().
Related
I've made a C++ program that lives in gke and takes some videos as input using ffmpeg, then does something with that input using opengl(not relevant), then finally encodes those edited videos as a single output. Normally the program works perfectly fine on my local machine, it encodes just as I want it to with no warnings or valgrind errors whatsoever. Then, after encoding the said video, I want my program to upload that video to the google cloud storage. This is where the problem comes, I have tried 2 methods for this: First, I tried using curl to upload to the cloud using a signed url. Second, I tried mounting the google storage using gcsfuse(I was already mounting the bucket to access the inputs in question). Both of those methods yielded undefined, weird behaviour's ranging from: Outputing a 0byte or 44byte file, (This is the most common one:) encoding in the correct file size ~500mb but the video is 0 seconds long, outputing a 0.4 second video or just encoding the desired output normally (really rare).
From the logs I can't see anything unusual, everything seems to work fine and ffmpeg does not give any errors or warnings, so does valgrind. Everything seems to work normally, even when I use curl to upload the video to the cloud the output is perfectly fine when it first encodes it (before sending it with curl) but the video gets messed up when curl uploads it to the cloud.
I'm using the muxing.c example of ffmpeg to encode my video with the only difference being:
void video_encoder::fill_yuv_image(AVFrame *frame, struct SwsContext *sws_context) {
const int in_linesize[1] = { 4 * width };
//uint8_t* dest[4] = { rgb_data, NULL, NULL, NULL };
sws_context = sws_getContext(
width, height, AV_PIX_FMT_RGBA,
width, height, AV_PIX_FMT_YUV420P,
SWS_BICUBIC, 0, 0, 0);
sws_scale(sws_context, (const uint8_t * const *)&rgb_data, in_linesize, 0,
height, frame->data, frame->linesize);
}
rgb_data is the data I got after editing the inputs. Again, this works fine and I don't think there are any errors here.
I'm not sure where the error is and since the code is huge I can't provide a replicable example. I'm just looking for someone to point me to the right direction.
Running the cloud's output in mplayer wields this result (This is when the video is the right size but is 0 seconds long, the most common one.):
MPlayer 1.4 (Debian), built with gcc-11 (C) 2000-2019 MPlayer Team
do_connect: could not connect to socket
connect: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.
Playing /media/c36c2633-d4ee-4d37-825f-88ae54b86100.
libavformat version 58.76.100 (external)
libavformat file format detected.
[mov,mp4,m4a,3gp,3g2,mj2 # 0x7f2cba1168e0]moov atom not found
LAVF_header: av_open_input_stream() failed
libavformat file format detected.
[mov,mp4,m4a,3gp,3g2,mj2 # 0x7f2cba1168e0]moov atom not found
LAVF_header: av_open_input_stream() failed
RAWDV file format detected.
VIDEO: [DVSD] 720x480 24bpp 29.970 fps 0.0 kbps ( 0.0 kbyte/s)
X11 error: BadMatch (invalid parameter attributes)
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory
[vdpau] Error when calling vdp_device_create_x11: 1
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
libavcodec version 58.134.100 (external)
[dvvideo # 0x7f2cb987a380]Requested frame threading with a custom get_buffer2() implementation which is not marked as thread safe. This is not supported anymore, make your callback thread-safe.
Selected video codec: [ffdv] vfm: ffmpeg (FFmpeg DV)
==========================================================================
Load subtitles in /media/
==========================================================================
Opening audio decoder: [libdv] Raw DV Audio Decoder
Unknown/missing audio format -> no sound
ADecoder init failed :(
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
[dvaudio # 0x7f2cb987a380]Decoder requires channel count but channels not set
Could not open codec.
ADecoder init failed :(
ADecoder init failed :(
Cannot find codec for audio format 0x56444152.
Audio: no sound
Starting playback...
[dvvideo # 0x7f2cb987a380]could not find dv frame profile
Error while decoding frame!
[dvvideo # 0x7f2cb987a380]could not find dv frame profile
Error while decoding frame!
V: 0.0 2/ 2 ??% ??% ??,?% 0 0
Exiting... (End of file)
Edit: Since the code runs on a VM, I'm using xvfb-run ro start my application, but again even when using xvfb-run it works completely fine on when not encoding to the cloud.
Apparently, I'm assuming for security reasons, the google cloud storage does not allow us to do multiple continuous operations on a file, just a singular read/write operation. So I found a workaround by encoding my video to a local file inside the pod and then doing a copy operation to the cloud.
I use the library of ffmpeg to decode stream from [TTQ HD Camera] and encode it to a rtmp stream.
but I receive a lot of warnings like the picture below.
i try to set qmin and qmax , it seems a little better. but still not totally resolve the problem.
encoder_context->qmin = 10;
encoder_context->qmax = 51;
who knows this is why ?
[dshow # 04bfc640] real-time buffer [TTQ HD Camera] [video input] too full or near too full (101% of size: 3041280 [rtbufsize parameter])! frame dropped!
Have you tried increasing the -rtbufsize parameter to something larger than 3041280? If you have the RAM for it, try something like 2000M. It should be defined before the -i of the camera.
So something like:
ffmpeg -f dshow -video_size 1920x1080 -rtbufsize 2147.48M -framerate 30 -pixel_format bgr0 -i video=...
Note that the resolution and frame rate are just examples and you would have to fill in your values that you have used in the ffmpeg command already.
The Problem:
I'm working on a reader for multiple ip camera streams. The application needs to run on a ubuntu AWS EC2 instance. I've been unsuccessful in trying to reliably fetch and decode the frames from RTSP h.264 streams.
What I've Tried:
I've used OpenCV's and SciKit-Video's VideoCapture classes, neither of which could both fetch and decode the frames from my test stream. I have verified that my test stream is readable using VLC and openRTSP, so I believe this is an encoding issue.
I've also attempted to build some solutions using python's subprocess module to run the aforementioned command-line applications. This allows me to read the stream reliably, but it raises the issue that the decoder fails due to it (apparently) not finding the keyframe data it needs to decode the stream.
Below is the code for this example. It tells openRTSP to periodically save some amount of video as a separate file and uses an openCV VideoCapture to get a single frame from each of those samples. Code:
def openrtsp_thread(queue, feed_name, source, sample_time, intruder, cleanup=True):
(major, minor, subminor) = (cv2.__version__).split('.')
cmd = 'openRTSP -V -4 -v -P ' + str(sample_time) + ' ' + source
out_dir = '/aws_odw/frame-store/'+feed_name.replace(' ','-')+'/'
try:
os.mkdir(out_dir)
except:
print '[?]['+feed_name+']: Not creating new frame directory'
pass
os.chdir(out_dir)
p = subprocess.Popen(cmd.split(' '))
while True:
try:
for dirs, files, filenames in os.walk(out_dir):
for f in filenames:
cap = cv2.VideoCapture(os.path.join(out_dir, f))
if int(major) < 3:
fps = cap.get(cv2.cv.CV_CAP_PROP_FPS)
else:
fps = cap.get(cv2.CAP_PROP_FPS)
#for i in range(int(float(sample_time)*fps*0.5)):
ret, frame = cap.read()
cap.release()
print 'enqueueing...'
queue.put((feed_name, frame, intruder))
except (KeyboardInterrupt, SystemExit):
print '[x]['+feed_name+']: keyboard interrupt, cleaning up...'
break
p.send_signal(signal.SIGUSR1)
p.wait()
print '[*]['+feed_name+']: exiting gracefully.'
Can anyone offer any pointers? I don't know much about video encoding, so I'm feeling pretty lost. Any help would be very appreciated.
Edit: the end goal here is to queue the frames in python for real-time precessing in a computer vision application.
I have searched all around and can not find any examples or tutorials on how to access a webcam using ffmpeg in C++. Any sample code or any help pointing me to some documentation, would greatly be appreciated.
Thanks in advance.
I have been working on this for months now. Your first "issue" is that ffmpeg (libavcodec and other ffmpeg libs) does NOT access web cams, or any other device.
For a basic USB webcam, or audio/video capture card, you first need driver software to access that device. For linux, these drivers fall under the Video4Linux (V4L2 as it is known) category, which are modules that are part of most distros. If you are working with MS Windows, then you need to get an SDK that allows you to access the device. MS may have something for accessing generic devices, (but from my experience, they are not very capable, if they work at all) If you've made it this far, then you now have raw frames (video and/or audio).
THEN you get to the ffmpeg part - libavcodec - which takes the raw frames (audio and/or video) and encodes them into a streams, which ffmpeg can then mux into your final container.
I have searched, but have found very few examples of all of these, and most are piece-meal.
If you don't need to actually code of this yourself, the command line ffmpeg, as well as vlc, can access these devices, capture and save to files, and even stream.
That's the best I can do for now.
ken
For windows use dshow
For Linux (like ubuntu) use Video4Linux (V4L2).
FFmpeg can take input from V4l2 and can do the process.
To find the USB video path type : ls /dev/video*
E.g : /dev/video(n) where n = 0 / 1 / 2 ….
AVInputFormat – Struct which holds the information about input device format / media device format.
av_find_input_format ( “v4l2”) [linux]
av_format_open_input(AVFormatContext , “/dev/video(n)” , AVInputFormat , NULL)
if return value is != 0 then error.
Now you have accessed the camera using FFmpeg and can continue the operation.
sample code is below.
int CaptureCam()
{
avdevice_register_all(); // for device
avcodec_register_all();
av_register_all();
char *dev_name = "/dev/video0"; // here mine is video0 , it may vary.
AVInputFormat *inputFormat =av_find_input_format("v4l2");
AVDictionary *options = NULL;
av_dict_set(&options, "framerate", "20", 0);
AVFormatContext *pAVFormatContext = NULL;
// check video source
if(avformat_open_input(&pAVFormatContext, dev_name, inputFormat, NULL) != 0)
{
cout<<"\nOops, could'nt open video source\n\n";
return -1;
}
else
{
cout<<"\n Success !";
}
} // end function
Note : Header file < libavdevice/avdevice.h > must be included
This really doesn't answer the question as I don't have a pure ffmpeg solution for you, However, I personally use Qt for webcam access. It is C++ and will have a much better API for accomplishing this. It does add a very large dependency on your code however.
It definitely depends on the webcam - for example, at work we use IP cameras that deliver a stream of jpeg data over the network. USB will be different.
You can look at the DirectShow samples, eg PlayCap (but they show AmCap and DVCap samples too). Once you have a directshow input device (chances are whatever device you have will be providing this natively) you can hook it up to ffmpeg via the dshow input device.
And having spent 5 minutes browsing the ffmpeg site to get those links, I see this...
I am trying to record a video acquired from a webcam connect to the usb device. I am workin with Qt5.1.0 in Linux 64 bit.
I have the following code:
camera = new QCamera(this);
viewFinder = new QCameraViewfinder(this);
camera->setViewfinder(viewFinder);
recorder = new QMediaRecorder(camera,this);
QVideoEncoderSettings settings = recorder->videoSettings();
settings.setResolution(1280,720);
settings.setQuality(QMultimedia::VeryHighQuality);
settings.setFrameRate(30.0);
recorder->setVideoSettings(settings);
camera->setCaptureMode(QCamera::CaptureVideo);
camera->start();
QString name = filename + QDateTime::currentDateTime().toString("dd.MM.yy-h-m-s");
recorder->setOutputLocation(QUrl::fromLocalFile(outputpath + "/" + name + ".mp4"));
recorder->record();
When I run this code I get the following warning and error
CameraBin error: "Internal data flow error."
CameraBin error: "Could not negotiate format"
And in fact nothing is recorded.
If I change the line
camera->setCaptureMode(QCamera::CaptureVideo);
for
camera->setCaptureMode(QCamera::CaptureViewFinder);
No error is output, the file is generated, but it only contains one frame (fixed image)
If I remove this piece of code:
QVideoEncoderSettings settings = recorder->videoSettings();
settings.setResolution(1280,720);
settings.setQuality(QMultimedia::VeryHighQuality);
settings.setFrameRate(30.0);
I get two different errors:
CameraBin warning: "A lot of buffers are being dropped."
CameraBin error: "Could not encode stream."
But the video is actually recorded.
I'm having almost the same problem, with capturing an image. I discovered that it is only working with the default resolution 640 x 480. If you set the resolution to a higher value it does not work. I've also tried with 2 different cameras with no success, so it seems to be a qt5 problem. You could try to don't set a resolution, then you should be able to record a video, but only with the default resolution of 640 x 480.