I'm trying to connect to several USB cameras that are connected to different USB buses (hopefully cancelling the bandwidth bottleneck caused by the USB).
Before accessing to the cameras, I'm probing them with V4L2 API directly to see if they're accessible. For now, there are 3 cameras and V4L2 sees them all.
Then I'm trying to access them using openCV like this, each in its own object:
this->camera = new cv::VideoCapture(camera_port);
if(this->camera->isOpened()) {
....
cv::Mat capturedImage;
bool read;
read = this->camera->read(capturedImage);
....
}
where camera_port is 0,1,2.
Obviously this->camera is called with release() on program closure.
The problem arises when I'm accessing more than 1 camera. Only one camera out of the three returns an image. The others return errors that I'm seeing on the console (not the same on every run):
libv4l2: error turning on stream: Connection timed out,
VIDIOC_STREAMON: Connection timed out
libv4l2: error turning on stream: Input/output error,
VIDIOC_STREAMON: Input/output error
libv4l2: error turning on stream: Invalid argument,
VIDIOC_STREAMON: Invalid argument
Some other errors, but the above are the most frequent
However, this does work on the first run after replugging-in the USB cameras, but not on further runs of the program.
Thoughts?
Related
I’ve implemented custom IMFMediaSink for use with sink writer. Works OK, receives h264 video samples. I don’t have any container, I’m consuming raw h264 video samples. I have not implemented custom writer, I'm using MFCreateSinkWriterFromMediaSink API to wrap my custom media sink into a framework-provided writer.
I’m unable to implement graceful shutdown, IMFSinkWriter::Finalize() never returns. When I implemented IMFSinkWriterCallback, IMFSinkWriter::Finalize() returns immediately but my IMFSinkWriterCallback::OnFinalize was never called.
The problem reproduces in 100% tests with both nvenc and MS software encoder.
Writer attributes:
MF_LOW_LATENCY = TRUE
MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS = TRUE (1)
MF_READWRITE_DISABLE_CONVERTERS = FALSE (2)
MF_SINK_WRITER_DISABLE_THROTTLING = TRUE
MF_SINK_WRITER_D3D_MANAGER
MF_SINK_WRITER_ASYNC_CALLBACK
(1) Tried both, same result
(2) Need the converters because nvenc only supports YUV and I have RGB textures on input.
Output media type (it’s fixed, I’m using the built-in handler created by MFCreateSimpleTypeHandler API).
MF_MT_MAJOR_TYPE = MFMediaType_Video
MF_MT_SUBTYPE = MFVideoFormat_H264
MF_MT_INTERLACE_MODE = MFVideoInterlace_Progressive
MF_MT_AVG_BITRATE = 40*1000*1000
MF_MT_FRAME_SIZE = { 3840, 2160 }
MF_MT_FRAME_RATE = { 60, 1 }
MF_MT_PIXEL_ASPECT_RATIO = { 1, 1 }
Input media type:
MF_MT_MAJOR_TYPE = MFMediaType_Video
MF_MT_SUBTYPE = MFVideoFormat_RGB32
MF_MT_INTERLACE_MODE = MFVideoInterlace_Progressive
MF_MT_FRAME_SIZE = { 3840, 2160 }
MF_MT_FRAME_RATE = { 60, 1 }
MF_MT_PIXEL_ASPECT_RATIO = { 1, 1 }
When not using IMFSinkWriterCallback, here’s a call stack at the time of hang:
ntdll.dll!_NtWaitForSingleObject#12 ()
KernelBase.dll!WaitForSingleObjectEx()
mfreadwrite.dll!CMFSinkWriter::InternalFinalize(void)
mfreadwrite.dll!CMFSinkWriter::Finalize(void)
MFTrace doesn’t have anything related to finalize even with -k All:
13700,3C60 19:01:25.79566 CMFTransformDetours::ProcessOutput #02EA6E3C failed hr=0xC00D6D72 MF_E_TRANSFORM_NEED_MORE_INPUT
13700,2A98 19:01:25.80250 CMFTransformDetours::ProcessOutput #1A6CEF38 Stream ID 0, Sample #1C244F30, Time 1216ms, Duration 16ms, Buffers 1, Size 12441600B, MFSampleExtension_CleanPoint=1;MFSampleExtension_Interlaced=0
13700,2098 19:01:25.80254 CMFTransformDetours::ProcessInput #02EA6E3C Stream ID 0, Sample #1C244F30, Time 1216ms, Duration 16ms, Buffers 1, Size 12441600B, MFSampleExtension_CleanPoint=1;MFSampleExtension_Interlaced=0
13700,2A98 19:01:25.80256 CMFTransformDetours::ProcessOutput #1A6CEF38 failed hr=0xC00D6D72 MF_E_TRANSFORM_NEED_MORE_INPUT
13700,2A98 19:01:25.80266 CMFTransformDetours::ProcessMessage #1A6CEF38 Message type=0x00000001 MFT_MESSAGE_COMMAND_DRAIN, param=00000000
13700,2A98 19:01:25.80267 CMFTransformDetours::ProcessOutput #1A6CEF38 failed hr=0xC00D6D72 MF_E_TRANSFORM_NEED_MORE_INPUT
13700,2098 19:01:25.81669 CMFTransformDetours::ProcessOutput #02EA6E3C Stream ID 0, Sample #1FB68CF8, Time 1216ms, Duration 16ms, Buffers 1, Size 680B, {2B5D5457-5547-4F07-B8C8-B4A3A9A1DAAC}=1;{73A954D4-09E2-4861-BEFC-94BD97C08E6E}=12166667 (0,12166667);{9154733F-E1BD-41BF-81D3-FCD918F71332}=65535;{973704E6-CD14-483C-8F20-C9FC0928BAD5}=1;MFSampleExtension_CleanPoint=0;{B2EFE478-F979-4C66-B95E-EE2B82C82F36}=16 (0,16)
13700,82C 19:01:25.81674 CMFTransformDetours::ProcessOutput #02EA6E3C failed hr=0xC00D6D72 MF_E_TRANSFORM_NEED_MORE_INPUT
13700,82C 19:01:25.81674 CMFTransformDetours::ProcessMessage #02EA6E3C Message type=0x00000001 MFT_MESSAGE_COMMAND_DRAIN, param=00000000
13700,82C 19:01:25.81674 CMFTransformDetours::ProcessOutput #02EA6E3C failed hr=0xC00D6D72 MF_E_TRANSFORM_NEED_MORE_INPUT
13700,1F54 19:01:27.24237 CKernel32ExportDetours::OutputDebugStringA # D3D11 WARNING: Process is terminating. Using simple reporting. Please call ReportLiveObjects() at runtime for standard reporting. [ STATE_CREATION WARNING #0: UNKNOWN]
13700,1F54 19:01:27.24255 CKernel32ExportDetours::OutputDebugStringA # D3D11 WARNING: Live Producer at 0x0311D91C, Refcount: 13. [ STATE_CREATION WARNING #0: UNKNOWN]
Warnings about live D3D resources are expected as I terminated the process after the hang.
Any ideas what’s going on? I think the writer probably waits for these SPS/PPS magic blobs to arrive but it never happens. Is there a way to instruct the h264 encoder to output SPS/PPS somewhere?
You’ve implemented custom IMFMediaSink, so i suppose you 've also implemented IMFStreamSink.
Doing this in the usual way with Mediafoundation, you have circular COM reference between IMFMediaSink and IMFStreamSink. That's why the Shutdown method from IMFMediaSink interface exists.
If a program that uses your custom MediaSink does not call Shutdown at the right place, there will be memory leaks.
From your IMFSinkWriterCallback problem, we don't have enough information to find where the problem is.
Also it is not clear about "custom IMFMediaSink" and "IMFSinkWriter". Are you also implementing a IMFSinkWriter...
EDIT1
Just two things :
MFCreateSinkWriterFromMediaSink
Call CoInitialize(Ex) and MFStartup before calling this function.
When you are done using the media sink, call the media sink's IMFMediaSink::Shutdown method. (The sink writer does not shut down the media sink.) Release the sink writer before calling Shutdown on the media sink.
do you release interfaces correctly ?
IMFSinkWriter::Finalize
Internally, this method calls IMFStreamSink::PlaceMarker to place end-of-segment markers for each stream on the media sink.
Do you handle this message (MFSTREAMSINK_MARKER_ENDOFSEGMENT) ?
We don't know how you handle CriticalSection/Event/CircularReference, so it's hard to found the problem.
EDIT2
Is there a way to instruct the h264 encoder to output SPS/PPS somewhere?
Normally, for h264 video format, you need to get the attributes MF_MT_MPEG_SEQUENCE_HEADER (BLOB type) when SetCurrentMediaType is called on your IMFStreamSink (assuming you implements IMFMediaTypeHandler).
EDIT3
Could you provide the real one (this is how i think the app should be) :
I don't remember if your custom sink creates a mp4 file. If it is, in the IMFSinkWriter::Finalize, you have to generate ftpy/moov atom.
EDIT4
Also you can read this : Video Recording Hangs on IMFSinkWriter->Finalize();
With no source code, this is the only answer i can give.
I want to list all connected webcams (USB-webcams and internal webcams), using C++, OpenCV 2.4.11, Windows 8.1 and Qt Creator 3.4.2.
For me it is enough to get the number of accessible webcams in the following way:
VideoCapture videoCapture(0); // Will access my internal laptop webcam.
VideoCapture videoCapture(1); // Will access the first connected usb webcam.
Here is my code:
// Following procedure detects, how many webcams are accessible from 0 on upwards.
numberOfDevices = 0;
bool noError = true;
while (noError)
{
try
{
// Check if camera is available.
VideoCapture videoCapture(numberOfDevices); // Will crash if not available, hence try/catch.
// ...
}
catch (...)
{
noError = false;
}
// If above call worked, we have found another camera.
numberOfDevices++;
}
The code in the try-block works if I have activated my internal webcam. The call fails when I deactivate the internal camera in the hardware manager (and no other camera is attached to my laptop) with the following error message (debug mode):
Exception Triggered
---------------------------
The inferior stopped because it triggered an exception.<p>Stopped in thread 0 by:
Exception at 0x7ff8533d9090, code: 0xc0000005: read access violation at: 0x0, flags=0x0 (first chance).
and the following 2 build problems:
Exception at 0x7ff871af8b9c, code: 0xa1a01db1: , flags=0x0 (first chance)
Exception at 0x7ff8533d9090, code: 0xc0000005: read access violation at: 0x0, flags=0x0 (first chance)
How can I fetch the occuring error? As you see, try/catch does not work.
Or is there a method where I can access all available webcams in OpenCV without such a dirty loop?
There is still no any functionality related to camera count in OpenCV at the current moment (3.0.0 version) - see corresponding ticket.
Proper camera handling seems like OpenCV internal problem (for example, described here or here). Usually it appears in capture code after physically disabling camera while it is still opened in OpenCV (when we try to read destroyed file descriptor).
Generally you can even implement your own handler for access violations (please look into this thread), but it's really dirty trick.
I created this C++ class that allows enumerating devices (including the ID) to be used inside OpenCV. It is hosted on GitHub.
https://github.com/studiosi/OpenCVDeviceEnumerator
The idea is to use DirectShow to get all the devices that have the category with GUID CLSID_VideoInputDeviceCategory, and then, through an enumerator, you get in which order they appear on the system, which is the ID you need to open them on OpenCV by creating a VideoCapture object (by using the constructor that receives the ID, which would be the index of the device on the enumeration). Obviously, this approach only works on Windows.
I am trying to record a video acquired from a webcam connect to the usb device. I am workin with Qt5.1.0 in Linux 64 bit.
I have the following code:
camera = new QCamera(this);
viewFinder = new QCameraViewfinder(this);
camera->setViewfinder(viewFinder);
recorder = new QMediaRecorder(camera,this);
QVideoEncoderSettings settings = recorder->videoSettings();
settings.setResolution(1280,720);
settings.setQuality(QMultimedia::VeryHighQuality);
settings.setFrameRate(30.0);
recorder->setVideoSettings(settings);
camera->setCaptureMode(QCamera::CaptureVideo);
camera->start();
QString name = filename + QDateTime::currentDateTime().toString("dd.MM.yy-h-m-s");
recorder->setOutputLocation(QUrl::fromLocalFile(outputpath + "/" + name + ".mp4"));
recorder->record();
When I run this code I get the following warning and error
CameraBin error: "Internal data flow error."
CameraBin error: "Could not negotiate format"
And in fact nothing is recorded.
If I change the line
camera->setCaptureMode(QCamera::CaptureVideo);
for
camera->setCaptureMode(QCamera::CaptureViewFinder);
No error is output, the file is generated, but it only contains one frame (fixed image)
If I remove this piece of code:
QVideoEncoderSettings settings = recorder->videoSettings();
settings.setResolution(1280,720);
settings.setQuality(QMultimedia::VeryHighQuality);
settings.setFrameRate(30.0);
I get two different errors:
CameraBin warning: "A lot of buffers are being dropped."
CameraBin error: "Could not encode stream."
But the video is actually recorded.
I'm having almost the same problem, with capturing an image. I discovered that it is only working with the default resolution 640 x 480. If you set the resolution to a higher value it does not work. I've also tried with 2 different cameras with no success, so it seems to be a qt5 problem. You could try to don't set a resolution, then you should be able to record a video, but only with the default resolution of 640 x 480.
Maybe anybody will know the answer to the next question...
I have plugged in usb digital camera (it has one input pin - 0).
I check input status every 5 seconds via structure: v4l2_input (here is the example: http://pastebin.com/FFvNAkQ3), to process situation if one of the flags is set (V4L2_IN_ST_NO_POWER or V4L2_IN_ST_NO_SIGNAL or V4L2_IN_ST_NO_H_LOCK).
The problem is that even i unplug my usb digital camera, the input status is always 0 (0x00). Why the driver does not change input status flag when the device is unplugged?
Maybe it is possible to check device status not only input pin? If yes, then which ioctl request should be set...?
I have a working opencv code that takes feed from my webcam and displays it. (The code is modified from the one here
The only issue is that when I try to print the frames-per-second value, as
int fps = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_FPS);
printf ("Frames per Second: %d\n",fps);
-1 gets printed.
Another (side) issue is that there appears to be some run time errors/warnings
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
select timeout
HIGHGUI ERROR: V4L2: Unable to get property <unknown property string>(5) - Invalid argument
HIGHGUI ERROR: V4L: Property <unknown property string>(2) not supported by device
HIGHGUI ERROR: V4L2: Unable to get property <unknown property string>(1) - Invalid argument
HIGHGUI ERROR: V4L: Property <unknown property string>(2) not supported by device
But the code does what it's supposed to do, ie, display video from the webcam.
Any ideas how to sort out the problems? I am using opencv on Eclipse with CDT on Ubuntu 11.10
Thanks in advance.
Judging from the error messages, this sounds like a problem your webcam driver. The error messages are from Video4Linux (V4L or V4L2 in the error messages), which is the part of the Linux kernel that contains webcam drivers. Getting a select() failure and the "not supported by device" error message probably means that your webcam didn't implement this part of the V4L2 API.
You can verify that this is the problem by testing your code with a camera that has a known-good driver. Unfortunately, this is very common in the webcam drivers included in the kernel. Many of them are reverse engineered, so it's quite a feat just access the video stream.
Try this for the HIGHGUI ERROR errors:
export LD_PRELOAD=/usr/lib/libv4l/v4l1compat.so