My OpenAL C++ audio streaming buffer gliching - c++

I have for the first time coding sound generating with OpenAL in C++.
What I want to do is to generate endless sinus wave into a double buffering way.
And the problem is that the sound is glittering/lags. I Think it is between the buffering and I don't know why it is like that.
My code:
void _OpenALEngine::play()
{
if(!m_running && !m_threadRunning)
{
ALfloat sourcePos[] = {0,0,0};
ALfloat sourceVel[] = {0,0,0};
ALfloat sourceOri[] = {0,0,0,0,0,0};
alGenSources(1, &FSourceID);
alSourcefv (FSourceID, AL_POSITION, sourcePos);
alSourcefv (FSourceID, AL_VELOCITY, sourceVel);
alSourcefv (FSourceID, AL_DIRECTION, sourceOri);
GetALError();
ALuint FBufferID[2];
alGenBuffers( 2, &FBufferID[0] );
GetALError();
// Gain
ALfloat listenerPos[] = {0,0,0};
ALfloat listenerVel[] = {0,0,0};
ALfloat listenerOri[] = {0,0,0,0,0,0};
alListenerf( AL_GAIN, 1.0 );
alListenerfv(AL_POSITION, listenerPos);
alListenerfv(AL_VELOCITY, listenerVel);
alListenerfv(AL_ORIENTATION, listenerOri);
GetALError();
alSourceQueueBuffers( FSourceID, 2, &FBufferID[0] );
GetALError();
alSourcePlay(FSourceID);
GetALError();
m_running = true;
m_threadRunning = true;
Threading::Thread thread(Threading::ThreadStart(this, &_OpenALEngine::threadPlaying));
thread.Start();
}
}
Void _OpenALEngine::threadPlaying()
{
while(m_running)
{
// Check how much data is processed in OpenAL's internal queue.
ALint Processed;
alGetSourcei( FSourceID, AL_BUFFERS_PROCESSED, &Processed );
GetALError();
// Add more buffers while we need them.
while ( Processed-- )
{
alSourceUnqueueBuffers( FSourceID, 1, &BufID );
runBuffer(); // <--- Generate the sinus wave and submit the Array to the submitBuffer method.
alSourceQueueBuffers( FSourceID, 1, &BufID );
ALint val;
alGetSourcei(FSourceID, AL_SOURCE_STATE, &val);
if(val != AL_PLAYING)
{
alSourcePlay(FSourceID);
}
}
// Don't kill the CPU.
Thread::Sleep(1);
}
m_threadRunning = false;
return Void();
}
void _OpenALEngine::submitBuffer(byte* buffer, int length)
{
// Submit more data to OpenAL
alBufferData( BufID, AL_FORMAT_MONO8, buffer, length * sizeof(byte), 44100 );
}
I generate the sinus wave in the runBuffer() method. And the sinus generator is correct because when I increase the buffer array from 4096 to 40960 the glittering/lags sound with bigger interval. Thank you very much if some one know the problem and will share it :)

Similar Problems are all over the internet and I'm not 100% sure this is the solution to this on. But it probably is, and if not it might at least help others. Most other threads are on different forums and I'm not registering everywhere just to share my knowledge...
The code below is what I came up after 2 days of experimenting. Most solutions I found did not work for me...
(it's not exactly my code, I stripped it of some parts special to my case, so I'm sorry if there are typos or similar that prevent it from being copied verbatim)
My experiments were on an iPhone. Some of the the things I found out, might be iOS-specific.
The problem is that there is no guaranty at what point a processed buffer is marked as such and is available for unqueueing. Trying to build a version that sleeps until a buffer becomes available again I saw that this might be much(I use very small buffers) later than expected. So I realised that the common idea to wait until a buffer is available(which works for most frameworks, but not openAL) is wrong. Instead you should wait until the time you should enqueue another buffer.
With that you have to give up the idea of double-buffering. When the time comes you should check if a buffer exists and unqueue it. But if none is available you need to create a 3rd...
Waiting for when a buffer should be enqueue can be done by calculating times relative to the system-clock, which worked fairly well for me but I decided to go for a version where I rely on a time source that is definitivly in sync with openAL. Best I came up with was wait depending on what s left in the queue. Here, iOS seems not fully in accordance to openAL-spec because AL_SAMPLE_OFFSET should be exact to one sample but I never saw anything but multiples of 2048. That's about 45 microseconds #44100, this is where the 50000 in the code comes from(little more than the smalest unit iOS handles)
Depending on the block-size this can easily be bigger. But with that code I had 3 times that alSourcePlay() was needed again in the last ~hour(compared to up to 10 per minute with other implementations that claimed to be the solution)
uint64 enqueued(0); // keep track of samples in queue
while (bKeepRunning)
{
// check if enough in buffer and wait
ALint off;
alGetSourcei(m_Source, AL_SAMPLE_OFFSET, &off);
uint32 left((enqueued-off)*1000000/SAMPLE_RATE);
if (left > 50000) // at least 50000 mic-secs in buffer
usleep(left - 50000);
// check for available buffer
ALuint buffer;
ALint processed;
alGetSourcei(m_Source, AL_BUFFERS_PROCESSED, &processed);
switch (processed)
{
case 0: // no buffer to unqueue->create new
alGenBuffers(1, &buffer);
break;
case 1: // on buffer to unqueue->use that
alSourceUnqueueBuffers(m_Source, 1, &buffer);
enqueued -= BLOCK_SIZE_SAMPLES;
break;
default: // multiple buffers to unqueue->take one,delete on
{ // could also delete more if processed>2
// but doesn't happen often
// therefore simple implementation(will del. in next loop)
ALuint bufs[2];
alSourceUnqueueBuffers(m_Source, 2, bufs);
alDeleteBuffers(1, bufs);
buffer = bufs[1];
enqueued -= 2*BLOCK_SIZE_SAMPLES;
}
break;
}
// fill block
alBufferData(buffer, AL_FORMAT_STEREO16, pData,
BLOCK_SIZE_SAMPLES*4, SAMPLE_RATE);
alSourceQueueBuffers(m_Source, 1, &buffer);
//check state
ALint state;
alGetSourcei(m_Source, AL_SOURCE_STATE, &state);
if (state != AL_PLAYING)
{
enqueued = BLOCK_SIZE_SAMPLES;
alSourcePlay(m_Source);
}
else
enqueued += BLOCK_SIZE_SAMPLES;
}

I have written OpenAL streaming servers so I know your pain - my instinct is to confirm you have spawned separate threads for the I/O logic which available your streaming audio data - separate from the thread to hold your above OpenAL code ??? If not this will cause your symptoms. Here is a simple launch of each logical chunk into its own thread :
std::thread t1(launch_producer_streaming_io, chosen_file, another_input_parm);
std::this_thread::sleep_for (std::chrono::milliseconds( 100));
std::thread t2(launch_consumer_openal, its_input_parm1, parm2);
// -------------------------
t1.join();
t2.join();
where launch_producer_streaming_io is a method being called with its input parms which services the input/output to continuously supply the audio data ... launch_consumer_openal is a method launched in its own thread where you instantiate your OpenAL class

Related

Getting External Headphones (8): EXC_BAD_ACCESS (code=1, address=0x0) error when I am trying to use maximilian

I am testing out using the maximilian library with JUCE. I am trying to use the maxiSample feature and I have implemented it exactly how the example code says to. Whenever I run the standalone app, I get the error "External Headphones (8): EXC_BAD_ACCESS (code=1, address=0x0)" and it gives me a breakpoint at line 747 of maximilian.cpp. It's not my headphones as it does the same thing with any playback device. Truly at a loss.
I've attached my MainComponent.cpp below. Any advice would be great, thank you!
#include "MainComponent.h"
#include "maximilian.h"
//==============================================================================
MainComponent::MainComponent()
{
// Make sure you set the size of the component after
// you add any child components.
setSize (800, 600);
// Some platforms require permissions to open input channels so request that here
if (juce::RuntimePermissions::isRequired (juce::RuntimePermissions::recordAudio)
&& ! juce::RuntimePermissions::isGranted (juce::RuntimePermissions::recordAudio))
{
juce::RuntimePermissions::request (juce::RuntimePermissions::recordAudio,
[&] (bool granted) { setAudioChannels (granted ? 2 : 0, 2); });
}
else
{
// Specify the number of input and output channels that we want to open
setAudioChannels (2, 2);
}
}
MainComponent::~MainComponent()
{
// This shuts down the audio device and clears the audio source.
shutdownAudio();
sample1.load("/Users/(username)/JuceTestPlugins/maxiSample/Source/kick.wav");
}
//==============================================================================
void MainComponent::prepareToPlay (int samplesPerBlockExpected, double sampleRate)
{
// This function will be called when the audio device is started, or when
// its settings (i.e. sample rate, block size, etc) are changed.
// You can use this function to initialise any resources you might need,
// but be careful - it will be called on the audio thread, not the GUI thread.
// For more details, see the help for AudioProcessor::prepareToPlay()
}
void MainComponent::getNextAudioBlock (const juce::AudioSourceChannelInfo& bufferToFill)
{
// Your audio-processing code goes here!
// For more details, see the help for AudioProcessor::getNextAudioBlock()
// Right now we are not producing any data, in which case we need to clear the buffer
// (to prevent the output of random noise)
//bufferToFill.clearActiveBufferRegion();
for(int sample = 0; sample < bufferToFill.buffer->getNumSamples(); ++sample){
//float sample2 = sample1.
//float wave = tesOsc.sinewave(200);
//double sample2 = sample1.play();
// leftSpeaker[sample] = (0.25 * wave);
// rightSpeaker[sample] = leftSpeaker[sample];
double *output;
output[0] = sample1.play();
output[1] = output[0];
}
}
void MainComponent::releaseResources()
{
// This will be called when the audio device stops, or when it is being
// restarted due to a setting change.
// For more details, see the help for AudioProcessor::releaseResources()
}
//==============================================================================
void MainComponent::paint (juce::Graphics& g)
{
// (Our component is opaque, so we must completely fill the background with a solid colour)
g.fillAll (getLookAndFeel().findColour (juce::ResizableWindow::backgroundColourId));
// You can add your drawing code here!
}
void MainComponent::resized()
{
// This is called when the MainContentComponent is resized.
// If you add any child components, this is where you should
// update their positions.
}
Can't say for sure, but couple of things catch my attention.
In getNextAudioBlock() you are dereferencing invalid pointers:
double *output;
output[0] = sample1.play();
output[1] = output[0];
The pointer variable output is uninitialised and will probably be filled with garbage or zeros, which will make the program read from invalid memory. This problem is most likely to cause the EXC_BAD_ACCESS. This method is called from the realtime audio thread, so you probably get a crash on a non-main thread (in this case the thread of External Headphones (8)).
It also is no clear to me what exactly it is you're trying to do here, so it's hard for me to say how it should be. What I can say is that assigning the result of sample1.play() to a double value looks suspicious.
Normally, when dealing with juce::AudioSourceChannelInfo you would get pointers to the audio buffers like so:
auto** bufferPointer = bufferToFill.buffer->getArrayOfWritePointers()
Further, you are loading a file inside the destructor of MainComponent. This at least is suspicious, why would you load a file during destruction?
MainComponent::~MainComponent()
{
// This shuts down the audio device and clears the audio source.
shutdownAudio();
sample1.load("/Users/(username)/JuceTestPlugins/maxiSample/Source/kick.wav");
}

Higher Than Expected Inherent Latency in OpenAL Playback (Windows, C++)

TL/DR: A simple echo program that records and plays back audio immediately is showing higher than expected latency.
I am working on a real-time audio broadcasting application. I have decided to use OpenAL to both capture and playback audio samples. I am planning on sending UDP packets with raw PCM data across a LAN network. My ideal latency between recording on one machine and playing back on another machine is 30ms. (A lofty goal).
As a test, I made a small program that records samples from a microphone and immediately plays them back to the host speakers. I did this to test the baseline latency.
However, I'm seeing an inherent latency of about 65 - 70 ms from simply recording the audio and playing it back.
I have reduced the buffer size that openAL uses to 100 samples at 44100 samples per second. Ideally, this would yield a latency of 2 - 3 ms.
I have yet to try this on another platform (MacOS / Linux) to determine if this is an OpenAL issue or a Windows issue.
Here's the code:
using std::list;
#define FREQ 44100 // Sample rate
#define CAP_SIZE 100 // How much to capture at a time (affects latency)
#define NUM_BUFFERS 10
int main(int argC, char* argV[])
{
list<ALuint> bufferQueue; // A quick and dirty queue of buffer objects
ALuint helloBuffer[NUM_BUFFERS], helloSource;
ALCdevice* audioDevice = alcOpenDevice(NULL); // Request default audio device
ALCcontext* audioContext = alcCreateContext(audioDevice, NULL); // Create the audio context
alcMakeContextCurrent(audioContext);
// Request the default capture device with a ~2ms buffer
ALCdevice* inputDevice = alcCaptureOpenDevice(NULL, FREQ, AL_FORMAT_MONO16, CAP_SIZE);
alGenBuffers(NUM_BUFFERS, &helloBuffer[0]); // Create some buffer-objects
// Queue our buffers onto an STL list
for (int ii = 0; ii < NUM_BUFFERS; ++ii) {
bufferQueue.push_back(helloBuffer[ii]);
}
alGenSources(1, &helloSource); // Create a sound source
short* buffer = new short[CAP_SIZE]; // A buffer to hold captured audio
ALCint samplesIn = 0; // How many samples are captured
ALint availBuffers = 0; // Buffers to be recovered
ALuint myBuff; // The buffer we're using
ALuint buffHolder[NUM_BUFFERS]; // An array to hold catch the unqueued buffers
alcCaptureStart(inputDevice); // Begin capturing
bool done = false;
while (!done) { // Main loop
// Poll for recoverable buffers
alGetSourcei(helloSource, AL_BUFFERS_PROCESSED, &availBuffers);
if (availBuffers > 0) {
alSourceUnqueueBuffers(helloSource, availBuffers, buffHolder);
for (int ii = 0; ii < availBuffers; ++ii) {
// Push the recovered buffers back on the queue
bufferQueue.push_back(buffHolder[ii]);
}
}
// Poll for captured audio
alcGetIntegerv(inputDevice, ALC_CAPTURE_SAMPLES, 1, &samplesIn);
if (samplesIn > CAP_SIZE) {
// Grab the sound
alcCaptureSamples(inputDevice, buffer, samplesIn);
// Stuff the captured data in a buffer-object
if (!bufferQueue.empty()) { // We just drop the data if no buffers are available
myBuff = bufferQueue.front(); bufferQueue.pop_front();
alBufferData(myBuff, AL_FORMAT_MONO16, buffer, samplesIn * sizeof(short), FREQ);
// Queue the buffer
alSourceQueueBuffers(helloSource, 1, &myBuff);
// Restart the source if needed
// (if we take too long and the queue dries up,
// the source stops playing).
ALint sState = 0;
alGetSourcei(helloSource, AL_SOURCE_STATE, &sState);
if (sState != AL_PLAYING) {
alSourcePlay(helloSource);
}
}
}
}
// Stop capture
alcCaptureStop(inputDevice);
alcCaptureCloseDevice(inputDevice);
// Stop the sources
alSourceStopv(1, &helloSource);
alSourcei(helloSource, AL_BUFFER, 0);
// Clean-up
alDeleteSources(1, &helloSource);
alDeleteBuffers(NUM_BUFFERS, &helloBuffer[0]);
alcMakeContextCurrent(NULL);
alcDestroyContext(audioContext);
alcCloseDevice(audioDevice);
return 0;
}
Here is an image of the waveform showing the delay in input sound and the resulting echo. This example is shows a latency of about 70ms.
Waveform with 70ms echo delay
System specs:
Intel Core i7-9750H
24 GB Ram
Windows 10 Home: V 2004 - Build 19041.508
Sound driver: Realtek Audio (Driver version 10.0.19041.1)
Input device: Logitech G330 Headset
Issue is reproducible on other Windows Systems.
EDIT:
I tried to use PortAudio to do a similar thing and achieved a similar result. I've determined that this is due to Windows audio drivers.
I rebuilt PortAudio with ASIO audio only and installed ASIO4ALL audio driver. This has achieved an acceptable latency of <10ms.
I ultimately resolved this issue by ditching OpenAL in favor of PortAudio and Steinberg ASIO. I installed ASIO4ALL and rebuilt PortAudio to accept only ASIO device drivers. I needed to use the ASIO SDK from Steinberg to do this. (Followed guide here). This has allowed me to achieve a latency of between 5 and 10 ms.
This post helped a lot: input delay with PortAudio callback and ASIO sdk

Set up a QBuffer as a FIFO with a restricted size ? QT

I have a continuous audio stream flow from which I'd like to keep only the last 50 seconds.
I do it like so for now but I have issues:
The buffer grows beyond 50 seconds. I tried to use resize on a QByteArray to get closer from a FIFO , but it doesn't seem to care much about it and continue to grow so I guess I misunderstood the doc.
The sound get suddenly ugly when we pass the 15 sec -> I have no clue why the sound suddenly rips (if I load a 5min local sound in the same buffer, everything works great).
Last, but not least of course, the buffer doesn't behave like a FIFO and this is what I'm looking for to keep only the last 50sec
Here is my code: (Windows10 - Qt5)
buffer = new QBuffer;
arr = new QByteArray;
arr->resize(200000);
buffer->setData(*arr); // buffer->setBuffer(*arr);
buffer->open(QIODevice::ReadWrite);
dataStream.setDevice(buffer); `
m_player = new QMediaPlayer(this);
m_player->setMedia(QMediaContent(), buffer);
[...]
connect(m_player, &QMediaPlayer::durationChanged, this, &MainWindow::durationChanged);
connect(m_player, &QMediaPlayer::positionChanged, this, &MainWindow::positionChanged);
I refresh the buffer duration in the latest slot.
So the buffer gets bigger than 50sec of course and the sound get chopped after 15 sec.
Does anyone know how to set up a FIFO buffer with a restricted size with Qt?
---------------------------------------------------------- EDIT ------------------------------------------------------------------
I found this to get closer but I miss something in it :
//.cpp :
#define SAMPLE_RATE 22050
#define CHANNELS 1
#define SAMPLE_SIZE 16
#define SAMPLE_TYPE SignedInt
myAudio::myAudio()
{
formatIn.setSampleRate(SAMPLE_RATE);
formatIn.setChannelCount(CHANNELS);
formatIn.setSampleSize(SAMPLE_SIZE);
[...]
formatIn.setByteOrder(QAudioFormat::LittleEndian);
formatIn.setSampleType(QAudioFormat::SAMPLE_TYPE);
formatOut. //same than formatIn
[...]
//configure device
audioOut = new QAudioOutput(deviceOut,formatOut,0);
audioIn = new QAudioInput (deviceIn, formatIn,0);
buff.resize(0x10000); //create a rx buffer
pbuff=buff.data(); //get the buff address;
RXbuff=0; //set RX buffer pointer
qDebug()<<"File open"<<open(QIODevice::ReadWrite);
qDebug()<<"is device Sequential="<<isSequential();
audioIn->start(this); //start reading device
audioOut->setVolume(0.5); //volume 0 to 1.0
audioOut->start(this); //start writing to device
}
//QIODevice Class (Protected Functions)This function is called by QIODevice.
//send to output(Speaker)
qint64 myAudio::readData(char *data, qint64 len)
{
static quint64 TXbuff=0;
qint64 total = 0;
while (len > total && RXbuff>TXbuff)//write and synchonise buffers
{
//write data to speaker
memcpy(&data[total],&pbuff[TXbuff%0x10000], 2); //copy 2 Bytes
TXbuff+=2; //point to next buffer 16 bit location
total+=2;
}
return total; //the reset interval
}
//audio input (from Microphone)
qint64 myAudio::writeData(const char *data, qint64 len)
{
int total=0;
while (len > total)
{
memcpy(&pbuff[RXbuff%0x10000],&data[total], 2); //write 2Bytes into
circular buffer(64K)
RXbuff+=2; //next 16bit buffer location
total+=2; //next data location
}
return (total); //return total number of bytes received
}
qint64 myAudio::bytesAvailable() const{return 0;}
What I miss is pretty basic... does anyone know When/How are the methods called ?!

VIDIOC_DQBUF hangs on camera disconnection

My application is using v4l2 running in a separate thread. If a camera gets disconnected then the user is given an appropriate message before terminating the thread cleanly. This works in the vast majority of cases. However, if the execution is inside the VIDIOC_DQBUF ioctl when the camera is disconnected then the ioctl doesn't return causing the entire thread to lock up.
My system is as follows:
Linux Kernel: 4.12.0
OS: Fedora 25
Compiler: gcc-7.1
The following is a simplified example of the problem function.
// Get Raw Buffer from the camera
void v4l2_Processor::get_Raw_Frame(void* buffer)
{
struct v4l2_buffer buf;
memset(&buf, 0, sizeof (buf));
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
// Grab next frame
if (ioctl(m_FD, VIDIOC_DQBUF, &buf) < 0)
{ // If the camera becomes disconnected when the execution is
// in the above ioctl, then the ioctl never returns.
std::cerr << "Error in DQBUF\n";
}
// Queue for next frame
if (ioctl(m_FD, VIDIOC_QBUF, &buf) < 0)
{
std::cerr << "Error in QBUF\n";
}
memcpy(buffer, m_Buffers[buf.index].buff,
m_Buffers[buf.index].buf_length);
}
Can anybody shed any light on why this ioctl locks up and what I might do to solve this problem?
I appreciate any help offered.
Amanda
I am currently having the same issue. However, my entire thread doesn't lock up. The ioctl times out (15s) but thats way too long.
Is there a what to query V4L2 (that wont hang) if video is streaming? or at least change the ioctl timeout ?
UPDATE:
#Amanda you can change the timeout of the dequeue in the v4l2_capture driver source & rebuild the kernel/kernel module
modify the timeout in the dqueue function:
if (!wait_event_interruptible_timeout(cam->enc_queue,
cam->enc_counter != 0,
50 * HZ)) // Modify this constant
Best of luck!

Does v4l2 camera capture with mmap ring buffer make sense for tracking application

I'm working on a v4l2 API for capturing images from a raw sensor on embedded platform. My capture routine is related to the example on [1]. The proposed method for streaming is using mmaped buffers as a ringbuffer.
For initialization, buffers (default = 4 buffers) are requested using ioctl with VIDIOC_REQBUFS identifier. Subsequently, they are queued using VIDIOC_QBUF. The entire streaming procedure is described here [2]. As soon as streaming starts, the driver fills the queued buffers with data. The timestamp of v4l2_buffer struct indicates the time of first byte captured which in my case results in a time interval of approximately 8.3 ms (=120fps) between buffers. So far so good.
Now what I would expect of a ring buffer is that new captures automatically overwrite older ones in a circular fashion. But this is not what happens. Only when a buffer is queued again (VIDIOC_QBUF) after it has been dequeued (VIDIOC_DQBUF) and processed (demosaic, tracking step,..), a new frame is assigned to a buffer. If I do meet the timing condition (processing < 8.3 ms) I don't get the latest captured frame when dequeuing but the oldest captured one (according to FIFO), so the one of 3x8.3 ms before the current one. If the timing condition is not met the time span gets even larger, as the buffers are not overwritten.
So I have several questions:
1. Does it even make sense for this tracking application to have a ring buffer as I don't really need history of frames? I certainly doubt that, but by using the proposed mmap method drivers mostly require a minimum amount of buffers to be requested.
2. Should a seperate thread continously DQBUF and QBUF to accomplish the buffer overwrite? How could this be accomplished?
3. As a workaround one could probably dequeue and requeue all buffers on every capture, but this doesn't sound right. Is there someone with more experience in real time capture and streaming who can point to the "proper" way to go?
4. Also currently, I'm doing the preprocessing step (demosaicing) between DQBUF and QBUF and the tracking step afterwards. Should the tracking step also be executed before QBUF is called again?
So the main code basically performs Capture() and Track() subsequently in a while loop. The Capture routine looks as follows:
cv::Mat v4l2Camera::Capture( size_t timeout ) {
fd_set fds;
FD_ZERO(&fds);
FD_SET(mFD, &fds);
struct timeval tv;
tv.tv_sec = 0;
tv.tv_usec = 0;
const bool threaded = true; //false;
// proper register settings
this->format2registerSetting();
//
if( timeout > 0 )
{
tv.tv_sec = timeout / 1000;
tv.tv_usec = (timeout - (tv.tv_sec * 1000)) * 1000;
}
//
const int result = select(mFD + 1, &fds, NULL, NULL, &tv);
if( result == -1 )
{
//if (EINTR == errno)
printf("v4l2 -- select() failed (errno=%i) (%s)\n", errno, strerror(errno));
return cv::Mat();
}
else if( result == 0 )
{
if( timeout > 0 )
printf("v4l2 -- select() timed out...\n");
return cv::Mat(); // timeout, not necessarily an error (TRY_AGAIN)
}
// dequeue input buffer from V4L2
struct v4l2_buffer buf;
memset(&buf, 0, sizeof(v4l2_buffer));
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP; //V4L2_MEMORY_USERPTR;
if( xioctl(mFD, VIDIOC_DQBUF, &buf) < 0 )
{
printf("v4l2 -- ioctl(VIDIOC_DQBUF) failed (errno=%i) (%s)\n", errno, strerror(errno));
return cv::Mat();
}
if( buf.index >= mBufferCountMMap )
{
printf("v4l2 -- invalid mmap buffer index (%u)\n", buf.index);
return cv::Mat();
}
// emit ringbuffer entry
printf("v4l2 -- recieved %ux%u video frame (index=%u)\n", mWidth, mHeight, (uint32_t)buf.index);
void* image_ptr = mBuffersMMap[buf.index].ptr;
// frame processing (& tracking step)
cv::Mat demosaic_mat = demosaic(image_ptr,mSize,mDepth,1);
// re-queue buffer to V4L2
if( xioctl(mFD, VIDIOC_QBUF, &buf) < 0 )
printf("v4l2 -- ioctl(VIDIOC_QBUF) failed (errno=%i) (%s)\n", errno, strerror(errno));
return demosaic_mat;
}
As my knowledge is limited regarding capturing and streaming video I appreciate any help.