Traverse an raw video more efficiently - c++

I have a raw video file , and i make in qt , an app that reads frame by frame from this file .At large raw files when i press an button that goes to the next frame there is a big delay almost one sec .
Here is my code that returns an frame from raw file :
void RawVideoReader::getFrame(int offset)
{
std::cout<<"getFrame"<<std::endl;
file.seek((unsigned long long int)(((unsigned long long int)width * (unsigned long long int)height) * (unsigned long long int)offset));
QByteArray array = file.read(width * height);
const std::size_t count = array.size();
hex = std::unique_ptr<unsigned char>(new unsigned char[count]);
std::memcpy(hex.get(), array.constData(), count);
}

You can read directly into the buffer you desire - the question is: why do you want to manage this memory buffer using unique_ptr? QByteArray already does that job. Furthermore, you probably want to keep the same buffer, and not reallocate it over and over.
class RawVideoReader : ... {
QByteArray frame;
uint8_t *frameData() const { return frame.size() ? static_cast<uint8_t*>(frame.constData()) : nullptr; }
size_t frameSize() const { return static_cast<size_t>(frame.size()); }
...
};
bool RawVideoReader::getFrame(int frameNo) {
qDebug() << __FUNCTION__;
frame.resize(width * height * 1);
file.seek(qint64(frame.size()) * qint64(frameNo));
auto const hadRead = file.read(frame.data(), frame.size());
return hadRead == frame.size();
}

Related

What's the difference between initializing a vector in Class Header or Class constructor body?

I encountered a strange behavior in my C++ program that I don't understand and I don't know how to search for more information. So I ask for advice here hoping someone might know.
I have a class Interface that has a 2 dimensional vector that I initialize in the header :
class Interface {
public:
// code...
const unsigned short int SIZE_X_ = 64;
const unsigned short int SIZE_Y_ = 32;
std::vector<std::vector<bool>> screen_memory_ =
std::vector<std::vector<bool>>(SIZE_X_, std::vector<bool>(SIZE_Y_, false));
// code...
};
Here I expect that I have a SIZE_X_ x SIZE_Y_ vector filled with false booleans.
Later in my program I loop at a fixed rate like so :
void Emulator::loop() {
const milliseconds intervalPeriodMillis{static_cast<int>((1. / FREQ) * 1000)};
//Initialize the chrono timepoint & duration objects we'll be //using over & over inside our sleep loop
system_clock::time_point currentStartTime{system_clock::now()};
system_clock::time_point nextStartTime{currentStartTime};
while (!stop) {
currentStartTime = system_clock::now();
nextStartTime = currentStartTime + intervalPeriodMillis;
// ---- Stuff happens here ----
registers_->trigger_timers();
interface_->toogle_buzzer();
interface_->poll_events();
interface_->get_keys();
romParser_->step();
romParser_->decode();
// ---- ------------------ ----
stop = stop || interface_->requests_close();
std::this_thread::sleep_until(nextStartTime);
}
}
But then during the execution I get a segmentation fault
[1] 7585 segmentation fault (core dumped) ./CHIP8 coin.ch8
I checked with the debugger and some part of the screen_memory_ cannot be accessed anymore. And it seems to happen at random time.
But when I put the initialization of the vector in the constructor body like so :
Interface::Interface(const std::shared_ptr<reg::RegisterManager> & registers, bool hidden)
: registers_(registers) {
// code ...
screen_memory_ =
std::vector<std::vector<bool>>(SIZE_X_, std::vector<bool>(SIZE_Y_, false));
// code ...
}
The segmentation fault doesn't happen anymore. So the solution is just to initialize the vector in the constructor body.
But why ? what is happening there ?
I don't understand what I did wrong, I'm sure someone knows.
Thanks for your help !
[Edit] I found the source of the bug (Or at least what to change so it doesnt give me a segfault anymore).
In my class Interface I use the SDL and SDL_audio libraries to create the display and the buzzer sound. Have a special look where I set the callback want_.callback, the callback Interface::forward_audio_callback and Interface::audio_callback. Here's the code :
// (c) 2021 Maxandre Ogeret
// Licensed under MIT License
#include "Interface.h"
Interface::Interface(const std::shared_ptr<reg::RegisterManager> & registers, bool hidden)
: registers_(registers) {
if (SDL_Init(SDL_INIT_AUDIO != 0) || SDL_Init(SDL_INIT_VIDEO) != 0) {
throw std::runtime_error("Unable to initialize rendering engine.");
}
want_.freq = SAMPLE_RATE;
want_.format = AUDIO_S16SYS;
want_.channels = 1;
want_.samples = 2048;
want_.callback = Interface::forward_audio_callback;
want_.userdata = &sound_userdata_;
if (SDL_OpenAudio(&want_, &have_) != 0) {
SDL_LogError(SDL_LOG_CATEGORY_AUDIO, "Failed to open audio: %s", SDL_GetError());
}
if (want_.format != have_.format) {
SDL_LogError(SDL_LOG_CATEGORY_AUDIO, "Failed to get the desired AudioSpec");
}
window = SDL_CreateWindow("CHIP8", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED,
SIZE_X_ * SIZE_MULTIPLIER_, SIZE_Y_ * SIZE_MULTIPLIER_,
hidden ? SDL_WINDOW_HIDDEN : 0);
renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_SOFTWARE);
bpp_ = SDL_GetWindowSurface(window)->format->BytesPerPixel;
SDL_Delay(1000);
// screen_memory_ = std::vector<std::vector<bool>>(SIZE_X_, std::vector<bool>(SIZE_Y_, false));
}
Interface::~Interface() {
SDL_CloseAudio();
SDL_DestroyWindow(window);
SDL_Quit();
}
// code ...
void Interface::audio_callback(void * user_data, Uint8 * raw_buffer, int bytes) {
audio_buffer_ = reinterpret_cast<Sint16 *>(raw_buffer);
sample_length_ = bytes / 2;
int & sample_nr(*(int *) user_data);
for (int i = 0; i < sample_length_; i++, sample_nr++) {
double time = (double) sample_nr / (double) SAMPLE_RATE;
audio_buffer_[i] = static_cast<Sint16>(
AMPLITUDE * (2 * (2 * floor(220.0f * time) - floor(2 * 220.0f * time)) + 1));
}
}
void Interface::forward_audio_callback(void * user_data, Uint8 * raw_buffer, int bytes) {
static_cast<Interface *>(user_data)->audio_callback(user_data, raw_buffer, bytes);
}
}
In the function Interface::audio_callback, replacing the class variable assignation :
sample_length_ = bytes / 2;
By an int creation and assignation :
int sample_length = bytes / 2;
which gives :
void Interface::audio_callback(void * user_data, Uint8 * raw_buffer, int bytes) {
audio_buffer_ = reinterpret_cast<Sint16 *>(raw_buffer);
int sample_length = bytes / 2;
int &sample_nr(*(int*)user_data);
for(int i = 0; i < sample_length; i++, sample_nr++)
{
double time = (double)sample_nr / (double)SAMPLE_RATE;
audio_buffer_[i] = (Sint16)(AMPLITUDE * sin(2.0f * M_PI * 441.0f * time)); // render 441 HZ sine wave
}
}
The class variable sample_length_ is defined and initialized as private in the header like so :
int sample_length_ = 0;
So I had an idea and I created the variable sample_length_ as public and it works ! So the problem was definitely a scope problem of the class variable sample_length_. But it doesn't explain why the segfault disappeared when I moved the init of some other variable in the class constructor... Did I hit some undefined behavior with my callback ?
Thanks for reading me !

How to compress/decompress buffer using Fast-LZMA2

I want to compress/decompress a unsigned char buffer using fast-LZMA2 by 7Zip : https://github.com/conor42/fast-lzma2
In the sample there's two function :
static int compress_file(FL2_CStream *fcs)
{
unsigned char in_buffer[8 * 1024];
unsigned char out_buffer[4 * 1024];
FL2_inBuffer in_buf = { in_buffer, sizeof(in_buffer), sizeof(in_buffer) };
FL2_outBuffer out_buf = { out_buffer, sizeof(out_buffer), 0 };
size_t res = 0;
size_t in_size = 0;
size_t out_size = 0;
do {
if (in_buf.pos == in_buf.size) {
in_buf.size = fread(in_buffer, 1, sizeof(in_buffer), fin);
in_size += in_buf.size;
in_buf.pos = 0;
}
res = FL2_compressStream(fcs, &out_buf, &in_buf);
if (FL2_isError(res))
goto error_out;
fwrite(out_buf.dst, 1, out_buf.pos, fout);
out_size += out_buf.pos;
out_buf.pos = 0;
} while (in_buf.size == sizeof(in_buffer));
do {
res = FL2_endStream(fcs, &out_buf);
if (FL2_isError(res))
goto error_out;
fwrite(out_buf.dst, 1, out_buf.pos, fout);
out_size += out_buf.pos;
out_buf.pos = 0;
} while (res);
fprintf(stdout, "\t%ld -> %ld\n", in_size, out_size);
return 0;
error_out:
fprintf(stderr, "Error: %s\n", FL2_getErrorName(res));
return 1;
}
static int decompress_file(FL2_DStream *fds)
{
unsigned char in_buffer[4 * 1024];
unsigned char out_buffer[8 * 1024];
FL2_inBuffer in_buf = { in_buffer, sizeof(in_buffer), sizeof(in_buffer) };
FL2_outBuffer out_buf = { out_buffer, sizeof(out_buffer), 0 };
size_t res;
size_t in_size = 0;
size_t out_size = 0;
do {
if (in_buf.pos == in_buf.size) {
in_buf.size = fread(in_buffer, 1, sizeof(in_buffer), fout);
in_size += in_buf.size;
in_buf.pos = 0;
}
res = FL2_decompressStream(fds, &out_buf, &in_buf);
if (FL2_isError(res))
goto error_out;
/* Discard the output. XXhash will verify the integrity. */
out_size += out_buf.pos;
out_buf.pos = 0;
} while (res && in_buf.size);
fprintf(stdout, "\t%ld -> %ld\n", in_size, out_size);
return 0;
error_out:
fprintf(stderr, "Error: %s\n", FL2_getErrorName(res));
return 1;
}
But I have no idea how to make it work with a buffer and also without size limit like 8*1024
like zlib deflate compression.
I want something like
LZMA2_Compress(void* buffer,size_t bufferSize);
and LZMA2_Decompress(void* buffer,size_t bufferSize);
I want to use this algorithm on some heavy files and Fast LZMA2 is the fastest high ratio compression I found, Please don't suggest me using other methods.
Here's my test code, It's working but just need to correct information:
https://gist.github.com/Bit00009/3241bb66301f8aaba16074537d094e61
Check the header file for all of the functions available. This one looks like the one you need. You will need to cast your buffers as (void *).
High level functions
fast-lzma2.h
...
/*! FL2_compress() :
* Compresses `src` content as a single LZMA2 compressed stream into already allocated `dst`.
* Call FL2_compressMt() to use > 1 thread. Specify nbThreads = 0 to use all cores.
* #return : compressed size written into `dst` (<= `dstCapacity),
* or an error code if it fails (which can be tested using FL2_isError()). */
FL2LIB_API size_t FL2LIB_CALL FL2_compress(void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
int compressionLevel);
...
Management of memory and options
To do explicit memory management (set dictionary size, buffer size, etc.) you need to create a context:
fast-lzma2.h
/*= Compression context
* When compressing many times, it is recommended to allocate a context just once,
* and re-use it for each successive compression operation. This will make workload
* friendlier for system's memory. The context may not use the number of threads requested
* if the library is compiled for single-threaded compression or nbThreads > FL2_MAXTHREADS.
* Call FL2_getCCtxThreadCount to obtain the actual number allocated. */
typedef struct FL2_CCtx_s FL2_CCtx;
FL2LIB_API FL2_CCtx* FL2LIB_CALL FL2_createCCtx(void);
than you can use FL2_CCtx_setParameter() to set the parameters in the context. The possible values for the paramters are listed in FL2_cParameter , and the value FL2_p_dictionarySize will allow you to set the dictionary size.
/*! FL2_CCtx_setParameter() :
* Set one compression parameter, selected by enum FL2_cParameter.
* #result : informational value (typically, the one being set, possibly corrected),
* or an error code (which can be tested with FL2_isError()). */
FL2LIB_API size_t FL2LIB_CALL FL2_CCtx_setParameter(FL2_CCtx* cctx, FL2_cParameter param, size_t value);
Finally you can compress the buffer by calling FL2_compressCCtx()
/*! FL2_compressCCtx() :
* Same as FL2_compress(), but requires an allocated FL2_CCtx (see FL2_createCCtx()). */
FL2LIB_API size_t FL2LIB_CALL FL2_compressCCtx(FL2_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
int compressionLevel);

SoundTouch library messes up the ending when pitch-shifting

I'm using the SoundTouch library to pitch-shift some audio files. Everything works well, except the last few hundred milliseconds of the new audio file are not like the original file. Here is the original file:
And here's what I get after pitch-shifting:
As you can see the ending is not right. It's like there was nothing there in the original file, when there certainly is.
Here's the code I'm using:
int generateFile(WavInFile *file, SoundTouch *st, string fileName, int semitones)
{
const bool speech = true;
SAMPLETYPE samples[BUFF_SIZE];
WavOutFile *out = new WavOutFile(fileName.c_str(), (int)file->getSampleRate(), (int)file->getNumBits(), (int)file->getNumChannels());
int nChannels = (int)file->getNumChannels();
assert(nChannels > 0);
int num, nSamples;
int buffSizeSamples = BUFF_SIZE / nChannels;
st->setSampleRate((int)file->getSampleRate());
st->setChannels(nChannels);
st->setPitchSemiTones(semitones);
if (!speech)
{
st->setSetting(SETTING_USE_QUICKSEEK, 0);
st->setSetting(SETTING_USE_AA_FILTER, 0);
}
else
{
st->setSetting(SETTING_USE_QUICKSEEK, 0);
st->setSetting(SETTING_SEQUENCE_MS, 40);
st->setSetting(SETTING_SEEKWINDOW_MS, 15);
st->setSetting(SETTING_OVERLAP_MS, 8);
}
while (file->eof() == 0)
{
num = file->read(samples, BUFF_SIZE);
nSamples = num / (int)file->getNumChannels();
st->putSamples(samples, nSamples);
do
{
nSamples = st->receiveSamples(samples, buffSizeSamples);
out->write(samples, nSamples * nChannels);
} while (nSamples != 0);
}
st->flush();
do
{
nSamples = st->receiveSamples(samples, buffSizeSamples);
out->write(samples, nSamples * nChannels);
} while (nSamples != 0);
delete out;
return 0;
}
And yes, I delete WavInFile *file later in my code. So my question is- Why is SoundTouch doing this and how can I fix it?
Also I cannot simply cut the wrong part of the new audio file because I'm generating hundreds of files this way so cutting every single one of them would be...

Packging LEFT and RIGHT channel data

I am decoding FLAC audio into memory, and passing the decoded audio data to the OpenAL: void alBufferData (ALuint bufferName, ALenum format, const ALvoid *data, ALsizei size, ALsizei frequency);
The data from the decoded audio goes into mine std::vector<FLAC__int32> data_;. Into which I am attempting to package the LEFT and RIGHT channels (AL_FORMAT_STEREO16). However, I don't understand how I am to store/align these channels within my data_ vector.
So I have the libFLAC virtual callback member function:
FLAC__StreamDecoderWriteStatus
Source::write_callback (
FLAC__Frame const* _frame, FLAC__int32 const *const _buffer[])
{
for(size_t i(0); i < _frame->header.blocksize; i++) {
data_[index_] = _buffer[0][i]; // channel audio on the left
++index_;
data_[index_] = _buffer[1][i]; // what about the right channel?
} // jump
return FLAC__STREAM_DECODER_WRITE_STATUS_CONTINUE;
} // main
At the moment, during audio playback, I am hearing only the LEFT channel. There is static sound after the sound has finished playing which I am assuming is the missing RIGHT channel data. How do i get the RIGHT channel to work also?
Also, this is the metadata callback signature as per libFLAC:
void
Source::metadata_callback (const ::FLAC__StreamMetadata *metadata)
{
total_samples_ = metadata->data.stream_info.total_samples;
rate_ = metadata->data.stream_info.sample_rate;
channels_ = metadata->data.stream_info.channels;
bps_ = metadata->data.stream_info.bits_per_sample;
switch (bps_) {
case 16 :
if (channels_ > 1) {
format_ = AL_FORMAT_STEREO16; } else {
format_ = AL_FORMAT_MONO16; }
break;
case 8 :
if (channels_ > 1) {
format_ = AL_FORMAT_STEREO8; } else {
format_ = AL_FORMAT_MONO8; }
break;
default:
break;
}
size_ = (ALuint)(rate_ * channels_ * (bps_ / 8));
data_.resize(total_samples_); index_ = 0;
} // main
A solution which works, was to assagned the below struct as the vector data type like so:
struct Data
{
FLAC__int16 channelLeft_;
FLAC__int16 channelRight_;
};
std::vector<Source::Data> data_;
than assign the size_ like so:
size_ = total_samples_ * sizeof(Source::Data);
finaly the data loop should now be:
for(size_t i(0); i < _frame->header.blocksize; i++) {
data_[index_].channelLeft_ = _buffer[0][i];
data_[index_].channelRight_ = _buffer[1][i];
++index_;
} // jump

Resource instead of external file C++

I am reusing some old code(originally developed on c, not c++) with some functions to open/read/manipulate text-files. The path to the text-files is passed to the functions as a string (char*) then opened using: FileToUse = fopen(filename, "rb"); then multiple calls to fread() and fseek() are used. This code is known to work for external text-files, but now I would like to include the textfiles as resources in my project (MFC C++ in visual studio).
I found some examples on the web on how to use resources rusulting in this code:
HINSTANCE hInst = AfxGetResourceHandle();
HRSRC hResource = FindResource(hInst, MAKEINTRESOURCE(IDR_TEXTFILE1), "TEXTFILE");
if (hResource){
HGLOBAL hLoadedResource = LoadResource(hInst, hResource);
if (hLoadedResource){
const char* pResource = LockResource(hLoadedResource);
if (pResource){
DWORD dwResourceSize = SizeofResource(hInst, hResource);
if (0 != dwResourceSize){ // if(FileToUse){
memcpy(&Buffer, pResource, (15 * 2)); // fread($Buffer, 15, 2, FileToUse);
pTemp = pResource + 200; // fseek(FileToUse, 200, SEEK_SET);
pTemp = pTemp + 100; // fseek(FileToUse, 100, SEEK_CUR);
pTemp = pResource + (dwResourceSize - 1) - 40; // fseek(FileToUse, -40, SEEK_END);
}
}
}
}
I replaced the fread call by memcpy() as shown, but I'm missing the return value of fread (actual read items) and in the original code the filepointer was moved by fseek, I wonder whether my approach using a temporary pointer is correct.
My ultimate goal is to simulate the fread and fseek calls for resources with similar function prototypes:
size_t resread( void* buffer, size_t size, size_t count, char* resource );
int resseek( char* resource, long offset, int origin );
Any suggestions are much appreciated.
Thanks for your help, based on the Agent_L's suggestion this is what I came up with:
Text-resource type:
struct _resource {
const char * content; // File content
size_t size; // File size
size_t ptrloc; // 'Pointer' location
};
typedef struct _resource RES_TXT;
resread based on fread:
size_t resread( void* buffer, size_t size, size_t count, RES_TXT * resource)
{
size_t actualCount = ( resource->size - resource->ptrloc ) / size;
actualCount = min( count, actualCount );
if ( actualCount <= 0 ) return 0;
memcpy(buffer, (resource->_ptr + resource->ptrloc), (actualCount * size) );
resource->ptrloc += (actualCount * size);
return actualCount;
}
and to complete resseek based on fseek:
int resseek( RES_TXT * resource, long offset, int origin ) {
size_t nextloc;
switch ( origin ) {
case SEEK_SET: nextloc = 0;
break;
case SEEK_CUR: nextloc = resource->ptrloc;
break;
case SEEK_END: nextloc = resource->size;
break;
default: return -1;
}
nextloc += offset;
if ( nextloc >= 0 && nextloc < resource->size )
resource->ptrloc = nextloc;
else
return -1;
return 0;
}
Any call to fseek and fread can now be replaced to use a resource instead of an external file.
The file handle contains not only the data but also it's length and current position. You have to duplicate that.
(handwirtten code, unproven):
struct resFile
{
char* pData;
int iLenght;
int iCurrPosition;
};
size_t resread( void* buffer, size_t size, size_t count, resFile* resource)
{
int ActualRead = min(size*count, resource->iLenght - resource->iCurrPosition);
memcpy(buffer, resource->pData + resource->iCurrPosition, ActualRead);
resource->iCurrPostion += ActualRead;
return ActualRead;
}
Let me notify you that fread shifts current file position. This means that you don't need invoke fseek each time. From this perspective may be you code can avoid implementation of resseek by simple increasing Buffer pointer