I am doing my assignment to read a .rgb video file and display it in the window. I have only known how to read and display an image in C++. What should I do when reading the video and display it frame by frame. I don't want to use third party libraries, just pure C++ and windows programming.
My idea is: firstly load the whole video file into the program using fopen and allocate the buffer for it. Then just like display an image, I wanna treat the whole video as an array of frames, so after rendering the first frame, I will go to the next frame. In addition, how to keep the video display at a constant fps? If you have any learning resources or coding pieces, it would be very helpful!
Thanks
Since you haven't mentioned platform you are using.
But this snippet will help you to read file frame by frame.
#include <stdio.h>
int main()
{
FILE * fp = NULL;
int size = 800 * 600 * 2;
unsigned char * rawData = NULL;
fp = fopen("raw.rgb", "r+b");
rawData = (unsigned char *)malloc(size);
if (NULL == rawData)
return -1;
if (fp)
{
while(!feof(fp))
{
fread(rawData, size, 1, fp);
// GOT FRAME
}
fclose(fp);
fp = NULL;
}
}
Doing this without using any third-party library will be a lot of lot of works!
You may use the OpenCV library to do the work. Check http://opencv.org/
Related
I am writing a program using OpenCV that shall work on Windows as well as on Linux. Now the problem with OpenCV is, that its cv::imread function can not handle filepaths that contain non-ASCII characters on Windows. A workaround is to first read the file into a buffer using other libraries (for example std-libraries or Qt) and then read the file from that buffer using the cv::imdecode function. This is what I currently do. However, it's not very fast and much slower than just using cv::imread. I have a TIF image that is about 1GB in size. Reading it with cv::imread takes approx. 1s, reading it with the buffer method takes about 14s. I assume that imread just reads those parts of the TIF that are necessary for displaying the image (no layers etc.). Either this, or my code for reading a file into a buffer is bad.
Now my question is if there is a better way to do it. Either a better way with regard to OpenCV or a better way with regard to reading a file into a buffer.
I tried two different methods for the buffering, one using the std libraries and one using Qt (actually they both use QT for some things). They both are equally slow.:
Method 1
std::shared_ptr<std::vector<char>> readFileIntoBuffer(QString const& path) {
#ifdef Q_OS_WIN
std::ifstream file(path.toStdWString(), std::iostream::binary);
#else
std::ifstream file(path.toStdString(), std::iostream::binary);
#endif
if (!file.good()) {
return std::shared_ptr<std::vector<char>>(new std::vector<char>());
}
file.exceptions(std::ifstream::badbit | std::ifstream::failbit | std::ifstream::eofbit);
file.seekg(0, std::ios::end);
std::streampos length(file.tellg());
std::shared_ptr<std::vector<char>> buffer(new std::vector<char>(static_cast<std::size_t>(length)));
if (static_cast<std::size_t>(length) == 0) {
return std::shared_ptr<std::vector<char>>(new std::vector<char>());
}
file.seekg(0, std::ios::beg);
try {
file.read(buffer->data(), static_cast<std::size_t>(length));
} catch (...) {
return std::shared_ptr<std::vector<char>>(new std::vector<char>());
}
file.close();
return buffer;
}
And then for reading the image from the buffer:
std::shared_ptr<std::vector<char>> buffer = utility::readFileIntoBuffer(path);
cv::Mat image = cv::imdecode(*buffer, cv::IMREAD_UNCHANGED);
Method 2
QByteArray readFileIntoBuffer(QString const & path) {
QFile file(path);
if (!file.open(QIODevice::ReadOnly)) {
return QByteArray();
}
return file.readAll();
}
And for decoding the image:
QByteArray buffer = utility::readFileIntoBuffer(path);
cv::Mat matBuffer(1, buffer.size(), CV_8U, buffer.data());
cv::Mat image = cv::imdecode(matBuffer, cv::IMREAD_UNCHANGED);
UPDATE
Method 3
This method maps the file into memory using QFileDevice::map and then uses cv::imdecode.
QFile file(path);
file.open(QIODevice::ReadOnly);
unsigned char * fileContent = file.map(0, file.size(), QFileDevice::MapPrivateOption);
cv::Mat matBuffer(1, file.size(), CV_8U, fileContent);
cv::Mat image = cv::imdecode(matBuffer, cv::IMREAD_UNCHANGED);
However, also this approach didn't result in a shorter time than the other two. I also did some time measurements and found that reading the file in the memory or mapping it to the memory is actually not the bottleneck. The operation that takes the majority of the time is the cv::imdecode. I don't know why this is the case, since using cv::imread with the same image only takes a fraction of the time.
Potential Workaround
I tried obtaining an 8.3 pathname on Windows for files that contain non-ascii characters using the following code:
QString getShortPathname(QString const & path) {
#ifndef Q_OS_WIN
return QString();
#else
long length = 0;
WCHAR* buffer = nullptr;
length = GetShortPathNameW(path.toStdWString().c_str(), nullptr, 0);
if (length == 0) return QString();
buffer = new WCHAR[length];
length = GetShortPathNameW(path.toStdWString().c_str(), buffer, length);
if (length == 0) {
delete[] buffer;
return QString();
}
QString result = QString::fromWCharArray(buffer);
delete[] buffer;
return result;
#endif
}
However, I had to find out that 8.3 pathname generation is disabled on my machine, so it potentially is on others as well. So I wasn't able to test this yet and it does not seem to provide a reliable workaround. I also have the problem that the function doesn't tell me that 8.3 pathname generation is disabled.
There is an open ticket on this in OpenCV GitHub: https://github.com/opencv/opencv/issues/4292
One of the comments there suggest a workaround without reading the whole file to memory by using memory-mapped file (with help from Boost):
mapped_file map(path(L"filename"), ios::in);
Mat file(1, numeric_cast<int>(map.size()), CV_8S, const_cast<char*>(map.const_data()), CV_AUTOSTEP);
Mat image(imdecode(file, 1));
I have never worked with binary files before. I opened an .mp3 file using the mode ios::binary, read data from it, assigned 0 to each byte read and then rewrote them to another file opened in ios::binary mode. I opened the output file on a media player, it sounds corrupted but I can still hear the song. I want to know what happened physically.
How can I access/modify the raw data ( bytes ) of an audio ( video, images, ... ) using C++ ( to practice file encryption/decryption later )?
Here is my code:
#include <iostream>
#include <fstream>
#include <cstring>
using namespace std;
int main(){
char buffer[256];
ifstream inFile;
inFile.open("Backstreet Boys - Incomplete.mp3",ios::binary);
ofstream outFile;
outFile.open("Output.mp3",ios::binary);
while(!inFile.eof()){
inFile.read(buffer,256);
for(int i = 0; i<strlen(buffer); i++){
buffer[i] = 0;
}
outFile.write(buffer,256);
}
inFile.close();
outFile.close();
}
What you did has nothing to do with binary files or audio. You simply copied the file while zeroing some of the bytes. (The reason you didn't zero all of the bytes is because you use i<strlen(buffer), which simply counts up to the first zero byte rather than reporting the size of the buffer. Also you modify the buffer which means strlen(buffer) will report the length as zero after you zero the first byte.)
So the exact change in audio you get is entirely dependent on the mp3 file format and the audio compression it uses. MP3 is not an audio format that can be directly manipulated in useful ways.
If you want to manipulate digital audio, you need to learn about how raw audio is represented by computers.
It's actually not too difficult. For example, here's a program that writes out a raw audio file containing just a 400Hz tone.
#include <fstream>
#include <limits>
int main() {
const double pi = 3.1415926535;
double tone_frequency = 400.0;
int samples_per_second = 44100;
double output_duration_seconds = 5.0;
int output_sample_count =
static_cast<int>(output_duration_seconds * samples_per_second);
std::ofstream out("signed-16-bit_mono-channel_44.1kHz-sample-rate.raw",
std::ios::binary);
for (int sample_i = 0; sample_i < output_sample_count; ++sample_i) {
double t = sample_i / static_cast<double>(samples_per_second);
double sound_amplitude = std::sin(t * 2 * pi * tone_frequency);
// encode amplitude as a 16-bit, signed integral value
short sample_value =
static_cast<short>(sound_amplitude * std::numeric_limits<short>::max());
out.write(reinterpret_cast<char const *>(&sample_value),
sizeof sample_value);
}
}
To play the sound you need a program that can handle raw audio, such as Audacity. After running the program to generate the audio file, you can File > Import > Raw data..., to import the data for playing.
How can I access/modify the raw data ( bytes ) of an audio ( video, images, ... ) using C++ ( to practice file encryption/decryption later )?
As pointed out earlier, the reason your existing code is not completely zeroing out the data is because you are using an incorrect buffer size: strlen(buffer). The correct size is the number of bytes read() put into the buffer, which you can get with the function gcount():
inFile.read(buffer,256);
int buffer_size = inFile.gcount();
for(int i = 0; i < buffer_size; i++){
buffer[i] = 0;
}
outFile.write(buffer, buffer_size);
Note: if you were to step through your program using a debugger you probably would have pretty quickly seen the problem yourself when you noticed the inner loop executing less than you expected. Debuggers are a really handy tool to learn how to use.
I notice you're using open() and close() methods here. This is sort of pointless in this program. Just open the file in the constructor, and allow the file to be automatically closed when inFile and outFile go out of scope:
{
ifstream inFile("Backstreet Boys - Incomplete.mp3",ios::binary);
ofstream outFile("Output.mp3",ios::binary);
// don't bother calling .close(), it happens automatically.
}
I'm working on a project that will involve having to process PCM audio data through fft as its being played, preferably in sync. I'm using a linux g++ compiler and currently reading and playing audio data using OpenAL.
My question is this: is there a better way to process PCM audio data with an fft live as the audio is playing then using threads? If not, then what threading library would be best to use for these purposes.
this is my function that loads the wave data into an array of bytes, these can later be cast to ints for processing and all I use to play the data is OpenAL.
char* loadWAV(const char* fn, int& chan, int& samplerate, int& bps, int& size){
char buffer[4];
ifstream in(fn, ios::binary);
in.read(buffer, 4); //ChunkID "RIFF"
if(strncmp(buffer, "RIFF", 4) != 0){
cerr << "this is not a valid wave file";
return NULL;
}
in.read(buffer,4); //ChunkSize
in.read(buffer,4); //Format "WAVE"
in.read(buffer,4); // "fmt "
in.read(buffer,4); // 16
in.read(buffer,2); // 1
in.read(buffer,2); // NUMBER OF CHANNELS
chan = convertToInt(buffer,2);
in.read(buffer,4); // SAMPLE RATE
samplerate = convertToInt(buffer,4);
in.read(buffer,4); // ByteRate
in.read(buffer,2); // BlockAlign
in.read(buffer,2); // bits per sample
bps = convertToInt(buffer,2);
in.read(buffer,4); // "data"
in.read(buffer,4);
size = convertToInt(buffer,4);
char * data = new char[size];
in.read(data,size);
return data;
}
thank you for any and all help.
edit: to anyone who might be interested I wrote the function using this as a reference to know
how a WAV file is formated
Are you hoping to perform the FFT using OpenAL? I don't know if that's possible. Your code will likely be performing the FFT.
You don't need to explicitly set up any threads. However, your audio output library will probably do so on your behalf. I'm not familiar with OpenAL, but the way that a lot of audio libraries operate is by letting you specify a callback that will feed more audio into the output. Thus, your main program will load audio from the audio file, stuff it into a buffer (likely protected by a mutex) for the audio callback to read, compute an FFT over the audio window, and perhaps visualize the data for the user.
Again, the audio library will probably be managing the threading so you don't need to worry about the exact threading implementation or library. But be sure to manage shared data correctly with a mutex.
I'm trying to get an SDL_Surface* from a custom resource file.
This custom resource file, is get with this code;
http://content.gpwiki.org/index.php/C:Custom_Resource_Files
I packed a folder, wich contain a bitmap, a jpeg and a WAV sound.
I have a function returning a buffer, then with this buffer i'm able to load a surface using SDL_Rworps*.
It works fine when i'm tryin to get my BMP image, with SDL.
But my problem is to get the same effect with JPG and PNG using sdl_image.
Here are some code;
This function read the resource file (*resourcefilename) , and search for the file (*resourcename) we want to get. The last int param is a pointer handling the file size
char *GetBufferFromResource(char *resourcefilename, char *resourcename, int *filesize)
{
//Try to open the resource file in question
int fd = open(resourcefilename, O_RDONLY);
if (fd < 0){perror("Error opening resource file"); exit(1);}
//Make sure we're at the beginning of the file
lseek(fd, 0, SEEK_SET);
//Read the first INT, which will tell us how many files are in this resource
int numfiles;
read(fd, &numfiles, sizeof(int));
//Get the pointers to the stored files
int *filestart = (int *) malloc(sizeof(int) * numfiles); // this is probably wrong in the zip
read(fd, filestart, sizeof(int) * numfiles);
//Loop through the files, looking for the file in question
int filenamesize;
char *buffer;
int i;
for(i=0;i<numfiles;i++)
{
char *filename;
//Seek to the location
lseek(fd, filestart[i], SEEK_SET);
//Get the filesize value
read(fd, filesize, sizeof(int));
//Get the size of the filename string
read(fd, &filenamesize, sizeof(int));
//Size the buffer and read the filename
filename = (char *) malloc(filenamesize + 1);
read(fd, filename, filenamesize);
//Remember to terminate the string properly!
filename[filenamesize] = '\0';
//Compare to the string we're looking for
if (strcmp(filename, resourcename) == 0)
{
//Get the contents of the file
buffer = (char *) malloc(*filesize);
read(fd, buffer, *filesize);
free(filename);
break;
}
//Free the filename buffer
free(filename);
}
//Release memory
free(filestart);
//Close the resource file!
close(fd);
//Did we find the file within the resource that we were looking for?
if (buffer == NULL)
{
printf("Unable to find '%s' in the resource file!\n", resourcename);
exit(1);
}
//Return the buffer
return buffer;
}
Now that's my function returning a SDL_Surface* ( for BMP ) note that this function use SDL_Image "IMG_LoadBMP_RW()"
SDL_Surface *LoadBMP(char *resourcefilename, char *imagefilename){
//Get the image's buffer and size from the resource file
int filesize = 0;
char *buffer = GetBufferFromResource(resourcefilename, imagefilename, &filesize);
//Load the buffer into a surface using RWops
SDL_RWops *rw = SDL_RWFromMem(buffer, filesize);
if(IMG_isBMP(rw))
printf("This is a BMP file.\n");
else
printf("This is not a BMP file, or BMP support is not available.\n");
SDL_Surface *temp = IMG_LoadBMP_RW(rw);
free(buffer);
//Return our loaded image
printf("IMG size: %d x %d\n", temp->w, temp->h);
SDL_Surface *image;
image = SDL_DisplayFormat(temp);
SDL_FreeSurface(temp);
return image;
}
But when i try to use the same function, modified for JPG, i get on my stdout:
This is not a JPG file, or JPG support is not available.
I ask for your help, if someone want, i can upload entire source code, or at least a simplified version with the resource file.
I recently followed the same SDL custom resource tutorial, extending its functionality to allow unpacking of resource files into the original file formats. I think I'm having a problem which relates to yours, and may provide more insight into the issue.
The code works fine packing and unpacking the specific file types the tutorial addresses, namely BMP and WAV. It also packs and unpacks OGG files with no issues. However, the process does strip TXT files of all line break formatting. And somewhere in the packing / unpacking process PNG files are completely corrupted. I have not tried with JPG, but they could be suffering the same fate as the PNG, and if this happens during packing, then this could explain why your JPGs won't load using SDL_Rwops.
I will test with a JPG tonight, and run the before and after files through a checksum to see if there are any discrepancies.
Hey all, I'm writing an application which records microphone input to a WAV file. Previously, I had written this to fill a buffer of a specified size and that worked fine. Now, I'd like to be able to record to an arbitrary length. Here's what I'm trying to do:
Set up 32 small audio buffers (circular buffering)
Start a WAV file with ofstream -- write the header with PCM length set to 0
Add a buffer to input
When a buffer completes, append its data to the WAV file and update the header; recycle the buffer
When the user hits "stop", write the remaining buffers to file and close
It kind of works in that the files are coming out to the correct length (header and file size and are correct). However, the data is wonky as hell. I can make out a semblance of what I said -- and the timing is correct -- but there's this repetitive block of distortion. It basically sounds like only half the data is getting into the file.
Here are some variables the code uses (in header)
// File writing
ofstream mFile;
WAVFILEHEADER mFileHeader;
int16_t * mPcmBuffer;
int32_t mPcmBufferPosition;
int32_t mPcmBufferSize;
uint32_t mPcmTotalSize;
bool mRecording;
Here is the code that prepares the file:
// Start recording audio
void CaptureApp::startRecording()
{
// Set flag
mRecording = true;
// Set size values
mPcmBufferPosition = 0;
mPcmTotalSize = 0;
// Open file for streaming
mFile.open("c:\my.wav", ios::binary|ios::trunc);
}
Here's the code that receives the buffer. This assumes the incoming data is correct -- it should be, but I haven't ruled out that it isn't.
// Append file buffer to output WAV
void CaptureApp::writeData()
{
// Update header with new PCM length
mPcmBufferPosition *= sizeof(int16_t);
mPcmTotalSize += mPcmBufferPosition;
mFileHeader.bytes = mPcmTotalSize + sizeof(WAVFILEHEADER);
mFileHeader.pcmbytes = mPcmTotalSize;
mFile.seekp(0);
mFile.write(reinterpret_cast<char *>(&mFileHeader), sizeof(mFileHeader));
// Append PCM data
if (mPcmBufferPosition > 0)
{
mFile.seekp(mPcmTotalSize - mPcmBufferPosition + sizeof(WAVFILEHEADER));
mFile.write(reinterpret_cast<char *>(&mPcmBuffer), mPcmBufferPosition);
}
// Reset file buffer position
mPcmBufferPosition = 0;
}
And this is the code that closes the file:
// Stop recording
void CaptureApp::stopRecording()
{
// Save remaining data
if (mPcmBufferSize > 0)
writeData();
// Close file
if (mFile.is_open())
{
mFile.flush();
mFile.close();
}
// Turn off recording flag
mRecording = false;
}
If there's anything here that looks like it would result in bad data getting appended to the file, please let me know. If not, I'll triple check the input data (in the callback). This data should be good, because it works if I copy it to a larger buffer (eg, two minutes) and then save that out.
I am just wondering, how
void CaptureApp::writeData()
{
mPcmBufferPosition *= sizeof(int16_t); // mPcmBufferPosition = 0, so 0*2 = 0;
// (...)
mPcmBufferPosition = 0;
}
works (btw. sizeof int16_t is always 2). Are you setting mPcmBufferPosition somewhere else?
void CaptureApp::writeData()
{
// Update header with new PCM length
long pos = mFile.tellp();
mPcmBufferBytesToWrite *= 2;
mPcmTotalSize += mPcmBufferBytesToWrite;
mFileHeader.bytes = mPcmTotalSize + sizeof(WAVFILEHEADER);
mFileHeader.pcmbytes = mPcmTotalSize;
mFile.seekp(0);
mFile.write(reinterpret_cast<char *>(&mFileHeader), sizeof(mFileHeader));
mFile.seekp(pos);
// Append PCM data
if (mPcmBufferBytesToWrite > 0)
mFile.write(reinterpret_cast<char *>(mPcmBuffer), mPcmBufferBytesToWrite);
}
Also mPcmBuffer is a pointer, so don't know why you use & in write.
The most likely reason is you're writing from the address of the pointer to your buffer, not from the buffer itself. Ditch the "&" in the final mFile.write. (It may have some good data in it if your buffer is allocated nearby and you happen to grab a chunk of it, but that's just luck that your write hapens to overlap your buffer)
In general, if you find yourself in this sort of situation, you could try to think how you can test this code in isolation from the recording code: Set up a buffer that has the values 0..255 in it, and then set your "chunk size" to 16 and see if it writes out a continuous sequence of 0..255 across 16 separate write operations. That will quickly verify if your buffering code is working or not.
I won't debug your code, but will try to give you checklist of the things you can try to check and determine where's the error:
always have referent recorder or player handy. It can be something as simple as Windows Sound Recorder, or Audacity, or Adobe Audition. Have a recorder/player that you are CERTAIN that will record and play files correctly.
record the file with your app and try to play it with reference player. Working?
try to record the file with reference recorder, and play it with your player. Working?
when you write SOUND data to the WAV file in your recorder, write it to one extra file. Open that file in RAW mode with the player (Windows Sound Recorder won't be enough here). Does it play correctly?
when playing the file in your player, and writing to the soundcard, write output to the RAW file, to see if you are playing the data correctly at all or you have soundcars issues. Does it play correctly?
Try all this, and you'll have much better idea of where something went wrong.
Shoot, sorry -- had a late night of work and am a bit off today. I forgot to show y'all the actual callback. This is it:
// Called when buffer is full
void CaptureApp::onData(float * data, int32_t & size)
{
// Check recording flag and buffer size
if (mRecording && size <= BUFFER_LENGTH)
{
// Save the PCM data to file and reset the array if we
// don't have room for this buffer
if (mPcmBufferPosition + size >= mPcmBufferSize)
writeData();
// Copy PCM data to file buffer
copy(mAudioInput.getData(), mAudioInput.getData() + size, mPcmBuffer + mPcmBufferPosition);
// Update PCM position
mPcmBufferPosition += size;
}
}
Will try y'alls advice and report.