I am implementing an audio player application that pre-buffers a small part of the audio data and reads the rest of the data when it is required to do so, for example when the play command arrives. It's a real time application, to it's really important that theres near zero latency between the play command and the start of the playback.
Example: my audio stream is 10 Mb, I read part of it when the file is selected and start creating a buffer like this:
// Stuff to do as soon as the file is selected
// Allocate new memory for the current sample
// contains sample length in number of samples
sampleSize = SampleLib.smp.len;
// assume it's a 16-bit audio file, each sample is 2 bytes long
byteSize = sampleSize * sizeof(short);
// Allow 10 extra samples and fill with zeroes
SampleData = new short[sampleSize + 10]();
// PRELOAD_BYTESIZE is set to 65535 bytes
preloadByteSize = byteSize > PRELOAD_BYTESIZE ? PRELOAD_BYTESIZE : byteSize;
// Set pointer in file - WavePointer contains the exact location where the sample data starts in file
fseek(inFile, WavePointer, SEEK_SET);
// read preloadByteSize from inFile into SampleData
fread(SampleData, 1, preloadByteSize, inFile);
At this point my buffer SampleData contains only part of the audio data to start playing back as soon as the play command arrives. At the same time, the program should fill the rest of the buffer and continue playing up until the end of the audio sample with no interruption.
// Stuff to do as soon the playback starts
// Load the rest of the sample data
// If file is already in memory, avoid reading it again
if (preloadByteSize < ByteSize)
{
// Set pointer in file at stample start + preload size
fseek(fpFile, WavePointers + preloadByteSize, SEEK_SET);
// read the remaining bytes from inFile and fill the empty part of the buffer
fread(SampleData + preloadByteSize / sizeof(short), 1, ByteSize - preloadByteSize, inFile);
// remember the number of loaded bytes
preloadByteSize = ByteSize;
}
I expect that the second part of the code is executed in the background while the file is playing back, but actually it's all serial processing, so playback won't start until the rest of the buffer is loaded, thus retarding the playback.
Ho can I have a background thread that loads the file data without interfering with the audio task? Can I do this with OpenMP?
You might be able to do this with OpenMP, but this involves concurrency more than parallelism, so I would look at pthreads or C++11 threads:
pthreads (link)
C++11 threads (link)
The BackgroundWorker Class
Some good example here:
BackgroundWorker Class Microsoft
BackgroundWorker Class CodeProject
Here I launch three threads using pthread. It might give you something to work from ... enjoy :
// g++ -o audio *.cpp ../common/*.cpp -std=c++11 -lm -lpthread
#include "cpp_openal_opengl_dna.h"
#include <thread>
#include <exception>
#include <mutex>
void launch_producer(Circular_Buffer * given_circular_buffer,
struct_sample_specs * ptr_struct_sample_specs, std::string chosen_file) {
}
void launch_mvc_visualization(Audio_Model * given_audio_model) {
}
void launch_audio_playback(Circular_Buffer * given_circular_buffer, Audio_Model * given_audio_model) {
}
int main() {
std::cout << "hello Corinde" << std::endl; // prints hello Corinde
// here we launch three threads
// thread t1 reads an input file to populate audio buffer
// notice the first parameter is the function above followed by its input parms
std::thread t1(launch_producer, circular_buffer, ptr_struct_sample_specs,
all_file_names[WHICH_FILE_INPUT]);
Audio_Model * audio_model = new Audio_Model(MAX_SIZE_CIRCULAR_BUFFER);
// thread t2 does real time OpenGL visualization of audio buffer data
std::thread t2(launch_mvc_visualization, audio_model); // OpenGL graphics visualization
// thread t3 renders the audio buffers as sound to your speakers
std::thread t3(launch_audio_playback, circular_buffer, audio_model);
// -------------------------
std::cout << "all three threads now launched" << std::endl;
t1.join();
t2.join();
t3.join();
std::cout << "processing is complete" << std::endl;
// ----------
return 0;
}
I think I have just solved using std::thread with method detach().
For doing so, I must re-open the file every time I have to load new sample data from it so I have now a global variable that stores the filename and call the function this way:
// The loading function that will be executed in a new thread
void continuePreload(unsigned long ByteSize)
{
// Re-open the file 'openFile'
FILE *fpFile = fopen(openFile, "rb");
// Set pointer in file at stample start + preload size
fseek(fpFile, WavePointers + preloadByteSize, SEEK_SET);
// Read the remaining bytes
fread(SampleData + preloadByteSize / sizeof(short), 1, ByteSize - preloadByteSize, fpFile);
// Close file
fclose(fpFile);
// Remember how many bytes we loaded
preloadByteSize = ByteSize;
}
Within the Play Event function...
// Get the size in bytes
const unsigned long ByteSize = SampleLib.smp.len * sizeof(short);
if (preloadByteSize < ByteSize)
{
std::thread loadSample(&myClass::continuePreload, this, ByteSize);
loadSample.detach();
}
The program is now acting exactly how I expected: whenever the play event arrives, it starts playing back audio from the sample buffer using what was previously preloaded, in the meantime a new thread finishes loading the remaining part of the file and fills the buffer completely.
As long as loading from disk is faster than the audio playback, we have no race conditions. In case loading is too slow, I can still increase the preload size, slowing down a bit the initial loading time.
Related
I have never worked with binary files before. I opened an .mp3 file using the mode ios::binary, read data from it, assigned 0 to each byte read and then rewrote them to another file opened in ios::binary mode. I opened the output file on a media player, it sounds corrupted but I can still hear the song. I want to know what happened physically.
How can I access/modify the raw data ( bytes ) of an audio ( video, images, ... ) using C++ ( to practice file encryption/decryption later )?
Here is my code:
#include <iostream>
#include <fstream>
#include <cstring>
using namespace std;
int main(){
char buffer[256];
ifstream inFile;
inFile.open("Backstreet Boys - Incomplete.mp3",ios::binary);
ofstream outFile;
outFile.open("Output.mp3",ios::binary);
while(!inFile.eof()){
inFile.read(buffer,256);
for(int i = 0; i<strlen(buffer); i++){
buffer[i] = 0;
}
outFile.write(buffer,256);
}
inFile.close();
outFile.close();
}
What you did has nothing to do with binary files or audio. You simply copied the file while zeroing some of the bytes. (The reason you didn't zero all of the bytes is because you use i<strlen(buffer), which simply counts up to the first zero byte rather than reporting the size of the buffer. Also you modify the buffer which means strlen(buffer) will report the length as zero after you zero the first byte.)
So the exact change in audio you get is entirely dependent on the mp3 file format and the audio compression it uses. MP3 is not an audio format that can be directly manipulated in useful ways.
If you want to manipulate digital audio, you need to learn about how raw audio is represented by computers.
It's actually not too difficult. For example, here's a program that writes out a raw audio file containing just a 400Hz tone.
#include <fstream>
#include <limits>
int main() {
const double pi = 3.1415926535;
double tone_frequency = 400.0;
int samples_per_second = 44100;
double output_duration_seconds = 5.0;
int output_sample_count =
static_cast<int>(output_duration_seconds * samples_per_second);
std::ofstream out("signed-16-bit_mono-channel_44.1kHz-sample-rate.raw",
std::ios::binary);
for (int sample_i = 0; sample_i < output_sample_count; ++sample_i) {
double t = sample_i / static_cast<double>(samples_per_second);
double sound_amplitude = std::sin(t * 2 * pi * tone_frequency);
// encode amplitude as a 16-bit, signed integral value
short sample_value =
static_cast<short>(sound_amplitude * std::numeric_limits<short>::max());
out.write(reinterpret_cast<char const *>(&sample_value),
sizeof sample_value);
}
}
To play the sound you need a program that can handle raw audio, such as Audacity. After running the program to generate the audio file, you can File > Import > Raw data..., to import the data for playing.
How can I access/modify the raw data ( bytes ) of an audio ( video, images, ... ) using C++ ( to practice file encryption/decryption later )?
As pointed out earlier, the reason your existing code is not completely zeroing out the data is because you are using an incorrect buffer size: strlen(buffer). The correct size is the number of bytes read() put into the buffer, which you can get with the function gcount():
inFile.read(buffer,256);
int buffer_size = inFile.gcount();
for(int i = 0; i < buffer_size; i++){
buffer[i] = 0;
}
outFile.write(buffer, buffer_size);
Note: if you were to step through your program using a debugger you probably would have pretty quickly seen the problem yourself when you noticed the inner loop executing less than you expected. Debuggers are a really handy tool to learn how to use.
I notice you're using open() and close() methods here. This is sort of pointless in this program. Just open the file in the constructor, and allow the file to be automatically closed when inFile and outFile go out of scope:
{
ifstream inFile("Backstreet Boys - Incomplete.mp3",ios::binary);
ofstream outFile("Output.mp3",ios::binary);
// don't bother calling .close(), it happens automatically.
}
I'm trying to receive a number from an Arduino as an integer in C++. The full code is below:
#define STRICT
#include <tchar.h>
#include <windows.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include "Serial.h"
#include <boost\lexical_cast.hpp>
enum { EOF_Char = 27 };
int __cdecl _tmain(int /*argc*/, char** /*argv*/)
{
CSerial serial;
LONG lLastError = ERROR_SUCCESS;
// Attempt to open the serial port (COM4)
lLastError = serial.Open(_T("COM4"), 0, 0, false);
// Setup the serial port (9600,8N1, which is the default setting)
lLastError = serial.Setup(CSerial::EBaud9600, CSerial::EData8, CSerial::EParNone, CSerial::EStop1);
// Register only for the receive event
lLastError = serial.SetMask(CSerial::EEventBreak |
CSerial::EEventCTS |
CSerial::EEventDSR |
CSerial::EEventError |
CSerial::EEventRing |
CSerial::EEventRLSD |
CSerial::EEventRecv);
// Use 'non-blocking' reads, because we don't know how many bytes
// will be received. This is normally the most convenient mode
// (and also the default mode for reading data).
lLastError = serial.SetupReadTimeouts(CSerial::EReadTimeoutNonblocking);
// Keep reading data, until an EOF (CTRL-Z) has been received
bool fContinue = true;
do
{
// Wait for an event
lLastError = serial.WaitEvent();
// Save event
const CSerial::EEvent eEvent = serial.GetEventType();
// Handle data receive event
if (eEvent & CSerial::EEventRecv)
{
// Read data, until there is nothing left
DWORD dwBytesRead = 0;
char szBuffer[101];
do
{
// Read data from the COM-port
lLastError = serial.Read(szBuffer, sizeof(szBuffer) - 1, &dwBytesRead);
if (dwBytesRead > 0)
{
// Finalize the data, so it is a valid string
szBuffer[dwBytesRead] = '\0';
// Display the data
printf("%s", szBuffer);
// Check if EOF (CTRL+'[') has been specified
if (strchr(szBuffer, EOF_Char))
fContinue = false;
}
} while (dwBytesRead == sizeof(szBuffer) - 1);
}
} while (fContinue);
// Close the port again
serial.Close();
return 0;
}
I have my Arduino constantly sending out the number 51. This code works fine and consistently displays "51". However, I want an int to manipulate in C++.
First I added
std::stringstream str(szBuffer);
int tester;
str >> tester;
printf("My number is: %d\n", tester+1);
right after
printf("%s", szBuffer);
A typical result looks like:
51My number is: 52
51My number is: 52
51My number is: 52
51My number is: 52
51My number is: 52
5My number is: 6
1My number is: 2
After doing it perfectly 5 or 6 times, the output always separates the incoming digits once or twice in a row (I haven't been able to find a specific pattern yet, but it's always 5-6 and 1-2).
My other attempt was to use the boost library:
int tester = boost::lexical_cast<int>(szBuffer);
printf("My number is: %d\n", tester);
right after
printf("%s", szBuffer);
and I get the same result (1-2 errors after 5-6 correct ones). I don't think the Arduino is sending bad data, since just a
printf("%s", szBuffer);
will never deviate from the number it's supposed to be. Could the conversion be messing up the receiving of data? Thanks.
EDIT: The Arduino code is:
void setup() {
Serial.begin(9600); // same as in your c++ script
}
void loop() {
Serial.print(51);
delay(1000);
}
With serial ports, there is no mechanism where a transmitter can inform a receiver how many bytes were transmitted as a block. I.e. there's no "hidden" marker where Serial.print(51); tells the receiver that it sent two characters as one number. You have to add some kind of indication (spaces, commas, line ends, initial byte counts, whatever) to your serial protocol.
Because of this, the number of characters you get from serial.Read depends on the number of characters you asked it to read (the second parameter) and how many characters are in the serial port's receive buffer, whichever is smaller. Most of the time, it seems the Arduino sends both digits before you call serial.Read, but sometimes it only gets one out in time... and the second is read the next time through the loop.
So let's assume you decided to use line ends to separate your numbers. All you have to do on the Arduino end is change to Serial.println(51);. The receive end is a little more complex.
I don't know what your serial library has in it. Most have some kind of "read line" function, and you would just replace the serial.Read call with something like:
serial.Readline(szBuffer, sizeof(szBuffer) - 1);
and it will take care of null-terminating the output. If it doesn't take care of null-termination, you'll need to find the line end and change it to a \0 yourself. From this point on, your code will work fine, because the serial.Readline function will block until it gets the whole line.
If you don't have a "read line" or at least a "read until this character" function, it's a bit harder. You have to repeatedly call serial.Read, moving through your buffer, until you see the line end character. Further, you run the risk of reading part or all of the next line, so you can't just discard all the data you read when you're done reading the number; you have to move teh data in the buffer so the next line's data (and further) is at the start of the buffer.
If you're using Boost (are you? it has no CSerial that I see), it looks like it has a read_until function. This takes three parameters: the stream you're reading from, a stream buffer to store the data in, and something to stop on. In this case, the stream buffer for storage is the one in your std::stringstream:
std::stringstream buffer;
size_t chars = boost::asio::read_until(serial, buffer.rdbuf(), '\n');
if(chars == 0) return;
int tester;
buffer >> tester;
printf("My number is: %d\n", tester+1);
I'm working on a project that will involve having to process PCM audio data through fft as its being played, preferably in sync. I'm using a linux g++ compiler and currently reading and playing audio data using OpenAL.
My question is this: is there a better way to process PCM audio data with an fft live as the audio is playing then using threads? If not, then what threading library would be best to use for these purposes.
this is my function that loads the wave data into an array of bytes, these can later be cast to ints for processing and all I use to play the data is OpenAL.
char* loadWAV(const char* fn, int& chan, int& samplerate, int& bps, int& size){
char buffer[4];
ifstream in(fn, ios::binary);
in.read(buffer, 4); //ChunkID "RIFF"
if(strncmp(buffer, "RIFF", 4) != 0){
cerr << "this is not a valid wave file";
return NULL;
}
in.read(buffer,4); //ChunkSize
in.read(buffer,4); //Format "WAVE"
in.read(buffer,4); // "fmt "
in.read(buffer,4); // 16
in.read(buffer,2); // 1
in.read(buffer,2); // NUMBER OF CHANNELS
chan = convertToInt(buffer,2);
in.read(buffer,4); // SAMPLE RATE
samplerate = convertToInt(buffer,4);
in.read(buffer,4); // ByteRate
in.read(buffer,2); // BlockAlign
in.read(buffer,2); // bits per sample
bps = convertToInt(buffer,2);
in.read(buffer,4); // "data"
in.read(buffer,4);
size = convertToInt(buffer,4);
char * data = new char[size];
in.read(data,size);
return data;
}
thank you for any and all help.
edit: to anyone who might be interested I wrote the function using this as a reference to know
how a WAV file is formated
Are you hoping to perform the FFT using OpenAL? I don't know if that's possible. Your code will likely be performing the FFT.
You don't need to explicitly set up any threads. However, your audio output library will probably do so on your behalf. I'm not familiar with OpenAL, but the way that a lot of audio libraries operate is by letting you specify a callback that will feed more audio into the output. Thus, your main program will load audio from the audio file, stuff it into a buffer (likely protected by a mutex) for the audio callback to read, compute an FFT over the audio window, and perhaps visualize the data for the user.
Again, the audio library will probably be managing the threading so you don't need to worry about the exact threading implementation or library. But be sure to manage shared data correctly with a mutex.
I have a roughly 11.1G binary file where stores a series of the depth frames from the Kinect. There are 19437 frames in this file. To read one frame per time, I use ifstream in fstream but it reaches eof before the real end of the file. (I only got the first 20 frames, and the function stops because of the eof flag)
However, all frames can be read by using fread in stdio instead.
Can anyone explain this situation? Thank you for precious time on my question.
Here are my two functions:
// ifstream.read() - Does Not Work: the loop will stop after 20th frame because of the eof flag
ifstream depthStream("fileName.dat");
if(depthStream.is_open())
{
while(!depthStream.eof())
{
char* buffer = new char[640*480*2];
depthStream.read(buffer, 640*480*2);
// Store the buffer data in OpenCV Mat
delete[] buffer;
}
}
// fread() - Work: Get 19437 frames successfully
FILE* depthStream
depthStream = fopen("fileName.dat", "rb");
if(depthStream != NULL)
{
while(!feof(depthStream))
{
char* buffer = new char[640*480*2];
fread(buffer, 1, 640*480*2, depthStream);
// Store the buffer data in OpenCV Mat
delete[] buffer;
}
Again, thank you for precious time on my question
You need to open the stream in binary mode, otherwise it will stop at the first byte it sees with a value of 26.
ifstream depthStream("fileName.dat", ios_base::in | ios_base::binary);
As for why 26 is special, it's the code for Ctrl-Z which was a convention used to mark the end of a text file. The history behind this was recorded in Raymond Chen's blog.
Hey all, I'm writing an application which records microphone input to a WAV file. Previously, I had written this to fill a buffer of a specified size and that worked fine. Now, I'd like to be able to record to an arbitrary length. Here's what I'm trying to do:
Set up 32 small audio buffers (circular buffering)
Start a WAV file with ofstream -- write the header with PCM length set to 0
Add a buffer to input
When a buffer completes, append its data to the WAV file and update the header; recycle the buffer
When the user hits "stop", write the remaining buffers to file and close
It kind of works in that the files are coming out to the correct length (header and file size and are correct). However, the data is wonky as hell. I can make out a semblance of what I said -- and the timing is correct -- but there's this repetitive block of distortion. It basically sounds like only half the data is getting into the file.
Here are some variables the code uses (in header)
// File writing
ofstream mFile;
WAVFILEHEADER mFileHeader;
int16_t * mPcmBuffer;
int32_t mPcmBufferPosition;
int32_t mPcmBufferSize;
uint32_t mPcmTotalSize;
bool mRecording;
Here is the code that prepares the file:
// Start recording audio
void CaptureApp::startRecording()
{
// Set flag
mRecording = true;
// Set size values
mPcmBufferPosition = 0;
mPcmTotalSize = 0;
// Open file for streaming
mFile.open("c:\my.wav", ios::binary|ios::trunc);
}
Here's the code that receives the buffer. This assumes the incoming data is correct -- it should be, but I haven't ruled out that it isn't.
// Append file buffer to output WAV
void CaptureApp::writeData()
{
// Update header with new PCM length
mPcmBufferPosition *= sizeof(int16_t);
mPcmTotalSize += mPcmBufferPosition;
mFileHeader.bytes = mPcmTotalSize + sizeof(WAVFILEHEADER);
mFileHeader.pcmbytes = mPcmTotalSize;
mFile.seekp(0);
mFile.write(reinterpret_cast<char *>(&mFileHeader), sizeof(mFileHeader));
// Append PCM data
if (mPcmBufferPosition > 0)
{
mFile.seekp(mPcmTotalSize - mPcmBufferPosition + sizeof(WAVFILEHEADER));
mFile.write(reinterpret_cast<char *>(&mPcmBuffer), mPcmBufferPosition);
}
// Reset file buffer position
mPcmBufferPosition = 0;
}
And this is the code that closes the file:
// Stop recording
void CaptureApp::stopRecording()
{
// Save remaining data
if (mPcmBufferSize > 0)
writeData();
// Close file
if (mFile.is_open())
{
mFile.flush();
mFile.close();
}
// Turn off recording flag
mRecording = false;
}
If there's anything here that looks like it would result in bad data getting appended to the file, please let me know. If not, I'll triple check the input data (in the callback). This data should be good, because it works if I copy it to a larger buffer (eg, two minutes) and then save that out.
I am just wondering, how
void CaptureApp::writeData()
{
mPcmBufferPosition *= sizeof(int16_t); // mPcmBufferPosition = 0, so 0*2 = 0;
// (...)
mPcmBufferPosition = 0;
}
works (btw. sizeof int16_t is always 2). Are you setting mPcmBufferPosition somewhere else?
void CaptureApp::writeData()
{
// Update header with new PCM length
long pos = mFile.tellp();
mPcmBufferBytesToWrite *= 2;
mPcmTotalSize += mPcmBufferBytesToWrite;
mFileHeader.bytes = mPcmTotalSize + sizeof(WAVFILEHEADER);
mFileHeader.pcmbytes = mPcmTotalSize;
mFile.seekp(0);
mFile.write(reinterpret_cast<char *>(&mFileHeader), sizeof(mFileHeader));
mFile.seekp(pos);
// Append PCM data
if (mPcmBufferBytesToWrite > 0)
mFile.write(reinterpret_cast<char *>(mPcmBuffer), mPcmBufferBytesToWrite);
}
Also mPcmBuffer is a pointer, so don't know why you use & in write.
The most likely reason is you're writing from the address of the pointer to your buffer, not from the buffer itself. Ditch the "&" in the final mFile.write. (It may have some good data in it if your buffer is allocated nearby and you happen to grab a chunk of it, but that's just luck that your write hapens to overlap your buffer)
In general, if you find yourself in this sort of situation, you could try to think how you can test this code in isolation from the recording code: Set up a buffer that has the values 0..255 in it, and then set your "chunk size" to 16 and see if it writes out a continuous sequence of 0..255 across 16 separate write operations. That will quickly verify if your buffering code is working or not.
I won't debug your code, but will try to give you checklist of the things you can try to check and determine where's the error:
always have referent recorder or player handy. It can be something as simple as Windows Sound Recorder, or Audacity, or Adobe Audition. Have a recorder/player that you are CERTAIN that will record and play files correctly.
record the file with your app and try to play it with reference player. Working?
try to record the file with reference recorder, and play it with your player. Working?
when you write SOUND data to the WAV file in your recorder, write it to one extra file. Open that file in RAW mode with the player (Windows Sound Recorder won't be enough here). Does it play correctly?
when playing the file in your player, and writing to the soundcard, write output to the RAW file, to see if you are playing the data correctly at all or you have soundcars issues. Does it play correctly?
Try all this, and you'll have much better idea of where something went wrong.
Shoot, sorry -- had a late night of work and am a bit off today. I forgot to show y'all the actual callback. This is it:
// Called when buffer is full
void CaptureApp::onData(float * data, int32_t & size)
{
// Check recording flag and buffer size
if (mRecording && size <= BUFFER_LENGTH)
{
// Save the PCM data to file and reset the array if we
// don't have room for this buffer
if (mPcmBufferPosition + size >= mPcmBufferSize)
writeData();
// Copy PCM data to file buffer
copy(mAudioInput.getData(), mAudioInput.getData() + size, mPcmBuffer + mPcmBufferPosition);
// Update PCM position
mPcmBufferPosition += size;
}
}
Will try y'alls advice and report.