SDL only making beeping noises instead of the actual audio file - c++

I recently managed to get past the errors of using SDL for sound.
Now that it's running and I'm not running into errors, my program is only playing beeping noises instead of the file I've provided.
I want the program to play the .wav file I'm passing to the SDL_LoadWAV.
I've tried with two different .wav files of different length and size, and checked the header files to find comments and tips on what format is required for the SDL to play the .wav file, haven't gotten anywhere with either of it.
The myAudioCallback function is responsible for handling the SDL callback.
void myAudioCallback(void* userdata, Uint8* stream, int len)
{
AudioData* audio = (AudioData*)userdata;
if (audio->length == 0)
return;
Uint32 length = (Uint32)len;
length = (length > audio->length ? audio->length : length); // if length is more than the audio length, then set length to be the audio.length, if not, set it to be the length passed to the function
std::cout << "Audio Length " << audio->length << std::endl;
std::cout << "Audio Position " << audio->position << std::endl;
SDL_memcpy(stream, audio->position, length); // audio callback is called by SDL, this ensures that the stream and data that is sent, is copied over to our struct, so we can use it and manipulate it
audio->position += length;
audio->length -= length;
}
My loadAudio function is responsible for loading the audio file and saving information about the audio file to the various variables I've declared in the .h (see further down for my .h)
void mainEngineCW4::loadAudio() // this function is for the sole purpose of loading the .wav file
{
SDL_Init(SDL_INIT_AUDIO); // loads the SDL to initialise audio
char* audioFile = "backgroundmusic.wav"; // a char pointer for the file path
// LoadWAV loads the wav file, and by putting it in an if statement like this, we can simutaneously check if the result is null, meaning an error occured while loading it.
if (SDL_LoadWAV(audioFile, &wavSpec, &wavStart, &wavLength) == NULL)
std::cerr << "Error: file could not be loaded as an audio file." << std::endl;
else
std::cout << audioFile << " loaded" << std::endl;
}
The playAudio function is responsible for loading the audio device and playing the audio through the Audio device
void mainEngineCW4::playAudio() // this function is for loading an audio device, and playing the audio through that device
{
audio.position = wavStart; // define where we start in the audio file
audio.length = wavLength; // define the length of the audio file
wavSpec.callback = myAudioCallback; // the callback variable needs a function that its going to run to be able to call back the audio that is played. assigning the function name to the variable allows it to call that function when needed
wavSpec.userdata = &audio; // the userdata is the audio itself
audioDevice = SDL_OpenAudioDevice(NULL, 0, &wavSpec, NULL, SDL_AUDIO_ALLOW_ANY_CHANGE); // opens the audio device, also having it play the audio information found at memory address for wavspec
if (audioDevice == 0) {
std::cerr << SDL_GetError() << std::endl;
return; }
SDL_PauseAudioDevice(audioDevice, 0); // mildly confused by why they decided to call the function for starting to play audio for "PauseAudioDevice" but yeah. this plays audio.
}
Here's my .h. I've defined myAudioCallback outside of the class, since SDL doesn't like the additional hidden parameter of a member function
struct AudioData
{
Uint8* position;
Uint32 length;
};
void myAudioCallback(void* userdata, Uint8* stream, int len);
class mainEngineCW4 :
public BaseEngine
{
public:
void loadAudio();
void playAudio();
void endAudio();
private:
// variables and pointers for audio information
AudioData audio;
SDL_AudioSpec wavSpec;
SDL_AudioDeviceID audioDevice;
Uint8* wavStart;
Uint32 wavLength;
};
I've removed the other functions and variables that are irrelevant to the issue I'm having
My problem is that I want my program to play the audio file I pass in, not just beeping noises.
Any help is greatly appreciated
EDIT: I realised I'm crap at providing info and explanation to things, so I edited in more information, explanation and the header file. If there is anything else I can provide, please let me know

With the help of a friend, I managed to fix the issue.
It seems like SDL didn't like it when I passed
SDL_OpenAudioDevice(NULL, 0, &wavSpec, NULL, SDL_AUDIO_ALLOW_ANY_CHANGE);
so instead, I passed
SDL_OpenAudioDevice(NULL, 0, &wavSpec, NULL, 0);
instead. This made the file play nicely, as well as having additional variables for the length and position of the audio file.
Another issue I ran into even after I got the file playing, was that the beeping was still playing with the audio file. I wasn't able to fix this directly myself, instead, when I cleaned the solution the day after, the beeping was gone, and the only thing playing was the audio file.
I've attached the code that works below.
In addition to the struct I created in the .h
struct AudioData
{
Uint8* position;
Uint32 length;
};
Defining the audio_position and audio_length as a global variable also helped in copying over information in the audio callback function.
static Uint8* audio_position;
static Uint32 audio_length;
void myAudioCallback(void* userdata, Uint8* stream, int len)
{
if (audio_length == 0)
return;
len = (len > audio_length ? audio_length : len); // if length is more than the audio length, then set length to be the audio.length, if not, set it to be the length passed to the function
SDL_memcpy(stream, audio_position, len); // audio callback is called by SDL, this ensures that the stream and data that is sent, is copied over to our struct, so we can use it and manipulate it
audio_position += len;
audio_length -= len;
}
For the load audio, I made sure that I actually load all the information that would be considered "loading", including storing the AudioSpec callback function, and setting the length and position of the audio file.
void mainEngineCW4::loadAudio() // this function is for the sole purpose of loading the .wav file
{
if (SDL_Init(SDL_INIT_AUDIO) < 0 || audioPlaying == true) // loads the SDL to initialise audio
return;
char* filePath = "backgroundmusic.wav"; // a char pointer for the file path
// LoadWAV loads the wav file, and by putting it in an if statement like this, we can simutaneously check if the result is null, meaning an error occured while loading it.
if (SDL_LoadWAV(filePath, &desiredSpec, &wavStart, &wavLength) == NULL)
std::cerr << "Error: file could not be loaded as an audio file." << std::endl;
else
std::cout << filePath << " loaded" << std::endl;
desiredSpec.callback = myAudioCallback; // the callback variable needs a function that its going to run to be able to call back the audio that is played. assigning the function name to the variable allows it to call that function when needed
desiredSpec.userdata = &audioInfo; // the userdata is the audio itself
audio_position = wavStart; // define where we start in the audio file
audio_length = wavLength; // define the length of the audio file
}
I also added a boolean to the class, so that when this returns true, it means that the audio has already been playing or has already been loaded, as to ensure SDL won't play the same thing simultaneously.
void mainEngineCW4::playAudio() // this function is for loading an audio device, and playing the audio through that device
{
if (audioPlaying == true)
return;
audioDevice = SDL_OpenAudioDevice(NULL, 0, &desiredSpec, NULL, 0); // opens the audio device, also having it play the audio information found at memory address for wavspec
if (audioDevice == 0)
{
std::cerr << SDL_GetError() << std::endl;
return;
}
SDL_PauseAudioDevice(audioDevice, 0); // mildly confused by why they decided to call the function for starting to play audio for "PauseAudioDevice" but yeah. this plays audio.
audioPlaying = true;
}
void mainEngineCW4::endAudio()
{
SDL_CloseAudioDevice(audioDevice);
SDL_FreeWAV(wavStart);
audioPlaying = false;
}

Related

How can I perform a realtime FFT on SDL2 audio stream data

I am trying to create a music visualiser in C++ using SDL2 and FFTW3.
My aim is to load a .wav audio file and then simultaneously play the audio and perform a realtime Fast Fourier Transform using an SDL2 Callback function.
I want to get the frequency spectrum data so that I can implement the graphical visualiser at a later date.
I followed an SDL YouTube guide on loading the .wav and playing the audio using the callback function, but I don't understand how to perform the FFT on this data. I followed yet another guide on using FFTW and SDL with C to produce a similar effect but I'm still unsure how to actually implement it.
Uint8* sampData;
SDL_AudioSpec wavSpec;
Uint8* wavStart;
Uint32 wavLength;
SDL_AudioDeviceID aDevice;
struct AudioData {
Uint8* filePosition;
Uint32 fileLength;
};
void PlayAudioCallback(void* userData, Uint8* stream, int streamLength) {
AudioData* audio = (AudioData*)userData;
sampData = new Uint8;
if (audio->fileLength == 0) {
return;
}
Uint32 length = (Uint32)streamLength;
length = (length > audio->fileLength ? audio->fileLength : length);
SDL_memcpy(stream, audio->filePosition, length);
// HERE is where i'd like to implement the FFT on 'stream' data
// but i don't know how to implement this using FFTW
audio->filePosition += length;
audio->fileLength -= length;
}
int main() {
SDL_Init(SDL_INIT_AUDIO);
// Load .wav file
if (SDL_LoadWAV(FILE_PATH, &wavSpec, &wavStart, &wavLength) == NULL) {
cerr << "Couldnt load file: " << FILE_PATH << endl;
getchar();
}
cout << "Loaded " << FILE_PATH << endl;
AudioData audio;
audio.filePosition = wavStart;
audio.fileLength = wavLength;
wavSpec.callback = PlayAudioCallback;
wavSpec.userdata = &audio;
// Open audio playback endpoint
aDevice = SDL_OpenAudioDevice(NULL, 0, &wavSpec, NULL, SDL_AUDIO_ALLOW_ANY_CHANGE);
if (aDevice == 0) {
cerr << "Audio Device connection failed: " << SDL_GetError() << endl;
getchar();
}
// Play audio on playback endpoint
SDL_PauseAudioDevice(aDevice, 0);
// Do nothing while there's still data to be played
while (audio.fileLength > 0) {
SDL_Delay(100);
}
}
From previous experience I used NumPy to unpack .wav data into a NumPy array, before sending it the built-in NumPy-FFT function, but I'm clueless on what to do with the SDL stream data that I have here.
What you want is the short term FFT. You collect a buffer of samples from your stream and apply a window function to the samples before performing the FFT. You then collect another buffer, keeping some samples from the first buffer and appending new samples. Repeat until all the data has been processed.
Since your input data is real, the FFT is symmetric, hence you only want the first N/2+1 complex output bins. These represent frequencies from d.c. to Fs/2. Take their magnitudes and plot. Repeat for each FFT.

C++/C FFmpeg artifact build up across video frames

Context:
I am building a recorder for capturing video and audio in separate threads (using Boost thread groups) using FFmpeg 2.8.6 on Ubuntu 16.04. I followed the demuxing_decoding example here: https://www.ffmpeg.org/doxygen/2.8/demuxing_decoding_8c-example.html
Video capture specifics:
I am reading H264 off a Logitech C920 webcam and writing the video to a raw file. The issue I notice with the video is that there seems to be a build-up of artifacts across frames until a particular frame resets. Here is my frame grabbing, and decoding functions:
// Used for injecting decoding functions for different media types, allowing
// for a generic decode loop
typedef std::function<int(AVPacket*, int*, int)> PacketDecoder;
/**
* Decodes a video packet.
* If the decoding operation is successful, returns the number of bytes decoded,
* else returns the result of the decoding process from ffmpeg
*/
int decode_video_packet(AVPacket *packet,
int *got_frame,
int cached){
int ret = 0;
int decoded = packet->size;
*got_frame = 0;
//Decode video frame
ret = avcodec_decode_video2(video_decode_context,
video_frame, got_frame, packet);
if (ret < 0) {
//FFmpeg users should use av_err2str
char errbuf[128];
av_strerror(ret, errbuf, sizeof(errbuf));
std::cerr << "Error decoding video frame " << errbuf << std::endl;
decoded = ret;
} else {
if (*got_frame) {
video_frame->pts = av_frame_get_best_effort_timestamp(video_frame);
//Write to log file
AVRational *time_base = &video_decode_context->time_base;
log_frame(video_frame, time_base,
video_frame->coded_picture_number, video_log_stream);
#if( DEBUG )
std::cout << "Video frame " << ( cached ? "(cached)" : "" )
<< " coded:" << video_frame->coded_picture_number
<< " pts:" << pts << std::endl;
#endif
/*Copy decoded frame to destination buffer:
*This is required since rawvideo expects non aligned data*/
av_image_copy(video_dest_attr.video_destination_data,
video_dest_attr.video_destination_linesize,
(const uint8_t **)(video_frame->data),
video_frame->linesize,
video_decode_context->pix_fmt,
video_decode_context->width,
video_decode_context->height);
//Write to rawvideo file
fwrite(video_dest_attr.video_destination_data[0],
1,
video_dest_attr.video_destination_bufsize,
video_out_file);
//Unref the refcounted frame
av_frame_unref(video_frame);
}
}
return decoded;
}
/**
* Grabs frames in a loop and decodes them using the specified decoding function
*/
int process_frames(AVFormatContext *context,
PacketDecoder packet_decoder) {
int ret = 0;
int got_frame;
AVPacket packet;
//Initialize packet, set data to NULL, let the demuxer fill it
av_init_packet(&packet);
packet.data = NULL;
packet.size = 0;
// read frames from the file
for (;;) {
ret = av_read_frame(context, &packet);
if (ret < 0) {
if (ret == AVERROR(EAGAIN)) {
continue;
} else {
break;
}
}
//Convert timing fields to the decoder timebase
unsigned int stream_index = packet.stream_index;
av_packet_rescale_ts(&packet,
context->streams[stream_index]->time_base,
context->streams[stream_index]->codec->time_base);
AVPacket orig_packet = packet;
do {
ret = packet_decoder(&packet, &got_frame, 0);
if (ret < 0) {
break;
}
packet.data += ret;
packet.size -= ret;
} while (packet.size > 0);
av_free_packet(&orig_packet);
if(stop_recording == true) {
break;
}
}
//Flush cached frames
std::cout << "Flushing frames" << std::endl;
packet.data = NULL;
packet.size = 0;
do {
packet_decoder(&packet, &got_frame, 1);
} while (got_frame);
av_log(0, AV_LOG_INFO, "Done processing frames\n");
return ret;
}
Questions:
How do I go about debugging the underlying issue?
Is it possible that running the decoding code in a thread other than the one in which the decoding context was opened is causing the problem?
Am I doing something wrong in the decoding code?
Things I have tried/found:
I found this thread that is about the same problem here: FFMPEG decoding artifacts between keyframes
(I cannot post samples of my corrupted frames due to privacy issues, but the image linked to in that question depicts the same issue I have)
However, the answer to the question is posted by the OP without specific details about how the issue was fixed. The OP only mentions that he wasn't 'preserving the packets correctly', but nothing about what was wrong or how to fix it. I do not have enough reputation to post a comment seeking clarification.
I was initially passing the packet into the decoding function by value, but switched to passing by pointer on the off chance that the packet freeing was being done incorrectly.
I found another question about debugging decoding issues, but couldn't find anything conclusive: How is video decoding corruption debugged?
I'd appreciate any insight. Thanks a lot!
[EDIT] In response to Ronald's answer, I am adding a little more information that wouldn't fit in a comment:
I am only calling decode_video_packet() from the thread processing video frames; the other thread processing audio frames calls a similar decode_audio_packet() function. So only one thread calls the function. I should mention that I have set the thread_count in the decoding context to 1, failing which I would get a segfault in malloc.c while flushing the cached frames.
I can see this being a problem if the process_frames and the frame decoder function were run on separate threads, which is not the case. Is there a specific reason why it would matter if the freeing is done within the function, or after it returns? I believe the freeing function is passed a copy of the original packet because multiple decode calls would be required for audio packet in case the decoder doesnt decode the entire audio packet.
A general problem is that the corruption does not occur all the time. I can debug better if it is deterministic. Otherwise, I can't even say if a solution works or not.
A few things to check:
are you running multiple threads that are calling decode_video_packet()? If you are: don't do that! FFmpeg has built-in support for multi-threaded decoding, and you should let FFmpeg do threading internally and transparently.
you are calling av_free_packet() right after calling the frame decoder function, but at that point it may not yet have had a chance to copy the contents. You should probably let decode_video_packet() free the packet instead, after calling avcodec_decode_video2().
General debugging advice:
run it without any threading and see if that works;
if it does, and with threading it fails, use thread debuggers such as tsan or helgrind to help in finding race conditions that point to your code.
it can also help to know whether the output you're getting is reproduceable (this suggests a non-threading-related bug in your code) or changes from one run to the other (this suggests a race condition in your code).
And yes, the periodic clean-ups are because of keyframes.

SDL2 & SMPEG2 - Empty sound buffer trying to read a MP3

I'm trying to load a MP3 in a buffer using the SMPEG2 library, which comes with the SDL2. Every SMPEG function calls returns without error, but when I'm done, the sound buffer is full of zeros.
Here's the code :
bool LoadMP3(char* filename)
{
bool success = false;
const Uint32 Mp3ChunkLen = 4096;
SMPEG* mp3;
SMPEG_Info infoMP3;
Uint8 * ChunkBuffer;
Uint32 MP3Length = 0;
// Allocate a chunk buffer
ChunkBuffer = (Uint8*)malloc(Mp3ChunkLen);
SDL_RWops *mp3File = SDL_RWFromFile(filename, "rb");
if (mp3File != NULL)
{
mp3 = SMPEG_new_rwops(mp3File, &infoMP3, 1, 0);
if(mp3 != NULL)
{
if(infoMP3.has_audio)
{
Uint32 readLen;
// Inform the MP3 of the output audio specifications
SMPEG_actualSpec(mp3, &asDeviceSpecs); // static SDL_AudioSpec asDeviceSpecs; containing valid values after a call to SDL_OpenAudioDevice
// Enable the audio and disable the video.
SMPEG_enableaudio(mp3, 1);
SMPEG_enablevideo(mp3, 0);
// Play the MP3 once to get the size of the needed finale buffer
SMPEG_play(mp3);
while ((readLen = SMPEG_playAudio(mp3, ChunkBuffer, Mp3ChunkLen)) > 0)
{
MP3Length += readLen;
}
SMPEG_stop(mp3);
if(MP3Length > 0)
{
// Reallocate the buffer with the new length (if needed)
if (MP3Length != Mp3ChunkLen)
{
ChunkBuffer = (Uint8*)realloc(ChunkBuffer, MP3Length);
}
// Replay the entire MP3 into the new ChunkBuffer.
SMPEG_rewind(mp3);
SMPEG_play(mp3);
bool readBackSuccess = (MP3Length == SMPEG_playAudio(mp3, ChunkBuffer, MP3Length));
SMPEG_stop(mp3);
if(readBackSuccess)
{
// !!! Here, ChunkBuffer contains only zeros !!!
success = true;
}
}
}
SMPEG_delete(mp3);
mp3 = NULL;
}
SDL_RWclose(mp3File);
mp3File = NULL;
}
free(ChunkBuffer);
return success;
}
The code's widely based on SDL_Mixer, which I cannot use for my projet, based on its limitations.
I know Ogg Vorbis would be a better choice of file format, but I'm porting a very old project, and it worked entirely with MP3s.
I'm sure the sound system is initialized correctly because I can play WAV files just fine. It's intialized with a frequency of 44100, 2 channels, 1024 samples, and the AUDIO_S16SYS format (the latter which is, as I understood from the SMPEG source, mandatory).
I've calculated the anticipated buffer size, based on the bitrate, the amount of data in the MP3 and the OpenAudioDevice audio specs, and everything is consistent.
I cannot figure why everything but the buffer data seems to be working.
UPDATE #1
Still trying to figure out what's wrong, I thought the support for MP3 might not be working, so I created the following function :
SMPEG *mpeg;
SMPEG_Info info;
mpeg = SMPEG_new(filename,&info, 1);
SMPEG_play(mpeg);
do { SDL_Delay(50); } while(SMPEG_status(mpeg) == SMPEG_PLAYING);
SMPEG_delete(mpeg);
The MP3 played. So, the decoding should actually be working. But that's not what I need ; I really need the sound buffer data so I can send it to my mixer.
After much tinkering, research and digging through the SMPEG source code, I realized that I had to pass 1 as the SDLAudio parameter to SMPEG_new_rwops function.
The comment found in smpeg.h is misleading :
The sdl_audio parameter indicates if SMPEG should initialize the SDL audio subsystem. If not, you will have to use the SMPEG_playaudio() function below to extract the decoded data.
Since the audio subsystem was already initialized and I was using the SMPEG_playaudio() function, I had no reason to think I needed this parameter to be non-zero. In SMPEG, this parameter triggers the audio decompression at opening time, but even though I called SMPEG_enableaudio(mp3, 1); the data is never reparsed. This might be a bug/a shady feature.
I had another problem with the freesrc parameter which needed to be 0, since I freed the SDL_RWops object myself.
For future reference, once ChunkBuffer has the MP3 data, it needs to pass through SDL_BuildAudioCVT/SDL_ConvertAudio if it's to be played through an already opened audio device.
The final working code is :
// bool ReadMP3ToBuffer(char* filename)
bool success = false;
const Uint32 Mp3ChunkLen = 4096;
SDL_AudioSpec mp3Specs;
SMPEG* mp3;
SMPEG_Info infoMP3;
Uint8 * ChunkBuffer;
Uint32 MP3Length = 0;
// Allocate a chunk buffer
ChunkBuffer = (Uint8*)malloc(Mp3ChunkLen);
memset(ChunkBuffer, 0, Mp3ChunkLen);
SDL_RWops *mp3File = SDL_RWFromFile(filename, "rb"); // filename is a char* passed to the function.
if (mp3File != NULL)
{
mp3 = SMPEG_new_rwops(mp3File, &infoMP3, 0, 1);
if(mp3 != NULL)
{
if(infoMP3.has_audio)
{
Uint32 readLen;
// Get the MP3 audio specs for later conversion
SMPEG_wantedSpec(mp3, &mp3Specs);
SMPEG_enablevideo(mp3, 0);
// Play the MP3 once to get the size of the needed buffer in relation with the audio specs
SMPEG_play(mp3);
while ((readLen = SMPEG_playAudio(mp3, ChunkBuffer, Mp3ChunkLen)) > 0)
{
MP3Length += readLen;
}
SMPEG_stop(mp3);
if(MP3Length > 0)
{
// Reallocate the buffer with the new length (if needed)
if (MP3Length != Mp3ChunkLen)
{
ChunkBuffer = (Uint8*)realloc(ChunkBuffer, MP3Length);
memset(ChunkBuffer, 0, MP3Length);
}
// Replay the entire MP3 into the new ChunkBuffer.
SMPEG_rewind(mp3);
SMPEG_play(mp3);
bool readBackSuccess = (MP3Length == SMPEG_playAudio(mp3, ChunkBuffer, MP3Length));
SMPEG_stop(mp3);
if(readBackSuccess)
{
SDL_AudioCVT convertedSound;
// NOTE : static SDL_AudioSpec asDeviceSpecs; containing valid values after a call to SDL_OpenAudioDevice
if(SDL_BuildAudioCVT(&convertedSound, mp3Specs.format, mp3Specs.channels, mp3Specs.freq, asDeviceSpecs.format, asDeviceSpecs.channels, asDeviceSpecs.freq) >= 0)
{
Uint32 newBufferLen = MP3Length*convertedSound.len_mult;
// Make sure the audio length is a multiple of a sample size to avoid sound clicking
int sampleSize = ((asDeviceSpecs.format & 0xFF)/8)*asDeviceSpecs.channels;
newBufferLen &= ~(sampleSize-1);
// Allocate the new buffer and proceed with the actual conversion.
convertedSound.buf = (Uint8*)malloc(newBufferLen);
memcpy(convertedSound.buf, ChunkBuffer, MP3Length);
convertedSound.len = MP3Length;
if(SDL_ConvertAudio(&convertedSound) == 0)
{
// Save convertedSound.buf and convertedSound.len_cvt for future use in your mixer code.
// Dont forget to free convertedSound.buf once it's not used anymore.
success = true;
}
}
}
}
}
SMPEG_delete(mp3);
mp3 = NULL;
}
SDL_RWclose(mp3File);
mp3File = NULL;
}
free(ChunkBuffer);
return success;
NOTE : Some MP3 files I tried lost a few milliseconds and cutoff too early during playback when I resampled them with this code. Some others didn't. I could reproduce the same behaviour in Audacity, so I'm not sure what's going on. There may still have a bug with my code, a bug in SMPEG, or it maybe a known issue with the MP3 format itself. If someone can provide and explanation in the comments, that would be great!

(FFMPEG) avformat_write_header crashes (MSVC2013) (C++) (Qt)

I just downloaded FFMPEG and now I'm trying to use it in Qt with MSVC2013 compiler.
To understand how it works, I started reading the documentation and the API.
According to this figure, I was trying to make a little test with libavformat.
I did all they said in the demuxing module, then the muxing module. But, my program crashes when I call the avformat_write_header() function.
I would like to know what I did wrong and if you could help me to understand that.
In the main:
av_register_all();
if(!decode())
return;
The decode() methode:
bool MainWindow::decode()
{
AVFormatContext *formatContext = NULL;
AVPacket packet;
/**************** muxing varaiables ******************/
AVFormatContext *muxingContext = avformat_alloc_context();
AVOutputFormat *outputFormat = NULL;
AVIOContext *contextIO = NULL;
AVCodec *codecEncode = avcodec_find_encoder(AV_CODEC_ID_WMAV2);
AVStream *avStream = NULL;
AVCodecContext *codecContext = NULL;
/******************* demuxing **************************/
//open a media file
if(avformat_open_input(&formatContext,"h.mp3",NULL,NULL)!=0)
{
qDebug() << "paka ouve fichier";
return false;
}
//function which tries to read and decode a few frames to find missing
information.
if(avformat_find_stream_info(formatContext,NULL)<0)
{
qDebug()<<"paka find stream";
return false;
}
/**************** muxing *************************/
//The oformat field must be set to select the muxer that will be used.
muxingContext->oformat = outputFormat;
//Unless the format is of the AVFMT_NOFILE type, the pb field must be set to
//an opened IO context, either returned from avio_open2() or a custom one.
if(avio_open2(&contextIO,"out.wma",AVIO_FLAG_WRITE,NULL,NULL)<0)
{
qDebug() <<"paka kreye fichier soti";
return false;
}
muxingContext->pb = contextIO;
//Unless the format is of the AVFMT_NOSTREAMS type, at least
//one stream must be created with the avformat_new_stream() function.
avStream = avformat_new_stream(muxingContext,codecEncode);
//The caller should fill the stream codec context information,
//such as the codec type, id and other parameters
//(e.g. width / height, the pixel or sample format, etc.) as known
codecContext = avStream->codec;
codecContext->codec_type = AVMEDIA_TYPE_AUDIO;
codecContext->codec_id = AV_CODEC_ID_WMAV2;
codecContext->sample_fmt = codecEncode->sample_fmts[0];
codecContext->bit_rate = 128000;
codecContext->sample_rate = 44000;
codecContext->channels = 2;
//The stream timebase should be set to the timebase that the caller desires
//to use for this stream (note that the timebase actually used by the muxer
//can be different, as will be described later).
avStream->time_base = formatContext->streams[0]->time_base;
qDebug()<<formatContext->streams[0]->time_base.num <<"/"
<<formatContext- >streams[0]->time_base.den;
//When the muxing context is fully set up, the caller must call
//avformat_write_header()
//to initialize the muxer internals and write the file header
qDebug() << "does not crash yet";
if(avformat_write_header(muxingContext,NULL) <0)
{
qDebug()<<"cannot write header";
return false;
}
qDebug() << "OOps you can't see me (John Cena)";
///////////////////// Reading from an opened file //////////////////////////
while(av_read_frame(formatContext,&packet)==0)
{
//The data is then sent to the muxer by repeatedly calling
//av_write_frame() or av_interleaved_write_frame()
if(av_write_frame(muxingContext,&packet)<0)
qDebug()<<"paka write frame";
else
qDebug()<<"writing";
}
//Once all the data has been written, the caller must call
//av_write_trailer() to flush any buffered packets and finalize
//the output file, then close the IO context (if any) and finally
//free the muxing context with avformat_free_context().
if(av_write_trailer(muxingContext)!=0)
{
qDebug()<<"paka ekri trailer";
return false;
}
return true;
}
The program shows the message does not crash yet. But not OOps you can't see me (John Cena)
And there is no error. I used an MP3 file as input and I would like to ouput it in WMA.
Instead of avformat_alloc_context(), use avformat_alloc_output_context2(). This will set muxingContext->oformat.

How to read YUV8 data from avi file?

I have avi file that contains uncompressed gray video data. I need to extract frames from it. The size of file is 22 Gb.
How do i do that?
I have already tried ffmpeg, but it gives me "could not find codec parameters for video stream" message - because there is no codec at work, just frames.
Since Opencv just uses ffmpeg to read video, that rules out opencv as well.
The only path that seems to be left is to try and dig into the raw data, but i do not know how.
Edit: this is the code i use to read from the file with opencv. The failure occurs inside the second if. Running ffmpeg binary on the file also fails with the message above (could not find codec aprameters etc)
/* register all formats and codecs */
av_register_all();
/* open input file, and allocate format context */
if (avformat_open_input(&fmt_ctx, src_filename, NULL, NULL) < 0) {
fprintf(stderr, "Could not open source file %s\n", src_filename);
ret = 1;
goto end;
}
fmt_ctx->seek2any = true;
/* retrieve stream information */
int res = avformat_find_stream_info(fmt_ctx, NULL);
if (res < 0) {
fprintf(stderr, "Could not find stream information\n");
ret = 1;
goto end;
}
Edit:
Here is sample code i have tried to make the extraction: pastebin. The result i get is an unchanging buffer after every call to AVIStreamRead.
If you do not need cross platform functionality Video for Windows (VFW) API is a good alternative (http://msdn.microsoft.com/en-us/library/windows/desktop/dd756808(v=vs.85).aspx), i will not put an entire code block, since there's quite much to do, but you should be able to figure it out from the reference link. Basically, you do a AVIFileOpen, then get the video stream via AVIFileGetStream with streamtypeVIDEO, or alternatively do it at once with AVIStreamOpenFromFile and then read samples from the stream with AVIStreamRead. If you get to a point where you fail I can try to help, but it should be pretty straightforward.
Also, not sure why ffmpeg is failing, I have been doing raw AVI reading with ffmpeg without any codecs involved, can you post what call to ffpeg actually fails?
EDIT:
For the issue that you are seeing when the read data size is 0. The AVI file has N slots for frames in each second where N is the fps of the video. In real life the samples won't come exactly at that speed (e.g. IP surveillance cameras) so the actual data sample indexes can be non continuous like 1,5,11,... and VFW would insert empty samples between them (that is from where you read a sample with a zero size). What you have to do is call AVIStreamRead with NULL as buffer and 0 as size until the bRead is not 0 or you run past last sample. When you get an actual size, then you can again call AVIStreamRead on that sample index with the buffer pointer and size. I usually do compressed video so i don't use the suggested size, but at least according to your code snipplet I would do something like this:
...
bRead = 0;
do
{
aviOpRes = AVIStreamRead(ppavi,smpS,1,NULL,0,&bRead,&smpN);
} while (bRead == 0 && ++smpS < si.dwLength + si.dwStart);
if(smpS >= si.dwLength + si.dwStart)
break;
PUCHAR tempBuffer = new UCHAR[bRead];
aviOpRes = AVIStreamRead(ppavi,smpS,1,tempBuffer,bRead,&bRead,&smpN);
/* do whatever you need */
delete tempBuffer;
...
EDIT 2:
Since this may come in handy to someone or yourself to make a choice between VFW and FFMPEG I also updated your FFMPEG example so that it parsed the same file (sorry for the code quality since it lacks error checking but i guess you can see the logical flow):
/* register all formats and codecs */
av_register_all();
AVFormatContext* fmt_ctx = NULL;
/* open input file, and allocate format context */
const char *src_filename = "E:\\Output.avi";
if (avformat_open_input(&fmt_ctx, src_filename, NULL, NULL) < 0) {
fprintf(stderr, "Could not open source file %s\n", src_filename);
abort();
}
/* retrieve stream information */
int res = avformat_find_stream_info(fmt_ctx, NULL);
if (res < 0) {
fprintf(stderr, "Could not find stream information\n");
abort();
}
int video_stream_index = 0; /* video stream is usualy 0 but still better to lookup in case it's not present */
for(; video_stream_index < fmt_ctx->nb_streams; ++video_stream_index)
{
if(fmt_ctx->streams[video_stream_index]->codec->codec_type == AVMEDIA_TYPE_VIDEO)
break;
}
if(video_stream_index == fmt_ctx->nb_streams)
abort();
AVPacket *packet = new AVPacket;
while(av_read_frame(fmt_ctx, packet) == 0)
{
if (packet->stream_index == video_stream_index)
printf("Sample nr %d\n", packet->pts);
av_free_packet(packet);
}
Basically you open the context and read packets from it. You will get both audio and video packets so you should check if the packet belongs to the stream of interest. FFMPEG will save you the trouble with empty frames and give only those samples that have data in them.