I am developing a gstreamer plugin similar to v4l2src that captures live video from our camera and display using gstreamer.
My plugin is able to sucesfully generate a yuv file.
Now I need to copy camera image buffer to gstreamer buffer and push it to the next element. For this I used the below method:
GstBuffer *outbuf1
outbuf1 = gst_buffer_new_allocate(NULL,*buflen,NULL);
ret_size = gst_buffer_fill(outbuf1,pImgBuf->pBuffer[0],pImgBuf->iFilledLen[0]);
ret = gst_pad_push(Video->gst_filter->srcpad,outbuf1);
Here I get the error code GST_FLOW_FLUSHING
Further debugging this I also generated one yuv file from the data in image buffer before copying image buffer to gstreamer buffer:
fwrite( pImgBuf->pBuffer[0], pImgBuf->iFilledLen[0], 1, Video->File );
This yuv file is sucesfully generated and played using vooya.
Now when I try to generate the yuv file using the gstreamer buffer yuv file is not properly generated as it is not played on vooya and gives the error message "no good data"
Here is my code flow for more details:
static void gst_imx219src_init (Gstimx219src * filter)
{
printf("================================Inside gst_imx219src_init============== \n");
filter->sinkpad = gst_pad_new_from_static_template (&sink_factory, "sink");
gst_pad_set_event_function (filter->sinkpad,
GST_DEBUG_FUNCPTR(gst_imx219src_sink_event));
-----------------code for camera init and recording -------------------
ret = start_processing_on_event_dequeue( hHandle, filter );
start_processing_on_event_dequeue : This function will start the recording process in camera.This function internally calls the below function:
void Video_DeliverInput( void *hVideo, CAM_ImageBuffer *pImgBuf , CAM_StreamID strmId)
{
VideoChain *Video = (VideoChain*)Video;
CAM_Error error = CAM_ERROR_NONE;
int retCode = *buflen;
GstBuffer *outbuf1,*outbuf2,*outbuf3;
GstFlowReturn ret;
buflen = pImgBuf->iFilledLen;
outbuf1 = gst_buffer_new_allocate(NULL,*buflen,NULL);
outbuf2 = gst_buffer_new_allocate(NULL,*buflen,NULL);
outbuf3 = gst_buffer_new_allocate(NULL,*buflen,NULL);
gst_buffer_memset(outbuf1, 0, '\0', *buflen);
gst_buffer_memset(outbuf2, 0, '\0', *buflen);
gst_buffer_memset(outbuf3, 0, '\0', *buflen);
if ( Video->iEncoderType == 0 ) // yuv
{
if ( Video->File != NULL )
{
fwrite( pImgBuf->pBuffer[0], pImgBuf->iFilledLen[0], 1, Video->File );
fwrite( pImgBuf->pBuffer[1], pImgBuf->iFilledLen[1], 1, Video->File );
fwrite( pImgBuf->pBuffer[2], pImgBuf->iFilledLen[2], 1, Video->File );
}
}
if(outbuf1)
{
ret_size = gst_buffer_fill (outbuf1, 0, pImgBuf->pBuffer[0],pImgBuf->iFilledLen[0]);
ret_size = gst_buffer_fill (outbuf2, 0, pImgBuf->pBuffer[1],pImgBuf->iFilledLen[1]);
ret_size = gst_buffer_fill (outbuf3, 0, pImgBuf->pBuffer[2],pImgBuf->iFilledLen[2]);
}
else
{
printf("===========failed to create buffer \n");
}
gst_pad_set_active (Video->gst_filter->srcpad,TRUE);
ret = gst_pad_is_active (Video->gst_filter->srcpad);
ret = gst_pad_push(Video->gst_filter->srcpad,outbuf1);
#if defined( PLATFORM_PROCESSOR_ULC )
if(strmId == CAM_STREAM1)
error = CAM_PortEnqueueBuffer( hVideo->hCameraEngine, CAM_PORT_PREVIEW, pImgBuf );
else
error = CAM_PortEnqueueBuffer( hVideo->hCameraEngine, CAM_PORT_STILL, pImgBuf );
ASSERT_CAM_ERROR( error );
#else
error = CAM_PortEnqueueBuffer( hVideo->hCameraEngine, CAM_PORT_VIDEO, pImgBuf );
ASSERT_CAM_ERROR( error );
#endif
//}
return;
}
filter->srcpad = gst_pad_new_from_static_template (&src_factory, "src");
ret = start_processing_on_event_dequeue( hHandle, filter );
Please can anyone help me with this?
Thanks for the help in advance.
Related
I seem to be having a bit of trouble using FFMPEG audio filters in C++ code. If I only have "abuffer" and "abuffersink" filters, and grab the audio frame from the filtergraph, it sounds perfect. Once I add another filter into the graph (in this case, it's a "volume" filter), there is a lot of noise introduced. I can't figure out what would be causing this.
This isn't the case for all filters - "aecho" works just fine, for example. Any thoughts? Here's the relevant code:
Filter Creation
char args[512];
int ret = 0;
_filterGraph = avfilter_graph_alloc();
// abuffer must be the first filter used -- it feeds data into the filter graph
/******************
ABUFFER FILTER
*******************/
_abufferFilter = avfilter_get_by_name("abuffer");
/*buffer audio source : decoded frames will be
inserted here. */
if (!_inAudioCodecContext->channel_layout)
{
_inAudioCodecContext->channel_layout = av_get_default_channel_layout(_inAudioStream->codec->channels);
}
snprintf(args, sizeof(args),
"sample_rate=%d:sample_fmt=%s:channel_layout=0x%" PRIx64,
_inAudioCodecContext->sample_rate,
av_get_sample_fmt_name(_inAudioCodecContext->sample_fmt),
_inAudioCodecContext->channel_layout);
ret = avfilter_graph_create_filter(&_abufferFilterCtx, _abufferFilter, "abuffer", args, NULL, _filterGraph);
char *errorCode = new char[256];
av_strerror(ret, errorCode, 256);
/******************
VOLUME FILTER
*******************/
snprintf(args, sizeof(args),
"%f",
2.0f);
_volumeFilter = avfilter_get_by_name("volume");
ret = avfilter_graph_create_filter(&_volumeFilterCtx, _volumeFilter, "volume", args, NULL, _filterGraph);
char *errorCode = new char[256];
av_strerror(ret, errorCode, 256);
/******************
ABUFFERSINK FILTER
*******************/
// abuffersink must be the last filter used -- it gets data out of the filter graph
_abuffersinkFilter = avfilter_get_by_name("abuffersink");
ret = avfilter_graph_create_filter(&_abufferSinkFilterCtx, _abuffersinkFilter, "abuffersink", NULL, NULL, _filterGraph);
// Link the source buffer to the volume filter
// If I link this to the sink buffer and comment out the next line
// Audio sounds perfect
ret = avfilter_link(_abufferFilterCtx, 0, _volumeFilterCtx, 0);
// Link the volume filter to the sink buffer
ret = avfilter_link(_volumeFilterCtx, 0, _abufferSinkFilterCtx, 0);
ret = avfilter_graph_config(_filterGraph, NULL);
return ret;
Read frames from buffer
// Read a frame from the audio stream/file
ret = av_read_frame(_inFormatContext, &_packet);
int frameFinished = 0;
// Decode the resulting packet into a single frame for processing
int length = avcodec_decode_audio4(_inAudioCodecContext, _audioFrame, &frameFinished, &_packet);
if (frameFinished)
{
// Insert the frame into the source filter
ret = av_buffersrc_write_frame(_abufferFilterCtx, _audioFrame);
while (true)
{
// Pull a frame from the filter graph
ret = av_buffersink_get_frame(_abufferSinkFilterCtx, _audioFrame);
// EOF or EAGAIN is expected when filtering frames, set the error to "0"
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
{
ret = 0;
break;
}
if (ret < 0)
{
break;
}
}
// This keeps going, doing some resampling and conversion based on codec output selection, but that isn't relevant to the issue
I figured this out. It was defaulting to "float" precision, so I had to add the following if the format was fixed:
snprintf(args, sizeof(args),
"volume=%f:precision=fixed",
1.0f);
_volumeFilter = avfilter_get_by_name("volume");
ret = avfilter_graph_create_filter(&_volumeFilterCtx, _volumeFilter, "volume", args, NULL, _filterGraph);
(I'm not fluent in english i'll try to do my best)
I try to code (C++) a simple mkv player. I'm very new in this subject, so I discover all I need little by little. For the beginning, I use VP8 codec for video and Vorbis for audio.
The video side seem ok for now, but I'm in trouble with audio.
I can't figure out the loop logic to decode the audio frames I get from mkvparser with the libvorbis.
I looked up to this sample and this brief explanation but can't manage to make it work in my case. And I didn't find other simple examples.
Here is a chunk of my code:
const mkvparser::Block* const pBlock = m_pMkvContext->pBlockEntry->GetBlock();
const mkvparser::Track* const pTrack = m_pMkvContext->pTracks->GetTrackByNumber( (unsigned long)pBlock->GetTrackNumber() );
if ( pTrack != NULL )
{
const long long trackType = pTrack->GetType();
const int frameCount = pBlock->GetFrameCount();
if ( frameCount > 0 )
{
const mkvparser::Block::Frame& oFrame = pBlock->GetFrame( 0 );
unsigned char* pData = (unsigned char*)malloc( (size_t)oFrame.len );
oFrame.Read( &m_pMkvContext->oReader, pData );
if ( trackType == mkvparser::Track::kVideo )
{
// i'm ok here
}
else if ( trackType == mkvparser::Track::kAudio )
{
// what to do here with my audio frame data ?
}
free( pData );
}
}
And maybe the way I get frames is good for video and not for audio...
Do you guys know some good resources to share about it? Or some advices?
Thanks for help !
[EDIT] : I forgot to add one of my try:
bool MoviePlayer::DecodeAudioData( unsigned char* pData, uint32 iSize )
{
int ret;
char* pBuffer = NULL;
pBuffer = ogg_sync_buffer( &m_pOVContext->oOggSyncState, iSize );
memcpy( pBuffer, pData, iSize );
ogg_sync_wrote( &m_pOVContext->oOggSyncState, iSize );
ret = ogg_sync_pageout( &m_pOVContext->oOggSyncState, &m_pOVContext->oOggPage );
ret = ogg_stream_init( &m_pOVContext->oOggStreamState, ogg_page_serialno(&m_pOVContext->oOggPage) );
ret = ogg_stream_pagein( &m_pOVContext->oOggStreamState, &m_pOVContext->oOggPage );
int iPacketsCount = ogg_page_packets( &m_pOVContext->oOggPage );
for ( int i = 0; i < iPacketsCount; ++i )
{
ret = ogg_stream_packetout(&m_pOVContext->oOggStreamState, &m_pOVContext->oOggPacket);
// do something with the packet...
}
return true;
}
It crashes at ogg_sync_pageout, as my ogg_page was not correctly initialized.
But, not coming from a proper .ogg file as in the examples i found, i don't know how to correctly initialize the vorbis structures.
https://matroska.org/technical/specs/codecid/index.html
in A_VORBIS section
The private data contains the first three Vorbis packet in order....
and the codec private is here
https://matroska.org/technical/specs/index.html
"CodecPrivate 3 [63][A2]"
I'm currently having issues trying to encapsulate raw H264 nal packets into a mp4 container. Instead of writing them to disk however, I want to have the result stored in memory. I followed this approach Raw H264 frames in mpegts container using libavcodec but haven't been successful so far.
First, is this the right way to write to memory? I have a small struct in my header
struct IOOutput {
uint8_t* outBuffer;
int bytesSet;
};
where I initialize the buffer and bytesset. I then initialize my AVIOContext variable
AVIOContext* pIOCtx = avio_alloc_context(pBuffer, iBufSize, 1, outptr, NULL, write_packet, NULL);
where outptr is a void pointer to IOOutput output, and write_packet looks like the following
int write_packet (void *opaque, uint8_t *buf, int buf_size) {
IOOutput* out = reinterpret_cast<IOOutput*>(opaque);
memcpy(out->outBuffer+out->bytesSet, buf, buf_size);
out->bytesSet+=buf_size;
return buf_size;
}
I then set
fc->pb = pIOCtx;
fc->flags = AVFMT_FLAG_CUSTOM_IO;
on my AVFormatContext *fc variable.
Then, whenever I encode the nal packets I have from a frame, I write them to the AVFormatContext via av_interleaved_write_frame and then get the mp4 contents via
void getBufferContent(char* buffer) {
memcpy(buffer, output.outBuffer, output.bytesSet);
output.bytesSet=0;
}
and thus reset the variable bytesSet, so during the next writing operation bytes will be inserted at the start of the buffer. Is there a better way to do this? Is this actually a valid way to do it? Does FFMPEG do any reading operation if I only do call av_interleaved_write_frame and avformat_write_header in order to add packets?
Thank you very much in advance!
EDIT
Here is the code regarding the muxing process - in my encode Function I have something like
int frame_size = x264_encoder_encode(obj->mEncoder, &obj->nals, &obj->i_nals, obj->pic_in, obj->pic_out);
int total_size=0;
for(int i=0; i<obj->i_nals;i++)
{
if ( !obj->fc ) {
obj->create( obj->nals[i].p_payload, obj->nals[i].i_payload );
}
if ( obj->fc ) {
obj->write_frame( obj->nals[i].p_payload, obj->nals[i].i_payload);
}
}
// Here I get the output values
int currentBufferSize = obj->output.bytesSet;
char* mem = new char[currentBufferSize];
obj->getBufferContent(mem);
And the create and write functions look like this
int create(void *p, int len) {
AVOutputFormat *of = av_guess_format( "mp4", 0, 0 );
fc = avformat_alloc_context();
// Add video stream
AVStream *pst = av_new_stream( fc, 0 );
vi = pst->index;
void* outptr = (void*) &output;
// Create Buffer
pIOCtx = avio_alloc_context(pBuffer, iBufSize, 1, outptr, NULL, write_packet, NULL);
fc->oformat = of;
fc->pb = pIOCtx;
fc->flags = AVFMT_FLAG_CUSTOM_IO;
pcc = pst->codec;
AVCodec c= {0};
c.type= AVMEDIA_TYPE_VIDEO;
avcodec_get_context_defaults3( pcc, &c );
pcc->codec_type = AVMEDIA_TYPE_VIDEO;
pcc->codec_id = codec_id;
pcc->bit_rate = br;
pcc->width = w;
pcc->height = h;
pcc->time_base.num = 1;
pcc->time_base.den = fps;
}
void write_frame( const void* p, int len ) {
AVStream *pst = fc->streams[ vi ];
// Init packet
AVPacket pkt;
av_init_packet( &pkt );
pkt.flags |= ( 0 >= getVopType( p, len ) ) ? AV_PKT_FLAG_KEY : 0;
pkt.stream_index = pst->index;
pkt.data = (uint8_t*)p;
pkt.size = len;
pkt.dts = AV_NOPTS_VALUE;
pkt.pts = AV_NOPTS_VALUE;
av_interleaved_write_frame( fc, &pkt );
}
See the AVFormatContext.pb documentation. You set it correctly, but you shouldn't touch AVFormatContext.flags. Also, make sure you set it before calling avformat_write_header().
When you say "it doesn't work", what exactly doesn't work? Is the callback not invoked? Is the data in it not of the expected type/format? Something else? If all you want to do is write raw nal packets, then you could just take encoded data directly from the encoder (in the AVPacket), that's the raw nal data. If you use libx264's api directly, it even gives you each nal individually so you don't need to parse it.
I couldn't figure out how to create my own sound player, so I have opted to use one from ChiliTomatoNoodle's framework.
The issue I'm having, however, is I have a 180s wave file, that's only playing the first second or so. What do I have to do to make it play longer?
Sound.h:
#pragma once
#include <windows.h>
#include <mmsystem.h>
#include <dsound.h>
#include <stdio.h>
class DSound;
class Sound
{
friend DSound;
public:
Sound( const Sound& base );
Sound();
~Sound();
const Sound& operator=( const Sound& rhs );
void Play( int attenuation = DSBVOLUME_MAX );
private:
Sound( IDirectSoundBuffer8* pSecondaryBuffer );
private:
IDirectSoundBuffer8* pBuffer;
};
class DSound
{
private:
struct WaveHeaderType
{
char chunkId[4];
unsigned long chunkSize;
char format[4];
char subChunkId[4];
unsigned long subChunkSize;
unsigned short audioFormat;
unsigned short numChannels;
unsigned long sampleRate;
unsigned long bytesPerSecond;
unsigned short blockAlign;
unsigned short bitsPerSample;
char dataChunkId[4];
unsigned long dataSize;
};
public:
DSound( HWND hWnd );
~DSound();
Sound CreateSound( char* wavFileName );
private:
DSound();
private:
IDirectSound8* pDirectSound;
IDirectSoundBuffer* pPrimaryBuffer;
};
Sound.cpp:
#include "Sound.h"
#include <assert.h>
#pragma comment(lib, "dsound.lib")
#pragma comment(lib, "dxguid.lib")
#pragma comment(lib, "winmm.lib" )
DSound::DSound( HWND hWnd )
: pDirectSound( NULL ),
pPrimaryBuffer( NULL )
{
HRESULT result;
DSBUFFERDESC bufferDesc;
WAVEFORMATEX waveFormat;
result = DirectSoundCreate8( NULL,&pDirectSound,NULL );
assert( !FAILED( result ) );
// Set the cooperative level to priority so the format of the primary sound buffer can be modified.
result = pDirectSound->SetCooperativeLevel( hWnd,DSSCL_PRIORITY );
assert( !FAILED( result ) );
// Setup the primary buffer description.
bufferDesc.dwSize = sizeof(DSBUFFERDESC);
bufferDesc.dwFlags = DSBCAPS_PRIMARYBUFFER | DSBCAPS_CTRLVOLUME;
bufferDesc.dwBufferBytes = 0;
bufferDesc.dwReserved = 0;
bufferDesc.lpwfxFormat = NULL;
bufferDesc.guid3DAlgorithm = GUID_NULL;
// Get control of the primary sound buffer on the default sound device.
result = pDirectSound->CreateSoundBuffer( &bufferDesc,&pPrimaryBuffer,NULL );
assert( !FAILED( result ) );
// Setup the format of the primary sound bufffer.
// In this case it is a .WAV file recorded at 44,100 samples per second in 16-bit stereo (cd audio format).
waveFormat.wFormatTag = WAVE_FORMAT_PCM;
waveFormat.nSamplesPerSec = 44100;
waveFormat.wBitsPerSample = 16;
waveFormat.nChannels = 2;
waveFormat.nBlockAlign = (waveFormat.wBitsPerSample / 8) * waveFormat.nChannels;
waveFormat.nAvgBytesPerSec = waveFormat.nSamplesPerSec * waveFormat.nBlockAlign;
waveFormat.cbSize = 0;
// Set the primary buffer to be the wave format specified.
result = pPrimaryBuffer->SetFormat( &waveFormat );
assert( !FAILED( result ) );
}
DSound::~DSound()
{
if( pPrimaryBuffer )
{
pPrimaryBuffer->Release();
pPrimaryBuffer = NULL;
}
if( pDirectSound )
{
pDirectSound->Release();
pDirectSound = NULL;
}
}
// must be 44.1k 16bit Stereo PCM Wave
Sound DSound::CreateSound( char* wavFileName )
{
int error;
FILE* filePtr;
unsigned int count;
WaveHeaderType waveFileHeader;
WAVEFORMATEX waveFormat;
DSBUFFERDESC bufferDesc;
HRESULT result;
IDirectSoundBuffer* tempBuffer;
IDirectSoundBuffer8* pSecondaryBuffer;
unsigned char* waveData;
unsigned char* bufferPtr;
unsigned long bufferSize;
// Open the wave file in binary.
error = fopen_s( &filePtr,wavFileName,"rb" );
assert( error == 0 );
// Read in the wave file header.
count = fread( &waveFileHeader,sizeof( waveFileHeader ),1,filePtr );
assert( count == 1 );
// Check that the chunk ID is the RIFF format.
assert( (waveFileHeader.chunkId[0] == 'R') &&
(waveFileHeader.chunkId[1] == 'I') &&
(waveFileHeader.chunkId[2] == 'F') &&
(waveFileHeader.chunkId[3] == 'F') );
// Check that the file format is the WAVE format.
assert( (waveFileHeader.format[0] == 'W') &&
(waveFileHeader.format[1] == 'A') &&
(waveFileHeader.format[2] == 'V') &&
(waveFileHeader.format[3] == 'E') );
// Check that the sub chunk ID is the fmt format.
assert( (waveFileHeader.subChunkId[0] == 'f') &&
(waveFileHeader.subChunkId[1] == 'm') &&
(waveFileHeader.subChunkId[2] == 't') &&
(waveFileHeader.subChunkId[3] == ' ') );
// Check that the audio format is WAVE_FORMAT_PCM.
assert( waveFileHeader.audioFormat == WAVE_FORMAT_PCM );
// Check that the wave file was recorded in stereo format.
assert( waveFileHeader.numChannels == 2 );
// Check that the wave file was recorded at a sample rate of 44.1 KHz.
assert( waveFileHeader.sampleRate == 44100 );
// Ensure that the wave file was recorded in 16 bit format.
assert( waveFileHeader.bitsPerSample == 16 );
// Check for the data chunk header.
assert( (waveFileHeader.dataChunkId[0] == 'd') &&
(waveFileHeader.dataChunkId[1] == 'a') &&
(waveFileHeader.dataChunkId[2] == 't') &&
(waveFileHeader.dataChunkId[3] == 'a') );
// Set the wave format of secondary buffer that this wave file will be loaded onto.
waveFormat.wFormatTag = WAVE_FORMAT_PCM;
waveFormat.nSamplesPerSec = 44100;
waveFormat.wBitsPerSample = 16;
waveFormat.nChannels = 2;
waveFormat.nBlockAlign = (waveFormat.wBitsPerSample / 8) * waveFormat.nChannels;
waveFormat.nAvgBytesPerSec = waveFormat.nSamplesPerSec * waveFormat.nBlockAlign;
waveFormat.cbSize = 0;
// Set the buffer description of the secondary sound buffer that the wave file will be loaded onto.
bufferDesc.dwSize = sizeof(DSBUFFERDESC);
bufferDesc.dwFlags = DSBCAPS_CTRLVOLUME;
bufferDesc.dwBufferBytes = waveFileHeader.dataSize;
bufferDesc.dwReserved = 0;
bufferDesc.lpwfxFormat = &waveFormat;
bufferDesc.guid3DAlgorithm = GUID_NULL;
// Create a temporary sound buffer with the specific buffer settings.
result = pDirectSound->CreateSoundBuffer( &bufferDesc,&tempBuffer,NULL );
assert( !FAILED( result ) );
// Test the buffer format against the direct sound 8 interface and create the secondary buffer.
result = tempBuffer->QueryInterface( IID_IDirectSoundBuffer8,(void**)&pSecondaryBuffer );
assert( !FAILED( result ) );
// Release the temporary buffer.
tempBuffer->Release();
tempBuffer = 0;
// Move to the beginning of the wave data which starts at the end of the data chunk header.
fseek( filePtr,sizeof(WaveHeaderType),SEEK_SET );
// Create a temporary buffer to hold the wave file data.
waveData = new unsigned char[ waveFileHeader.dataSize ];
assert( waveData );
// Read in the wave file data into the newly created buffer.
count = fread( waveData,1,waveFileHeader.dataSize,filePtr );
assert( count == waveFileHeader.dataSize);
// Close the file once done reading.
error = fclose( filePtr );
assert( error == 0 );
// Lock the secondary buffer to write wave data into it.
result = pSecondaryBuffer->Lock( 0,waveFileHeader.dataSize,(void**)&bufferPtr,(DWORD*)&bufferSize,NULL,0,0 );
assert( !FAILED( result ) );
// Copy the wave data into the buffer.
memcpy( bufferPtr,waveData,waveFileHeader.dataSize );
// Unlock the secondary buffer after the data has been written to it.
result = pSecondaryBuffer->Unlock( (void*)bufferPtr,bufferSize,NULL,0 );
assert( !FAILED( result ) );
// Release the wave data since it was copied into the secondary buffer.
delete [] waveData;
waveData = NULL;
return Sound( pSecondaryBuffer );
}
Sound::Sound( IDirectSoundBuffer8* pSecondaryBuffer )
: pBuffer( pSecondaryBuffer )
{}
Sound::Sound()
: pBuffer( NULL )
{}
Sound::Sound( const Sound& base )
: pBuffer( base.pBuffer )
{
pBuffer->AddRef();
}
Sound::~Sound()
{
if( pBuffer )
{
pBuffer->Release();
pBuffer = NULL;
}
}
const Sound& Sound::operator=( const Sound& rhs )
{
this->~Sound();
pBuffer = rhs.pBuffer;
pBuffer->AddRef();
return rhs;
}
// attn is the attenuation value in units of 0.01 dB (larger
// negative numbers give a quieter sound, 0 for full volume)
void Sound::Play( int attn )
{
attn = max( attn,DSBVOLUME_MIN );
HRESULT result;
// check that we have a valid buffer
assert( pBuffer != NULL );
// Set position at the beginning of the sound buffer.
result = pBuffer->SetCurrentPosition( 0 );
assert( !FAILED( result ) );
// Set volume of the buffer to attn
result = pBuffer->SetVolume( attn );
assert( !FAILED( result ) );
// Play the contents of the secondary sound buffer.
result = pBuffer->Play( 0,0,0 );
assert( !FAILED( result ) );
}
Thanks for your help in advance!
Assuming you have a .wav file, and you are loading the sound file somewhere along the lines of:
yourSound = audio.CreateSound("fileName.WAV"); //Capslock on WAV
yourSound.Play();
With this comes the declaration of the Sound in the header:
Sound yourSound;
Now because you have probably done this already and this is not working it likely has to do with your file as playing sounds 160 seconds+ should not be a problem.
Are you using a .WAV file for the sound? If so did you happen to convert that (as it is probably a background sound?). If you did try converting it with this converter:
Converter MP3 -> WAV
Please let me know if this works!
Your buffer is probably only large enough to play the first second or so. What you need to do is setup "notifications". See the documentation.
Notifications are a way to ask the audio hardware to let you know when they have reached a specific point in the buffer.
The idea is to setup a notification in the middle of the buffer and at the end of the buffer. When you receive the notification from the notification in the middle, you fill the first half of the buffer with more data. When you receive the notification from the end, you fill the second half of the buffer with more data. This way, you can stream an infinite amount of data with a single buffer.
I am trying to play a sound on OSX, from a buffer (eg: Equivalent of Windows "PlaySound" function).
I have put together some C++ code to play audio with AudioQueue (as it is my understanding that this is the easiest way to play audio on OSX).
However, no sound is ever generated, and the audio callback function is never called.
Does anybody know what I'm doing wrong, or does anyone have a simple C/C++ example of how to play a sound on OSX?
#include
#include
#define BUFFER_COUNT 3
static struct AQPlayerState {
AudioStreamBasicDescription desc;
AudioQueueRef queue;
AudioQueueBufferRef buffers[BUFFER_COUNT];
unsigned buffer_size;
} state;
static void audio_callback (void *aux, AudioQueueRef aq, AudioQueueBufferRef bufout)
{
printf("I never get called!\n");
#define nsamples 4096
short data[nsamples];
for (int i=0;imAudioDataByteSize = nsamples * sizeof(short) * 1;
assert(bufout->mAudioDataByteSize mAudioData, data, bufout->mAudioDataByteSize);
AudioQueueEnqueueBuffer(state.queue, bufout, 0, NULL);
}
void audio_init()
{
int i;
bzero(&state, sizeof(state));
state.desc.mFormatID = kAudioFormatLinearPCM;
state.desc.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
state.desc.mSampleRate = 44100;
state.desc.mChannelsPerFrame = 1;
state.desc.mFramesPerPacket = 1;
state.desc.mBytesPerFrame = sizeof (short) * state.desc.mChannelsPerFrame;
state.desc.mBytesPerPacket = state.desc.mBytesPerFrame;
state.desc.mBitsPerChannel = (state.desc.mBytesPerFrame*8)/state.desc.mChannelsPerFrame;
state.desc.mReserved = 0;
state.buffer_size = state.desc.mBytesPerFrame * state.desc.mSampleRate;
if (noErr != AudioQueueNewOutput(&state.desc, audio_callback, 0, NULL, NULL, 0, &state.queue)) {
printf("audioqueue error\n");
return;
}
// Start some empty playback so we'll get the callbacks that fill in the actual audio.
for (i = 0; i mAudioDataByteSize = state.buffer_size;
AudioQueueEnqueueBuffer(state.queue, state.buffers[i], 0, NULL);
}
if (noErr != AudioQueueStart(state.queue, NULL)) printf("AudioQueueStart failed\n");
printf("started audio\n");
}
int main() {
audio_init();
while (1) {
printf("I can't hear anything!\n");
}
}
REFS:
developer.apple.com Audio Queue Services Programming Guide: Playing Audio
developer.apple.com Audio Queue Services Reference
where to start with audio synthesis on iPhone Answer by Andy J Buchanan
Note that I had to explicitly set mAudioDataByteSize to the size I allocated.
In the docs they mention that it is initially set to zero, and that's what I found.
The docs don't say why but I suspect it's to allow for variable size buffers or something?
/* Ben's Audio Example for OSX 10.5+ (yeah Audio Queue)
Ben White, Nov, 2011
Makefile:
example: example.c
gcc -o $# $< -Wimplicit -framework AudioToolbox \
-framework CoreFoundation -lm
*/
#include "AudioToolbox/AudioToolbox.h"
typedef struct {
double phase, phase_inc;
int count;
} PhaseBlah;
void callback (void *ptr, AudioQueueRef queue, AudioQueueBufferRef buf_ref)
{
OSStatus status;
PhaseBlah *p = ptr;
AudioQueueBuffer *buf = buf_ref;
int nsamp = buf->mAudioDataByteSize / 2;
short *samp = buf->mAudioData;
int ii;
printf ("Callback! nsamp: %d\n", nsamp);
for (ii = 0; ii < nsamp; ii++) {
samp[ii] = (int) (30000.0 * sin(p->phase));
p->phase += p->phase_inc;
//printf("phase: %.3f\n", p->phase);
}
p->count++;
status = AudioQueueEnqueueBuffer (queue, buf_ref, 0, NULL);
printf ("Enqueue status: %d\n", status);
}
int main (int argc, char *argv[])
{
AudioQueueRef queue;
PhaseBlah phase = { 0, 2 * 3.14159265359 * 450 / 44100 };
OSStatus status;
AudioStreamBasicDescription fmt = { 0 };
AudioQueueBufferRef buf_ref, buf_ref2;
fmt.mSampleRate = 44100;
fmt.mFormatID = kAudioFormatLinearPCM;
fmt.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
fmt.mFramesPerPacket = 1;
fmt.mChannelsPerFrame = 1; // 2 for stereo
fmt.mBytesPerPacket = fmt.mBytesPerFrame = 2; // x2 for stereo
fmt.mBitsPerChannel = 16;
status = AudioQueueNewOutput(&fmt, callback, &phase, CFRunLoopGetCurrent(),
kCFRunLoopCommonModes, 0, &queue);
if (status == kAudioFormatUnsupportedDataFormatError) puts ("oops!");
else printf("NewOutput status: %d\n", status);
status = AudioQueueAllocateBuffer (queue, 20000, &buf_ref);
printf ("Allocate status: %d\n", status);
AudioQueueBuffer *buf = buf_ref;
printf ("buf: %p, data: %p, len: %d\n", buf, buf->mAudioData, buf->mAudioDataByteSize);
buf->mAudioDataByteSize = 20000;
callback (&phase, queue, buf_ref);
status = AudioQueueAllocateBuffer (queue, 20000, &buf_ref2);
printf ("Allocate2 status: %d\n", status);
buf = buf_ref2;
buf->mAudioDataByteSize = 20000;
callback (&phase, queue, buf_ref2);
status = AudioQueueSetParameter (queue, kAudioQueueParam_Volume, 1.0);
printf ("Volume status: %d\n", status);
status = AudioQueueStart (queue, NULL);
printf ("Start status: %d\n", status);
while (phase.count < 15)
CFRunLoopRunInMode (
kCFRunLoopDefaultMode,
0.25, // seconds
false // don't return after source handled
);
return 0;
}
Based somewhat on the answer by bw1024, I created this complete ogg vorbis player with libvorbisfile.
It expands on the previous answer, demonstrates how to use a function to fill in the audio buffers (as in, doesn't generate the sound by itself) and adds end-of-playback detection with an event listener callback.
The code itself is heavily commented, which hopefully explains everything that needs to be explained.
I tried to keep it as close to "production quality" of both Audio Queues and libvorbisfile, so it contains "real" error conditions and checks for exceptional circumstances; such as variable sample rate in the vorbis file, which it can't handle.
I hope none of the noise distracts from its value as a sample.
// vorplay.c - by Johann `Myrkraverk' Oskarsson <johann#myrkraverk.com>
// In the interest of example code, it's explicitly licensed under the
// WTFPL, see the bottom of the file or http://www.wtfpl.net/ for details.
#include <pthread.h> // For pthread_exit().
#include <vorbis/vorbisfile.h>
#include <AudioToolbox/AudioToolbox.h>
#include <stdio.h>
// This is a complete example of an Ogg Vorbis player based on the vorbisfile
// library and the audio queue API in OS X.
// It can be either taken as an example of how to use libvorbisfile, or
// audio queue programming.
// There are many "magic number" constants in the code, and understanding
// them requires looking up the relevant documentation. Some, such as
// the number of buffers in the audio queue and the size of each buffer
// are the result of experimentation. A "real application" may benefit
// from allowing the user to tweak these, in order to resolve audio stutters.
// Error handling is done very simply in order to focus on the example code
// while still resembling "production code." Here, we use the
// if ( status = foo() ) { ... }
// syntax for error checking. The assignment in if()s is not an error.
// If your compiler is complaining, you can use its equivalent of the
// GCC switch -Wno-parentheses to silence it.
// Assuming you'll want to use libvorbisfile from mac ports, you can
// compile it like this.
// gcc -c -I/opt/local/include \
// vorplay.c \
// -Wno-parentheses
// And link with
// gcc -o vorplay vorplay.o \
// -L/opt/local/lib -lvorbisfile \
// -framework AudioToolbox
// The start/stop listener...
void listener( void *vorbis, AudioQueueRef queue, AudioQueuePropertyID id )
{
// Here, we're only listening for start/stop, so don't need to check
// the id; it's always kAudioQueueProperty_IsRunning in our case.
UInt32 running = 0;
UInt32 size = sizeof running;
/* OggVorbis_File *vf = (OggVorbis_File *) vorbis; */
OSStatus status = -1;
if ( status = AudioQueueGetProperty( queue, id, &running, &size ) ) {
printf( "AudioQueueGetProperty status = %d; running = %d\n",
status, running );
exit( 1 );
}
if ( !running ) {
// In a "real example" we'd clean up the vf pointer with ov_clear() and
// the audio queue with AudioQueueDispose(); however, the latter is
// better not called from within the listener function, so we just
// exit normally.
exit( 0 );
// In a "real" application, we might signal the termination with
// a pthread condition variable, or something similar, instead;
// where the waiting thread would call AudioQueueDispose(). It is
// "safe" to call ov_clear() here, but there's no point.
}
}
// The audio queue callback...
void callback( void *vorbis, AudioQueueRef queue, AudioQueueBufferRef buffer )
{
OggVorbis_File *vf = (OggVorbis_File *) vorbis;
int section = 0;
OSStatus status = -1;
// The parameters here are congruent with our format specification for
// the audio queue. We read directly into the audio queue buffer.
long r = ov_read( vf, buffer->mAudioData, buffer->mAudioDataBytesCapacity,
0, 2, 1, §ion );
// As an extra precaution, check if the current buffer is the same sample
// rate and channel number as the audio queue.
{
vorbis_info *vinfo = ov_info( vf, section );
if ( vinfo == NULL ) {
printf( "ov_info status = NULL\n" );
exit( 1 );
}
AudioStreamBasicDescription description;
UInt32 size = sizeof description;
if ( status = AudioQueueGetProperty( queue,
kAudioQueueProperty_StreamDescription,
&description,
&size ) ) {
printf( "AudioQueueGetProperty status = %d\n", status );
exit( 1 );
}
// If we were using some other kind of audio playback API, such as OSS4
// we could simply change the sample rate and channel number on the fly.
// However, with an audio queue, we'd have to use a different
// one, afaict; so we don't handle it at all in this example.
if ( vinfo->rate != description.mSampleRate ) {
printf( "We don't handle changes in sample rate.\n" );
exit( 1 );
}
if ( vinfo->channels != description.mChannelsPerFrame ) {
printf( "We don't handle changes in channel numbers.\n" );
exit( 1 );
}
}
// The real "callback"...
if ( r == 0 ) { // No more data, stop playing.
// Flush data, to make sure we play to the end.
if ( status = AudioQueueFlush( queue ) ) {
printf( "AudioQueueFlush status = %d\n", status );
exit( 1 );
}
// Stop asynchronously.
if ( status = AudioQueueStop( queue, false ) ) {
printf( "AudioQueueStop status = %d\n", status );
exit( 1 );
}
} else if ( r < 0 ) { // Some error?
printf( "ov_read status = %ld\n", r );
exit( 1 );
} else { // The normal course of action.
// ov_read() may not return exactly the number of bytes we requested.
// so we update the buffer size per call.
buffer->mAudioDataByteSize = r;
if ( status = AudioQueueEnqueueBuffer( queue, buffer, 0, 0 ) ) {
printf( "AudioQueueEnqueueBuffer status = %d, r = %ld\n", status, r );
exit( 1 );
}
}
}
int main( int argc, char *argv[] )
{
// The very simple command line argument check.
if ( argc != 2 ) {
printf( "Usage: vorplay <file>\n" );
exit( 1 );
}
FILE *file = fopen( argv[ 1 ], "r" );
if ( file == NULL ) {
printf( "Unable to open input file.\n" );
exit( 1 );
}
OggVorbis_File vf;
// Using OV_CALLBACKS_DEFAULT means ov_clear() will close the file.
// However, this particular example doesn't use that function.
// A typical place for it might be the listener(), when we stop
// playing.
if ( ov_open_callbacks( file, &vf, 0, 0, OV_CALLBACKS_DEFAULT ) ) {
printf( "ov_open_callbacks() failed. Not an Ogg Vorbis file?\n" );
exit( 1 );
}
// For the sample rate and channel number in the audio.
vorbis_info *vinfo = ov_info( &vf, -1 );
if ( vinfo == NULL ) {
printf( "ov_info status = NULL\n" );
exit( 1 );
}
// The audio queue format specification. This structure must be set
// to values congruent to the ones we use with ov_read().
AudioStreamBasicDescription format = { 0 };
// First, the constants. The format is quite hard coded, both here
// and in the calls to ov_read().
format.mFormatID = kAudioFormatLinearPCM;
format.mFormatFlags =
kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
format.mFramesPerPacket = 1;
format.mBitsPerChannel = 16;
// Load the sample rate and channel number from the vorbis file.
format.mSampleRate = vinfo->rate;
format.mChannelsPerFrame = vinfo->channels;
// The number of bytes depends on the channel number.
format.mBytesPerPacket =
format.mBytesPerFrame = 2 * vinfo->channels; // times two, for 16bit
OSStatus status = -1;
AudioQueueRef queue;
// Create the audio queue with the desired format. Notice that we
// use the OggVorbis_File pointer as the data far the callback.
if ( status = AudioQueueNewOutput( &format, callback,
&vf, NULL, NULL, 0, &queue ) ) {
printf( "AudioQueueNewOutput status = %d\n", status );
exit( 1 );
}
// For me distortions happen with 3 buffers; hence the magic number 5.
AudioQueueBufferRef buffers[ 5 ];
for ( int i = 0; i < sizeof buffers / sizeof (AudioQueueBufferRef); i++ ) {
// For each buffer...
// The size of the buffer is a magic number. 4096 is good enough, too.
if ( status = AudioQueueAllocateBuffer( queue, 8192, &buffers[ i ] ) ) {
printf( "AudioQueueAllocateBuffer status = %d\n", status );
exit( 1 );
}
// Enqueue buffers, before play. According to the process outlined
// in the Audio Queue Services Programming Guide, we must do this
// before calling AudioQueueStart() and it's simplest to do it like
// this.
callback( &vf, queue, buffers[ i ] );
}
// We set the volume to maximum; even though the docs say it's the
// default.
if ( status = AudioQueueSetParameter( queue,
kAudioQueueParam_Volume, 1.0 ) ) {
printf( "AudioQueueSetParameter status = %d\n", status );
exit( 1 );
}
// Here, we might want to call AudioQueuePrime if we were playing one
// of the supported compressed formats. However, since we only have
// raw PCM buffers to play, I don't see the point. Maybe playing will
// start faster with it, after AudioQueueStart() but I still don't see
// the point for this example; if there's a delay, it'll happen anyway.
// We add a listener for the start/stop event, so we know when to call
// exit( 0 ) and terminate the application. We also give it the vf
// pointer, even though it's not used in our listener().
if ( status = AudioQueueAddPropertyListener( queue,
kAudioQueueProperty_IsRunning,
listener,
&vf ) ) {
printf( "AudioQueueAddPropertyListener status = %d\n", status );
exit( 1 );
}
// And then start to play the file.
if ( status = AudioQueueStart( queue, 0 ) ) {
printf( "AudioQueueStart status = %d\n", status );
exit( 1 );
}
// Work's For Me[tm]. This trick to make sure the process doesn't
// terminate before the song has played "works for me" on
// OS X 10.10.3. If you're going to use this same trick in production
// code, you might as well turn off the joinability of the main thread,
// with pthread_detach() and also make sure no callback or listener is
// using data from the stack. Unlike this example.
pthread_exit( 0 );
return 0; // never reached, left intact in case some compiler complains.
}
// DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
// Version 2, December 2004
//
// Copyright (C) 2015 Johann `Myrkraverk' Oskarsson
// <johann#myrkraverk.com>
//
// Everyone is permitted to copy and distribute verbatim or modified
// copies of this license document, and changing it is allowed as long
// as the name is changed.
//
// DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
// TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
//
// 0. You just DO WHAT THE FUCK YOU WANT TO.