I've followed the tutorial along Microsoft's website, and created my own SoundEngine and Sound class structure to have the code abstracted away in main, however whenever I make a call such as batmanWav->play(), it will only play the audio if I write std::cin.get() or system("PAUSE")
Why is this? I will be trying to port this to a game that's already been worked on, but I obviously don't want the game to stop every time a sound is played.
EDIT: I was asked to show some code which has the issues
int main()
{
CoInitializeEx(nullptr, COINIT_MULTITHREADED);
Sound batman("batman.wav");
Sound alien("alien.wav");
alien.play();
batman.play();
system("PAUSE");
CoUninitialize();
}
Sound.cpp
HRESULT Sound::play()
{
HRESULT hr = S_OK;
if (FAILED(hr = pSourceVoice->Start(0)))
return hr;
return hr;
}
For each Sound object I initialized a source voice, and each one refers to the same mastering voice and IXAudio2 *pXAudio2 object. The code I used to load wave file data was taken straight off MSDN docs.
As someone already noted, the problem is that your application exits before the sound really starts playing. XAudio2 is a non-blocking API, which means that you have to keep the memory live and the audio graph active until the sounds complete playing.
In other words, when you called IXAudio2SourceVoice::Start, nothing happens except that it logs that you want to start the audio. Another thread which was created as part of your IXAudio2 object then sees the request and begins to process the playback request. By that point, your application has exited and the process terminated unless you 'pause'.
Try something like the following from XAudio2BasicSound:
hr = pSourceVoice->Start( 0 );
// Let the sound play
BOOL isRunning = TRUE;
while( SUCCEEDED( hr ) && isRunning )
{
XAUDIO2_VOICE_STATE state;
pSourceVoice->GetState( &state );
isRunning = ( state.BuffersQueued > 0 ) != 0;
Sleep( 10 );
}
You should look at these samples and at DirectX Tool Kit for Audio.
Related
So we are using a stack consisting of c++ media foundation code in order to playback video files. An important requirement is the ability to play these videos in constantly repeating sequences, so every single video slot will periodically change the video it is playing. In our current example we are creating 16 HWNDs to render video into and 16 corresponding player objects. The main application loops over all of them one after another and does the following:
Shutdown the last player
Release the object
CoCreateinstance for a new player
Initialize the player with the (old) HWND
Start Playback
The media player is called "MediaPlayer2", this needs to be built and registered as COM (regsvr32). The main application is to be found in the TAPlayer2 Project. It searches for the CLSID of the player in the registry and instantiates it. As current test file we use a test.mp4 that has to reside on the disk like C:\test.mp4
Now everything goes fine initially. The program loops through the players and the video keeps restarting and playing. The memory footprint is normal and all goes smooth. After a timeframe of anything between 20 minutes and 4 days, all of the sudden things will get weird. At this point it seems as if calls to "InitializeRenderer" by the EVR slow down and eventually don't go through anymore at all. With this, also thread count and memory footprint will start to increase drastically and after a certain amount of time depending on existing RAM all the memory will be exhausted and our application crashes, usually somewhere in the GPU driver or near the EVR DLL.
I am happy to try out any other code examples that propose to solve my issue: displaying multiple video windows at the same time, and looping through them like in a playlist. Needs to be running on Windows 10!
I have been going at this for quite a while now and am pretty hard stuck. I uploaded the above mentioned code example and added the link to this post. This should work out of the box afaik. I can also provide code excerpts in here in the thread if that is preferred.
Any help is appreciated, thanks
Thomas
Link to demo project (VS2015): https://filebin.net/l8gl79jrz6fd02vt
edit: the following code from the end of winmain.cpp is used to restart the players:
do
{
for (int i = 0; i < PLAYER_COUNT; i++)
{
hr = g_pPlayer[i]->Shutdown();
SafeRelease(&g_pPlayer[i]);
hr = CoCreateInstance(CLSID_AvasysPlayer, // CLSID of the coclass
NULL, // no aggregation
CLSCTX_INPROC_SERVER, // the server is in-proc
__uuidof(IAvasysPlayer), // IID of the interface we want
(void**)&g_pPlayer[i]); // address of our interface pointer
hr = g_pPlayer[i]->InitPlayer(hwndPlayers[i]);
hr = g_pPlayer[i]->OpenUrl(L"C:\\test.mp4");
}
} while (true);
Some MediaFoundation interface like
IMFMediaSource
IMFMediaSession
IMFMediaSink
need to be Shutdown before Release them.
At this point it seems as if calls to "InitializeRenderer" by the EVR slow down and eventually don't go through anymore at all.
... usually somewhere in the GPU driver or near the EVR DLL.
a good track to make a precise search in your code.
In your file PlayerTopoBuilder.cpp, at CPlayerTopoBuilder::AddBranchToPartialTopology :
if (bVideo)
{
if (false) {
BREAK_ON_FAIL(hr = CreateMediaSinkActivate(pSD, hVideoWnd, &pSinkActivate));
BREAK_ON_FAIL(hr = AddOutputNode(pTopology, pSinkActivate, 0, &pOutputNode));
}
else {
//// try directly create renderer
BREAK_ON_FAIL(hr = MFCreateVideoRenderer(__uuidof(IMFMediaSink), (void**)&pMediaSink));
CComQIPtr<IMFVideoRenderer> pRenderer = pMediaSink;
BREAK_ON_FAIL(hr = pRenderer->InitializeRenderer(nullptr, nullptr));
CComQIPtr<IMFGetService> getService(pRenderer);
BREAK_ON_FAIL(hr = getService->GetService(MR_VIDEO_RENDER_SERVICE, __uuidof(IMFVideoDisplayControl), (void**)&pVideoDisplayControl));
BREAK_ON_FAIL(hr = pVideoDisplayControl->SetVideoWindow(hVideoWnd));
BREAK_ON_FAIL(hr = pMediaSink->GetStreamSinkByIndex(0, &pStreamSink));
BREAK_ON_FAIL(hr = AddOutputNode(pTopology, 0, &pOutputNode, pStreamSink));
}
}
You create a IMFMediaSink with MFCreateVideoRenderer and pMediaSink. pMediaSink is release because of the use of CComPtr, but never Shutdown.
You must keep a reference on the Media Sink and Shutdown/Release it when the Player Shutdown.
Or you can use a different approach with MFCreateVideoRendererActivate.
IMFMediaSink::Shutdown
If the application creates the media sink, it is responsible for calling Shutdown to avoid memory or resource leaks.
In most applications, however, the application creates an activation object for the media sink, and the Media Session uses that object to create the media sink.
In that case, the Media Session — not the application — shuts down the media sink. (For more information, see Activation Objects.)
I also suggest you to use this kind of code at the end of CPlayer::CloseSession (after release all others objects) :
if(m_pSession != NULL){
hr = m_pSession->Shutdown();
ULONG ulMFObjects = m_pSession->Release();
m_pSession = NULL;
assert(ulMFObjects == 0);
}
For the use of MFCreateVideoRendererActivate, you can look at my MFNodePlayer project :
MFNodePlayer
EDIT
I rewrote your program, but i tried to keep your logic and original source code, like CComPtr/Mutex...
MFMultiVideo
Tell me if this program has memory leaks.
It will depend on your answer, but then we can talk about best practices with MediaFoundation.
Another thought :
Your program uses 1 to 16 IMFMediaSession. On a good computer configuration, you could use only one IMFMediasession, i think (Never try to aggregate 16 MFSource).
Visit :
CustomVideoMixer
to understand the other way to do it.
I think your approach to use 16 IMFMediasession is not the best approach on a modern computer. VuVirt talk about this.
EDIT2
I've updated MFMultiVideo using Work Queues.
I think the problem can be that you call MFStartup/MFShutdown for each players.
Just call MFStartup/MFShutdown once in winmain.cpp for example, like my program does.
I have a problem I am not able to solve. My application should be able to switch the default audio device during runtime. To achieve this I am using XAudio2 from the DirectXTK.
I implemented the IMMNotificationClient into my audio class to be able to react on default device changes.
For instance when the default device changes I stop my current source voice, reset the audio engine and start the source voice again. Everything works as expected.
However when my default device is a USB soundcard and I unplug it during the source voice is playing the application freezes.
The reason for this is that the source voice hangs when stopping the voice. Sometimes also when flushing the source buffer. Seems like the source voice could not be stopped anymore when the audio device was removed the source voice was using.
Does someone had the same problem and was able to solve this?
Here's the function I am using to reset the audio engine.
HRESULT DX::XAudioEngine::ResetAudioDevice()
{
HRESULT hr = S_OK;
this->m_retryAudio = TRUE;
if (SUCCEEDED(hr) && m_pSourceVoice)
{
hr = m_pSourceVoice->Stop();
}
if (SUCCEEDED(hr))
{
hr = m_pSourceVoice->FlushSourceBuffers();
}
if (m_audEngine && m_pSourceVoice)
{
// Get the source voice back from the smart pointer
IXAudio2SourceVoice* ptmpSrcVoice = nullptr;
ptmpSrcVoice = m_pSourceVoice.release();
// Destroy the voice
ptmpSrcVoice->DestroyVoice();
}
m_audEngine->Reset(&m_waveFormat, NULL);
// Create source voice
IXAudio2SourceVoice* ptmpSrcVoice = nullptr;
m_audEngine->AllocateVoice(
(WAVEFORMATEX*)&m_waveFormat,
SoundEffectInstance_Default,
false,
&ptmpSrcVoice
);
// Add source voice to smart pointer
m_pSourceVoice.reset(ptmpSrcVoice);
// Set the input volume
if (this->m_inputVolume != 1.0f) {
hr = m_pSourceVoice->SetVolume(this->m_inputVolume);
}
hr = m_pSourceVoice->Start(0);
this->m_retryAudio = FALSE;
return hr;
}
I have an Directshow based mediaplayer application . It works very well without any issues during normal playabck . But occasionally i am facing one issue when the Mediaplayer started just after system boot .
HRESULT CSDirectShow::RenderOutputPins (IBaseFilter* pFilter)
{
const char* funcName = "CSDirectShow::RenderOutputPins()";
HRESULT hr = S_OK;
// Enumerate all pins on the source filter,
// looking for the output pins so that I can call Render() on them
//
CComPtr< IEnumPins > pEnumPin;
if (!FAILED (pFilter->EnumPins (&pEnumPin)))
{
while (true)
{
// get the next pin
//
CComPtr< IPin > pPin;
if (pEnumPin->Next (1L, &pPin, NULL) != S_OK) break;
// I'm not interested in connected pins
// if this pin is an unconnected output pin, then render it.
//
CComPtr< IPin > pConnectedPin;
if (pPin->ConnectedTo (&pConnectedPin) == VFW_E_NOT_CONNECTED)
{
PIN_DIRECTION pinDirection;
PIN_INFO pinInfo;
//Get the information of the pin
if (pPin->QueryDirection (&pinDirection) == S_OK
&& pinDirection == PINDIR_OUTPUT
&& pPin->QueryPinInfo(&pinInfo) == S_OK
&& strstr((char*)pinInfo.achName,"~")==NULL)
{
if (FAILED (hr = m_pGB->Render (pPin)))
{
SafeRelease(&pinInfo.pFilter);
return hr;
}
}
SafeRelease(&pinInfo.pFilter);
}
}
}
TraceMsg ("%s: exit",funcName);
return S_OK;
}
When m_pGB->Render (pPin) is called ,This function never returns and it is blocked inside .I confirmed using logs .This issue happens only when i start my application immediately after bootup . When issues occures if I close and restart my application it works like a charm .Since application is designed to start automatically after system bootup this behaviour has become a bigger concern .Kindly help
IGraphBuilder.Render call does a lot internally, and specifically it goes over enumeration of potentially suitable filter, which in turn attempts to load additional DLLs registered with DirectShow environment. Such file could have missing dependencies, or dependencies on remote or temporarily inaccessible drivers (just one example).
If you experience a deadlock, you can troubleshoot it further (debug it) and get details on locked state, and on activity during Render call.
If the problem is caused by third party filters (esp. codec packs registering a collection of filters at once without thinking too much on compatibility) registered with the system in a not so good way, perhaps you could identify them and uninstall.
If you want to improve the player on your side, you should avoid Render call, and build your filter graph with smaller increments - adding specific filter and connecting pins, without leaving big tasks at mercy of Intelligent Connect, which works well in general but is sensitive to compatibility problems as mentioned above.
I have an issue when finalizing a video recording into an .mp4 using Media Foundation where the call to IMFSinkWriter->Finalize(); hangs forever. It doesn't always happen, and can happen on almost any machine (seen on Windows server, 7, 8, 10). Flush() is called on the audio and video streams before hand and no new samples are added between Flush and Finalize. Any ideas on what could cause Finalize to hang forever?
Things I've tried:
Logging all HRESULTs to check for any issues (was already checking them before proceeding to the next line of code)
Everything comes back as S_OK, not seeing any issues
Added the IMFSinkWriterCallback on the stream to get callbacks when
the stream process markers (adding markers every 10 samples) and finishes Finalize()
Haven't been able to reproduce since adding this but this would give the best information about what's going on when I get it working.
Searched code samples online to see how others are setting up the
Sink Writer and how Finalize() is used
Didn't find many samples and it looks like my code is similar to the ones that were found
Looked at encoders available and used by each system including version of the encoder dll
Encoders varied between AMD H.264 Hardware MFT Encoder and H264 Encoder MFT on machines that could reproduce the issue. Versions didn't seem to matter and some of the machines were up to date with video drivers.
Here are some code samples without any HRESULT checking (that doubled the amount of code so I took it out)
Building the sink sample:
CComPtr<IMFAttributes> pAttr;
::MFCreateAttributes( &pAttr, 4 );
pAttr->SetGUID( MF_TRANSCODE_CONTAINERTYPE, GetFileContainerType() );
pAttr->SetUINT32( MF_LOW_LATENCY, FALSE ); // Allows better multithreading
pAttr->SetUINT32( MF_SINK_WRITER_DISABLE_THROTTLING, TRUE ); // Does not block
pAttr->SetUINT32( MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, TRUE );
m_pCallback.Attach( new MFSinkWriterCallback() );
pAttr->SetUnknown( MF_SINK_WRITER_ASYNC_CALLBACK, m_pCallback );
::MFCreateSinkWriterFromURL( m_strFilename.c_str(), NULL, pAttr, &m_pSink );
if ( m_pVideoInputType && m_pVideoOutputType )
{
m_pSink->AddStream( m_pVideoOutputType, &m_dwVideoStreamId );
// Attributes for encoding?
CComPtr<IMFAttributes> pAttrVideo;
// Not sure if these are needed
//::MFCreateAttributes( &pAttrVideo, 5 );
m_pSink->SetInputMediaType( m_dwVideoStreamId, m_pVideoInputType, pAttrVideo );
}
if ( m_pAudioInputType && m_pAudioOutputType )
{
m_pSink->AddStream( m_pAudioOutputType, &m_dwAudioStreamId );
// Attributes for encoding?
CComPtr<IMFAttributes> pAttrAudio;
// Not sure if these are needed
//::MFCreateAttributes( &pAttrAudio, 2 );
//pAttrAudio->SetGUID( MF_MT_SUBTYPE, MFAudioFormat_AAC );
//pAttrAudio->SetUINT32( MF_MT_AUDIO_BITS_PER_SAMPLE, 16 );
m_pSink->SetInputMediaType( m_dwAudioStreamId, m_pAudioInputType, pAttrAudio );
}
m_pSink->BeginWriting();
Stopping the recording sample:
if ( m_dwVideoStreamId != (DWORD)-1 )
{
m_sink->Flush( m_dwVideoStreamId );
}
if ( m_dwAudioStreamId != (DWORD)-1 )
{
m_sink->Flush( m_dwAudioStreamId );
}
m_sink->Finalize();
There is a lot of situation where a Media Foundation application can hang :
Calling MFShutDown/CoUninitialize when using Media Foundation objects.
Using a GUI, and doing bad use of windows message pump in multithreaded application.
Bad use of MTA/STA components.
Bad use of critical section/wait for event function.
Forget to call EndXXX() function when BeginXXX() function are used.
Bad use of Callback function.
Forget to call AddRef when necessary, and releasing the object used by another thread.
A bug in Media Foundation (there are some on Windows Seven).
and so on...
When i say a minimal source code, i mean, isolate the source code that do the encoding process, and provide it to Github if it's too large. It's better if we can compile and try the source code, because deadlock are difficult to find.
I want to capture audio from both the mic and the speaker - separately. How can I distinguish between them? I can capture one or the other using the Wave API, e.g., WaveInOpen().
When I enumerate the devices using waveInGetNumDevs() and waveInGetDevCaps()/waveoutGetDevCaps(), there seems to be no information related to a particular end-point device (e.g., mic or speaker). I only see the following, which are adapter devices:
HD Read Audio Input
HD Read Audio Output
Webcam ...
I've actually no knowledge of the windows API so my answer isn't probably the best and there maybe even better ways.
HRESULT hr = CoInitialize(NULL);
IMMDeviceEnumerator *pEnum = NULL;
hr = CoCreateInstance(__uuidof(MMDeviceEnumerator), NULL, CLSCTX_ALL, __uuidof(IMMDeviceEnumerator), (void**)&pEnum);
if(SUCCEEDED(hr))
{
IMMDeviceCollection *pDevices;
// Enumerate the output devices.
hr = pEnum->EnumAudioEndpoints(eAll, DEVICE_STATE_ACTIVE, &pDevices);
// You can choose between eAll, eCapture or eRender
}
With that you'd be able to distinguish between input (capture) and output (render).
(That's what you wanted right?)
The code is taken from this article. You may look at it for the correct API calls and libraries, it even might give you some more information.
Hope that's helpfull.