Implementing frame step on CBaseRenderer - c++

For Video mixing renderer, I have
//IFilterGraph * m_FilterGraph
IVideoFrameStep * ivfs=0;
HRESULT hr = m_FilterGraph->QueryInterface(IID_IVideoFrameStep, (void**)&ivfs );
if ( SUCCEEDED(hr) )
{
ivfs->Step( 1, 0 );
}
SAFE_RELEASE( ivfs );
and it works, but I cannot make the same using a derived class of baseclasses CBaseRenderer. Reading the reference pages, I tried to implement IKsPropertySet interface on the renderer, but it is never queried so that's about it...
So how to enable frame stepping on a custom renderer?
I'm also curious to know how to implement the actual stepping.
For a dirty quick solution, due to the fact that I seem to own a direct pointer to the renderer, I just call
//CBaseRenderer * m_pRenderer
m_pRenderer->BeginFlush();
m_pRenderer->EndFlush();
from the application thread. It works until I switch from stepping state (=paused) to playing state. Essentially, it never breaks down, but there seems to be a lag equal to the duration of the stepping state. Obviously it is not correct.

Ok, I got it after I found EC_STEP_COMPLETE notification. So this'll do it.
STDMETHODIMP CSteppingBaseRenderer::NonDelegatingQueryInterface(const IID &riid, void **ppv){
if (!ppv)
return E_POINTER;
if ( riid == IID_IKsPropertySet ){
return GetInterface( (IKsPropertySet*)this, ppv );
}
return CBaseRenderer::NonDelegatingQueryInterface(riid,ppv);
}
STDMETHODIMP CSteppingBaseRenderer::Set(
REFGUID guidPropSet,
DWORD dwPropID,
LPVOID pInstanceData,
DWORD cbInstanceData,
LPVOID pPropData,
DWORD cbPropData)
{
if ( guidPropSet == AM_KSPROPSETID_FrameStep )
{
if (dwPropID == AM_PROPERTY_FRAMESTEP_STEP )
{
m_Stepping = 1;
return S_OK;
}else if ( dwPropID == AM_PROPERTY_FRAMESTEP_CANSTEP )
{
return S_OK;
}
}
return E_PROP_SET_UNSUPPORTED;
}
HRESULT CSteppingBaseRenderer::DoRenderSample(IMediaSample *pMediaSample){
//...
if ( m_Stepping && !(--m_Stepping) )
{
this->NotifyEvent( EC_STEP_COMPLETE, 0, 0 );
}
return NOERROR;
}

Related

Console app vs Win32 app - DirectSound capture-device enumeration gives different results

I am looking for a reliable method to map a DirectShow capture device GUID to its corresponding waveID value.
I found the following project by Chris_P:
The solution works great, and it relies on an a rather obscure IKsPropertySet interface to retrieve the mapping.
Unfortunately, if I attempt the same technique from a C++/CLI library, the code fails with E_NOTIMPL (this behavior has been described on this question, - see the answer by Vladimir Hmelyoff)
So, I figured that I could write a simple console-based auxiliary app to retrieve the mappings and print them. My library could then launch this auxiliary app and parse the redirected output to obtain the mappings.
The console program runs fine, however, the GUIDs that are being passed to the enumeration callback are completely different to the ones passed by Chris_P's solution.
In fact they all share the same basic structure:
lpGuid = 0x02ad0808 {BDF35A00-B9AC-11D0-A619-00AA00A7C000}
The only variation occurs in the last digits of the GUID, where coincidentally, they match the mapped waveId value.
Another weird thing is that the capture device description is truncated to 31 characters, as if the enumeration was being performed using WaveIn APIs!
It would almost seem that some DirectSound facade is now wrapping the WaveIn API.
Any pointers on what could be happening?, Can I disable this behavior and enumerate the same GUIDS that the WIN32 app is enumerating?
Here is the code for the console application:
#include "stdafx.h"
#include <mmreg.h>
#include <initguid.h>
#include <Dsound.h>
#include <dsconf.h>
static BOOL CALLBACK DSEnumCallback(
LPGUID lpGuid,
LPCTSTR lpcstrDescription,
LPCTSTR lpcstrModule,
LPVOID lpContext
);
static BOOL GetInfoFromDSoundGUID(GUID i_sGUID, DWORD &dwWaveID);
static HRESULT DirectSoundPrivateCreate(OUT LPKSPROPERTYSET * ppKsPropertySet);
typedef WINUSERAPI HRESULT(WINAPI *LPFNDLLGETCLASSOBJECT) (const CLSID &, const IID &, void **);
BOOL GetInfoFromDSoundGUID(GUID i_sGUID, DWORD &dwWaveID) {
LPKSPROPERTYSET pKsPropertySet = NULL;
HRESULT hr;
BOOL retval = FALSE;
PDSPROPERTY_DIRECTSOUNDDEVICE_DESCRIPTION_DATA psDirectSoundDeviceDescription = NULL;
DSPROPERTY_DIRECTSOUNDDEVICE_DESCRIPTION_DATA sDirectSoundDeviceDescription;
memset(&sDirectSoundDeviceDescription, 0, sizeof(sDirectSoundDeviceDescription));
hr = DirectSoundPrivateCreate(&pKsPropertySet);
if( SUCCEEDED(hr) ) {
ULONG ulBytesReturned = 0;
sDirectSoundDeviceDescription.DeviceId = i_sGUID;
// On the first call the final size is unknown so pass the size of the struct in order to receive
// "Type" and "DataFlow" values, ulBytesReturned will be populated with bytes required for struct+strings.
hr = pKsPropertySet->Get(DSPROPSETID_DirectSoundDevice,
DSPROPERTY_DIRECTSOUNDDEVICE_DESCRIPTION,
NULL,
0,
&sDirectSoundDeviceDescription,
sizeof(sDirectSoundDeviceDescription),
&ulBytesReturned
);
if( ulBytesReturned ) {
// On the first call it notifies us of the required amount of memory in order to receive the strings.
// Allocate the required memory, the strings will be pointed to the memory space directly after the struct.
psDirectSoundDeviceDescription = (PDSPROPERTY_DIRECTSOUNDDEVICE_DESCRIPTION_DATA)new BYTE[ulBytesReturned];
*psDirectSoundDeviceDescription = sDirectSoundDeviceDescription;
hr = pKsPropertySet->Get(DSPROPSETID_DirectSoundDevice,
DSPROPERTY_DIRECTSOUNDDEVICE_DESCRIPTION,
NULL,
0,
psDirectSoundDeviceDescription,
ulBytesReturned,
&ulBytesReturned
);
dwWaveID = psDirectSoundDeviceDescription->WaveDeviceId;
delete[] psDirectSoundDeviceDescription;
retval = TRUE;
}
pKsPropertySet->Release();
}
return retval;
}
HRESULT DirectSoundPrivateCreate(OUT LPKSPROPERTYSET * ppKsPropertySet) {
HMODULE hLibDsound = NULL;
LPFNDLLGETCLASSOBJECT pfnDllGetClassObject = NULL;
LPCLASSFACTORY pClassFactory = NULL;
LPKSPROPERTYSET pKsPropertySet = NULL;
HRESULT hr = DS_OK;
// Load dsound.dll
hLibDsound = LoadLibrary(TEXT("dsound.dll"));
if( !hLibDsound ) {
hr = DSERR_GENERIC;
}
// Find DllGetClassObject
if( SUCCEEDED(hr) ) {
pfnDllGetClassObject =
(LPFNDLLGETCLASSOBJECT)GetProcAddress(hLibDsound, "DllGetClassObject");
if( !pfnDllGetClassObject ) {
hr = DSERR_GENERIC;
}
}
// Create a class factory object
if( SUCCEEDED(hr) ) {
hr = pfnDllGetClassObject(CLSID_DirectSoundPrivate, IID_IClassFactory, (LPVOID *)&pClassFactory);
}
// Create the DirectSoundPrivate object and query for an IKsPropertySet
// interface
if( SUCCEEDED(hr) ) {
hr = pClassFactory->CreateInstance(NULL, IID_IKsPropertySet, (LPVOID *)&pKsPropertySet);
}
// Release the class factory
if( pClassFactory ) {
pClassFactory->Release();
}
// Handle final success or failure
if( SUCCEEDED(hr) ) {
*ppKsPropertySet = pKsPropertySet;
} else if( pKsPropertySet ) {
pKsPropertySet->Release();
}
FreeLibrary(hLibDsound);
return hr;
}
BOOL CALLBACK DSEnumCallback(
LPGUID lpGuid,
LPCTSTR lpcstrDescription,
LPCTSTR lpcstrModule,
LPVOID lpContext
) {
LPWSTR psz = NULL;
StringFromCLSID(*lpGuid, &psz);
DWORD WaveID = 0xFFFFFFFF;
if( lpGuid ) {
GUID i_guid = *lpGuid;
GetInfoFromDSoundGUID(i_guid, WaveID);
}
if( WaveID != 0xFFFFFFFF )
wprintf(_T("%d %s\r\n"), WaveID, psz);
if( psz ) {
CoTaskMemFree(psz);
}
return TRUE;
}
int main()
{
DirectSoundCaptureEnumerate(DSEnumCallback, NULL);
Sleep(10000);
return 0;
}
It turns out I was not initializing COM.
I added the following snippet at the beginning of my main() procedure and the program retrieved the expected GUIDs:
HRESULT hr = NULL;
hr = CoInitialize(NULL);
if( FAILED(hr) ) {
printf("Failed to initialize COM");
return -1;
}
So I guess that if COM is not initialized, the DirectSound engine falls back to the WaveIn API (creating a DirectShow facade around it).

ListView accessibility issue -- how to make Windows Narrator read the entire selected row, including subitems?

I'm trying to add accessibility support to the WC_LISTVIEW control in my Win32/MFC application. I'm using Windows Narrator tool in Windows 10 to test the results. And by default it only reads the main item name of a selected row. For instance, in this case:
it will read only the country, when I need it to read the whole line.
So I found that I can set up a Server annotation for the list-view control using this example.
I would first set it up as such:
CAccPropServer_ListView* pMyPropSrv = NULL;
HRESULT hr;
CComPtr<IAccPropServices> pAccPropSvc = NULL;
hr = ::CoCreateInstance(CLSID_AccPropServices, NULL, CLSCTX_SERVER, IID_IAccPropServices, (void **)&pAccPropSvc);
if(SUCCEEDED(hr) &&
pAccPropSvc )
{
pMyPropSrv = new (std::nothrow) CAccPropServer_ListView( pAccPropSvc );
if( pMyPropSrv )
{
MSAAPROPID propids[] = {
PROPID_ACC_NAME,
};
hr = pAccPropSvc->SetHwndPropServer( hWndListCtrl, OBJID_CLIENT,
CHILDID_SELF, propids, 1, pMyPropSrv, ANNO_CONTAINER);
pMyPropSrv->Release();
}
}
where the CAccPropServer_ListView class does all the work:
class CAccPropServer_ListView: public IAccPropServer
{
ULONG m_Ref;
IAccPropServices * m_pAccPropSvc;
public:
CAccPropServer_ListView( IAccPropServices * pAccPropSvc )
: m_Ref( 1 ),
m_pAccPropSvc( pAccPropSvc )
{
m_pAccPropSvc->AddRef();
}
~CAccPropServer_ListView()
{
m_pAccPropSvc->Release();
}
/* TODO: Addref/Release/QI go here...
Skipped them for brevity...
*/
HRESULT STDMETHODCALLTYPE GetPropValue (const BYTE * pIDString,
DWORD dwIDStringLen, MSAAPROPID idProp, VARIANT * pvarValue,
BOOL * pfGotProp )
{
if(!pfGotProp)
return E_POINTER;
pvarValue->vt = VT_EMPTY;
*pfGotProp = FALSE;
HWND hwnd;
DWORD idObject;
DWORD idChild;
if( S_OK != m_pAccPropSvc->DecomposeHwndIdentityString( pIDString,
dwIDStringLen, &hwnd, &idObject, &idChild ) )
{
return S_OK;
}
if( idChild != CHILDID_SELF )
{
if( idProp == PROPID_ACC_NAME )
{
CString str;
str.Format(L"Line index %d", idChild);
BSTR bstr = ::SysAllocString((LPCTSTR)str.GetString());
pvarValue->vt = VT_BSTR;
pvarValue->bstrVal = bstr;
*pfGotProp = TRUE;
}
}
return S_OK;
}
};
So my question is concerning GetPropValue method above that actually generates the text prompt for Narrator to read out loud.
How do I get an index of a row read by the Narrator from the idChild that is returned by DecomposeHwndIdentityString?
In my example above, purely experimentally, I was getting the following values:
"Line index 17"
"Line index 33"
"Line index 49"
"Line index 65"
and so on
which would translate to 0x11, 0x21, 0x31, 0x41 that are not row indexes. Are those IDs documented anywhere for a SysListView32?

Obtaining the Excel.Application IDispatch* within a dll that's been loaded into Excel

Does anyone know how to get hold of the Excel.Application IDispatch* pointer associated with an excel process into which an dll has been loaded?
A key thing here is that the process is excel.exe, and the pointer I need must belong to that process. Using the Running Object Table will not fly since Excel only registers its first instance with that.
I'm hoping there is some low-level COM trickery, but I'm not an expert in that field.
EDITED II Code is under the WTFPL license version 2.
EDITED: Add PID parameter to allow filtering when several Excel processes are currently running, as per comment suggestion from #EricBrown.
I managed to get a working IDispatch* to an Excel "Application" object without using the ROT. The trick is to use MSAA. My code works as a stand alone console application, but I think that if the code is executed in an Excel process, via DLL Injection, it MAY works fine. You may have to be in a dedicated thread. Let me know if you want me to push the expriment to the DLL injection level.
Tested OK on Window7 64b, with a UNICODE builds (32 bits and 64 bits).
Excel version 2010 64 bits (version "14")
I get the IDispatch via the "application" property from an "Worksheet" object. Consequence: there must be an opened worksheet. In order to find the good MSSA Window, I need the class name of the Top Level Excel Frame Window. In Excel 2010, it's "XLMAIN". The class name for worksheets is "EXCEL7" and that seems to be a "standard".
I was not able to directly get a working IDispatch* from the main Excel Window, but have not tried very hard. That may involve #import with a automation DLL from Excel, in order to QueryInterface the IDispatch that MSAA gives for the main Window (that IDispatch is NOT for an Application object)
#include <atlbase.h>
#pragma comment( lib, "Oleacc.lib" )
HRESULT GetExcelAppDispatch( CComPtr<IDispatch> & spIDispatchExcelApp, DWORD dwExcelPID ) {
struct ew {
struct ep {
_TCHAR* pszClassName;
DWORD dwPID;
HWND hWnd;
};
static BOOL CALLBACK ewp( HWND hWnd, LPARAM lParam ) {
TCHAR szClassName[ 64 ];
if ( GetClassName( hWnd, szClassName, 64 ) ) {
ep* pep = reinterpret_cast<ep*>( lParam );
if ( _tcscmp( szClassName, pep->pszClassName ) == 0 ) {
if ( pep->dwPID == 0 ) {
pep->hWnd = hWnd;
return FALSE;
} else {
DWORD dwPID;
if ( GetWindowThreadProcessId( hWnd, &dwPID ) ) {
if ( dwPID == pep->dwPID ) {
pep->hWnd = hWnd;
return FALSE;
}
}
}
}
}
return TRUE;
}
};
ew::ep ep;
ep.pszClassName = _TEXT( "XLMAIN" );
ep.dwPID = dwExcelPID;
ep.hWnd = NULL;
EnumWindows( ew::ewp, reinterpret_cast<LPARAM>( &ep ) );
HWND hWndExcel = ep.hWnd;
if ( ep.hWnd == NULL ) {
printf( "Can't Find Main Excel Window with EnumWindows\n" );
return -1;
}
ep.pszClassName = _TEXT( "EXCEL7" );
ep.dwPID = 0;
ep.hWnd = NULL;
EnumChildWindows( hWndExcel, ew::ewp, reinterpret_cast<LPARAM>( &ep ) );
HWND hWndWorkSheet = ep.hWnd;
if ( hWndWorkSheet == NULL ) {
printf( "Can't Find a WorkSheet with EnumChildWindows\n" );
return -1;
}
CComPtr<IDispatch> spIDispatchWorkSheet;
HRESULT hr = AccessibleObjectFromWindow( hWndWorkSheet, OBJID_NATIVEOM, IID_IDispatch,
reinterpret_cast<void**>( &spIDispatchWorkSheet ) );
if ( FAILED( hr ) || ( spIDispatchWorkSheet == 0 ) ) {
printf( "AccessibleObjectFromWindow Failed\n" );
return hr;
}
CComVariant vExcelApp;
hr = spIDispatchWorkSheet.GetPropertyByName( CComBSTR( "Application" ), &vExcelApp );
if ( SUCCEEDED( hr ) && ( vExcelApp.vt == VT_DISPATCH ) ) {
spIDispatchExcelApp = vExcelApp.pdispVal;
return S_OK;
}
return hr;
}
int _tmain(int argc, _TCHAR* argv[])
{
DWORD dwExcelPID = 0;
if ( argc > 1 ) dwExcelPID = _ttol( argv[ 1 ] );
HRESULT hr = CoInitialize( NULL );
bool bCoUnInitializeTodo = false;
if ( SUCCEEDED( hr ) ) {
bCoUnInitializeTodo = true;
CComPtr<IDispatch> spDispatchExcelApp;
hr = GetExcelAppDispatch( spDispatchExcelApp, dwExcelPID );
if ( SUCCEEDED( hr ) && spDispatchExcelApp ) {
CComVariant vExcelVer;
hr = spDispatchExcelApp.GetPropertyByName( CComBSTR( "Version" ), &vExcelVer );
if ( SUCCEEDED( hr ) && ( vExcelVer.vt == VT_BSTR ) ) {
wprintf( L"Excel Version is %s\n", vExcelVer.bstrVal );
}
}
}
if ( bCoUnInitializeTodo ) CoUninitialize();
return 0;
}
You should be able to find out how to do this by reviewing the code in ExcelDNA. This project contains code that hooks back into Excel from the extension library. The code is likely to be more elaborate that you need, but will implement the reference you require.
This is how I do it: (acknowledge #manuell). dispatch_wrapper is a class, here is the constructor to set m_disp_application:
dispatch_wrapper(void)
{
DWORD target_process_id = ::GetProcessId(::GetCurrentProcess());
if (getProcessName() == "excel.exe"){
HWND hwnd = ::FindWindowEx(0, 0, "XLMAIN", NULL);
while (hwnd){
DWORD process_id;
::GetWindowThreadProcessId(hwnd, &process_id);
if (process_id == target_process_id){
HWND hwnd_desk = ::FindWindowEx(hwnd, 0, "XLDESK", NULL);
HWND hwnd_7 = ::FindWindowEx(hwnd_desk, 0, "EXCEL7", NULL);
IDispatch* p = nullptr;
if (SUCCEEDED(::AccessibleObjectFromWindow(hwnd_7, OBJID_NATIVEOM, IID_IDispatch, (void**)&p))){
LPOLESTR name[1] = {L"Application"};
DISPID dispid;
if (SUCCEEDED(p->GetIDsOfNames(IID_NULL, name, 1U, LOCALE_SYSTEM_DEFAULT, &dispid))){
CComVariant v;
DISPPARAMS dp;
::memset(&dp, NULL, sizeof(DISPPARAMS));
EXCEPINFO ei;
::memset(&ei, NULL, sizeof(EXCEPINFO));
if (SUCCEEDED(p->Invoke(dispid, IID_NULL, LOCALE_SYSTEM_DEFAULT, DISPATCH_PROPERTYGET, &dp, &v, &ei, NULL))){
if (v.vt == VT_DISPATCH){
m_disp_application = v.pdispVal;
m_disp_application->AddRef();
return;
}
}
}
}
}
hwnd = ::FindWindowEx(0, hwnd, "XLMAIN", NULL);
}
}
m_disp_application = nullptr;
}
getProcessName() returns lower case.
Because Office applications register their documents in the ROT, you can attach to instances beside the first one (which is already in the ROT) by getting IDispatch for documents in the ROT, then you can use document.Application.hwnd (this is VBA, you need to translate to IDispatch::GetIDsOfNames and IDispatch::Invoke with DISPATCH_PROPERTYGET) to get the window handles of all Excel instances.
Now you have a mapping between IDispatch and Windows handles of all Excel instances, it is time to find your own Excel instance. You can call GetWindowThreadProcessId on the window handles to get the process ids, then compare to your own process id returned by GetCurrentProcessId to see which excel window belongs to your current process, and look up in the HWND to IDispatch mapping to find your current Excel application's IDispatch interface.

Not releasing filter com object causing a crash

I am using Viveks capture filter with vlc (http://tmhare.mvps.org/downloads/vcam.zip) to emulate a capture source. When the filter is open and I close vlc I get a crash. The stack trace indicates that 2 COM objects still exist (Im guessing the filter and pin) which should be released before a CoUninitialize call. My problem is I am not sure where to release the filter and pin COM objects, I have a destructor for the fitler and pin but they are never called when vlc closes. Someone who had a similar problem (Unreleased DirectShow CSource filter makes program crash at process shutdown).
This is the important registering part of the dll.
STDAPI RegisterFilters( BOOL bRegister )
{
HRESULT hr = NOERROR;
WCHAR achFileName[MAX_PATH];
char achTemp[MAX_PATH];
ASSERT(g_hInst != 0);
if( 0 == GetModuleFileNameA(g_hInst, achTemp, sizeof(achTemp)))
return AmHresultFromWin32(GetLastError());
MultiByteToWideChar(CP_ACP, 0L, achTemp, lstrlenA(achTemp) + 1,
achFileName, NUMELMS(achFileName));
hr = CoInitialize(0);
if(bRegister)
{
hr = AMovieSetupRegisterServer(CLSID_VirtualCam, L"Virtual Cam", achFileName, L"Both", L"InprocServer32");
}
if( SUCCEEDED(hr) )
{
IFilterMapper2 *fm = 0;
hr = CreateComObject( CLSID_FilterMapper2, IID_IFilterMapper2, fm );
if( SUCCEEDED(hr) )
{
if(bRegister)
{
IMoniker *pMoniker = 0;
REGFILTER2 rf2;
rf2.dwVersion = 1;
rf2.dwMerit = MERIT_DO_NOT_USE;
rf2.cPins = 1;
rf2.rgPins = &AMSPinVCam;
hr = fm->RegisterFilter(CLSID_VirtualCam, L"Virtual Cam", &pMoniker, &CLSID_VideoInputDeviceCategory, NULL, &rf2);
}
else
{
hr = fm->UnregisterFilter(&CLSID_VideoInputDeviceCategory, 0, CLSID_VirtualCam);
}
}
// release interface
//
if(fm)
fm->Release();
}
if( SUCCEEDED(hr) && !bRegister )
hr = AMovieSetupUnregisterServer( CLSID_VirtualCam );
CoFreeUnusedLibraries();
CoUninitialize();
return hr;
}
STDAPI DllRegisterServer()
{
return RegisterFilters(TRUE);
}
STDAPI DllUnregisterServer()
{
return RegisterFilters(FALSE);
}
extern "C" BOOL WINAPI DllEntryPoint(HINSTANCE, ULONG, LPVOID);
BOOL APIENTRY DllMain(HANDLE hModule, DWORD dwReason, LPVOID lpReserved)
{
return DllEntryPoint((HINSTANCE)(hModule), dwReason, lpReserved);
}
The important filter part
CVCamStream::CVCamStream(HRESULT *phr, CVCam *pParent, LPCWSTR pPinName) :
CSourceStream(NAME("Virtual Cam"),phr, pParent, pPinName), m_pParent(pParent)
{
// Set the default media type as 320x240x24#15
GetMediaType(4, &m_mt);
}
CVCamStream::~CVCamStream()
{
m_pParent->Release();
}
Leaked COM references are a sort of hard to nail down. Assuming that COM client - in your case VLC - is doing everything right (which might be not the case, but it's okay to start with this assumption), the problem is on your code. It is typically one of the two:
You are dealing with raw pointers and there is somewhere no matching Release for AddRef done earlier
There are circular references and object keep each other alive
As you already see you have two objects floating around, a good strategy would be to identify what classes are they exactly, and trace reference counter changes to see where is the lost IUnknown::Release.

MS Kinect FaceTracker creating IFTResult

I have a fairly simple application that contains the following:
context->mFaceTracker = FTCreateFaceTracker();
hr = context->mFaceTracker->Initialize( &mVideoCameraConfig, &mDepthCameraConfig, NULL, NULL );
which works fine and returns S_OK and mFaceTracker is (as far as I can tell) initialized properly. However, the next line is:
hr = context->mFaceTracker->CreateFTResult( &context->mFTResult );
which always returns FT_ERROR_UNINITIALIZED, doesn't allocate the pointer, and has me puzzled. I've tried many different strategies for getting this to work from changing how the threading for the device and detector works, to changing my FTcontext object from a class to a struct to match the samples, all with no success. The Kinect SDK samples all work fine, but trying to use them in my own application doesn't seem to, despite my closely mirroring how they initialize the device and the Face Tracker. I'm curious if anyone else has run into this or similar problems around initializing either IFTFaceTracker or the IFTResult. Also, I'm curious how else I can test the IFTFaceTracker for correct intialization, other than testing the HRESULT that Initialize() returns. Thanks in advance,
=== edit
I've had a few request for more code. It's built on Cinder and is using this block for Cinder: https://github.com/BanTheRewind/Cinder-KinectSdk
I can't post all of the code, but I've posted at least most of the relevant Kinect initialization code here:
void Kinect::start( const DeviceOptions &deviceOptions )
{
if ( !mCapture ) {
// Copy device options
mDeviceOptions = deviceOptions;
string deviceId = mDeviceOptions.getDeviceId();
int32_t index = mDeviceOptions.getDeviceIndex();
// Clamp device index
if ( index >= 0 ) {
index = math<int32_t>::clamp( index, 0, math<int32_t>::max( getDeviceCount() - 1, 0 ) );
}
// Initialize device instance
long hr = S_OK;
if ( index >= 0 ) {
hr = NuiCreateSensorByIndex( index, &mSensor );
if ( FAILED( hr ) ) {
trace( "Unable to create device instance " + toString( index ) + ": " );
error( hr );
return;
}
} else if ( deviceId.length() > 0 ) {
_bstr_t id = deviceId.c_str();
hr = NuiCreateSensorById( id, &mSensor );
if ( FAILED( hr ) ) {
trace( "Unable to create device instance " + deviceId + ":" );
error( hr );
return;
}
} else {
trace( "Invalid device name or index." );
return;
}
// Check device
hr = mSensor != 0 ? mSensor->NuiStatus() : E_NUI_NOTCONNECTED;
if ( hr == E_NUI_NOTCONNECTED ) {
error( hr );
return;
}
// Get device name and index
if ( mSensor != 0 ) {
mDeviceOptions.setDeviceIndex( mSensor->NuiInstanceIndex() );
BSTR id = ::SysAllocString( mSensor->NuiDeviceConnectionId() );
_bstr_t idStr( id );
if ( idStr.length() > 0 ) {
std::string str( idStr );
mDeviceOptions.setDeviceId( str );
}
::SysFreeString( id );
} else {
index = -1;
deviceId = "";
}
flags |= NUI_INITIALIZE_FLAG_USES_COLOR;
}
hr = mSensor->NuiInitialize( flags );
if ( FAILED( hr ) ) {
trace( "Unable to initialize device " + mDeviceOptions.getDeviceId() + ":" );
error( hr );
return;
}
hr = mSensor->NuiSkeletonTrackingEnable( 0, flags );
if ( FAILED( hr ) ) {
trace( "Unable to initialize skeleton tracking for device " + mDeviceOptions.getDeviceId() + ": " );
error( hr );
return;
}
mIsSkeletonDevice = true;
mThread = CreateThread(NULL, 0, &Kinect::StaticThread, (PVOID)this, 0, 0);
}
}
DWORD WINAPI Kinect::StaticThread(PVOID lpParam)
{
Kinect* device = static_cast<Kinect*>(lpParam);
if (device)
{
return device->run();
}
return 0;
}
void run() {
if(mSensor) {
if(mEnabledFaceTracking)
{
if(mNeedFaceTracker) {
mFaceTracker = new FaceTracker(
mDeviceOptions.getVideoSize().x,
mDeviceOptions.getVideoSize().y,
mDeviceOptions.getDepthSize().x,
mDeviceOptions.getDepthSize().y,
1.0,
1 );
mNeedFaceTracker = false;
}
// make sure we have both color && depth buffers to work with
if(newDepth || newVideo)
{
FT_SENSOR_DATA sensorData(mFTColorImage, mFTDepthImage);
FT_VECTOR3D hint[2]; // this is initialized elsewhere
mFaceTracker->checkFaces( (NUI_SKELETON_FRAME*) &skeletonFrame, mFTColorImage, mFTDepthImage, 1.0, 0);
if(mFaceTracker->getNumFaces() > 0) {
cout << " we have a face " << mFaceTracker->getNumFaces() << endl;
mNewFaceTrackData = true;
mFaceData.clear();
for( int i = 0; i < mFaceTracker->getNumFaces(); i++) {
Face newFace;
mFaceTracker->getProjectedShape(0, newFace.scale, newFace.rotation, newFace.transform, newFace.screenPositions);
mFaceData.push_back(newFace);
}
}
}
}
Sleep( 8 );
}
}
It looks like you never call (or omitted including in the code sample) NuiImageStreamOpen(), such as this code snippet from the SingleFace sample, KinectSensor.cpp in the Init method:
hr = NuiImageStreamOpen(
colorType,
colorRes,
0,
2,
m_hNextVideoFrameEvent,
&m_pVideoStreamHandle );
if (FAILED(hr))
{
return hr;
}
hr = NuiImageStreamOpen(
depthType,
depthRes,
(bNearMode)? NUI_IMAGE_STREAM_FLAG_ENABLE_NEAR_MODE : 0,
2,
m_hNextDepthFrameEvent,
&m_pDepthStreamHandle );
Calling those before you call CreateFTResult() may fix the uninitialized error.
Additionally, you call CreateThread() and then call run(), but there is no while loop so that thread will exit almost immediately, certainly without enough time for the Kinect to start providing data to the FaceTracking.
It doesn't look like you have included the Thread or event loop that is checking the sensor for new data, updating mFTColorImage and mFTDepthImage, and setting the newDepth and newVideo flags. This could be in the same thread that you create above (provided you create a while loop, and ignoring performance or other classes needing the Kinect data), or could be a different thread as in the SingleFace Kinect SDK sample.