ATL structures break on COM objects from dll - c++

So, we are currently upgrading an ancient program from visual studio 2008 (where everything works) to 2017. We use a dll, whose classes the application connects to through ATL and a project-dependency.
Unfortunately all attempts at calling functions from these classes return exceptions, presumably because it can't find them.
The dll's classes are successfully added to the registry through regedit when built, and the uuids correspond correctly with the registered values. The classes can also be found in the OLE/COM-viewer.
It also only breaks when attempting to call from one of our classes. An attempt to call functions from IDispatch, which the classes in question inherits from, worked correctly.
In the below code the first attempt 'm_pRenderer' throws the exception. The second attempt 'test2' does not enter it's if-statement as CoCreateInstance returns a bad variable type-error
HRESULT res = CoCreateInstance(__uuidof(CBSNullRenderer), NULL, CLSCTX_INPROC_SERVER, __uuidof(IDispatch), (void**)&m_pRenderer);
Log("\nCreateInstance: %ld", res);
ICBSNullRendererPtr test2 = NULL;
HRESULT res2 = CoCreateInstance(__uuidof(CBSNullRenderer), NULL, CLSCTX_INPROC_SERVER, __uuidof(ICBSNullRenderer), (void**)&test2);
Log("\nres2: %ld", res2);
wireHWND mainHwnd = (wireHWND)GetParent(p_hWnd);
if(mainHwnd == NULL)
Log("\nWARNING mainHWND is NULL!");
try {
if (test2)
{
Log("\nDid create NullRenderer!");
test2->SetMainWnd(mainHwnd);
}
if (m_pRenderer)
{
Log("\nDid create NullRenderer!");
m_pRenderer->SetMainWnd(mainHwnd);
}
}
catch (...)
{
}
Which breaks on line 2 for m_pRenderer for:
inline HRESULT ICBSNullRenderer::SetMainWnd ( wireHWND hwnd ) {
HRESULT _hr = raw_SetMainWnd(hwnd);
if (FAILED(_hr)) _com_issue_errorex(_hr, this, __uuidof(this));
return _hr;
}
In the dll's .tli file.
The error creates an "Exception Thrown" dialog with the following:
Exception thrown at 0x00007FF9153ED7F2 (oleaut32.dll) in
AnimgramPro.exe: 0xC0000005: Access violation executing location
0x00007FF9153ED7F2
We also attempted to use QueryInterface on the m_pRenderer with the nullrenderer's uuid. This ended in another bad variable type-error.
Any advice or information of the errors which could be associated would be greatly appreciated.

So, I solved the problem. Apparently properly calling a dll-function in my program requires that 'Common Language Runtime Support' is turned off and that 'Whole Program Optimization' is set to 'Use Link Time Code Generation'

Related

When is calling CoInitialize required for a Windows console application

The code below, derived from https://learn.microsoft.com/en-us/windows/desktop/shell/folder-info#determining-an-objects-parent-folder, works as expected when compiled and run via Visual Studios 2017:
#include "stdafx.h"
#include <shlobj.h>
#include <shlwapi.h>
#include <objbase.h>
#pragma comment(lib, "shlwapi")
int main()
{
IShellFolder *psfParent = NULL;
LPITEMIDLIST pidlSystem = NULL;
LPCITEMIDLIST pidlRelative = NULL;
STRRET strDispName;
TCHAR szDisplayName[MAX_PATH];
HRESULT hr;
hr = SHGetFolderLocation(NULL, CSIDL_SYSTEM, NULL, NULL, &pidlSystem);
hr = SHBindToParent(pidlSystem, IID_IShellFolder, (void **)&psfParent, &pidlRelative);
if (SUCCEEDED(hr))
{
hr = psfParent->GetDisplayNameOf(pidlRelative, SHGDN_NORMAL, &strDispName);
hr = StrRetToBuf(&strDispName, pidlSystem, szDisplayName, sizeof(szDisplayName));
_tprintf(_T("%s\n"), szDisplayName);
}
psfParent->Release();
CoTaskMemFree(pidlSystem);
Sleep(5000);
return 0;
}
If I replace CSIDL_SYSTEM with CSIDL_MYDOCUMENTS, though, the GetDisplayNameOf method call fails with:
onecore\com\combase\objact\objact.cxx(812)\combase.dll!74EA3270: (caller: 74EA201B) ReturnHr(1) tid(d4c) 800401F0 CoInitialize has not been called.
onecoreuap\shell\windows.storage\regfldr.cpp(1260)\windows.storage.dll!76FE4FA3: (caller: 76E9F7EE) ReturnHr(1) tid(d4c) 80040111 ClassFactory cannot supply requested class
Adding CoInitialize(NULL); before the call to SHGetFolderLocation fixes the issue.
Why is calling CoInitialize required in one case but not the other?
Also, it seems like CoInitialize should always be called, but it's interesting that the sample code doesn't call it. I'm curious why this is the case. I couldn't get the sample code compiling as is - <iostream.h> couldn't be found, which is why I replaced the cout printing code with a call to _tprintf... Maybe that's an indication of the problem? Does the C++ runtime call CoInitialize for you, and maybe VS is trying to build a C application for me or something (like how on Linux, compiling with gcc and g++ has different implications).
As a rule, you should initialize COM/OLE before creating shell COM objects that inherit from IUnknown, use drag & drop etc. This also applies to functions that might use COM internally which could in theory be most of the SH* functions in shell32 and shlwapi.
Why did it work with CSIDL_SYSTEM?
The Windows 95 shell could run without loading COM/OLE. To do this it provided its own mini-COM implementation. Shell extensions could mark themselves as not requiring real COM and things implemented inside shell32 would call a special CoCreateInstance that tries to load things directly from shell32. This was to avoid loading ole32.dll because it is a very big file to load on a Intel 386 machine with 4 MiB of RAM (Windows 95 minimum requirements).
The IShellFolder implementation that deals with the filesystem is implemented in shell32 and does not require COM and is therefore able to handle a path like c:\Windows\system32.
CSIDL_MYDOCUMENTS however, is not a normal folder, it is a namespace extension and parts of its implementation is in mydocs.dll. And as you found out, parts of it does require COM.
All of this is of course a implementation detail and you should never assume that any of this is going to work without initializing COM.
SHGetFolderLocation may delegate execution to an extension that requires COM initialization. Although the documentation does not explicitly say so, you can find a remark about that for ShellExecute which is part of the same module (shell32.dll).
Because ShellExecute can delegate execution to Shell extensions (data
sources, context menu handlers, verb implementations) that are
activated using Component Object Model (COM), COM should be
initialized before ShellExecute is called. Some Shell extensions
require the COM single-threaded apartment (STA) type. In that case,
COM should be initialized as shown here:
CoInitializeEx(NULL, COINIT_APARTMENTTHREADED | COINIT_DISABLE_OLE1DDE)
There are certainly instances where ShellExecute does not use one of
these types of Shell extension and those instances would not require
COM to be initialized at all. Nonetheless, it is good practice to
always initalize COM before using this function.
You can use the following helper class to automatically initialize the COM library on the current thread.
class COMRuntime
{
public:
COMRuntime() {
::CoInitializeEx(nullptr, COINIT_APARTMENTTHREADED | COINIT_DISABLE_OLE1DDE);
}
~COMRuntime() {
::CoUninitialize();
}
};
Then just declare one instance of that class:
int main()
{
COMRuntime com;
// the rest of your code
}

How can I create a JavaScript object in Internet Explorer from C++

I'm using an external C++ application to control Internet Explorer (version is 11, but that seems unimportant for this example). Part of this control is being able to run arbitrary JavaScript in the browser's context. To enable that control, I'd like to be able to create a JavaScript object (or array, or function, etc.), represented by an appropriate IDispatch variant, that I can use in subsequent script calls. There are some examples online, including some from Microsoft, that would indicate such a thing should be possible, by finding and invoking the Object constructor.
Below is some sample code, cribbed from the aforementioned examples, that should do the trick. However, when I execute the code, I get an E_INVALIDARG ("One or more arguments are invalid") HRESULT returned. What am I doing wrong, and how do I fix the issue?
// Example assumes that you're using an ATL project which give access to
// the "CCom" wrapper classes, and that you have the ability to retrieve
// an IHTMLDocument2 object pointer.
int CreateJavaScriptObject(IHTMLDocument2* script_engine_host, CComVariant* created_object) {
// NOTE: Proper return code checking and error handling
// has been omitted for brevity
int status_code = 0;
CComPtr<IDispatch> script_dispatch;
HRESULT hr = script_engine_host->get_Script(&script_dispatch);
CComPtr<IDispatchEx> script_engine;
hr = script_dispatch->QueryInterface<IDispatchEx>(&script_engine);
// Create the variables we need
DISPPARAMS no_arguments_dispatch_params = { NULL, NULL, 0, 0 };
CComVariant created_javascript_object;
DISPID dispatch_id;
// Find the javascript object using the IDispatchEx of the script engine
std::wstring item_type = L"Object";
CComBSTR name(item_type.c_str());
hr = script_engine->GetDispID(name, 0, &dispatch_id);
// Create the jscript object by calling its constructor
// The below InvokeEx call returns E_INVALIDARG in this case
hr = script_engine->InvokeEx(dispatch_id,
LOCALE_USER_DEFAULT,
DISPATCH_CONSTRUCT,
&no_arguments_dispatch_params,
&created_javascript_object,
NULL,
NULL);
*created_object = created_javascript_object;
return status_code;
}
Potential answerers desiring a fully compilable solution can find a version of the above code in a GitHub repo. The repo includes a Visual Studio 2017 solution that creates a console application which will reproduce the issue, including launching Internet Explorer and obtaining a reference to the required IHTMLDocument2 object. Note that the version in the repo has somewhat more complete error handling than the above sample, in which it was omitted for brevity.

_com_ptr_t CreateInstance returns REGDB_E_CLASSNOTREG

I have an ATL project where I need to perform various initialization routines inside CComObjectRootEx::FinalConstruct. For demonstration purposes, consider the following implementation:
HRESULT FinalConstruct()
{
return m_bInit ? S_OK : E_FAIL;
}
This should return the appropriate HRESULT to the caller, indicating whether the initialization of the object was successful.
However, the client always receives REGDB_E_CLASSNOTREG instead of E_FAIL in case of failure, when trying to create the server:
#include <Windows.h>
#import <finalconstructtest.dll>
int main()
{
HRESULT hr = CoInitialize(0);
{
finalconstructtestLib::IFCClassPtr p;
// Returns REGDB_E_CLASSNOTREG
hr = p.CreateInstance(__uuidof(finalconstructtestLib::FCClass));
}
CoUninitialize();
return 0;
}
When I change the class context to CLSCTX_INPROC_SERVER however, the intended HRESULT is returned correctly:
// Returns E_FAIL
hr = p.CreateInstance(__uuidof(finalconstructtestLib::FCClass), nullptr, CLSCTX_INPROC_SERVER);
I've seen this post, where a similar behavior could be observed. However, I can't seem to find any reason why the class context affects the return value of FinalConstruct. Is this intended and perhaps documented somewhere?
CoCreateInstance API is not promising to forward the internal failure code to the caller. The implementation prefers to indicate the the source of the problem its own way, by returning REGDB_E_CLASSNOTREG meaning that it was unable to complete instantiation. Which is in turn true: it indeed was unable to create an instance (and the problem is not, for instance, related to missing interface, marshaling, permissions etc.). That is, the API prefers to suppress internal failure code to replace it with a documented one.
If you prefer to indicate a specific instantiate failure, you are better off with succeeding with initial creating an instance, and then you can implement an interface with a property or method exposing status, or giving relevant HRESULT error (with or without IErrorInfo support etc).

C++ Variables for wix

Im currently working with Visual Studio 2010 C++ Custom Action Project
I have a custom action like this:
extern "C" UINT __stdcall RegProductName(MSIHANDLE hInstall)
{
HRESULT hr = S_OK;
UINT er = ERROR_SUCCESS;
char szLocalPath[MAX_PATH];
hr = WcaInitialize(hInstall, "RegProductName");
ExitOnFailure(hr, "Failed to initialize");
WcaLog(LOGMSG_STANDARD, "Initialized.");
strcpy(szLocalPath, Orc_Get_Product_Name());
MsiSetProperty(hInstall, "ProductName", szLocalPath);
LExit:
er = SUCCEEDED(hr) ? ERROR_SUCCESS : ERROR_INSTALL_FAILURE;
return WcaFinalize(er);
}
It does not throw up any errors but I'm not sure if i'm doing what I want correctly.
In this line:
strcpy(szLocalPath, Orc_Get_Product_Name());
I'm getting the Product name from the function and copying it to szLocalPath. However I want to use this variable many times in other functions and also in my wix project.
Is this the correct way?
MsiSetProperty(hInstall, "ProductName", szLocalPath);
Many thanks
I can see no issue here but potential buffer overflow in strcpy. If your function Orc_Get_Product_Name() returns path longer than MAX_PATH. However, you'd better use TCHAR type instead of plain char to support Unicode/ANSI builds, and you should really build with Unicode enabled.
And I'm sure you want to assign the result of MsiSetProperty to hr variable.
Additionally, it looks strange that you copy Product Name into variable called Local Path.
And I can't see where the label LExit is used.
The function ExitOnFailure does not seem to return control: if WcaInitialize was unsuccessful, you should simply return ERROR_INSTALL_FAILURE or another error code back to MSI engine (rather than terminate the process where your custom action is running).

COM: how to get more details about COM errors?

Greets,
When working with DirectX, you get this nice header to #include called DxErr9.h which has really helpful functions like:
DXGetErrorString9
and
DXGetErrorDescription9
They tell you everything you need to know about the error given the HR.
But now working with COM and OLE, I find I'm kind of on my own with the HRESULTS that come back from COM functions. Is it really just me and MSDN at this point, or are there similar helper functions in OLE DB that I just haven't come across yet?
Additionally, you should look at the error info. Part of the COM system is the concept of the error information, which is a per-thread global which can be set and cleared at various times. You query for it in response to an error, and if it is set, it will have more useful information than just looking at the HRESULT.
HRESULT hr=something();
if (FAILED(hr))
{
CComPtr<IErrorInfo> err;
::GetErrorInfo(0, &err);
if (err)
{
CComBSTR description;
err->GetDescription(&description);
// description will be a more descriptive error message than just formatting the
// HRESULT because it is set by the COM server code at the point of the error
}
}
Use _com_error to get a meaningful string:
#include <comdef.h>
HRESULT hr = SomeComFunc();
if ( FAILED(hr) )
{
_com_error err(hr);
LPTCSTR szErrMsg = err.ErrorMessage();
// log szErrMsg or whatever
}