Related
I would like to understand the purposes of the files mentioned in this article and link the knowledge to my current COM server and COM client scenario, so that I can implement my COM server to use the COM server: this
I am having a COM server which is an exe, or service, that runs in the background. For now, I know there is an exposed interface inherited both from IUnknown and IDispatch. Besides I have the following files generated:
xxx_i.c defines all the CLSIDs and IIDs
xxx_i.h defines all the method the interface supports
xxx_p.c ?
dlldata.c ?
I am now using the automation way, IDispatch -> Invoke(), to access the interface methods. Although this way seems work fine without using any files mentioned above, I still would like to understand the purposes of them while using the normal way, IUnknown -> QueryInterface(), to access the methods.
Since I am new to the COM world, any suggested reading would be appreciated! Thanks!
In its most simple form, COM is only the vtable binary contract plus the mother of all interfaces: IUnknown. COM is a way to reuse code without source, with components, it's some kind of a dynamic casting mechanism. Provided I know the coclasses you support (their CLSID), the interfaces they expose (their IID), and what are these interfaces' methods layout, their parameters, order, type, etc., I can use your COM server.
But to ease "communication" between your COM clients and your COM server, you can/should use some standard mechanisms/documentation and add tooling so plumbing stuff like marshaling (=serialization) will be taken care w/o any effort. This is crucial in the out-of-process case, less important with in-process (I will elude the "apartment" concept here...)
So, lots of things you'll find in COM (like registration, tooling, IDL, typelibs, etc.) are in fact optional, but also very useful (so they kinda become mandatory in the end). The purpose of things like idl (for "interface language definition") is to define and expose to your COM clients what your COM server supports so tooling can generate a lot of code automatically for you and your clients (.c, .h, .tlb). Note that nothing prevents you from implementing interfaces or coclasses without defining them in idl. Nothing obliges you to provide your .idl or your .tlb. In this case, I will only be able to use them if I known their IID, method layout etc.
Then, on top of IUnknown, Microsoft created a universal interface called IDispatch (this is also known as "Automation", or "Late binding" as opposed to "Early binding" for IUnknown), at that time targeting VB/VBA clients (before even VBScript, JScript, and lots of other COM clients, .NET supports IUnknown and IDispatch). IDispatch, if you go that route, could be the last interface you'll ever have to implement, because its semantics allows full discovery and invocation of any method, provided it supports a finite set of defined data types, the "Automation types": BSTR, VARIANT, etc.
So, if you support IDispatch, provide a TLB (typelibs) and restrict all types to Automation types, then you don't need to handle marshaling, you don't need proxies and stubs, you can forget about all this, even in out-of-process scenarios, because Microsoft implements that automatically. Back in the days, we used to call "oleaut32.dll" the "universal marshaler".
Dual interfaces are interfaces that support both IUnknown and derivates and IDispatch at the same time. They mostly exist to support C/C++ clients and Automation clients at the same time. Using Automation (BSTR, VARIANT, etc.) is a bit painful in C/C++ because they were not intended originally to be used by C/C++ clients... Note Microsoft proposes C++ smart wrappers classes: CComBSTR and CComVARIANT with ATL, or _variant_t and _bstr_t with the Windows SDK.
Requests for reading material are out of scope for StackOverflow, but I can't help but to reccomend the seminal work by Don Box: Essential COM which is in print and available as an ebook elsewhere. Here's Don's description of the topic:
Box, Don. Essential COM. Addison-Wesley, 1998, pp. 350:
COM is based on client programs having a priori knowledge of an
interface's definition at development time. This is accomplished
either through C++ header files (for C++ clients) or through type
libraries (for Java and Visual Basic clients). In general, this is not
a problem, as programs written in these languages typically go through
some sort of compilation phase prior to being deployed. Some languages
do not go through such a compilation phase at development time and
instead are deployed in source code form to be interpreted at runtime.
Perhaps the most pervasive of such languages are HTML based scripting
languages (e.g., Visual Basic Script, JavaScript) that execute in the
context of either a Web browser or a Web server. In both of these
cases, script text is stored in its raw form embedded in an HTML file,
and the surrounding runtime executes the script text on the fly as the
HTML is parsed. To provide a rich programming environment, these
environments allow scripts to invoke methods on COM objects that may
be created in the script text itself or perhaps elsewhere in the HTML
stream (e.g., a control that is also part of the Web page). In these
environments, it is currently impossible to use type libraries or
other a priori means to provide the runtime engine with a description
of the interfaces being used. This means that the objects themselves
must assist the interpreter in translating the raw script text into
meaningful method invocations.
To allow objects to be used from
interpretive environments such as Visual Basic Script and JavaScript,
COM defines an interface that expresses the functionality of
interpretation.
Tl;dr: there are two ways to do everything in COM (ignoring IInspectable and dual interfaces):
IUnknownStandard virtual method invocation. Fast, no extra code. Requires compile time interface information (.h or .tlb) on the client calls
IDispatch"Late Binding". Slow, lots of interpreting code. No client compilation or interface spec needed.
Practically speaking, unless you are calling from VBA, VBScript or have some old VB6 clients then you are better off sticking with IUnknown exclusively.
Creating an ATL project in MSVC seems to create not one but two projects; the latter named the same as the former but with PS appended to its name. What is the purpose of this second project and how can I tell whether I need it?
COM supports making interface method calls across two different threads, two different processes or two different machines. This is called marshaling. Two different threads is the most common case, a COM server is often not thread-safe. COM implements thread-safety for such single-threaded coclasses by marshaling the call from the 'wrong' thread to the thread that created the server. Marshaling between processes occurs when you write an out-of-process server. Between different machines across a network is called DCOM.
This is implemented by creating an instance of the interface that looks exactly like the original. But all the methods of the interface are actually substitutes that do the job of the marshaling the call. This is the proxy. On the other end of the wire there's a substitute that looks exactly like the interface but does the opposite job. This is the stub. The proxy and stub work together to create the illusion that you're making a simple method call in your program.
The primary job of the proxy is to serialize the arguments of the method call into a memory buffer or network packet. This can be pretty untrivial, especially when you use pointers to variable-sized structures. COM needs help to get that right and that's the job of your FooPS project. When you run midl.exe on your .idl file, midl auto-generates code from the interface definitions to implement the proxy and the stub. This is quite often good enough but you may need to implement your own if the built-in keywords in IDL are not sufficient to describe your data.
Last but not least, Windows provides a standard marshaller that can marshal simple interfaces. Designed to support the sub-set of COM that's defined by COM Automation. In other words, interfaces that derive from IDispatch and only use Automation compatible types. You only need to get the registry entries right to enable it and don't otherwise need the proxy/stub generated by midl. And of course, if you only make simple in-process calls on one thread then you won't need it either. This is pretty common.
As #ebutusov said, *PS project contains implementations for Proxy and Stub. They are not standard, instead they are generated by MIDL for interfaces exported from your ATL server. These interfaces are declared in the *.IDL file. The ouput of the project is DLL. You may read this article to get more details.
You may remove PS project from the solution in case if you do not define any custom interfaces in you *.IDL file or if you define only interfaces which have dual and oleautomation modifiers. In that case a standard typelib marshaller will be used.
In order to be able to make use of the standard typelib marshaller, one has to register a typelibrary (which is done automatically since you are using ATL)
It's proxy/stub code, which contains non-standard data marshallers needed to transfer data between different apartments (threading related). It's used when application, which calls your COM object, uses different COM threading model. There was an option in ATL/COM wizard to merge this code into main library. In many common scenarios you don't have to worry about it (i.e. when your COM dll runs in the client context), unless you want to write a custom marshaller.
I have a VB6 COM DLL. I want to use it from C++. I know how to register it, generate a tlb file from the DLL, and #import it in C++.
I'd like however, to load and use DLLs like this dynamically, at runtime, without knowing them in advance. Is this possible?
Thanks,
Yes, but you need to get the question clearer.
Sometimes, you do know the COM interface upfront, just not the implementation. In that case, you can create a dummy implementation of the interface and #import that. At runtime, you'd still register the real component, get an object from it (via CoCreateInstance probably) and store that in an appropriate smart pointer.
With VB6, it's a bit less direct. This adds a level of indirection. Read up on IDispatch. You need to get that known interface to describe an unknown interface. That way, the unknown interface can be obtained at runtime.
IMHO, You need at least some common interface (so you known what to call in the C++ side).
I'd do something like:
Define a common interface (in its own DLL/TLB)
Implement this interface in one or more COM servers
Import this interface in the C++ side (let's call it client)
Define a way to pass the progid of the COM server you want to work with (load dynamically) in the client.
Hope this helps
Take a look at these two MSDN articles about Registration-Free Activation of COM Components:
Registration-Free Activation of COM Components: A Walkthrough
Escape DLL Hell: Simplify App Deployment with ClickOnce and Registration-Free COM
There also have been some similar question here on StackOverflow:
Generate manifest files for registration-free COM
Windows/C++: how to use a COM dll which is not registered
I am quite new to COM so the question may seem naive.
Q1. About Windows DLL
Based on my understanding, a Windows DLL can export functions, types(classes) and global variables. Is this understanding all right?
Q2. About COM
My naive understanding is that: a COM DLL seems to be just a new logical way to organize the functions and types exported by a standard Windows DLL. A COM DLL exports both functions such as DllRegisterServer() and DllGetClassObject(), and also the Classes which implements the IUnknown interface. Is this understanding all right?
Q3. *.def & *.idl
*.def is used to define the functions exported by a Windows DLL in the traditional way, such as DllGetClassObject().
*.idl is used to define the interface implemented by a COM coclass.
Thanks in advance.
Think of COM as a binary compatible way to share interfaces across DLL boundaries. C++ classes can't easily be exported from DLLs because of the non-standard name mangling done between various compiler versions. COM allows code in one DLL or executable, to create an implementation of an interface from another DLL or EXE as long as that implementation follows a defined interface and calling convention. This is what allows a COM class to be written in C# and invoked from C++, Python, and many other COM-aware languages.
COM interfaces are just standard C++ classes containing all pure virtual functions and derived from IUnknown. IUnknown is a pre-defined interface that all compliant COM interfaces must derive from that provides common facilities like reference counting and the ability to query whether an object implements a particular interface.
DLLs that wish to advertise the fact they can create implementations of COM interfaces do so by exporting 4 functions:
DllGetClassObject => Return a class factory of a requested interface
DllCanUnloadNow => Whether all instances handed out have since been released
DllRegisterServer => Register the types this DLL supplies in the registry
DllUnregisterServer => Unregister this DLL and its types from the registry
So to answer your questions:
Q1. About Windows DLL Based on my understanding, a Windows DLL can export functions,
types(classes) and global variables. Is this understanding all right?
DLLs can export functions and classes (not sure about global variables, but you don't want to be exporting them DLLs even if you can! :-) ) However, exported classes will be name mangled and hence only usable by other DLLs or EXEs that happen to have the same name mangling (not a good way to do business). Functions with C-style calling convention are not name mangled and hence can be exported and called from elsewhere without problems.
Q2. A COM DLL exports both functions such as DllRegisterServer() and DllGetClassObject(),
and also the Classes which implements the IUnknown interface. Is this understanding all
right?
Halfway, there are 4 functions to export to be a full COM compliant DLL (shown above). You can search MSDN for any of those names to see the full signatures fro them. Your implementation for DllGetClassObject is going to be the main one which outside parties will use. They can use that to get an IClassFactory to an interface your DLL provides, and then use that to create an instance.
COM is a large and complicated beast, but its foundations are fairly straightforward. You can also check out this link COM Intro for some more info. Good luck!
To add on to cpalmer's nice answer, there's no particular reason why you need to use the registry and the four recommended functions to export COM from your DLL.
You could use a registry-free COM, or COM-lite approach, where you simply export factory methods, such as
__declspec(dllexport) void MyDllCreateFoo(IFoo ** ppFoo);
DLL users would call your factory to create your class CMyFoo which implements IFoo. What DllRegisterServer et. al. do is allow CMyFoo and other classes to be looked up in the registry, among other things.
Q3: You are correct in fact, if not in spirit. .def files and .idl files are pretty different beasts. .def files are used by the linker only, and aren't even necessary - you can export all the functions you want using
__declspec(dllexport) void foo() {
}
inside your C++ code.
.idl files are used to generate C++ headers (.h files) which are included both by the DLL as well as its clients. It generates the interfaces plus some glue code that does things like parameter mashalling.
Again, you don't techincally need to use IDL to use COM, but it can make things more convenient.
The reason I'm breaking it down like this is to illustrate that COM is not a big monolithic thing, but rather a really small thing with a bunch of stuff built up around it that you could opt to use, or not, to taste.
You have it all exactly right so far. The only bit I would clarify is "the Classes which implements the IUnknown interface." A class is represented by an object that implements the IClassFactory interface. You can get such a class factory by calling DllGetClassObject in a DLL, and then you can ask the class factory to make an object of the class. This is similar to many other object-oriented systems: there is a variety of objects that represent classes and which can be used to manufacture instances.
Other useful facts about COM:
There is a small but complicated library of helper functions, mostly named with the prefix Co. It is these which actually load the DLLs, locate the exported functions and call them. Client code would not normally call the DLL exports directly (although it is a fairly straightforward matter to write your own hosting system for simple in-process COM objects).
There is an important interface IDispatch which adds another layer of abstraction such that it is possible to dynamically identify methods to call on the object, using a string to find the method and passing an array of argument values. This allows scripting languages to call COM objects safely (i.e. in a way that tolerates mistakes by the programmer instead of crashing).
There is a more complex "marshalling" system for allowing objects to be called from other threads, other processes or even other computers; in its remote (and security-aware) for form it was called DCOM. It did not succeed on anything like the scale that other COM usages did.
The cross-process support was the basis of OLE2, a gimmick that every application rushed to support in the early 1990s, most of them implementing the (very complicated) interfaces wrongly and so crashing all over the place.
Far more successful were OLE Controls, which were technically a lot simpler (not using the marshalling) and which offered a way to extend Visual Basic.
There is at least one widely distributed COM-inspired system that is not compatible with COM but shares exactly the same concepts (at least the simple stuff), which is called XPCOM and is the basis of the Mozilla Firefox Web browser.
If you use COM pervasively, you will have a great way to integrate with the .NET framework, as it includes superb interoperability with COM. But you must follow the rules of COM precisely: AddRef/Release must work exactly as defined, so an object must never be destroyed while its reference count is greater than zero, it must be possible to use QueryInterface to find IUnknown from any interface, and from there to get back to the original interface, the IUnknown obtained from any interface must always have the same memory address for the lifetime of the object (so it can be used for identity comparison). So for example: don't create COM objects on the stack, and don't return separate objects from QueryInterface (that is, objects that have their own QueryInterface that returns different interfaces).
A major gotcha with COM is that it has at least two standard ways of representing strings. Neither uses reference counting. Why they never defined an IString interface is beyond me, but they didn't.
You'll go insane if you try to write COM code without smart pointers to hold references, or without a base class to help you implement interfaces, e.g. class MyClass : COM::Object<IThis, IThat> {...)
Assume that you work only in the C++ world (cross-language interop is not required). What advantages/inconvenients do you see in using COM instead of a plain basic DLL? Do you think using COM is worth the trouble if you are not going to use the interface from different languages?
Everybody is mentioning things that are in COM's plus column. I'll mention a couple of detractions.
When you implement your system using COM, you need to register the COM 'servers' (be they in-proc or out-of-proc) at setup and unregister them at uninstall. This could increase the complexity of your setup system slightly and tends to require a reboot unless the user carefully tears down running processes first.
COM is slow compared to other standard ways of doing the same thing. This comment will probably generate a lot of hate and maybe some downvotes, but the fact of the matter is that at some point you will need to marshall data, and that is expensive.
According to the Rules of COM, once an interface has been published it can never be changed. That in itself is not a negative, and you might even argue that it forces you to do thorough design before shipping the interface. But the truth is there's no such thing as never, and in production code interfaces change. You will undoubtedly need to either add methods or change the signatures of existing methods. In order to accomplish this you have to either break the rules of COM -- which has bad effects -- or follow the rules of COM which are more complicated than just adding a parameter to a function like you would with a astraight DLL.
COM can be useful in plain old C++ for:
Interprocess communication
Plugin architectures
Late binding scenarios
"Much, much, more..." (tm)
That said, if you don't need it, don't use it.
With DLL you can get much closer coupling, while COM limits interactions very precisely. This is the root of both the advantages and the disadvantages!
You get more power and flexibility (e.g. inherit from classes defined in the DLL, not feasible in COM) but the dependency is thereby much stronger (need to rebuild the user for certain changes to the DLL, etc).
Often especially galling is that all DLLs and the EXE must use the same kind of runtime library and options (e.g. all dynamically linked to the non-debug multithreaded version of msvcrt* for example -- can't rebuild just one to use the debug version without incurring very likely errors!).
The looser coupling of COM is therefore often preferable, unless you really need the closer-coupling kinds of interactions in a specific case (e.g., a framework, which definitely requires user-code to inherit from its classes, should be a DLL).
If you can avoid don't use it. In my last project COM brought pretty much limitations into C++ interfaces being used. Just imagine, that you can't simply pass a std::string but have to use an array of characters. In that case you build the string, an then copy it to an array which can be handled by COM.
You also can only use very limited set of fundamental types, have casts and proprietary memory management. You can't use new/delete, but have to use COM own functions.
You also can't simply throw an exception, but have to initialize some COM interface IErrorInfo, which will be rethrown at the other end.
So if you don't need, don't use it. It will definitely screw your design. And if you need it, try to evaluate other interop possibilities: boost::interprocess, zeroc ice...
Regards,
Ovanes
Registration and discovery
Out-of-process
Remote invocation
are the few extra features that you would have got. Even transactional support can flow without the need for COM support these days.
The IUnknown interface is a good base level to support anyway -- gets you a way to add features without breaking old clients (QueryInterface) and pervasive reference counting. You can implement this without buying into everything in COM.
Then, whenever you are adding a feature to a class, if you use the COM interface for it, you at least get an interface that is known -- for example IDispatch if you want reflection features.
Your only delta away from being able to be called by another language would then be the registration and the class factory.
Because interfaces are independent of any particular DLL, at its simplest level, a COM like approach at the very least frees you to change the dll serving an interface under the hood, without having to recompile your app against the new dll name.
Using Full COM with MIDL defined interfaces and proxy stub dlls means that you can use COM to manage thread safety in-process, interprocess comms on the same PC, or even connect to the COM server object on a remote PC.