Related
Hope this question isn't going to be too vague. Reading through the COM spec and Don Box's Essential COM book, there is plenty of talk of the "problems that COM solves" - and they all sound important, relevant and current.
So how are the problems that COM addresses dealt with on other systems (linux, unix, OSX, android)? I'm thinking of things like:
binary compatibility across compilers and compiler versions
binary component reuse
compiling an application such that it has run-time dependencies rather than load-time ones (so that it runs even when a dependency is missing)
access to library functionality from languages other than the library's own
reasonably low-overhead remote procedure calls to components loaded in the address space of a different process
etc (I'm sure the list goes on)
I'm basically just trying to understand why for instance on Linux CORBA isn't a thing like COM is a thing on Windows (if that makes any sense). Does maybe software development on Linux subscribe to a different philosophy than the component-based model proposed by COM?
And finally, is COM a C/C++ thing? Several times I've come across comments from people saying COM is made "obsolete" by .NET but without really explaining what they meant by that.
For the remainder of this post, I'm going to use Linux as an example of open-source software. Where I mention "Linux" it's mostly a short/simple way to refer to open source software in general though, not anything specific to Linux.
COM vs. .NET
COM isn't actually restricted to C and C++, and .NET doesn't actually replace COM. However, .NET does provide alternatives to COM for some situations. One common use of COM is to provide controls (ActiveX controls). .NET provides/supports its own protocol for controls that allows somebody to write a control in one .NET language, and use that control from any other .NET language--more or less the same sort of thing that COM provides outside the .NET world.
Likewise, .NET provides Windows Communication Foundation (WCF). WCF implements SOAP (Simple Object Access Protocol)--which may have started out simple, but grew into something a lot less simple at best. In any case, WCF provides many of the same kinds of capabilities as COM does. Although WCF itself is specific to .NET, it implements SOAP, and a SOAP server built using WCF can talk to one implemented without WCF (and vice versa). Since you mention overhead, it's probably worth mentioning that WCF/SOAP tend to add more overhead that COM (I've seen anywhere from nearly equal to about double the overhead, depending on the situation).
Differences in Requirements
For Linux, the first two points tend to have relatively low relevance. Most software is open source, and many users are accustomed to building from source in any case. For such users, binary compatibility/reuse is of little or no consequence (in fact, quite a few users are likely to reject all software that isn't distributed in source code form). Although binaries are commonly distributed (e.g., with apt-get, yum, etc.) they're basically just caching a binary built for a specific system. That is, on Windows you might have a single binary for use on anything from Windows XP up through Windows 10, but if you use apt-get on, say, Ubuntu 18.02, you're installing a binary built specifically for Ubuntu 18.02, not one that tries to be compatible with everything back to Ubuntu 10 (or whatever).
Being able to load and run (with reduced capabilities) when a component is missing is also most often a closed-source problem. Closed source software typically has several versions with varying capabilities to support different prices. It's convenient for the vendor to be able to build one version of the main application, and give varying levels of functionality depending on which other components are supplied/omitted.
That's primarily to support different price levels though. When the software is free, there's only one price and one version: the awesome edition.
Access to library functionality between languages again tends to be based more on source code instead of a binary interface, such as using SWIG to allow use of C or C++ source code from languages like Python and Ruby. Again, COM is basically curing a problem that arises primarily from lack of source code; when using open source software, the problem simply doesn't arise to start with.
Low-overhead RPC to code in other processes again seems to stem primarily from closed source software. When/if you want Microsoft Excel to be able to use some internal "stuff" in, say, Adobe Photoshop, you use COM to let them communicate. That adds run-time overhead and extra complexity, but when one of the pieces of code is owned by Microsoft and the other by Adobe, it's pretty much what you're stuck with.
Source Code Level Sharing
In open source software, however, if project A has some functionality that's useful in project B, what you're likely to see is (at most) a fork of project A to turn that functionality into a library, which is then linked into both the remainder of project A and into Project B, and quite possibly projects C, D, and E as well--all without imposing the overhead of COM, cross-procedure RPC, etc.
Now, don't get me wrong: I'm not trying to act as a spokesperson for open source software, nor to say that closed source is terrible and open source is always dramatically superior. What I am saying is that COM is defined primarily at a binary level, but for open source software, people tend to deal more with source code instead.
Of course SWIG is only one example among several of tools that support cross-language development at a source-code level. While SWIG is widely used, COM is different from it in one rather crucial way: with COM, you define an interface in a single, neutral language, and then generate a set of language bindings (proxies and stubs) that fit that interface. This is rather different from SWIG, where you're matching directly from one source to one target language (e.g., bindings to use a C library from Python).
Binary Communication
There are still cases where it's useful to have at least some capabilities similar to those provided by COM. These have led to open-source systems that resemble COM to a rather greater degree. For example, a number of open-source desktop environments use/implement D-bus. Where COM is mostly an RPC kind of thing, D-bus is mostly an agreed-upon way of sending messages between components.
D-bus does, however, specify things it calls objects. Its objects can have methods, to which you can send signals. Although D-bus itself defines this primarily in terms of a messaging protocol, it's fairly trivial to write proxy objects that make invoking a method on a remote object look pretty much like invoking one on a local object. The big difference is that COM has a "compiler" that can take a specification of the protocol, and automatically generate those proxies for you (and corresponding stubs in the far end to receive the message, and invoke the proper function based on the message it received). That's not part of D-bus itself, but people have written tools to take (for example) an interface specification and automatically generate proxies/stubs from that specification.
As such, although the two aren't exactly identical, there's enough similarity that D-bus can be (and often is) used for many of the same sorts of things as COM.
Systems Similar to DCOM
COM also allows you to build distributed systems using DCOM (Distributed COM). That is, a system where you invoke a method on one machine, but (at least potentially) execute that invoked method on another machine. This adds more overhead, but since (as pointed out above with respect to D-bus) RPC is basically communication with proxies/stubs attached to the ends, it's pretty easy to do the same thing in a distributed fashion. The difference in overhead, however, tends to lead to differences in how systems need to be designed to work well, though, so the practical advantage of using exactly the same system for distributed systems as local systems tends to be fairly minimal.
As such, the open source world provides tools for doing distributed RPC, but doesn't usually work hard at making them look the same as non-distributed systems. CORBA is well known, but generally viewed as large and complex, so (at least in my experience) current use is fairly minimal. Apache Thrift provides some of the same general type of capabilities, but in a rather simpler, lighter-weight fashion. In particular, where CORBA attempts to provide a complete set of tools for distributed computing (complete with everything from authentication to distributed time keeping), Thrift follows the Unix philosophy much more closely, attempting to meet exactly one need: generate proxies and stubs from an interface definition (written in a neutral language). If you want to do those CORBA-like things with Thrift you undoubtedly can, but in a more typical case of building internal infrastructure where the caller and callee trust each other, you can avoid a lot of overhead and just get on with the business at hand. Likewise, google RPC provides roughly the same sorts of capabilities as Thrift.
OS X Specific
Cocoa provides distributed objects that are fairly similar to COM. This is based on Objective-C though, and I believe it's now deprecated.
Apple also offers XPC. XPC is more about inter-process communication than RPC, so I'd consider it more directly comparable to D-bus than to COM. But, much like D-bus, it has a lot of the same basic capabilities as COM, but in different form that places more emphasis on communication, and less on making things look like local function calls (and many now prefer messaging to RPC anyway).
Summary
Open source software has enough different factors in its design that there's less demand for something providing the same mix of capabilities as Microsoft's COM provides on Windows. COM is largely a single tool that tries to meet all needs. In the open-source world, there's less drive to provide that single, all-encompassing solution, and more tendency toward a kit of tools, each doing one thing well, that can be put together into a solution for a specific need.
Being more commercially oriented, Apple OS X probably has what are (at least arguably) closer analogs to COM than most of the more purely open-source world.
A quick answer on the last question: COM is far from being obsolete. Almost everything in the Microsoft world is COM-based, including the .NET engine (the CLR), and including the new Windows 8.x's Windows Runtime.
Here is what Microsoft says about .NET in it latest C++ pages Welcome Back to C++ (Modern C++):
C++ is experiencing a renaissance because power is king again.
Languages like Java and C# are good when programmer productivity is
important, but they show their limitations when power and performance
are paramount. For high efficiency and power, especially on devices
that have limited hardware, nothing beats modern C++.
PS: which is a bit of a shock for a developer who has invested more than 10 years on .NET :-)
In the Linux world, it is more common to develop components that are statically linked, or which run in separate processes and communicate by piping text (maybe JSON or XML) back and forth.
Some of this is due to tradition. UNIX developers have been doing stuff like this long before CORBA or COM existed. It's "the UNIX way".
As Jerry Coffin says in his answer, when you have the source code for everything, binary interfaces are not as important, and in fact just make everything more difficult.
COM was invented back when personal computers were a lot slower than they are today. In those days, loading components into your app's process space and invoking native code was often necessary to achieve reasonable performance. Now, parsing text and running interpreted scripts aren't things to be afraid of.
CORBA never really caught on in the open-source world because the initial implementations were proprietary and expensive, and by the time high-quality free implementations were available, the spec was so complicated that nobody wanted to use it if they weren't required to do so.
To a large extent, the problems solved by COM are simply ignored on Linux.
It is true that binary compatibility is less important when you have the source code available. However, you still have to worry about modularisation and versioning. If two different programs depend on different versions of the same library, you need to somehow support that.
Then there is the case of the same program using different versions of the same library. This is often useful when working on large legacy programs, where upgrading everything can be prohibitively expensive but you would like to use new features anyway. With COM, the old parts of the program can just be left alone, since new library versions can more easily be made backwards compatible.
In addition, having to compile from source instead of binary compatibility is a huge hassle. Especially if you are actually developing software, since binary incompatibility means you have to recompile much more often. If one tiny part changes in a large C++ program, you may have to wait for a 30 minute recompile. If the different pieces are compatible, only the part which changed has to be recompiled.
COM and DCOM in particular have been around in windows for some considerable time now and naturally windows developers have made use of this powerful framework.
We are now in the cross platform age and when porting such applications to other platforms we are faced with challenges which in many cases can be mitigated or eliminated altogether unless the application we are porting is more than just one simple standalone app.
If your dealing with a whole suite of modules running on different machines all communicating using windows specific technologies such as DCE/RPC, DCOM or even windows named pipes then your job just became an order of magnitude harder.
DCE/RPC DCOM and windows named pipes all are very windows specific, non portable and of course subject to windows security access control.
For instance anyone familar with OPC DA (an industrial automation protocol based on DCOM still very much in use but now superceded by OPC UA (which avoids DCOM))) will know that there are no elegant solutions here if the client (or server) needs to be available for Linux!!
Sure there appear to be some technical hurdles here given that the MS code is not in the public domain but projects such as Wine have a partly ok DCE/RPC implementation and MS do publish some of the protocol docs. Try searching and you will probably find little information and few products open source or otherwise to help you.
Perhaps the lack of open source or affordable options here is more due to legal concerns - I wonder!
Some simpler solutions simply involve installing a "gateway service" on the windows machines to allow an alternative means of access to DCOM interfaces on that machine. This is fine if the windows machine does not belong to an unwilling 3rd party which unfortunately is sometimes the case!!! I know we'll just chuck another Windows machine as the gateway in the middle is the usual global warming enhancing solution to that problem.
I would conclude that Linux to Windows DCOM interoperability is certainly not impossible but it does appear to be a topic that few are interested in talking about unless you get your wallet out!
I am looking for a library for native C++ (library source code has to be gcc compatible and portable across Linux and Windows) that does what WCF does in its very basic form - i.e. OperationContracts and DataContracts in a client-server environment, with data exchanges in binary format (binary serialization).
Ideally I'd like to use a library to achieve this. So if there's a library already available that compiles OperationContracts and DataContracts into rich C++ classes with metadata for reflection which can be consumed in our code and with client-server TCP communications built-in (i.e. a rudimentary implementation of WCF's functionality without the need to be compatible with WCF at all), please point me to it.
If not, implementing them myself (unlikely due to time constraints), I could use boost::serialization for DataContracts but how would I implement OperationContracts?
It's not necessarily compatible with gcc--so it's somewhat tangential to the precise question asked here--but I'd like to include a reference to the Windows Web Services API, Microsoft's native-code counterpart to WCF (for Windows systems).
From this secondary article:
WWS is designed from the ground up to be a completely native-code implementation of SOAP, including support for many of the WS-* protocols. WWS is, strictly speaking, exposed through a C API, making interoperability with other languages and runtimes very straightforward, but it is the C++ developer who will likely benefit the most.
I have found one that fits the purpose called "RCF" (Remote Call Framework) by Delta V Software. It's open source (GPLv2 or US$195 closed source). So far in my testing, it's working very well. According to the site, companies like HP, Ericsson and Siemens are users of the library.
Apache Thrift is another option you might consider,
http://en.wikipedia.org/wiki/Apache_Thrift
I'm starting development on a new Application, and although my background is mainly Mac/iOS based, I need to work on a Windows application for participation in their Imagine cup.
This project includes communication between clients through a socket connection (to a server, not ad-hoc), and I need for both Mac and Windows clients to be able to communicate with each other. I'd also like to not have to write this networking code twice, and simply write different native-UI code on both platforms. This makes the networking easier (I'm confident that two different platforms are not going to be interacting with the server in different ways) and allows for a native UI on both platforms.
Is C++ the best language for this task? Is the standard library the same on both platforms? I understand that I'll have to use Microsoft's Visual C++ library, as it seems as though it is hard to utilize C++ code from C#; is this true?
I've never really written a cross-platform application before, especially one that deals with networking between platforms.
If you're going to use C++, I second "the_mandrill" above by strongly recommending you look at ASIO in Boost - that is a great way to write the code once and support both platforms.
As to whether or not C++ is the right language is rather more subjective. My personal feelings are that if you need to ask, odds on, it's not the best approach.
Reasons for selecting C++ to implement networking code are:
Possible to achieve very low latencies and high throughput with very carefully designed and implemented code.
Possible to take explicit control of memory management - avoiding variations in performance associated with garbage collection.
Potential for tight integration of networking code in-process with other native libraries.
Potential to build small components suitable for deployment in resource constrained environments.
Low level access to C socket API exposes features such as socket options to use protocols beyond vanilla TCP/IP and UDP.
Reasons for avoiding C++ are:
Lower productivity in developing code compared with higher-level languages (e.g. Java/C#/Python etc. etc.)
Greater potential for significant implementation errors (pointer-abuse etc.)
Additional effort required to verify code compiles on each platform.
Alternatives you might consider include:
Python... directly using low level socket support or high-level Twisted library. Rapid development using convenient high-level idioms and minimal porting effort. Biggest disadvantage - poor support to exploit multiple processor cores for high-throughput systems.
Ruby(socket)/Perl(IO::Socket)... High level languages which might be particularly suited if the communicated information is represented as text strings.
Java... garbage collection simplifies memory management; wide range of existing libraries.
CLR languages (C# etc.) also garbage collected - like Java... WCF is an effective framework within which bespoke protocols can be developed... which may prove useful.
You also asked: "I understand that I'll have to use Microsoft's Visual C++ library, as it seems as though it is hard to utilize C++ code from C#; is this true?"
You can avoid Visual C++ libraries entirely - either by developing using a language other than C++, or using an alternate C++ compiler - for example Cygwin or MinGW offer G++... though I'd recommend using Visual C++ to build C and C++ code for Windows.
It's not hard to use C++ code from C# - though I don't recommend it as an approach.. it's likely overly complicated. Visual C++ can (optionally) compile "Managed Code" from C++ source (there are a few syntax extensions to grasp and there's a slightly different syntax for interoperation using Mono rather than Visual C++, but these are not major issues IMHO.) These CLR objects interact directly with C# objects and can be linked together into a single assembly without issue. It's also easy to access native DLLs (potentially written using C++ for the native architecture) using Pinvoke. All this is somewhat irrelevant, however, as the .Net framework has good support for low level networking (similar to that provided by Winsock[2]) in system.net - this provides a convenient C#-oriented interface to similar facilities, and will likely provide a more convenient API to develop against if using C# (or VB.Net or any of the other CLR languages.)
I would suggest that you take a look at Qt. IMO Qt is one of the best C++ libraries for doing cross-platform application. The benefits of Qt when comparing with Boost is that Qt have even GUI classes.
Best language is very subjective, but if you want portable fast code with useful abstractions and C style syntax the C++ is a good choice. Note if you do not know any C++ already there is a steep learning curve.
The library as defined by the ISO standard is by definition the same on each platform, however the implementation of it my be less or more compliant. That said, both gcc, clang and MSVC(post .net) all implement C++98 very well. So long as you don't use compiler specific extensions.
I strong recommend looking at boost asio (and the boost library in general) for your networking, it uses the proactor design pattern which is very efficient. However it does take some time getting your head around it.
http://www.boost.org/doc/libs/1_48_0/doc/html/boost_asio.html
Stick with the Standard library and boost and for the most part cross platform problems are not a major concern.
Lastly, I would avoid using the C++11 features for writing cross platform code, because MSVC, GCC and Clang have all implemented different parts of the standard.
If you want to spend a year and put in a 1000 hrs sure use boost::asio. Or you can use a library that's built around boost::asio that invokes C++ network templates. This is crossplatform network and you can find it here:
https://bitbucket.org/ptroen/crossplatformnetwork/src/master/
It compiles on Windows,Android, Macos and Linux.
That is not to say if your absolutely expert level in boost::asio you can do a tiny bit better in performance but if you want to get like 98% of the performance gains you may find it useful. It also supports HTTP,HTTPS,NACK,RTP,TCP ,UDP,MulticastServer and Multicast Client.
Examples:
TCPServer: https://bitbucket.org/ptroen/crossplatformnetwork/src/master/OSI/Application/Stub/TCPServer/main.cc
HTTPServer: https://bitbucket.org/ptroen/crossplatformnetwork/src/master/OSI/Application/Stub/HTTPServer/httpserver.cc
OSI::Transport::TCP::TCP_ClientTransport<SampleProtocol::IncomingPayload<OSI::Transport::Interface::IClientTransportInitializationParameters>, SampleProtocol::OutgoingPayload<OSI::Transport::Interface::IClientTransportInitializationParameters>,
SampleProtocol::SampleProtocolClientSession<OSI::Transport::Interface::IClientTransportInitializationParameters>, OSI::Transport::Interface::IClientTransportInitializationParameters> tcpTransport(init_parameters);;
SampleProtocol::IncomingPayload< OSI::Transport::Interface::IClientTransportInitializationParameters> request(init_parameters);
request.msg = init_parameters.initialPayload;
std::string ipMsg=init_parameters.ipAddress;
LOGIT1(ipMsg)
tcpTransport.RunClient(init_parameters.ipAddress, request);
I'm biased because I wrote the library.
You can also check for this communication library:
finalmq
This library has the following properties:
C++, cross-platform, async/non-blocking, multiple protocols (TCP, HTTP, mqtt5), multiple encoding formats (json, protobuf). Check the examples.
I have a set of functions in C/C++ that I need to be available to accept calls and return values to C/C++ code in a remote location, similar to RMI on the java platform. With RMI the Java methods are set up through the rmiregistry and remain available in memory to accept requests. I'm looking for similar functionality in C/C++, but i'm a bit confused with all the options that are out there. Is this type of scenario that CORBA was intended for and if so, is this still the best technology to use or are there better options out there. I've read about XML-RPC, CORBA, and a few others but i'm not sure which of these is what i need.
Thanks for your help.
Mike
Is this type of scenario that CORBA was intended for and if so, is this still the best technology to use or are there better options out there.
Yes, this is what CORBA was intended to solve. Whether it's "best" is subjective and argumentative. :) I can say, from my personal experience, I don't miss my short experience with CORBA and would suggest you explore other options.
I've read about XML-RPC, CORBA, and a few others but i'm not sure which of these is what i need.
As you seem to be aware, you're looking for any technology that implements RMI (also frequently called RPC). It's not built-in to C/C++.
On Linux, there is SunRPC. I would also recommend looking at Google protocol buffers, which provide a mechanism for serializing data as well as an interface for defining RPC services. There are several service implementations available, but I don't have experience with the service implementations.
On Unix-like platforms, you're probably looking for Sun RPC (remote procedure calls).
CORBA is also relevant but has a more natural binding to languages with object oriented capability.
There's no built in method for accomplishing this in C or C++. That said there are several libraries that can accomplish this.
If you're on Windows, then the best answer is probably DCOM, which is part of the OS itself. I'm not sure about other platforms.
I shall suggest either CORBA or any webservice library available
CORBA is a reasonable choice for you (though it may be a little bit old technology for now). I have been using CORBA for several years in my previous job.
I should say, learning curve for CORBA is kind of steep, and you need to cater a lot of extra setup, but once it it done correct, it become smooth to use. (The problem is it takes really some time to use it correctly)
Webservice is an de facto industrial standard now, and I believe C++ will have some reasonable implementation and library for that. CORBA cover more features than WS but those feature are seldom used in simple systems.
I'm a game's developer and am currently in the processing of writing a cross-platform, multi-threaded engine for our company. Arguably, one of the most powerful tools in a game engine is its scripting system, hence I'm on the hunt for a new scripting language to integrate into our engine (currently using a relatively basic in-house engine).
Key features for the desired scripting system (in order of importance) are:
Performance - MUST be fast to call & update scripts
Cross platform - Needs to be relatively easy to port to multiple platforms (don't mind a bit of work, but should only take a few days to port to each platform)
Offline compilation - Being able to pre-parse the script code offline is almost essential (helps with file sizes and load times)
Ability to integrate well with c++ - Should be able to support OO code within the language, and integrate this functionality with c++
Multi-threaded - not required, but desired. Would be best to be able to run separate instances of it on multiple threads that don't interfere with each other (i.e. no globals within the underlying code that need to be altered while running). Critical Section and Mutex based solutions need not apply.
I've so far had experience integrating/using Lua, Squirrel (OO language, based on Lua) and have written an ActionScript 2 virtual machine.
So, what scripting system do you recommend that fits the above criteria? (And if possible, could you also post or link to any comparisons to other scripting languages that you may have)
Thanks,
Grant
Lua has the advantage of being time-tested by a number of big-name video game developers and a good base of knowledgeable developers thanks to Blizzard-Activision's adoption of it as the primary platform for developing World of Warcraft add-ins.
Lua is a very good match for your needs. I'll take them in the same order.
Lua is one of the fastest scripting languages. It's fast to compile and fast to run.
Lua compiles on any platform with an ANSI C compiler, which afaik includes all gaming platforms.
Lua can be pre-compiled, but as a very dynamic languages most errors are only detectable at runtime. Also precompiled code (as bytecode) is often larger in terms of size than source code.
There are many Lua/C++ binding tools.
It doesn't support multi-threading (you cannot access a single instance of the interpreter from multiple threads), but you can have several instances of the interpreter, one per thread, or even one per game object.
Lua have been used in video-game industry for years. Lightweight and efficient.
That being said, ChaiScript and Falcon are good candidates matching your needs and with higher level language than Lua but with less history and community support.
Lua
Boost Python
SWIG
We've had good luck with Squirrel so far. Lua is so popular it's on its way to becoming a standard.
I recommend you worry more about memory than speed. Most scripting languages are "fast enough" and if they get slow you can always push some of that functionality back down into C++. Many of them burn through lots of memory, though, and on a console memory is an even more scarce resource than CPU time. Unbounded memory consumption will crash you eventually, and if you have to allocate 4MB just for the interpreter, that's like having to throw 30 textures out the window to make room.
Lua, and then LuaJIT for extra blaziness!
just don't expect too much from automatic C++ binding libraries, most are slow and restrictive. better do your own binding for your own objects.
as for concurrency, either LuaLanes, or roll your own. if your C++ program is already multithreaded, just call separate LuaStates from each thread, and use your own C++ shared structures as communications channels if needed.
as you might already know, the most often repeated answer in Lua is 'roll your own', and it's often the best advice! except when it's about bindings to common C/C++ libraries, in that case it's quite probable there's already one.
If you haven't looked at it yet I would suggest you check out Angelscript.
I have successfully used it in a cross platform environment (Windows and Linux with only a recompile) and it is designed to integrate well with C++ (both objects and code).
It is lightweight and supports multi-threading (in the sense that the question was asked), performs well and compiles to byte code which could be done in advance.
Start with Python.
If you can prove that you need more speed, then look at Stackless Python. That's what EVE Online uses for their game.
JavaScript may be a reasonable option, because of the mountains of effort that have gone into optimizing the various implementations for use in web-browsers.
These come to mind:
Lua
Python with boost::python
MzScheme or Guile
Ruby with SWIG