Related
I'm mainly a C++ guy. As C++ lacks an official ABI I always use a COM-like approach for component designs that support more than one compiler.
Recently I came across the question whether Objective-C would be a replacement for the COM-like approach. Obviously for Objective-C to be a replacement one would need a stable ABI, therefor I'd like to know if a stable ABI for Objective-C exists (on all major OSes [OSX, GNU/Linux, Windows]) and how easy it would be to use Objective-C(++) as "glue" between components created by different compilers.
EDIT:
As Nikolai Ruhe pointed out a short description of COM may be helpful. COM is essentially a "binary standard" that allows mixing binarys of different compilers (and in a variety of languages). The vehicle of COM are interfaces, which define methods (which map to C++'s virtual functions). Components implement a at least one interfaces and are distributed as DLLs. They can be located anywhere on the system (the position is specified in the Registry) and can be loaded by any COM-client via the ID of the interface they implement.
I can only speak for Apple's implementation, as I have no experience with the GNU or other ports.
Objective-C relies on C's ABI for the most part (like function calls and memory layout of structs).
It's own ABI underwent a couple of changes in Apple's implementation, like non-fragile instance variables introduced with the "Modern Runtime", introduction of properties, faster exception handling, garbage collection, __weak support for ARC.
Some of the changes were backwards compatible, some not. But since the whole system and frameworks are provided by Apple and the changes were usually introduced with other non-compatible changes (switch to Intel, and LP64) this was without consequences to users.
Edit: One thing you should have in mind is that Objective-C does not only rely on a fixed ABI but also on a compatible runtime. That's one more headache to care about for your purpose.
What does it offer to an object oriented language such as C++? or is it not possible to use GTK+ without it?
Is the GObject implementation of objects is of a similar quality to that of C++ in terms of the size and speed of an executable assuming both examples use the same compiler? Or are there some trade-offs where GObject would be slower on the account of additional capabilities it provides?
GObject (a bit like COM in the Windows world) is a C API designed with cross language interoperability in mind.
This means that you can use GObjects in any language which supports calling C functions, but this makes it very difficult to write GObjects in a non-C language which are truly reusable from any language (if you write a GObject derived class in say, Python, you'd have to embed a Python interpreter every time you wanted to use objects from this class in C).
It is possible to semi-automate the creation of bindings for many languages (eg. Python, Perl, JS etc), and here lies one of the strengths of GObject. This accounts for the somewhat opaque API that GObject provides, which is, I confess, quite difficult to understand thoroughly.
Unfortunately, it doesn't fit well within the C++ language either. GObjects have no trivial relationship with C++ classes, and even if bindings are available (Gtkmm) it is not possible to easily write a C++ class "inheriting from GObject" and expose it to the world. You have to write C for this.
[What the world would need would be some kind of extensions to the C++ language which would make it easy to interop with GObject, a little like C++Cx on Windows, but 1) it is a difficult task, perhaps achievable through a GCC plugin, and 2) there is no momentum towards C++ in the Gnome world, or generally in the Linux world (KDE being a notable exception). For now we are stuck with the Gtkmm bindings.]
The article on GObject from Wikipedia includes a comparison with C++. Some of the things they mention is the lack of multiple inheritance, and the presence of signals. Additionally, GObject benefits from the fact that the names of exported C functions do not, unlike C++, depend on your choice of compiler. So if you were to develop an object-oriented library using GObject, it would probably be easier to link with than a C++ one.
It would also be interesting to look at the Vala programming language, which targets GObject.
Just a little elaboration on something hinted by Vlad: A major point in favour of C is that it makes interoptability between compilers or languages 'possible' (guaranteed), in that it standardises an ABI. This (pardon me if I'm oversimplifying) enables guarantees about how callers from any C compiler or other language can use exported symbols. Hence why GTK+ has bindings to various other languages - including C++ in GTKmm. The latter is the best of both worlds IMHO: the well-established API of GTK+ but with the language features of C++.
C++ as yet does not have an official standard ABI, though all is not yet lost, as the A-Team are working on it: https://isocpp.org/files/papers/n4028.pdf
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm planning to code a library that should be usable by a large number of people in on a wide spectrum of platforms. What do I have to consider to design it right? To make this questions more specific, there are four "subquestions" at the end.
Choice of language
Considering all the known requirements and details, I concluded that a library written in C or C++ was the way to go. I think the primary usage of my library will be in programs written in C, C++ and Java SE, but I can also think of reasons to use it from Java ME, PHP, .NET, Objective C, Python, Ruby, bash scrips, etc... Maybe I cannot target all of them, but if it's possible, I'll do it.
Requirements
It would be to much to describe the full purpose of my library here, but there are some aspects that might be important to this question:
The library itself will start out small, but definitely will grow to enormous complexity, so it is not an option to maintain several versions in parallel.
Most of the complexity will be hidden inside the library, though
The library will construct an object graph that is used heavily inside. Some clients of the library will only be interested in specific attributes of specific objects, while other clients must traverse the object graph in some way
Clients may change the objects, and the library must be notified thereof
The library may change the objects, and the client must be notified thereof, if it already has a handle to that object
The library must be multi-threaded, because it will maintain network connections to several other hosts
While some requests to the library may be handled synchronously, many of them will take too long and must be processed in the background, and notify the client on success (or failure)
Of course, answers are welcome no matter if they address my specific requirements, or if they answer the question in a general way that matters to a wider audience!
My assumptions, so far
So here are some of my assumptions and conclusions, which I gathered in the past months:
Internally I can use whatever I want, e.g. C++ with operator overloading, multiple inheritance, template meta programming... as long as there is a portable compiler which handles it (think of gcc / g++)
But my interface has to be a clean C interface that does not involve name mangling
Also, I think my interface should only consist of functions, with basic/primitive data types (and maybe pointers) passed as parameters and return values
If I use pointers, I think I should only use them to pass them back to the library, not to operate directly on the referenced memory
For usage in a C++ application, I might also offer an object oriented interface (Which is also prone to name mangling, so the App must either use the same compiler, or include the library in source form)
Is this also true for usage in C# ?
For usage in Java SE / Java EE, the Java native interface (JNI) applies. I have some basic knowledge about it, but I should definitely double check it.
Not all client languages handle multithreading well, so there should be a single thread talking to the client
For usage on Java ME, there is no such thing as JNI, but I might go with Nested VM
For usage in Bash scripts, there must be an executable with a command line interface
For the other client languages, I have no idea
For most client languages, it would be nice to have kind of an adapter interface written in that language. I think there are tools to automatically generate this for Java and some others
For object oriented languages, it might be possible to create an object oriented adapter which hides the fact that the interface to the library is function based - but I don't know if its worth the effort
Possible subquestions
is this possible with manageable effort, or is it just too much portability?
are there any good books / websites about this kind of design criteria?
are any of my assumptions wrong?
which open source libraries are worth studying to learn from their design / interface / souce?
meta: This question is rather long, do you see any way to split it into several smaller ones? (If you reply to this, do it as a comment, not as an answer)
Mostly correct. Straight procedural interface is the best. (which is not entirely the same as C btw(**), but close enough)
I interface DLLs a lot(*), both open source and commercial, so here are some points that I remember from daily practice, note that these are more recommended areas to research, and not cardinal truths:
Watch out for decoration and similar "minor" mangling schemes, specially if you use a MS compiler. Most notably the stdcall convention sometimes leads to decoration generation for VB's sake (decoration is stuff like #6 after the function symbol name)
Not all compilers can actually layout all kinds of structures:
so avoid overusing unions.
avoid bitpacking
and preferably pack the records for 32-bit x86. While theoretically slower, at least all compilers can access packed records afaik, and the official alignment requirements have changed over time as the architecture evolved
On Windows use stdcall. This is the default for Windows DLLs. Avoid fastcall, it is not entirely standarized (specially how small records are passed)
Some tips to make automated header translation easier:
macros are hard to autoconvert due to their untypeness. Avoid them, use functions
Define separate types for each pointer types, and don't use composite types (xtype **) in function declarations.
follow the "define before use" mantra as much as possible, this will avoid users that translate headers to rearrange them if their language in general requires defining before use, and makes it easier for one-pass parsers to translate them. Or if they need context info to auto translate.
Don't expose more than necessary. Leave handle types opague if possible. It will only cause versioning troubles later.
Do not return structured types like records/structs or arrays as returntype of functions.
always have a version check function (easier to make a distinction).
be careful with enums and boolean. Other languages might have slightly different assumptions. You can use them, but document well how they behave and how large they are. Also think ahead, and make sure that enums don't become larger if you add a few fields, break the interface. (e.g. on Delphi/pascal by default booleans are 0 or 1, and other values are undefined. There are special types for C-like booleans (byte,16-bit or 32-bit word size, though they were originally introduced for COM, not C interfacing))
I prefer stringtypes that are pointer to char + length as separate field (COM also does this). Preferably not having to rely on zero terminated. This is not just because of security (overflow) reasons, but also because it is easier/cheaper to interface them to Delphi native types that way.
Memory always create the API in a way that encourages a total separation of memory management. IOW don't assume anything about memory management. This means that all structures in your lib are allocated via your own memory manager, and if a function passes a struct to you, copy it instead of storing a pointer made with the "clients" memory management. Because you will sooner or later accidentally call free or realloc on it :-)
(implementation language, not interface), be reluctant to change the coprocessor exception mask. Some languages change this as part of conforming to their standards floating point error(exception-)handling.
Always pair a callbacks with an user configurable context. This can be used by the user to give the the callback state without defining global variables. (like e.g. an object instance)
be careful with the coprocessor status word. It might be changed by others and break your code, and if you change it, other code might stop working. The status word is generally not saved/restored as part of calling conventions. At least not in practice.
don't use C style varargs parameters. Not all languages allow variable number of parameters in an unsafe way
(*) Delphi programmer by day, a job that involves interfacing a lot of hardware and thus translating vendor SDK headers. By night Free Pascal developer, in charge of, among others, the Windows headers.
(**)
This is because what "C" means binary is still dependant on the used C compiler, specially if there is no real universal system ABI. Think of stuff like:
C adding an underscore prefix on some binary formats (a.out, Coff?)
sometimes different C compilers have different opinions on what to do with small structures passed by value. Officially they shouldn't support it at all afaik, but most do.
structure packing sometimes varies, as do details of calling conventions (like skipping
integer registers or not if a parameter is registerable in a FPU register)
===== automated header conversions ====
While I don't know SWIG that well, I know and use some delphi specific header tools( h2pas, Darth/headconv etc).
However I never use them in fully automatic mode, since more often then not the output sucks. Comments change line or are stripped, and formatting is not retained.
I usually make a small script (in Pascal, but you can use anything with decent string support) that splits a header up, and then try a tool on relatively homogeneous parts (e.g. only structures, or only defines etc).
Then I check if I like the automated conversion output, and either use it, or try to make a specific converter myself. Since it is for a subset (like only structures) it is often way easier than making a complete header converter. Of course it depends a bit what my target is. (nice, readable headers or quick and dirty). At each step I might do a few substitutions (with sed or an editor).
The most complicated scheme I did for Winapi commctrl and ActiveX/comctl headers. There I combined IDL and the C header (IDL for the interfaces, which are a bunch of unparsable macros in C, the C header for the rest), and managed to get the macros typed for about 80% (by propogating the typecasts in sendmessage macros back to the macro declaration, with reasonable (wparam,lparam,lresult) defaults)
The semi automated way has the disadvantage that the order of declarations is different (e.g. first constants, then structures then function declarations), which sometimes makes maintenance a pain. I therefore always keep the original headers/sdk to compare with.
The Jedi winapi conversion project might have more info, they translated about half of the windows headers to Delphi, and thus have enormous experience.
I don't know but if it's for Windows then you might try either a straight C-like API (similar to the WINAPI), or packaging your code as a COM component: because I'd guess that programming languages might want to be able to invoke the Windows API, and/or use COM objects.
Regarding automatic wrapper generation, consider using SWIG. For Java, it will do all the JNI work. Also, it is able to translate complex OO-C++-interfaces properly (provided you follow some basic guidelines, i.e. no nested classes, no over-use of templates, plus the ones mentioned by Marco van de Voort).
Think C, nothing else. C is one of the most popular programming languages. It is widely used on many different software platforms, and there are few computer architectures for which a C compiler does not exist. All popular high-level languages provide an interface to C. That makes your library accessible from almost all platforms in existence. Don't worry too much about providing an Object Oriented interface. Once you have the library done in C, OOP, functional or any other style interface can be created in appropriate client languages. No other systems programming language will give you C's flexibility and potability.
NestedVM I think is going to be slower than pure Java because of the array bounds checking on the int[][] that represents the MIPS virtual machine memory. It is such a good concept but might not perform well enough right now (until phone manufacturers add NestedVM support (if they do!), most stuff is going to be SLOW for now, n'est-ce pas)? Whilst it may be able to unpack JPEGs without error, speed is of no small concern! :)
Nothing else in what you've written sticks out, which isn't to say that it's right or wrong! The principles sound (mainly just listening to choice of words and language to be honest) like roughly standard best practice but I haven't thought through the details of everything you've said. As you said yourself, this really ought to be several questions. But of course doing this kind of thing is not automatically easy just because you're fixed on perhaps a slightly different architecture to the last code base you've worked on...! ;)
My thoughts:
All your comments on C interface compatibility sound sensible to me, pretty much best practice except you don't seem to properly address memory management policy - some sentences a bit ambiguous/vague/wrong-sounding. The design of the memory management will be to a large extent determined by the access patterns made in your application, rather than the functionality per se. I suiggest you study others' attempts at making portable interfaces like the standard ANSI C API, Unix API, Win32 API, Cocoa, J2SE, etc carefully.
If it was me, I'd write the library in a carefully chosen subset of the common elements of regular Java and Davlik virtual machine Java and also write my own custom parser that translates the code to C for platforms that support C, which would of course be most of them. I would suggest that if you restrict yourself to data types of various size ints, bools, Strings, Dictionaries and Arrays and make careful use of them that will help in cross-platform issues without affecting performance much most of the time.
your assumptions seem ok, but i see trouble ahead, much of which you have already spotted in your assumptions.
As you said, you can't really export c++ classes and methods, you will need to provide a function based c interface. What ever facade you build around that, it will remain a function based interface at heart.
The basic problem i see with that is that people choose a specific language and its runtime because their way of thinking (functional or object oriented) or the problem they address (web programming, database,...) corresponds to that language in some way or other.
A library implemented in c will probably never feel like the libraries they are used to, unless they program in c themselves.
Personally, I would always prefer a library that "feels like python" when I use python, and one that feels like java when I do Java EE, even though I know c and c++.
So your effort might be of little actual use (other than your gain in experience), because people will probably want to stick with their mindset, and rather re-implement the functionality than use a library that does the job, but does not fit.
I also fear the desired portability will seriously hamper development. Just think of the infinite build settings needed, and tests for that. I have worked on a project that tried to maintain compatibility for 5 operating systems (all posix-like, but still) and about 10 compilers, the builds were a nightmare to test and maintain.
Give it an XML interface, whether passed as a parameter and return value or as files through a command-line invocation. This may not seem as direct as a normal function interface, but is the most practical way to access an executable from, e.g., Java.
This might be a dumb question to some of you and maybe I asked this question wrong, because I am new to C++. But I notice when working in a lot of Win32 applications, you use a lot of resources that are pointers. Why do you have to always acquire a objects pointer? Why not initiate a new instance of the class. and speaking of that, I notice in most cases you never initiate new objects, but always call on methods that return that pointer. What if that pointer is being used somewhere else. Couldn't you mess something up if you alter that pointer and it is being used somewhere else?
Windows APIs were designed for C, which was and still is the most used language for system programming; C APIs are the de-facto standard for system APIs, and for this almost all other languages had and have some way to call external C functions, so writing a C API helps to be compatible with other languages.
C APIs need just a simple ABI, that consists of almost just the definition for the calling convention to use for functions (and something about the structures layout). C++ and other object oriented languages, on the contrary, require a complex ABI, that must define how objects are laid out in memory, how to handle inheritance, how to lay out the vtable, how to propagate exceptions, where to put RTTI data, ... Moreover not all the languages are object oriented, and using APIs thought for C++ with other non-object oriented languages may be a real pain (if you ever used COM from C you know what I mean).
As an aside, when Windows was initially designed C++ wasn't so widespread on PCs, and also C wasn't used so much: actually, a large part of Windows 3.11 and many applications were still written in assembly, since the memory and CPU constraints at the era were very tight; compilers were also less smart than now are, especially C++ ones. On machines where hand-forged assembly was often the only solution, the C++ overhead was really unacceptable.
For the pointers thing: Windows APIs use almost always handles, i.e. opaque pointers, to be able to change the underlying nature of every resource without affecting the existing applications and to stop applications to mess around with internal structures. It doesn't matter if the structure used by the window manager to represent a window internally is changed: all the applications use simply an HWND, which is always of the size of a pointer. You may think at this as some kind of PIMPL idiom.
However, Windows is in some way object-oriented (see for example the whole "window class" concept, or, at a deeper level, the inner working of the NT kernel, which is heavily based on the "object" concept), however its most basic APIs, being simple C functions, somehow hide this OO nature. The shell, on the other side, being designed many years after, is written mostly in C++ and it provides a really object-oriented COM interface.
Interestingly, you can see in COM all the tradeoffs that you must face in building a cross-language but still C++ biased object-oriented interface: the result is quite complicated, in some respects ugly and not really simple to use from any language. The Windows APIs, instead, being simple functions are generally more easily callable.
If you are interested in a system based on C++ APIs you may have a look at Haiku; personally, this is one of the aspects because of which I am quite interested in that project.
By the way, if you are going to do Win32 programming just with the APIs you'd better get a good book to get used to these "particularities" and to other Win32 idioms. Two well-known ones are the Rector-Newcomer and the Petzhold.
Because Win32 Api are written on plain C, not C++. So any program on almost any language can make call to those API.
Plus, there are not simple mechanism to use objects across diferent modules, and diferent languages. I.e. you can't export C++ class to python. Of course, there are technoligies like OLE/COM, but they still written on plain C. And they are bit comlicated to use.
At other hand - calls to plain C functions are standardized. So you can call routines from DLL or static lib in any language.
Win32 was designed to work with the C language not C++.
That's why you will see return types of the defined BOOL instead of bool for example.
bool is specific to C++ and doesn't exist in C.
For Microsoft's object oriented wrapper of Win32 see MFC.
A newer framework from Microsoft since then is the .Net Framework.
The .Net framework is based on managed code though and does not run natively. The most modern way to do GUI programming on Windows is WPF or even Silverlight.
The most modern way to do unmanaged GUI programming is still using MFC, although some people still prefer using straight Win32.
Note working with pointers is not specific to C, it is still very common in C++.
First reason is because passing pointers around is cheap. Pointers are 4 bytes on x86 and 8 bytes on x64. While the structure or class it points to can occupy a whole lot more in memory. So instantiating a class means reserving new memory again and again. This isn't efficient from speed and memory consumption POVs.
Another way is to pass references to objects or smart pointers or similar constructs. But win32 api was designed in the C era, so this is where it is up to now ;)
As about potential messing up with pointers - it's possible of course. But most of the time their lifetime is explicitly stated in the API (if not obvious).
Probably because the Win32 API is "older" than mainstream object-oriented programming, it's not a C++ API at its core.
Its almost like you should try one of the many OO wrappers. Like MFC or .net.
The Windows API is plain old C, hence the use of pointers everywhere. Also, the reason you ask Windows for a new pointer is because Windows needs to keep track of all objects... it allocates things and tells you a pointer (or sometimes just a numeric ID) to let you work with them.
Having C functions as the API allows both C and C++ programmers use it.
Windows APIs goes way back - C was famous during those days.
All the HWND, HANDLE, HDC are but a weak attempt to make clamped, objects-like data types (using struct). C FAQ has a question on this -> http://c-faq.com/struct/oop.html.
To understand the pointers you might want to read the CPlusPlus.com tutorial on pointers.
I have heard that C++ offers no native support for multithreading. I assume that multithreaded C++ apps depended on managed code for multithreading; that is, for example, a Visual C++ app used MFC or .NET or something along those lines to provide multithreading capability. I further assume that some or all of those managed-code capabilities are unavailable to unmanaged applications. But I have read about unmanaged multithreaded applications. How is this possible? Which of my assumptions is false?
It is wholly up to the operating system to provide support for multi-threading. On Windows, the necessary functionality is available via the Win32 API. Frameworks such as MFC provide wrappers over the low-level threading functions to simplify things, while of course .NET/CLR has its own managed interface for accessing Win32 multi-threading capabilities.
A good explanation is offerred in this article (Multithreading in C++).
Why Doesn’t C++ Contain Built-In
Support for Multithreading?
C++ does not contain any built-in
support for multithreaded
applications. Instead, it relies
entirely upon the operating system to
provide this feature. Given that both
Java and C# provide built-in support
for multithreading, it is natural to
ask why this isn’t also the case for
C++. The answers are efficiency,
control, and the range of applications
to which C++ is applied. Let’s examine
each.
By not building in support for
multithreading, C++ does not attempt
to define a “one size fits all”
solution. Instead, C++ allows you to
directly utilize the multithreading
features provided by the operating
system. This approach means that your
programs can be multithreaded in the
most efficient means supported by the
execution environment. Because many
multitasking environments offer rich
support for multithreading, being able
to access that support is crucial to
the creation of high-performance,
multithreaded programs.
Multithreading in C++ does not require managed code.
In very much the same way that C++ does not provide native support for displaying graphics or emitting sounds or reading input from a mouse, the operating system that's being used will provide a C++ API for utilizing these features.
It's not a matter of C++ not being able to do it. It simply hasn't been written into the C++ standard yet.
Some of your assumptions are not quite right. The operating system (I'm talking about win32 since you mention .NET) provides support for threading. There are lots of good threading libs. that build ontop of the OS functionality in C++ to make multithreading "easier" :) -- pthreads for example. Here is more at MSDN.
The ISO standard for the programming language C++ neither defines nor prohibits multithreading. An implementation is allowed to provide extensions if it wishes. A program is allowed to use implementation extensions if it wishes, and then the program will only run on systems that provide those extensions.
For comparison, the ISO standard for the programming language C++ neither defines nor prohibits the use of a mouse. A program is allowed to use implementation extensions and then it will only run on systems that provide those extensions. For another comparison, the ISO standard for C++ neither defines nor prohibits UTF-8, so your program can depend on Latin-1 and then your program will only run on systems that provide Latin-1.
Native C++ does not offer "built in" multithreading support simply because it was not intended to, or in fact, needed.
Your misconception is that this is a fault, while it is in fact a strength of the language. By being "oblivious" to multithreading, C++ seamlessly integrates with the MT support offered by the OS your code will compile and run on, thereby offering much more flexibility and efficiency than if it came with it's own "MT baggage" so to speak.
You mention MFC and .NET as examples - be aware that these libraries/wrappers are merely a layer over basic Win32 API's. Using C++ as intended will provide you with efficient code that will run multithrededly on ANY OS, as long as you seperate the logic from the OS-specific MT API calles (i.e thread creation etc), so that porting between OS's is greatly facilitated.
Unlike Java, which defined language constructs and JVM specs, the C++ standard is oblivious to threading (so is C). As far as these languages are concerned, anything thread-related consists of function calls to OS functionality. Libraries compiled for multithreading simply make sense of the same calls, but from a language perspectives they are plain old code.
I think you misunderstand the definition of 'managed' code. 'Managed' code is a Microsoft-specific term meaning code that is uses the .NET framework and thus is subject to the various aspects of .NET. 'Unmanaged' code means code that runs outside that and does not operate through the .NET layer. MFC code is 'unmanaged'; it's merely a spectacularly bad wrapper for the ubiquitous Win32 API (which isn't even the lowest level API available on Windows).
The .NET libraries (including multithreading) are almost all, at some level, interfaces for the more basic system APIs used by traditional, 'unmanaged' applications. There is, generally speaking, no functionality available to 'managed' code that cannot be replicated in 'unmanaged' code with sufficient effort, though the reverse is not true (this is called the abstraction penalty, if you wish to know more). While it may be easier to do in 'managed' code, that's just because somewhere, some 'unmanaged' code is doing it for you, more or less. In the case of a threading API, it is in turn an interface to the operating system kernel, which itself accesses the processor's capabilities to allow a process to run in multiple places concurrently (if using multiple cores; if not, then it's just a pseudo-concurrency).
The C++ standard currently provides no definition of threads (the upcoming C++1x standard does). There are a number different threading libraries available, including those provided by Win32 and MFC, the pthreads library found on POSIX systems, and Boost.Thread, which will use the platform's local threading library.
The next C++ standard (named c++0x) will have support for multithreading.
Will include atomic operations.