Related
While developing a new product, we decided to go for a mix of C++ and C#, haven been told that bridging them to allow the C# code to call the C++ code would be easy (spoiler, it's not).
We're pretty experienced C++ programmers and not at all C# programmers so we pretty much just had to believe what we've read. A few attempts to call C and Objective-C was promising and we even found a few articles that showed how to make an unmanaged C++ class available in C# -- or at least we thought. The C++ code in the articles, wasn't C++, but instead the horrible monster C++/CLI that Microsoft seems to think is C++. Since we're doing the C# stuff to get some bits "for free" in macOS and Windows, C++/CLI isn't an option either :-(.
Anyway, plenty of people have claimed that it's easy to call C++ code from some specific programming language, but so far, I haven't seen a single one that will allow me to do so (I haven't been paying too much attention to this, so please provide me with an obvious example). C++ invariably always means C with no C++ stuff at all; no namespaces, classes, no stl, lambdas or anything. Just plain dumb C.
So, are there any languages, besides C++(/CLI) that will allow me to do the following:
Create an instance of a class, using a C++ constructor, and dispatch it with a C++ destructor.
Call member functions on an object ( foo f; f.foo();) with a C++ class.
Use std::vector, std::find_if, std::string and other stuff from the stl. Complete coverage of the stl is not required.
Use overloaded functions (i.e. f(), f(int), f(std::string))
Use overloaded operators (foo operator + (foo, foo))
Use C++11, C++14 and/or C++17 features.
Use RAII (rather important IMHO).
Use namespaces.
No. There is no such language.
Unless you count Objective-C++. But that falls pretty much in the same bucket as C++/CLI, in being C++ with some extensions. And C++/CX is another such beast.
There are some interop tools that can work with C++ classes (SWIG for example), but I have never heard of a tool that is capable of instantiating C++ templates (like vector or find_if) on demand.
What languages will call C++ with no explicit bridging?
The short answer to this question is: NONE
Remember that a programming language is a specification written in some technical report, usually in English. For examples, read n1570 (the spec of C11) or R5RS (the spec of Scheme). For C++, see n3337.
Actually, you are interested in implementations, e.g. in compilers and interpreters for your programming languages. These implementations are practically software. And then the answer might become: it depends (notably on the ABI used & targetted by your compiler and system).
See for examples this list of ABIs for Linux.
plenty of people have claimed that it's easy to call C++ code from some specific programming language,
The C calling conventions are quite common, and it might help to declare every C++ function callable from outside as extern "C". But there is no silver bullet, and details matter a lot.
So, are there any languages, besides C++(/CLI) that will allow me to do the following:
list of C++ features skipped
Probably not.
You probably need at least to understand more about memory management approaches. I recommend understanding more about garbage collection, e.g. by reading the GC handbook (at least for underlying concepts & terminology). Learn more about foreign function interfaces (in some cases, the libffi might help) and language bindings.
You might also consider generating some of the C++ or C glue code, maybe with SWIG (or write your own C++ glue code generator).
On operating systems providing dynamic linking capable of loading plugins at runtime (e.g. Linux with dlopen(3)/dlsym(3); but other OSes often have similar facilities) you could even consider generating some C or C++ glue code at runtime in some temporary file, compile it as a temporary plugin, and dynamically loading that plugin. You could also consider JIT-compiling libraries like GCCJIT or LLVM (or libjit).
I recommend reading SICP, the Dragon Book, and probably Lisp In Small Pieces. Of course, learn something about OSes, e.g. Operating Systems: Three Easy Pieces. Reading about Linkers and Loaders could also help.
As an excellent example of cleverly gluing C++, look into CLASP and see this video.
But whatever approach you take, you'll need a lot of work (years, not weeks).
C++ as a language does not have a defined ABI (Application Binary Interface) - which basically means that there is no universal standard of what a C++ class/function call/template would look like in binary form on any given platform, or across platforms.
What that means is that there is no universal way to call C++ code from other languages, on different platforms, or even across compilers on the same platform. It also means that the people who are telling you "it's easy to call C++ code from XYZ language" are mostly incorrect (or at least incredibly incomplete).
Where there are interfaces it's either because the provider of the interface controls the ABI (C++/CLI with .NET), or because there is a translation layer from C++ to something like the C calling convention (Boost::python).
Some work has been done towards attempting to define an ABI per-platform (http://open-std.org/JTC1/SC22/WG21/docs/papers/2014/n4028.pdf), but as far as I'm aware it has not yet been accepted into C++17.
You can look into using C++ interpreter, which allows for the fine-grained control you ask for. But I don't know of any that "just works", see also:
Have you used any of the C++ interpreters (not compilers)?
I would like to create a domain specific language as an augmented-C++ language. I will need mostly two types of contructs:
Top-level constructs for specialized types or declarations
In-code constructs, i.e. to add primitives to make functions calls or idiom easier
The language will be used for scientific computing purposes, and will ultimately be translated into plain C++. C++ has been chosen as it seems to offer a good compromise between: ease of use, efficiency and availability of a wide range of libraries.
A previous attempt using flex and bison failed due to the complexity of the C++ syntax. The existing parser can still fail on some constructs. So we want to start over, but on better bases.
Do you know about similar projects? And if you attempted to do so, what tools would you use? What would be the main pitfalls? Would you have recommendations in term of syntax?
There are many (clever) attempts to have domain specific languages within the C++ language.
It's usually called DSEL for Domain Specific Embedded Language. For example, you could look up the Boost.Spirit syntax, or Boost.rdb (in the boost vault).
Those are fully compliant C++ libraries which make use of C++ syntax.
If you want to hide some complexity, you might add in a few macros.
I would be happy to provide some examples if you gave us something to work with :)
You can try extending an open source Elsa C++ parser (it is now a part of a Mozilla's Pork project):
https://wiki.mozilla.org/Pork
The way to extend C++ is not to try to extend the language, which will be extremely difficult and probably break as new base compiler releases implement new features, but to write class libraries to support your problem domain. This has been what C++ programming has been all about since the language's inception.
If you really want to extend C++, you'll need a full C++ parser plus name and type resolution. As you've found out, this is pretty hard. Your best solution is to get an existing one and modify it.
Our DMS Software Reengineering Toolkit is an infrastructure for implementing langauge processors. It is
designed to support the construction of tools that parse languages, carry out transformations, and spit out the same language (with enhanced code) or a different language/dialect.
DMS has a full C++ Front End, that parses C++, builds abstract syntax trees and symbol tables (e.g., all that name and type resolution stuff).
The DMS/C++ front end is provided with DMS in source form, so that it can be customized to achieve the kind of effect you want. You'd define your DSL as an extension of the C++ front end, and then write transformations that convert your special constructs into "vanilla" C++ constructs, and then spit out compilable result.
DMS/C++ have been used for a wide variety of transformation tasks, including ones that involved extending C++ as you've described, and including tasks that carry out massive reorganizations of large C++ applications. (See the Publications at that website).
To solve you first bullet, maybe you can use C++0x new features "initializer lists", and "user defined litterals" avoiding the need for a new parser. They may help for the second bullet, too.
This may seem a trivial question, but it's one that's bothered me a lot lately. Why do some programmers refer to "C++/STL" like it's a different language? The STL is part of the C++ standard library -- and therefore is part of the language, "C++". It's not a separate component, and it does not live alone in the scope of things C++. Yet some continually act like it's a different language altogether. Why?
It's possible to be a competent and experienced C++ programmer and never use the STL. You may be using Boost or ACE, or been an MFC windows programmer for 10 years.
If you want someone experienced in using the STL, asking for someone who knows C++ is no guarantee that you'll get one.
Also for my mind, writing code that's heavily dependent on the STL feels very different to writing, say, MFC code. They might as well be different languages. They certainly won't look particularly similar.
An understanding of the STL isn't necessary to understand C++. It's useful to have when you need ADTs, but you can go (could have gone?) through your whole C++ career without needing it.
The above answers are really good; I'm only going to add to their content in a broader context.
Developers might refer to language/api|library e.g. C/Win32, Java/Struts, Java/Spring, C#/.net MVC because there are in essence two knowledge bases - knowledge of the language in question and knowledge of how to use that specific library, API or framework. Something like Win32 is pretty huge, as is say Django, which I'm currently learning. Django itself works in a very specific way and knowing that is what I'm learning, not Python.
The same is true of C++/MFC or C++/Boost or C++/STL. The language is C++ - the API/library you're using is MFC, Boost or STL.
Probably because STL came a little late to the C++ game, and many people have written code that does not use any STL. For example, think early win32 programming with MFC.
Guess:
When C++ was first released, the STL did not exist. It came into existence later as an optional addition and then was incorporated into the standard.
When writing a resume, people would often list C/C++ as a language, which, in many cases means they don't know either.
Sometime resumes would list "Visual C++" as a language, trying to indicate they don't know what a language is.
This, together with "great knowledge of C++ and PHP" statements, go strait into recycle bin at my firm. Not because they are necessarily bad programmers - but because the interview time waste is not worth it.
The MFC has all class names that start with C. For example, CFile and CGdiObject. Has anyone seen it used elsewhere? Is there an official naming convention guide from Microsoft that recommends this style? Did the idea originate with MFC or was it some other project?
It's evil. Don't use Hungarian Notation for anything but abstracted things.
For instance, btnSubmit is ok to describe a button named Submit(which would have an accompanying lblSubmit for the label next to the button)
But things like CMyClass for Class and uiCount for unsigned integer named count does not help programmers and just leads to extra wasteful typing.
Something a bit similar is used in Symbian C++, where the convention is that:
T classes are "values", for example TChar, TInt32, TDes
R classes are handles to kernel (or other) resources, for example RFile, RSocket
M classes are mixins, which includes interfaces (construed as mixins with no function implementations). The guideline is that multiple inheritance should involve at most 1 non-M class.
C classes are pretty much everything else, and derive from CBase, which has some stuff in it to help with resource-handling.
HBufC exists primarily to generate confused posts on Symbian forums, and having its very own prefix is just the start. The H stands for "huh?", or possibly "Haw, haw! You have no STL!" ;-)
This is close in spirit to Apps Hungarian Notation rather than Systems Hungarian notation. The prefix tells you something about the class which you could look up in the documentation, but which you would not know otherwise. The whole point of naming anything in programming is to provide such hints and reminders, otherwise you'd just call your classes "Class001", "Class002", etc.
Systems Hungarian just tells you the type of a variable, which IMO is nothing to get very excited about, especially in a language like C++ where types tend to be either repeated constantly or else completely hidden by template parameters. Its analogue when naming types is the Java practice of naming all interfaces with I. Again, I don't get very excited about this (and neither do the standard Java libraries), but if you're going to define an interface for every class, in addition to the interfaces which are actually used for polymorphism in non-test situations, then you need some way to distinguish the two.
That was a old C++ coding style, and MFC was probably one of the last things to use it.
It was usually just a convention of C++ (and maybe a few other languages), and hence it started falling out of favor as the languages became more interoperable, through COM and then .NET.
You still see it's cousin, the "I" prefix for interfaces, pretty often. I've always found it interesting that "I" survived when "C" died, but that was probably because interfaces were used so heavily in COM interoperability.
I remember Borland compilers were comming with libraries where class names started with 'T'. Probably for "type" :)
I can't answer all your questions, but as far as I know, it's just to distinguish the MFC classes from other classes -- a form of Hungarian Notation.
Interestingly, it's apparently controversial not just outside MS, but inside as well.
Years ago naming convention is crucial to help identifying the class, type of even the grouping of the class. Dont forget back then there was no namespace and no/limited intellisense available. C is a form of Hungarian notation but certainly made popular by MFC. Borland and Delphi was using T - as prefix for Type
While MFC and lots of software written for Windows used the "C" convention for classes, you generally don't find the latter in software written for UNIX platforms. I think it was a habit very strongly encouraged by Visual C++. I remember that Visual C++ 6.0 would prefix a "C" to any classes that one created with the class wizard.
See here : http://www.jelovic.com/articles/stupid_naming.htm for a long article on this issue.
Such conventions for variables are useful for languages like Fortran where you don't need to declare the types of your variables before using them. I seem to recall that variables who's names started with "i" or "j" defaulted to integers, and variables who's names started with "r" and other letters defaulted to real (float) values.
That people use similar for languages where you do need to declare variables - or for class definitions - is probably just a relic of someone misunderstanding the old code conventions from languages like Fortran where it actually mattered.
When writing applications that use the Qt libraries, we use a naming convention that distinguishes classes that are directly or indirectly derived from QObject from classes that aren't. This is useful because you can tell from the class name whether or not it supports signals/slots, properties, and all the other goodies that come from QObject.
we use it at work, like many other naming conventions
by many I meant C for classes, p for pointer, m_ for members, s_ for static members, n for integer ... not many documents
personally I find that hungarian notation helps me, in that I can look at a screen full of variables and instantly know what they are as I try and absorb the logic. Your only argument against it I see is "extra typing"
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm planning to code a library that should be usable by a large number of people in on a wide spectrum of platforms. What do I have to consider to design it right? To make this questions more specific, there are four "subquestions" at the end.
Choice of language
Considering all the known requirements and details, I concluded that a library written in C or C++ was the way to go. I think the primary usage of my library will be in programs written in C, C++ and Java SE, but I can also think of reasons to use it from Java ME, PHP, .NET, Objective C, Python, Ruby, bash scrips, etc... Maybe I cannot target all of them, but if it's possible, I'll do it.
Requirements
It would be to much to describe the full purpose of my library here, but there are some aspects that might be important to this question:
The library itself will start out small, but definitely will grow to enormous complexity, so it is not an option to maintain several versions in parallel.
Most of the complexity will be hidden inside the library, though
The library will construct an object graph that is used heavily inside. Some clients of the library will only be interested in specific attributes of specific objects, while other clients must traverse the object graph in some way
Clients may change the objects, and the library must be notified thereof
The library may change the objects, and the client must be notified thereof, if it already has a handle to that object
The library must be multi-threaded, because it will maintain network connections to several other hosts
While some requests to the library may be handled synchronously, many of them will take too long and must be processed in the background, and notify the client on success (or failure)
Of course, answers are welcome no matter if they address my specific requirements, or if they answer the question in a general way that matters to a wider audience!
My assumptions, so far
So here are some of my assumptions and conclusions, which I gathered in the past months:
Internally I can use whatever I want, e.g. C++ with operator overloading, multiple inheritance, template meta programming... as long as there is a portable compiler which handles it (think of gcc / g++)
But my interface has to be a clean C interface that does not involve name mangling
Also, I think my interface should only consist of functions, with basic/primitive data types (and maybe pointers) passed as parameters and return values
If I use pointers, I think I should only use them to pass them back to the library, not to operate directly on the referenced memory
For usage in a C++ application, I might also offer an object oriented interface (Which is also prone to name mangling, so the App must either use the same compiler, or include the library in source form)
Is this also true for usage in C# ?
For usage in Java SE / Java EE, the Java native interface (JNI) applies. I have some basic knowledge about it, but I should definitely double check it.
Not all client languages handle multithreading well, so there should be a single thread talking to the client
For usage on Java ME, there is no such thing as JNI, but I might go with Nested VM
For usage in Bash scripts, there must be an executable with a command line interface
For the other client languages, I have no idea
For most client languages, it would be nice to have kind of an adapter interface written in that language. I think there are tools to automatically generate this for Java and some others
For object oriented languages, it might be possible to create an object oriented adapter which hides the fact that the interface to the library is function based - but I don't know if its worth the effort
Possible subquestions
is this possible with manageable effort, or is it just too much portability?
are there any good books / websites about this kind of design criteria?
are any of my assumptions wrong?
which open source libraries are worth studying to learn from their design / interface / souce?
meta: This question is rather long, do you see any way to split it into several smaller ones? (If you reply to this, do it as a comment, not as an answer)
Mostly correct. Straight procedural interface is the best. (which is not entirely the same as C btw(**), but close enough)
I interface DLLs a lot(*), both open source and commercial, so here are some points that I remember from daily practice, note that these are more recommended areas to research, and not cardinal truths:
Watch out for decoration and similar "minor" mangling schemes, specially if you use a MS compiler. Most notably the stdcall convention sometimes leads to decoration generation for VB's sake (decoration is stuff like #6 after the function symbol name)
Not all compilers can actually layout all kinds of structures:
so avoid overusing unions.
avoid bitpacking
and preferably pack the records for 32-bit x86. While theoretically slower, at least all compilers can access packed records afaik, and the official alignment requirements have changed over time as the architecture evolved
On Windows use stdcall. This is the default for Windows DLLs. Avoid fastcall, it is not entirely standarized (specially how small records are passed)
Some tips to make automated header translation easier:
macros are hard to autoconvert due to their untypeness. Avoid them, use functions
Define separate types for each pointer types, and don't use composite types (xtype **) in function declarations.
follow the "define before use" mantra as much as possible, this will avoid users that translate headers to rearrange them if their language in general requires defining before use, and makes it easier for one-pass parsers to translate them. Or if they need context info to auto translate.
Don't expose more than necessary. Leave handle types opague if possible. It will only cause versioning troubles later.
Do not return structured types like records/structs or arrays as returntype of functions.
always have a version check function (easier to make a distinction).
be careful with enums and boolean. Other languages might have slightly different assumptions. You can use them, but document well how they behave and how large they are. Also think ahead, and make sure that enums don't become larger if you add a few fields, break the interface. (e.g. on Delphi/pascal by default booleans are 0 or 1, and other values are undefined. There are special types for C-like booleans (byte,16-bit or 32-bit word size, though they were originally introduced for COM, not C interfacing))
I prefer stringtypes that are pointer to char + length as separate field (COM also does this). Preferably not having to rely on zero terminated. This is not just because of security (overflow) reasons, but also because it is easier/cheaper to interface them to Delphi native types that way.
Memory always create the API in a way that encourages a total separation of memory management. IOW don't assume anything about memory management. This means that all structures in your lib are allocated via your own memory manager, and if a function passes a struct to you, copy it instead of storing a pointer made with the "clients" memory management. Because you will sooner or later accidentally call free or realloc on it :-)
(implementation language, not interface), be reluctant to change the coprocessor exception mask. Some languages change this as part of conforming to their standards floating point error(exception-)handling.
Always pair a callbacks with an user configurable context. This can be used by the user to give the the callback state without defining global variables. (like e.g. an object instance)
be careful with the coprocessor status word. It might be changed by others and break your code, and if you change it, other code might stop working. The status word is generally not saved/restored as part of calling conventions. At least not in practice.
don't use C style varargs parameters. Not all languages allow variable number of parameters in an unsafe way
(*) Delphi programmer by day, a job that involves interfacing a lot of hardware and thus translating vendor SDK headers. By night Free Pascal developer, in charge of, among others, the Windows headers.
(**)
This is because what "C" means binary is still dependant on the used C compiler, specially if there is no real universal system ABI. Think of stuff like:
C adding an underscore prefix on some binary formats (a.out, Coff?)
sometimes different C compilers have different opinions on what to do with small structures passed by value. Officially they shouldn't support it at all afaik, but most do.
structure packing sometimes varies, as do details of calling conventions (like skipping
integer registers or not if a parameter is registerable in a FPU register)
===== automated header conversions ====
While I don't know SWIG that well, I know and use some delphi specific header tools( h2pas, Darth/headconv etc).
However I never use them in fully automatic mode, since more often then not the output sucks. Comments change line or are stripped, and formatting is not retained.
I usually make a small script (in Pascal, but you can use anything with decent string support) that splits a header up, and then try a tool on relatively homogeneous parts (e.g. only structures, or only defines etc).
Then I check if I like the automated conversion output, and either use it, or try to make a specific converter myself. Since it is for a subset (like only structures) it is often way easier than making a complete header converter. Of course it depends a bit what my target is. (nice, readable headers or quick and dirty). At each step I might do a few substitutions (with sed or an editor).
The most complicated scheme I did for Winapi commctrl and ActiveX/comctl headers. There I combined IDL and the C header (IDL for the interfaces, which are a bunch of unparsable macros in C, the C header for the rest), and managed to get the macros typed for about 80% (by propogating the typecasts in sendmessage macros back to the macro declaration, with reasonable (wparam,lparam,lresult) defaults)
The semi automated way has the disadvantage that the order of declarations is different (e.g. first constants, then structures then function declarations), which sometimes makes maintenance a pain. I therefore always keep the original headers/sdk to compare with.
The Jedi winapi conversion project might have more info, they translated about half of the windows headers to Delphi, and thus have enormous experience.
I don't know but if it's for Windows then you might try either a straight C-like API (similar to the WINAPI), or packaging your code as a COM component: because I'd guess that programming languages might want to be able to invoke the Windows API, and/or use COM objects.
Regarding automatic wrapper generation, consider using SWIG. For Java, it will do all the JNI work. Also, it is able to translate complex OO-C++-interfaces properly (provided you follow some basic guidelines, i.e. no nested classes, no over-use of templates, plus the ones mentioned by Marco van de Voort).
Think C, nothing else. C is one of the most popular programming languages. It is widely used on many different software platforms, and there are few computer architectures for which a C compiler does not exist. All popular high-level languages provide an interface to C. That makes your library accessible from almost all platforms in existence. Don't worry too much about providing an Object Oriented interface. Once you have the library done in C, OOP, functional or any other style interface can be created in appropriate client languages. No other systems programming language will give you C's flexibility and potability.
NestedVM I think is going to be slower than pure Java because of the array bounds checking on the int[][] that represents the MIPS virtual machine memory. It is such a good concept but might not perform well enough right now (until phone manufacturers add NestedVM support (if they do!), most stuff is going to be SLOW for now, n'est-ce pas)? Whilst it may be able to unpack JPEGs without error, speed is of no small concern! :)
Nothing else in what you've written sticks out, which isn't to say that it's right or wrong! The principles sound (mainly just listening to choice of words and language to be honest) like roughly standard best practice but I haven't thought through the details of everything you've said. As you said yourself, this really ought to be several questions. But of course doing this kind of thing is not automatically easy just because you're fixed on perhaps a slightly different architecture to the last code base you've worked on...! ;)
My thoughts:
All your comments on C interface compatibility sound sensible to me, pretty much best practice except you don't seem to properly address memory management policy - some sentences a bit ambiguous/vague/wrong-sounding. The design of the memory management will be to a large extent determined by the access patterns made in your application, rather than the functionality per se. I suiggest you study others' attempts at making portable interfaces like the standard ANSI C API, Unix API, Win32 API, Cocoa, J2SE, etc carefully.
If it was me, I'd write the library in a carefully chosen subset of the common elements of regular Java and Davlik virtual machine Java and also write my own custom parser that translates the code to C for platforms that support C, which would of course be most of them. I would suggest that if you restrict yourself to data types of various size ints, bools, Strings, Dictionaries and Arrays and make careful use of them that will help in cross-platform issues without affecting performance much most of the time.
your assumptions seem ok, but i see trouble ahead, much of which you have already spotted in your assumptions.
As you said, you can't really export c++ classes and methods, you will need to provide a function based c interface. What ever facade you build around that, it will remain a function based interface at heart.
The basic problem i see with that is that people choose a specific language and its runtime because their way of thinking (functional or object oriented) or the problem they address (web programming, database,...) corresponds to that language in some way or other.
A library implemented in c will probably never feel like the libraries they are used to, unless they program in c themselves.
Personally, I would always prefer a library that "feels like python" when I use python, and one that feels like java when I do Java EE, even though I know c and c++.
So your effort might be of little actual use (other than your gain in experience), because people will probably want to stick with their mindset, and rather re-implement the functionality than use a library that does the job, but does not fit.
I also fear the desired portability will seriously hamper development. Just think of the infinite build settings needed, and tests for that. I have worked on a project that tried to maintain compatibility for 5 operating systems (all posix-like, but still) and about 10 compilers, the builds were a nightmare to test and maintain.
Give it an XML interface, whether passed as a parameter and return value or as files through a command-line invocation. This may not seem as direct as a normal function interface, but is the most practical way to access an executable from, e.g., Java.