I have being using stlport to develop wince based custom OS, but from now on I am thinking about using stl provided by windows. I read that functionally they are not different from each other so currently what matters is my image's size. Unfortunately I cannot give both of them a try like first use stl and make a run time image and then use stlport, then compare both images' sizes, because I have a lot of other problems that I need to solve in order to succesfully build the OS. Hence I wanted to get an expert idea:
Which one do you think would be more lightweight? I know how stlport is attached, loaded etc but I am not quite sure about STL. I looked into STL headers and all I saw were thousands of inline functions. But is that all? I need to be sure about it. Does STL link any other libraries inside or does it simply include the headers and use those inline functions?
Best
Ps: I am using VS2012 and working on wec2013
Ps2: I know what STL and stlport stands for and how to build an application by using them. My actual question is which one would consume less memory, use smaller size on HDD? (Considering things like stlport is a lib but stl is not etc.)
I assume that by STL you mean your compiler's standard library. This is a common misunderstanding, as STL was the original name of a library that was proposed and accepted into the language, but it has evolved from that. Taking this into account, the question becomes:
Should I use the standard library provided with my compiler or use stlport [or other alternatives]?
The answer is that it will depend on your use case, but the good thing is that as long as you use the library as defined in the standard (i.e. without extensions) then you should be able to easily switch from building with one or the other, and that means that you can test this yourself. You can also test building with different compiler flags. This is specially important in VS, as by default the library uses checked iterators, that are good for debugging but at the cost of extra memory and processing.
STLPort is designed to be used on platforms that does not provide STL for some reasons (for example, embedded platforms without C++ exceptions support), or native STL support is outdated.
So, usually you do not need to replace native STL. There should be strong reasons to use STLPort in your project. In my experience, I used it for some embedded DSP platforms (no native STL), and for a UEFI platform (not really embedded, but no native STL as well, also runtime does not support C++ exceptions).
STLPort is highly customizable (you can disable exceptions, streams, etc), and can be used on almost any platform with basic C++ support.
Related
I'm trying to understand the correct way, or right approach, to provide a reasonably large C++ API for a non-open source project. I do not want to provide a "header only" library as the code base is fairly large and meant to be closed source. The goals are as follows:
Provide a native C++ API that users may instantiate C++ classes, pass data around, all in C++, without a C-only wraper
Allow methods to take as parameters and return C++ objects, especially STL types (std::string, std::vector, etc)
No custom allocators
Most industry standard / canonical method of doing this possible, if such a standard exists
No re-creating COM or using MS COM
Assume all C++ compilers are at least C++11 "compliant"
I am targeting Windows as well as other platforms (Linux).
My understanding is creating a DLL or shared library is out of the question because of the DLL boundary issue. To deal with DLL boundary issue, the DLL and the calling code must be compiled with dynamically linked runtime (and the right version, multithreaded/debug/etc on Windows), and then all compiler settings would need to match with respect to debug symbols (iterator debugging settings, etc). One question I have this is whether, if, say on Windows, we ensure that the compiler settings match in terms of /MD using either default "Debug" and "Releas" settings in Visual Studio, can we really be "safe" in using the DLL in this way (that is passing STL objects back and forth and various things that would certainly be dangerous/fail if there was a mismatch)? Do shared object, *.so in Linux under gcc have the same problem?
Does using static libraries solve this problem? How much do compiler settings need to match between a static library and calling code for which it is linked? Is it nearly the same problem as the DLL (on Windows)?
I have tried to find examples of libraries online but cannot find much guidance on this. Many resources discuss Open Source solution, which seems to be copying header and implementation files into the code base (for non-header-only), which does not work for closed source.
What's the right way to do this? It seems like it should be a common issue; although I wonder if most commercial vendors just use C interfaces.
I am ok with static libraries if that solves the problem. I could also buy into the idea of having a set of X compilers with Y variations of settings (where X and Y are pre-determined list of options to support) and having a build system that generated X * Y shared binary libraries, if that was "safe".
Is the answer is really only to do either C interfaces or create Pure Abstract interfaces with factories? (if so, is there a canonical book or guide for doing this write, that is not implementing Microsoft COM?).
I am aware of Stefanus DuToit's Hourglass Pattern:
https://www.youtube.com/watch?v=PVYdHDm0q6Y
I worry that it is a lot of code duplication.
I'm not complaining about the state of things, I just want to understand the "right" way and hopefully this will serve as a good question for others in similar position.
I have reviewed these Stackoverflow references:
When to use dynamic vs. static libraries
How do I safely pass objects, especially STL objects, to and from a DLL?
Distributing Windows C++ library: how to decide whether to create static or dynamic library?
Static library API question (std::string vs. char*)
Easy way to guarantee binary compatibility for C++ library, C linkage?
Also have reviewed:
https://www.acodersjourney.com/cplusplus-static-vs-dynamic-libraries/
https://blogs.msmvps.com/gdicanio/2016/07/11/the-perils-of-c-interface-dlls/
Given your requirements, you'll need a static library (e.g. .lib under windows) that is linked into your program during the build phase.
The interface to that library will need to be in a header file that declares types and functions.
You might choose to distribute as a set of libraries and header files, if they can be cleanly broken into separate pieces - if you manage the dependencies between the pieces. That's optional.
You won't be able to define your own templated functions or classes, since (with most compilers) that requires distributing source for those functions or classes. Or, if you do, you won't be able to use them as part of the interface to the library (e.g. you might use templated functions/classes internally within the library, but not expose them in the library header file to users of the library).
The main down side is that you will need to build and distribute a version of the library for each compiler and host system (in combination that you support). The C++ standard specifically encourages various types of incompatibilities (e.g. different name mangling) between compilers so, generally speaking, code built with one compiler will not interoperate with code built with another C++ compiler. Practically, your library will not work with a compiler other than the one it is built with, unless those compilers are specifically implemented (e.g. by agreement by both vendors) to be compatible. If you support a compiler that supports a different ABI between versions (e.g. g++) then you'll need to distribute a version of your library built with each ABI version (e.g. the most recent version of the compiler that supports each ABI)
An upside - which follows from having a build of your library for every compiler and host you support - is that there will be no problem using the standard library, including templated types or functions in the standard library. Passing arguments will work. Standard library types can be member of your classes. This is simply because the library and the program using it will be built with the same compiler.
You will need to be rigorous in including standard headers correctly (e.g. not relying on one standard header including another, unless the standard says it actually does - some library vendors are less than rigorous about this, which can cause your code to break when built with different compilers and their standard libraries).
There will mostly be no need for "Debug" and "Release" versions (or versions with other optimisation settings) of your library. Generally speaking, there is no problem with having parts of a program being linked that are compiled with different optimisation settings. (It is possible to cause such things to break,if you - or a programmer using your library in their program - use exotic combinations of build options, so keep those to a minimum). Distributing a "Debug" version of your library will permit stepping through your library with a debugger, which seems counter to your wishes.
None of the above prevents you using custom allocators, but doesn't require it either.
You will not need to recreate COM unless you really want to. In fact, you should aim to ensure your code is as standard as possible - minimise use of compiler-specific features, don't make particular assumptions about sizes of types or layout of types, etc. Using vendor specific features like COM is a no-no, unless those features are supported uniformly by all target compilers and systems you support. Which COM is not.
I don't know if there is a "standard/canonical" way of doing this. Writing code that works for multiple compilers and host systems is a non-trivial task, because there are variations between how different compiler vendors interpret or support the standard. Keeping your code simple is best - the more exotic or recent the language or standard library feature you use, the more likely you are to encounter bugs in some compilers.
Also, take the time to set up a test suite for your library, and maintain it regularly, to be as complete as possible. Test your library with that suite on every combination of compiler/system you support.
Provide a native C++ API that users may instantiate C++ classes, pass data around, all in C++, without a C-only wraper
This excludes COM.
Allow methods to take as parameters and return C++ objects, especially STL types (std::string, std::vector, etc)
This excludes DLLs
Most industry standard / canonical method of doing this possible, if such a standard exists
Not something "standard", but common practises are there. For example, in DLL, pass only raw C stuff.
No re-creating COM or using MS COM
This requires DLL/COM servers
Assume all C++ compilers are at least C++11 "compliant"
Perhaps. Normally yes.
Generally: If source is to be available, use header only (if templates) or h +cpp.
If no source, the best is DLL. Static libraries - you have to build for many compilers and one has to carry on your lib everywhere and link to it.
On linux, gcc uses libstdc++ for the c++ standard library while clang can use libc++ or libstdc++. If your users are building with clang and libc++, they won't be able to link if you only build with gcc and libstdc++ (libc++ and libstdc++ are not binary compatible). So if you wan't to get target both you need two version of you library, one for libstdc++, and another for libc++.
Also, binaries on linux (executable, static library, dynamic library) are not binary compatible between distros. It might work on your distro and not work on someone else distro or even a different version of your distro. Be super careful to test that it work on whichever distro you want to target. Holy Build Box could help you to produce cross-distribution binaries. I heard good thing about it, just never tried it. Worst case you might need to build on all the linux distro you want to support.
https://phusion.github.io/holy-build-box/
For our current project, we are thinking to use Boost framework.
However, the project should be truly cross-platform and might be shipped to some exotic platforms. Therefore, we would like to use only Boost packages (libraries) that does not contain any platform specific code: pure C++ and that's all.
Boost has the idea of header-only packages (libraries).
Can one assume that these packages (libraries) are free from platform specific code?
In case if not, is there a way to identify these kind of packages of Boost?
All C++ code is platform-specific to some extent. On the one side, there is this ideal concept of "pure standard C++ code", and on the other side, there is reality. Most of the Boost libraries are designed to maintain the ideal situation on the user-side, meaning that you, as the user of Boost, can write platform-agnostic standard C++ code, while all the underlying platform-specific code is hidden away in the guts of those Boost libraries (for those that need them).
But at the core of this issue is the problem of how to define platform-specific code versus standard C++ code in the real world. You can, of course, look at the standard document and say that anything outside of it is platform-specific, but that's nothing more than an academic discussion.
If we start from this scenario: assume we have a platform that only has a C++ compiler and a C++ standard library implementation, and no other OS or OS-specific API to rely on for other things that aren't covered by the standard library. Well, at that point, you still have to ask yourself:
What compiler is this? What version?
Is the standard library implementation correct? Bug-free?
Are those two entirely standard-compliant?
As far as I know, there is essentially no universal answer to this and there are no realistic guarantees. Most exotic platforms rely on exotic (or old) compilers with partial or non-compliant standard library implementations, and sometimes have self-imposed restrictions (e.g., no exceptions, no RTTI, etc.). An enormous amount of "pure standard C++ code" would never compile on these platforms.
Then, there is also the reality that most platforms today, even really small embedded systems have an operating system. The vast majority of them are POSIX compliant to some level (except for Windows, but Windows doesn't support any exotic platform anyways). So, in effect, platform-specific code that relies on POSIX functions is not really that bad since it is likely that most exotic platforms have them, for the most part.
I guess what I'm really getting at here is that this pure dividing line that you have in your mind about "pure C++" versus platform-specific code is really just an imaginary one. Every platform (compiler + std-lib + OS + ext-libs) lies somewhere along a continuum of level of support for standard language features, standard library features, OS API functions, and so on. And by that measure, all C++ code is platform-specific.
The only real question is how wide of a net it casts. For example, most Boost libraries (except for recent "flimsy" ones) generally support compilers down to a reasonable level of C++98 support, and many even try to support as far back as early 90s compilers and std-libs.
To know if a library, part of Boost or not, has wide enough support for your intended applications or platforms, you have the define the boundaries of that support. Just saying "pure C++" is not enough, it means nothing in the real world. You cannot say that you will be using C++11 compilers just after you've taken Boost.Thread as an example of a library with platform-specific code. Many C++11 implementations have very flimsy support for std::thread, but others do better, and that issue is as much of a "platform-specific" issue as using Boost.Thread will ever be.
The only real way to ever be sure about your platform support envelope is to actual set up machines (e.g., virtual machines, emulators, or real hardware) that will provide representative worst-cases. You have to select those worst-case machines based on a realistic assessment of what your clients may be using, and you have to keep that assessment up to date. You can create a regression test suite for your particular project, that uses the particular (Boost) libraries, and test that suite on all your worst-case test environments. Whatever doesn't pass the test, doesn't pass the test, it's that simple. And yes, you might find out in the future that some Boost library won't work under some new exotic platform, and if that happens you need to either get the Boost dev-team to add code to support it, or you have to re-write your code to get around it, but that's what software maintenance is all about, and it's a cost you have to anticipate, and such problems will come not only from Boost, but from the OS and from the compiler vendors too! At least, with Boost, you can fix the code yourself and contribute it to Boost, which you can't always do with OS or compiler vendors.
We had "Boost or not" discussion too. We decided not to use it.
We had some untypical hardware platforms to serve with one source code. Especially running boost on AVR was simply impossible because RTTI and exceptions, which Boost requires for a lot of things, aren't available.
There are parts of boost which use compiler specific "hacks" to e.g. get information about class structure.
We tried splitting the packages, but the inter dependency is quite high (at least 3 or 4 years ago).
In the meantime, C++11 was underway and GCC started supporting more and more. With that many reasons to use from boost faded (Which Boost features overlap with C++11?). We implemented the rest of our needs from scratch (with relative low effort thanks to variadic templates and other TMP features in C++11).
After a steep learning curve we have all we need without external libraries.
At the same time we have pondered the future of Boost. We expected the newly standardized C++11 features would be removed from boost. I don't know the current roadmap for Boost, but at the time our uncertainty made us vote against Boost.
This is not a real answer to your question, but it may help you decide whether to use Boost. (And sorry, it was to large for a comment)
With the advent of threading facilities in the STL for the new C++ standard (C++0x), will it be better to change existing code that is using POSIX threading or even Windows threading to use STL threading?
You could always hedge your bets... write your own simple threading API that is just featureful enough to do what your application needs to be done, and change your code to target your threading API only. Then you can implement the internals of your custom threading API using Windows or Posix or STL or whatever, and change the implementation whenever you need to without having to touch your entire codebase each time.
By doing it that way, you could start with the STL implementation, and then if it turns out that e.g. Windows has a difficult-to-resolve problem using that, you could just include an alternate Windows-API implementation inside my_threading_api.cpp (inside of an #ifdef WIN32) and you'd be back in business.
A great deal will depend on how much you care about portability, and how much you're taking advantage of features your native API may provide that the standard library doesn't. The standard library threading is sufficiently similar to POSIX that (at least offhand) I can't think of much you'd gain by staying with POSIX (and due to the similarity, porting to use the standard library should usually be pretty easy).
Windows threading is enough different that you're more likely to run into something that will be non-trivial to port to using the standard library, and even at best porting will probably be non-trivial.
Don't change unless you really need it. I assume that your existing code is working well.
There's no telling how long it will be before the C++0x library features are commonly supported, so the answer might well depend on how tied to a particular compiler you might want to be.
You might also want to consider a framework or library that works on top of the native or C++0x library implementation, such as Boost Threads or the Intel Threading Building Blocks and let that library handle the details of whether it's using C++0x features or platform APIs.
It depends.
C++0x threading isn't widely supported yet (I think GCC implements it, but MSVC doesn't, and I don't know when they're planning to add that support, but I might suspect they consider it a low priority feature)
If your code works as it is, why change it?
For new C++ applications, and assuming compiler support, I'd go with C++0x threads, simply because they're standard, and it's a much nicer API than either Win32 or POSIX threads.
I need to develop a Windows/Linux command line utility. The utility will talk to middleware that has a standard API on both platforms. I have done some cross-platform development before, on FreeBSD/Linux, which was considerably easier - and I had people in the group with experience that I could talk to.
At this point there is no one in my group who has tackled a Windows/Linux development project. I am looking for advice on how to best set it up. I'm kind of a newbie to C++ too, I have mostly developed C#/.Net GUI applications and Linux device driver level "stuff". Kind of a weird mix.
I was thinking that it would be best to define my own data types and not use either the Linux or the Windows defined types - keep the OS specific code in separate folders and include conditionally. That's kind of what we did for the Linux/BSD work. So it seemed like a good start.
One of the developers here is a big fan of Boost... another thought the TCLAP command line parser library was easier to use... Obviously everything has to be compatible with the licensing.
The code will be open sourced, but it is production code - so I don't want to be sloppy. What else should I be doing or looking for? Are there any best practices out there?
Boost is good, as is ACE. Between the 2 of those they cover pretty much anything you would want to do in a cross-platform manner. I have also gone the route of getting posix libraries for windows and using gcc on cygwin, but I don't recommend it.
Use a portable runtime that is supported on both platforms. I have had good luck with the Apache Portable Runtime.
Use standard C or C++ for most of the project. Only use platform specific functions when necessary. If possible, put those in a wrapper in isolated files so that the build (makefile) can substitute in the correct version for the appropriate platform.
Refrain from using #ifdef LINUX or #ifdef WINDOWS or similar conditional compilation. Those get really hard to debug and there are error prone when the keyword is not supplied to the compiler.
Use Boost. Among other things, you'll get a portable implementation of a subset of TR1, which is worth it if only for <cstdint> and the types within - int32_t etc. As well, shared_ptr is essential for many moderately complicated data structures.
Boost also has a slew of helper types which are extremely convenient in day-to-day C++ tasks. Two specific ones that come to mind right away are optional, and ptr_... polymorphic container types come to mind right away. String algorithm library is also very handy, considering the lack of very commonly needed string functions, such as case conversion or trimming, in the standard library.
Speaking of more heavyweight components, Boost.Filesystem is a very decent cross-platform abstraction for filesystem navigation, also a relatively common task in command-line tools. Then there's Boost.MultiIndex is a swiss army knife of containers - rarely truly needed, but when it is, it's indispensable.
I did a gig this summer in .NET and just ported to Mono. Worked great.
Although there are some good cross platform libraries out there (like Boost), remember that they are probably not there by default. This is especially problematic if you are shipping binaries only. The target platform is unlikely to have the library (or correct version of the library) that you need.
First prize is to stick with standard C++ (even if you need to implement simple stuff yourself). This avoid library dependence altogether.
If you must use a library, try statically linking against it (although this may create big binaries). This will allow you to avoid runtime failure due to lack of binaries.
If you must ship DLLs (or .so on some unixes) make sure that the correct version is shipped with your product and some way to avoid conflicts with the wrong version.
If you are shipping code, include the library with the code and build the library as well as your utility.
Also beware of GPL and possibly LGPL code. If you release a library with a GPL dependency (or modify an LGPL library) you will need to supply the code and allow redistribution as per the GPL.
TCLAP is the only header-only CLI parsing option that I'm aware of. As such, it strikes me as the most portable and is probably your best bet (it's currently what I use and recommend for exactly those reasons). It also helps that the API for TCLAP is very developer friendly and automatically generates decently formatted help messages for you.
Boost program_options has a shard library component to it, which is irritating to maintain ABI with. It also gets around nuisance parsing incompatibilities and behaviors from the getopt family of arguments.
I have used libparamset, that is cross platform (Windows, OS X, Linux) CLI parser. It provides flexible and powerful CLI parser and various UI building features (input error handling, wildcards, typo detection, task resolving, help formatting ...) to build a good command-line tool. It is suitable for both C and C++ projects.
I was curious about STL implementations outside of what's packaged with gcc or Visual Studio, so a quick Google search turned up a few results, such as:
Apache stdcxx
uSTL
rdeSTL
Under what circumstances should one use an alternative standard template library?
For instance, Apache's page has a list including items such as "full conformance to the C++ standard" and "optimized for fast compiles and extremely small executable file sizes". If it's so good, why wouldn't it replace libstdc++?
For the sake of completeness, here are some of the other STL implementations:
STLPort
STXXL (which is sort of special purpose, meant for large data sets that won't fit in memory)
Dinkumware (commercial)
SGI STL
libstdc++ (GCC's implementation)
I never had to use an STL version other than the one packed with the compiler. But here are some points that come into my mind.
Thread-safety: The STL from apache provides a compile switch to turn on/off some thread-safety features.
Localization: Again the STL from apache comes with nice support for many different locales.
Data structures: You might need a basic_string implementation that is based on COW (copy-on-write) and the STL version that came with your compiler doesn't offer that.
Non-standard extensions: Particular features you like from some other STL implementations. For example, hash_map (and related) versions from Dinkumware (which ships with Visual Studio) have a significantly different design from hash_map (and related) from STLPort.
Binary issues: Constraints in some environment (embedded software) due to code size. In such case, if you don't need the whole STL it could be interesting to use a reduced version.
Performance: What if you discovered, after profiling, that the "other" STL implementation gives you significant better performance for a particular application. (With so many details concerning algorithms and data structures this could actually be possible.)
Debug mode: Some STL implementation provide nice features for debugging. For instance, checking ranges of iterators.
I sometimes use STLPort rather than the STL that ships with Visual Studio. Back when VC6 was supported the STL that shipped with it was buggy and so using STLPort (or another STL) made a lot of sense (especially if you were building multi-threaded code).
Now it's often more about performance (size or speed). For example, the STL that ships with VS2008 isn't that friendly in a multi-threaded situation as it uses locking around locale objects which causes things that you wouldn't expect to synchronise across threads. (See here Convert a number to a string with specified length in C++ for details of one example of this).
Third parties can implement improved versions of STL that attempt to offer various things, such as smaller size, faster execution, etc. You might choose one of these alternative implementations because you want one of those attributes of their implementation. You might also choose one of them when doing cross-platform development because you want to avoid running into differences in behavior between the gcc and Visual Studio versions of your product (as just one example).
There is no need to wait for a new release of a compiler with a bundled implementation of STL in order to reach out for a fresh implementation of it if you have specific needs.
I've never had a need to use an alternative STL, but I could envision some scenarios where it might be useful to use, for example, the Apache version, if you need small executables because you're developing for an embedded platform.
Another reason might be to use an STL version that guarantees certain things which are not necessarily guaranteed by the standard. For example, to ensure you have non-COW strings so you can write thread-safe code.
STLport has what they call a "power debug mode" that does a whole slew of run-time checking for "the correctness of iterators and containers usage". Helps catch a number of errors that would not be immediately obvious. I highly recommend use of STLport while debugging and testing.
The C++ standard library can be implemented in variety of ways. Some implementers try to cope with modern ideas. So, using an optimized implementation may result in faster and smaller executables.
Take SCARY for example. Some implementers didn't do it yet, although it reduces the bloat of STL to a great extent. When you do the following:
vector<int> f;
vector<int, MyAllocator> s;
size_t fc = count(f.begin(), f.end(), SomeValue);
size_t sc = count(s.begin(), s.end(), SomeOtherValue);
An "old" implementation could produce two different count functions in the result executable, because the type of f is not the same as of s. Thats because the iterator type depends on the type of the vector itself, although it doesn't need to be like that. A better idea is to separate the type of the iterator in a separate class, and provide a typedef in vector, and the compiler would produce one count only. That was just an example, but I think there are more to say about the quality of some implementations.
Aside from the reasons already given, I could imagine using a different STL due to debugging support or as a way to guarantee I was not relying on vendor extensions.
It would also be a first step in testing whether a library I was shipping worked well on other platforms.
Folks mentioning STLport have cited performance and portability, but there's also a pretty good debug mode available. I think that's a great reason to be using a different STL, if your current compiler's library is limited in this way.
Aaand ... it looks like Max and I tied on mentioning debugging! ;^)~
One reason is for better thread-safety. I was using the default STL that came with Visual Studio (VC6 infact) then had to shift to STLPort as it had much better thread-safety.
We are currently using STLPort - external implementation of STL because we have to use (for various reasons) quite old Microsoft Visual C++ 6.0 compiler (1998 release date) and compiler supplied library (by Dimkunware) is of course very out of date.
Debugging has been mentioned by several people, in terms of the ability to switch on extra diagnostic information, however another important aspect is that if you're using the platform's own STL then you may get a better experience in the debugger. This is especially true if you're using Visual Studio which has visualisers for all the standard containers.
STLPort has support for files bigger than 2GB through std::fstreams. Visual Studio 2005/2008 cannot handle files bigger than 2GB.
You can test your STL implementation by displaying: std::numeric_limits<std::streamsize>::max()
Both MSVC++ and GNU g++ come with pretty good implementations of the C++ Standard Library, but there are compilers that don't, and if I had to support such compilers I would look for a 3rd party implementation of STL.