This question already has answers here:
Where to get the source code for the C++ standard library? [closed]
(10 answers)
Closed 7 years ago.
I'm curious about learning how certain C++ features work. I'm trying to learn C++11 concepts such as std::function, but I keep hitting walls like INVOKE(function, arguments, return) that I don't understand. People tell me, "Oh, just ignore it and use auto" but I want to have a truly deep understanding of how C++ and its standard library works, so I wanted to find the source code of the standard library.
I would guess that the C++ Standard Library is somewhat related with the C Standard Library and the messy assembly/binary implementations at the lowest level for stuff like std::iostream and such, but I'm interested in more higher-level abstractions like smart pointers and std::function. Given that many of the C++11 libraries were once Boost ones, how might I find the source for C++ Standard Library implementations?
The two most popular open source implementations of standard C++ library are:
libstdc++, part of gcc project
libc++, part of LLVM project
Both websites contain links to git/svn repositories with source code.
You might dive into the source code of libstdc++ if you care about GCC. Indeed it sometimes leverages above the standard C library (e.g. ::operator new might call malloc, etc...)
Notice that since the C++ library is part of the standard, some of it might be implemented in a compiler specific way.
In principle, nothing requires standard headers to be real operating-system files; a compiler could parse #include <vector> in a particular way not involving any file. I know no compiler doing that much!
In particular, libstdc++ is using some GCC builtins and some GCC attributes (which happens to be also understood by Clang/LLVM).
And some standard types require (or profit from) internal support in the compiler. For example, GCC has some specific code to deal with va_list and with std::initializer_list (and of course basic types like int...), etc. Also, the compiler's implementation of C++ closures (or lambda functions) is related to other classes, etc...
Also, some optimization passes of GCC (it has several hundreds of them) are designed with some features of libstdc++ implementation in mind.
BTW, using a recent gdb debugger with e.g. the libstdc++6-4.9-dbg debian package might also help.
Related
So apparently STL is known as standard template library, which includes common data structures, classes, functions, or methods. However, the STL is not built into C++ language even though it holds the code to use such common things in the C++ language? I thought all these common data structures and methods were built into C++ language itself, but we must keep including the preprocessor directives to access them. Also, is there one preprocessor directive for all of the STL? Why have separate preprocessor directives that built up STL collectively. Should not the STL be represented by one thing?
The STL is the name of a software library, developed originally by Alexander Stepanov, and proposed for consideration by the C++ standardisation committee in 1993.
During the standardisation process, which eventually resulted in the first C++ standard being ratified in 1998, the specification of the library evolved. The C++ standard specifies the C++ standard library.
Because of this history, the STL influenced the specification of the C++ standard library. During the standardisation process before 1998, the STL was was evolved and extended. In 1994, this work resulted in a proposal for a C++ standard library by Alexander Stepanov and Meng Lee being voted and incorporated into the (then) draft C++ standard.
Technically, it is possible to distinguish between the C++ language (rules of syntax, semantics, etc) and the C++ standard library (which provides a set of types and functions that build on and support the language). A lot of the C++ standard library can be implemented in the C++ language (e.g. templated parts of the library, such as standard algorithms, which originated in STL). Some elements (e.g. the specification of the range of integral types represented by the templated std::numeric_limits) are implementation-defined. Some parts cannot be implemented in C++ at all (e.g. at some level they access "compiler magic", or use facilities of the host system (OS-specific API, machine instructions, etc).
There is not a single preprocessor directive for the C++ standard library, and never was for the STL. The philosophy is that code only accesses functionality it needs (e.g. if doing console-based I/O, it includes <iostream> but doesn't need to include <numeric> (which provides common math functions)). Practically, with most implementations, including parts of the standard library that are not needed by a program, tends to increase the time and resources needed for preprocessing and subsequent phases of translation (parsing the contents of the headers), and therefore increases build times by a substantial amount. Since significant projects can have build-from-scratch times measured in weeks or months, and using "include everything headers" can easily increase build times by orders of magnitude, it is usually considered good practice to avoid them.
The "Standard Template Library" is no more. It's the "Standard Library" now. STL was from the SGI era and has been significantly reworked and developed since then.
The reason it's broken up into various components instead of one gigantic #include file is because processing these files comes with a cost, and it can also pollute your root namespace if you're using namespace std.
C++, like C, requires inclusion of header files for the tools that you're using, nothing is supplied automatically. This isn't all that unusual, other languages force you to require or import or use other code which serves the same purpose.
Now the Standard Library is not part of the C++ language per-se, the language does not require using it, but it is an expected feature of any standard-compliant C++ compiler and the Standard Library has been improved in conjunction with the language through each major release. It's not built into the language, but it is built into the compiler, if you care for such distinctions. It'd also something that makes C++ far more useful than having just the core language.
So the Standard Library is the baseline for a C++ compiler. Many developers choose to go beyond that using things like Boost, where Boost itself is an unofficial standard library of sorts. Many features from Boost have been reworked and absorbed into the Standard Library, a trend that's likely to continue.
You can make an omni-include file that includes everything if you prefer, but you'll probably see much, much slower build times. The entire library is a considerable amount of header information to process.
So apparently STL is known as standard template library
Yes, but STL is nowadays a misnomer. It is best to call it the standard library.
However, the STL is not built into C++ language even though it holds the code to use such common things in the C++ language?
The standard library is part of the ISO C++ language.
Even for freestanding implementations (e.g. without an operating system), the standard requires implementing a fair amount of headers and things intended to support the language.
I thought all these common data structures and methods were built into C++ language itself
For hosted implementations (the ones you usually deal with unless you work on things like embedded), they are!
but we must keep including the preprocessor directives to access them.
That does not mean they are not part of the language, just that you need to include the headers like for any other library.
The language could have an "auto-detection" feature for things in the std namespace (and sometimes compilers have one to suggest you things on errors), but it is not the case.
Also, is there one preprocessor directive for all of the STL?
No, but that is not usually a problem: you can make one if you really want that (although it is usually best to just declare whatever you need in each translation unit).
Why have separate preprocessor directives that built up STL collectively. Should not the STL be represented by one thing?
The language could have specified such a thing (some languages have a "prelude" of sorts), but currently that isn't the case and you need to be explicit.
Per Wikipedia
In the C++ programming language, the C++ Standard Library is a collection of classes and functions, which are written in the core language and part of the C++ ISO Standard itself.
Is C++ Standard Template Library (STL) on Windows different than the one on Linux OR any other Platform? Are the headers of STL differ with platform as well OR is STL just a header library and its implementation in CRT?
We know that compiler differ with platform and also the C Runtime Library differ with Platform. Going by this is it true that even Standard Template Library (STL) differ with platform?
Please clarify this doubt.
Also, what is the name of the C++ STL on windows and whats on Linux?
I have been trying to understand this by going various articles online and trying to frame a single workflow in my mind to understand the terms better.
The specification of the C++ Standard Library is not contingent on any particular platform or compiler, although it does depend on the target C++ standard, and its behaviour is dependent function on various properties of a platform.
But the implementation of the C++ Standard Library is extremely dependent on the compiler and operating system. Some of the C++ Standard Library can even hardcoded into compilers.
But you use it in the same way. E.g. for std::cout and std::cin you should always write #include <iostream> as that's what the documentation says you're supposed to do. The names of header files can vary between implementations, but the ones you are supposed to use directly never differ.
This is why it's a good idea not to rely on #includes being available implicitly via other headers, or by using fancy non-standard #includes like <bits/...>. If you do so then you are not writing portable C++.
C++ was designed for abstract hardware but Standard C++ Library implementations differ, yes. Library vendors are required to follow the C++ Standard rules but are free to provide their implementation where allowed (the implementation defined wording in the standard). Headers also differ.
I have searched the net without any conclusive answers to issue related to lack of C++ ABI when it comes to exporting c++ classes accross dll boundaries in windows.
I can use extern c and provide a c like api to a library , but I would like end users to be able to use classes that us stl containers.
What generic patterns do you use for exporting a class that uses stl container across dll boundaries , in a safe manner? e.g. best practice.
This question is for experienced library authors.
Regards
Niladri
There's no defined C++ ABI and it does differ between compilers in terms of memory layout, name mangling, RTL etc.
It gets worse than that though. If you are targeting MSVC compiler for example, your dll/exe can be built with different macros that configure the STL to not include iterator checks so that it is faster. This modifies the layout of STL classes and you end up breaking the One Definition Rule (ODR) when you link (but the link still succeeds). If the ODR is violated then your program will crash seemingly randomly.
I would recommend maybe reading Imperfect C++ which has a chapter on the subject of C++ ABIs.
The upshot is that either:
you compile the dll specifically for the target compiler that is going to link to it and name the dll's appropriately (like boost does it). In this case you can't avoid ODR violations so the user of the library must be able to recompile the library themselves with different options.
Provide a portable C API and provide C++ wrapper classes for convenience that are compiled on the client side of the API. This is time consuming but portable.
Take a look at CppComponents https://github.com/jbandela/cppcomponents
You can export classes and interfaces across dll boundaries even across different compilers.
The library is header-only so nothing to build. To use it, you will need MSVC 2013 and/or GCC 4.8 as it uses C++11 variadic templates extensively.
See presentation from C++Now
https://github.com/boostcon/cppnow_presentations_2014/blob/master/files/cppnow2014_bandela_presentation.pdf?raw=true
https://github.com/jbandela/cppcomponents_cppnow_examples has the examples from the presentation.
I am new user of the boost library. I find my self thinking more about adopting boost for a number of reasons. From what I can tell, it seems that the boost library is a sort of skunkworks sandbox where various C++ TR features for upcoming standardization are tried out before being adopted by the C++ committee - think boost::filesystem and boost::regex,
As an example, I was trying out some of the C++11 regex features in visual studio via the #include header - this worked great until I ported to a target power pc platform, which, at the time used CodeSourcery's GCC 4.7.3. Unfortunately, I realized that at run-time, that much of the regex implementation was incomplete or empty (even thought it compiled) - With a bit of homework, I should have realized this beforehand, however now that GCC 4.8.x is out, the implementation is part of the v3 standard C++ library so it is a different story now.
In an ideal world, the standard library should be like developing for Java - write once, deploy everywhere - but that is not a reality. I would eventually like to move to the standard library implementation rather than Boost's regex and filesystem implementations.
My question given the above regex history, is how should developers use boost, is it possible to do a simple search and replace of the boost headers and namespaces when the features are adopted by the standard library or are there more many things to consider. I would like to use pure C++11 code without dependency on 3rd party libraries.
The amount of work required to move from a Boost library to its C++11 conterpart depends on the degree of C++11 conformance of a particular Boost library. In the simplest case it can be a matter of including another set of headers and using another namespace.
In a more complicated case, Boost library may have some subtle incompliancy with C++11 (eg. in Boost.Thread V1 ~thread used to call detach()) - such things might "silently" break the code correctness, but they are easy to fix.
Finally, Boost library may implement funcionality that doesn't exist in C++11 (eg. boost::bind can be extended using get_pointer function). Apparantly, porting such a code to C++11 would be quite not trivial.
Let's begin with your statement
I would like to use pure C++11 code without dependency on 3rd party
libraries.
It is clear that this is not possible now. You will have to use 3rd party libraries for any non-trivial program.
Unfortunately, C++ with Boost is not a platform also. You need 3rd party libraries to do things available out of the box in languages like Java, C#, Python etc.
So, you have to select libraries according to your requirements: performance, supported platforms, multithreading etc.
Again, Boost shouldn't be your default choice. It is not that useful now as it was 10 years ago. Most of must have stuff went into C++ standard library already.
If you support existing C++ codebase, find the best C++ library for your needs (e.g. re2 for regex). If you start a new project, I would suggest using Qt as a platform.
A "simple" way to migrate usage may be to use preprocessor defines to define a "Using Boost" directive. By putting all boost code in an #if-#else and carefully writing the code to not break (or at least have expected results) for sections that do not have a C++11 equivalent. You can simply not provide a definition for "Using Boost" before at the beginning of your code and C++11 features would be used instead.
See this and this
One link points to an old stackoverflow question, the other to an interesting talk performed by Stephan Lavavej
Do libraries such as boost, STL, ACE (which often make inclusions in namespace std) contain any special kind of coding techniques which is not possible to be coded/used by a usual programmer ?
In a broader sense, do they leverage any compiler or implementation specific utilities, which is not exposed to the general programmers ?
These are all written in the same code available to everyone. However, the code is often hard to read (at least for me) because they go to great lengths to ensure the generality of the libraries. Here is the sgi implementation of the STL. Browse through it and see for yourself.
Since the standard library is part of the C++ specification, your question is not well-founded.
For example, the implementation of std::fstream (or at least, std::filebuf) must use OS-dependent interfaces. Do those count as "implementation specific utilities"?
The bottom line is that the spec does not separate out the standard library from the rest of the language. It is all just part of the language, and its facilities are available to "usual programmers".
Boost is mostly written in standard C++, but they do take advantage of platform-specific features when that can yield performance improvements, and they occasionally need compiler-dependent extensions for features. The documentation will generally mention when a feature is not available on all platforms.
I do not know about ACE.
The STL (and the others) is written in 'pure C++'. See here for a very similar question.
C, on the other hand, has many system calls (unix/Windows/etc) in its standard library files to make everything work.
The C++0x STL also uses some compiler magic to make some new language features work.