Do libraries such as boost, STL, ACE (which often make inclusions in namespace std) contain any special kind of coding techniques which is not possible to be coded/used by a usual programmer ?
In a broader sense, do they leverage any compiler or implementation specific utilities, which is not exposed to the general programmers ?
These are all written in the same code available to everyone. However, the code is often hard to read (at least for me) because they go to great lengths to ensure the generality of the libraries. Here is the sgi implementation of the STL. Browse through it and see for yourself.
Since the standard library is part of the C++ specification, your question is not well-founded.
For example, the implementation of std::fstream (or at least, std::filebuf) must use OS-dependent interfaces. Do those count as "implementation specific utilities"?
The bottom line is that the spec does not separate out the standard library from the rest of the language. It is all just part of the language, and its facilities are available to "usual programmers".
Boost is mostly written in standard C++, but they do take advantage of platform-specific features when that can yield performance improvements, and they occasionally need compiler-dependent extensions for features. The documentation will generally mention when a feature is not available on all platforms.
I do not know about ACE.
The STL (and the others) is written in 'pure C++'. See here for a very similar question.
C, on the other hand, has many system calls (unix/Windows/etc) in its standard library files to make everything work.
The C++0x STL also uses some compiler magic to make some new language features work.
Related
When the C++ committee publish a new feature that will be part of the standard library in the next standard of the language, do they also release some source code or some kind of guidance on how to implement that feature?
Let's take unique_ptr as an example. The language committee just defines an interface for that class template and let the compiler vendor implement it as it wants? How exactly does this process of implementation of the standard library features occur?
Can anyone implement parts of the standard library for a platform that doesn't have support for them yet? Let say I would like to implement some cool features of the C++ standard library to use it on a microcontroller environment. How could I do that? Where should I look for information? If I decide to open source my project, can I do that? Will I need to follow exactly what the standard says, or I can write a non-compliant version?
Usually,
every new library feature goes through a proposal.
If the proposal makes it to the C++ committee's Library Evolution Working Group, it goes through a series of iterations (a "tough ground" as I am aware).
It undergoes a series of refinement process as described here
Should it require a (TS) Technical Specification (since C++11), it goes there to be baked. Take for example, the #include <filesystem> was in a Filesystem TS prior to C++17.
One thing I believe the committee likes, is an implementation experience.
More information can be found on the ISOCpp site
Well, as to the implementation:
There are quite a number of "library features" that cannot be implemented purely as a library. they require compiler support. And in these case, compilers provides "intrinsic" that you could hook on to. Take for example, clang provides intrinsics for certain type_traits
Most library features have some implementation experience, mostly from the Boost libraries.
You could actually look into the source code for the default standard library that ships with your compiler:
libc++ for Clang
libstdc++ for GCC
Sadly most of the implementations use a whole bunch of underscores. Mostly because they are reserved for use by the "Standard Library".
Can anyone implement parts of the standard library for a platform that doesn't have support for it yet?
Yes, you can, so far your compiler supports that platform, and the platform or Operating System provides usable API. For example. std::cout, elements of std::ifstream, and so much more requires platform specific support.
Let say I would like to implement some cool features of the C++ standard library to use it on a microcontroller environment. How could I do that?
You can look into the code of others and start from there. We learn from giants. Some Open Source Examples:
ETL
StandardCPlusPlus
uClib++
How could I do that? Where should I look for information?
You could check the paper that introduced the feature into the C++ library. For example, std::optional has a stand-alone implementation here which was used as a reference implementation during the proposal stages.
You could check the standard library, and do a laborious study. :-)
Search the internet. :-)
Or write it from scratch as specified by the standard
Will I need to follow exactly what the standard say or I can write a non-compliant version?
There's is no compulsion to follow what the C++ standard library specifies. That would be your "own" library.
Formally, no. As with all standards out there, C++ Standard sets the rules, and does not gives implementation. However, from the practical standpoint, it is nearly impossible to introduce a new feature into Standard Library without proposed implementation, so you often can find those attached to proposals.
As for your questions on "can you write non-compliant version", you can do whatever you want. Adoption might depend on your compliance, or might not - a super-widely adopted MSVC is known to violate C++ standard.
Typically, a new feature is not standardized, unless the committee has some solid evidence that it can be implemented, and will be useful. This very often consists of a prototype implementation in boost, a GNU library, or one of the commercial compiler vendors.
The standard itself does not contain any implementation guidance - it is purely a specification. The compiler vendors (or their subcontractors) choose how to implement that specification.
In the specific case of unique_ptr, it was adopted into the standard from boost::unique_ptr - and you can still use the latter. If you have a compiler that will compile for your microcontroller, it is almost certain that it will be able to build enough of boost to make unique_ptr work.
There is nothing stopping you from writing a non-conforming implementation (apart from the trivial point that if you sold it as being standards-conforming, and it wasn't you might get your local equivalent of Trading Standards come knocking.)
The committee does not release any reference implementations. In the early days, things got standardized and then the tool developers went away and implemented the standard. This has changed, and now the committee looks for features that have been implemented and tested before standardization.
Also major developments usually don't go directly into the standard. First they become experimental features called a Technical Specification or TS. These TS may then be incorporated into the main standard at a later date.
You are free to write you own implementation of the C++ standard library. Plum Hall has a test suite (commercial, I have no connection, but Plum Hall are very involved with C++ standardization).
I don't see any issue with not being conformant. Almost all implementations have some extensions. Just don't make any false claims, especially if you want to sell your product.
If you're interested in getting involved, this can be done via your 'National Body' (ANSI for the USA, BSI for the UK etc.). The isocpp web site has a section on standardization which would be a good starting place.
So apparently STL is known as standard template library, which includes common data structures, classes, functions, or methods. However, the STL is not built into C++ language even though it holds the code to use such common things in the C++ language? I thought all these common data structures and methods were built into C++ language itself, but we must keep including the preprocessor directives to access them. Also, is there one preprocessor directive for all of the STL? Why have separate preprocessor directives that built up STL collectively. Should not the STL be represented by one thing?
The STL is the name of a software library, developed originally by Alexander Stepanov, and proposed for consideration by the C++ standardisation committee in 1993.
During the standardisation process, which eventually resulted in the first C++ standard being ratified in 1998, the specification of the library evolved. The C++ standard specifies the C++ standard library.
Because of this history, the STL influenced the specification of the C++ standard library. During the standardisation process before 1998, the STL was was evolved and extended. In 1994, this work resulted in a proposal for a C++ standard library by Alexander Stepanov and Meng Lee being voted and incorporated into the (then) draft C++ standard.
Technically, it is possible to distinguish between the C++ language (rules of syntax, semantics, etc) and the C++ standard library (which provides a set of types and functions that build on and support the language). A lot of the C++ standard library can be implemented in the C++ language (e.g. templated parts of the library, such as standard algorithms, which originated in STL). Some elements (e.g. the specification of the range of integral types represented by the templated std::numeric_limits) are implementation-defined. Some parts cannot be implemented in C++ at all (e.g. at some level they access "compiler magic", or use facilities of the host system (OS-specific API, machine instructions, etc).
There is not a single preprocessor directive for the C++ standard library, and never was for the STL. The philosophy is that code only accesses functionality it needs (e.g. if doing console-based I/O, it includes <iostream> but doesn't need to include <numeric> (which provides common math functions)). Practically, with most implementations, including parts of the standard library that are not needed by a program, tends to increase the time and resources needed for preprocessing and subsequent phases of translation (parsing the contents of the headers), and therefore increases build times by a substantial amount. Since significant projects can have build-from-scratch times measured in weeks or months, and using "include everything headers" can easily increase build times by orders of magnitude, it is usually considered good practice to avoid them.
The "Standard Template Library" is no more. It's the "Standard Library" now. STL was from the SGI era and has been significantly reworked and developed since then.
The reason it's broken up into various components instead of one gigantic #include file is because processing these files comes with a cost, and it can also pollute your root namespace if you're using namespace std.
C++, like C, requires inclusion of header files for the tools that you're using, nothing is supplied automatically. This isn't all that unusual, other languages force you to require or import or use other code which serves the same purpose.
Now the Standard Library is not part of the C++ language per-se, the language does not require using it, but it is an expected feature of any standard-compliant C++ compiler and the Standard Library has been improved in conjunction with the language through each major release. It's not built into the language, but it is built into the compiler, if you care for such distinctions. It'd also something that makes C++ far more useful than having just the core language.
So the Standard Library is the baseline for a C++ compiler. Many developers choose to go beyond that using things like Boost, where Boost itself is an unofficial standard library of sorts. Many features from Boost have been reworked and absorbed into the Standard Library, a trend that's likely to continue.
You can make an omni-include file that includes everything if you prefer, but you'll probably see much, much slower build times. The entire library is a considerable amount of header information to process.
So apparently STL is known as standard template library
Yes, but STL is nowadays a misnomer. It is best to call it the standard library.
However, the STL is not built into C++ language even though it holds the code to use such common things in the C++ language?
The standard library is part of the ISO C++ language.
Even for freestanding implementations (e.g. without an operating system), the standard requires implementing a fair amount of headers and things intended to support the language.
I thought all these common data structures and methods were built into C++ language itself
For hosted implementations (the ones you usually deal with unless you work on things like embedded), they are!
but we must keep including the preprocessor directives to access them.
That does not mean they are not part of the language, just that you need to include the headers like for any other library.
The language could have an "auto-detection" feature for things in the std namespace (and sometimes compilers have one to suggest you things on errors), but it is not the case.
Also, is there one preprocessor directive for all of the STL?
No, but that is not usually a problem: you can make one if you really want that (although it is usually best to just declare whatever you need in each translation unit).
Why have separate preprocessor directives that built up STL collectively. Should not the STL be represented by one thing?
The language could have specified such a thing (some languages have a "prelude" of sorts), but currently that isn't the case and you need to be explicit.
Per Wikipedia
In the C++ programming language, the C++ Standard Library is a collection of classes and functions, which are written in the core language and part of the C++ ISO Standard itself.
When the C++ committee publish a new feature that will be part of the standard library in the next standard of the language, do they also release some source code or some kind of guidance on how to implement that feature?
Let's take unique_ptr as an example. The language committee just defines an interface for that class template and let the compiler vendor implement it as it wants? How exactly does this process of implementation of the standard library features occur?
Can anyone implement parts of the standard library for a platform that doesn't have support for them yet? Let say I would like to implement some cool features of the C++ standard library to use it on a microcontroller environment. How could I do that? Where should I look for information? If I decide to open source my project, can I do that? Will I need to follow exactly what the standard says, or I can write a non-compliant version?
Usually,
every new library feature goes through a proposal.
If the proposal makes it to the C++ committee's Library Evolution Working Group, it goes through a series of iterations (a "tough ground" as I am aware).
It undergoes a series of refinement process as described here
Should it require a (TS) Technical Specification (since C++11), it goes there to be baked. Take for example, the #include <filesystem> was in a Filesystem TS prior to C++17.
One thing I believe the committee likes, is an implementation experience.
More information can be found on the ISOCpp site
Well, as to the implementation:
There are quite a number of "library features" that cannot be implemented purely as a library. they require compiler support. And in these case, compilers provides "intrinsic" that you could hook on to. Take for example, clang provides intrinsics for certain type_traits
Most library features have some implementation experience, mostly from the Boost libraries.
You could actually look into the source code for the default standard library that ships with your compiler:
libc++ for Clang
libstdc++ for GCC
Sadly most of the implementations use a whole bunch of underscores. Mostly because they are reserved for use by the "Standard Library".
Can anyone implement parts of the standard library for a platform that doesn't have support for it yet?
Yes, you can, so far your compiler supports that platform, and the platform or Operating System provides usable API. For example. std::cout, elements of std::ifstream, and so much more requires platform specific support.
Let say I would like to implement some cool features of the C++ standard library to use it on a microcontroller environment. How could I do that?
You can look into the code of others and start from there. We learn from giants. Some Open Source Examples:
ETL
StandardCPlusPlus
uClib++
How could I do that? Where should I look for information?
You could check the paper that introduced the feature into the C++ library. For example, std::optional has a stand-alone implementation here which was used as a reference implementation during the proposal stages.
You could check the standard library, and do a laborious study. :-)
Search the internet. :-)
Or write it from scratch as specified by the standard
Will I need to follow exactly what the standard say or I can write a non-compliant version?
There's is no compulsion to follow what the C++ standard library specifies. That would be your "own" library.
Formally, no. As with all standards out there, C++ Standard sets the rules, and does not gives implementation. However, from the practical standpoint, it is nearly impossible to introduce a new feature into Standard Library without proposed implementation, so you often can find those attached to proposals.
As for your questions on "can you write non-compliant version", you can do whatever you want. Adoption might depend on your compliance, or might not - a super-widely adopted MSVC is known to violate C++ standard.
Typically, a new feature is not standardized, unless the committee has some solid evidence that it can be implemented, and will be useful. This very often consists of a prototype implementation in boost, a GNU library, or one of the commercial compiler vendors.
The standard itself does not contain any implementation guidance - it is purely a specification. The compiler vendors (or their subcontractors) choose how to implement that specification.
In the specific case of unique_ptr, it was adopted into the standard from boost::unique_ptr - and you can still use the latter. If you have a compiler that will compile for your microcontroller, it is almost certain that it will be able to build enough of boost to make unique_ptr work.
There is nothing stopping you from writing a non-conforming implementation (apart from the trivial point that if you sold it as being standards-conforming, and it wasn't you might get your local equivalent of Trading Standards come knocking.)
The committee does not release any reference implementations. In the early days, things got standardized and then the tool developers went away and implemented the standard. This has changed, and now the committee looks for features that have been implemented and tested before standardization.
Also major developments usually don't go directly into the standard. First they become experimental features called a Technical Specification or TS. These TS may then be incorporated into the main standard at a later date.
You are free to write you own implementation of the C++ standard library. Plum Hall has a test suite (commercial, I have no connection, but Plum Hall are very involved with C++ standardization).
I don't see any issue with not being conformant. Almost all implementations have some extensions. Just don't make any false claims, especially if you want to sell your product.
If you're interested in getting involved, this can be done via your 'National Body' (ANSI for the USA, BSI for the UK etc.). The isocpp web site has a section on standardization which would be a good starting place.
Many utilities of boost have been included as part of extended C++ TR1 currently.
Is the complete boost library going to be included once the standard is officially out ? In other words, do I need boost library, if I have complete standard conforming C++11 compiler ?
If not then any reason for that (Reliability cannot be an issue; as far as I know it's written by many people from standard committee) ?
Boost is huge, and of generally high but still varying quality. A lot of the APIs - even the techniques and functionality - are quite "experimental" in the sense that they're still regularly modified as real-world feedback comes in. By way of contrast, the Standard is expected to get it right and need minimal revision, especially when that breaks backwards compatibility.
The standard for review for Standard libraries is much higher than boost's, which is not to say that many boost libraries wouldn't meet the bar - just that many wouldn't too, and that the review process itself is time consuming. There are terrific programmers coordinating and contributing to boost, but they naturally focus their time on their own development interests and things they see as more relevant, so if something is a little specialised, doesn't appeal to their coding style, etc. it may not receive the same scrutiny. The Standard library needs to be scrutinised much more thoroughly as the consequences of change are so much more painful.
While portability is a factor in acceptance of a library into boost, it's not a hard and fast requirement, where-as compiler vendors are expected to make the Standard library run on all C++ compilers, so taking boost more or less as-is and expecting the functionality to be universal on Standard-compliant compiler vendors would put a huge workload on those vendors.
No, in fact very few parts of Boost are "included" in the C++0x revisions to the C++ Standard Library. The parts that are "included" are some of the most commonly used parts of Boost, though.
Really, "included" isn't correct anyway: there are many differences between the Boost libraries and the corresponding additions to the C++ Standard Library. Further, the Boost libraries continue to grow and evolve; the C++0x Standard Library is now finished.
No, Boost is not going to be included in its entirety in C++0x.
Parts of Boost will be, like boost::shared_ptr, Boost.Array, and a couple of other things. But most of Boost is not being included.
Is it possible to write the complete C++ standard library (including STL of course, but self-contained, only internal dependencies) using only C++? I would imagine containers and <cstdlib> functionality would be doable in terms of chars, bitshifts, and for loops and other byte fancy things, but stuff like exceptions and perhaps std::cout and std::cin seem hard to me without a dependency to begin with. Let's say there is a set of OS functions available, that are completely implemented in assembly (to avoid any C contamination).
I'm assuming the compiler understands everything from classes and virtual functions to templates and function overloading, these are language level things and have no place in a library IMHO.
If this has been asked before or is a trivially stupid question, please forgive me. I'm not trying to start a C<->C++ war here, just trying to figure out the limitations of implementing a beast such as the Standard library...
Thanks!
Since pretty much anything written in C can be rewritten fairly easily in C++, you're asking whether assembly code is needed, and the answer is generally no.
Unless we're talking about embedded programming, operating systems have all the necessary file and I/O functionality available through system calls, usually (nowadays) in C format. The library needs to call them, likely through extern "C"{ ... } declarations. The operating system functions are not considered part of the C++ library, and typically aren't exact matches to anything defined in the C++ Standard.
To implement a C++ standard library, you would need to be familiar with the language itself, know the OS calls you're going to use, and have the algorithms you're going to use. At that point, it's a relatively straightforward matter of writing the software.
First thing, it doesn't matter if it's C, C++, or D. Any compilable programming language at the end gives you (mostly) the same assembly object file.
Second thing, STL is written in C++, you cannot write C++ library in C or any other language (well, you can but I assume, that we're talking on reasonable solutions). You cannot implement STL containers in C, because the're strongly use templates.
GCC generate really nice output in asm for exceptions now. I recommend to read about C++ ABI (if you're interested in it).
C++ compiler understands all C++ specific features really nice nowadays. Thanks to really advanced code analysis and optimizations, it's able to produce fast executables (see first paragraph).
I hope that I've at least partly answered your question.
The only piece of C++ that needs assembly is the exception handling. I suppose that it might be doable in C++ if there exist libraries to handle the necessary register and stack management.
Of course, those libraries would then include assembly. There just isn't any other way to do direct register management.
The STL is very heavily reliant on #includeed header files. Those pretty much have to be C++.
Anything that isn't in one of those header files though could in theory be implemented in C, Ada, Assembly, or the other systems-programming language of your choice. However, you'd probably have to be maintaining two interfaces if you don't make at least the top layer C++.