When the C++ committee publish a new feature that will be part of the standard library in the next standard of the language, do they also release some source code or some kind of guidance on how to implement that feature?
Let's take unique_ptr as an example. The language committee just defines an interface for that class template and let the compiler vendor implement it as it wants? How exactly does this process of implementation of the standard library features occur?
Can anyone implement parts of the standard library for a platform that doesn't have support for them yet? Let say I would like to implement some cool features of the C++ standard library to use it on a microcontroller environment. How could I do that? Where should I look for information? If I decide to open source my project, can I do that? Will I need to follow exactly what the standard says, or I can write a non-compliant version?
Usually,
every new library feature goes through a proposal.
If the proposal makes it to the C++ committee's Library Evolution Working Group, it goes through a series of iterations (a "tough ground" as I am aware).
It undergoes a series of refinement process as described here
Should it require a (TS) Technical Specification (since C++11), it goes there to be baked. Take for example, the #include <filesystem> was in a Filesystem TS prior to C++17.
One thing I believe the committee likes, is an implementation experience.
More information can be found on the ISOCpp site
Well, as to the implementation:
There are quite a number of "library features" that cannot be implemented purely as a library. they require compiler support. And in these case, compilers provides "intrinsic" that you could hook on to. Take for example, clang provides intrinsics for certain type_traits
Most library features have some implementation experience, mostly from the Boost libraries.
You could actually look into the source code for the default standard library that ships with your compiler:
libc++ for Clang
libstdc++ for GCC
Sadly most of the implementations use a whole bunch of underscores. Mostly because they are reserved for use by the "Standard Library".
Can anyone implement parts of the standard library for a platform that doesn't have support for it yet?
Yes, you can, so far your compiler supports that platform, and the platform or Operating System provides usable API. For example. std::cout, elements of std::ifstream, and so much more requires platform specific support.
Let say I would like to implement some cool features of the C++ standard library to use it on a microcontroller environment. How could I do that?
You can look into the code of others and start from there. We learn from giants. Some Open Source Examples:
ETL
StandardCPlusPlus
uClib++
How could I do that? Where should I look for information?
You could check the paper that introduced the feature into the C++ library. For example, std::optional has a stand-alone implementation here which was used as a reference implementation during the proposal stages.
You could check the standard library, and do a laborious study. :-)
Search the internet. :-)
Or write it from scratch as specified by the standard
Will I need to follow exactly what the standard say or I can write a non-compliant version?
There's is no compulsion to follow what the C++ standard library specifies. That would be your "own" library.
Formally, no. As with all standards out there, C++ Standard sets the rules, and does not gives implementation. However, from the practical standpoint, it is nearly impossible to introduce a new feature into Standard Library without proposed implementation, so you often can find those attached to proposals.
As for your questions on "can you write non-compliant version", you can do whatever you want. Adoption might depend on your compliance, or might not - a super-widely adopted MSVC is known to violate C++ standard.
Typically, a new feature is not standardized, unless the committee has some solid evidence that it can be implemented, and will be useful. This very often consists of a prototype implementation in boost, a GNU library, or one of the commercial compiler vendors.
The standard itself does not contain any implementation guidance - it is purely a specification. The compiler vendors (or their subcontractors) choose how to implement that specification.
In the specific case of unique_ptr, it was adopted into the standard from boost::unique_ptr - and you can still use the latter. If you have a compiler that will compile for your microcontroller, it is almost certain that it will be able to build enough of boost to make unique_ptr work.
There is nothing stopping you from writing a non-conforming implementation (apart from the trivial point that if you sold it as being standards-conforming, and it wasn't you might get your local equivalent of Trading Standards come knocking.)
The committee does not release any reference implementations. In the early days, things got standardized and then the tool developers went away and implemented the standard. This has changed, and now the committee looks for features that have been implemented and tested before standardization.
Also major developments usually don't go directly into the standard. First they become experimental features called a Technical Specification or TS. These TS may then be incorporated into the main standard at a later date.
You are free to write you own implementation of the C++ standard library. Plum Hall has a test suite (commercial, I have no connection, but Plum Hall are very involved with C++ standardization).
I don't see any issue with not being conformant. Almost all implementations have some extensions. Just don't make any false claims, especially if you want to sell your product.
If you're interested in getting involved, this can be done via your 'National Body' (ANSI for the USA, BSI for the UK etc.). The isocpp web site has a section on standardization which would be a good starting place.
Related
When the C++ committee publish a new feature that will be part of the standard library in the next standard of the language, do they also release some source code or some kind of guidance on how to implement that feature?
Let's take unique_ptr as an example. The language committee just defines an interface for that class template and let the compiler vendor implement it as it wants? How exactly does this process of implementation of the standard library features occur?
Can anyone implement parts of the standard library for a platform that doesn't have support for them yet? Let say I would like to implement some cool features of the C++ standard library to use it on a microcontroller environment. How could I do that? Where should I look for information? If I decide to open source my project, can I do that? Will I need to follow exactly what the standard says, or I can write a non-compliant version?
Usually,
every new library feature goes through a proposal.
If the proposal makes it to the C++ committee's Library Evolution Working Group, it goes through a series of iterations (a "tough ground" as I am aware).
It undergoes a series of refinement process as described here
Should it require a (TS) Technical Specification (since C++11), it goes there to be baked. Take for example, the #include <filesystem> was in a Filesystem TS prior to C++17.
One thing I believe the committee likes, is an implementation experience.
More information can be found on the ISOCpp site
Well, as to the implementation:
There are quite a number of "library features" that cannot be implemented purely as a library. they require compiler support. And in these case, compilers provides "intrinsic" that you could hook on to. Take for example, clang provides intrinsics for certain type_traits
Most library features have some implementation experience, mostly from the Boost libraries.
You could actually look into the source code for the default standard library that ships with your compiler:
libc++ for Clang
libstdc++ for GCC
Sadly most of the implementations use a whole bunch of underscores. Mostly because they are reserved for use by the "Standard Library".
Can anyone implement parts of the standard library for a platform that doesn't have support for it yet?
Yes, you can, so far your compiler supports that platform, and the platform or Operating System provides usable API. For example. std::cout, elements of std::ifstream, and so much more requires platform specific support.
Let say I would like to implement some cool features of the C++ standard library to use it on a microcontroller environment. How could I do that?
You can look into the code of others and start from there. We learn from giants. Some Open Source Examples:
ETL
StandardCPlusPlus
uClib++
How could I do that? Where should I look for information?
You could check the paper that introduced the feature into the C++ library. For example, std::optional has a stand-alone implementation here which was used as a reference implementation during the proposal stages.
You could check the standard library, and do a laborious study. :-)
Search the internet. :-)
Or write it from scratch as specified by the standard
Will I need to follow exactly what the standard say or I can write a non-compliant version?
There's is no compulsion to follow what the C++ standard library specifies. That would be your "own" library.
Formally, no. As with all standards out there, C++ Standard sets the rules, and does not gives implementation. However, from the practical standpoint, it is nearly impossible to introduce a new feature into Standard Library without proposed implementation, so you often can find those attached to proposals.
As for your questions on "can you write non-compliant version", you can do whatever you want. Adoption might depend on your compliance, or might not - a super-widely adopted MSVC is known to violate C++ standard.
Typically, a new feature is not standardized, unless the committee has some solid evidence that it can be implemented, and will be useful. This very often consists of a prototype implementation in boost, a GNU library, or one of the commercial compiler vendors.
The standard itself does not contain any implementation guidance - it is purely a specification. The compiler vendors (or their subcontractors) choose how to implement that specification.
In the specific case of unique_ptr, it was adopted into the standard from boost::unique_ptr - and you can still use the latter. If you have a compiler that will compile for your microcontroller, it is almost certain that it will be able to build enough of boost to make unique_ptr work.
There is nothing stopping you from writing a non-conforming implementation (apart from the trivial point that if you sold it as being standards-conforming, and it wasn't you might get your local equivalent of Trading Standards come knocking.)
The committee does not release any reference implementations. In the early days, things got standardized and then the tool developers went away and implemented the standard. This has changed, and now the committee looks for features that have been implemented and tested before standardization.
Also major developments usually don't go directly into the standard. First they become experimental features called a Technical Specification or TS. These TS may then be incorporated into the main standard at a later date.
You are free to write you own implementation of the C++ standard library. Plum Hall has a test suite (commercial, I have no connection, but Plum Hall are very involved with C++ standardization).
I don't see any issue with not being conformant. Almost all implementations have some extensions. Just don't make any false claims, especially if you want to sell your product.
If you're interested in getting involved, this can be done via your 'National Body' (ANSI for the USA, BSI for the UK etc.). The isocpp web site has a section on standardization which would be a good starting place.
Different C++ compilers implement the various language features at different points in time (see, e.g., clang C++ status and gcc c++ status; likewise for other compilers). When creating a C++ library it is often desirable to support the latest features to improve the user experience. When supporting new features rather than the common subset implemented everywhere it is helpful to know what features a compilers support without having to support a set of version numbers for each compiler.
Is there are reasonably standardized set of feature tests which can be used at compile time to determine if a specific language feature is supported by the compiler?
You probably can't do better than the Boost.Config library. It defines preprocessor macros for various C++11 and C++14 features which aren't universally supported on C++11/14-ish compilers like VC++. It's as close as you're going to get to standard.
IIRC, it works similarly to autoconf, by prebuilding (and, when necessary, executing) a bunch of simple test programs. I don't think you're going to get anything which operates entirely at compile-time, simply because there are things which are keywords in one implementation and syntax errors in another.
I haven't tried to use the recommendations but at the C++ committee meetings the Feature Test SG (SG10) meets and updates a list of recommendations. Here is the latest document listing the current feature test macros: there are macros for various language level features. The expectation is that the document P0096rx will be updated when new features are voted into the working draft.
This document is not a standards document: the standard mandates implementation of a language standard and it doesn't make sense to standardize macros indicating whether a specific features is implemented! An implementation is either fully conforming or not. However, the expectation is that compiler vendors do use these macros as guidance to help users.
I work on large scale, multi platform, real time networked applications. The projects I work on lack any real use of containers or the Standard Library in general, no smart pointers or really any "modern" C++ language features. Lots of raw dynamically allocated arrays are common place.
I would very much like to start using the Standard Library and some of the C++11 spec, however, there are many people also working on my projects that are against because "STL / C++11 isn't as portable, we take a risk using it". We do run software on a wide variety of embedded systems as well as fully fledged Ubuntu/Windows/Mac OS systems.
So, to my question; what are the actual issues of portability with concern to the Standard Library and C++11? Is it just a case of having g++ past a certain version? Are there some platforms that have no support? Are compiled libraries required and if so, are they difficult to obtain/compile? Has anyone had serious issues being burnt by non-portable pure C++?
Library support for the new C++11 Standard is pretty complete for either Visual C++ 2012, gcc >= 4.7 and Clang >= 3.1, apart from some concurrency stuff. Compiler support for all the individual language features is another matter. See this link for an up to date overview of supported C++11 features.
For an in-depth analysis of C++ in an embedded/real-time environment, Scott Meyers's presentation materials are really great. It discusses costs of virtual functions, exception handling and templates, and much more. In particular, you might want to look at his analysis of C++ features such as heap allocations, runtime type information and exceptions, which have indeterminate worst-case timing guarantees, which matter for real-time systems.
It's those kind of issues and not portability that should be your major concern (if you care about your granny's pacemaker...)
Any compiler for C++ should support some version of the standard library. The standard library is part of C++. Not supporting it means the compiler is not a C++ compiler. I would be very surprised if any of the compilers you're using at the moment don't portably support the C++03 standard library, so there's no excuse there. Of course, the compiler will have to be have been updated since 2003, but unless you're compiling for some archaic system that is only supported by an archaic compiler, you'll have no problems.
As for C++11, support is pretty good at the moment. Both GCC and MSVC have a large portion of the C++11 standard library supported already. Again, if you're using the latest versions of these compilers and they support the systems you want to compile for, then there's no reason you can't use the subset of the C++11 standard library that they support - which is almost all of it.
C++ without the standard library just isn't C++. The language and library features go hand in hand.
There are lists of supported C++11 library features for GCC's libstdc++ and MSVC 2012. I can't find anything similar for LLVM's libc++, but they do have a clang c++11 support page.
The people you are talking to are confusing several different
issues. C++11 isn't really portable today. I don't think any
compiler supports it 100% (although I could be wrong); you can
get away with using large parts of it if (and only if) you limit
yourself to the most recent compilers on two or three platforms
(Windows and Linux, and probably Apple). While these are the
most visible platforms, they represent but a small part of all
machines. (If you're working on large scale networked
applications, Solaris will probably be important, and Sun CC.
Unless Sun have greatly changed since I last worked on it, that
means that there are even parts of C++03 that you can't count
on.)
The STL is a completely different issue. It depends partially
on what you mean by the STL, but there is certainly no
portability problem today in using std::vector. locale
might be problematic on a very few compilers (it was with Sun
CC—with both the Rogue Wave and the Stlport libraries),
and some of the algorithms, but for the most part, you can
pretty much count on all of C++03.
And in the end, what are the alternatives? If you don't have
std::vector, you end up implementing something pretty much
like it. If you're really worried about the presence of
std::vector, wrap it in your own class—if ever it's not
available (highly unlikely, unless you go back with a time
machine), just reimplement it, exactly like we did in the
pre-standard days.
Use STLPort with your existing compiler, if it supports it. This is no more than a library of code, and you use other libraries without problem, right?
Every permitted implementation-defined behaviour is listed in publicly available standard draft. There is next to nothing less portable in C+11 than in C++98.
Someone brought this article to my attention that claims (I'm paraphrasing) the STL term is misused to refer to the entire C++ Standard Library instead of the parts that were taken from SGI STL.
(...) it refers to the "STL", despite the fact that very few people still use the STL (which was designed at SGI).
Parts of the C++ Standard Library were based on parts of the STL, and it is these parts that many people (including several authors and the notoriously error-ridden cplusplus.com) still refer to as "the STL". However, this is inaccurate; indeed, the C++ standard never mentions "STL", and there are content differences between the two.
(...) "STL" is rarely used to refer to the bits of the stdlib that happen to be based on the SGI STL. People think it's the entire standard library. It gets put on CVs. And it is misleading.
I hardly know anything about C++'s history so I can't judge the article's correctness. Should I refrain from using the term STL? Or is this an isolated opinion?
The "STL" was written by Alexander Stepanov in the days long before C++ was standardised. C++ existed through the 80s, but what we now call "C++" is the language standardised in ISO/IEC 14882:2014 (and earlier versions, such as ISO/IEC 14882:2011).
The STL was already widely used as a library for C++, giving programmers access to containers, iterators and algorithms. When the standardisation happened, the language committee designed parts of the C++ Standard Library (which is part of the language standard) to very closely match the STL.
Over the years, many people — including prominent book authors, and various websites — have continued to refer to the C++ Standard Library as "the STL", despite the fact that the two entities are separate and that there are some differences. These differences are even more pronounced in the upcoming new C++ standard, which includes various features and significantly alters some classes.
The original STL is now often called "an implementation of the C++ Standard Template Library" (rather backwards to actual history!), in the same way that your Microsoft Visual Studio or GCC ships an implementation of the C++ Standard Library. But the "Standard Template Library" and the "Standard Library" are not the same thing.
The battle is about whether the current Standard Library should be called "the STL" in whole or in part, and/or whether it matters what it's called.
For "STL"
There is a school of thought that says that everybody knows now that "STL" means the standard library, just as everybody now knows that "C++" is the ISO-standardised language.
It also includes those who believe that it doesn't really matter as long as all parties understand what is being talked about.
It's a term made even more prevalent by the nature of the beast, much of which makes heavy use of the C++ feature known as "templates".
For "C++ Standard Library" (or stdlib)
However, there is another school of thought — to which I subscribe — that says that this is confusing. People learning C++ for the first time do not know this distinction, and may not notice small language differences.
The author of that article has numerous times encountered people who believe that the entire C++ Standard Library is the STL, including features that were never part of the STL itself. Most vocal proponents of "the STL", in contrast, know exactly what they mean by it and refuse to believe that not everybody "gets it". Clearly, the term's usage is not uniform.
In addition, there are some STL-like libraries that are in fact implementations of the original STL, not the C++ Standard Library. Until recently, STLPort was one of them (and even there, the confusion abounds!).
Further, the C++ Standard does not contain the text "STL" anywhere, and some people habitually employ phrases like "the STL is included in the C++ Standard Library", which is plain incorrect.
It's my belief that continuing to propagate the usage of the term in this way will just lead to the misunderstanding going on forever. Alas, it may be entirely counter-productive to attempt to change things, even if it's supposed to be for the better. We may just be stuck with double-meanings forever.
Conclusion
I appreciate that this post has been a little biased: I wrote the article you linked to. :) Anyway, I hope this helps to explain the battle a bit better.
Update 13/04/2011
Here are three perfect examples of someone who is using "the STL" to refer to the entire C++ Standard Library. It continues to baffle me that so many people swear blind that nobody ever does this, when it's plain to see almost on a daily basis.
There is no one answer that's really correct. Alexander Stepanov developed a library he called STL (working for HP at the time). That library was then proposed for inclusion in the C++ standard.
That basically "forked" development. The committee included some parts, rejected others completely, and redesigned a few (with Alexander's participation). Development of the original library was later moved to Silicon Graphics, but continued separately from the C++ standard library.
After those pieces were added to the standard library, some other parts of the standard library were modified to fit better with what was added (e.g., begin, end, rbegin and rend were added to std::string so it could be used like a container). Around the same time, most of the library (even pieces that were completely unrelated) were made into templates to accommodate different types (e.g., standard streams).
Some people also use STL as just a short form of "STandard Library".
That means when somebody uses the term "STL" they could be referring to any of about half a dozen different things. For better or worse, most people who use it seem to ignore the multiplicity of meanings, and assume that everybody else will recognize what they're referring to. This leads to many misunderstandings, and at least a few serious flame-wars that made most of the participants look foolish because they were simply talking about entirely different things.
Unfortunately, the confusion is likely to continue unabated. It's much more convenient to refer to "STL" than something like "the containers, iterators, and algorithms in the C++ standard library, but not including std::string, even though it can act like a container." Even though "C++ standard library" isn't quite as long and clumsy as that, "STL" is still a lot shorter and simpler still. Until or unless somebody invents terms that are more precise (when necessary), and just as convenient, "STL" will continue to be used and confusion will continue to result.
The term "STL" or "Standard Template Library" does not show up anywhere in the ISO 14882 C++ standard. So referring to the C++ standard library as STL is wrong. The term "C++ Standard Library" or "standard library" is what's officially used by ISO 14882:
ISO 14882 C++ Standard:
17 - Library introduction [lib.library]:
This clauses describes the contents of the C++ Standard Library, how
a well-formed C++ program makes use of
the library, and how a conforming
implementation may provide the
entities in the library.
...
STL is a library originally designed by Alexander Stepanov, independent of the C++ standard. However, some components of the C++ standard library include STL components like vector, list and algorithms like copy and swap.
But of course the C++ standard includes much more things outside the STL, so the term "C++ standard library" is more correct (and is what's actually used by the standards documents).
I've made this same argument recently, but I believe a little tolerance can be allowed. If Scott Meyers makes the same mistake, you're in good company.
From the GNU Standard C++ Library (libstdc++) FAQ:
The STL (Standard Template Library) was the inspiration for large chunks of the C++ Standard Library, but the terms are not interchangeable and they don't mean the same thing. The C++ Standard Library includes lots of things that didn't come from the STL, and some of them aren't even templates, such as std::locale and std::thread.
Libstdc++-v3 incorporates a lot of code from the SGI STL (the final merge was from release
3.3). The code in libstdc++ contains many fixes and changes compared to the original SGI code.
In particular, string is not from SGI and makes no use of their "rope" class (although that is included as an optional extension), neither is valarray nor some others. Classes like vector<> were from SGI, but have been extensively modified.
More information on the evolution of libstdc++ can be found at the API evolution and backwards compatibility documentation.
The FAQ for SGI's STL is still recommended reading.
FYI, as of March 2018 even the official STL web site www.sgi.com/tech/stl/ is gone.
In layman words: STL is part of Standard Library.
C++ Standard Library is group into:
Standard Functional Library
-I/O,
-String and character handling,
-Mathematical,
-Time, date, and localization,
-Dynamic allocation,
-Miscellaneous,
-Wide-character functions
Standard OOP and Generics Library
-The Standard C++ I/O Classes
-The String Class
-The Numeric Classes
-The STL Container Classes
-The STL Algorithms
-The STL Function Objects
-The STL Iterators
-The STL Allocators
-The Localization library
-Exception Handling Classes
-Miscellaneous Support Library
So if you are talking about STL as Standard Library, it is OK and just remember that STL implementations allow for generics and others are more specific to one type.
Please refer to https://www.tutorialspoint.com/cplusplus/cpp_standard_library.htm
C++ Standard Library includes C++ STL
The contents of the C++ standard library are:
C++ version of C language header file
C++ IO header file
C++ STL
So please don’t confuse the C++ standard library with STL.
Do libraries such as boost, STL, ACE (which often make inclusions in namespace std) contain any special kind of coding techniques which is not possible to be coded/used by a usual programmer ?
In a broader sense, do they leverage any compiler or implementation specific utilities, which is not exposed to the general programmers ?
These are all written in the same code available to everyone. However, the code is often hard to read (at least for me) because they go to great lengths to ensure the generality of the libraries. Here is the sgi implementation of the STL. Browse through it and see for yourself.
Since the standard library is part of the C++ specification, your question is not well-founded.
For example, the implementation of std::fstream (or at least, std::filebuf) must use OS-dependent interfaces. Do those count as "implementation specific utilities"?
The bottom line is that the spec does not separate out the standard library from the rest of the language. It is all just part of the language, and its facilities are available to "usual programmers".
Boost is mostly written in standard C++, but they do take advantage of platform-specific features when that can yield performance improvements, and they occasionally need compiler-dependent extensions for features. The documentation will generally mention when a feature is not available on all platforms.
I do not know about ACE.
The STL (and the others) is written in 'pure C++'. See here for a very similar question.
C, on the other hand, has many system calls (unix/Windows/etc) in its standard library files to make everything work.
The C++0x STL also uses some compiler magic to make some new language features work.