Is it possible to portablize all #pragma commands? #pragma once can be portablize as the following:
#ifndef FOO
#define FOO
//Code here
#endif
However, I am not aware of universal implementations for #pragma pack and others. Is this possible?
The specification of pragmas in the standard is essentially an implementation-defined hook - the results of every pragma is implementation-defined. The results can include causing translation to fail, the implementation or emitted code behaving in a non-conforming manner (i.e. do things with any code construct differently than the standard requires). Implementations are required to ignore pragmas they don't recognise.
By definition, therefore, there are no universal definitions. There is also nothing preventing two different compilers from doing completely different things with the same pragma.
To "portabilize" such things, it is necessary to have a consistent specification of the features and behaviours that compiler vendors can implement (and, then, that programmers can consistently use). Getting such a specification is challenging but not impossible. Firstly, a specification needs to be shared - formally (e.g. in a standard) or informally (e.g. a specification agreed between some vendors, even if it is not actually in a standard). Second, it requires requires all vendors to agree to implement it - which is not a given, even if it is in a ratified standard. Third, it requires any variations between implementations to be understood, and accepted by developers (or, at the least, a reasonable majority).
When agreement is required between multiple players (vendors, developers, committees, the list goes on) it takes more effort to get agreement. With C++, there is no body that has sufficient "pull" to guarantee that some feature will be successfully "portabilized" - or, to put it another way, there is nobody with a trump vote who can be convinced to make something universal. Getting 100% "portabilization" will be hard - practically, there will often be a "near enough" result rather than 100%.
No, pragma implementations are left up to the preprocessor or compiler author and are not specified in the C++ Standard.
See https://isocpp.org/std/the-standard.
It's easier to read than one might expect.
What you posted will not result in precisely the same behavior. #pragma once in VS does not equate to the inclusion guard #ifdef you wrote.
Related
P0040R3 (adopted 2016-06, see also N4603) introduced some extended memory management algorithms like std::uninitialized_move_n into the draft, and finally it became parts of ISO C++17. Some of them had an extra overload with a ExecutionPolicy parameter for potential support of parallelism.
However, as of now (Aug 2018), I don't find any standard library implementation shipped with the implementations of these overloads. And the documentation of implementations I've checked does not clarify it well. Specifically, (currently) they are:
libstdc++ shows it does not support P0040R3 in trunk, but actually at least std::destroy_at and std::uninitialized_move_n without ExecutionPolicy are in GCC 8.2.
libc++ has "Complete" support of P0040R3 since 4.0, but the overloads with ExecutionPolicy are actually missing.
Microsoft VC++ has support of P0040R3 since VS 2017 15.3 with /std:c++17 or /std:c++latest, but the overloads with ExecutionPolicy are actually missing.
The only implementation with ExecutionPolicy overloads I know is in HPX, but this is not a full implementation of the standard library. If I want to use the features portably, I have to adapt to custom implementation likewise, rather than direct use of std names. But I still want to use std implementation in future as preference (unless they have known bugs). (The reason is that implementation-defined execution policies are tightly coupled with concrete implementations, so external implementations as well as their client code would likely have less opportunity to utilize various execution policies in general; although this is not necessarily true for client code which is not guaranteed portable in the sense of conforming to standard.) Thus, I want something available for conditional inclusion in my potable adaptive layer for implementations - to get the specified features with using std::... when they are provided by the standard library, and complement it with my implementations as the fallback of missing parts from the standard library implementation only when necessary.
As I have known, the SD-6 feature testing macros as well as P0941R2 shows __cpp_lib_raw_memory_algorithms is sufficient for the features in P0040R3. On the other hand, __cpp_lib_parallel_algorithm seems not related to <memory> at all. So there is no way to express the state like current libc++ and MSVC implementations - with std names from P0040R3 but lack of ExecutionPolicy overloads. And I'm not sure __has_include<execution> would ever work. The reality may be quirkier, e.g. P0336R1 is even not supported by libc++.
So, how to get it perfectly portable in my code when the features become (hopefully) available in some newer version of the standard library implementations, except inspecting the source of each version of them, or totally reinventing my wheels of the whole P0040R3?
Edited:
I know the intended use of feature testing macros and I think libstdc++ has done the right thing. However, there is room to improve. More specifically, my code of the portable layer would play the role of the implementation (like HPX), but more "lightweight" in the sense of not reinventing wheels when they are already provided by the standard library implementation:
namespace my
{
#if ???
//#if __cpp_lib_raw_memory_algorithms
using std::uninitialized_move_n;
// XXX: Is this sufficient???
#else
// ... wheels here ... not expected to be more efficient to std counterparts in general
#endif
}
so my client code can be:
my::uninitialized_move_n(???::par, iter, size, d_iter);
rather than (copied from Barry's answer):
#if __cpp_lib_raw_memory_algorithms
std::uninitialized_move_n(std::execution::par, iter, size, d_iter);
#else
// ???
#endif
Both pieces of the code can work, but obviously checking __cpp_lib_raw_memory_algorithms directly everywhere in client code is more costly.
Ideally I should have some complete up-to-date standard library implementation, but that is not always the case I can guarantee (particularly working with environments where the standard library is installed as parts of system libraries). I need the adaption to ease the clients' work anyway.
The fallback is obvious: avoiding the using std::uninitialized_move_n; path totally. I'm afraid this would be a pessimistic implementation so I want to avoid this approach when possible.
Second update:
Because "perfectly portable" sounds unclear, I have illustrated some code in the edit above. Although the question is not changed and still covered by the title, I will make it more concrete here.
The "perfectly portable" way I want in the question is restricted as, given the code like the edit above, filling up any parts marked in ???, without relying on any particular versions of language implementations (e.g., nothing like macro names depended on implementations should be used for the purpose).
See here and here for the code examples fail to meet the criteria. (Well, these versions are figured out via inspection of commit logs... certainly imperfect, and, still buggy in some cases.) Note this is not related to the overloads with ExecutionPolicy yet, because they are missing in the mentioned standard library implementations, and my next action is depending on the solution of this question. (But the future of the names in std should be clear.)
A perfect (enough) solution can be, for example, adding a new feature testing macro to make the overloads independent from __cpp_lib_raw_memory_algorithms so in future I can just add my implementation of the overloads with ExecutionPolicy when they are not detected by the stand-alone new feature testing macro, without messing up the condition of #if again. But certainly I can't guarantee this way would be feasible; it ultimately depends on the decision of the committee and vendors.
I'm not sure whether there can be other directions.
The initial version of P0941 contained a table which made it clear that P0040R3 has the corresponding feature-test macro __cpp_lib_raw_memory_algorithms. This implies that the correct, portable way to write code to conditionally use this feature is:
#if __cpp_lib_raw_memory_algorithms
std::uninitialized_move_n(std::execution::par, iter, size, d_iter);
#else
// ???
#endif
The imposed requirement is that if that macro is defined, then that function exists and does what the standard prescribes. But that macro not being defined does not really say anything. As you point out, there are parts of P0040R3 that are implemented in libstdc++ - parts, but not all, which is why the feature-test macro is not defined.
There is currently a concerted effort to implement the parallel algorithms in libstdc++.
As to what to do in the #else branch there, well... you're kind of on your own.
In functions like std::atomic::compare_exchange, there's runtime parameters like std::memory_order_release, std::memory_order_relaxed.
(http://en.cppreference.com/w/cpp/atomic/atomic/compare_exchange)
I'm not sure is these memory order flags are guaranteed to exist in all kinds of CPU/architects, if some CPU don't support a flag, does this flag lead to crash or? Seems some of these flags are designed for intel Itanium, etc, not sure if std::memory_order related code is portable or not.
Would you give some suggestions?
The C++ standard does have the concept of so-called "freestanding" implementations, which can support a subset of the standard library. However, it also defines bare-minimum functionality that even a freestanding implementation must support. Within that list is the entirety of the <atomic> header.
So yes, implementations must support these.
However, that doesn't mean that a particular flag will do exactly and only what that flag describes. The flags represent the minimum memory barrier, the specific things that are guaranteed to be visible. The implementation could issue a full memory barrier even for flags that don't require it, if the hardware implementation doesn't have lower-tier memory barriers.
So you should write code against what the standard says, and let the compiler sort out the details. If it proves to be inefficient on a platform, you can inspect the assembly to see if you might be able to improve matters.
But to answer your main question, yes, atomic-based code is portable (modulo compiler bugs).
In general, compilers are free to only provide the strongest memory guarantees regardless of what you ask for.
On some platforms, there are relaxed guarantees that are sufficient. Not all platforms support these relaxed guarantees. On those platforms, compilers must provide strictly stronger guarantees.
So they are portable, in that comforming compilers must provide that guarantee or better when you ask for a paticular guarantee.
Note that it isn't just the hardware that is of concern. Certain optimizations and reorderings may be legal or not depending on what memory order guarantee you ask for. I am unaware of any compiler that relies on that, but I am not a compiler expert.
In practice, consume semantic is always strengthened to acquire by current compilers because it turned out to be very hard to implement safely without doing that. Either:
the architecture provides acquire on all loads, and then load consume does the same thing as load acquire which does the same as load relaxed: nothing special (like x86);
the architecture requires an acquire barrier even on dependent accesses (very very rare, maybe only DEC Alpha): then the compiler will use an acquire barrier on consume;
The ISA guarantees dependency ordering for loads in asm, but full acquire needs a barrier. (This is what consume is trying to expose to the programmer). The compiler should provide you the benefit of avoiding the barrier with logical (not crazy) uses of consume
either the compiler tries to do that, but it's tricky and fails in some corner cases that the compiler back end optimizations break (the front end often does not communicate with its back end enough to avoid these just for consume);
or you don't trust the compiler, set optimization to zero, which doesn't even guarantee that no trivial optimization done implicitly by the back end will break consume (and make performance very bad);
or the compiler writers did not care about efficiency or knew they couldn't do a reliable job providing consume so they provide acquire instead, which is semantically correct but really less efficient, and not the intent of the standard.
(And C++'s consume semantic is crazy. It's one of the most inconsistent part of C++ which tells you a lot.)
The nonstandard #pragma once feature is implemented on practically all C++ compilers, but the C++ standard excludes it.
The usual explanation of why #pragma once, or some language construct that does what #pragma once does, has been excluded from the C++ standard is that hard links and copied header files either break #pragma once or provoke the compiler to heuristics. Fair enough, heuristics are normally incompatible with the C++ philosophy anyway, but regarding plain breakage: there are many useful language features you can break, not only #pragma once. The normal C++ way to manage such breakage is to let the compiler issue an optional warning in doubtful instances. After all, C++ is purposely designed to let one program unsafely and/or unportably when one wishes to do so. Besides, the unsafety and/or unportability of #pragma once is pretty minimal. It just isn't that easy to abuse.
Why is #pragma once excluded from the standard when other abusable but useful language features are typically included? Is there something special about #pragma once?
Also, where can one read the recent deliberations of the standards committee in the matter? Has some committee member, or committee follower, published a recent summary of the debate?
There are a few simple reasons:
It is harder than generally assumed to implement and specify it. The argument that it is implemented doesn't hold much water as the implementations generally do not deal with approaches to subvert the feature.
Committee time is much more reasonably spent on working on modules which make most of the preprocessor unnecessary than trying to improve something we want to get rid of.
There is a simple work-around around (include guards) for the absence of #pragma once, i.e., it isn't considered a problem.
It seems, existing implementations actually do behave different which seems to be the root of one of the recent discussions. Of course, this means that standardization would be good but then it immediately starts conflicting with 2. and the discussions won't be simple because different parties would want their respective behavior to be retained.
I didn't do a too thorough search but I didn't see a proposal, either: if nobody writes a proposal [and lobbies it through the process] nothing will be standardized. That said, I'd fully expect the reasons given above to stop a proposal to add #pragma once have a sufficient majority for it to be stopped quite quickly.
There was a recent discussion on the proposals mailing list (see isocpp.org for how to sign up; I can't get to this site at the moment, though). I didn't follow it too thoroughly, though. Quickly browsing over it I saw the four reasons given above (the forth I added after browsing).
Here are some references from the recent mailing list discussion:
Is #pragma once part of the standard?
Why isn't C/C++s #pragma once standard?
modules proposal
From my understanding, #pragma once is an implementation specific instance of the standard #pragma directive as described in Section §16.6 of the Standard (draft):
16.6 Pragma directive [cpp.pragma]
A preprocessing directive of the form
# pragma pp-tokens opt new-line causes the implementation to behave in an implementation-
defined manner. The behavior might cause translation
to fail or cause the translator or the resulting program to behave in
a non-conforming manner. Any pragma that is not recognized by the
implementation is ignored.
Having pragma once standardized would introduce quite a bit of complexity.
Give also a look here: https://stackoverflow.com/a/1696194/2741329
Traditionally, the standard and portable way to avoid multiple header inclusions in C++ was/is to use the #ifndef - #define - #endifpre-compiler directives scheme also called macro-guard scheme (see code snippet below).
#ifndef MY_HEADER_HPP
#define MY_HEADER_HPP
...
#endif
In most implementations/compilers (see picture below) however, there's a more "elegant" alternative that serves the same purpose as the macro-guard scheme called #pragma once. #pragma once has several advantages compared to the macro-guard scheme, including less code, avoidance of name clashes, and sometimes improved compile speed.
Doing some research, I realized that although #pragma once directive is supported by almost all known compilers, there's a turbidness on whether #pragma once directive is part of the C++11 standard or not.
Questions:
Could someone clarify whether #pragma once directive is part of the C++11 standard or not?
If it's not part of the C++11 standard, are there any plans on including it on later releases (e.g., C++14 or later)?
It would also be nice if someone could further elaborate on the advantages/disadvantages in using either one of the techniques (i.e., macro-guard versus #pragma once).
#pragma once is not standard. It is a widespread (but not
universal) extension, which can be used
if your portability concerns are limited, and
you can be sure that all of your include files are always on a local disk.
It was considered for standardization, but rejected because it
cannot be implemented reliably. (The problems occur when you
have files accessible through several different remote mounts.)
It's fairly easy to ensure that there are no include guard
conflicts within a single development. For libraries, which may
be used by many different developments, the obvious solution is
to generate a lot of random characters for the include guard
when you create it. (A good editor can be set up to do this for
you whenever you open a new header.) But even without this,
I've yet to encounter any problems with conflicts between
libraries.
Section §16.6 of the Standard (N3936 draft) describes #pragma directives as:
A preprocessing directive of the form
# pragma pp-tokensopt new-line
causes the implementation to behave in an implementation-defined
manner. The behavior might cause translation to fail or cause the
translator or the resulting program to behave in a non-conforming
manner. Any pragma that is not recognized by the implementation is
ignored.
Basically #pragma once is an implementation specific instance of a #pragma directive, and no, it's not standard. Yet.
It is often widely supported by most "major compilers" including GCC and Clang and is therefore sometimes recommended to avoid include-guards boilerplate.
All applicants to our company must pass a simple quiz using C as part of early screening process.
It consists of a C source file that must be modified to provide the desired functionality. We clearly state that we will attempt to compile the file as-is, with no changes.
Almost all applicants user "strlen" but half of them do not include "string.h", so it does not compile until I include it.
Are they just lazy or are there compilers that do not require you to include standard library files, such as "string.h"?
GCC will happily compile the following code as is:
main()
{
printf("%u\n",strlen("Hello world"));
}
It will complain about incompatible implicit declaration of built-in function ‘printf’ and strlen(), but it will still produce an executable.
If you compile with -Werror it won't compile.
I'm pretty sure it's non-conformant for a compiler to include headers that aren't asked for. The reason for this is that the C standard says that various names are reserved, if the relevant header is included. I think this implies they aren't reserved if they aren't included, since compilers aren't allowed to reserve names the standard doesn't say are reserved (unless of course they include a non-standard header which happens to be provided by the compiler, and is documented elsewhere to reserve extra names. Which is what happens when you use POSIX).
This doesn't fully answer your question - there do exist non-conformant compilers. As for your applicants, maybe they're just used to including "windows.h", and so have never thought before about what header strlen might be defined in by the C standard. I assume without testing that MSVC does in principle require you to include "string.h". But since "windows.h" does that for you, for the vast majority of practical Windows programs you don't need to know that you have to include "string.h".
They might be lazy, but you can't tell. Standard library implementations (as opposed to compilers, though of course each compiler usually has "its own" stdlib impl.) are allowed to include other headers. For example, #include <stdlib.h> could include every other library described in the standard. (I'm talking in the context of "C/C++", not strictly C.)
As a result, programmers get accustomed to such things, even if not strictly guaranteed, and it's easy to forget whether some function comes from a general catch-all like stdlib.h or something else—many people forget that memcpy is from string.h too.
If they do not include any headers, I would count them as wrong. If you don't allow them to test it with a particular implementation, however, it's hard to say they're wrong. And if you don't provide them with man pages (which represent the resources they'll need to know how to use on the job), then you're wrong.
At that point, you can certainly say the don't follow the exact letter of the standard; but do you want coders that get things done and know how to fix problems when they see them, or coders that worry about minutiea that won't matter?
If you provide a C file to start working with, make it have all the headers that could be needed from the beginning and ask the applicants to remove the unused ones.
The most common engineering experience is to add (or delete) a few lines of code to/from an application with thousands of lines already working correctly. It would be extremely rare in such a case to need another header file when adding a call to printf() or strlen().
It would be interesting to look over the shoulder of experienced engineers—not just graduated from school, but with extensive experience in the trenches—to see if they simply add strlen() and try compiling, or if they check to see if stdlib.h or string.h is already included before compiling. I bet the overwhelming majority do the former.
C implementations usually still allow implicit function declarations.
Anyway, I wouldn't consider all the boilerplate a required part of an interview, unless you specifically ask for it (e.g. "please don't omit anything you'd normally have in a source file").
(And with Visual Assist's "add ... include" I know less and less where they comde from ;))
Most compilers provide some kind of option to force headers inclusion.
Eg. the GCC compiler has the -include option which is the equivalent of #include preprocessor directive.
TCC will also happily compile a file such as the accepted answer's example:
int main()
{
printf("%u\n", strlen("hello world"));
}
without any warnings (unless you pass -Wall); as an added bonus, you can just call tcc -run file.c to execute file.c without compiling to an output file first.
In C89 (the most commonly applied standard), a function that is not declared will be assumed to return an int and have unknown arguments, and will compile (probably with warnings). If it does not, teh compiler is not compliant. If on the other hand you compiled the code as C++ it will fail, andf must do so if the C++ compiler is compliant.
Why not simply specify in the question that all necessary headers must be included (and perhaps irrelevant ones omitted).
Alternatively, sit the candidates at a machine with a compiler and have them check their own code, and state that the code must compile at the maximum warning level, without warnings.
I'm doing C/C++ for 20 years (even taught classes) and I guess there's a 50% probability that I'd forget the include too (99% of the time when I code, the include is already there somewhere). I know this isn't exactly answering your question, but if someone knows strlen() they will know within seconds what to do with that specific compiler error, so from a job qualification standpoint the slip is practically irrelevant.
Rather than putting emphasis on stuff like that, checking for the subtleties that require real understanding of the language should be far more relevant, i.e. buffer overruns, strncpy not appending a final \0 when hitting the limits, asking someone to make a safe (truncating) copy to a buffer of limited length, etc. Especially in C/C++ programming, the errors that do not generate a compiler error are the ones which will cause you/your company the real trouble.