T.C. left an interesting comment to my answer on this question:
Why aren't include guards in c++ the default?
T.C. states:
There's "header" and there's "source file". "header"s don't need to be
actual files.
What does this mean?
Perusing the standard, I see plenty of references to both "header files" and "headers". However, regarding #include, I noticed that the standard seems to make reference to "headers" and "source files". (C++11, § 16.2)
A preprocessing directive of the form
# include < h-char-sequence> new-line
searches a sequence of implementation-defined places for a header identified uniquely
by the specified sequence between the < and > delimiters, and causes the replacement
of that directive by the entire contents of the header. How the places are specified
or the header identified is implementation-defined.
and
A preprocessing directive of the form
# include " q-char-sequence" new-line
causes the replacement of that directive by the entire contents of the source *file*
identified by the specified sequence between the " delimiters. The named source *file*
is searched for in an implementation-defined manner.
I don't know if this is significant. It could be that "headers" in a C++ context unambiguously means "header files" but the word "sources" would be ambiguous so "headers" is a shorthand but "sources" is not. Or it could be that a C++ compiler is allowed leeway for bracket includes and only needs to act as if textual replacement takes place.
So when are header (files) not files?
The footnote mentioned by T.C. in the comments below is quite direct:
174) A header is not necessarily a source file, nor are the sequences
delimited by < and > in header names necessarily valid source file
names (16.2).
For the standard header "files" the C++ standard doesn't really make a mandate that the compiler uses a file or that the file, if it uses one, actually looks like a C++ file. Instead, the standard header files are specified to make a certain set of declarations and definitions available to the C++ program.
An alternative implementation to a file could be a readily packaged set of declarations represented in the compiler as data structure which is made available when using the corresponding #include-directive. I'm not aware of any compiler which does exactly that but clang started to implement a module system which makes the headers available from some already processed format.
They do not have to be files, since the C and C++ preprocessor are nearly identical it is reasonable to look into the C99 rationale for some clarity on this. If we look at the Rationale for International Standard—Programming Languages—C it says in section 7.1.2 Standard headers says (emphasis mine):
In many implementations the names of headers are the names of files in
special directories. This implementation technique is not required,
however: the Standard makes no assumptions about the form that a file
name may take on any system. Headers may thus have a special status if
an implementation so chooses. Standard headers may even be built into
a translator, provided that their contents do not become “known” until
after they are explicitly included. One purpose of permitting these
header “files” to be “built in” to the translator is to allow an
implementation of the C language as an interpreter in a free-standing
environment where the only “file” support may be a network interface.
It really depends on the definition of files.
If you consider any database which maps filenames to contents to be a filesystem, then yes, headers are files. If you only consider files to be that which is recognized by the OS kernel open system call, then no, headers don't have to be files.
They could be stored in a relational database. Or a compressed archive. Or downloaded over the network. Or stored in alternate streams or embedded resources of the compiler executable itself.
In the end, though, textual replacement is done, and the text comes from some sort of indexed-by-name database.
Dietmar mentioned modules and loading already processed content... but this is generally NOT allowable behavior for #include according to the C++ standard (modules will have to use a different syntax, or perhaps #include with a completely new quotation scheme other than <> or ""). The only processing that could be done in advance is tokenization. But contents of headers and included source files are subject to stateful preprocessing.
Some compilers implement "precompiled headers" which have done more processing than mere tokenization, but eventually you find some behavior that violates the Standard. For example, in Visual C++:
The compiler ... skips to just beyond the #include directive associated with the .h file, uses the code contained in the .pch file, and then compiles all code after filename.
Ignoring the actual source code prior to #include definitely does not conform to the Standard. (That doesn't prevent it from being useful, but you need to be aware that edits may not produce the expected behavior changes)
Related
A slightly strange question, however, if I remember correctly, C++ source code doesn't require a file system to store its files.
Having a compiler that scans handwritten papers via a camera would be a conforming implementation. Although practically not making that much sense.
However C++20 now adds source location with file_name. Does this now imply that source code should always be stored in a file?
No, source code doesn't have to come from a file (nor go to a file).
You can compile (and link) C++ completely within a pipe, putting your compiler in the middle, e.g.
generate_source | g++ -o- -xc++ - | do_something_with_the_binary
and it's been like that for decades. See also:
Is it possible to get GCC to read from a pipe?
How to make GCC output to stdout?
The introduction of std::source_location in C++20 doesn't change this state of affairs. It's just that some code will not have a well-defined source location (or it may be well-defined, but not very meaningful). Actually, I'd say that the insistence on defining std::source_location using files is a bit myopic... although in fairness, it's just a macro-less equivalent of __FILE__ and __LINE__ which already exist in C++ (and C).
#HBv6 notes that if you print the value of __FILE__ when compiling using GCC from the standard input stream:
echo -e '#include <iostream>\n int main(){std::cout << __FILE__ ;}' | g++ -xc++ -
running the resulting executable prints <stdin>.
Source code can even come from the Internet.
#Morwenn notes that this code:
#include <https://raw.githubusercontent.com/Morwenn/poplar-heap/master/poplar.h>
// Type your code here, or load an example.
void poplar_sort(int* data, size_t size) {
poplar::make_heap(data, data + size);
poplar::sort_heap(data, data + size);
}
works on GodBolt (but won't work on your machine - no popular compiler supports this.)
Are you a language lawyer? Ok, so let's consult the standard..
The question of whether C++ program sources need to come from files is not answered clearly in the language standard. Looking at a draft of the C++17 standard (n4713), section 5.1 [lex.separate] reads:
The text of the program is kept in units called source files in this document. A source file together with all the headers (20.5.1.2) and source files included (19.2) via the preprocessing directive #include, less any source lines skipped by any of the conditional inclusion (19.1) preprocessing directives, is called a translation unit.
So, the source code is not necessarily kept in a file per se, but in a "unit called a source file". But then, where do the includes come from? One would assume they come from named files on the filesystem... but that too is not mandated.
At any rate, std::source_location does not seem to change this wording in C++20 or to affect its interpretation.
Even before C++20, the standard has had:
__FILE__
The presumed name of the current source file (a character string literal).
The definition is the same for source_location::file_name.
As such, there has not been a change in regard to support for file system-less implementations in C++20.
The standard doesn't exactly define what "source file" means, so whether it refers to a file system may be up to interpretation. Presumably, it could be conforming for an implementation to produce "the handwritten note that you gave to me just then" if that indeed identifies the "source file" in that implementation of the language.
In conclusion: Yeah, sources are referred to as "files" by the standard, but what a "file" is and whether a file system is involved is unspecified.
T.C. left an interesting comment to my answer on this question:
Why aren't include guards in c++ the default?
T.C. states:
There's "header" and there's "source file". "header"s don't need to be
actual files.
What does this mean?
Perusing the standard, I see plenty of references to both "header files" and "headers". However, regarding #include, I noticed that the standard seems to make reference to "headers" and "source files". (C++11, § 16.2)
A preprocessing directive of the form
# include < h-char-sequence> new-line
searches a sequence of implementation-defined places for a header identified uniquely
by the specified sequence between the < and > delimiters, and causes the replacement
of that directive by the entire contents of the header. How the places are specified
or the header identified is implementation-defined.
and
A preprocessing directive of the form
# include " q-char-sequence" new-line
causes the replacement of that directive by the entire contents of the source *file*
identified by the specified sequence between the " delimiters. The named source *file*
is searched for in an implementation-defined manner.
I don't know if this is significant. It could be that "headers" in a C++ context unambiguously means "header files" but the word "sources" would be ambiguous so "headers" is a shorthand but "sources" is not. Or it could be that a C++ compiler is allowed leeway for bracket includes and only needs to act as if textual replacement takes place.
So when are header (files) not files?
The footnote mentioned by T.C. in the comments below is quite direct:
174) A header is not necessarily a source file, nor are the sequences
delimited by < and > in header names necessarily valid source file
names (16.2).
For the standard header "files" the C++ standard doesn't really make a mandate that the compiler uses a file or that the file, if it uses one, actually looks like a C++ file. Instead, the standard header files are specified to make a certain set of declarations and definitions available to the C++ program.
An alternative implementation to a file could be a readily packaged set of declarations represented in the compiler as data structure which is made available when using the corresponding #include-directive. I'm not aware of any compiler which does exactly that but clang started to implement a module system which makes the headers available from some already processed format.
They do not have to be files, since the C and C++ preprocessor are nearly identical it is reasonable to look into the C99 rationale for some clarity on this. If we look at the Rationale for International Standard—Programming Languages—C it says in section 7.1.2 Standard headers says (emphasis mine):
In many implementations the names of headers are the names of files in
special directories. This implementation technique is not required,
however: the Standard makes no assumptions about the form that a file
name may take on any system. Headers may thus have a special status if
an implementation so chooses. Standard headers may even be built into
a translator, provided that their contents do not become “known” until
after they are explicitly included. One purpose of permitting these
header “files” to be “built in” to the translator is to allow an
implementation of the C language as an interpreter in a free-standing
environment where the only “file” support may be a network interface.
It really depends on the definition of files.
If you consider any database which maps filenames to contents to be a filesystem, then yes, headers are files. If you only consider files to be that which is recognized by the OS kernel open system call, then no, headers don't have to be files.
They could be stored in a relational database. Or a compressed archive. Or downloaded over the network. Or stored in alternate streams or embedded resources of the compiler executable itself.
In the end, though, textual replacement is done, and the text comes from some sort of indexed-by-name database.
Dietmar mentioned modules and loading already processed content... but this is generally NOT allowable behavior for #include according to the C++ standard (modules will have to use a different syntax, or perhaps #include with a completely new quotation scheme other than <> or ""). The only processing that could be done in advance is tokenization. But contents of headers and included source files are subject to stateful preprocessing.
Some compilers implement "precompiled headers" which have done more processing than mere tokenization, but eventually you find some behavior that violates the Standard. For example, in Visual C++:
The compiler ... skips to just beyond the #include directive associated with the .h file, uses the code contained in the .pch file, and then compiles all code after filename.
Ignoring the actual source code prior to #include definitely does not conform to the Standard. (That doesn't prevent it from being useful, but you need to be aware that edits may not produce the expected behavior changes)
I am aware that questions about the difference between <header> and <header.h> have been asked before. And after reading those answers, I have enlisted the following differences
Of course iostream.h is deprecated, it is not supported by newer standard-complying compilers
iostream.hdoesn't contain everything inside the std namespace and doesn't make use of templates.
Okay.
But, after reading a few books and a few answers (like this), I have inferred that #include<iostream.h> includes a specific file called iostream.h in our program whereas, #include<iostream> is NOT even required to map to a file at all. It simply guarantees that everything belonging to the iostream library is included in our program. Am I correct?
No "system" header is required to be a file. Inclusion using <> is specified thusly:
C++11 16.2 [cpp.include]/2: searches a sequence of implementation-defined places for a header identified uniquely by the specified sequence between the < and > delimiters, and causes the replacement of that directive by the entire contents of the header. How the places are specified or the header identified is implementation-defined.
So the declarations from headers known to the implementation (which might or might not include current and/or obsolete standard library headers) can be made available without loading and preprocessing a text file, if the implementor deems that to be a good idea.
Including with "" will first search for a file (in implementation-defined places), and fall back to <> if that fails.
Accelerated C++: Practical Programming by Example book says the following..
... system header files need not be implemented as files. Even though the #include
directive is used to access both our own header files and system headers, there
is no requirement that they be implemented in the same way
What exactly does this mean? If not as a file how else can a system header file be implemented?
Imagine you write your own compiler and C++ standard library. You could make it so that #include <vector> does not open any file, but instead simply loads some state into the compiler which makes it understand std::vector. You could then implement your vector class in some language other than C++, so long as your compiler understands enough to make it work "as if" you had written an actual C++ source file called vector.
The compiler could have hardcoded that when it sees:
#include <iostream>
then it makes available all definitions of things that are specified as being declared by this directive, etc.
Or it could store the definitions in a database, or some other encoded file, or the cloud, or whatever. The point is that the standard does not restrict the compiler in any way, so long as the end goal is achieved that the specified things get declared.
The way in which headers are included into your "source file stream" is left mostly up to the implementation.
C++11 (but this has been the case for a long time, both in C++ and C) 16.2 Source file inclusion states:
A #include directive shall identify a header or source file that can be processed by the implementation.
A preprocessing directive of the form # include < h-char-sequence> new-line searches a sequence of implementation-defined places for a header identified uniquely by the specified sequence between the < and > delimiters, and causes the replacement of that directive by the entire contents of the header. How the places are specified or the header identified is implementation-defined.
(and then further description of the " and naked variants of #include).
So the header may be in a file.
It may also be injected by the compiler from hard-coded values.
Or read from a server located on one of the planets orbiting Betelgeuse (though, without FTL transmissions, such a compiler wouldn't last long in the marketplace).
The possibilities are many and varied, most of them bordering on lunacy but none of them actually forbidden by the standard itself.
For the purposes of this question, I am interested only in Standard-Compliant C++, not C or C++0x, and not any implementation-specific details.
Questions arise from time to time regarding the difference between #include "" and #include <>. The argument typically boils down to two differences:
Specific implementations often search different paths for the two forms. This is platform-specific, and not in the scope of this question.
The Standard says #include <> is for "headers" whereas #include "" is for a "source file." Here is the relevant reference:
ISO/IEC 14882:2003(E)
16.2 Source file inclusion [cpp.include]
1 A #include directive shall identify a header or source file that can be processed by the implementation.
2 A preprocessing directive of the form
# include < h-char-sequence > new-line
searches a sequence of implementation-defined places for a header identified uniquely by the specified sequence between the < and > delimiters, and causes the replacement of that directive by the entire contents of the header. How the places are specified or the header identified is implementation-defined.
3 A preprocessing directive of the form
# include "q-char-sequence" new-line
causes the replacement of that directive by the entire contents of the source file identified by the specified sequence between the " delimiters. The named source file is searched for in an implementation-defined manner. If this search is not supported, or if the search fails, the directive is reprocessed as if it read
# include < h-char-sequence > new-line
with the identical contained sequence (including > characters, if any) from the original directive.
(Emphasis in quote above is mine.) The implication of this difference seems to be that the Standard intends to differentiate between a 'header' and a 'source file', but nowhere does the document define either of these terms or the difference between them.
There are few other places where headers or source files are even mentioned. A few:
158) A header is not necessarily a source file, nor are the sequences delimited by in header names necessarily valid source file names (16.2).
Seems to imply a header may not reside in the filesystem, but it doesn't say that source files do, either.
2 Lexical conventions [lex]
1 The text of the program is kept in units called source files in this International Standard. A source file together with all the headers (17.4.1.2) and source files included (16.2) via the preprocessing directive #include, less any source lines skipped by any of the conditional inclusion (16.1) preprocessing directives, is called a translation unit. [Note: a C + + program need not all be translated at the same time. ]
This is the closest I could find to a definition, and it seems to imply that headers are not the "text of the program." But if you #include a header, doesn't it become part of the text of the program? This is a bit misleading.
So what is a header? What is a source file?
My reading is that the standard headers, included by use of <> angle brackets, need not be actual files on the filesystem; e.g. an implementation would be free to enable a set of "built-in" operations providing the functionality of iostream when it sees #include <iostream>.
On the other hand, "source files" included with #include "xxx.h" are intended to be literal files residing on the filesystem, searched in some implementation-dependent manner.
Edit: to answer your specific question, I believe that "headers" are limited only to those #includeable facilities specified in the standard: iostream, vector and friends---or by the implementation as extensions to the standard. "Source files" would be any non-standard facilities (as .h files, etc.) the programmer may write or use.
Isn't this saying that a header may be implemented as a source file, but there again may not be? as for "what is a source file", it seems very sensible for the standard not to spell this out, given the many ways that "files" are implemented.
The standard headers (string, iostream) don't necessarily have to be files with those names, or even files at all. As long as when you say
#include <iostream>
a certain list of declarations come into scope, the Standard is satisfied. Exactly how that comes about is an implementation detail. (when the Standard was being written, DOS could only handle 8.3 filenames, but some of the standard header names were longer than that)
As your quotes say: a header is something included using <>, and a source file is the file being compiled, or something included using "". Exactly where the contents of these come from, and what non-standard headers are available, is up to the implementation. All the Standard specifies is what is defined if you include the standard headers.
By convention, headers are generally system-wide things, and source files are generally local to a project (for some definition of project), but the standard wisely doesn't get bogged down in anything to do with project organisation; it just gives very general definitions that are compatible with such conventions, leaving the details to the implementation and/or the user.
Nearly all of the standard deals with the program after it's been preprocessed, at which time there are no such things as source files or headers, just the translations units that your last quote defines.
Hmmm...
My casual understanding has been that the distinction between <> includes and "" includes was inherited from c and (though not defined by the standards) the de facto meaning was that <> searched paths for system and compiler provided headers and "" also searched local and user specified paths.
The definition above seem to agree in some sense with that usage, but restricts the use of "header" to things provided by the compiler or system exclusive of code provided by the user, even if they have the traditional "interface goes in the header" form.
Anyway, very interesting.