It is known that in C++ string literals are immutable and the result of modifying a string literal is undefined. For example
char * str = "Hello!";
str[1] = 'a';
This will bring to an undefined behavior.
Besides that string literals are placed in static memory. So they exists during whole program. I would like to know why do string literals have such properties.
There are a couple of different reasons.
One is to allow storing string literals in read-only memory (as others have already mentioned).
Another is to allow merging of string literals. If one program uses the same string literal in several different places, it's nice to allow (but not necessarily require) the compiler to merge them, so you get multiple pointers to the same memory, instead of each occupying a separate chunk of memory. This can also apply when two string literals aren't necessarily identical, but do have the same ending:
char *foo = "long string";
char *bar = "string";
In a case like this, it's possible for bar to be foo+5 (if I'd counted correctly).
In either of these cases, if you allow modifying a string literal, it could modify the other string literal that happens to have the same contents. At the same time, there's honestly not a lot of point in mandating that either -- it's pretty uncommon to have enough string literals that you could overlap that most people probably want the compiler to run slower just to save (maybe) a few dozen bytes or so of memory.
By the time the first standard was written, there were already compilers that used all three of these techniques (and probably a few others besides). Since there was no way to describe one behavior you'd get from modifying a string literal, and nobody apparently thought it was an important capability to support, they did the obvious: said even attempting to do so led to undefined behavior.
It's undefined behaviour to modify a literal because the standard says so. And the standard says so to allow compilers to put literals in read only memory. And it does this for a number of reasons. One of which is to allow compilers to make the optimisation of storing only one instance of a literal that is repeated many times in the source.
I believe you ask about the reason why literals are placed in
read-only memory, not about technical details of linker doing this and
that or legal details of a standard forbidding such and such.
When modification of string literals works, it leads to subtle bugs
even in the absence of string merging (which we have reasons to
disallow if we decided to permit modification). When you see code like
char *str="Hello";
.../* some code, but str and str[...] are not modified */
printf("%s world\n", str);
it's a natural conclusion that you know what's going to be printed,
because str (and its content) were not modified in a particular
place, between initialization and use.
However, if string literals are writable, you don't know it any
more: str[0] could be overwritten later, in this code or inside a
deeply nested function call, and when the code is run again,
char *str="Hello";
won't guarantee anything about the content of str anymore. As we
expect, this initialization is implemented as moving the address known
in link time into a place for str. It does not check that str
contains "Hello" and it does not allocate a new copy of it. However,
we understand this code as resetting str to "Hello". It's hard to
overcome this natural understanding, and it's hard to reason about the
code where it is not guaranteed. When you see an expression like
x+14, what if you had to think about 14 being possibly overwritten
in other code, so it became 42? The same problem with strings.
That's the reason to disallow modification of string literals, both in
the standard (with no requirement to detect the failure early) and in
actual target platforms (providing the bonus of detecting potential
bugs).
I believe that many attempts to explain this thing suffer from the
worst kind of circular reasoning. The standard forbids writing to
literals because the compiler can merge strings, or they can be placed
in read-only memory. They are placed in read-only memory to catch the
violation of the standard. And it's valid to merge literals because
the standard forbids... is it a kind of explanation you asked for?
Let's look at other
languages. Common Lisp standard
makes modification of literals undefined behaviour, even though the
history of preceding Lisps is very different with the history of C
implementations. That's because writable literals are logically
dangerous. Language standards and memory layouts only reflect that
fact.
Python language has exactly one place where something resembling
"writing to literals" can happen: parameter default values, and this
fact confuses people all the time.
Your question is tagged C++, and I'm unsure of its current state
with respect to implicit conversion to non-const char*: if it's a
conversion, is it deprecated? I expect other answers to provide a
complete enlightenment on this point. As we talk of other languages
here, let me mention plain C. Here, string literals are not const,
and an equivalent question to ask would be why can't I modify string
literals (and people with more experience ask instead, why are
string literals non-const if I can't modify them?). However, the
reasoning above is fully applicable to C, despite this difference.
Because is K&R C, there was not such thing as "const". And similarly in pre-ANSI C++. Hence there was a lot of code which had things like char * str = "Hello!"; If the Standards committee made text literals const, all those programs would have no longer compiled. So they made a compromise. Text literals are official const char[], but they have a silent implicit conversion to char*.
In C++, string literals are const because you aren't allowed
to modify them. In standard C, they would have been const as
well, except that when const was introduced into C, there was
so much code along the lines of char* p = "somethin"; that
making them const would have broke, that it was deemed
unacceptable. (The C++ committee chose a different solution to
this problem, with a deprecated implicit conversion which allows
the above.)
In the original C, string literals were not const, and were
mutable, and it was garanteed that no two string literals shared
any memory. This was quickly realized to be a serious error,
allowing things like:
void
mutate(char* p)
{
static char c = 'a';
*p = a ++;
}
and in another module:
mutate( "hello" ); // Can't trust what is written, can you.
(Some early implementations of Fortran had a similar issue,
where F(4) might call F with just about any integral value.
The Fortran committee fixed this, just like the C committee
fixed string literals in C.)
Related
I'm currently learning pointers and my professor provided this piece of code as an example:
//We cannot predict the behavior of this program!
#include <iostream>
using namespace std;
int main()
{
char * s = "My String";
char s2[] = {'a', 'b', 'c', '\0'};
cout << s2 << endl;
return 0;
}
He wrote in the comments that we can't predict the behavior of the program. What exactly makes it unpredictable though? I see nothing wrong with it.
The behaviour of the program is non-existent, because it is ill-formed.
char* s = "My String";
This is illegal. Prior to 2011, it had been deprecated for 12 years.
The correct line is:
const char* s = "My String";
Other than that, the program is fine. Your professor should drink less whiskey!
The answer is: it depends on what C++ standard you're compiling against. All the code is perfectly well-formed across all standards‡ with the exception of this line:
char * s = "My String";
Now, the string literal has type const char[10] and we're trying to initialize a non-const pointer to it. For all other types other than the char family of string literals, such an initialization was always illegal. For example:
const int arr[] = {1};
int *p = arr; // nope!
However, in pre-C++11, for string literals, there was an exception in §4.2/2:
A string literal (2.13.4) that is not a wide string literal can be converted to an rvalue of type “pointer to char”; [...]. In either case, the result is a pointer to the first element of the array. This conversion is considered only when there is an explicit appropriate pointer target type, and not when there is a general need to convert from an lvalue to an rvalue. [Note: this conversion is deprecated. See Annex D. ]
So in C++03, the code is perfectly fine (though deprecated), and has clear, predictable behavior.
In C++11, that block does not exist - there is no such exception for string literals converted to char*, and so the code is just as ill-formed as the int* example I just provided. The compiler is obligated to issue a diagnostic, and ideally in cases such as this that are clear violations of the C++ type system, we would expect a good compiler to not just be conforming in this regard (e.g. by issuing a warning) but to fail outright.
The code should ideally not compile - but does on both gcc and clang (I assume because there's probably lots of code out there that would be broken with little gain, despite this type system hole being deprecated for over a decade). The code is ill-formed, and thus it does not make sense to reason about what the behavior of the code might be. But considering this specific case and the history of it being previously allowed, I do not believe it to be an unreasonable stretch to interpret the resulting code as if it were an implicit const_cast, something like:
const int arr[] = {1};
int *p = const_cast<int*>(arr); // OK, technically
With that, the rest of the program is perfectly fine, as you never actually touch s again. Reading a created-const object via a non-const pointer is perfectly OK. Writing a created-const object via such a pointer is undefined behavior:
std::cout << *p; // fine, prints 1
*p = 5; // will compile, but undefined behavior, which
// certainly qualifies as "unpredictable"
As there is no modification via s anywhere in your code, the program is fine in C++03, should fail to compile in C++11 but does anyway - and given that the compilers allow it, there's still no undefined behavior in it†. With allowances that the compilers are still [incorrectly] interpreting the C++03 rules, I see nothing that would lead to "unpredictable" behavior. Write to s though, and all bets are off. In both C++03 and C++11.
†Though, again, by definition ill-formed code yields no expectation of reasonable behavior
‡Except not, see Matt McNabb's answer
Other answers have covered that this program is ill-formed in C++11 due to the assignment of a const char array to a char *.
However the program was ill-formed prior to C++11 also.
The operator<< overloads are in <ostream>. The requirement for iostream to include ostream was added in C++11.
Historically, most implementations had iostream include ostream anyway, perhaps for ease of implementation or perhaps in order to provide a better QoI.
But it would be conforming for iostream to only define the ostream class without defining the operator<< overloads.
The only slightly wrong thing that I see with this program is that you're not supposed to assign a string literal to a mutable char pointer, though this is often accepted as a compiler extension.
Otherwise, this program appears well-defined to me:
The rules that dictate how character arrays become character pointers when passed as parameters (such as with cout << s2) are well-defined.
The array is null-terminated, which is a condition for operator<< with a char* (or a const char*).
#include <iostream> includes <ostream>, which in turn defines operator<<(ostream&, const char*), so everything appears to be in place.
You can't predict the behaviour of the compiler, for reasons noted above. (It should fail to compile, but may not.)
If compilation succeeds, then the behaviour is well-defined. You certainly can predict the behaviour of the program.
If it fails to compile, there is no program. In a compiled language, the program is the executable, not the source code. If you don't have an executable, you don't have a program, and you can't talk about behaviour of something that doesn't exist.
So I'd say your prof's statement is wrong. You can't predict the behaviour of the compiler when faced with this code, but that's distinct from the behaviour of the program. So if he's going to pick nits, he'd better make sure he's right. Or, of course, you might have misquoted him and the mistake is in your translation of what he said.
As others have noted, the code is illegitimate under C++11, although it was valid under earlier versions. Consequently, a compiler for C++11 is required to issue at least one diagnostic, but behavior of the compiler or the remainder of the build system is unspecified beyond that. Nothing in the Standard would forbid a compiler from exiting abruptly in response to an error, leaving a partially-written object file which a linker might think was valid, yielding a broken executable.
Although a good compiler should always ensure before it exits that any object file it is expected to have produced will be either valid, non-existent, or recognizable as invalid, such issues fall outside the jurisdiction of the Standard. While there have historically been (and may still be) some platforms where a failed compilation can result in legitimate-appearing executable files that crash in arbitrary fashion when loaded (and I've had to work with systems where link errors often had such behavior), I would not say that the consequences of syntax errors are generally unpredictable. On a good system, an attempted build will generally either produce an executable with a compiler's best effort at code generation, or won't produce an executable at all. Some systems will leave behind the old executable after a failed build, since in some cases being able to run the last successful build may be useful, but that can also lead to confusion.
My personal preference would be for disk-based systems to to rename the output file, to allow for the rare occasions when that executable would be useful while avoiding the confusion that can result from mistakenly believing one is running new code, and for embedded-programming systems to allow a programmer to specify for each project a program that should be loaded if a valid executable is not available under the normal name [ideally something which which safely indicates the lack of a useable program]. An embedded-systems tool-set would generally have no way of knowing what such a program should do, but in many cases someone writing "real" code for a system will have access to some hardware-test code that could easily be adapted to the purpose. I don't know that I've seen the renaming behavior, however, and I know that I haven't seen the indicated programming behavior.
A quick question: why doesn't C++11 offer a "user-" (really, standard library) defined literal for creating a std::string, such as
auto str = "hello world"s; // str is a std::string
Like C++, Objective-C supports both C-style strings and a more user-friendly library type NSString; however, unlike C++, nobody ever gets confused between the two, because it's trivial to create an NSString by prefixing a string literal with #. In fact, it's rare to see Objective-C code where a literal doesn't have the # prefix. Everybody just knows that's the way to do it, uses NSString and gets on with it.
C++11 user-defined literals would allow this, and in fact the UDL section on Stroustrup's C++11 FAQ uses exactly this example. Furthermore, UDLs without a leading underscore are reserved, so there would be no problem in allowing a plain s as above -- there's no possibility of it clashing with anything else.
Perhaps I'm missing something, but it seems like this would be a very valuable and risk-free addition to the language, so does anybody know why C++11 doesn't offer it? Is it likely to appear in C++14?
User-defined literals were new enough in C++11 that this wasn't included, but it is in C++14( in §21.7, in case anybody cares). And yes, it uses s as the suffix. The precise result type depends on the type of the string literal itself -- a "narrow" literal gives an std::string, a u16 literal gives a u16string, and a u32 literal gives a u32string. Oh, and yeah, if you insist on using a wide string literal it'll produce a wstring.
Note that the s suffix is also used to mean seconds (defined in <chrono>) but there's no real conflict between the two--s on a string literal means a string, and s on a number means seconds.
Both are equivalent in that they return the length of the null-terminated character sequence. Are there reasons of preferring one to the other?
Use the simpler alternative. std::char_traits::length is great and all, but for C strings it does the same and is much longer code.
Do yourself a favour and avoid code bloat. I’m a huge fan of C++ functions over C equivalent (e.g. I will never use std::strcpy or std::memcpy, there’s a perfectly fine std::copy). But avoiding std::strlen is just silly.
One reason to use C++ functions exclusively is interface uniformity: for instance, both std::strcpy and std::memcpy have atrocious interfaces. However, std::strlen is a perfectly fine algorithm in the best tradition of C++. It doesn’t generalise, true, but the neither do other class-specific free functions found in the standard library.
std::strlen() is a holdover from the C Standard Library and only operates on a const char* (it is unsafe in that it has undefined behavior if the string is not null terminated). If the string is using a wide character set (e.g. const unsigned short*), std::strlen() is useless.
std::char_traits<T>::length() will operate on whatever the T type is (e.g. if it is an unsigned short, it will still operate properly, but also requires a null terminated value - that is the last value must be T(0) - if an array of T's passed to it is not null terminated, behavior is undefined as well).
In general, when dealing with strings, it is better to use std::string::length() instead of using C-style character strings.
Both Zack Howland and Konrad Rudolph have a point. Thanks. I accept both answers. The summarized reply would be:
There doesn't seem to be any except personal preference either for shorter code or the C++ standard library (I leave out generalization since it wasn't the point of the question as can be seen from the title).
std::strlen() is a C standard library compatibility (even though it is part of ISO C++) function that takes const char* as an argument. length() is a method of the std::string family of classes. So if you want to use strlen() on std::string you'd have to write:
strlen(mystring.c_str())
which is less tidy than mystr.length(). Apart from that, there should be no tangible difference (for char type, that is).
With this code, I get a segmentation fault:
char* inputStr = "abcde";
*(inputStr+1)='f';
If the code was:
const char* inputStr = "abcde";
*(inputStr+1)='f';
I will get compile error for "assigning read-only location".
However, for the first case, there is no compile error; just the segmentation fault when the assign operation actually happened.
Can anyone explain this?
Here is what the standard says about string literals in section [2.13.4/2]:
A string literal that does not begin with u, U, or L is an ordinary string literal, also referred to as a narrow string literal. An ordinary string literal has type “array of n const char”, where n is the size of the string as defined below; it has static storage duration (3.7) and is initialized with the given characters.
So, strictly speaking, "abcde" has type
const char[6]
Now what happens in your code is an implicit cast to
char*
so that the assignment is allowed. The reason why it is so is, likely, compatibility with C. Have a look also at the discussion here: http://learningcppisfun.blogspot.com/2009/07/string-literals-in-c.html
Once the cast is done, you are syntactically free to modify the literal, but it fails because the compiler stores the literal in a non writable segment of memory, as the standard itself allow.
This gets created in the code segment:
char *a = "abcde";
Essentially it's const.
If you wish to edit it, try:
char a[] = "abcde";
The standard states that you are not allowed to modify string literals directly, regardless of whether you mark them const or not:
Whether all string literals are distinct (that is, are stored in nonoverlapping
objects) is implementation-defined. The effect of attempting to modify a string literal is undefined.
In fact, in C (unlike C++), string literals are not const but you're still not allowed to write to them.
This restriction on writing allows certain optimisations to take place, such as sharing of literals along the lines of:
char *ermsg = "invalid option";
char *okmsg = "valid option";
where okmsg can actually point to the 'v' character in ermsg, rather than being a distinct string.
String literals are typically stored in read-only memory. Trying to change this memory will kill your program.
Here's a good explanation: Is a string literal in c++ created in static memory?
It is mostly ancient history; once upon a long time ago, string literals were not constant.
However, most modern compilers place string literals into read-only memory (typically, the text segment of your program, where your code also lives), and any attempt to change a string literal will yield a core dump or equivalent.
With G++, you can most certainly get the compilation warning (-Wall if it is not enabled by default). For example, G++ 4.6.0 compiled on MacOS X 10.6.7 (but running on 10.7) yields:
$ cat xx.cpp
int main()
{
char* inputStr = "abcde";
*(inputStr+1)='f';
}
$ g++ -c xx.cpp
xx.cpp: In function ‘int main()’:
xx.cpp:3:22: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
$
So the warning is enabled by default.
What happened is that the compiler put the constant "abcde" in some read-only memory segment. You pointed your (non-const) char* inputStr at that constant, and kaboom, segfault.
Lesson to be learned: Don't invoke undefined behavior.
Edit (elaboration)
However, for the first case, there is no compile error, just segmentation fault when the assign operation actually happened.
You need to enabled your compiler warnings. Always set your compiler warnings as high as possible.
Even though "abcde" is a string literal, which should not be modified, you've told the compiler that you don't care about that by having a non-const char* point to it.
The compiler will happily assume that you know what you're doing, and not throw an error. However, there's a good chance that the code will fail at runtime when you do indeed try to modify the string literal.
String literals, while officially non-const, are almost always stored in read-only memory. In your setup, this is apparently only the case if it is declared as const char array.
Note that the standard forbids you to modify any string literal.
a little bit of history of string literals in Ritchie's words.
mostly about the orgin and the evolution of string literals from K&R 1.
Hope this might clarify a thing or two about const and string literals.
"From: Dennis Ritchie
Subject: Re: History question: String literals.
Date: 02 Jun 1998
Newsgroups: comp.std.c
At the time that the C89 committee was working, writable
string literals weren't "legacy code" (Margolin) and what standard
there existed (K&R 1) was quite explicit (A.2.5) that
strings were just a way of initializing a static array.
And as Barry pointed out there were some (mktemp) routines
that used this fact.
I wasn't around for the committee's deliberations on the
point, but I suspect that the BSD utility for fiddling
the assembler code to move the initialization of strings
to text instead of data, and the realization that most
literal strings were not in fact overwritten, was more
important than some very early version of gcc.
Where I think the committee might have missed something
is in failure to find a formulation that explained
the behavior of string literals in terms of const.
That is, if "abc" is an anonymous literal of type
const char [4]
then just about all of its properties (including the
ability to make read-only, and even to share its storage
with other occurrences of the same literal) are nearly
explained.
The problem with this was not only the relatively few
places that string literals were actually written on, but much
more important, working out feasible rules for assignments
to pointers-to-const, in particular for function's actual
arguments. Realistically the committee knew that whatever
rules they formulated could not require a mandatory
diagnostic for every func("string") in the existing world.
So they decided to leave "..." of ordinary char array
type, but say one was required not to write over it.
This note, BTW, isn't intended to be read as a snipe
at the formulation in C89. It is very hard to get things
both right (coherent and correct) and usable (consistent
enough, attractive enough).
Dennis
"
Is there any good reason to use C-strings in C++ nowadays? My textbook uses them in examples at some points, and I really feel like it would be easier just to use a std::string.
The only reasons I've had to use them is when interfacing with 3rd party libraries that use C style strings. There might also be esoteric situations where you would use C style strings for performance reasons, but more often than not, using methods on C++ strings is probably faster due to inlining and specialization, etc.
You can use the c_str() method in many cases when working with those sort of APIs, but you should be aware that the char * returned is const, and you should not modify the string via that pointer. In those sort of situations, you can still use a vector<char> instead, and at least get the benefit of easier memory management.
A couple more memory control notes:
C strings are POD types, so they can be allocated in your application's read-only data segment. If you declare and define std::string constants at namespace scope, the compiler will generate additional code that runs before main() that calls the std::string constructor for each constant. If your application has many constant strings (e.g. if you have generated C++ code that uses constant strings), C strings may be preferable in this situation.
Some implementations of std::string support a feature called SSO ("short string optimization" or "small string optimization") where the std::string class contains storage for strings up to a certain length. This increases the size of std::string but often significantly reduces the frequency of free-store allocations/deallocations, improving performance. If your implementation of std::string does not support SSO, then constructing an empty std::string on the stack will still perform a free-store allocation. If that is the case, using temporary stack-allocated C strings may be helpful for performance-critical code that uses strings. Of course, you have to be careful not to shoot yourself in the foot when you do this.
Because that's how they come from numerous API/libraries?
Let's say you have some string constants in your code, which is a pretty common need. It's better to define these as C strings than as C++ objects -- more lightweight, portable, etc. Now, if you're going to be passing these strings to various functions, it's nice if these functions accept a C string instead of requiring a C++ string object.
Of course, if the strings are mutable, then it's much more convenient to use C++ string objects.
If a function needs a constant string I still prefer to use 'const char*' (or const wchar_t*) even if the program uses std::string, CString, EString or whatever elsewhere.
There are just too many sources of strings in a large code base to be sure the caller will have the string as a std::string and 'const char*' is the lowest common denominator.
Textbooks feature old-school C strings because many basic functions still expect them as arguments, or return them. Additionally, it gives some insight into the underlying structure of the string in memory.
Memory control. I recently had to handle strings (actually blobs from a database) about 200-300 MB in size, in a massively multithreaded application. It was a situation where just-one-more copy of the string might have burst the 32bit address space. I had to know exactly how many copies of the string existed. Although I'm an STL evangelist, I used char * then because it gave me the guarantee that no extra memory or even extra copy was allocated. I knew exactly how much space it would need.
Apart from that, standard STL string processing misses out on some great C functions for string processing/parsing. Thankfully, std::string has the c_str() method for const access to the internal buffer. To use printf() you still have to use char * though (what a crazy idea of the C++ team to not include (s)printf-like functionality, one of the most useful functions EVER in C. I hope boost::format will soon be included in the STL.
If the C++ code is "deep" (close to the kernel, heavily dependent on C libraries, etc.) you may want to use C strings explicitly to avoid lots of conversions in to and out of std::string. Of, if you're interfacing with other language domains (Python, Ruby, etc.) you might do so for the same reason. Otherwise, use std::string.
Some posts mention memory concerns. That might be a good reason to shun std::string, but char* probably is not the best replacement. It's still an OO language. Your own string class is probably better than a char*. It may even be more efficient - you can apply the Small String Optimization, for instance.
In my case, I was trying to get about 1GB worth of strings out of a 2GB file, stuff them in records with about 60 fields and then sort them 7 times of different fields. My predecessors code took 25 hours with char*, my code ran in 1 hour.
1) "string constant" is a C string (const char *), converting it to const std::string& is run-time process, not necessarily simple or optimized.
2) fstream library uses c-style strings to pass file names.
My rule of thumb is to pass const std::string& if I am about to use the data as std::string anyway (say, when I store them in a vector), and const char * in other cases.
After spending far, far, too much time debugging initialization rules and every conceivable string implementation on several platforms we require static strings to be const char*.
After spending far, far, too much time debugging bad char* code and memory leaks I suggest that all non-static strings be some type of string object ... until profiling shows that you can and should do something better ;-)
Legacy code that doesn't know of std::string. Also, before C++11 opening files with std::ifstream or std::ofstream was only possible with const char* as an input to the file name.
Given the choice, there is generally no reason to choose primitive C strings (char*) over C++ strings (std::string). However, often you don't have the luxury of choice. For instance, std::fstream's constructors take C strings, for historical reasons. Also, C libraries (you guessed it!) use C strings.
In your own C++ code it is best to use std::string and extract the object's C string as needed by using the c_str() function of std::string.
It depends on the libraries you're using. For example, when working with the MFC, it's often easier to use CString when working with various parts of the Windows API. It also seems to perform better than std::string in Win32 applications.
However, std::string is part of the C++ standard, so if you want better portability, go with std::string.
For applications such as most embedded platforms where you do not have the luxury of a heap to store the strings being manipulated, and where deterministic preallocation of string buffers is required.
c strings don't carry the overhead of being a class.
c strings generally can result in faster code, as they are closer to the machine level
This is not to say, you can't write bad code with them. They can be misused, like every other construct.
There is a wealth of libary calls that demand them for historical reasons.
Learn to use c strings, and stl strings, and use each when it makes sense to do so.
STL strings are certainly far easier to use, and I don't see any reason to not use them.
If you need to interact with a library that only takes C-style strings as arguments, you can always call the c_str() method of the string class.
The usual reason to do it is that you enjoy writing buffer overflows in your string handling. Counted strings are so superior to terminated strings it's hard to see why the C designers ever used terminated strings. It was a bad decision then; it's a bad decision now.