My question is in relation to this little code snippet:
typedef std::map<std::string, std::string> my_map_t;
std::string_view get_value_worse(const my_map_t& input_map, std::string_view value)
{
auto retrieved = input_map.find(value.data());
return retrieved != input_map.cend() ? retrieved->second : "";
}
std::string_view get_value_better(const my_map_t& input_map, std::string_view value)
{
auto retrieved = input_map.find(value.data());
if (retrieved != input_map.cend())
{
return retrieved->second;
}
return "";
}
int main()
{
my_map_t my_map = {
{"key_0", "value_0"},
{"key_1", "value_1"},
};
std::cout << (get_value_worse(my_map, "key_0") == get_value_better(my_map, "key_0")) << std::endl;
}
Under the latest gcc with no optimisations this prints 0 for false, while under -O3 this prints 1 for true.
I believe the un-optimised behaviour is because the second and third comparison operator arguments are expressions, not statements - and so the retrieved->second in retrieved != arguments.cend() ? retrieved->second : "" gets evaluated as a string construction on the stack, and returning a string_view to that is bad.
I can also see that with -O3 the compiler would be able to inline all of this, remove branching, and be done with it... but I would have expected -O3 to act exactly "as if" I had compiled with -O0.
Can anyone explain why the compiler gets to elide the copy construction I believe is happening in the -O0 version?
In the conditional expression, a temporary std::string object is constructed. Temporary object are usually constructed on the stack, although this is an implementation detail that is not important. The important thing is that the temporary object is destroyed at the end of the return statement, so the returned std::string_view is dangling. Attempting to access the data it points to (using the == operator, or otherwise) results in undefined behaviour.
When a program contains undefined behaviour, the compiler can do whatever it wants with it. In particular, the compiler is permitted to optimize by assuming that a condition that implies undefined behaviour will always be false. If it turns out that this assumption is wrong, then the compiler is off the hook (because it means undefined behaviour is occurring). What kind of assumption exactly is being made by your compiler is not clear. It also doesn't really matter, because you can't depend on the behaviour that you see now. You should just rewrite your program to remove the undefined behaviour.
Related
The C++ standard states that returning reference to a local variable (on the stack) is undefined behaviour, so why do many (if not all) of the current compilers only give a warning for doing so?
struct A{
};
A& foo()
{
A a;
return a; //gcc and VS2008 both give this a warning, but not a compiler error
}
Would it not be better if compilers give a error instead of warning for this code?
Are there any great advantages to allowing this code to compile with just a warning?
Please note that this is not about a const reference which could lengthen the lifetime of the temporary to the lifetime of the reference itself.
It is almost impossible to verify from a compiler point of view whether you are returning a reference to a temporary. If the standard dictated that to be diagnosed as an error, writing a compiler would be almost impossible. Consider:
bool not_so_random() { return true; }
int& foo( int x ) {
static int s = 10;
int *p = &s;
if ( !not_so_random() ) {
p = &x;
}
return *p;
}
The above program is correct and safe to run, in our current implementation it is guaranteed that foo will return a reference to a static variable, which is safe. But from a compiler perspective (and with separate compilation in place, where the implementation of not_so_random() is not accessible, the compiler cannot know that the program is well-formed.
This is a toy example, but you can imagine similar code, with different return paths, where p might refer to different long-lived objects in all paths that return *p.
Undefined behaviour is not a compilation error, it's just not a well-formed C++ program. Not every ill-formed program is incompilable, it's just un-predictable. I'd wager a bet that it's not even possible in principle for a computer to decide whether a given program text is a well-formed C++ program.
You can always add -Werror to gcc to make warnings terminate compilation with an error!
To add another favourite SO topic: Would you like ++i++ to cause a compile error, too?
If you return a pointer/reference to a local inside function the behavior is well defined as long as you do not dereference the pointer/reference returned from the function.
It is an Undefined Behavior only when one derefers the returned pointer.
Whether it is a Undefined Behavior or not depends on the code calling the function and not the function itself.
So just while compiling the function, the compiler cannot determine if the behavior is Undefined or Well Defined. The best it can do is to warn you of a potential problem and it does!
An Code Sample:
#include <iostream>
struct A
{
int m_i;
A():m_i(10)
{
}
};
A& foo()
{
A a;
a.m_i = 20;
return a;
}
int main()
{
foo(); //This is not an Undefined Behavior, return value was never used.
A ref = foo(); //Still not an Undefined Behavior, return value not yet used.
std::cout<<ref.m_i; //Undefined Behavior, returned value is used.
return 0;
}
Reference to the C++ Standard:
section 3.8
Before the lifetime of an object has started but after the storage which the object will occupy has been allo-cated 34) or, after the lifetime of an object has ended and before the storage which the object occupied is reused or released, any pointer that refers to the storage location where the object will be or was located may be used but only in limited ways. Such a pointer refers to allocated storage (3.7.3.2), and using the
pointer as if the pointer were of type void*, is well-defined. Such a pointer may be dereferenced but the resulting lvalue may only be used in limited ways, as described below. If the object will be or was of a class type with a non-trivial destructor, and the pointer is used as the operand of a delete-expression, the program has undefined behavior. If the object will be or was of a non-POD class type, the program has undefined behavior if:
— .......
Because standard does not restrict us.
If you want to shoot to your own foot you can do it!
However lets see and example where it can be useful:
int &foo()
{
int y;
}
bool stack_grows_forward()
{
int &p=foo();
int my_p;
return &my_p < &p;
}
Compilers should not refuse to compile programs unless the standard says they are allowed to do so. Otherwise it would be much harder to port programs, since they might not compile with a different compiler, even though they comply with the standard.
Consider the following function:
int foobar() {
int a=1,b=0;
return a/b;
}
Any decent compiler will detect that I am dividing by zero, but it should not reject the code since I might actually want to trigger a SIG_FPE signal.
As David Rodríguez has pointed out, there are some cases which are undecidable but there are also some which are not. Some new version of the standard might describe some cases where the compiler must/is allowed to reject programs. That would require the standard to be very specific about the static analysis which is to be performed.
The Java standard actually specifies some rules for checking that non-void methods always return a value. Unfortunately I haven't read enough of the C++ standard to know what the compiler is allowed to do.
You could also return a reference to a static variable, which would be valid code so the code must be able to compile.
It's pretty much super-bad practice to rely on this, but I do believe that in many cases (and that's never a good wager), that memory reference would still be valid if no functions are called between the time foo() returns and the time the calling function uses its return value. In that case, that area of the stack would not have an opportunity to get overwritten.
In C and C++ you can choose to access arbitrary sections of memory anyway (within the process's memory space, of course) via pointer arithmetic, so why not allow the possibility of constructing a reference to wherever one so chooses?
We are in a hot discussion with my friends about the code:
#include <iostream>
#include <string>
using namespace std;
string getString() {
return string("Hello, world!");
}
int main() {
char const * str = getString().c_str();
std::cout << str << "\n";
return 0;
}
This code produces different outputs on g++, clang and vc++:
g++ and clang output is the same:
Hello, world!
However vc++ outputs nothing (or just spaces):
What behavior is correct? Is this may be a change in standard according to temporaries lifetime ?
As far as I can see by reading IR of clang++, it works as following:
store `getString()`'s return value in %1
std::cout << %1.c_str() << "\n";
destruct %1
Personally, I think gcc works this way too (I've tested it with rvo/move verbosity (custom ctors and dtors which prints to std::cout). Why does vc++ works other way?
clang = Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
g++ = gcc version 4.9.2 (Debian 4.9.2-10)
Your program has undefined behaviour! You are "printing" a dangling pointer.
The result of getString(), a temporary string, lives no longer than that const char* declaration; accordingly neither does the result of invoking c_str() on that temporary.
So both compilers are "correct"; it is you and your friends who are wrong.
This is why we shall not store the result of std::string::c_str(), unless we really, really need to.
Both are right, undefined behaviour is undefined.
char const * str = getString().c_str();
getString() returns a temporary, which will be destroyed at the end of the full expression which contains it. So after that line is finished, str is an invalid pointer and trying to inspect it will plunge you into the land of undefined behaviour.
Some standards quotes, as requested (from N4140):
[class.temporary]/3: Temporary objects are destroyed as the last step in evaluating the full-expression that (lexically) contains the point where they were created.
basic_string::c_str is specified like so:
[string.accessors]/1: A pointer p such that p + i == &operator[](i) for each i in [0,size()].
Since strings have their contents stored contiguously ([string.require]/4) this essentially means "return a pointer to the start of the buffer".
Obviously when a std::string is destructed it will reclaim any memory which was allocated, making that pointer invalid (if your friends don't believe that, they have other problems).
That is undefined behavior so anything can happen (including printing the string "correctly").
Making things "working" anyway happens quite often with UB, unless the program is actually running on a paying customer's computer or if it's shown on the big screen in front of a vast audience ;-)
The problem is that you're taking a const char * pointing inside a temporary object that is destroyed before your use of the pointer.
Note that this is not the same situation as with:
const std::string& str = getString(); // Returns a temporary
std::cout << str << "\n";
because in this case instead there is a very specific rule about references bound to temporaries in the C++ standard. In this case the lifetime of the temporary will be extended until the reference str is also destroyed. The rule only applies to references and only if directly bound to the temporary or to a sub-object of the temporary (like const std::string& s = getObj().s;) and not to the result of calling methods of a temporary object.
The C++ standard states that returning reference to a local variable (on the stack) is undefined behaviour, so why do many (if not all) of the current compilers only give a warning for doing so?
struct A{
};
A& foo()
{
A a;
return a; //gcc and VS2008 both give this a warning, but not a compiler error
}
Would it not be better if compilers give a error instead of warning for this code?
Are there any great advantages to allowing this code to compile with just a warning?
Please note that this is not about a const reference which could lengthen the lifetime of the temporary to the lifetime of the reference itself.
It is almost impossible to verify from a compiler point of view whether you are returning a reference to a temporary. If the standard dictated that to be diagnosed as an error, writing a compiler would be almost impossible. Consider:
bool not_so_random() { return true; }
int& foo( int x ) {
static int s = 10;
int *p = &s;
if ( !not_so_random() ) {
p = &x;
}
return *p;
}
The above program is correct and safe to run, in our current implementation it is guaranteed that foo will return a reference to a static variable, which is safe. But from a compiler perspective (and with separate compilation in place, where the implementation of not_so_random() is not accessible, the compiler cannot know that the program is well-formed.
This is a toy example, but you can imagine similar code, with different return paths, where p might refer to different long-lived objects in all paths that return *p.
Undefined behaviour is not a compilation error, it's just not a well-formed C++ program. Not every ill-formed program is incompilable, it's just un-predictable. I'd wager a bet that it's not even possible in principle for a computer to decide whether a given program text is a well-formed C++ program.
You can always add -Werror to gcc to make warnings terminate compilation with an error!
To add another favourite SO topic: Would you like ++i++ to cause a compile error, too?
If you return a pointer/reference to a local inside function the behavior is well defined as long as you do not dereference the pointer/reference returned from the function.
It is an Undefined Behavior only when one derefers the returned pointer.
Whether it is a Undefined Behavior or not depends on the code calling the function and not the function itself.
So just while compiling the function, the compiler cannot determine if the behavior is Undefined or Well Defined. The best it can do is to warn you of a potential problem and it does!
An Code Sample:
#include <iostream>
struct A
{
int m_i;
A():m_i(10)
{
}
};
A& foo()
{
A a;
a.m_i = 20;
return a;
}
int main()
{
foo(); //This is not an Undefined Behavior, return value was never used.
A ref = foo(); //Still not an Undefined Behavior, return value not yet used.
std::cout<<ref.m_i; //Undefined Behavior, returned value is used.
return 0;
}
Reference to the C++ Standard:
section 3.8
Before the lifetime of an object has started but after the storage which the object will occupy has been allo-cated 34) or, after the lifetime of an object has ended and before the storage which the object occupied is reused or released, any pointer that refers to the storage location where the object will be or was located may be used but only in limited ways. Such a pointer refers to allocated storage (3.7.3.2), and using the
pointer as if the pointer were of type void*, is well-defined. Such a pointer may be dereferenced but the resulting lvalue may only be used in limited ways, as described below. If the object will be or was of a class type with a non-trivial destructor, and the pointer is used as the operand of a delete-expression, the program has undefined behavior. If the object will be or was of a non-POD class type, the program has undefined behavior if:
— .......
Because standard does not restrict us.
If you want to shoot to your own foot you can do it!
However lets see and example where it can be useful:
int &foo()
{
int y;
}
bool stack_grows_forward()
{
int &p=foo();
int my_p;
return &my_p < &p;
}
Compilers should not refuse to compile programs unless the standard says they are allowed to do so. Otherwise it would be much harder to port programs, since they might not compile with a different compiler, even though they comply with the standard.
Consider the following function:
int foobar() {
int a=1,b=0;
return a/b;
}
Any decent compiler will detect that I am dividing by zero, but it should not reject the code since I might actually want to trigger a SIG_FPE signal.
As David Rodríguez has pointed out, there are some cases which are undecidable but there are also some which are not. Some new version of the standard might describe some cases where the compiler must/is allowed to reject programs. That would require the standard to be very specific about the static analysis which is to be performed.
The Java standard actually specifies some rules for checking that non-void methods always return a value. Unfortunately I haven't read enough of the C++ standard to know what the compiler is allowed to do.
You could also return a reference to a static variable, which would be valid code so the code must be able to compile.
It's pretty much super-bad practice to rely on this, but I do believe that in many cases (and that's never a good wager), that memory reference would still be valid if no functions are called between the time foo() returns and the time the calling function uses its return value. In that case, that area of the stack would not have an opportunity to get overwritten.
In C and C++ you can choose to access arbitrary sections of memory anyway (within the process's memory space, of course) via pointer arithmetic, so why not allow the possibility of constructing a reference to wherever one so chooses?
Is this piece of code valid (and defined behavior)?
int &nullReference = *(int*)0;
Both g++ and clang++ compile it without any warning, even when using -Wall, -Wextra, -std=c++98, -pedantic, -Weffc++...
Of course the reference is not actually null, since it cannot be accessed (it would mean dereferencing a null pointer), but we could check whether it's null or not by checking its address:
if( & nullReference == 0 ) // null reference
References are not pointers.
8.3.2/1:
A reference shall be initialized to
refer to a valid object or function.
[Note: in particular, a null reference
cannot exist in a well-defined
program, because the only way to
create such a reference would be to
bind it to the “object” obtained by
dereferencing a null pointer, which
causes undefined behavior. As
described in 9.6, a reference cannot
be bound directly to a bit-field. ]
1.9/4:
Certain other operations are described
in this International Standard as
undefined (for example, the effect of
dereferencing the null pointer)
As Johannes says in a deleted answer, there's some doubt whether "dereferencing a null pointer" should be categorically stated to be undefined behavior. But this isn't one of the cases that raise doubts, since a null pointer certainly does not point to a "valid object or function", and there is no desire within the standards committee to introduce null references.
The answer depends on your view point:
If you judge by the C++ standard, you cannot get a null reference because you get undefined behavior first. After that first incidence of undefined behavior, the standard allows anything to happen. So, if you write *(int*)0, you already have undefined behavior as you are, from a language standard point of view, dereferencing a null pointer. The rest of the program is irrelevant, once this expression is executed, you are out of the game.
However, in practice, null references can easily be created from null pointers, and you won't notice until you actually try to access the value behind the null reference. Your example may be a bit too simple, as any good optimizing compiler will see the undefined behavior, and simply optimize away anything that depends on it (the null reference won't even be created, it will be optimized away).
Yet, that optimizing away depends on the compiler to prove the undefined behavior, which may not be possible to do. Consider this simple function inside a file converter.cpp:
int& toReference(int* pointer) {
return *pointer;
}
When the compiler sees this function, it does not know whether the pointer is a null pointer or not. So it just generates code that turns any pointer into the corresponding reference. (Btw: This is a noop since pointers and references are the exact same beast in assembler.) Now, if you have another file user.cpp with the code
#include "converter.h"
void foo() {
int& nullRef = toReference(nullptr);
cout << nullRef; //crash happens here
}
the compiler does not know that toReference() will dereference the passed pointer, and assume that it returns a valid reference, which will happen to be a null reference in practice. The call succeeds, but when you try to use the reference, the program crashes. Hopefully. The standard allows for anything to happen, including the appearance of pink elephants.
You may ask why this is relevant, after all, the undefined behavior was already triggered inside toReference(). The answer is debugging: Null references may propagate and proliferate just as null pointers do. If you are not aware that null references can exist, and learn to avoid creating them, you may spend quite some time trying to figure out why your member function seems to crash when it's just trying to read a plain old int member (answer: the instance in the call of the member was a null reference, so this is a null pointer, and your member is computed to be located as address 8).
So how about checking for null references? You gave the line
if( & nullReference == 0 ) // null reference
in your question. Well, that won't work: According to the standard, you have undefined behavior if you dereference a null pointer, and you cannot create a null reference without dereferencing a null pointer, so null references exist only inside the realm of undefined behavior. Since your compiler may assume that you are not triggering undefined behavior, it can assume that there is no such thing as a null reference (even though it will readily emit code that generates null references!). As such, it sees the if() condition, concludes that it cannot be true, and just throw away the entire if() statement. With the introduction of link time optimizations, it has become plain impossible to check for null references in a robust way.
TL;DR:
Null references are somewhat of a ghastly existence:
Their existence seems impossible (= by the standard),
but they exist (= by the generated machine code),
but you cannot see them if they exist (= your attempts will be optimized away),
but they may kill you unaware anyway (= your program crashes at weird points, or worse).
Your only hope is that they don't exist (= write your program to not create them).
I do hope that will not come to haunt you!
clang++ 3.5 even warns on it:
/tmp/a.C:3:7: warning: reference cannot be bound to dereferenced null pointer in well-defined C++ code; comparison may be assumed to
always evaluate to false [-Wtautological-undefined-compare]
if( & nullReference == 0 ) // null reference
^~~~~~~~~~~~~ ~
1 warning generated.
If your intention was to find a way to represent null in an enumeration of singleton objects, then it's a bad idea to (de)reference null (it C++11, nullptr).
Why not declare static singleton object that represents NULL within the class as follows and add a cast-to-pointer operator that returns nullptr ?
Edit: Corrected several mistypes and added if-statement in main() to test for the cast-to-pointer operator actually working (which I forgot to.. my bad) - March 10 2015 -
// Error.h
class Error {
public:
static Error& NOT_FOUND;
static Error& UNKNOWN;
static Error& NONE; // singleton object that represents null
public:
static vector<shared_ptr<Error>> _instances;
static Error& NewInstance(const string& name, bool isNull = false);
private:
bool _isNull;
Error(const string& name, bool isNull = false) : _name(name), _isNull(isNull) {};
Error() {};
Error(const Error& src) {};
Error& operator=(const Error& src) {};
public:
operator Error*() { return _isNull ? nullptr : this; }
};
// Error.cpp
vector<shared_ptr<Error>> Error::_instances;
Error& Error::NewInstance(const string& name, bool isNull = false)
{
shared_ptr<Error> pNewInst(new Error(name, isNull)).
Error::_instances.push_back(pNewInst);
return *pNewInst.get();
}
Error& Error::NOT_FOUND = Error::NewInstance("NOT_FOUND");
//Error& Error::NOT_FOUND = Error::NewInstance("UNKNOWN"); Edit: fixed
//Error& Error::NOT_FOUND = Error::NewInstance("NONE", true); Edit: fixed
Error& Error::UNKNOWN = Error::NewInstance("UNKNOWN");
Error& Error::NONE = Error::NewInstance("NONE");
// Main.cpp
#include "Error.h"
Error& getError() {
return Error::UNKNOWN;
}
// Edit: To see the overload of "Error*()" in Error.h actually working
Error& getErrorNone() {
return Error::NONE;
}
int main(void) {
if(getError() != Error::NONE) {
return EXIT_FAILURE;
}
// Edit: To see the overload of "Error*()" in Error.h actually working
if(getErrorNone() != nullptr) {
return EXIT_FAILURE;
}
}
I've just been bitten by a nasty undefined behavior due the returning a reference to a local variable.
We know it's evil, and generally the compiler prints a nice warning to tell us so... well gcc (3.4.2) does not seem to push the checks too far though.
std::string get_env_value(std::string const& key);
std::string const& get_phase()
{
std::string const& phase = get_env_value("PHASE"); // [1]
std::cout << "get_phase - " << phase << '\n';
return phase; // [2]
}
This compiles without a glitch, and yet we fall in the nasty realm of undefined behavior.
Line [1] is okay because the standard specifies that the lifetime of a variable bound to a const reference should be extended to match the lifetime of the const reference.
Line [2] seems okay too...
Do the C++ specifications cover this case ?
Does anyone know if this is usually diagnosed ? (I may miss a flag or something...)
It seems to me that static analysis should be able to tell that having use a "lifetime extension" for [1], [2] is not safe, but it could get ugly rapidly I guess...
The standard does not cover [2]. It allows an rvalue to be bound to a const reference, but that doesn't allow you to return a const reference and have the lifetime of the rvalue it is bound to extended as well.
And true, static analysis could catch this, but as always it's a trade-off. C++ compilation is slow enough as it is, so compiler writers have to weigh the benefits of further static analysis which might allow them to produce better diagnostics, against the increased compilation time.
No, I don't think Standard mentions / covers this specific case.
VS 2010 gives compilation warnings (/Za, /W4).
So, definitely it looks to be a diagnosable condition.
So, I tweaked the function slightly as follows, just to create multiple return paths:
std::string const& get_phase()
{
std::string const& phase = get_env_value("PHASE"); // [1]
std::cout << "get_phase - " << phase << '\n';
if(1){
while(1){
return phase;
}
}
return phase; // [2]
}
Now, VS does not report the warning as earlier.
As an example, at a first glance, it looks that it should be easy for the compiler to detect and catch that not all paths return a value. But compilers (e.g. VS) do not.
int get_phase()
{
char ch;
if(ch){
return 0;
}
// nothing returned from here.
}
So, I guess the code in OP has probably the same complexities to diagnose the condition as the example shown just above, though I am not sure. The only good thing is that the Standard is clear on this case.
$6.6.3/2 - "Flowing off the
end of a function is equivalent to a
return with no value; this results in
undefined behavior in a
value-returning function."
Coming back to the code in OP, I guess the standard does not mandate this condition to be a diagnosable condition, and hence compilers are free to do as they please. It is basically understood that the returned reference points to an object which is already destructed. So accessing such an object will lead to an undefined behavior