I am receiving an segmentation fault (SIGSEGV) when I try to reinterpret_cast a struct that contains an vector. The following code does not make sense on its own, but shows an minimal working (failing) example.
// compiler: g++ -std=c++17
struct Table
{
std::vector<int> ids;
};
std::vector<std::byte> storage;
// put that table into the storage
Table table = {.ids = {3, 5}};
auto convert = [](Table x){ return reinterpret_cast<std::byte*>(&x); };
std::byte* bytes = convert(table);
storage.insert(storage.end(), bytes, bytes + sizeof(Table));
// ...
// get that table back from the storage
Table& tableau = *reinterpret_cast<Table*>(&storage.front());
assert(tableau.ids[0] == 3);
assert(tableau.ids[1] == 5);
The code works fine if I inline the convert function, so my guess is that some underlying memory is deleted. The convert function makes a local copy of the table and after leaving the function, the destructor for the local copy's ids vector is called. Recasting just
returns the vector, but the ids are already deleted.
So here are my questions:
Why does the segmentation fault happen? (Is my guess correct?)
How could I resolve this issue?
Thanks in advance :D
I see at least three reasons for undefined behavior in the shown code, that fatally undermines what the shown code is attempting to do. One or some combination of the following reasons is responsible for your observed crash.
struct Table
{
std::vector<int> ids;
};
Reason number 1 is that this is not a trivially copyable object, so any attempt to copy it byte by byte, as the shown code attempts to do, results in undefined behavior.
storage.insert(storage.end(), bytes, bytes + sizeof(Table));
Reason number 2 is that sizeof() is a compile time constant. You might be unaware that the sizeof of this Table object is always the same, whether or not its vector is empty or contains the first billion digits of π. The attempt here to copy the whole object into the byte buffer, this way, therefore fails for this fundamental reason.
auto convert = [](Table x){ return reinterpret_cast<std::byte*>(&x); };
Reason number 3 is that this lambda, for all practical purposes, is the same as any other function with respect to its parameters: its x parameter goes out of scope and gets destroyed as soon as this function returns.
When a function receives a parameter, that parameter is just like a local object in the function, and is a copy of whatever the caller passed to it, and like all other local objects in the function it gets destroyed when the function returns. This function ends up returning a pointer to a destroyed object, and subsequent usage of this pointer also becomes undefined behavior.
In summary, what the shown code is attempting to do is, unfortunately, going against multiple core fundamentals of C++, and manifests in a crash for one or some combination of these reasons; C++ simply does not work this way.
The code works fine if I inline the convert function
If, by trial and error, you come up with some combination of compiler options, or cosmetic tweaks, that avoids a crash, for some miraculous reason, it doesn't fix any of the underlying problems and, at some point later down the road you'll get a crash anyway, or the code will fail to work correctly. Guaranteed.
How could I resolve this issue?
The only way for you to resolve this issue is, well, not do any of this. You also indicated that what you're trying to do is just "store multiple vectors of different types in the same container". This happens to be what std::variant can easily handle, safely, so you'll want to look into that.
Related
I'm using C++ 17 and I have a rather large (two dimensional) array of numbers I'm trying to initialize at the namespace scope. This array is intended to be a pre-computed lookup table that's going to be used in many parts of my code. It's definition looks something like this:
using table_type = std::array<std::array<uint64_t, SOME_BIG_NUMBER>, SOME_OTHER_BIG_NUMBER>;
table_type MyTable = ...
Where the total size of the table is > 200,000 or so. Now, all of the values in the table could be known at compile time, so initially I went ahead and did something like this:
Attempt 1
// Header.h
constexpr table_type MyTable = []()
{
table_type table{};
// code to initialize table...
return table;
}();
My initial fears were realized when MSVC refused to compile this code. The table is simply too large to be computed at compile time. I could have fiddled with the settings and increased the maximum allowed steps but I didn't really feel like getting into that.
Attempt 2
// Header.h
const extern table_type MyTable;
// Implementation.cpp
const table_type MyTable = []()
{
table_type table{};
// code to initialize table...
return table;
}();
I had a feeling this also wouldn't work when I was coding it up and I was right. MSVC warns that I'm using over 2MB of stack memory in the lambda function, and though it compiles the executable immediately crashes upon startup due to the stack being blown up.
Attempt 3
// Header.h
// MyTable is no longer const
extern table_type MyTable;
// Implementation.cpp
table_type MyTable{};
int init_table()
{
// code to initialize table...
// we initialize it by directly writing to it, e.g
// table[0][0] = 5;
// table[0][1] = 4;
// etc.
return 0;
}
const auto _Unused = init_table();
This works. MSVS, GCC, and Clang will compile this code without complaint and the resulting executable runs as desired. Nevertheless, I found this method unsatisfying for several reasons. The first obvious issue is that I had to make my table non-const which means that it can potentially be mutated from anywhere else in the code. I could just be careful not to do that, but if someone else ever comes along and uses my code (it's a static library) they might not be so careful. I'd like to avoid giving myself and others unneeded opportunities to shoot ourselves in the foot.
To top it all off, the initialization code is ugly. Rather than being able to use an immediately evaluated lambda I instead have to declare a free function and then call it in another location. Furthermore, if I want this to happen automatically when the program launches I need to have the init_table function return something and then store that result in a variable somewhere. C++ doesn't seem to allow for something like init_table to be called at namespace scope unless the result is being stored somewhere. So now I have an initialization function that returns a useless value and I am required to save that value in a namespace scoped variable. Ugly.
Attempt 4
// Header.h
const extern table_type MyTable;
// Implementation.cpp
const table_type MyTable = []() -> table_type
{
auto table_p = std::make_unique<table_type>();
// code to initialize table_p...
return *table_p;
}();
The idea is to avoid blowing up the stack by allocating a temporary table on the heap. That table is initialized and then copied into MyTable. MSVC, GCC, and Clang accept this code with no warnings and the resulting executables run fine.
My question
The problem is I didn't expect this to work and I can't quite wrap my head around why it works. The initial table is allocated on the heap without issue, but I don't completely understand what's happening when it's returned from the lambda. I added the -> table_type explicit return type to the lambda to make sure that it didn't deduce the return type as table_type& as that would result in a dangling reference. But since it's returning the table by copy wouldn't a temporary r-value need to be made on the call stack when the lambda returns? And that should result in the same crash as attempt #2.
I'm aware of RVO in C++ and that it was enhanced further with C++ 17. But in this case I'm attempting to return an object that's allocated on the heap and managed by a unique_ptr. I don't see how RVO could apply here because once the return statement is hit the destructor of the unique_ptr will free the memory used to store the table so there's no way that MyTable could then be initialized by simply copying the contents of the memory pointed to by table_p.
I understand that if I directly returned a stack allocated object by value the compiler can optimize away the call to the destructor of that object and effectively memcpy its contents into the new value it's being stored in. But in this case the object being returned by value is on the heap. It appears to me that what the compiler is doing is copying the contents of the memory pointed to by table_p into the memory used to store MyTable, and it is doing this before the destructor of table_p is run. Since destructor calls (as far as I'm aware) are considered part of the function body in which they are called, this would mean that MyTable is being fully intialized before the function that produces the value that initializes it actually exits. This sounds very strange to me.
I've been puzzling about this all day and I just can't figure out what's going on here. I also can't seem to find much online that's related to this specific scenario. My fear is that I'm relying on undefined/implementation defined behavior that the three major compilers just happen to work nicely with. Could another conforming compiler come along and produce an executable that blows up the stack here?
MyTable is declared at namespace scope, so it's not stored on the stack. It's stored in the "static area" or whatever you call it.
The object that table_p points to is stored on the heap.
So what happens when the lambda returns? First, the object that is stored on the heap is copied directly to the object that is stored in the static area (without having to go through a large temporary stack object). Then, table_p is destroyed (and the heap object with it).
The C++17 "guaranteed copy elision" feature ensures that when the lambda returns, a temporary object is not created unless it needs to be. The call expression is a prvalue, which means that it does not designate an object, but is a "recipe" that describes how to initialize an object. The compiler determines which object is the target of the prvalue (i.e., the memory location on which the "recipe" will be run in order to create and initialize an object). In this case, the target is MyTable itself, not a temporary. The return statement directly initializes that target object, not a temporary.
In the small sample below:
#include<iostream>
using namespace std;
int z(){
return 5 + 10; // returns 15
}
int main(){
z(); // what happens to this return?
cout << "Did not fail";
return 0;
}
What happens to the 15? I tried running it in debugger but I can't find it anywhere. I assume that because it didn't get assigned to anything it just vanished but I feel like that's wrong.
I asked my TA about this today and he told me it's stored on the call stack but when I viewed it in debugger I see that it is not.
The C++ standard imposes the "as-if" rule. That rule means that a C++ compiler can do anything to a program as long as all side effects (inputs and outputs that are visible to the rest of the system, like writing to a file or showing stuff on the screen) are respected. Going back to my cheeky philosophical comment, this means that in C++, when a tree falls in the forest and no one is there to hear it, it doesn't have to make a sound (but it can).
In the case of your program, at a high level, since your function does nothing, the compiler may or may not create a call to it, or could even remove it from the compiled binary. If it does include and call it, the return value will go to whatever return slot your platform's application binary interface specifies. On almost every x86_64 system, that will be the rax register for an integer return value. The return value is there but will never be read and will be overwritten at some point.
If it was a non-trivial object instead of an int, its destructor would be invoked immediately.
In general: when a function returns a non-void value and the value does not get stored anywhere, the value is destroyed.
Specifically: natural datatypes, like ints and doubles, or pointers, don't have an explicit destructor, so nothing really happens. The returned value simply gets ignored.
If a function returns a class instance, the class instance gets destroyed, which results in an invocation of the class's defined destructor, or a default destructor.
In the small sample below:
#include<iostream>
using namespace std;
int z(){
return 5 + 10; // returns 15
}
int main(){
z(); // what happens to this return?
cout << "Did not fail";
return 0;
}
What happens to the 15? I tried running it in debugger but I can't find it anywhere. I assume that because it didn't get assigned to anything it just vanished but I feel like that's wrong.
I asked my TA about this today and he told me it's stored on the call stack but when I viewed it in debugger I see that it is not.
The C++ standard imposes the "as-if" rule. That rule means that a C++ compiler can do anything to a program as long as all side effects (inputs and outputs that are visible to the rest of the system, like writing to a file or showing stuff on the screen) are respected. Going back to my cheeky philosophical comment, this means that in C++, when a tree falls in the forest and no one is there to hear it, it doesn't have to make a sound (but it can).
In the case of your program, at a high level, since your function does nothing, the compiler may or may not create a call to it, or could even remove it from the compiled binary. If it does include and call it, the return value will go to whatever return slot your platform's application binary interface specifies. On almost every x86_64 system, that will be the rax register for an integer return value. The return value is there but will never be read and will be overwritten at some point.
If it was a non-trivial object instead of an int, its destructor would be invoked immediately.
In general: when a function returns a non-void value and the value does not get stored anywhere, the value is destroyed.
Specifically: natural datatypes, like ints and doubles, or pointers, don't have an explicit destructor, so nothing really happens. The returned value simply gets ignored.
If a function returns a class instance, the class instance gets destroyed, which results in an invocation of the class's defined destructor, or a default destructor.
I have encountered this problem and am wondering what its cause is.
My code is as follows:
struct node{
bool leaf;
std::string label;
};
//in main
std::vector<node> graph;
graph.reserve(5);
Now, if i try to assign
graph[3].leaf = true;,
everything works out.
However, if i try to do the same with an object type, like
graphA[i].label = "01";,
I get a segmentation fault.
If i change the code to
struct node{
bool leaf;
std::string * label;
};
And allocate the memory for the string in every instance of node in the vector, I can now assign a value to graphA[i]->label without a problem.
Why is this? Any responses will be appreciated.
Now, if i try to assign graph[3].leaf = true;
You're invoking undefined behavior.
graph has no element at index 3. In fact, it has no elements at all. You only reserved memory for the vector, but didn't add any elements to it.
You can add 5 default-constructed elements using resize:
graph.resize(5);
Now, if i try to assign graph[3].leaf = true;, everything works out.
It's important to stress that this is a mere coincidence. Nothing works out - you have undefined behaviour because you access graph[3] when graph is empty. It would be much better if the program crashed right away, so that you notice something is wrong.
graph is empty because you confused std::vector's reserve and resize member functions. You don't set the vector's element count to 5, you just ask it to prepare its internally held memory for at least 5 elements in the future. std::vector is even allowed to ignore this request.
When you do graphA[i].label = "01";, you also have undefined behaviour, of course.
So why do the first version and the pointer "fix" (which also invokes undefined behaviour) seem to work fine while the other one crashes? C++ as a programming language does not distinguish between different kinds of undefined behaviour. Anything can happen; you may as well experience situations in which the first crashes and the second one "works out". Such is the nature of undefined behaviour.
What probably happens here in practice is that in the bool and std::string* case, you are coincidentally writing to a memory location your program is allowed to write to, because you are just dealing with a single bool or with a single pointer. In the std::string version, with all of std::string's automatic dynamic memory management happening behind the scenes, more places in memory are involved and so it happens that you hit a forbidden one.
But that's just a very speculative theory. If you really want to know for sure, debug the code and step into each individual operation. But remember that undefined behaviour is always undefined behaviour.
I tried a few Google searches before making this post, but to be honest I don't know what to search for. I have a C++ project and have been happily going about using the GNU compilers (g++). Today I tried to compile with clang++ and got a segfault.
Fine, ok, I can deal with this. After perusing my code and printing some stuff I was able to fix the problem. However the solution deeply troubles and confuses me.
Here's the situation: I'm using a tree-like data structure that stores a class called Ligament, but I'm storing it in a std::vector. I do this by storing a vector of "children" which are really just integer offsets between parent and child within the vector. In this way I can access children by using the this pointer, i.e
child = this[offset];
However, none of that's important. Here's this issue: I have an Ligament::addChild(int) function that takes an integer and pushes it to the back of a vector that is a member of Ligament:
void Ligament::addChild(uint32_t offset){
children.push_back(offset);
}
Very simple stuff. In general I pass to addChild an argument that gets returned from a recursive function called fill:
//starting at root
uint32_t fill(vector<Ligament>& lVec, TiXmlElement * el){
//store current size here, as size changes during recursion
uint32_t curIdx = lVec.size();
lVec.push_back(createLigament());
//Add all of this Ligament's children
TiXmlElement * i = el->FirstChildElement("drawable");
for (; i; i=i->NextSiblingElement("drawable")){
uint32_t tmp = fill(lVec, i) - curIdx;
lVec[curIdx].addChild(tmp);
//Does not work in clang++, but does in g++
//lVec[curIdx].addChild(fill(lVec,i)-curIdx);
}
//return the ligament's index
return curIdx;
}
The fill function gets called on an XML element and goes through its children, depth first.
Sorry if all that was unclear, but the core of the problem seems to be what's in that for loop. For some reason I have to store the return value of the fill call in a variable before I send it to the addChild function.
If I don't store it in a temporary variable, it seems as though the addChild function does not change the size of children, but I can't imagine why.
To check all this I printed out the size of the children vector before and after these calls, and it never went above 1. Only when I called addChild with a value that wasn't directly returned from a function did it seems to work.
I also printed out the values of offset inside the addChild function as well as inside the for loop before it was called. In all cases the values were the same, both in clang++ and in g++.
Since the issue is resolved I was able to move forward, but this is something I'd expect to work. Is there something I'm doing wrong?
Feel free to yell at me if I could do more to make this question clearer.
ALSO: I realize now that passing lVec around by reference through these recursions may be bad, as a push_back call may cause the address to change. Is this a legitimate concern?
EDIT:
So as people have pointed out, my final concern turned out to be related to the issue. The fill call has the potential to resize the vector, while the lVec[curIdx] = modifier will change an element in the vector. The order in which these things occurs can have drastic consequences.
As a follow up, is using the tmp variable acceptable? There's still the issue of a reallocation occuring...I think I will use SHR's suggestion of a map, then convert it to a vector when all is said and done.
// Does not work in clang++, but does in g++:
lVec[curIdx].addChild(fill(lVec,i)-curIdx);
The bug you are seeing is due to dependence on order of evaluation. Since fill(lVec, i) may cause lVec to reallocate its elements, the program will have undefined behavior if lVec[curIdx] is evaluated before fill(lVec,i). The order of evaluation of function arguments - and the postfix expression that determines which function to call - is unspecified.
I think it is undefined behavior.
you push into vector, and change it in the same command.
one compiler may do the fill first and the other may get lVec[curIdx] first.
if it is the case it will work for both compilers when you use map<uint32_t,uint32_t> instead of the vector. since map doesn't require the memory to be sequential.