I'm getting a segfault on MacOSX ("Segmentation fault: 11", in gdb "Program received signal SIGSEGV, Segmentation fault"), appearing in the destructor in which a container is looped over with an iterator and memory deleted. I've tried with clang++, g++ (both part of LLVM) and homebrew g++. The segfault appears when the iterator is incremented for the first time, with the gdb message (having compiled with clang++)
"0x000000010001196d in std::__1::__tree_node_base<void*>* std::__1::__tree_next<std::__1::__tree_node_base<void*>*>(std::__1::__tree_node_base<void*>*) ()"
When starting the program in gdb I also get warnings saying "warning: Could not open OSO archive file".
On a cluster linux node, with gcc 4.8.1, I don't get a segfault. Any ideas what might be wrong and how I can avoid the segfault on my mac (preferably with clang)? I really don't know much about compilers and such.
EDIT:
I think I found the problem, however I'd like to understand still why this works on one platform but not another. Here's a minimal example:
class Word:
#ifndef WORD_H
#define WORD_H
#include <string>
#include <map>
class Word {
public:
/*** Constructor ***/
Word(std::string w) : m_word(w) {
// Add word to index map, if it's not already in there
std::map<std::string, Word*>::iterator it = index.find(w);
if (it == index.end()) {
index[w] = this;
}
}
~Word() { index.erase(m_word); } // Remove from index
static void DeleteAll() { // Clear index, delete all allocated memory
for (std::map<std::string, Word*>::const_iterator it = index.begin();
it != index.end();
++it)
{ delete it->second; }
}
private:
std::string m_word;
static std::map<std::string, Word*> index; // Index holding all words initialized
};
#endif
WordHandler class:
#ifndef _WORDHANDLER_H_
#define _WORDHANDLER_H_
#include <string>
#include "Word.h"
class WordHandler {
WordHandler() {}
~WordHandler() { Word::DeleteAll(); } // clear memory
void WordHandler::NewWord(const std::string word) {
Word* w = new Word(word);
}
};
#endif
Main program:
#include <iostream>
#include "WordHandler.h"
int main () {
std::cout << "Welcome to the WordHandler. " << std::endl;
WordHandler wh;
wh.NewWord("hallon");
wh.NewWord("karl");
std::cout << "About to exit WordHandler after having added two new words " << std::endl;
return 0;
}
So the segfault occurs upon exiting of the program, when the destructor ~WordHandler is called. The reason I found, is the Word destructor: the Word object is erased from the map, which makes the DeleteAll() function weird because the map is altered while it's being iterated over (some sort of double delete I suppose). The segfault disappears either by removing the DeleteAll completely, or removing the Word destructor.
So I'm still wondering why the segfault doesn't appear on linux with g++ from gcc 4.8.1. (Also, I guess off topic, I'm wondering about the programming itself – what would be the proper way to treat index erasing/memory deletion in this code?)
EDIT 2:
I don't think this is a duplicate of Vector.erase(Iterator) causes bad memory access, because my original question had to do with why I get a segfault on one platform and not another. It's possible that the other question explains the segfault per se (not sure how get around this problem... perhaps removing the Word destructor and calling erase from DeleteAll() rather than "delete"? But that destructor makes sense to me though...), but if it's truly a bug in the code why isn't picked up by gcc g++?
This is a problem:
~Word() { index.erase(m_word); } // Remove from index
static void DeleteAll() { // Clear index, delete all allocated memory
for (std::map<std::string, Word*>::const_iterator it = index.begin();
it != index.end();
++it)
{ delete it->second; }
}
delete it->second invokes ~Word which erases from the map that you are iterating over. This invalidates your active iterator, leading to undefined behaviour. Because it is UB, the fact that it works on one platform but not another is basically just luck (or lack thereof).
To fix this, you can either make a copy of index and iterate over that, consider a different design that doesn't mutate the index as you delete it, or use the fact that erase returns the next valid iterator to make the loop safe (which means hoisting the erase into DeleteAll).
Related
Here I have a class definition. It is a little long, but the focus will be on the move constructor and the destructor. Below the class definition is a short test.
#include <cassert>
#include <iostream>
#include <utility>
template <typename T>
class SharedPtr {
public:
SharedPtr() {}
explicit SharedPtr(T* input_pointer) : raw_ptr_(input_pointer), ref_count_(new size_t(1)) {}
SharedPtr(const SharedPtr& other) : raw_ptr_(other.raw_ptr_), ref_count_(other.ref_count_) {
if (ref_count_) {
++*ref_count_;
}
}
SharedPtr(SharedPtr&& other) : raw_ptr_(other.raw_ptr_), ref_count_(other.ref_count_) {}
SharedPtr& operator=(SharedPtr other) {
swap(other, *this);
return *this;
}
size_t use_count() const {
return ref_count_ ? *ref_count_ : 0;
}
~SharedPtr() {
if (ref_count_) {
--*ref_count_;
if (*ref_count_ == 0) {
delete raw_ptr_;
delete ref_count_;
}
}
}
private:
T* raw_ptr_ = nullptr;
size_t* ref_count_ = nullptr;
friend void swap(SharedPtr<T>& left, SharedPtr<T>& right) {
std::swap(left.raw_ptr_, right.raw_ptr_);
std::swap(left.ref_count_, right.ref_count_);
}
};
int main() {
// Pointer constructor
{
SharedPtr<int> p(new int(5));
SharedPtr<int> p_move(std::move(p));
assert(p_move.use_count() == 1);
}
std::cout << "All tests passed." << std::endl;
return 0;
}
If I run the code I get an error message indicating memory corruption:
*** Error in `./a.out': corrupted size vs. prev_size: 0x0000000001e3dc0f ***
======= Backtrace: =========
...
======= Memory map: ========
...
Aborted (core dumped)
We may suspect something is wrong with the move constructor: if we move from a SharedPtr and then later destruct that SharedPtr, it will still destruct as if it were an "active" SharedPtr. So we could fix that by setting the other object's pointers to nullptr in the move constructor.
But that's not the interesting thing about this code. The interesting thing is what happens if I don't do that, and instead simply add std::cout << "x" << std::endl; to the move constructor.
The new move constructor is given below, and the rest of the code is unchanged.
SharedPtr(SharedPtr&& other) : raw_ptr_(other.raw_ptr_), ref_count_(other.ref_count_) {
std::cout << "x" << std::endl;
}
The code now runs without error on my machine and yields the output:
x
All tests passed.
So my questions are:
Do you get the same results as I do?
Why does adding a seemingly innocuous std::cout line cause the program to run "successfully"?
Please note: I am not under any sort of impression that error message gone implies bug gone.
bolov's answer explains the cause of the undefined behavior (UB), when the move constructor of SharedPtr does not invalidate the moved-from pointer.
I disagree with bolov's view that it is pointless to understand UB. The question why code changes result in different behavior, when facing UB, is extremely interesting. Knowing what happens can help debugging, on one hand, and it can help intruders intrude the system, on the other.
The difference in the code in question comes from adding std::cout << something. In fact, the following change also makes the crash go away:
{
SharedPtr<int> p(new int(5));
SharedPtr<int> p_move(std::move(p));
assert(p_move.use_count() == 1);
std::cout << "hi\n"; // <-- added
}
The std::cout << allocates some internal buffer, which std::cout << uses. The allocation in cout happens only once, and the question is if this allocation happens before or after the double free. Without the additional std::cout, this allocation happens after the double free, when the heap is corrupted. When the heap is corrupted, the allocation in std::cout << triggers the crash. But when there is a std::cout << before the double-free, there is no allocation after the double-free.
Let's have few other experiments to validate this hypothesis:
Remove all std::cout << lines. All works fine.
Move two calls to new int(some number) right before the end:
int main() {
int *p2 = nullptr;
int *cnt = nullptr;
// Pointer constructor
{
SharedPtr<int> p(new int(5));
SharedPtr<int> p_move(std::move(p));
assert(p_move.use_count() == 1);
}
p2 = new int(100);
cnt = new int(1); // <--- crash
return 0;
}
This crashes, since the new is attempted on a corrupted heap.
(you can try it out here)
Now move the two new lines to slightly up, right before the closing } of the inner block. In this case, the new is performed before the heap is corrupted, so nothing triggers a crash. The delete simply puts the data in the free list, which is not corrupted. As long as the corrupted heap is not touched, then things will work fine. One can call new int, and get a pointer of one of the lately released pointers, and nothing bad will happen.
{
SharedPtr<int> p(new int(5));
SharedPtr<int> p_move(std::move(p));
assert(p_move.use_count() == 1);
p2 = new int(100);
cnt = new int(1);
}
delete p2;
delete cnt;
p2 = new int(100); // No crash. We are reusing one of the released blocks
cnt = new int(1);
(you can try it out here)
The interesting fact is that the corrupted heap can be undetected to much later in the code. The computer may run millions of unrelated lines of code, and suddenly crash on a completely unrelated new in a completely different part of the code. This is why sanitizers and the likes of valgrind are needed: debugging memory corruption can be practically impossible to debug otherwise.
Now, the really interesting question is "can this be exploited more than for denial of service?". Yes it can. It depends on the kind of object that is destroyed twice, and what it does in the destructor. It also depends on what happens between the first destruction of the pointer, and its second free. In this trivial example, nothing substantial seems to be possible.
SharedPtr(SharedPtr&& other) : raw_ptr_(other.raw_ptr_), ref_count_(other.ref_count_) {}
When you move the moved from object remains the same. This means that at some point in your program you will delete raw_ptr_ twice for the same memory. The same for ref_count_. This is Undefined Behavior.
The behaviour you observe falls well within Undefined Behavior because that's what UB means: the standard doesn't mandate absolutely any kind of behavior from your program. Trying to understand why exactly happens what happens on your particular compiler and your particular version on your particular platform with your specific flags is ... kind of pointless.
I have ran into a rather confusing problem. It seems like the IF statement in my program is causing me a segmentation error.
I am working with extern libraries, and calling the code from external libraries in the IF statement, so I can't provide the whole code of those functions because I don't have it either.
Basic example of what happens. So this example causes me a Segmentation fault.
IRank *rank;
//Generating wavelet tree from BWT with sdsl library
if(true) {
std::cout << "I am in IF" << endl; // this gets printed on the screen
wt_huff<> wt; // right after that - segm fault
construct_im(wt, BWT, 1);
WTRank wtrank(&wt);
rank = &wtrank;
}
However, the same example, but without an IF, when I comment it out, does not cause Segmentation fault, and executes normally.
IRank *rank;
//Generating wavelet tree from BWT with sdsl library
//if(true) {
std::cout << "I am in IF" << endl; // again this gets printed
wt_huff<> wt; // no segmentation error this time
construct_im(wt, BWT, 1);
WTRank wtrank(&wt);
rank = &wtrank;
//}
Original example:
// // Decide what rank function to use
IRank *rank;
if(m_wt) {
// Multiary Wavelet Tree rank function :: student implementation
mwt::node *m_wtree = mwt::generateMultiaryWT(BWT, ary);
MultiWTRank m_wt_rank(m_wtree, ary);
rank = &m_wt_rank;
} else if(b_wt) {
// Binary Wavelet Tree rank function :: SDSL implementation
wt_huff<> b_wtree;
construct_im(b_wtree, BWT, 1);
WTRank b_wt_rank(&b_wtree);
rank = &b_wt_rank;
} else if(non_wt) {
// Implementation of rank function not using Wavelet Tree
LinRank lin_rank(BWT);
rank = &lin_rank;
} else {
// should not happen
}
//...
run(rank);
What happens here, it is so confusing?
EDIT: example of other code being called from this snipper
#include "IRank.h"
#include "mwt.h"
class MultiWTRank : public IRank {
private:
mwt::node *wt;
int n_ary;
public:
MultiWTRank(mwt::node *root, int ary) {
wt = root;
n_ary = ary;
}
~MultiWTRank() {
}
index_type rank(index_type index, symbol_type symbol);
};
So this is being constructed in the first IF.
EDIT2: Providing a code that generates a pointer to the tree that could cause the trouble
class mwt {
public:
// Structure of a MW tree node
typedef struct node {
vector<int> data;
vector<node*> next;
} node;
// ...
static node* generateMultiaryWT(string input, int ary) {
//...
return root;
}
Node is created like this:
static node* InitRoot(int ary){
node *root = new node;
for(int iter = 0; iter < ary; iter++){
root->next.push_back(NULL);
}
return root;
}
Declare the 'wt' and 'wtrank' variables before the if. If you declare it inside the block following the if, its scope is limited to that block. After the } it is out of scope and the 'rank' pointer becomes dangling, so accessing it later may cause a segfault.
Your problem is almost certainly some other code you have not shown doing something untoward - molesting a pointer, falling off the end of an array, accessing value of an uninitialised variable, etc.
Introducing an if (true) around some block, at most, will change memory layout of your program (e.g. if storage is set aside to hold the value true, and if the compiler emits some code to test it before executing the subsequent code). Because the memory layout changes, the implications of misbehaving code (i.e. what gets clobbered) can change.
Naturally, in this case, the possible change depends on the compiler. An aggressive optimisation may detect that true is always (well) true, and therefore eliminate the if (true) entirely from emitted code. In this case, there will be no difference on program behaviour of having it or not. However, not all compilers (or compiler settings) do that.
Incidentally, the advice to change where you define the variable wt might or might not work for similar reasons. Moving the definition might simply change the order of actions in code (machine instructions, etc), or the layout of memory as used by your program (particularly if the constructor for that object allocates significant resources). So it is not a solution, even if it might appear to work. Because it is not guaranteed to work. And may break because of other changes (of your code, compiler, compilation settings, etc) in future.
The thing is, the real problem might be in code you have shown (impact of functions being called, constructors being invoked, etc) or it might be in code executed previously in your program. Such is the nature of undefined behaviour - when a problem occurs, the symptom may not become visible immediately, but may affect behaviour of unrelated code.
Given where the problem occurs, the rank = &wtrank statement is not the cause. The cause will be in previous code. However, that dangling pointer will be another problem for subsequently executed code - once this problem is fixed.
Why would you want the declaration of > wt in the IF statement?
I'm checking for memory leaks in my Qt program using QtCreator and Valgrind. I am deleting a few entries in a QHash in my destructor like this:
QHash<QString, QVariant*> m_Hash;
/**
* #brief
* Destruct a Foo Class instance
*/
Foo ::~Foo ()
{
// Do Cleanup here
// Delete hash leftovers
foreach( QString key, m_Hash.keys() )
{
qDebug() << "Deleting an entry..";
// Delete the hash item
delete m_Hash.take(key);
}
}
If I debug with Valgrind this code is fine and deletes the contents when the destructor is called:
>> Deleting an entry..
>> Deleting an entry..
If I launch with GDB within QtCreator, launch without GDB from QtCreator, or just run my Qt App from the command line I get Segmentation Faults!
Signal name :
SIGSEGV
Signal meaning :
Segmentation fault
If I commend out the 'delete' line then I can run my app just fine using any method but I do leak memory.
What gives? Does valgrind introduce some sort of delay that allows my destructor to work? How can I solve this?
hyde's answer is correct; however, the simplest possible way to clear your particular hash is as follows:
#include <QtAlgorithms>
Foo::~Foo()
{
qDeleteAll(m_Hash);
m_Hash.clear();
}
Note that the above technique would not work if the key of the hash table was a pointer (e.g. QHash<QString*, QVariant>).
You can not modify the container you iterate over with foreach. Use iterators instead. Correct code using method iterator QHash::erase ( iterator pos ):
QMap<QString, QVariant* >::iterator it = m_Hash.begin();
// auto it = m_Hash.begin(); // in C++11
while (it != m_Hash.end()) {
delete it.value();
it = m_Hash.erase(it);
}
Also, any particular reason why you are storing QVariant pointers, instead of values? QVariant is usually suitable for keeping as value, since most data you'd store in QVariant is either implicitly shared, or small.
The documentation does not explicitly mention it, but it is a problem that you are mutating the container you are iterating over.
The code to foreach is here.
Looking at the code, it and your code do basically the same as if you'd write:
for (QHash::iterator it=m_Hash.begin(), end=m_Hash.end();
it!=end;
++it)
{
delete m_Hash.take(key);
}
however, the take member function may trigger an invalidation of existing iterators (it and end), so your iterators might have become dangling, yielding undefined behavior.
Possible solutions:
* do not modify the container you iterator over while iterating
* make sure it is valid before the next iteration begins, and don't store an end-iterator (this solution forbids the use of foreach)
Maybe there is problem with the foreach keyword. Try replacing:
foreach( QString key, m_Hash.keys() )
{
qDebug() << "Deleting an entry..";
delete m_Hash.take(key); // take changes the m_Hash object
}
with:
for (QHash<QString, QVariant*>::iterator it = m_Hash.begin();
it != m_Hash.end(); ++it)
{
qDebug() << "Deleting an entry..";
delete it.value(); // we delete only what it.value() points to, but the
// m_Hash object remains intact.
}
m_Hash.clear();
This way the hash table remains unchanged while you iterate through it. It is possible that the foreach macro expands into a construct where you "delete the hashtable from under your feet". That is the macro probably creates an iterator, which becomes invalid or "dangling" as a side effect of calling
m_Hash.take(key);
I am writing a program to implement a stack which works like a real world stack means it topples when the size of a stack reaches threshold and therefore need to create a new stack for inserting that new element.
Below is my program for this:
#include <iostream>
#include<vector>
#include<stack>
using namespace std;
class stack_of_plates
{
vector<stack<int> > stacks;
unsigned int stack_size;
public:
stack_of_plates(unsigned int size=100)
{
stack_size=size;
}
void push(int data)
{
if(stacks.empty())
{
stack<int> *sptr= new stack<int>; //on debugging Segmentation fault at thisline
stacks.push_back(*sptr);
}
vector<stack<int> >::iterator it=stacks.end();
if(it->size()==stack_size)
{
stack<int> *sptr= new stack<int>; //on debugging Segmentation fault at thisline
stacks.push_back(*sptr);
}
it->push(data);
}
void pop()
{
if(stacks.empty())
{
cout<<"\nEmpty Stack";
return ;
}
vector<stack<int> >::iterator it=stacks.end();
if(it->empty())
{
it--;
}
it->pop();
}
int top()
{
if(stacks.empty())
{
cout<<"\nEmpty Stack";
return 0;
}
vector<stack<int> >::iterator it=stacks.end();
if(it->empty())
{
it--;
}
return it->top();
}
};
int main()
{
stack_of_plates ss;
ss.push(1);
ss.push(2);
cout<<ss.top();
return 0;
}
On compiling it gives no error or warning. However program terminates with unusual error.
On debugging its giving segmentation fault error indicating problem in allocating new stack.
Kindly help me how should i change my code while allocating the new stack. Please help me removing this error.
stacks.end(); refers to the (nonexistent) element after the end of the vector. You can't dereference it; doing so will cause undefined behaviour, possibly a segmentation fault.
It's not quite clear what you're doing there, but if you want an iterator for the last element, then either decrement it:
vector<stack<int> >::iterator it=stacks.end(); // points past the end
--it; // points to last element
or use a reverse iterator (in which case, you use ++ rather than -- to move backwards through the sequence):
vector<stack<int> >::reverse_iterator it=stacks.rbegin();
Adding an element to a vector can invalidate it, so the it->push_back(data) at the end of push() is incorrect. You could avoid using an iterator here:
void push() {
if (stacks.empty() || stacks.back().size()==stack_size) {
// See below for explanation of this change
stacks.push_back(stack<int>());
}
stacks.back().push(data);
}
In pop(), you probably want to remove the last stack if it's empty; otherwise, you'll end up with two empty stacks at the end, and your code will erroneously try to pop from one of those. Again, doing that could cause a segmentation fault or other undefined behavoiur. You probably want something like:
void pop() {
if (stacks.empty()) {
cout<<"\nEmpty Stack";
return ;
}
stacks.back().pop();
if (stacks.back().empty()) {
stacks.pop_back();
}
}
And now we've established an invariant that the last stack is never empty, top can be a bit simpler:
int top() {
if (stacks.empty()) {
cout<<"\nEmpty Stack";
return 0;
}
return stacks.back().top();
}
Also, you usually don't want to create objects using new, especially in a case like this where you're putting a copy of the object into the vector and then discarding the pointer, leaking the allocated memory. You can add an empty stack to the vector like this:
stacks.push_back(stack<int>());
Sometimes (but not in this case) you might want to store pointers to allocated objects in a container; in that case, either remember to delete them when they're removed from the container, or store smart pointers such as std::unique_ptr. But in this case, just store objects.
There are many problems with the code, so it is hard to say which one is the direct cause of your problem. You need the clean them up one by one and then retest. If you still have a problem, post your new code here.
Here is the list:
You have a memory leak from your allocation with new. Since you have a vector of stacks, all you need to do is resize the vector and a new stack will be allocated. So
stacks.resize(stacks.size() + 1);
instead of
stack<int> *sptr= new stack<int>;
stacks.push_back(*sptr);
vector<>.end() returns an iterator that point to an element AFTER the last one, which is why #Joachim suggested that you need to decrement the iterator before you use it.
You have a logical error when you check whether to transfer storage to a new stack - after checking is the size of the last stack is the max, and creating a new one, you keep pushing on the old one.
I hope this helps.
std::stack<int> already has the functionality you show in your example, so there is no need for a std::vector< std::stack<int> >. By just pushing and popping to the std::stack<int> you avoid most of the issues your having in your code. There is no reason to limit std::stack<int> to stack_size.
Next to that, when you need the last entry in a container, use back() instead of end().
My code seems to work(I haven't tried it with large datasets because of the above error).
Code:
#include <iostream>
#include <queue>
#include <stxxl/queue>
int main()
{
//queue<int> q; //this works
stxxl::queue<int> q; //does not work
for (int i = 0; i<100; i++) {
q.push(i);
}
std::cout << "done copying" << std::endl;
while (q.empty() == false) {
std::cout << q.front() << std::endl;
q.pop();
}
std::cout << "done poping" << std::endl;
return 0;
}
my simple .stxxl is simply: disk=./testfile,0,syscall
But my error is:
stackexchangeexample(3884) malloc: *** error for object 0x101c04000: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
The program has unexpectedly finished.
I'm not sure how to troubleshoot it, do I need to free memory in this case? I'm still learning c++ so sorry if this is really basic(this only happens when I use the stxxl queue).
I've never used stxxl before but since it's a template you can take a look at the code here: http://algo2.iti.kit.edu/stxxl/trunk/queue_8h_source.html. And since you're a newbie I'll explain a few things. This goofy queue maintains a queue of pointers. Line 00054 shows typedef ValTp value_type, so now your int is a value_type. Line's 00072 & 00073 show that your front and back elements are of that value_type. Do you see how they will be maintained as pointers. Finally if you look at any constructor the pool_type* pool defined on line 00069 will be "new'd" up (which is the basis of your elements) and the init function is always called. And within init, pool->steal() is called, click on it if you want to learn more.
In short, you need to be pushing new'd up integers onto your queue. Bad interface, not your fault.