I have a python code base that sometimes invokes C++ programs to handle intensive workloads. One such code has to count all kmers of a certain size in a large text file. For each line it reads, it creates a temporary index that stores the position of each kmer. Here is the function that processes each line:
void process_read(char* read, int num) {
int l = strlen(read) ;
std::string seq(read) ;
// index kmers
std::unordered_map<std::string, std::vector<int>> index ;
for (int i = 0 ; i <= l - 1 - 15 ; i++) {
std::string k = seq.substr(i, 15) ;
if (global_index->find(k) == global_index->end()) {
continue ;
}
if (index.find(k) == index.end()) {
index.insert(std::make_pair(k, std::vector<int>(1, i))) ;
} else {
index[k].push_back(i) ;
}
}
// 50+ lines of code commented out. It returns here
}
The code crashes every time it reaches a certain line of input:
ACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCAAACCATAACCCTAAACCTCACGATAACCCAAACCATCACCAAAAAAAAAAAAAACACACCTACCGAAACCAACAACATA
Out of the kmers in this line, only AAAAAAAAAAAAAAC and CAAAAAAAAAAAAAA make it to index. The code always crashes when trying to insert CAAAAAAAAAAAAAA for some reason I don't understand. I guess is a problem with these keys being inserted into the unordered_map in sequence. Changing the function to this will still result in the same crash when inserting the second key:
void process_read(char* read, int num) {
std::unordered_map<std::string, std::vector<int>> index ;
index.insert(std::make_pair("AAAAAAAAAAAAAAC", std::vector<int>(1, 2))) ;
index.insert(std::make_pair("CAAAAAAAAAAAAAA", std::vector<int>(1, 2))) ;
}
Now this function is clearly not accessing any global state unlike the original one so the problem has to be with these specific keys (notice that one is a circular shift of the other, the hash function used might not be comfortable with that); however, putting this code at the start of the program or writing another small program that only does this doesn't seem to reproduce the crash so I'm really confused.
Any suggestion is appreciated.
Update:
I get this stack trace during the crash. For reasons, I can't use gdb to debug so I guess this is the best I'm going to get. But don't know how to interpret it.
*** Error in `src/python/kmer/c_counter.out': malloc(): memory corruption (fast): 0x0000000001eac690 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f189daa47e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x82651)[0x7f189daaf651]
/lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7f189dab1184]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_Znwm+0x18)[0x7f189e3a3e78]
/src/python/kmer/c_counter.out[0x41c5e4]
/src/python/kmer/c_counter.out[0x4146ea]
/src/python/kmer/c_counter.out[0x41453a]
/src/python/kmer/c_counter.out[0x41035b]
/src/python/kmer/c_counter.out[0x40b3d8]
/src/python/kmer/c_counter.out[0x40940a]
/src/python/kmer/c_counter.out[0x404528]
/src/python/kmer/c_counter.out[0x404f9d]
/src/python/kmer/c_counter.out[0x405f42]
/src/python/kmer/c_counter.out[0x4063d6]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f189da4d830]
/src/python/kmer/c_counter.out[0x403b39]
======= Memory map: ========
00400000-00440000 r-xp 00000000 00:2f 546796 src/python/kmer/c_counter.out
0063f000-00640000 rw-p 0003f000 00:2f 546796 src/python/kmer/c_counter.out
014a0000-01ebf000 rw-p 00000000 00:00 0 [heap]
7f1898000000-7f1898021000 rw-p 00000000 00:00 0
7f1898021000-7f189c000000 ---p 00000000 00:00 0
7f189da2d000-7f189dbed000 r-xp 00000000 fc:00 1439150 /lib/x86_64-linux-gnu/libc-2.23.so
7f189dbed000-7f189dded000 ---p 001c0000 fc:00 1439150 /lib/x86_64-linux-gnu/libc-2.23.so
7f189dded000-7f189ddf1000 r--p 001c0000 fc:00 1439150 /lib/x86_64-linux-gnu/libc-2.23.so
7f189ddf1000-7f189ddf3000 rw-p 001c4000 fc:00 1439150 /lib/x86_64-linux-gnu/libc-2.23.so
7f189ddf3000-7f189ddf7000 rw-p 00000000 00:00 0
7f189ddf7000-7f189de0d000 r-xp 00000000 fc:00 1439041 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f189de0d000-7f189e00c000 ---p 00016000 fc:00 1439041 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f189e00c000-7f189e00d000 rw-p 00015000 fc:00 1439041 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f189e00d000-7f189e115000 r-xp 00000000 fc:00 1439141 /lib/x86_64-linux-gnu/libm-2.23.so
7f189e115000-7f189e314000 ---p 00108000 fc:00 1439141 /lib/x86_64-linux-gnu/libm-2.23.so
7f189e314000-7f189e315000 r--p 00107000 fc:00 1439141 /lib/x86_64-linux-gnu/libm-2.23.so
7f189e315000-7f189e316000 rw-p 00108000 fc:00 1439141 /lib/x86_64-linux-gnu/libm-2.23.so
7f189e316000-7f189e488000 r-xp 00000000 fc:00 671990 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7f189e488000-7f189e688000 ---p 00172000 fc:00 671990 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7f189e688000-7f189e692000 r--p 00172000 fc:00 671990 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7f189e692000-7f189e694000 rw-p 0017c000 fc:00 671990 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7f189e694000-7f189e698000 rw-p 00000000 00:00 0
7f189e698000-7f189e6be000 r-xp 00000000 fc:00 1439146 /lib/x86_64-linux-gnu/ld-2.23.so
7f189e878000-7f189e89f000 rw-p 00000000 00:00 0
7f189e8bc000-7f189e8bd000 rw-p 00000000 00:00 0
7f189e8bd000-7f189e8be000 r--p 00025000 fc:00 1439146 /lib/x86_64-linux-gnu/ld-2.23.so
7f189e8be000-7f189e8bf000 rw-p 00026000 fc:00 1439146 /lib/x86_64-linux-gnu/ld-2.23.so
7f189e8bf000-7f189e8c0000 rw-p 00000000 00:00 0
7ffea4907000-7ffea4929000 rw-p 00000000 00:00 0 [stack]
7ffea49b6000-7ffea49b9000 r--p 00000000 00:00 0 [vvar]
7ffea49b9000-7ffea49bb000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Changing the function to this will still result in the same crash when inserting the second key:
[...]
putting this code at the start of the program or writing another small program that only does this doesn't seem to reproduce the crash so I'm really confused.
std::unordered_map has no relevant global state that could change between "everything is fine if I run this test function at the start" and "if I run this test function later, the map crashes". You have memory corruption due to undefined behavior somewhere else in your program - the observations you made are the strongest proof you could get for that.
The function signature suggests that you don't necessarily have a null ended string(one that has a \0 as the last character).
But you don't treat it as such(you should use variants that take num as a parameter). I suspect this pattern repeats in other places and at some point you corrupt your memory.
If I were you i would build the binary with Valgrind or some other memory analysis tool and run it again. It will catch the incorrect access where it happens.
The way you use unordered_map looks fine to me.
Related
I was trying to solve a hackerearth problems to understand c++ better.
The Task is as follows :
The first line will consists of one integer T denoting the number of test cases.
For each test case:
The first line consists of two integers N and K, N being the number of elements in the array and K denotes the number of steps of rotation.
The next line consists of N space separated integers , denoting the elements of the array A.
So,If the input is like
1
5 2
1 2 3 4 5
The output should be
4 5 1 2 3
The code that i have written is as follows :
#include <iostream>
#include<bits/stdc++.h>
using namespace std;
void ans(int r,vector<int> &a)
{
reverse(a.begin(),a.end());
reverse(a.begin(),a.begin()+r);
reverse(a.begin()+r,a.end());
}
int main() {
int testno;
cin>>testno;
int size,rotation,temp;
vector<vector<int>> arr(testno);
for(int i=0;i<testno;i++)
{
cin>>size;
cin>>rotation;
for(int j=0;j<size;j++)
{
cin>>temp;
arr[i].push_back(temp);
};
ans(rotation,arr[i]);
};
for(int i=0;i<arr.size();i++)
{
for(int j=0;j<arr[i].size();j++)
cout<<arr[i][j]<<" ";
cout<<"\n";
};
}
This code works fine for small test-cases or numbers.
But when the input is huge,I get the following error :
Execution failed.
Stack Trace:
*** Error in `/hackerearth/CPP17_a870_37de_35ba_2748/s_3a8f_d922_ff44_e648.cpp.out': malloc(): memory corruption (fast): 0x0000000001b0ad50 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777f5)[0x7f10c1df57f5]
/lib/x86_64-linux-gnu/libc.so.6(+0x82679)[0x7f10c1e00679]
/lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7f10c1e021d4]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_Znwm+0x15)[0x7f10c2405c75]
/hackerearth/CPP17_a870_37de_35ba_2748/s_3a8f_d922_ff44_e648.cpp.out[0x400f62]
/hackerearth/CPP17_a870_37de_35ba_2748/s_3a8f_d922_ff44_e648.cpp.out[0x400bbb]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f10c1d9e840]
/hackerearth/CPP17_a870_37de_35ba_2748/s_3a8f_d922_ff44_e648.cpp.out[0x400d49]
======= Memory map: ========
00400000-00402000 r-xp 00000000 07:00 28 /hackerearth/CPP17_a870_37de_35ba_2748/s_3a8f_d922_ff44_e648.cpp.out
00601000-00602000 r--p 00001000 07:00 28 /hackerearth/CPP17_a870_37de_35ba_2748/s_3a8f_d922_ff44_e648.cpp.out
00602000-00603000 rw-p 00002000 07:00 28 /hackerearth/CPP17_a870_37de_35ba_2748/s_3a8f_d922_ff44_e648.cpp.out
01af8000-01b2a000 rw-p 00000000 00:00 0 [heap]
7f10bc000000-7f10bc021000 rw-p 00000000 00:00 0
7f10bc021000-7f10c0000000 ---p 00000000 00:00 0
7f10c1a75000-7f10c1b7d000 r-xp 00000000 ca:01 48154 /lib/x86_64-linux-gnu/libm-2.23.so
7f10c1b7d000-7f10c1d7c000 ---p 00108000 ca:01 48154 /lib/x86_64-linux-gnu/libm-2.23.so
7f10c1d7c000-7f10c1d7d000 r--p 00107000 ca:01 48154 /lib/x86_64-linux-gnu/libm-2.23.so
7f10c1d7d000-7f10c1d7e000 rw-p 00108000 ca:01 48154 /lib/x86_64-linux-gnu/libm-2.23.so
7f10c1d7e000-7f10c1f3e000 r-xp 00000000 ca:01 48130 /lib/x86_64-linux-gnu/libc-2.23.so
7f10c1f3e000-7f10c213e000 ---p 001c0000 ca:01 48130 /lib/x86_64-linux-gnu/libc-2.23.so
7f10c213e000-7f10c2142000 r--p 001c0000 ca:01 48130 /lib/x86_64-linux-gnu/libc-2.23.so
7f10c2142000-7f10c2144000 rw-p 001c4000 ca:01 48130 /lib/x86_64-linux-gnu/libc-2.23.so
7f10c2144000-7f10c2148000 rw-p 00000000 00:00 0
7f10c2148000-7f10c215f000 r-xp 00000000 ca:01 18214 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f10c215f000-7f10c235e000 ---p 00017000 ca:01 18214 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f10c235e000-7f10c235f000 r--p 00016000 ca:01 18214 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f10c235f000-7f10c2360000 rw-p 00017000 ca:01 18214 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f10c2360000-7f10c2533000 r-xp 00000000 ca:01 16581 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
7f10c2533000-7f10c2732000 ---p 001d3000 ca:01 16581 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
7f10c2732000-7f10c273d000 r--p 001d2000 ca:01 16581 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
7f10c273d000-7f10c2740000 rw-p 001dd000 ca:01 16581 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
7f10c2740000-7f10c2743000 rw-p 00000000 00:00 0
7f10c2743000-7f10c2769000 r-xp 00000000 ca:01 48198 /lib/x86_64-linux-gnu/ld-2.23.so
7f10c2948000-7f10c294e000 rw-p 00000000 00:00 0
7f10c2967000-7f10c2968000 rw-p 00000000 00:00 0
7f10c2968000-7f10c2969000 r--p 00025000 ca:01 48198 /lib/x86_64-linux-gnu/ld-2.23.so
7f10c2969000-7f10c296a000 rw-p 00026000 ca:01 48198 /lib/x86_64-linux-gnu/ld-2.23.so
7f10c296a000-7f10c296b000 rw-p 00000000 00:00 0
7fff76e54000-7fff76e75000 rw-p 00000000 00:00 0 [stack]
7fff76f09000-7fff76f0b000 r--p 00000000 00:00 0 [vvar]
7fff76f0b000-7fff76f0d000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
I don't understand why there is a runtime malloc error? What exactly is happening that is leading to this error for large inputs? Why is there memory corruption? Am I trying to access memory not allocated to me during the runtime? If so,Where? The link to the problem,if needed is : https://www.hackerearth.com/practice/codemonk/
Turns out some test cases had rotation higher than the size of vector.So just updated rotation to rotation%size. Working fine now.
I've decided to parallelize a huge program I wrote and eventually I came across the new C++11 smart pointers.
I had a routine that should be executed many times (usually above 1000 times) that is somewhat expensive. It was ran in a dead simple for loop, what I did was to install this for loop in a method which would be ran by some worker threads.
Did it so, made some parameters wrapped by a std::shared_ptr, cared for race conditions, everything seemed fine.
But now, some times, process will be aborted and I am given one of the following errors:
Process finished with exit code 139 (interrupted by signal 11:
SIGSEGV)
or
malloc.c:2395: sysmalloc: Assertion `(old_top == initial_top (av) &&
old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse
(old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)' failed.
All of these errors occur while the parallel for is ongoing; not before, not right in the beginning, not in the end, but somewhere in between, which smells me like a race condition not covered.
The program is huge, but I created a miniature of it that is able to reproduce the problem:
#include <iostream>
#include <vector>
#include <unordered_map>
#include <thread>
#include <set>
#include <memory>
#include <atomic>
namespace std {
template <>
struct hash<std::multiset<unsigned long>>
{
std::size_t operator()(const std::multiset<unsigned long>& k) const
{
std::size_t r = 0;
bool shift = false;
for (auto&& it : k) {
r = (r >> !shift) ^ (std::hash<unsigned long>()(it) << shift);
shift = !shift;
}
return r;
}
};
}
typedef std::unordered_map<std::multiset<unsigned long>, int*> graphmap;
std::multiset<unsigned long>* bar(int pos) {
std::multiset<unsigned long> *label = new std::multiset<unsigned long>;
label->insert(pos%5);
label->insert(pos%2);
return label;
}
void foo(std::shared_ptr<graphmap> &kSubgraphs, int pos) {
int *v = (*kSubgraphs)[*bar(pos)];
if(v == nullptr) {
v = new int[pos+1]();
v[0]++;
} else {
v[pos]++;
}
}
void worker(std::shared_ptr<std::atomic_int> *counter, int n, std::shared_ptr<graphmap> *kSubgraphs)
{
for (int pos = (*counter)->fetch_add(1); pos <= n; pos = (*counter)->fetch_add(1)) {
if (pos%100==0) std::cout << pos << std::endl;
foo(*kSubgraphs, pos);
}
}
int main() {
int n = 1000;
std::vector<std::thread> threads;
std::shared_ptr<graphmap> kSubgraphs = std::make_shared<graphmap>();
std::shared_ptr<std::atomic_int> counter = std::make_shared<std::atomic_int>(0);
for (int i=0; i<5; i++) {
foo(kSubgraphs, n);
}
for (int i=0; i<4; i++) {
threads.push_back(std::thread(worker, &counter, n, &kSubgraphs));
}
for(auto& th : threads) th.join();
return 0;
}
This code basically mimics the behavior of the original code, which is to use an unordered_map keyed by a multiset with values that are pointers to int arrays. First some keys are inserted and the array initialized (maybe the problem is caused by the way I am initializing it?) and finally the worker threads run to update a unique position of the array of an entry of the unordered_map.
Two threads may access the same map entry simultaneously, but they will never write to the same index of the array at the same time.
Different from the original code, this code won't throw any errors when running over internet compilers such as ideone.com, I also tried to run it from CLion ide and no errors will occur (maybe it can occur errors if one tries enough times), but I got similar error as the original when running from the command line multiple times. I compiled it with:
g++ -std=c++11 -pthread -o test.exe test.cpp
And after running several times it eventually gives this error:
*** Error in `./test.exe': double free or corruption (fasttop): 0x00000000006a2d30 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x77725)[0x7fccc9d4f725]
/lib/x86_64-linux-gnu/libc.so.6(+0x7ff4a)[0x7fccc9d57f4a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fccc9d5babc]
./test.exe[0x404e9e]
./test.exe[0x40431b]
./test.exe[0x4045ed]
./test.exe[0x407c6c]
./test.exe[0x4078d6]
./test.exe[0x40742a]
./test.exe[0x40869e]
./test.exe[0x4086be]
./test.exe[0x4085dd]
./test.exe[0x40842d]
./test.exe[0x4023a2]
./test.exe[0x401d55]
./test.exe[0x401c4a]
./test.exe[0x401c66]
./test.exe[0x401702]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fccc9cf8830]
./test.exe[0x401199]
======= Memory map: ========
00400000-0040f000 r-xp 00000000 08:05 12202697 /home/rodrigo/test.exe
0060e000-0060f000 r--p 0000e000 08:05 12202697 /home/rodrigo/test.exe
0060f000-00610000 rw-p 0000f000 08:05 12202697 /home/rodrigo/test.exe
00691000-006c3000 rw-p 00000000 00:00 0 [heap]
7fcca8000000-7fcca8089000 rw-p 00000000 00:00 0
7fcca8089000-7fccac000000 ---p 00000000 00:00 0
7fccb0000000-7fccb008b000 rw-p 00000000 00:00 0
7fccb008b000-7fccb4000000 ---p 00000000 00:00 0
7fccb8000000-7fccb8089000 rw-p 00000000 00:00 0
7fccb8089000-7fccbc000000 ---p 00000000 00:00 0
7fccc0000000-7fccc007c000 rw-p 00000000 00:00 0
7fccc007c000-7fccc4000000 ---p 00000000 00:00 0
7fccc79cb000-7fccc79cc000 ---p 00000000 00:00 0
7fccc79cc000-7fccc81cc000 rw-p 00000000 00:00 0
7fccc81cc000-7fccc81cd000 ---p 00000000 00:00 0
7fccc81cd000-7fccc89cd000 rw-p 00000000 00:00 0
7fccc89cd000-7fccc89ce000 ---p 00000000 00:00 0
7fccc89ce000-7fccc91ce000 rw-p 00000000 00:00 0
7fccc91ce000-7fccc91cf000 ---p 00000000 00:00 0
7fccc91cf000-7fccc99cf000 rw-p 00000000 00:00 0
7fccc99cf000-7fccc9ad7000 r-xp 00000000 08:05 24126366 /lib/x86_64-linux-gnu/libm-2.23.so
7fccc9ad7000-7fccc9cd6000 ---p 00108000 08:05 24126366 /lib/x86_64-linux-gnu/libm-2.23.so
7fccc9cd6000-7fccc9cd7000 r--p 00107000 08:05 24126366 /lib/x86_64-linux-gnu/libm-2.23.so
7fccc9cd7000-7fccc9cd8000 rw-p 00108000 08:05 24126366 /lib/x86_64-linux-gnu/libm-2.23.so
7fccc9cd8000-7fccc9e98000 r-xp 00000000 08:05 24126374 /lib/x86_64-linux-gnu/libc-2.23.so
7fccc9e98000-7fccca097000 ---p 001c0000 08:05 24126374 /lib/x86_64-linux-gnu/libc-2.23.so
7fccca097000-7fccca09b000 r--p 001bf000 08:05 24126374 /lib/x86_64-linux-gnu/libc-2.23.so
7fccca09b000-7fccca09d000 rw-p 001c3000 08:05 24126374 /lib/x86_64-linux-gnu/libc-2.23.so
7fccca09d000-7fccca0a1000 rw-p 00000000 00:00 0
7fccca0a1000-7fccca0b9000 r-xp 00000000 08:05 24126373 /lib/x86_64-linux-gnu/libpthread-2.23.so
7fccca0b9000-7fccca2b8000 ---p 00018000 08:05 24126373 /lib/x86_64-linux-gnu/libpthread-2.23.so
7fccca2b8000-7fccca2b9000 r--p 00017000 08:05 24126373 /lib/x86_64-linux-gnu/libpthread-2.23.so
7fccca2b9000-7fccca2ba000 rw-p 00018000 08:05 24126373 /lib/x86_64-linux-gnu/libpthread-2.23.so
7fccca2ba000-7fccca2be000 rw-p 00000000 00:00 0
7fccca2be000-7fccca2d4000 r-xp 00000000 08:05 24121519 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fccca2d4000-7fccca4d3000 ---p 00016000 08:05 24121519 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fccca4d3000-7fccca4d4000 rw-p 00015000 08:05 24121519 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fccca4d4000-7fccca646000 r-xp 00000000 08:05 6029347 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7fccca646000-7fccca846000 ---p 00172000 08:05 6029347 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7fccca846000-7fccca850000 r--p 00172000 08:05 6029347 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7fccca850000-7fccca852000 rw-p 0017c000 08:05 6029347 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7fccca852000-7fccca856000 rw-p 00000000 00:00 0
7fccca856000-7fccca87c000 r-xp 00000000 08:05 24126370 /lib/x86_64-linux-gnu/ld-2.23.so
7fcccaa44000-7fcccaa4a000 rw-p 00000000 00:00 0
7fcccaa78000-7fcccaa7b000 rw-p 00000000 00:00 0
7fcccaa7b000-7fcccaa7c000 r--p 00025000 08:05 24126370 /lib/x86_64-linux-gnu/ld-2.23.so
7fcccaa7c000-7fcccaa7d000 rw-p 00026000 08:05 24126370 /lib/x86_64-linux-gnu/ld-2.23.so
7fcccaa7d000-7fcccaa7e000 rw-p 00000000 00:00 0
7ffc6b1c8000-7ffc6b1e9000 rw-p 00000000 00:00 0 [stack]
7ffc6b1fa000-7ffc6b1fc000 r--p 00000000 00:00 0 [vvar]
7ffc6b1fc000-7ffc6b1fe000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
[1] 26639 abort (core dumped) ./test.exe
At last, when running the original code with debug mode on in CLion set up to capture Exceptions as breakpoints, it shows a Signal interruption of SIGBUS(Bus error) or a SIGSEGV(Segmentation fault) sometimes and last line of code executed before the interruption maps to the line:
int *v = (*kSubgraphs)[*bar(pos)];
in the foo function of the code presented here.
I am a little bit lost with this one. My strongest assumption is that I am using smart pointers the wrong way, despite I do not see where.
Your program has undefined behavior, because you access the object pointed to by kSubgraphs from different threads without mutual exclusion. This happens in this line of code in function foo, which is called from your thread function worker:
int *v = (*kSubgraphs)[*bar(pos)];
I can't guess why you think shared_ptr would help your case in any way. The way you do, it is totally pointless to wrap your objects in shared_ptr.
shared_ptr is used when ownership of your objects is shared from different other objects. It has nothing to do with "sharing objects between threads".
I am trying to make a simple std::vector code on C++ with g++ on linux terminal but confused with the output on the terminal itself.
My code is simple.
#include <iostream>
#include <vector>
int main() {
double xl;
int nx=5;
double delta_x;
int i=0;
std::vector<double> x(nx);
for(i=0; i <=nx; i++) {
x[i]=0.0005;
std::cout<<x[i]<<std::endl;
}
return 0;
}
when I compile:
$ g++ grid2.cpp -o grid2
It succeeded and gives a grid2 executable. The problem arise when I run the executable:
$ ./grid2
gives me output like this on the terminal:
./grid2[0x400929]
======= Memory map: ========
00400000-00402000 r-xp 00000000 08:05 13372783 /mnt/home/FINALS/codeDNS_AKA_MY_MAIN_DIR/basic_CPP/grid2
00601000-00602000 rw-p 00001000 08:05 13372783 /mnt/home/FINALS/codeDNS_AKA_MY_MAIN_DIR/basic_CPP/grid2
01f23000-01f55000 rw-p 00000000 00:00 0 [heap]
7f13fc000000-7f13fc021000 rw-p 00000000 00:00 0
7f13fc021000-7f1400000000 ---p 00000000 00:00 0
7f1403971000-7f1403b0c000 r-xp 00000000 08:06 790319 /usr/lib/libc-2.22.so
7f1403b0c000-7f1403d0b000 ---p 0019b000 08:06 790319 /usr/lib/libc-2.22.so
7f1403d0b000-7f1403d0f000 r--p 0019a000 08:06 790319 /usr/lib/libc-2.22.so
7f1403d0f000-7f1403d11000 rw-p 0019e000 08:06 790319 /usr/lib/libc-2.22.so
7f1403d11000-7f1403d15000 rw-p 00000000 00:00 0
7f1403d15000-7f1403d2b000 r-xp 00000000 08:06 790384 /usr/lib/libgcc_s.so.1
7f1403d2b000-7f1403f2a000 ---p 00016000 08:06 790384 /usr/lib/libgcc_s.so.1
7f1403f2a000-7f1403f2b000 rw-p 00015000 08:06 790384 /usr/lib/libgcc_s.so.1
7f1403f2b000-7f1404028000 r-xp 00000000 08:06 790449 /usr/lib/libm-2.22.so
7f1404028000-7f1404227000 ---p 000fd000 08:06 790449 /usr/lib/libm-2.22.so
7f1404227000-7f1404228000 r--p 000fc000 08:06 790449 /usr/lib/libm-2.22.so
7f1404228000-7f1404229000 rw-p 000fd000 08:06 790449 /usr/lib/libm-2.22.so
7f1404229000-7f140439b000 r-xp 00000000 08:06 790552 /usr/lib/libstdc++.so.6.0.21
7f140439b000-7f140459b000 ---p 00172000 08:06 790552 /usr/lib/libstdc++.so.6.0.21
7f140459b000-7f14045a5000 r--p 00172000 08:06 790552 /usr/lib/libstdc++.so.6.0.21
7f14045a5000-7f14045a7000 rw-p 0017c000 08:06 790552 /usr/lib/libstdc++.so.6.0.21
7f14045a7000-7f14045ab000 rw-p 00000000 00:00 0
7f14045ab000-7f14045cd000 r-xp 00000000 08:06 790282 /usr/lib/ld-2.22.so
7f1404791000-7f1404797000 rw-p 00000000 00:00 0
7f14047ca000-7f14047cc000 rw-p 00000000 00:00 0
7f14047cc000-7f14047cd000 r--p 00021000 08:06 790282 /usr/lib/ld-2.22.so
7f14047cd000-7f14047ce000 rw-p 00022000 08:06 790282 /usr/lib/ld-2.22.so
7f14047ce000-7f14047cf000 rw-p 00000000 00:00 0
7ffec8a9b000-7ffec8abc000 rw-p 00000000 00:00 0 [stack]
7ffec8bef000-7ffec8bf1000 r--p 00000000 00:00 0 [vvar]
7ffec8bf1000-7ffec8bf3000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Aborted (core dumped)
But the (another) weird thing is, when I change the
std::vector<double>
into
std::vector<float>
Those trash are all gone. And I can see the clean output right on my terminal.
And the other thing, when I print the output to another file like this:
$ ./grid2 > test.txt
the results are the same for both double and float. No trash, clean output. It's like the trash only appear on my terminal.
I am novice on linux and c++, so please if anyone know what I'm facing, I'll be very happy to hear that :)
As already noted, you have undefined behavior.
Although changing the <= to < does work, it just makes this code correct. It doesn't "immunize" you against running into essentially the same problem again soon.
One alternative would be to use the "range-based" for loop like this:
#include <iostream>
#include <vector>
int main() {
double xl;
int nx=5;
double delta_x;
int i=0;
std::vector<double> x(nx);
for(double &d : x) {
d=0.0005;
std::cout<<d<<"\n";
}
}
Though in this case, you're filling it with all identical values, so it's probably better to just initialize it that way:
#include <iostream>
#include <vector>
int main() {
std::vector<double> x(5, 0.0005);
for (double const &d : x)
std::cout << d << "\n";
}
The range-based for loop makes it much more difficult to accidentally attempt to access outside the bounds of the collection.
As an aside: note the use of "\n" instead of std::endl. I would recommend against using std::endl in general--on the rare occasion that you need what it does, make that explicit with std::cout << "\n" << std::flush;. In the much more common case that you didn't really want that flush, just omit it. endl does both, even though the flush is almost never desired (and most people aren't aware it even happens).
x has 5 elements. Unfortunately you are accessing 6 - your for loop gives i values from 0 to 5 inclusive, which is a total of 6. Accessing x[5] is undefined behaviour; if you're lucky it crashes in an obvious way, but anything could happen.
You either need to make x bigger, or write your loop as
for(i=0; i < nx; i++)
(using < rather than <=).
Step 1. Create an instance of a class
Step 2. Push this instance to a vector
Step 3. Call delete this; in a member method of an instance
Step 4. Everything is Ok
Step 5. Push something to the vector and get this
*** glibc detected *** ./app: double free or corruption (fasttop): 0x0000000001017930 ***
======= Backtrace: =========
/lib/libc.so.6(+0x71bd6)[0x7f607d60cbd6]
/lib/libc.so.6(cfree+0x6c)[0x7f607d61194c]
./app[0x40231c]
./app[0x402290]
./app[0x4053c0]
./app[0x4048fe]
./app[0x404246]
./app[0x403fe0]
./app[0x402400]
./app[0x4035cb]
./app[0x4034d3]
/lib/libpthread.so.0(+0x68ca)[0x7f607e2b78ca]
/lib/libc.so.6(clone+0x6d)[0x7f607d66a92d]
======= Memory map: ========
00400000-0040f000 r-xp 00000000 09:03 60427370 /root/AHS/app
0060e000-0060f000 rw-p 0000e000 09:03 60427370 /root/AHS/app
01017000-01038000 rw-p 00000000 00:00 0 [heap]
7f6074000000-7f6074021000 rw-p 00000000 00:00 0
7f6074021000-7f6078000000 ---p 00000000 00:00 0
7f607a595000-7f607a596000 ---p 00000000 00:00 0
7f607a596000-7f607ad96000 rw-p 00000000 00:00 0
7f607ad96000-7f607ad97000 ---p 00000000 00:00 0
7f607ad97000-7f607b597000 rw-p 00000000 00:00 0
7f607b597000-7f607b598000 ---p 00000000 00:00 0
7f607b598000-7f607bd98000 rw-p 00000000 00:00 0
7f607bd98000-7f607bd99000 ---p 00000000 00:00 0
7f607bd99000-7f607c599000 rw-p 00000000 00:00 0
7f607c599000-7f607c59a000 ---p 00000000 00:00 0
7f607c59a000-7f607cd9a000 rw-p 00000000 00:00 0
7f607cd9a000-7f607cd9b000 ---p 00000000 00:00 0
7f607cd9b000-7f607d59b000 rw-p 00000000 00:00 0
7f607d59b000-7f607d6f4000 r-xp 00000000 09:03 60425052 /lib/libc-2.11.3.so
7f607d6f4000-7f607d8f3000 ---p 00159000 09:03 60425052 /lib/libc-2.11.3.so
7f607d8f3000-7f607d8f7000 r--p 00158000 09:03 60425052 /lib/libc-2.11.3.so
7f607d8f7000-7f607d8f8000 rw-p 0015c000 09:03 60425052 /lib/libc-2.11.3.so
7f607d8f8000-7f607d8fd000 rw-p 00000000 00:00 0
7f607d8fd000-7f607d913000 r-xp 00000000 09:03 60425245 /lib/libgcc_s.so.1
7f607d913000-7f607db12000 ---p 00016000 09:03 60425245 /lib/libgcc_s.so.1
7f607db12000-7f607db13000 rw-p 00015000 09:03 60425245 /lib/libgcc_s.so.1
7f607db13000-7f607db93000 r-xp 00000000 09:03 60425438 /lib/libm-2.11.3.so
7f607db93000-7f607dd93000 ---p 00080000 09:03 60425438 /lib/libm-2.11.3.so
7f607dd93000-7f607dd94000 r--p 00080000 09:03 60425438 /lib/libm-2.11.3.so
7f607dd94000-7f607dd95000 rw-p 00081000 09:03 60425438 /lib/libm-2.11.3.so
7f607dd95000-7f607de8b000 r-xp 00000000 09:03 60032880 /usr/lib/libstdc++.so.6.0.13
7f607de8b000-7f607e08b000 ---p 000f6000 09:03 60032880 /usr/lib/libstdc++.so.6.0.13
7f607e08b000-7f607e092000 r--p 000f6000 09:03 60032880 /usr/lib/libstdc++.so.6.0.13
7f607e092000-7f607e094000 rw-p 000fd000 09:03 60032880 /usr/lib/libstdc++.so.6.0.13
7f607e094000-7f607e0a9000 rw-p 00000000 00:00 0
7f607e0a9000-7f607e0b0000 r-xp 00000000 09:03 60425177 /lib/librt-2.11.3.so
7f607e0b0000-7f607e2af000 ---p 00007000 09:03 60425177 /lib/librt-2.11.3.so
7f607e2af000-7f607e2b0000 r--p 00006000 09:03 60425177 /lib/librt-2.11.3.so
7f607e2b0000-7f607e2b1000 rw-p 00007000 09:03 60425177 /lib/librt-2.11.3.so
7f607e2b1000-7f607e2c8000 r-xp 00000000 09:03 60425205 /lib/libpthread-2.11.3.so
7f607e2c8000-7f607e4c7000 ---p 00017000 09:03 60425205 /lib/libpthread-2.11.3.so
7f607e4c7000-7f607e4c8000 r--p 00016000 09:03 60425205 /lib/libpthread-2.11.3.so
7f607e4c8000-7f607e4c9000 rw-p 00017000 09:03 60425205 /lib/libpthread-2.11.3.so
7f607e4c9000-7f607e4cd000 rw-p 00000000 00:00 0
7f607e4cd000-7f607e4eb000 r-xp 00000000 09:03 60425293 /lib/ld-2.11.3.so
7f607e6da000-7f607e6df000 rw-p 00000000 00:00 0
7f607e6e7000-7f607e6ea000 rw-p 00000000 00:00 0
7f607e6ea000-7f607e6eb000 r--p 0001d000 09:03 60425293 /lib/ld-2.11.3.so
7f607e6eb000-7f607e6ec000 rw-p 0001e000 09:03 60425293 /lib/ld-2.11.3.so
7f607e6ec000-7f607e6ed000 rw-p 00000000 00:00 0
7fff4ee3b000-7fff4ee50000 rw-p 00000000 00:00 0 [stack]
7fff4efff000-7fff4f000000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Aborted
Could someone please tell me, what is this, why this occurs and how do I fix it?
It's because you freed memory you didn't own. The vector owns the memory containing its contents.
delete this; is like taking a rental car to the salvage yard when you're done with it. Don't do that, the rental company expects it back!
Be careful of the difference between owning memory, and merely having control of it loaned to you.
Doing delete this for an object belonging to a vector you are making two big mistakes:
it's the vector that manages the memory where that instance is stored (it allocates it, deallocates it, ...), so you shouldn't mess with it. In other words, since you don't own that memory you must leave it alone.
delete is a correct match if the object was allocated with new - but the object managed by the vector is not allocated like that; instead, vector grabs a chunk of memory from the allocator (as big as it thinks it's sensible) and copies there the elements that are pushed into it using placement new; so, not only you are freeing memory that you don't own, you are also using the wrong method to free it.
If you need to remove an element from a vector, just use the vector::erase method.
If you do delete this in any method, you have to be sure that no one else calls any method ( for that matter any code) of the instance after that statement. This includes the destructor.
When you push the instance into a vector, while being destroyed, the vector calls the destructor for the instance, hence the double free
If you need a vector, you can push pointers to the instance and it would be fine.
However, as others have said DO NOT use delete this unless absolutely necessary
You would also have the problems if you just create an local instance of the class in a function. Most likely you did not see this behavior because your program finished before the scope of the local variable ended. If you tried this :
void func() {
MyClass myClass;
myClass.theBadFunc();
}
When this returns, you will have a core dump.
Does this resemble the code that you're describing?
struct MyClass {
void f() { delete this; }
};
int main() {
MyClass c;
std::vector<MyClass> v;
v.push_back(c);
c.f();
return 0;
}
The vector is irrelevant. The problem is that delete this calls the destructor for c and frees the memory where c was built. That memory is on the stack, and cannot be freed. Don't ever delete stack objects. The compiler generates code to clean them up when they go out of scope, in this case, at the end of main.
I have a class call grid. The class holds two 2d char arrays for storing a grid...
The class has two functions for creating the memory for the grid and releasing the memory for the grid.
Grid.h
private:
char **gridOne;
char **gridTwo;
Grid.cpp
void Grid::allocateGridMem()
{
_gridOne = new char*[gridRowCount()];
_gridTwo = new char*[gridRowCount()];
for(int i =0; i < gridColumnCount(); ++i){
*(_gridOne + i) = new char[gridColumnCount()];
*(_gridTwo + i) = new char[gridColumnCount()];
}
}
void Grid::dealocateGridMem()
{
if(_gridOne != 0)
{
for(int i =0; i < gridRowCount(); ++i){
delete [] *(_gridOne + i);
}
delete [] _gridOne;
_gridOne = 0;
}
if(_gridTwo != 0)
{
for(int i =0; i < gridRowCount(); i++){
delete [] *(_gridTwo + i);
}
delete [] _gridTwo;
_gridTwo = 0;
}
}
The problem is happening in the deallocation of the memory which I receive the following error.
*** glibc detected *** ./a.out: double free or corruption (out): 0x088c9318 ***
======= Backtrace: =========
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0xb756c591]
/lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0xb756dde8]
/lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0xb7570ecd]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xb775c741]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xb775c79d]
./a.out[0x804a7b9]
./a.out[0x8049cb6]
./a.out[0x804b8f3]
./a.out[0x804c06a]
./a.out[0x804b71d]
./a.out[0x80498eb]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7517bd6]
./a.out[0x8049521]
======= Memory map: ========
08048000-0804f000 r-xp 00000000 08:02 920728 /home/a.out
0804f000-08050000 r--p 00006000 08:02 920728 /home/a.out
08050000-08051000 rw-p 00007000 08:02 920728 /home/a.out
088c7000-088e8000 rw-p 00000000 00:00 0 [heap]
b7300000-b7321000 rw-p 00000000 00:00 0
b7321000-b7400000 ---p 00000000 00:00 0
b7500000-b7501000 rw-p 00000000 00:00 0
b7501000-b7654000 r-xp 00000000 08:02 19796293 /lib/tls/i686/cmov/libc-2.11.1.so
b7654000-b7655000 ---p 00153000 08:02 19796293 /lib/tls/i686/cmov/libc-2.11.1.so
b7655000-b7657000 r--p 00153000 08:02 19796293 /lib/tls/i686/cmov/libc-2.11.1.so
b7657000-b7658000 rw-p 00155000 08:02 19796293 /lib/tls/i686/cmov/libc-2.11.1.so
b7658000-b765b000 rw-p 00000000 00:00 0
b765b000-b7678000 r-xp 00000000 08:02 19791955 /lib/libgcc_s.so.1
b7678000-b7679000 r--p 0001c000 08:02 19791955 /lib/libgcc_s.so.1
b7679000-b767a000 rw-p 0001d000 08:02 19791955 /lib/libgcc_s.so.1
b767a000-b767b000 rw-p 00000000 00:00 0
b767b000-b769f000 r-xp 00000000 08:02 19796301 /lib/tls/i686/cmov/libm-2.11.1.so
b769f000-b76a0000 r--p 00023000 08:02 19796301 /lib/tls/i686/cmov/libm-2.11.1.so
b76a0000-b76a1000 rw-p 00024000 08:02 19796301 /lib/tls/i686/cmov/libm-2.11.1.so
b76a1000-b778a000 r-xp 00000000 08:02 28708531 /usr/lib/libstdc++.so.6.0.13
b778a000-b778b000 ---p 000e9000 08:02 28708531 /usr/lib/libstdc++.so.6.0.13
b778b000-b778f000 r--p 000e9000 08:02 28708531 /usr/lib/libstdc++.so.6.0.13
b778f000-b7790000 rw-p 000ed000 08:02 28708531 /usr/lib/libstdc++.so.6.0.13
b7790000-b7797000 rw-p 00000000 00:00 0
b77a5000-b77a8000 rw-p 00000000 00:00 0
b77a8000-b77a9000 r-xp 00000000 00:00 0 [vdso]
b77a9000-b77c4000 r-xp 00000000 08:02 19791897 /lib/ld-2.11.1.so
b77c4000-b77c5000 r--p 0001a000 08:02 19791897 /lib/ld-2.11.1.so
b77c5000-b77c6000 rw-p 0001b000 08:02 19791897 /lib/ld-2.11.1.so
bf83a000
-bf84f000 rw-p 00000000 00:00 0 [stack]
Aborted
I have checked all my pointers that they are not being changed to something else in execution and that every check and balance one can think of is happening rite. I have been pulling my hair out for the last few hours and still nothing.
I am running this with gcc on ubuntu 10 system.
Should also note that I have changed names etc for the purpose of this post and have only included the code I though to be of value.
EDIT:
Fixed the syntax problem however the original code has this I just typed it out to quickly and did not proof read.
Any help is greatly appreciated and worth a gold star in my book.
I am very much an advanced users of gdb and have used this with this issue but my thinking that it is a problem maybe in external library. I can not see any issues with the memory and how it is scoped just hoping someone has seen something like this. For all purposes this code is fine.
change
for(int i =0; i < gridColumnCount(); ++i){
_gridOne = new char[gridColumnCount()];
_gridTwo = new char[gridColumnCount()];
}
to
for(int i =0; i < gridRowCount(); ++i){
_gridOne[i] = new char[gridColumnCount()];
_gridTwo[i] = new char[gridColumnCount()];
}
Besides, don't do
*(array + i)
but
array[i]
Probably it should be _gridOne[i] instead of _gridOne inside the loop of allocateGridMem. But please, avoid such low-level operations whenever possible and use a high-level component like boost::numeric::ublas::matrix instead.
Does that Grid class of yours have a copy constructor and an assignment operator? Otherwise, if you copy such objects, this error would be what happens.
I suggest you stop doing manual resource management and make Grid a thin two-dimensional wrapper around a std::vector<char> which manages memory.
sudo apt-get install valgrind, run valgrind myprogram and 99% of memory allocation bugs become obvious.
At least one of the problems is that you are assigning the row (should be _gridOne[i]) over the top of the array pointer. Compile with -Werror -Wall -W and many errors like this will become obvious at compile time.
Use the STL and vectors:
class Grid
{
std::vector<std::vector<char> > grid1;
std::vector<std::vector<char> > grid2;
public:
Grid(int col,int row)
: grid1(col, std::vector<char>(row))
, grid2(col, std::vector<char>(row))
{}
};
All done.
If you want to get fancy look at boost Matrix.