I'm trying to copy the contents of tempFile to CacheFile in a thread in safe and
efficient way.
what is myscript takes 1-2 min times to write to a file (ie. tempFile) and after sucessfull .I need to copy to cacheFile so who ever call the function (getStudentDetails) will get serve from cachefile rather than tempFile to avoid the delay(though first call always will be delay for 1-2 min since cacheFIle will empty).
Note: I NEED TO USE FILE NOT PRIMARY STORAGE
Below is my approach it working fine but can it be more efficient and safe?
bool unlock=true;
pthread_mutex_t count_mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t condition_var = PTHREAD_COND_INITIALIZER;
void * runScriptAndUpdateCacheFile ( void * arg )
{
std::cout<<"Inside thread lock"<<endl;
unlock=false; // to avoid multiple call to this thread till it get complete
char tempFile[32]={0};
strncpy(tempFile,"/tmp/smbXXXXXX",14); // tempFile is the main file where my script will dump the output
if( mkstemp(tempFile) < 1)
{
cout<<"fail to create temp file"<<endl;
return (NULL);
}
if (tempFile != NULL)
{
char command[256] = {0};
sprintf(command, " myscript > %s", tempFile);
int status=system(command); // write ouput of script to tempFile
if (status < 0)
{
std::cout << "Error: " << strerror(errno) << '\n';
return(NULL);
}
else
{
if (WIFEXITED(status)) // fetch exit code
{
std::cout << "Program returned normally, exit code " << WEXITSTATUS(status) << '\n';
std::ifstream src(tempFile, std::ios::binary);
std::ofstream CacheFile((char*)arg, std::ios::binary);
Cache << src.rdbuf(); // Copy to cacheFile
}
else
{
std::cout << "Program exited abnormaly\n";
return(NULL);
}
}
}
cout<<" thread is unlock"<<endl;
unlock=true;
}
getStudentDetails can be called multiple times
void getStudentDetails()
{
pthread_t threads;
char CacheFile[32]={0};
strncpy(CacheFile,"/tmp/CacheFile",19);
//Serve from CacheFile rather than orignal file coz this file
//will take 1-2 min to dump in it
std::ifstream fin(CacheFile);
if(unlock) // to avoid multiple call to thread when multile call made to getStudentDetails
// this unlock is made false when it enter to thread and made true when exit
{
pthread_mutex_lock( &count_mutex );
int rc = pthread_create(&threads, NULL, runSmbtree, CacheFile); // pass cache file to thread this file will get updated with
// orignal file create by script.
pthread_mutex_unlock( &count_mutex );
}
std::string line;
while (getline(fin, line))
{
// reading Cachefile and serveing when getStudentDetails called by client
}
}
Valgrind complains with below error
=3535== HEAP SUMMARY:
==3535== in use at exit: 864 bytes in 3 blocks
==3535== total heap usage: 169 allocs, 166 frees, 78,323 bytes allocated
==3535==
==3535== Searching for pointers to 3 not-freed blocks
==3535== Checked 25,373,616 byte
==3535==
==3535== Thread 1:
==3535== 864 bytes in 3 blocks are possibly lost in loss record 1 of 1
==3535== at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3535== by 0x4012E54: allocate_dtv (dl-tls.c:296)
==3535== by 0x4012E54: _dl_allocate_tls (dl-tls.c:460)
==3535== by 0x5359DA0: allocate_stack (allocatestack.c:589)
==3535== by 0x5359DA0: pthread_create##GLIBC_2.2.5 (pthread_create.c:500)
==3535==
==3535== LEAK SUMMARY:
==3535== definitely lost: 0 bytes in 0 blocks
==3535== indirectly lost: 0 bytes in 0 blocks
==3535== possibly lost: 864 bytes in 3 blocks
==3535== still reachable: 0 bytes in 0 blocks
==3535== suppressed: 0 bytes in 0 blocks
==3535==
==3535== ERROR SUMMARY: 3 errors from 2 contexts (suppressed: 0 from 0)
==3535==
==3535== 2 errors in context 1 of 2:
==3535== Thread 2:
==3535== Syscall param open(filename) points to unaddressable byte(s)
==3535== at 0x565B4CD: ??? (syscall-template.S:81)
==3535== by 0x55E9E07: _IO_file_open (fileops.c:228)
==3535== by 0x55E9E07: _IO_file_fopen##GLIBC_2.2.5 (fileops.c:333)
==3535== by 0x55DE2E3: __fopen_internal (iofopen.c:90)
==3535== by 0x4EB29BF: std::__basic_file::open(char const*, std::_Ios_Openmode, int) (in
/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==3535== by 0x4EEAEA9: std::basic_filebuf >::open(char const*, std::_Ios_Openmode) (in
/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==3535== by 0x4EEC747: std::basic_ofstream >::basic_ofstream(char const*,
std::_Ios_Openmode) (in
/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==3535== by 0x40200B: runScriptAndUpdateCacheFile(void*) (in ***)
==3535== by 0x5359181: start_thread (pthread_create.c:312)
==3535== by 0x566A30C: clone (clone.S:111)
==3535== Address 0xffefffba0 is on thread 1's stack
==3535== 384 bytes below stack pointer
==3535==
==3535== ERROR SUMMARY: 3 errors from 2 contexts (suppressed: 0 from 0)
Related
I use doubly linked list to implement Deque in C++.
Destructor:
Deque::~Deque()
{
while (this->left_p)
{
node *temp = this->left_p;
this->left_p = this->left_p->next;
delete temp;
}
this->right_p = NULL;
}
when i use valgrind --leak-check=full ./a.out to check memory leak just to test my destructor` I got the following output:
==2636==
==2636== HEAP SUMMARY:
==2636== in use at exit: 72,704 bytes in 1 blocks
==2636== total heap usage: 1,003 allocs, 1,002 frees, 97,760 bytes allocated
==2636==
==2636== 72,704 bytes in 1 blocks are still reachable in loss record 1 of 1
==2636== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2636== by 0x4EC3EFF: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==2636== by 0x40106B9: call_init.part.0 (dl-init.c:72)
==2636== by 0x40107CA: call_init (dl-init.c:30)
==2636== by 0x40107CA: _dl_init (dl-init.c:120)
==2636== by 0x4000C69: ??? (in /lib/x86_64-linux-gnu/ld-2.23.so)
==2636==
==2636== LEAK SUMMARY:
==2636== definitely lost: 0 bytes in 0 blocks
==2636== indirectly lost: 0 bytes in 0 blocks
==2636== possibly lost: 0 bytes in 0 blocks
==2636== still reachable: 72,704 bytes in 1 blocks
==2636== suppressed: 0 bytes in 0 blocks
==2636==
==2636== For counts of detected and suppressed errors, rerun with: -v
==2636== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
I can't figure out why there is still ONE out of 1003 allocs not being free.
Why do i have one memory leak? what is wrong with my destructor?
Test code here:
/* Deque Test Program 6 */
#include <cstring>
#include <iostream>
#include "Deque.h"
using namespace std ;
int main (int argc, char * const argv[]) {
cout << "\n\nDeque Class Test Program 6 - START\n\n";
// Make a Deque
Deque * dq1 = new Deque();
for( int i = 0 ; i<1 ; i++ ){
dq1->push_left(1);
// dq1->display();
}
cout << "Size=" << dq1->size() << endl ;
// The destructor should delete all the nodes.
delete dq1 ;
cout << "\n\nDeque Class Test Program 6 - DONE\n\n";
return 0;
}
edit: remove implementation code.
Essentially, it's not your code's fault, it's valgrind's.
Check this other question that has had the same problem:
Valgrind: Memory still reachable with trivial program using <iostream>
Quoting from the post:
First of all: relax, it's probably not a bug, but a feature. Many implementations of the C++ standard libraries use their own memory pool allocators. Memory for quite a number of destructed objects is not immediately freed and given back to the OS, but kept in the pool(s) for later re-use. The fact that the pools are not freed at the exit of the program cause Valgrind to report this memory as still reachable. The behaviour not to free pools at the exit could be called a bug of the library though.
Hope that helps :)
The memory leak reported by valgrind does not appear to be in your code:
==2636== 72,704 bytes in 1 blocks are still reachable in loss record 1 of 1
==2636== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2636== by 0x4EC3EFF: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==2636== by 0x40106B9: call_init.part.0 (dl-init.c:72)
==2636== by 0x40107CA: call_init (dl-init.c:30)
==2636== by 0x40107CA: _dl_init (dl-init.c:120)
This appears to be a heap allocation from within a constructor of a global object. (In theory, it could still come from your code if operator new is called as a tail call, so that it does not show up in the backtrace, but I don't see such an object declaration in your cdoe.)
It is also not an actual leak, it is just some data allocated on the heap at program start. If you install debugging information for libstdc++, then you might get a hint of what is actually being allocated. Then you could also set a breakpoint on call_init and step through the early process initialization, to see the constructors that are called.
I have a thread system that keeps allways opened a certain amount of threads. The problem with this program si that it is having some memory leak problems. I am starting the program at around 8MB (RAM memory) and in 1 day i see it at 800MB. I used valgrind to track down this issue and i found the following while running this command time valgrind --leak-check=yes ./test2; :
I get this when i use 1000 treads and it is not even finishing.
Thread 157: status = VgTs_WaitSys
==16172== at 0x4160F8B: ??? (in /lib/libc-2.5.so)
==16172== by 0x40FF58F: _IO_file_underflow##GLIBC_2.1 (in /lib/libc-2.5.so)
==16172== by 0x40FFC8A: _IO_default_uflow (in /lib/libc-2.5.so)
==16172== by 0x410100C: __uflow (in /lib/libc-2.5.so)
==16172== by 0x40F48B5: _IO_getline_info (in /lib/libc-2.5.so)
==16172== by 0x40F4800: _IO_getline (in /lib/libc-2.5.so)
==16172== by 0x40F3779: fgets (in /lib/libc-2.5.so)
==16172== by 0x80496CF: exec(char const*) (in /home/pthread_test)
==16172== by 0x80498C2: thread_test(void*) (in /home/pthread_test)
==16172== by 0x40A911: start_thread (in /lib/libpthread-2.5.so)
==16172== by 0x417047D: clone (in /lib/libc-2.5.so)
Thread 158: status = VgTs_WaitSys
==16172== at 0x4160F8B: ??? (in /lib/libc-2.5.so)
==16172== by 0x40FF58F: _IO_file_underflow##GLIBC_2.1 (in /lib/libc-2.5.so)
==16172== by 0x40FFC8A: _IO_default_uflow (in /lib/libc-2.5.so)
==16172== by 0x410100C: __uflow (in /lib/libc-2.5.so)
==16172== by 0x40F48B5: _IO_getline_info (in /lib/libc-2.5.so)
==16172== by 0x40F4800: _IO_getline (in /lib/libc-2.5.so)
==16172== by 0x40F3779: fgets (in /lib/libc-2.5.so)
==16172== by 0x80496CF: exec(char const*) (in /home/pthread_test)
==16172== by 0x80498C2: thread_test(void*) (in /home/pthread_test)
==16172== by 0x40A911: start_thread (in /lib/libpthread-2.5.so)
==16172== by 0x417047D: clone (in /lib/libc-2.5.so)
When using just ./test2 (running without valgrind), absolutely all threads finishes and i can see in the debug.log that every thread finishes
tail debug.log
END:1972:191
END:1984:191
END:1992:191
END:1974:191
END:1986:191
END:1979:191
END:1993:191
END:1983:191
END:1999:191 <-- even thread 1999 finished
END:1991:191
When i am doing with 250 threads i get this at the end of the program
THREAD [1993] ending
THREAD [1983] ending
THREAD [1999] ending
THREAD [1991] ending
==23152== Thread 25:
==23152== Invalid free() / delete / delete[]
==23152== at 0x400579D: free (vg_replace_malloc.c:325)
==23152== by 0x3B59DD: ??? (in /lib/libc-2.5.so)
==23152== by 0x3B5556: ??? (in /lib/libc-2.5.so)
==23152== by 0x40013B2: _vgnU_freeres (vg_preloaded.c:62)
==23152== by 0x333A73: _Exit (in /lib/libc-2.5.so)
==23152== by 0x40AAF8: start_thread (in /lib/libpthread-2.5.so)
==23152== by 0x37547D: clone (in /lib/libc-2.5.so)
==23152== Address 0x409b2c8 is not stack'd, malloc'd or (recently) free'd
==23152==
==23152==
==23152== HEAP SUMMARY:
==23152== in use at exit: 160,850 bytes in 12,002 blocks
==23152== total heap usage: 20,151 allocs, 8,150 frees, 2,640,139 bytes allocated
==23152==
==23152== Thread 1:
==23152== 152 bytes in 1 blocks are possibly lost in loss record 2 of 8
==23152== at 0x4004EC2: calloc (vg_replace_malloc.c:418)
==23152== by 0x291FF9: _dl_allocate_tls (in /lib/ld-2.5.so)
==23152== by 0x40B026: pthread_create##GLIBC_2.1 (in /lib/libpthread-2.5.so)
==23152== by 0x80493DF: main (in /home/freelancer1/pthreads_test/test2)
==23152==
==23152== 20,000 bytes in 2,000 blocks are definitely lost in loss record 3 of 8
==23152== at 0x4005B83: malloc (vg_replace_malloc.c:195)
==23152== by 0x3105DF: strdup (in /lib/libc-2.5.so)
==23152== by 0x8049C56: file_put_contents(char*, char*, char*) (in /home/freelancer1/pthreads_test/test2)
==23152== by 0x804981D: thread_test(void*) (in /home/freelancer1/pthreads_test/test2)
==23152== by 0x40A911: start_thread (in /lib/libpthread-2.5.so)
==23152== by 0x37547D: clone (in /lib/libc-2.5.so)
==23152==
==23152== 20,000 bytes in 2,000 blocks are definitely lost in loss record 4 of 8
==23152== at 0x4005B83: malloc (vg_replace_malloc.c:195)
==23152== by 0x3105DF: strdup (in /lib/libc-2.5.so)
==23152== by 0x8049C56: file_put_contents(char*, char*, char*) (in /home/freelancer1/pthreads_test/test2)
==23152== by 0x804944D: main (in /home/freelancer1/pthreads_test/test2)
==23152==
==23152== 20,000 bytes in 2,000 blocks are definitely lost in loss record 5 of 8
==23152== at 0x4005B83: malloc (vg_replace_malloc.c:195)
==23152== by 0x3105DF: strdup (in /lib/libc-2.5.so)
==23152== by 0x8049C56: file_put_contents(char*, char*, char*) (in /home/freelancer1/pthreads_test/test2)
==23152== by 0x8049950: thread_test(void*) (in /home/freelancer1/pthreads_test/test2)
==23152== by 0x40A911: start_thread (in /lib/libpthread-2.5.so)
==23152== by 0x37547D: clone (in /lib/libc-2.5.so)
==23152==
==23152== 26,890 bytes in 2,000 blocks are definitely lost in loss record 6 of 8
==23152== at 0x4005B83: malloc (vg_replace_malloc.c:195)
==23152== by 0x3105DF: strdup (in /lib/libc-2.5.so)
==23152== by 0x8049C64: file_put_contents(char*, char*, char*) (in /home/freelancer1/pthreads_test/test2)
==23152== by 0x8049950: thread_test(void*) (in /home/freelancer1/pthreads_test/test2)
==23152== by 0x40A911: start_thread (in /lib/libpthread-2.5.so)
==23152== by 0x37547D: clone (in /lib/libc-2.5.so)
==23152==
==23152== 30,890 bytes in 2,000 blocks are definitely lost in loss record 7 of 8
==23152== at 0x4005B83: malloc (vg_replace_malloc.c:195)
==23152== by 0x3105DF: strdup (in /lib/libc-2.5.so)
==23152== by 0x8049C64: file_put_contents(char*, char*, char*) (in /home/freelancer1/pthreads_test/test2)
==23152== by 0x804981D: thread_test(void*) (in /home/freelancer1/pthreads_test/test2)
==23152== by 0x40A911: start_thread (in /lib/libpthread-2.5.so)
==23152== by 0x37547D: clone (in /lib/libc-2.5.so)
==23152==
==23152== 42,890 bytes in 2,000 blocks are definitely lost in loss record 8 of 8
==23152== at 0x4005B83: malloc (vg_replace_malloc.c:195)
==23152== by 0x3105DF: strdup (in /lib/libc-2.5.so)
==23152== by 0x8049C64: file_put_contents(char*, char*, char*) (in /home/freelancer1/pthreads_test/test2)
==23152== by 0x804944D: main (in /home/freelancer1/pthreads_test/test2)
==23152==
==23152== LEAK SUMMARY:
==23152== definitely lost: 160,670 bytes in 12,000 blocks
==23152== indirectly lost: 0 bytes in 0 blocks
==23152== possibly lost: 152 bytes in 1 blocks
==23152== still reachable: 28 bytes in 1 blocks
==23152== suppressed: 0 bytes in 0 blocks
==23152== Reachable blocks (those to which a pointer was found) are not shown.
==23152== To see them, rerun with: --leak-check=full --show-reachable=yes
==23152==
==23152== For counts of detected and suppressed errors, rerun with: -v
==23152== ERROR SUMMARY: 8 errors from 8 contexts (suppressed: 36 from 8)
Here is the source code :
#include <iostream>
#include <cstdlib>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <cstring>
#include <stdexcept>
using namespace std;
void file_put_contents(char* filename, char* content, char* mode)
{
char* filename2;
filename2 = strdup(filename);
char* content2;
content2 = strdup(content);
char* writemode = "a";
if(strstr(mode,"FILE_APPEND"))
{
writemode = "a";
}
else if(strstr(mode,"FILE_OVERWRITE"))
{
writemode = "w";
}
else
{
writemode = "w";
}
FILE * file;
file = fopen(filename2,writemode);
fprintf(file,content2);
fclose(file);
}
int NUM_THREADS = 2000;
int MAX_THREADS = 500;
int THREADSTACK = 65536;
struct thread_struct{
int arg1;
int arg2;
};
pthread_mutex_t mutex_;
pthread_mutex_t mutex2_;
static unsigned int thread_count = 0;
string exec(const char* cmd)
{
int DEBUG=0;
char buffer[5000];
string result = "";
FILE* pipe = popen(cmd, "r");
if (!pipe && DEBUG) throw runtime_error("popen() failed!");
try
{
while (!feof(pipe))
{
if (fgets(buffer, 128, pipe) != NULL)
{
result += buffer;
}
}
}
catch(...)
{
pclose(pipe);
throw;
}
pclose(pipe);
return result;
}
void *thread_test(void *arguments)
{
pthread_mutex_lock(&mutex_);
thread_count++;
pthread_mutex_unlock(&mutex_);
struct thread_struct *args = (thread_struct*)arguments;
int thread_id = (int) args->arg1;
char debug_var[100];
// UGLY DEBUG
memset(debug_var,0,sizeof(debug_var)); sprintf(debug_var,"BEGIN:%d:%d\n",thread_id,__LINE__); file_put_contents("debug.log",debug_var,"FILE_APPEND");
int random_sleep;
random_sleep = rand() % 10 + 3;
char command[100];
memset(command,0,sizeof(command));
sprintf(command,"sleep %d",random_sleep);
exec(command);
pthread_mutex_lock(&mutex_);
thread_count--;
pthread_mutex_unlock(&mutex_);
// UGLY DEBUG
memset(debug_var,0,sizeof(debug_var)); sprintf(debug_var,"END:%d:%d\n",thread_id,__LINE__); file_put_contents("debug.log",debug_var,"FILE_APPEND");
printf("THREAD [%d] ending\n",thread_id);
pthread_exit(NULL);
}
int main()
{
// pthread_t threads[NUM_THREADS];
int rc;
printf("Beginning ...");
usleep(1000000);
srand ((unsigned)time(NULL));
unsigned int thread_count_now = 0;
pthread_attr_t attrs;
pthread_attr_init(&attrs);
pthread_attr_setstacksize(&attrs, THREADSTACK);
pthread_attr_setdetachstate(&attrs, PTHREAD_CREATE_DETACHED);
pthread_mutex_init(&mutex2_, NULL);
for(int i=0; i < NUM_THREADS; i++ )
{
create_thread:
pthread_mutex_lock(&mutex_);
thread_count_now = thread_count;
pthread_mutex_unlock(&mutex_);
struct thread_struct struct1;
struct1.arg1 = i;
struct1.arg2 = 999;
pthread_t temp_thread;
printf("CREATE thread [%d]\n",i);
rc = pthread_create(&temp_thread, &attrs, &thread_test, (void *)&struct1);
char debug_var[100];
memset(debug_var,0,sizeof(debug_var)); sprintf(debug_var,"THREAD:%d:RC=%d:%d\n",i,rc,__LINE__); file_put_contents("debug.log",debug_var,"FILE_APPEND");
if (rc)
{
printf("Unable to create thread %d\n",rc);
sleep(1);
pthread_detach(temp_thread);
goto create_thread;
}
usleep(10);
while(true)
{
if(thread_count >= MAX_THREADS)
{
// printf("Thread POOL full %d of %d\n",thread_count,MAX_THREADS);
}
else
{
break;
}
usleep(10);
}
}
pthread_attr_destroy(&attrs);
pthread_mutex_destroy(&mutex2_);
printf("Proccess completed!\n");
pthread_exit(NULL);
return 1;
}
Did i forgot somewhere to free memory?
I don't quite understand those logs perfectly, can you give me any clue? What is wrong? What do i have to change?
Thank you.
The following code is to move the "non-overlapping" TablePath from vector v to vector u. I am encountering segmentaion fault at the line "u.push_back(*it1);". I didn't copy the object(but instead only copy the pointer of the object) so I believe the problem doesn't lie in copying constructor. Can you give some hints on why segfault is occuring ?
#include <iostream>
#include <vector>
using namespace std;
class TablePath
{
private:
int source;
int destination;
public:
TablePath(int,int);
~TablePath();
int overlap(TablePath*);
void toString();
TablePath(const TablePath& that) : source(that.source), destination(that.destination)
{
}
};
TablePath::TablePath(int source=0,int destination=0)
{
this->source = source;
this->destination = destination;
}
int TablePath::overlap(TablePath* thatTablePath)
{
if (this->source >= thatTablePath->source and this->source <= thatTablePath->destination)
return 1;
else if (this->destination >= thatTablePath->source and this->destination <= thatTablePath->destination)
return 1;
else if (thatTablePath->source >= this->source and thatTablePath->source <= this->destination)
return 1;
else if (thatTablePath->destination >= this->source and thatTablePath->destination <= this->destination)
return 1;
else
return 0;
}
void TablePath::toString()
{
cout << this->source << " " << this->destination << endl;
}
int main()
{
int numofTests;
cin >> numofTests;
while(numofTests > 0)
{
int numofMoves;
vector<TablePath *> v;
cin >> numofMoves;
for (int i=0;i<numofMoves;i++)
{
int source,destination;
cin >> source >> destination;
TablePath* MyTablePath = new TablePath(source,destination);
v.push_back(MyTablePath);
}
vector<TablePath *> u;
vector<TablePath *>::iterator it1 = v.begin();
u.push_back(*it1);
v.erase(v.begin());
for(vector<TablePath *>::iterator it1 = v.begin(); it1 != v.end(); ++it1)
{
for(vector<TablePath *>::iterator it2 = u.begin(); it2 != u.end(); ++it2)
{
if ((*it1)->overlap((*it2)))
{
u.push_back(*it1);
}
}
}
cout << u.size() * 10;
v.erase(v.begin(),v.end());
u.erase(u.begin(),u.end());
numofTests--;
}
}
The following is the output I get from valgrind:
frank#frank-vm:~$ valgrind --tool=memcheck ./tablepath
==6172== Memcheck, a memory error detector
==6172== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==6172== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==6172== Command: ./tablepath
==6172==
1
3
10
20
15 30
20 50
==6172== Invalid read of size 4
==6172== at 0x8048BB0: main (in /home/frank/tablepath)
==6172== Address 0x4320184 is 0 bytes after a block of size 4 free'd
==6172== at 0x402ACFC: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==6172== by 0x8049616: __gnu_cxx::new_allocator<TablePath*>::deallocate(TablePath**, unsigned int) (in /home/frank/tablepath)
==6172== by 0x80493E8: std::_Vector_base<TablePath*, std::allocator<TablePath*> >::_M_deallocate(TablePath**, unsigned int) (in /home/frank/tablepath)
==6172== by 0x8049230: std::vector<TablePath*, std::allocator<TablePath*> >::_M_insert_aux(__gnu_cxx::__normal_iterator<TablePath**, std::vector<TablePath*, std::allocator<TablePath*> > >, TablePath* const&) (in /home/frank/tablepath)
==6172== by 0x8048E00: std::vector<TablePath*, std::allocator<TablePath*> >::push_back(TablePath* const&) (in /home/frank/tablepath)
==6172== by 0x8048BED: main (in /home/frank/tablepath)
==6172==
==6172== Invalid read of size 4
==6172== at 0x8048965: TablePath::overlap(TablePath*) (in /home/frank/tablepath)
==6172== by 0x8048BCA: main (in /home/frank/tablepath)
==6172== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==6172==
==6172==
==6172== Process terminating with default action of signal 11 (SIGSEGV)
==6172== Access not within mapped region at address 0x0
==6172== at 0x8048965: TablePath::overlap(TablePath*) (in /home/frank/tablepath)
==6172== by 0x8048BCA: main (in /home/frank/tablepath)
==6172== If you believe this happened as a result of a stack
==6172== overflow in your program's main thread (unlikely but
==6172== possible), you can try to increase the size of the
==6172== main thread stack using the --main-stacksize= flag.
==6172== The main thread stack size used in this run was 8388608.
==6172==
==6172== HEAP SUMMARY:
==6172== in use at exit: 48 bytes in 5 blocks
==6172== total heap usage: 8 allocs, 3 frees, 64 bytes allocated
==6172==
==6172== LEAK SUMMARY:
==6172== definitely lost: 0 bytes in 0 blocks
==6172== indirectly lost: 0 bytes in 0 blocks
==6172== possibly lost: 0 bytes in 0 blocks
==6172== still reachable: 48 bytes in 5 blocks
==6172== suppressed: 0 bytes in 0 blocks
==6172== Rerun with --leak-check=full to see details of leaked memory
==6172==
==6172== For counts of detected and suppressed errors, rerun with: -v
==6172== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)
std::vector<T>::push_back is (usually) allowed to invalidate all iterators associated with the vector object. So after you do:
u.push_back(*it1);
it is not safe to continue the inner for loop with ++it2.
You could use an index for the inner loop instead. Or, break out of the inner loop right after doing the push_back, if you don't really want multiple copies of the same TablePath* pointer in u.
Thats the Store Credit problem on Google code jam.
https://code.google.com/codejam/contest/351101/dashboard#s=p0
My code gives out a SIGSEGV after running the large test. But the answer is correct!
#include <cstdio>
#include <algorithm>
#include <vector>
using namespace std;
int ps[1000]={0};
vector<int> indice[1000];
int main() {
int cases; scanf("%d", &cases);
for(int j=1;j<=cases;j++) {
printf("Case #%d: ", j);
int c, is; scanf("%d%d", &c, &is);
for(int i=0;i<=c;i++) ps[i]=0;
for(int i=0;i<=c;i++) indice[i].clear();
for (int i = 0; i < is; i++) {
int it; scanf("%d", &it);
indice[it].push_back(i+1);
ps[it]=1;
if (c-it>0&&ps[c-it]) {
int a, b;
a = indice[it][0];
b = indice[c-it][0];
if(c==2*it&&indice[it].size()>1) {
b=indice[it][1];
}
if (a!=b) {
printf("%d %d\n", min(a,b),max(a,b));
}
}
}
}
return 0;
}
So I use valgrind to find out what's going on .. but it seems that it's not my problem.
==17599== Invalid free() / delete / delete[] / realloc()
==17599== at 0x4C2A4BC: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==17599== by 0x401669: __gnu_cxx::new_allocator<int>::deallocate(int*, unsigned long) (new_allocator.h:98)
==17599== by 0x4013CD: std::_Vector_base<int, std::allocator<int> >::_M_deallocate(int*, unsigned long) (stl_vector.h:156)
==17599== by 0x400F60: std::_Vector_base<int, std::allocator<int> >::~_Vector_base() (stl_vector.h:142)
==17599== by 0x400D8D: std::vector<int, std::allocator<int> >::~vector() (stl_vector.h:351)
==17599== by 0x400C48: __tcf_0 (a.cpp:6)
==17599== by 0x5383900: __run_exit_handlers (exit.c:78)
==17599== by 0x5383984: exit (exit.c:100)
==17599== by 0x5369773: (below main) (libc-start.c:258)
==17599== Address 0x1 is not stack'd, malloc'd or (recently) free'd
==17599==
==17599==
==17599== HEAP SUMMARY:
==17599== in use at exit: 128 bytes in 1 blocks
==17599== total heap usage: 4,527 allocs, 4,527 frees, 113,664 bytes allocated
==17599==
==17599== LEAK SUMMARY:
==17599== definitely lost: 0 bytes in 0 blocks
==17599== indirectly lost: 0 bytes in 0 blocks
==17599== possibly lost: 0 bytes in 0 blocks
==17599== still reachable: 128 bytes in 1 blocks
==17599== suppressed: 0 bytes in 0 blocks
==17599== Rerun with --leak-check=full to see details of leaked memory
==17599==
==17599== For counts of detected and suppressed errors, rerun with: -v
==17599== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 2 from 2)
I'm so confused... Could anybody tell me what's going on? I'm a newbie of C++ ..
Thank you very much.
As I understand valgrind, it can't detect that you are writing outside of the bounds of arrays that are staticly allocated. So let's allocate them on the heap.
vector<int> *indice = new vector<int>[1000];
int *ps = new int[1000];
Then, you will see errors come out of valgrind. Including:
==7168== Invalid read of size 8
==7168== at 0x4008D6: main (stl_vector.h:735)
==7168== Address 0x4c39e10 is 8 bytes after a block of size 24,008 alloc'd
==7168== at 0x4A07152: operator new[](unsigned long) (vg_replace_malloc.c:363)
==7168== by 0x400791: global constructors keyed to indice (foo.cc:6)
==7168== by 0x400C35: ??? (in /tmp/foo)
==7168== by 0x4005F2: ??? (in /tmp/foo)
==7168==
==7168== Invalid read of size 8
==7168== at 0x4008DA: main (stl_vector.h:735)
==7168== Address 0x4c39e18 is 16 bytes after a block of size 24,008 alloc'd
==7168== at 0x4A07152: operator new[](unsigned long) (vg_replace_malloc.c:363)
==7168== by 0x400791: global constructors keyed to indice (foo.cc:6)
==7168== by 0x400C35: ??? (in /tmp/foo)
==7168== by 0x4005F2: ??? (in /tmp/foo)
==7168==
And using gdb, I can see that the SIGSEGV occurs when you access indice[1433], which is outside of the bounds of indice.
I also imagine that your actual issue is that for the large dataset, the variables bounds are listed as:
N = 50
3 ≤ I ≤ 2000
Are you sure you shouldn't be allocating 2001 elements, instead of 1000?
On my createFileAddRecordAndRead() function I create an instance of a FixedLengthRecordFile object (which handles file I/O), and then use this to write a record into the file, and then read it back again.
FixedLengthRecordFile file;
// The following opens the file in "wb+" mode because
// in order to write a record I need to read something else first
file.create("fixedfile.txt",record.getRecordSize(),1);
file.write(0,record); // Write the record in the offset 0
file.close();
file.open("fixedfile.txt","rb");
file.read(0,readRecord); // Read record 0 and store it in a variable
But it doesn't work (actually, it does work, because the record gets written and read back, but valgrind complains). However, if I perform the task of writing & reading separately (i.e. with two instances - one to write, one to read), valgrind doesn't complain.
This is some of the code of the FixedLengthRecordFile class...
int FixedLengthRecordFile :: create (std::string fileName, int recSize, int p_maxRecs)
{
// Initialize file attributes
recordSize = recSize;
maxRecs = p_maxRecs;
file = fopen(fileName.c_str(),"wb+");
if (file == NULL)
return RES_ERROR;
if (initFileHeader() == RES_OK)
{
if (initEmptyRecords() == RES_OK)
return RES_OK;
}
return RES_ERROR;
}
int FixedLengthRecordFile :: open (std::string fileName, std::string mode)
{
file = fopen(fileName.c_str(),mode.c_str());
if (file == NULL) {
return RES_ERROR;
}
struct header head;
if (getFileHeader(head) == RES_ERROR) {
return RES_ERROR;
}
recordSize = head.h_recordSize;
maxRecs = head.h_maxRecords;
return RES_OK;
}
int FixedLengthRecordFile :: close ()
{
int result = fclose(file);
if (result == 0) {
std::cout << "File closed. " << std::endl;
return RES_OK;
}
return RES_ERROR;
}
int FixedLengthRecordFile :: write (const int numRec, const FixedLengthFieldsRecord & rec)
{
int positioning = seek (numRec);
if (positioning == RES_RECORD_DOESNT_EXIST || positioning == RES_ERROR)
return RES_ERROR;
char flag;
int res = fread(&flag,sizeof(flag),1,file);
if (res != 1) {
std::cout << "Couldn't read record flag of " << numRec << std::endl;
return RES_ERROR;
}
if (flag == 'O') // "occupied"
return RES_RECORD_EXISTS;
seek(numRec);
return rec.write(file);
}
int FixedLengthRecordFile :: read (int numRec, FixedLengthFieldsRecord & rec)
{
int positioning = seek (numRec);
if (positioning == RES_RECORD_DOESNT_EXIST)
return RES_RECORD_DOESNT_EXIST;
char flag;
int res = fread(&flag,sizeof(char),1,file);
if (res != 1) {
std::cout << "Couldn't read record flag of " << numRec << std::endl;
return RES_ERROR;
}
if (flag != 'O')
return RES_RECORD_DOESNT_EXIST;
return rec.read(file);
}
This is the relevant code from FixedLengthFieldsRecord class...
int FixedLengthFieldsRecord :: read (FILE* file)
{
int res = fread(&record,recordSize,1,file);
if (res != 1) {
std::cout << "Couldn't read record. " << std::endl;
return RES_ERROR;
}
std::cout << "Record read successfully! " << std::endl;
return RES_OK;
}
int FixedLengthFieldsRecord :: write (FILE* file) const
{
char flag = 'O';
int res = fwrite(&flag,sizeof(flag),1,file);
if (res != 1) {
std::cout << "Couldn't mark record as occupied. " << std::endl;
return RES_ERROR;
}
res = fwrite(&record,recordSize,1,file);
if (res != 1) {
std::cout << "Couldn't write record. " << std::endl;
return RES_ERROR;
}
std::cout << "Record written successfully! " << std::endl;
return RES_OK;
}
... and this is the output from valgrind:
X#notebook:~/Desktop/FixedRecordFile$ valgrind --leak-check=full --track-origins=yes --show-reachable=yes ./flrfile
Record written successfully!
==20368== Syscall param write(buf) points to uninitialised byte(s)
==20368== at 0x422A443: __write_nocancel (syscall-template.S:82)
==20368== by 0x41BCBA4: _IO_file_write##GLIBC_2.1 (fileops.c:1289)
==20368== by 0x41BCA83: new_do_write (fileops.c:543)
==20368== by 0x41BE0FD: _IO_do_write##GLIBC_2.1 (fileops.c:516)
==20368== by 0x8049DED: createFileAddRecordAndRead() (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368== by 0x8049FE3: main (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368== Address 0x4035021 is not stack'd, malloc'd or (recently) free'd
==20368== Uninitialised value was created by a stack allocation
==20368== at 0x8049C97: createFileAddRecordAndRead() (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368==
File closed.
Record read successfully!
==20368== Invalid read of size 4
==20368== at 0x41B131C: fclose##GLIBC_2.1 (iofclose.c:60)
==20368== by 0x8049F09: createFileAddRecordAndRead() (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368== by 0x8049FE3: main (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368== Address 0x8 is not stack'd, malloc'd or (recently) free'd
==20368==
==20368==
==20368== Process terminating with default action of signal 11 (SIGSEGV)
==20368== Access not within mapped region at address 0x8
==20368== at 0x41B131C: fclose##GLIBC_2.1 (iofclose.c:60)
==20368== by 0x8049F09: createFileAddRecordAndRead() (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368== by 0x8049FE3: main (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368== If you believe this happened as a result of a stack
==20368== overflow in your program's main thread (unlikely but
==20368== possible), you can try to increase the size of the
==20368== main thread stack using the --main-stacksize= flag.
==20368== The main thread stack size used in this run was 8388608.
==20368==
==20368== HEAP SUMMARY:
==20368== in use at exit: 478 bytes in 5 blocks
==20368== total heap usage: 22 allocs, 17 frees, 1,112 bytes allocated
==20368==
==20368== 12 bytes in 1 blocks are still reachable in loss record 1 of 5
==20368== at 0x402A5E6: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==20368== by 0x8048D2E: FixedLengthFieldsRecord::FixedLengthFieldsRecord(int, int) (FixedLengthFieldsRecord.cpp:44)
==20368== by 0x8049CDF: createFileAddRecordAndRead() (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368== by 0x8049FE3: main (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368==
==20368== 12 bytes in 1 blocks are definitely lost in loss record 2 of 5
==20368== at 0x402A5E6: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==20368== by 0x8048D2E: FixedLengthFieldsRecord::FixedLengthFieldsRecord(int, int) (FixedLengthFieldsRecord.cpp:44)
==20368== by 0x8049E9D: createFileAddRecordAndRead() (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368== by 0x8049FE3: main (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368==
==20368== 51 bytes in 1 blocks are still reachable in loss record 3 of 5
==20368== at 0x402A5E6: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==20368== by 0x8048D14: FixedLengthFieldsRecord::FixedLengthFieldsRecord(int, int) (FixedLengthFieldsRecord.cpp:43)
==20368== by 0x8049CDF: createFileAddRecordAndRead() (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368== by 0x8049FE3: main (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368==
==20368== 51 bytes in 1 blocks are definitely lost in loss record 4 of 5
==20368== at 0x402A5E6: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==20368== by 0x8048D14: FixedLengthFieldsRecord::FixedLengthFieldsRecord(int, int) (FixedLengthFieldsRecord.cpp:43)
==20368== by 0x8049E9D: createFileAddRecordAndRead() (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368== by 0x8049FE3: main (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368==
==20368== 352 bytes in 1 blocks are still reachable in loss record 5 of 5
==20368== at 0x402BE68: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==20368== by 0x41B1D17: __fopen_internal (iofopen.c:76)
==20368== by 0x8049E53: createFileAddRecordAndRead() (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368== by 0x8049FE3: main (in /home/X/Desktop/FixedRecordFile/flrfile)
==20368==
==20368== LEAK SUMMARY:
==20368== definitely lost: 63 bytes in 2 blocks
==20368== indirectly lost: 0 bytes in 0 blocks
==20368== possibly lost: 0 bytes in 0 blocks
==20368== still reachable: 415 bytes in 3 blocks
==20368== suppressed: 0 bytes in 0 blocks
==20368==
==20368== For counts of detected and suppressed errors, rerun with: -v
==20368== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)
Your error is in the pointer use.
res = fwrite(&record,recordSize,1,file);
should be
res = fwrite(record,recordSize,1,file);
because record is already a pointer.
Similarly
int res = fread(&record,recordSize,1,file);
should be
int res = fread(record,recordSize,1,file);
Might be worth renaming that variable so it's clear it's a pointer. When reading the code I initially assumed it was some kind of struct, so I didn't spot the error.