I have this piece of code which iterates over a directory, gets the filenames and pushes them in a vector.
std::vector<const char*> afp_filenames;
if ((dir = opendir (path.c_str())) != NULL) {
while ((ent = readdir (dir)) != NULL) {
if ( (ent->d_name[0] == '.') ) continue;
afp_filenames.push_back((const char*)ent->d_name);
cout <<afp_filenames[i]<<"\n";
i++;
}
}
cout prints the correct filenames. The issue comes afterwards when I print again the values stored in the vector. I get the same result when using an iterator.
for (j = 0; j < i; j++) {
cout <<afp_filenames[j]<<"\n";
}
Here is a sample of what I get in the console.
for the first loop:
afp_PortaAPorta_5_318_383.dat
afp_PortaAPorta_5_407_663.dat
afp_PortaAPorta_5_485_510.dat
afp_781_5_472_371.dat
afp_781_5_115_617.dat
afp_781_5_358_509.dat
afp_781_5_276_348.dat
and for the second one:
2.dat
.dat
afpò
9.dat
2.dat
5_356_572.dat
5_291_435.dat
afp_781_5_243_558.dat
afp_PortaAPorta_5_352_716.dat
afp_781_5_111_541.dat
You are storing const char *s in the vector but the pointers are no longer valid by the time you get to the second loop and so cout is trying to print out whatever is left in the memory they were pointing to.
From readdir:
On success, readdir() returns a pointer to a dirent structure. (This structure may be statically allocated; do not attempt to free(3) it.)
The lifetime of the struct returned by the system call seems to be managed by the kernel C library.
You are storing const char * pointers pointing to memory returned by readdir.
Per man 3 readdir:
The data returned by readdir() may be overwritten by subsequent calls to readdir() for the same directory stream.
Therefore, your ent->d_name entries are being overwritten each loop.
Use a vector<std::string> instead so that you store copies of the string (as an added benefit, you won't have to worry about allocating/deallocating the strings since it will be handled for you).
Probably you just need to copy the string as the pointer you get from d_name may end up pointing to garbage or something else some time later. Make a vector of std::strings and push strings into it.
Related
I'm putting all file names (in this case all files in the folder /Windows/Fonts/) into the char File_Names. Menawhile the names are rendered as well, but after I have put everything in the char (when while ends), I try to access it and use one of the names in something else (this happens after while), but every single pointer I try only gives the last file from the folder (this means, it only has the last pointer). How do I keep all different file names and can use them after while?
Here is a fragment from my program:
//-------------------- List files start
int File_Name_Y = Font_Window_Y+35-Mouse_Scroll*10;
char *File_Names[4096];
struct dirent *entry;
DIR *dir = opendir("/Windows/Fonts");
int t = 1;
while ((entry = readdir(dir)) != NULL)
{
t += 1;
File_Names[t] = entry->d_name;
if (File_Name_Y <= Font_Window_Y+H-65 && File_Name_Y >= Font_Window_Y+30)
{
draw_text (Font_Window_X+15, File_Name_Y, File_Names[t], {1, 1, 1}, 1.5);
}
File_Name_Y += 20;
}
closedir(dir);
//------------------ List files end
if (use_button (SelectFontButton))
{
if (Mouse_Pressed)
{
Mouse_Pressed = 0;
char Font_Path[1024];
strcpy (Font_Path, "/Windows/Fonts/");
strcat(Font_Path, File_Names[3]);
strcpy (Global_Font, Font_Path);
printf (Global_Font);
}
}
You are storing the exact same value, entry->d_name in every entry. Of course they're all going to point to the same string.
You may be thinking that readdir changes the value of entry->d_name to something different every time. But it doesn't. Every time, it sets the value of entry->d_name to point to the very same buffer that readdir uses to hold the entry name.
You need to save the actual name, not a pointer to a buffer controlled by some other piece of code. Allocating an array of pointers is insufficient -- where are the actual entry names going to be stored?
Hello I am having the following difficulty,
I am trying to read in a table of doubles (1 entry per line) and store it in an array, while dynamically changing this array's size (for each line/entry). This is for a school assignment and it forbids the use of vectors(would be much easier...). The main idea that I had is to have a main array which stores the value, then store the previous values and the next one into a new array and do this iteratively. Currently, the problem that I am having is that only the last value of the table is being stored. I am aware, that somehow I need to be passing the data by refference to the global function and that the pointers that I am working with become null ater they exit the following iteration of the while. However, since the exact length of the data is unknown, this seems impossible since intializing an array in the main() is impossible (exact length not known). Any help would be appreciated.
Code posted below.
EDIT: after consideration of the two comments I made the following changes to the code, however I am not sure, whether they will behave appropriately. I added a new function called add_new_datapoint, that should globally change the values of the pointer/length and this is done by passing the values by refference. Called in the problematic else statement as add_new_datapoint(data_ptr, data_len, new_dp). Also, I am not sure that reallocating new memory to the pointer variable, will not result in a memory leak. In essence (after I reallocate data_ptr is the memory that was 'being pointed to' released or do I have to delete it and then re-inialise it in the . In such case, can I refference the pointer 'data_ptr' again in the next iteration of the loop?
I think it will be easier to simplify your posted code than trying to find all the places where you could have errors.
If you expect to see only double values in your file, you can simplify the code for reading data from the file to:
while ( data_file >> new_data_pt )
{
// Use new_data_pt
}
If you expect that there might be values other than doubles, then you can use:
while ( getline(data_file, line) )
{
std::istringstream str(line);
while ( str >> new_data_pt )
{
// Use new_data_pt
}
}
but then you have to understand the code will not read any more values from a line after it encounters an error. If your line contains
10.2 K 25.4
the code will read 10.2, encounter an error at K, and will not process 25.4.
The code to process new_data_pt is that it needs to be stored in a dynamically allocated array. I would suggest putting that in a function.
double* add_point(double* data_ptr, int data_len, double new_data_pt)
Call that function as:
data_ptr = add_point(data_ptr, data_len, new_data_pt);
Assuming the first while loop, the contents of main become:
int main()
{
std::fstream data_file{ "millikan2.dat" };
// It is possible that the file has nothing in it.
// In that case, data_len needs to be zero.
int data_len{ 0 };
// There is no need to allocate memory when there is nothing in the file.
// Allocate memory only when data_len is greater than zero.
double* data_ptr = nullptr;
double new_data_pt;
if (!data_file.good()) {
std::cerr << "Cannot open file";
return 1;
}
while ( data_file >> new_data_pt )
{
++data_len;
data_ptr = add_point(data_ptr, data_len, new_data_pt);
}
// No need of this.
// The file will be closed when the function returns.
// data_file.close();
}
add_point can be implemented as:
double* add_point(double* data_ptr, int data_len, double new_data_pt)
{
double* new_data_ptr = new double[data_len];
// This works even when data_ptr is nullptr.
// When data_ptr is null_ptr, (data_len - 1) is zero. Hence,
// the call to std::copy becomes a noop.
std::copy(data_ptr, data_ptr + (data_len - 1); new_data_ptr);
// Deallocate old memory.
if ( data_ptr != nullptr )
{
delete [] data_ptr;
}
new_data_ptr[data_len-1] = new_data_pt;
return new_data_ptr;
}
The code to track the number of bad points is a lot more complex. Unless you are required to do it, I would advise to ignore it.
You already got an excellent answer but I figured it may be helpful to point out a few mistakes in your code, so you can understand why it won't work.
In the second else scope you declare data_ptr again, even though it is visible from the outer scope. (delete[] doesn't delete the pointer itself, it just deallocates the memory the pointer points to.)
else {
double* data_temp { new double[data_len] };
std::copy(data_ptr, data_ptr + data_len - 2, data_temp);
*(data_temp + data_len - 1) = new_data_pt;
delete[] data_ptr;
double* data_ptr{ new double[data_len] }; // <- Right here
//for (int j{1}; j < data_len; j++) *(data_ptr + j) = *(data_temp + j);
std::cout << std::endl;
}
Instead you could just write data_ptr = new double[data_len]. However, that alone won't make this work.
All of your data disappears because on every iteration you create a new array, pointed to by data_temp and copy the data there, and on the next iteration you set data_temp to point to a new array again. This means that on every iteration you lose all data from previous iterations. This also causes a memory leak, since you allocate more memory every time you hit this line:
double* data_temp { new double[data_len] };
but you don't call delete[] data_temp afterwards.
I hope this helps to understand why it doesn't work.
I was trying to erase pointer elements (the value in the map is a pointer) from the map and I saw the code here What happens to an STL iterator after erasing it in VS, UNIX/Linux?
for(map<T, S*>::iterator it = T2pS.begin(); it != T2pS.end(); T2pS.erase(it++)) {
// wilhelmtell in the comments is right: no need to check for NULL.
// delete of a NULL pointer is a no-op.
if(it->second != NULL) {
delete it->second;
it->second = NULL;
}
}
I am not sure if the 'delete it->second' with de-allocate the correct memory because the erase(it++) step already moves the iterator to the next object. By the time, it reaches the delete statement, it is pointing to the next element which we don't want to delete. Am I missing something?
I believe this will work as expected.
The third section of the for loop (where the iterator is erased and then incremented) executes after the first iteration, and so on for each relevant iteration. Thus, you're always erasing the element you've already "dealt with" in the loop contents.
A parallel example:
for (int i = 0; i < 1; ++i) { ...
You will still enter the loop and execute with i = 0 before incrementing i and checking the looping condition.
You may want to try another way:
while (T2pS.size() > 0) {
if (T2pS.begin()->second != NULL) {
delete T2pS.begin()->second;
}
T2pS.erase(T2pS.begin());
}
EDIT: Pastebin links to the entirety of the code at the bottom
for my CS215 course, I was given a class called String215 which is a basic string class to help in the understanding of dynamic memory allocation and pointer arithmetic with char arrays.
The class was given to me in a very basic skeleton form with prototypes but no implementations, along with a test function to test my implementations. I CAN NOT use any C String functions in this assignment.
The part of the program which is troubling is the append function, which just appends a parameter string215 object to the end of the current string215 object.
// Add a suffix to the end of this string. Allocates and frees memory.
void string215::append(const string215 &suffix)
{
char *output = new char[str_len(data)+suffix.length()+1];
for(int x = 0; x < str_len(data); x++) {
*output = *data;
output++;
data++;
}
for(int x = 0; x < suffix.length(); x++) {
*output = suffix.getchar(x);
output++;
}
*output = '\0';
output -= (str_len(data)+suffix.length()+1);
delete[] data;
data = output;
}
This portion of the code is tested in the 13th test of the test function as shown here:
string215 str("testing");
...
// Test 13: test that append works in a simple case.
curr_test++;
string215 suffix("123");
str.append(suffix);
if (strcmp(str.c_str(), "testing123") != 0) {
cerr << "Test " << curr_test << " failed." << endl;
failed++;
}
Here is the description of the append class:
Add the suffix to the end of this string. Allocates a new, larger, array; copies the old contents, followed by the suffix, to the new array; then frees the old array and updates the pointer to the new one.
My program aborts at the very end of the append function execution with the error message:
Debug Assertion Failed!
Program: [Source path]\dbgdel.cpp
Line: 52
Expression: _BLOCK_TYPE_IS_VALID(pHead->nBlockUse)
...
Abort || Retry || Ignore
I'm fairly certain it has something to do with my very poor memory management. I know it's not a lot to go on, but I've been struggling with this for hours on end and can't seem to figure it out.
Here's a pastebin of the .cpp and .h file for this program
string215.cpp: http://pastebin.com/Xh2SvDKJ
string215.h: http://pastebin.com/JfAJDEVN
Any help at all is greatly appreciated!
Thanks,
RAW-BERRY
You are changing data pointer before delete[]. You need to delete[] exactly the same value you got from new[].
Also, you are incrementing output pointer str_len(data)+suffix.length() times, and you take it back by str_len(data) + suffix.length() + 1.
I would use separate variables for iteration to solve these problems.
You increment output exactly str_len(data) + suffix.length() times. Note that you don't increment output after *output = '\0';.
So to go back to the start, you should use:
output -= (str_len(data) + suffix.length());
By the way, some of the code is not very efficient. For example, getchar uses a loop instead of simply returning data[index]. You use getchar in append, which means that the performance isn't great.
EDIT: As zch says, you use delete[] data after modifying data, but note that even before that you use str_len(data) after modifying data (when deciding how many bytes to go skip back), so the calculation is wrong (and my suggestion above is also wrong, because str_len(data) is now zero).
So I think your problem is with the line
for(int x = 0; x < str_len(data); x++) {
Notice that the size of 'data' is changing at each iteration of the loop. As you increment 'x', you are decreasing the length of 'data'. Suppose 'data' is a string holding "hello": in the first iteration of the loop x=0 and str_len(data)=5; in the second iteration x=1 and str_len(data)=4. Thus the for loop executes half as many times as you need it to and 'data' does not end up pointing to the end of the data string
I am working on a query processor that reads in long lists of document id's from memory and looks for matching id's. When it finds one, it creates a DOC struct containing the docid (an int) and the document's rank (a double) and pushes it on to a priority queue. My problem is that when the word(s) searched for has a long list, when I try to push the DOC on to the queue, I get the following exception:
Unhandled exception at 0x7c812afb in QueryProcessor.exe: Microsoft C++ exception: std::bad_alloc at memory location 0x0012ee88..
When the word has a short list, it works fine. I tried pushing DOC's onto the queue in several places in my code, and they all work until a certain line; after that, I get the above error. I am completely at a loss as to what is wrong because the longest list read in is less than 1 MB and I free all memory that I allocate. Why should there suddenly be a bad_alloc exception when I try to push a DOC onto a queue that has a capacity to hold it (I used a vector with enough space reserved as the underlying data structure for the priority queue)?
I know that questions like this are almost impossible to answer without seeing all the code, but it's too long to post here. I'm putting as much as I can and am anxiously hoping that someone can give me an answer, because I am at my wits' end.
The NextGEQ function reads a list of compressed blocks of docids block by block. That is, if it sees that the lastdocid in the block (in a separate list) is larger than the docid passed in, it decompresses the block and searches until it finds the right one. Each list starts with metadata about the list with the lengths of each compressed chunk and the last docid in the chunk. data.iquery points to the beginning of the metadata; data.metapointer points to wherever in the metadata the function currently is; and data.blockpointer points to the beginning of the block of uncompressed docids, if there is one. If it sees that it was already decompressed, it just searches. Below, when I call the function the first time, it decompresses a block and finds the docid; the push onto the queue after that works. The second time, it doesn't even need to decompress; that is, no new memory is allocated, but after that time, pushing on to the queue gives a bad_alloc error.
Edit: I cleaned up my code some more so that it should compile. I also added in the OpenList() and NextGEQ functions, although the latter is long, because I think the problem is caused by a heap corruption somewhere in it. Thanks a lot!
struct DOC{
long int docid;
long double rank;
public:
DOC()
{
docid = 0;
rank = 0.0;
}
DOC(int num, double ranking)
{
docid = num;
rank = ranking;
}
bool operator>( const DOC & d ) const {
return rank > d.rank;
}
bool operator<( const DOC & d ) const {
return rank < d.rank;
}
};
struct listnode{
int* metapointer;
int* blockpointer;
int docposition;
int frequency;
int numberdocs;
int* iquery;
listnode* nextnode;
};
void QUERYMANAGER::SubmitQuery(char *query){
listnode* startlist;
vector<DOC> docvec;
docvec.reserve(20);
DOC doct;
//create a priority queue to use as a min-heap to store the documents and rankings;
priority_queue<DOC, vector<DOC>,std::greater<DOC>> q(docvec.begin(), docvec.end());
q.push(doct);
//do some processing here; startlist is a pointer to a listnode struct that starts the //linked list
//point the linked list start pointer to the node returned by the OpenList method
startlist = &OpenList(value);
listnode* minpointer;
q.push(doct);
//start by finding the first docid in the shortest list
int i = 0;
q.push(doct);
num = NextGEQ(0, *startlist);
q.push(doct);
while(num != -1)
{
q.push(doct);
//the is where the problem starts - every previous q.push(doct) works; the one after
//NextGEQ(num +1, *startlist) gives the bad_alloc error
num = NextGEQ(num + 1, *startlist);
//this is where the exception is thrown
q.push(doct);
}
}
//takes a word and returns a listnode struct with a pointer to the beginning of the list
//and metadata about the list
listnode QUERYMANAGER::OpenList(char* word)
{
long int numdocs;
//create a new node in the linked list and initialize its variables
listnode n;
n.iquery = cache -> GetiList(word, &numdocs);
n.docposition = 0;
n.frequency = 0;
n.numberdocs = numdocs;
//an int pointer to point to where in the metadata you are
n.metapointer = n.iquery;
n.nextnode = NULL;
//an int pointer to point to the uncompressed block of data, if there is one
n.blockpointer = NULL;
return n;
}
int QUERYMANAGER::NextGEQ(int value, listnode& data)
{
int lengthdocids;
int lengthfreqs;
int lengthpos;
int* temp;
int lastdocid;
lastdocid = *(data.metapointer + 2);
while(true)
{
//if it's not the first chunk in the list, the blockpointer will be pointing to the
//most recently opened block and docpos to the current position in the block
if( data.blockpointer && lastdocid >= value)
{
//if the last docid in the chunk is >= the docid we're looking for,
//go through the chunk to look for a match
//the last docid in the block is in lastdocid; keep going until you hit it
while(*(data.blockpointer + data.docposition) <= lastdocid)
{
//compare each docid with the docid passed in; if it's greater than or equal to it, return a pointer to the docid
if(*(data.blockpointer + data.docposition ) >= value)
{
//return the next greater than or equal docid
return *(data.blockpointer + data.docposition);
}
else
{
++data.docposition;
}
}
//read through the whole block; couldn't find matching docid; increment metapointer to the next block;
//free the block's memory
data.metapointer += 3;
lastdocid = *(data.metapointer + 3);
free(data.blockpointer);
data.blockpointer = NULL;
}
//reached the end of a block; check the metadata to find where the next block begins and ends and whether
//the last docid in the block is smaller or larger than the value being searched for
//first make sure that you haven't reached the end of the list
//if the last docid in the chunk is still smaller than the value passed in, move the metadata pointer
//to the beginning of the next chunk's metadata; read in the new metadata
while(true)
// while(*(metapointers[index]) != 0 )
{
if(lastdocid < value && *(data.metapointer) !=0)
{
data.metapointer += 3;
lastdocid = *(data.metapointer + 2);
}
else if(*(data.metapointer) == 0)
{
return -1;
}
else
//we must have hit a chunk whose lastdocid is >= value; read it in
{
//read in the metadata
//the length of the chunk of docid's is cumulative, so subtract the end of the last chunk
//from the end of this chunk to get the length
//find the end of the metadata
temp = data.metapointer;
while(*temp != 0)
{
temp += 3;
}
temp += 2;
//temp is now pointing to the beginning of the list of compressed data; use the location of metapointer
//to calculate where to start reading and how much to read
//if it's the first chunk in the list,the corresponding metapointer is pointing to the beginning of the query
//so the number of bytes of docid's is just the first integer in the metadata
if( data.metapointer == data.iquery)
{
lengthdocids = *data.metapointer;
}
else
{
//start reading from the offset of the end of the last chunk (saved in metapointers[index] - 3)
//plus 1 = the beginning of this chunk
lengthdocids = *(data.metapointer) - (*(data.metapointer - 3));
temp += (*(data.metapointer - 3)) / sizeof(int);
}
//allocate memory for an array of integers - the block of docid's uncompressed
int* docblock = (int*)malloc(lengthdocids * 5 );
//decompress docid's into the block of memory allocated
s9decompress((int*)temp, lengthdocids /4, (int*) docblock, true);
//set the blockpointer to point to the beginning of the block
//and docpositions[index] to 0
data.blockpointer = docblock;
data.docposition = 0;
break;
}
}
}
}
Thank you very much, bsg.
QUERYMANAGER::OpenList returns a listnode by value. In startlist = &OpenList(value); you then proceed to take the address of the temporary object that's returned. When the temporary goes away, you may be able to access the data for a time and then it's overwritten. Could you just declare a non-pointer listnode startlist on the stack and assign it the return value directly? Then remove the * in front of other uses and see if that fixes the problem.
Another thing you can try is replacing all pointers with smart pointers, specifically something like boost::shared_ptr<>, depending on how much code this really is and how much you're comfortable automating the task. Smart pointers aren't the answer to everything, but they're at least safer than raw pointers.
Assuming you have heap corruption and are not in fact exhausting memory, the commonest way a heap can get corrupted is by deleting (or freeing) the same pointer twice. You can quite easily find out if this is the issue by simply commenting out all your calls to delete (or free). This will cause your program to leak like a sieve, but if it doesn't actually crash you have probably identified the problem.
The other common cause cause of a corrupt heap is deleting (or freeing) a pointer that wasn't ever allocated on the heap. Differentiating between the two causes of corruption is not always easy, but your first priority should be to find out if corruption is actually the problem.
Note this approach won't work too well if the things you are deleting have destructors which if not called break the semantics of your program.
Thanks for all your help. You were right, Neil - I must have managed to corrupt my heap. I'm still not sure what was causing it, but when I changed the malloc(numdocids * 5) to malloc(256) it magically stopped crashing. I suppose I should have checked whether or not my mallocs were actually succeeding! Thanks again!
Bsg