C++ Second dynamic allocation while first works (same size) - c++

I used to work with 3D dynamic allocated arrays but I read that 1D array has better performance so I tried to transorm my 3D array in a 1D array.
I work with two big arrays (about 240*320*1000 double) and the allocation look like this :
int numberFiles; //about 1000, can change
int nRow; //240
int nCol; //320
double *inputData;
double *outputData;
inputData = (double*)malloc(nCol*nRow*numberFiles*sizeof(double)); //Always works
outputData = (double*)malloc(nCol*nRow*numberFiles*sizeof(double)); //Always fails
The second allocation always fail. Since the first doesn't, it's not a size problem. And since, when allocate this in a 3D arrays, it's also working, it's not a memory space problem
Any ideas why the second allocation fails ?
Thanks.
Chen Song.
PS : Sorry for the poor english.
PS2: Tried new Double[], same problems.
Edit : So allocation in a simple code works but as soon as I try to put it in my app, it's not working anymore. I'm woking with QtCreator with mingw32 and have matio(libraries to read MATLAB files) to read data. Here a code example of my program which give me error :
//Needed include
using namespace std;
int main(int argc, char ** argv)
{
int k,i,j;
double *matriceBrute,*matriceSortie;
double* matData;
const char * c;
mat_t *matfp;
matvar_t *matvar;
QApplication app(argc,argv);
// Getting the directory of the data and the files in it
QString directoryPath = QFileDialog::getExistingDirectory();
QDir myDir(directoryPath);
myDir.setNameFilters(QStringList()<<"*.MAT");
QStringList filesList = myDir.entryList(QDir::Files | QDir::Hidden);
string fullPath= (directoryPath+"/"+filesList[0]).toLocal8Bit().constData();
c = fullPath.c_str();
// Loading the first one to get the size of the data
matfp = Mat_Open(c,MAT_ACC_RDONLY);
matvar = Mat_VarReadNextInfo(matfp);
int read_err = Mat_VarReadDataAll(matfp,matvar);
int numberFiles = (int)filesList.size();
int nCol = matvar->dims[1];
int nRow = matvar->dims[0];
//Allocating
matriceBrute = (double*)malloc(nCol*nRow*numberFiles*sizeof(double));
matriceSortie = (double*)malloc(nCol*nRow*numberFiles*sizeof(double));
//the first one works but not the second
//loading the data in the allocated memory
for (k=0;k<numberFiles;k++)
{
fullPath= (directoryPath+"/"+filesList[k]).toLocal8Bit().constData();
c = fullPath.c_str();
matfp = Mat_Open(c,MAT_ACC_RDONLY);
matvar = Mat_VarReadNext(matfp);
matData = (double*)(matvar->data);
for (i=0;i<nRow;i++)
{
for (j=0;j<nCol;j++)
{
matriceBrute[i + nRow*(j+k*nCol)]=*(matData+(i+j*nRow));
matriceSortie[i + nRow*(j+k*nCol)]=*(matData+(i+j*nRow));
}
}
Mat_VarFree(matvar);
Mat_Close(matfp);
}
}

No error, it works.
#include <iostream>
using namespace std;
int main()
{
int numberFiles = 1000; //about 1000, can change
int nRow = 240; //240
int nCol = 240; //320
double *inputData;
double *outputData;
inputData = (double*)malloc(nCol*nRow*numberFiles*sizeof(double)); //Always works
outputData = (double*)malloc(nCol*nRow*numberFiles*sizeof(double)); //Always fails
if (inputData == NULL)
cout << "inputData alloc failed" << endl;
if (outputData == NULL)
cout << "outputData alloc failed" << endl;
return 0;
}
It does not print as failed, so, don't worry, be happy.

May it be memory fragmentation problem? If, for example, you can't alloc ~0.6Gb (by one heap) twice? First time you alloc it, at second - it just can't find in your physical memory such big heap of free memory?
Which exception do you receive?

Related

How can I fix this? " HEAP CORRUPTION DETECTED after normal block.. CRT detected that the application wrote to memory after end of heap buffer "

I had a program working good (without prompting errors) using references, but I decided to use a pointer-to-pointer array allocated on virtual memory, because I can use a variable as the size of the array.
The error prompts when I break the while (m_Window.isOpen()) loop, in other words when I close the game window and the game is finished. I have noticed that the program breaks when I try to erase the virtual memory in the Engine::cleanVirtualMemory() function. I have noticed that it is there because I have put two flags (cout << "running1" << endl; and cout << "running2" << endl) and I can show just the first flag.
Then it prompts a window with the following message:
HEAP CORRUPTION DETECTED after normal block.. CRT detected that the application wrote to memory after end of heap buffer
main.cpp
#include "Engine.h"
using namespace sf;
int main(){
Engine Motor;
Motor.run();
Motor.cleanVirtualMemory();
}
Engine.h
#pragma once
#include <SFML/Graphics.hpp>
#include "StructureBuilder.h"
using namespace sf;
using namespace std;
class Engine{
private:
// Lots of variables ....
Vector2i m_ArenaSize;
Vector2f * vectorStructureArray = new Vector2f[m_ArenaSize.y * m_ArenaSize.x * 4];
int** logicStructureArray = new int*[m_ArenaSize.y];
// Lots of variables ....
//Gameloop
void Input();
void Update(dtAsSeconds);
void Draw();
public:
Engine();
void run();
void cleanVirtualMemory();
};
Engine.cpp
#include "Engine.h"
#include <iostream>
using namespace sf;
Engine::Engine() {
/// lots of variables and data ...
/// Making arena
m_ArenaSize = Vector2i(10, 10);
StructureBuilder(arenaStructures, vectorStructureArray, logicStructureArray, m_ArenaSize);
}
void Engine::run() {
//Timing
Clock clock;
while (m_Window.isOpen()) {
// Each time clock restarted, dt = time elapse (from 0 to now, then clock = 0)
Time dt = clock.restart();
// Convert time elapse to seconds
double dtAsSeconds = dt.asSeconds();
// Call each part of the game in turn
Input();
Update(dtAsSeconds);
Draw();
}
}
void Engine::cleanVirtualMemory() {
// Deallocate Virtual Memory
// first flag
cout << "running1" << endl;
for (int i = m_ArenaSize.x - 1; i > -1; i--) {
delete[] logicStructureArray[i];
}
delete[] logicStructureArray;
logicStructureArray = NULL;
delete[] vectorStructureArray;
vectorStructureArray = NULL;
// second flag
cout << "running2" << endl;
}
StructureBuilder.h
#pragma once
#include <SFML/Graphics.hpp>
#include <iostream>
#include <string.h>
using namespace sf;
using namespace std;
Vector2i StructureBuilder(VertexArray& rVA, Vector2f* rA, int** rLA, Vector2i ArenaSize);
In the following code, you can see where I use pointers, I have erased code just to simplify.
StructureBuilder.cpp
#include "StructureBuilder.h"
Vector2i StructureBuilder(VertexArray& rVA, Vector2f* rA, int** rLA, Vector2i ArenaSize) {
//Set map properties and VertexArrayType
double tileSize = 100;
double Height = ArenaSize.y * tileSize;
double angle = 30;
int offset = 0;
int Primitive = 4;
int currentVertex = 0;
rVA.setPrimitiveType(Quads);
rVA.resize(ArenaSize.x * ArenaSize.y * 4);
// First build graphically our map structures's using char strings
string stringArray[10];
stringArray[0] = "1000000000";
stringArray[1] = "0000000000";
stringArray[2] = "0000010000";
stringArray[3] = "0000000000";
stringArray[4] = "0000000000";
stringArray[5] = "0000000000";
stringArray[6] = "0000000000";
stringArray[7] = "0000000000";
stringArray[8] = "0000000000";
stringArray[9] = "0000000000";
// Convert stringArray to charArray, finally charArray to intArray
char** charArray = new char*[ArenaSize.y];
Vector2f Vector1;
Vector2f Vector2;
Vector2f Vector3;
Vector2f Vector4;
for (int i = 0; i < ArenaSize.x; i++) {
charArray[i] = new char[ArenaSize.x];
rLA[i] = new int[ArenaSize.x];
}
for (int i = 0; i < ArenaSize.x; i++) {
for (int j = 0; j < ArenaSize.y; j++) {
charArray[j][i] = stringArray[j][i];
rLA[j][i] = charArray[j][i] - 48;
// Check when we have a value greater or equal to 1, if yes build a structure.
if (rLA[j][i] == 1) {
Vector1 = Vector2f(Value..., Value...);// in order to understand I dont put the whole calculation here
Vector2 = Vector2f(Value..., Value ...); // is just trigonometry to find vertex
Vector3 = Vector2f(Value..., Value ...);
Vector4 = Vector2f(Value..., Value ...);
rVA[currentVertex + 0].position = Vector1;
rVA[currentVertex + 1].position = Vector2;
rVA[currentVertex + 2].position = Vector3;
rVA[currentVertex + 3].position = Vector4;
rVA[currentVertex + 0].texCoords = Vector2f(42, 0); // coords on my spritesheet
rVA[currentVertex + 1].texCoords = Vector2f(86, 24);
rVA[currentVertex + 2].texCoords = Vector2f(42, 49);
rVA[currentVertex + 3].texCoords = Vector2f(0, 24);
rA[currentVertex + 0] = Vector1; //Later I use this to tell the program where to construct restrictions (where the player can't move)
rA[currentVertex + 1] = Vector2;
rA[currentVertex + 2] = Vector3;
rA[currentVertex + 3] = Vector4;
}
currentVertex = currentVertex + Primitive;
}
}
// Deallocate Virtual Memory
for (int i = ArenaSize.x - 1; i > -1; i--) {
delete[] charArray[i];
}
delete[] charArray;
charArray = NULL;
return ArenaSize;
}
Let's take a look at the declaration of the Engine class.
class Engine{
private:
// Lots of variables ....
Vector2i m_ArenaSize;
Vector2f * vectorStructureArray = new Vector2f[m_ArenaSize.y * m_ArenaSize.x * 4];
The first member of your Engine class is called m_ArenaSize. This is the first class member that will get constructed when an Engine gets constructed. That's how object construction works in C++: when a new object gets constructed, all of the new object's members get constructed in declaration order.
The second member of your Engine class is this vectorStructureArray pointer. It will be allocated to point to an array, with dynamic size, that gets newed using two of the constructed m_ArenaSize's own members, x and y.
And now, let's take a look at Engine's constructor:
Engine::Engine() {
All right. That's your constructor. So, according to our plan, m_ArenaSize is going to get default-constructed since it is not explicitly constructed, in Engine's constructor's initialization section (there is none, here). If you investigate what m_ArenaSize's default constructor does you will discover that it defaults x and y to 0.
And that's what's going to construct the vectorStructureArray pointer, as the 2nd order of business of Engine's default initialization. And, because its x and y are 0, the pointer will point to a grand total of 0 values.
m_ArenaSize = Vector2i(10, 10);
And only now the default-constructed m_ArenaSize gets replaced with a different subject, with different x and y values. The shown code clearly expects the pointer to get reallocated, to reflect m_ArenaSize's new x and y values,. However C++ does not work this way. vectorStructureArray has already been constructed. It's not going to get constructed again simply because a different class member gets replaced. Subsequent code's assumption is the new x and y values, and this results in fairly obvious memory corruption.
The same exact bug occurs with logicStructureArray, too.
These are just the first two major bugs in the shown code regarding memory allocation. There are several ways to fix them, but the easiest way to fix bugs is to make it logically impossible for them to happen in the first place. It is logically impossible for new and delete-related bugs to occur if they are never used. Modern C++ code rarely uses new and delete, but rather employs the services of the C++ library's many containers.
Here, both pointers can be simply replaced by std::vector, with its resize() member taking care of allocating both vectors' sizes. And, as extra bonus, all allocated memory gets automatically freed, automatically preventing all memory leaks.
Replacing all the error-prone new and delete logic with std::vector will fix all memory-related issues in the shown code, the only thing you will have to make sure is that the vectors are correctly resize()d.

C++ Memory leak (or weird behavior) on Linux, but not Windows

I have come across some unexplained memory behavior on Linux in my imaging application. Specifically, it appears that the application holds onto a certain amount of resident memory and will never release it. In our application, it appears that when allocating larger chunks, such as four sessions that each use 3GBs, once all resources are released the baseline may jump 1GB and then stay constant for a while. At some later point, it might jump again. The exact same sequence of operations on Windows demonstrates the expected memory behavior where the baseline remains fixed i.e. memory usage comes back to what it was when the application started.
We have profiled the code extensively to confirm the same number of allocations / deletions, verified in valgrind and that has shown no leaks.
When the image buffer is a simple array of char, allocation and deallocation behave as expected. If the image buffer is a class which contains the array of char, that too works as expected. If I have a container class (DicomImageData) that hold the image buffer class, the problem occurs. It has been isolated and shown below.
First run with an input parameter (WORK_AROUND_LEAK in C++ code below) set to 0. The first run will allocate 2.9GBs of memory and then release it, as viewed by top. The second and all subsequent runs, it will allocate 2.9GBs of memory, but it will not be released.
Work around
I discovered a “work around” by pure luck, but it appears that if I create an intermediate variable to hold the vector of DicomImageData objects, this issue no longer occurs. To verify, run it again with the input parameter set to 1 and do the same sequence. Memory will continually grow to 2.3GB, and then be released. I have tested on various Ubuntu and RHEL implementations and all behave in a consistent manner.
For reference, the dev system is Ubuntu 18.04.4 LTS with 72 cores and 32GB of memory, however running with 8-cores and 16GB exhibits identical behavior, although the 16GB system will thrash much sooner. I’m using gcc 7.5.0.
Any help would be appreciated.
My cpp code and Makefile are as below
C++ source code
#define WORK_AROUND_LEAK 0
#include <stdio.h>
#include <iostream>
#include <vector>
#include <string.h>
int totalImages = 2299;
// Class to hold image data
class PixelData {
public:
char * buffer;
size_t size;
PixelData()
: buffer(NULL)
, size(0) {
}
~PixelData() {
if (buffer) {
delete[] buffer;
buffer = NULL;
}
size = 0;
}
};
// Class to hold above defined Pixel data and some other meta-data relevant to the image
class DicomImageData {
public:
PixelData * pixelData;
DicomImageData() {
pixelData = NULL;
}
~DicomImageData() {
if (pixelData) {
delete pixelData;
pixelData = NULL;
}
}
};
using namespace std;
int main() {
#if WORK_AROUND_LEAK
std::vector<DicomImageData *> imageDataArray[1];
#endif
char c = 'c';
int executionCount(0);
size_t pixelDataSize(2 * 1024 * 520);
do {
int numIterations = 1;
executionCount++;
cout << "\nStarting execution test1 " << executionCount << endl;
int iter = 0;
cout << "Starting execution test iterations " << iter << endl;
std::vector<DicomImageData*> imageData;
for (size_t i = 0; i < totalImages; i++) {
char * readPixelData = new char [pixelDataSize + 1 + (i*3)];
memset(readPixelData, '#', pixelDataSize + 1);
DicomImageData * dicomImageData = new DicomImageData();
if (dicomImageData) {
// Create new pixel data struct and set
PixelData * pixelData = new PixelData();
pixelData->buffer = readPixelData;
pixelData->size = pixelDataSize;
dicomImageData->pixelData = pixelData;
imageData.push_back(dicomImageData);
}
#if WORK_AROUND_LEAK
imageDataArray[0] = imageData;
#endif
}
printf("\nPress ENTER to release memory \n");
scanf("%c", &c);
iter = 0;
cout << "Starting release iterations " << iter << endl;
#if WORK_AROUND_LEAK
imageData = imageDataArray[0];
#endif
for (size_t i = 0; i < totalImages; i++) {
PixelData *pixelData = imageData[i]->pixelData;
delete[] pixelData->buffer;
pixelData->buffer = NULL;
delete imageData[i]->pixelData;
imageData[i]->pixelData = NULL;
delete imageData[i];
imageData[i] = NULL;
}
imageData.clear();
printf("Press ENTER to run another test or press X to exit \n");
scanf("%c", &c);
} while (c != 'x' && c != 'X');
}
Makefile if needed
all: memory_leak
memory_leak:
g++ -std=c++11 -O2 -D NDEBUG memory_leak.cpp -o memory_leak
clean:
rm memory_leak
To my knowledge C++ does not initialize variables to a null value so
~DicomImageData() {
if (pixelData) {
delete pixelData;
pixelData = NULL;
}
}
might be the problem. Try this
// Class to hold image data
class PixelData {
public:
char * buffer = nullptr;
size_t size;
PixelData()
: buffer(nullptr)
, size(0) {
}
~PixelData() {
if (buffer!=nullptr) {
delete[] buffer;
buffer = nullptr;
}
size = 0;
}
};
Do the same nullptr initialization for the other class that has a pointer member variable. Also, you can avoid this entirely by using a unique_ptr to hold your pointers. See this post on how to use a unique_ptr to hold an array. Always try to use a smart pointers until you can't.

Corrupted struct char arrays - sqlite C++

I am a bit puzzled by this one. Everything has been fine so far with using SQLite but now I am trying to store my query results in a simple struct. When I do this in my callback, all my data looks great in my SQLItems vector but as soon as the callback exits, my SQLItems vector holding my rows of data is suddenly corrupted. Any ideas what could be causing this?
// Simple struct to hold column name and row data
struct SQLrow {
char * Column;
char * Data;
};
// static Vector to hold SQL rows
static std::vector<SQLrow> SQLItems;
...
// static callback that handles placing query results into structs and into SQLItems vector
// SQLItems column/row data gets corrupted after this function exits
static int countTablesCallback(void *data, int count, char **rows, char **azColName) {
int i;
for (i = 0; i < count; i++) {
SQLrow newItem = { azColName[i] ,rows[i] };
SQLItems.push_back(newItem);
}
*static_cast<std::vector<SQLrow>*>(data) = SQLItems; // Tried this too but throws an exception
return 0;
}
I also thought maybe it is only possible to statically cast from the callback to save the vector but that is throwing an exception as well. Stumped here. Thanks for any advice!
Your vector is fine, the static_cast makes no sense there, unless data is actually used as an out parameter. Your problem is, most likely, that SQLrow holds char pointer and SQLite deletes the pointed-to strings after the callback returns. Changing your class to
struct SQLrow {
std::string Column;
std::string Data;
};
should solve the problem.
Just looking at the code, it appears that the data pointed to by rows will be invalidated/destroyed/changed once the callback returns. So you can't retain those pointers for later use, and will have to make a copy of the data.
One easy way is to change Column and Data from char * to std::string. Failing that, you'll have to do some sort of manual memory management (allocate space with new, then delete it later) which is error prone and not really advisable these days.
In my opinion, there are very few case in which you want/need to use raw string in c++ and yours isn't one of those. By the way I hope this will help you or someone else in some way:
#include <vector>
#include <stdio.h>
#include <string.h>
#include <iostream>
struct SQLrow {
char* Column;
char* Data;
};
void your_callback(int count, char **rows, char **azColName) {
std::vector<SQLrow> rows_list;
for (int i = 0; i < count; i++) {
/* Uncomment this if you want
your copy of the strings. If you
use this, don't forget to free the
memory yourself with delete[] s1 and
s2.
size_t s1_len = strlen(rows[i]);
size_t s2_len = strlen(azColName[i]);
char* s1 = new char [sizeof(char) * (s1_len + 1)];
char* s2 = new char [sizeof(char) * (s2_len + 1)];
memcpy(s1, rows[i], s1_len);
s1[s1_len] = '\0';
memcpy(s2, azColName[i], s2_len);
s2[s2_len] = '\0';
SQLrow r = { s1, s2 }; */
SQLrow r = { rows[i], azColName[i] };
rows_list.push_back(r);
}
// test the result
for (int i = 0; i < count; i++) {
SQLrow r = rows_list.at(i);
std::cout << "rows:" << r.Column << " azColName:" << r.Data << std::endl;
}
}
// this 2 lines are just for simulating the data
// you will get this 'warning: ISO C++ forbids converting a string constant to char*''
char* rows[] = {"row1", "row2" , "row3" };
char* colName[] = {"name1", "name2", "name3" };
int main()
{
your_callback(3, rows, colName);
return 0;
}

Why does the last sr5 object not occupy memory with overloaded new operator?

When I run this program sr1, sr2, sr3, sr4 objects are created and values are assigned to corresponding variables. But in sr5 object, the name remains blank while the roll_no percentage shows the correct value.
When change the value of
int MAX = 5;
to
int MAX = 6;
everything works fine.
Here is my code:
const int MAX = 5;
const int FREE = 0;
const int OCCUPIED = 1;
int flag = 0;
using namespace std;
void warning()
{
cout<<"\n------All memory occupied------"<<endl;
exit(1);
}
class student_rec
{
private:
char name[25];
int roll_no;
float percentage;
public:
student_rec(char *n, int r, float per)
{
strcpy(name, n);
roll_no = r;
percentage = per;
}
student_rec()
{
}
void set_rec(char *n, int r, float per)
{
strcpy(name, n);
roll_no = r;
percentage = per;
}
void show_rec()
{
cout<<"\n-------------------\n";
cout<<"Name= "<<name<<endl;
cout<<"Roll number= "<<roll_no<<endl;
cout<<"Percentage= "<<percentage<<endl;
}
void *operator new (size_t sz);
void operator delete (void *d);
};
struct memory_store
{
student_rec obj;
int status;
};
memory_store *m = NULL;
void *student_rec::operator new (size_t sz)
{
int i;
if(flag == 0)
{
m = (memory_store *) malloc(sz * MAX);
if(m == NULL)
warning();
for(i=0; i<MAX; i++)
m[i].status = FREE;
flag = 1;
m[0].status = OCCUPIED;
return &m[0].obj;
}
else
{
for(i=0; i<MAX; i++)
{
if(m[i].status == FREE)
{
m[i].status = OCCUPIED;
return &m[i].obj;
}
}
warning();
}
}
void student_rec::operator delete (void *d)
{
if(d == NULL)
return;
for(int i=0; i<MAX; i++)
{
if(d == &m[i].obj)
{
m[i].status = FREE;
strcpy(m[i].obj.name, "");
m[i].obj.roll_no = 0;
m[i].obj.percentage = 0.0;
}
}
}
int main()
{
student_rec *sr1, *sr2, *sr3, *sr4, *sr5, *sr6, *sr7;
sr1 = new student_rec("sandeep", 21, 78);
sr1->show_rec();
sr2 = new student_rec("sachin", 21, 78);
sr2->show_rec();
sr3 = new student_rec("sapna", 21, 78);
sr3->show_rec();
sr4 = new student_rec("vipin", 21, 78);
sr4->show_rec();
sr5 = new student_rec("niraj", 21, 78);
sr5->show_rec();
sr6 = new student_rec; // error all memory occupied.
return 0;
}
I run this code on linux machine.
This is terrible code. It is totally unaware of the C++ object model. Forget it and start with a good introductory book, that explains the object lifecycle, and how to properly create new objects.
More explanations about what goes wrong: flaw 1
The problem is in student_rec::operator new (). This line:
m = (memory_store *) malloc(sz * MAX);
let you think that m points to some valid array of memory_store objects. Unfortunately, the C malloc() is used to allocate raw memory. There are thus no valid objects in that memory. Otherwise said, the objects pointed to by m are in an unknown dirty state.
Later, the line
m[i].status = FREE;
handles the objects pointed by m as if they were already valid. This is undefined behavior. If you don't allocate objects the C++ way (e.g. new instead of malloc() ) you would first need to create them with a placement new.
Now for your simple object trivial object this will not cause too many damages. There's yet another flaw.
Even more explanations about what goes wrong: fatal flaw 2
There is a second serious problem: malloc only allocates sz * MAX bytes. As the operator is overloaded for student_rec, it will be called with sz being sizeof(student_rec). But your code assumes that it is sizeof(memory_store), so that the allocated memory is at least sizeof(int)*n bytes too short !!
This is why increasing MAX (and thus allocating more memory than needed for your 5 objects) seems to work.
Other remarks
Using global variables as you did, exposing m to the outside world, is very dangerous and error prone. Suppose that in some other functions you'd like to use a local variable m, but forget to declare it; you could corrupt your data structure much faster than you'd expect ! You'd better make it a private static member of student_rec.
Forget about fixed char arrays for storing C strings. If a name would be longer than expected, you'd get another serious problem that is difficult to spot (strcpy could result in memory corruption in such case). If you code in C++, take advantage of string in order not to worry about such details :-)
Stylistic remark: why not make flag a boolean and use true & false instead of 0 and 1 ?
Stylistic remark: The warning() function has a misleading name: warning() suggests that you issue a warning and continue. Why not giving it a self-documenting name like for example fatal_error() or warning_and_exit()

c++ read 3 columns from a text file into a 2D array or 3 individual arrays

I have a text file looks like this:
173865.385 444879.102 0.299
173864.964 444879.137 0.467
173864.533 444879.177 0.612
173864.113 444879.211 0.798
173863.699 444879.244 1.002
173863.27 444879.282 0.926
173862.85 444879.317 0.974
....
....
....(around 200000 rows)
I'm trying to put each column into one array.
now I have these cripts:
int ReadDataFromFile(double * DataList[] ,int DataListCount,string &FileName)
{
ifstream DataFile;
int CurrentDataIndex = 0;;
DataFile.open(FileName.c_str(),ios::in);
if(DataFile.is_open()==true)
{
char buffer[200];
while(DataFile.getline(buffer,200))
{
string strdata;
stringstream ss(buffer);
for(int i =0;i<DataListCount;++i)
{
getline(ss,strdata,' ');
DataList[i][CurrentDataIndex] = strtod(strdata.c_str(),NULL);
}
++CurrentDataIndex;
}
}
return CurrentDataIndex;
}
int _tmain(int argc, _TCHAR* argv[])
{
double a[200000],b[200000],c[200000];
double* DataList[] = {a,b,c};
int DataCount = ReadDataFromFile(DataList,3,string("D:\\read\\k0_test.txt"));
for(int i=0;i<DataCount;++i)
{
cout<<setw(10)<<a[i]<<setw(10)<<b[i]<<setw(10)<<c[i]<<endl;
}
system("pause");
return 0;
}
But it always tell an error "overflow". Is there any other way to relize this problem?
double a[200000],b[200000],c[200000];
used out all your program's stack space, try to use std::vector (preferred) or use dynamic array, which allocates memory on the heap instead.
For example: (for a only)
vector<double> a;
a.reserve(200000);
Or
vector<double> a(200000);
if use dynamic array:
double* a = new double[200000];
Don't forget to release memory when you are done using them:
delete [] a;
See STL Vector for more information.
2 solutions:
Move double a[200000],b[200000],c[200000]; outside the _tmain so that they can be global variables.
Or,
Declare a,b,c as:
double *a = new double[200000];
double *b = new double[200000];
double *c = new double[200000];
and don't forget to free them by delete[]
Hope this helps :)