I need to store an array on heap since I got a seg fault when running the program, due to it being too large. Normally this would be easy, but in this case it is a multidimensional array (3D specifically) and it's a variable length array too.
I tried to fit this answer for a 2D array (which I'm pretty sure works because I found it on another answer on SO) into one for a 3D array
int **ary = new int*[sizeY];
for(int i = 0; i < sizeY; ++i) {
ary[i] = new int[sizeX];
}
by doing this:
double **isoarray = new double*[nBinsX];
for(int xi = 0; xi < nBinsX; ++xi){
isoarray[xi] = new double[nBinsY];
for(int yi = 0; yi < nBinsY; ++yi){
isoarray[xi][yi] = new double[nShuffles];
}
}
Where I should mention that the array is meant to have dimensions nBinsX x nBinsY x nShuffles, but it isn't working, nor did I really think it would to be honest. Can anyone offer a suggestion on how I would do this? Thanks in advance!
Rather than heap-allocating arrays of pointers to more heap-allocated arrays and so on, you should make a single giant allocation and do appropriate indexing. This is a common technique:
double *isoarray = new double[nBinsX * nBinsY * nShuffles];
If you want to make a nice C++ solution out of it, store that pointer in a class which has an indexing method something like this:
double& at(x, y, shuffle) {
return isoarray[x * nBinsY * nShuffles + y * nShuffles + shuffle];
}
This way you have a single contiguous allocation which is better for performance when allocating, when using, and when deallocating. You can play with the indexing in terms of which dimension comes "first" to achieve even better performance depending on which way you usually traverse the data.
Related
I'm making a picture editing program, and I'm stuck in allocating memory.
I have no idea what is going on.
Ok.. So when I do this:
std::vector<unsigned char> h;
for (int a = 0; a < 10000 * 10000 * 3; a++) {
h.push_back(0);
}
this is fine(sorry I had to), but when I do this:
std::vector<std::vector<std::vector<unsigned char>>> h;
for (uint32_t a = 0; a < 10000; a++) {
h.push_back({});
for (uint32_t b = 0; b < 10000; b++) {
h.at(a).push_back({});
for (uint32_t c = 0; c < 3; c++) {
h.at(a).at(b).push_back(0xff);
}
}
}
my memory usage explodes, and I get error: Microsoft C++ exception: std::bad_alloc at memory location 0x009CF51C
I'm working with .bmp.
Currently, code is in testing mode so it's basically a giant mess...
I'm 15, so don't expect much of me.
I was searching for solutions, but all I found was like how to handle large integers and so on...
If you can give me maybe another solution, but I want my code to be as beginner friendly as it can get.
This is due to overhead of vector<char>. Each such object with 3 elements takes not 3 bytes, but probably 4 (due to reallocation policy), plus 3 pointers which probably take 3*8=24 bytes. Overall your structure takes 9.3 times the memory it could have.
If you replace the inner vector with an array, it will start working, since array does not have this overhead:
std::vector<std::vector<std::array<unsigned char, 3>>> h;
for (uint32_t a = 0; a < 10000; a++) {
h.emplace_back();
for (uint32_t b = 0; b < 10000; b++) {
h.at(a).emplace_back();
for (auto &c : h.at(a).at(b)) {
c = 0xff;
}
}
}
Another alternative is to put the smaller dimension first.
My guess would be that the memory is being heavily fragmented by the constant vector reallocation, resulting in madness. For data this large, I would suggest simply storing a 1-dimensional pre-allocated vector:
std::vector h(10000 * 10000 * 3);
And then come up with an array accessing scheme that takes the X/Y arguments and turns them into an index in your 1d array, eg.:
int get_index(int x, int y, int width) {
return ((y * width) + x) * 3;
}
If the image size is always fixed, you can also use std::array (see multi-dimensional arrays), since the size is defined at compile-time and it won't suffer the same memory issues as the dynamically allocated vectors.
I don't know if this will help your problem, but you could try allocating the memory for the vec of vecs of vecs all at the beginning, with the constructor.
std::vector<std::vector<std::vector<unsigned char>>> h(10000, std::vector<std::vector<unsigned char>>(10000, std::vector<unsigned char>(3,0xff)));
BTW, you're getting a good start writing C++ at 15! I didn't start studying computer science till I was in my 20s. It really is a very marketable career path, and there are a lot of intellectually stimulating, challenging things to learn. Best of luck!
I have a 2D array created dynamically.
int **abc = new int*[rows];
for (uint32_t i = 0; i < rows; i++)
{
abc[i] = new int[cols];
}
I want to fill the array with some value (say 1). I can loop over each item and do it.
But is there a simpler way. I am trying to use memset and std::fill_n as mentioned in this post.
std::fill_n(abc, rows * cols, 1);
memset(abc, 1, rows * cols * sizeof(int));
Using memset crashes my program. Using fill_n gives a compile error.
invalid conversion from 'int' to 'int*' [-fpermissive]
What am I doing wrong here ?
You could just use vector:
std::vector<std::vector<int>> abc(rows, std::vector<int>(cols, 1));
You cannot use std::fill_n or memset on abc directly, it simply will not work. You can only use either on the sub-arrays:
int **abc = new int*[rows];
for (uint32_t i = 0; i < rows; i++)
{
abc[i] = new int[cols];
std::fill_n(abc[i], cols, 1);
}
Or make the whole thing single-dimensional:
int *abc = new int[rows * cols];
std::fill_n(abc, rows*cols, 1);
Or I guess you could use std::generate_n in combination with std::fill_n, but this just seems confusing:
int **abc = new int*[rows];
std::generate_n(abc, rows, [cols]{
int* row = new int[cols];
std::fill_n(row, cols, 1);
return row;
});
I think that your main problem here is that you don't have an array of int values. You have an array of pointers to ints.
You probably should start with int* abc = new int[rows * cols]; and work from there, if I understand what you are trying to achieve here.
Just use with * inside the loop you already have:
for (uint32_t i = 0; i < rows; i++)
{
abc[i] = new int[cols];
std::fill_n(*(abc+i), cols, sizeof(int));
}
fill_n don't know where the memory maps the new int array, so you must be carefully coding that way.
I recommend to read:
A proper way to create a matrix in c++
Since you've already got good, workable answers to solve your problem, I want to add just two pointers left and right from the standard path ;-)
a) is just a link to the documentation of Boost.MultiArray
and b) is something I don't recommend you use, but it might help you to understand what you've initially tried. And since your profile shows visual studio tags, you might come in contact with something like this in the win32 api. If that is the case the documentation usually tells you not to use free()/LocalFree()/... on the elements and the "outer" pointer-pointer but to use a specialized function.
(note: I'm not trying to make this code look pretty or clever; it's a mishmash of c and a little c++-ish junk ;-))
const std::size_t rows = 3, cols =4;
int main()
{
std::size_t x,y;
// allocate memory for 0...rows-1 int* pointers _and_ cols*rows ints
int **abc = (int**)malloc( (rows*sizeof(int*)) + cols*rows*sizeof(int) );
// the memory behind abc is large enough to hold the pointers for abc[0...rows-1]
// + the actual data when accessing abc[0...rows-1][0....cols-1]
int* data = (int*)((abc+rows));
// data now points to the memory right after the int*-pointer array
// i.e. &(abc[0][0]) and data should point to the same location when we're done:
// make abc[0] point to the first row (<-> data+(cols*0)), abc[1] point the second row (<-> data+(cols*1)....
for(y=0;y<rows; y++) {
abc[y] = &(data[y*cols]);
}
// now you can use abc almost like a stack 2d array
for(y=0; y<rows; y++) {
for (x=0; x<cols; x++) {
abc[y][x] = 127;
}
}
// and -since the memory block is continuos- you can also (with care) use memset
memset(&abc[0][0], 1, sizeof(int)*rows*cols);
// and with equal care ....
std::fill_n( &(abc[0][0]), rows*cols, 127);
// and get rid of the whole thing with just one call to free
free(abc);
return 0;
}
I am trying to fill a vector with a matrix of values in c++. I'm not very self confident with this procedure (I don't know well about pointers and I don't know if I need it here) however I am trying this
int auxMat[gray.rows][gray.cols];
vector<int> collectionSum;
collectionSum.push_back(auxMat);
When I try to compile I receive an error which says
invalid arguments 'Candidates are: void push_back(const int &)
Can anyone tell me wether it's possible to do, how can I solve it?
I read something about erasing cache memory, changing my eclipse compiler, my c++ version, however I don't think the problem is so big.
You cannot push back a matrix into a vector. What you can do is preallocate memory for your vector (for speeding things up) then use the std::vector<>::assign member function to "copy" from the matrix into the vector:
vector<int> collectionSum(gray.rows * gray.cols); // reserve memory, faster
collectionSum.assign(*auxMat, *auxMat + gray.rows * gray.cols);
This should be pretty fast. Otherwise, you can push back each individual element in a loop.
EDIT
See May I treat a 2D array as a contiguous 1D array? for some technicalities regarding possible undefined behaviour (thanks #juanchopanza for the comment). I believe the code is safe, due to the fact that the storage of the matrix is contiguous.
Because the array auxMat is continuous in memory, you can just copy it directly from memory into your vector. Here, you are telling the vector constructor to copy from the start of auxMat until its end in memory using pointer arithmetic:
std::vector<int> collectionSum(auxMat, auxMat + (gray.rows * gray.cols));
EDIT:
Sorry, I read your question as being a 1D array (int*) rather than a 2D (int**) array. I honestly recommend switching over to a 1D array because often it is faster and easier to work with. Depending on whether your using row-first order or column-first order, you can access the element you want by:
elem = y * width + x; // for row-first order
elem = x * height + y; // for column-first order
For instance:
// Create a 3x3 matrix but represent it continuously as a 1D array
const int A[] = {1, 2, 3, 4, 5, 6, 7, 8, 9};
const unsigned width = 3;
const unsigned height = 3;
for (int y = 0; y < height; ++y)
{
for (int x = 0; x < width; ++x)
{
printf("%d ", A[y * width + x]);
}
printf("\n");
}
EDIT: I will improve this question. I will clarify it right in a little days.
first, I am writing a litlle bmp image analyzer. I have the following problem: The image is stored on plain bytes, without format as an array.
The image is 24 bits, and requires 3 bytes per pixel. I have tried with a solution that I have found on this stackoverflow page, but I can not adapt it for structures.
I have tried but it references invalid areas and bytes. Here's my complete code if you want to see it in TinyPaste (just for a better highlighting): The code in TinyPaste
EDIT 1: This code is in C++, I want to translate it to pure C for portability reasons. This is just the example from I taken the idea of convert a linear array to bidimensional. I have tried to adapt it to pure C for structs but I fail.
This snippet was taken from a stackoverflow question that made me think about this
//The resulting array
unsigned int** array2d;
// Linear memory allocation
unsigned int* temp = new unsigned int[sizeX * sizeY];
// These are the important steps:
// Allocate the pointers inside the array,
// which will be used to index the linear memory
array2d = new unsigned int*[sizeY];
// Let the pointers inside the array point to the correct memory addresses
for (int i = 0; i < sizeY; ++i)
{
array2d[i] = (temp + i * sizeX);
}
// Fill the array with ascending numbers
for (int y = 0; y < sizeY; ++y)
{
for (int x = 0; x < sizeX; ++x)
{
array2d[y][x] = x + y * sizeX;
}
}
I adapt it to reference structs, but it fails. I have tried multiplying by three in this line:
array2d[i] = (temp + i * sizeX /* multiply by 3*/);
But it still without work. I have also done the related castings from char to the struct bmp_pixel(char r, char g, char b).
Can somebody tell me how to adapt it to pure C for structs?? Thanks.
Can someone please point out what I am doing wrong in the following code?
int* a = NULL;
int* b = NULL;
a = new int[map->mapSize.width];
b = new int[map->mapSize.height];
layer->tileGids = new int[a][b];
Here's what the code uses:
typedef struct _size {
int width, height;
} size;
class Map {
size mapSize;
}
class Layer {
int * tileGids;
}
EDIT: Compiler-Errors (in line 6 of the first bit of code):
error: expression in new-declarator must have integral or enumeration type|
error: 'b' cannot appear in a constant-expression|
Solution:
I have decided to accept lightalchemist's answer. In essence, what works for me is use a vector instead of the array. Vector manages the memory for you and hence is a lot easier to deal with.
You can't pass a pointer for initializing the size of an array. Others have now mentioned this.
This post (it's not mine) seems like it might help you: http://eli.thegreenplace.net/2003/07/23/allocating-multi-dimensional-arrays-in-c/
You should also consider doing the allocation in the class Layer's constructor and then deleting the memory in it's destructor (i.e. RAII - resource acquisition is initialization). This is considered good style.
Finally, you might consider using continuous memory and a custom indexing scheme, which you could easily use Layer to encapsulate. This of course depends upon how big things will get. The bigger they get the better the case for continuous memory becomes.
This should give you a flavor.
#include <iostream>
#include <cstdlib>
int main()
{
const size_t ROWS = 5;
const size_t COLS = 2;
const size_t size = ROWS*COLS;
int* arr = new int[size];
int i = 0;
for ( size_t r = 0 ; r < ROWS; ++r )
{
for (size_t c = 0; c < COLS; ++c )
{
arr[r*COLS+c] = i++;
}
}
for ( int j = 0; j < i; ++j)
{
std::cout << arr[j] << std::endl;
}
delete [] arr;
}
Firstly, your variables "a" and "b" are pointers. Your code:
layer->tileGids = new int[a][b]
is the root cause of the problem.
I'm trying to guess your intention here and I think what you are trying to do is make layer.tileGids a 2 dimension array to reference a "grid" of size (mapSize.Width, mapSize.height) so that you can refer to each "cell" in the grid using layer.tileGids[x][y].
If you are indeed trying to create a 2 dimension array, there are 2 methods to do it.
Method 1:
class Layer {
int ** tileGids; // NOTE the "**" to indicate tileGids is a pointer to pointer i.e. 2D array.
}
To initialize it:
int width = map->mapSize.width;
int height = map->mapSize.height;
layer.tileGids = new int*[width]; // NOTE the "int*" to indicate tileGids is a new array of pointers to int.
for (int i = 0; i < width; i++) // Initialize each element in layer.tileGids[] to be a pointer to int.
{
layer.tileGids[i] = new int[height];
}
Now you can access the items in layer.tileGids using:
int value = layer.tileGids[x][y] // where 0 <= x < width and 0 <= y < height
To deallocate this data structure, similar to how you allocate it, you need to deallocate each dynamically allocated array in each "row":
for (int i = 0; i < width; i++)
{
delete [] layer.tileGids[i]; // Deallocate each row.
}
delete [] layer.tileGids; // Deallocate "array" to the pointers itself.
Method 2:
Now another easier, less messy method (avoid pointers) is to use the C++ vector class. You need to make the following changes:
#include <vector>
class Layer {
vector<vector<int> > tileGids; // Note the space at "<int> >".
}
To initialize:
int width = map->mapSize.width;
int height = map->mapSize.height;
layer.tileGids = vector<vector<int> >(width, vector<int>(height, 0)); // Initialize all entries to 0.
To access the elements:
int value = layer.tileGids[x][y]; // Where 0 <= x < width and 0 <= y < height
Note that for the second method using vectors, you do not have to do any memory cleanup as is required in the first method because the vector will automatically take care of it. However, because a vector can grow dynamically i.e. you can add items to it, you lose the safety of having a fixed size array i.e. someone could accidentally increase the size of your grid if you use the vector method but if he tries to do that when you intialized it using the first method above an error will occur and you will immediately know that something is wrong.
Can someone please point out what I am doing wrong in the following code?
A lot. You're allocating two single arrays (a "row array" and a "column array", not what you need), and then you try to do something strange.
Generally you can't (strictly speaking) dynamically allocate a 2D array in C++ (because the type system would still need the type, along with the dimensions, to be known at compile time). You can emulate it with an array of arrays or so, but the best way is to allocate an 1D array:
int width=5;
std::vector<int> tab(width*height);
and then access the element by calculating the coordinates manually:
// access the element (1,2)
tab[1 + 2*width] = 10;
This way you're essentially interpreting a 1D array as a 2D array (with performance equal to static 2D arrays).
Then it's best to wrap the indexing with a class for convenience; boost::multi_array also has this done for you already.
a and b are int* here:
layer->tileGids = new int[a][b];
Perhaps you meant to say this?
layer->tileGids = new int[*a][*b];