MPI send revc pointers - c++

I have a pointer to a list of pointers (each pointer of the list point to a row)
I need to "scatter" the list of pointers so that each processor has a certain number of rows.
I make an example to say how I want to assign the pointers.
If the list is composed by 5 pointers and there are 2 processors, I want that processor0 has pointers 4 0 1 2 3 and processor1 has 2 3 4 0 (this mean that each processor has the last pointer of the previous processor and the first pointer of the following processor)
This is part of the code:
int **vptr = NULL;
if(rank==0){
vptr = m.ptr();
}
//this definition comes from one of my class methods
then I have this part of code that decide how to assign the rows to each process (assuming at the outset that each processor has only the rows that others do not have)
int *elem;
elem = new int[p]; //number of rows for process
int *disp;
disp = new int[p]; //index first row of the process
int split = N / p;
int extra = N % p;
for(unsigned i = 0; i < extra; i++){
elem[i] = split + 1;
}
for(unsigned i = extra; i < p; i++){
elem[i] = split;
}
disp[0] = 0;
for(unsigned i = 1; i < p; i++){
disp[i] = disp[i-1] + elem[i-1];
}
int local_n = elem[rank]; //number of rows for this process
int local_f = disp[rank]; //index first row for this process
int *local_v;
local_v = new int[local_n + 2]; //+2 because now I consider that I also need the row above and the row below
here I need to use MPI_Send and MPI_Recv, I suppose I am making an erroror with the pointers
if(rank==0){
for(unsigned j = 0; j < local_n + 2; j++){
local_v[j] = *vptr[j];
}
for(unsigned i = 1; i < p; i++){
MPI_Send(&vptr[disp[i]-1], elem[i] + 2, MPI_INT, i, 1, MPI_COMM_WORLD);
}
}else{
MPI_Recv(&local_v[0], local_n + 2, MPI_INT, 0, 1, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
}

Comment converted to answer in the pursuit of vainglorious reputation ... (and the slightly more noble pursuit of providing an acceptable answer for future generations)
I'm not sure I entirely understand your code but there is no point sending pointers from one process to another. Pointers point to locations in the local address space of a process and can not be expected to point to a specific location in the local address space of another process. Indeed, they cannot be expected to continue to point to any location in the local address space of another process

Related

Program works fine with small arrays, but it is suspended when the arrays get larger. How to allocate memory to large arrays?

I'll explain my code first before going into my question in more details.
The program will continue to count the next number until the nth number is generated, and then print out that number.
Here's how it works.
With a given sequence of starting numbers, for example, 0,3,6
The 1st number is 0.
The 2nd number is 3.
The 3rd number is 6.
Now, consider the last number, 6. Since that was the first
time the number had been spoken, the 4th number spoken is 0.
(if the last number has been spoken before, then, the next number
is the difference between the turn number when it was last spoken
and the turn number of the time it was most recently spoken before
then. )
since the last number, which is the 4th number (0) has been
spoken before, the most recent place where 0 appears before the last
number is turn 1. Therefore, the 5th number is 4 - 1, which is 3.
...keep counting until the nth number.
My code works fine when n is 2022, but the program stop running when n = 30,000,000
The is how I allocate memory to my arrays
int *test_case_one = new int[30000000];
Below is my entire code
#include <iostream>
#include <string>
using namespace std;
void test_cases( int Array[]);
int isfound( int table[], int current, int range);
int main()
{
int *test_case_one = new int[30000000];
test_case_one[0] = 1;
test_case_one[1] = 3;
test_case_one[2] = 2;
test_cases(test_case_one);
delete[] test_case_one;
}
void test_cases( int Array[])
{
int *table = new int[30000000];
int turn;
int last;
table[0] = Array[0];
table[1] = Array[1];
table[2] = Array[2];
table[3] = 0;
for ( int i = 4; i < 30000000; i++)
{
last = table[i -1];
turn = isfound(table, last, i);
if ( turn != -1) {
table[i] = (i-1) - turn;
}
else {
table[i] = 0;
}
}
cout<< table[29999999] << endl;
delete[] table;
}
int isfound( int table[], int last, int range)
{
for ( int j = range-1; j > 0 ; j--) {
if ( last == table[j -1]) {
return (j - 1);
}
}
return -1;
}
How can I fix this memory overload issue?
You should not build a table of the values, but a table of last rank per value, initialized to 0. That way when you get a value, you have a direct access to next value and you algo become simply linear.
If you are sure that none of the initialization values will be greater than the expected number of iterations, then everything is fine because as other values will be index differences, this will also be less than that number.
Here is a simple code for your 0-3-6 example:
#include <iostream>
int main()
{
int number;
std::cout << "Total number (>=7): ";
std::cin >> number;
if (!std::cin || number < 5) return EXIT_FAILURE;
// 3 initial values are 0, 3, 6, so 4th will be 0
int* data = new int[number];
for (int i = 0; i < number; i++) data[i] = 0;
data[0] = 1;
data[3] = 2;
int val = 6;
for (int index = 3; index <= number; index++) {
int newval = data[val] ? index - data[val] : 0;
data[val] = index;
val = newval;
// uncomment next line to see intermediary values
//printf("%d: %d\n", index + 1, val);
}
delete[] data; // always release dynamic objects...
printf("Final %d: %d\n", number, val);
return 0;
}
BTW, in modern C++ you should rarely directly allocate a raw array with new, precisely because if you do, you will be responsible for its deletion. It is much more common and easy to use standard containers, here a std::vector. (Thanks to #EtiennedeMartel for the remark).

The array in my singleton isn't keeping information after leaving a function and then it crashes when it tries to access the information again

I have an array called int **grid that is set up in Amazon::initGrid() and is made to be a [16][16] grid with new. I set every array value to 0 and then set [2][2] to 32. Now when I leave initGrid() and come back in getGrid() it has lost its value and is now 0x0000.
I don't know what to try, the solution seems to be really simple, but I'm just not getting it. Somehow the data isn't being kept in g_amazon but I could post the code.
// Returns a pointer to grid
int** Amazon::getGridVal()
{
char buf[100];
sprintf_s(buf, "Hello %d\n", grid[2][2]);
return grid;
}
int Amazon::initGrid()
{
int** grid = 0;
grid = new int* [16];
for (int i = 0; i < 16; i++)
{
grid[i] = new int[16];
for (int j = 0; j < 16; j++)
{
grid[i][j] = 0;
}
}
grid[2][2] = 32;
return 0;
}
int **grid;
g_amazon = Amazon::getInstance();
g_amazon->initGrid();
grid = g_amazon->getGridVal();
for (int i = 0; i < 16; i++)
{
for (int j = 0; j < 16; j++)
{
int index;
index = (width * 4 * i) + (4 * j);
int gridval;
gridval = grid[i][j];
lpBits[index] = gridval;
lpBits[index + 1] = gridval;
lpBits[index + 2] = gridval;
}
}
It crashes when I run it at the line where sprintf_s prints out [2][2] and it also crashes when I get to gridval = grid[i][j] because it's at memory location 0x000000.
The variable
int** grid
in the initGrid() function is a local variable. Edit** When the function returns the variable is popped off the stack. However, since it was declared with the new operator the memory still exists on the heap; it is simply just not pointed to by your global grid variable.
#Dean said in comment:
I have grid as an int** grid; in class Amazon {}; so shouldn't it stay in memory or do I need a static var.
That is the problem:
local int **grid; on Amazon::initGrid::
is masking
member int **grid; on Amazon::
as the first context has higher priority in name lookup.
So initGrid() allocates memory referenced only by a local pointer. That pointer no longer exists when you return from this function, Amazon::grid was never touched on initialization and you're also left with some bad memory issues.
So, as commented by #Remy-Lebeau, I also suggest
Consider using std::vector> or std::array, 16> instead. There is no good reason to use new[] manually in this situation.

double free or corruption (out) while deallocating memory in c++

I have a function which returns a 2D array in c++ as follows
float** Input_data(float** train_data, int Nv, int N){
float** x_train=new float*[Nv];
int a = 0,b = 0;
for(a = 1;a<= Nv;a++){
x_train[a] = new float[N+1];
for(b = 1; b <= N+1; b++){
if(b == 1){
x_train[a][b] = 1;
}else{
x_train[a][b] = train_data[a][b-1];
}
}return x_train;}
the purpose of the above code is to add ones in the first column and add remaining data from train_data pointer into x_train. after processing and using x_train i am trying to deallocate x_train as follows
void destroyx_array(float**x_train,int Nv){
for (int free_x = 1;free_x <= Nv;free_x++){
delete[] x_train[free_x];}delete[] x_train;}
and calling the destroy function as follows
destroyx_array(x_train,Nv)
the Input_data functions works fine but when i try to destroy_x_array it gives me double free or corruption(out) aborted (core dumped) can anybody explain what wrong i am doing ? thank you
Simply put, your code corrupts memory. The best thing is to not use raw pointers and instead use container classes such as std::vector.
Having said that, to fix your current code, the issue is that you're writing beyond the bounds of the memory here:
for(a = 1;a<= Nv;a++)
when a == Nv, you are writing one "row" beyond what was allocated. This looks like a manifestation of attempting to fake 1-based arrays. Arrays in C++ start from 0, not 1. Trying to fake 1-based arrays invariably can lead to bugs and memory corruption.
The fix is to rewrite your function to start from 0, not 1, and ensure your loop iterates to n-1, where n is the total number of rows:
for (a = 0; a < Nv; ++a)
the purpose of the above code is to add ones in the first column and
add remaining data from train_data pointer into x_train
Instead of the loop you wrote to test for the first column, you could simplify this by simply using memcpy:
for (int i = 0; i < Nv; ++i)
{
x_train[i][0] = 1;
memcpy(&x_train[i][1], &train_data[i][0], N * sizeof(float));
}
Thus the entire function would look like this:
float** Input_data(float** train_data, int Nv, int N)
{
float** x_train=new float*[Nv];
for(int a = 0; a < Nv; a++)
x_train[a] = new float[N+1];
for (int a = 0; a < Nv; a++)
{
x_train[i][0] = 1;
memcpy(&x_train[i][1], &train_data[i][0], N * sizeof(float));
}
return x_train;
}

Define an array which the number of rows(clos) is unknown in C++

I have a 2048x2048 matrix of grayscale image,i want to find some points which value are > 0 ,and store its position into an array of 2 columns and n rows (n is also the number of founded points) Here is my algorithm :
int icount;
icount = 0;
for (int i = 0; i < 2048; i++)
{
for (int j = 0; j < 2048; j++)
{
if (iout.at<double>(i, j) > 0)
{
icount++;
temp[icount][1] = i;
temp[icount][2] = j;
}
}
}
I have 2 problems :
temp is an array which the number of rows is unknown 'cause after each loop the number of rows increases ,so how can i define the temp array ? I need the exact number of rows for another implementation later so i can't give some random number for it.
My algorithm above doesn't work,the results is
temp[1][1]=0 , temp[1][2]=0 , temp[2][1]=262 , temp[2][2]=655
which is completely wrong,the right one is :
temp[1][1]=1779 , temp[1][2]=149 , temp[2][1]=1780 , temp[2][2]=149
i got the right result because i implemented it in Matlab, it is
[a,b]=find(iout>0);
How about a std::vector of std::pair:
std::vector<std::pair<int, int>> temp;
Then add (i, j) pairs to it using push_back. No size needed to be known in advance:
temp.push_back(make_pair(i, j));
We'll need to know more about your problem and your code to be able to tell what's wrong with the algorithm.
When you define a variable of pointer type, you need to allocate memory and have the pointer point to that memory address. In your case, you have a multidimensional pointer so it requires multiple allocations. For example:
int **temp = new int *[100]; // This means you have room for 100 arrays (in the 2nd dimension)
int icount = 0;
for(int i = 0; i < 2048; i++) {
for(int j = 0; j < 2048; j++) {
if(iout.at<double>(i, j) > 0) {
temp[icount] = new int[2]; // only 2 variables needed at this dimension
temp[icount][1] = i;
temp[icount][2] = j;
icount++;
}
}
}
This will work for you, but it's only good if you know for sure you're not going to need any more than the pre-allocated array size (100 in this example). If you know exactly how much you need, this method is ok. If you know the maximum possible, it's also ok, but could be wasteful. If you have no idea what size you need in the first dimension, you have to use a dynamic collection, for example std::vector as suggested by IVlad. In case you do use the method I suggested, don't forget to free the allocated memory using delete []temp[i]; and delete []temp;

Array passed by reference in recursive function - last column reinitialized

I have a rather unexpected issue with one of my functions. Let me explain.
I'm writing a calibration algorithm and since I want to do some grid search (non-continuous optimization), I'm creating my own mesh - different combinations of probabilities.
The size of the grid and the grid itself are computed recursively (I know...).
So in order:
Get variables
Compute corresponding size recursively
Allocate memory for the grid
Pass the empty grid by reference and fill it recursively
The problem I have is after step 4 once I try to retrieve this grid. During step 4, I 'print' on the console the results to check them and everything is fine. I computed several grids with several variables and they all match the results I'm expecting. However, as soon as the grid is taken out of the recursive function, the last column is filled with 0 (all the values from before are replace in this column only).
I tried allocating one extra column for the grid in step 3 but this only made the problem worse (-3e303 etc. values). Also I have the error no matter what size I compute it with (very small to very large), so I assume it isn't a memory error (or at least a 'lack of memory' error). Finally the two functions used and their call have been listed below, this has been quickly programmed, so some variables might seem kind of useless - I know. However I'm always open to your comments (plus I'm no expert in C++ - hence this thread).
void size_Grid_Computation(int nVars, int endPoint, int consideredVariable, int * indexes, int &sum, int nChoices)
{
/** Remember to initialize r at 1 !! - we exclude var_0 and var_(m-1) (first and last variables) in this algorithm **/
int endPoint2 = 0;
if (consideredVariable < nVars - 2)
{
for (indexes[consideredVariable] = 0; indexes[consideredVariable] < endPoint; indexes[consideredVariable] ++)
{
endPoint2 = endPoint - indexes[consideredVariable];
size_Grid_Computation(nVars, endPoint2, consideredVariable + 1, indexes, sum, nChoices);
}
}
else
{
for (int i = 0; i < nVars - 2; i++)
{
sum -= indexes[i];
}
sum += nChoices;
return;
}
}
The above function is for the grid size. Below for the grid itself -
void grid_Creation(double* choicesVector, double** varVector, int consideredVariable, int * indexes, int endPoint, int nVars, int &r)
{
if (consideredVariable > nVars-1)
return;
for (indexes[consideredVariable] = 0; indexes[consideredVariable] < endPoint; indexes[consideredVariable]++)
{
if (consideredVariable == nVars - 1)
{
double sum = 0.0;
for (int j = 0; j <= consideredVariable; j++)
{
varVector[r][j] = choicesVector[indexes[j]];
sum += varVector[r][j];
printf("%lf\t", varVector[r][j]);
}
varVector[r][nVars - 1] = 1 - sum;
printf("%lf row %d\n", varVector[r][nVars - 1],r+1);
r += 1;
}
grid_Creation(choicesVector, varVector, consideredVariable + 1, indexes, endPoint - indexes[consideredVariable], nVars, r);
}
}
Finally the call
#include <stdio.h>
#include <stdlib.h>
int main()
{
int nVars = 5;
int gridPrecision = 3;
int sum1 = 0;
int r = 0;
int size = 0;
int * index, * indexes;
index = (int *) calloc(nVars - 1, sizeof(int));
indexes = (int *) calloc(nVars, sizeof(int));
for (index[0] = 0; index[0] < gridPrecision + 1; index[0] ++)
{
size_Grid_Computation(nVars, gridPrecision + 1 - index[0], 1, index, size, gridPrecision + 1);
}
double * Y;
Y = (double *) calloc(gridPrecision + 1, sizeof(double));
for (int i = 0; i <= gridPrecision; i++)
{
Y[i] = (double) i/ (double) gridPrecision;
}
double ** varVector;
varVector = (double **) calloc(size, sizeof(double *));
for (int i = 0; i < size; i++)
{
varVector[i] = (double *) calloc(nVars, sizeof(double *));
}
grid_Creation(Y, varVector, 0, indexes, gridPrecision + 1, nVars - 1, r);
for (int i = 0; i < size; i++)
{
printf("%lf\n", varVector[i][nVars - 1]);
}
}
I left my barbarian 'printf', they help narrow down the problem. Most likely, I have forgotten or butchered one memory allocation. But I can't see which one. Anyway, thanks for the help!
It seems to me that you have a principal mis-design, namely your 2D array. What you are programming here is not a 2D array but an emulation of it. It only makes sense if you want to have a sort of sparse data structure where you may leave out parts. In your case it looks as if it is just a plain old matrix that you need.
Nowadays it is neither appropriate in C nor in C++ to program like this.
In C, since that seems what you are after, inside functions you declare matrices even with dynamic bounds as
double A[n][m];
If you fear that this could smash your "stack", you may allocate it dynamically
double (*B)[m] = malloc(sizeof(double[n][m]));
You pass such beasts to functions by putting the bounds first in the parameter list
void toto(size_t n, size_t m, double X[n][m]) {
...
}
Once you have clean and readable code, you will find your bug much easier.