CUDA: using unified memory together with classes and arrays - c++

Question

I'm trying to get unified memory to work with classes, and to pass and manipulate arrays in unified memory with kernel calls. I want to pass everything by reference.
So I'm overriding the new method for classes and arrays so they are accessible by the GPU, but I think I need to add more code to have arrays in unified memory, but not quite sure how to do this. I get a memory access error when the fillArray() method is called.
If I have to do these sorts of operations (arithmetic on arrays and copying between different sized arrays) hundreds of times, is unified memory a good approach or should I stick with manually copying between cpu and gpu memory? Thank you very much!
#include "cuda_runtime.h"
#include "device_launch_parameters.h"
#include <iostream>
#include <stdio.h>
#define TILE_WIDTH 4
#ifdef __CUDACC__
#define CUDA_CALLABLE_MEMBER __host__ __device__
#else
#define CUDA_CALLABLE_MEMBER
#endif
__global__ void add1(int height, int width, int *a, int *resultArray)
{
int w = blockIdx.x * blockDim.x + threadIdx.x; // Col // width
int h = blockIdx.y * blockDim.y + threadIdx.y;
int index = h * width + w;
if ((w < width) && (h < height))
resultArray[index] = a[index] + 1;
}
class Managed
{
public:
void *operator new(size_t len)
{
void *ptr;
cudaMallocManaged(&ptr, len);
return ptr;
}
void Managed::operator delete(void *ptr)
{
cudaFree(ptr);
}
void* operator new[] (size_t len) {
void *ptr;
cudaMallocManaged(&ptr, len);
return ptr;
}
void Managed::operator delete[] (void* ptr) {
cudaFree(ptr);
}
};
class testArray : public Managed
{
public:
testArray()
{
height = 16;
width = 8;
myArray = new int[height*width];
}
~testArray()
{
delete[] myArray;
}
CUDA_CALLABLE_MEMBER void runTest()
{
fillArray(myArray);
printArray(myArray);
dim3 dimGridWidth((width - 1) / TILE_WIDTH + 1, (height - 1)/TILE_WIDTH + 1, 1);
dim3 dimBlock(TILE_WIDTH, TILE_WIDTH, 1);
add1<<<dimGridWidth,dimBlock>>>(height, width, myArray, myArray);
cudaDeviceSynchronize();
printArray(myArray);
}
private:
int *myArray;
int height;
int width;
void fillArray(int *myArray)
{
for (int i = 0; i < height; i++){
for (int j = 0; j < width; j++)
myArray[i*width+j] = i*width+j;
}
}
void printArray(int *myArray)
{
for (int i = 0; i < height; i++){
for (int j = 0; j < width; j++)
printf("%i ",myArray[i*width+j]);
printf("\n");
}
}
};
int main()
{
testArray *test = new testArray;
test->runTest();
//testArray test;
//test.runTest();
system("pause");
return 0;
}

Answer 1

I want to pass everything by reference so there's no copying.
__global__ void add1(int height, int width, int *&a, int *&resultArray)
Passing a pointer by reference has one use: to modify (reseat) the pointer in the caller's scope. Which you do not do. So the references are, in this case, superfluous. In fact, it's a pessimization, because you're introducing another level of indirection. Use the following signature instead:
__global__ void add1(int height, int width, int* a, int* resultArray)
This compiles and runs, but it seems that the +1 operation never occurs. Why is this?
I know I should have catch error statements, this code is just a simple example.
Well, it's really unfortunate, because adding proper error checking would probably have helped you find the error. In the future, consider adding error checking before asking on SO.
Your kernel expects its arguments to be in an address space it can access. That means it must be a pointer that was obtained through a call to any of the cudaMalloc variants.
But what are you passing?
myArray = new int[height*width]; // Not a cudaMalloc* variant
[...]
add1<<<dimGridWidth,dimBlock>>>(height, width, myArray, myArray);
Therefore the pointer you pass to your kernel has no meaning, because it is not in a "CUDA address space". Your kernel probably segfaults immediately.
I think your confusion may arise from the fact that the enclosing class of myArray (testArray) inherits from Managed. This means that new testArray will allocate a testArray in GPU-accessible address space, but it doesn't mean that using operator new on that class members will allocate them in that address space, too. They too need to be allocated through cudaMalloc* (for example, although not required, through an overloaded operator new that forwards the allocation to cudaMallocManaged). A simple solution is to allocate your array not with new but like this:
cudaMallocManaged(&myArray, width * height* sizeof(*myArray));
Replace the corresponding call to delete with cudaFree.
Additionally:
testArray test;
This does not allocate test on GPU-accessible space, because it is not allocated through operator new.