remove CUDA dependency? - c++

Question

An open source C++/Qt app I'm interested in depends on CUDA. My macbook pro (mid 2014) has the stock Intel Iris Pro, and no NVidia graphics card. Naturally, the pre-built app won't run.
I found this emulator: https://github.com/gtcasl/gpuocelot - but it's only tested against Linux, and there are several open issues about it not compiling on the Mac.
I have the source - can I replace the CUDA dependency with c++ equivalents, at the cost of slower processing? I'm hoping for something like
rename file extensions: .cu to .cpp
remove CUDA references from make file
replace CUDA headers with equivalent c++ std lib headers
adjust makefile, adding missing library references as needed
fix remaining missing function calls (hopefully only one or two) with c++ code (possibly cribbed from Ocelot)
But I'm afraid it's not that simple. I'd like a sanity check before I begin.

Answer 1

In the general case, I don't think there is a specific roadmap to "de-CUDA-fy" an application. Just as I don't think there is a specific "mechanical" roadmap to "CUDA-fy" an application, nor do I find specific roadmaps for programming problems in general.
Furthermore, I think the proposed roadmap has flaws. To pick just one example, a .cu file will normally have CUDA-specific references that will not be tolerated by an ordinary c++ compiler used to compile a .cpp code. Some of these references may be items that depend on the CUDA runtime API, such as cudaMalloc and cudaMemcpy, and although these could be made to pass through an ordinary c++ compiler (they are just library calls) it would not be sensible to leave those in-place for an application that has the CUDA character removed. Furthermore, some of the references may be CUDA specific language features such as declaration of device code via __global__ or __device__ or launching of a device "kernel" function with it's corresponding syntax <<<...>>>. These cannot be made to pass through an ordinary c++ compiler, and would have to be dealt with specifically. Furthermore, simply deleting those CUDA keywords and syntax would be very unlikely to produce useful results.
In short, the code would have to be refactored; there is no reasonably concise roadmap that explains a more-or-less mechanical process to do so. I suggest the complexity of the refactoring process would be approximately the same complexity as the original process (if there was one) to convert a non-CUDA version of the code to a CUDA version. At a minimum, some non-mechanical knowledge of CUDA programming would be required in order to understand the CUDA constructs.
For very simple CUDA codes, it might be possible to lay out a somewhat mechanical process to de-CUDA-fy the code. To recap, the basic CUDA processing sequence is as follows:
allocate space for data on the device (perhaps with cudaMalloc) and copy data to the device (perhaps with cudaMemcpy)
launch a function that runs on the device (a __global__ or "kernel" function) to process the data and create results
copy results back from the device (perhaps, again, with cudaMemcpy)
Therefore, a straightforward approach would be to:
eliminate the cudaMalloc/cudaMemcpy operations, thus leaving the data of interest in its original form, on the host
convert the cuda processing functions (kernels) to ordinary c++ functions, that perform the same operation on the host data
Since CUDA is a parallel processing architecture, one approach to convert an inherently parallel CUDA "kernel" code to ordinary c++ code (step 2 above) would be to use a loop or a set of loops. But beyond that the roadmap tends to get quite divergent, depending on what the code is actually doing. In addition, inter-thread communication, non-transformational algorithms (such as reductions), and use of CUDA intrinsics or other language specific features will considerably complicate step 2.
For example lets take a very simple vector ADD code. The CUDA kernel code for this would be distinguished by a number of characteristics that would make it easy to convert to or from a CUDA realization:
There is no inter-thread communication. The problem is "embarassingly parallel". The work done by each thread is independent of all other threads. This describes only a limited subset of CUDA codes.
There is no need or use of any CUDA specific language features or intrinsics (other than a globally unique thread index variable), so the kernel code is recognizable as almost completely valid c++ code already. Again, this characteristic probably describes only a limited subset of CUDA codes.
So the CUDA version of the vector add code might look like this (drastically simplified for presentation purposes):
#include <stdio.h>
#define N 512
// perform c = a + b vector add
__global__ void vector_add(const float *a, const float *b, float *c){
int idx = threadIdx.x;
c[idx]=a[idx]+b[idx];
}
int main(){
float a[N] = {1};
float b[N] = {2};
float c[N] = {0};
float *d_a, *d_b, *d_c;
int dsize = N*sizeof(float);
cudaMalloc(&d_a, dsize); // step 1 of CUDA processing sequence
cudaMalloc(&d_b, dsize);
cudaMalloc(&d_c, dsize);
cudaMemcpy(d_a, a, dsize, cudaMemcpyHostToDevice);
cudaMemcpy(d_b, b, dsize, cudaMemcpyHostToDevice);
vector_add<<<1,N>>>(d_a, d_b, d_c); // step 2
cudaMemcpy(c, d_c, dsize, cudaMemcpyDeviceToHost); // step 3
for (int i = 0; i < N; i++) if (c[i] != a[i]+b[i]) {printf("Fail!\n"); return 1;}
printf("Success!\n");
return 0;
}
We see that the above code follows the typical CUDA processing sequence 1-2-3 and the beginning of each step is marked in the comments. So our "de-CUDA-fy" roadmap is, again:
eliminate the cudaMalloc/cudaMemcpy operations, thus leaving the data of interest in its original form, on the host
convert the cuda processing functions (kernels) to ordinary c++ functions, that perform the same operation on the host data
For step 1, we will literally just delete the cudaMalloc and cudaMemcpy lines, and we will instead plan to operate directly on the a[], b[] and c[] variables in the host code. The remaining step, then, is to convert the vector_add CUDA "kernel" function to an ordinary c++ function. Again, some knowledge of CUDA fundamentals is necessary to understand the extent of the operation being performed in parallel. But the kernel code itself (other than the use of the threadIdx.x built-in CUDA variable) is completely valid c++ code, and there is no inter-thread communication or other complicating factors. So an ordinary c++ realization could just be the kernel code, placed into a suitable for-loop iterating over the parallel extent (N in this case), and placed into a comparable c++ function:
void vector_add(const float *a, const float *b, float *c){
for (int idx=0; idx < N; idx++)
c[idx]=a[idx]+b[idx];
}
Combining the above steps, we need to (in this trivial example):
delete the cudaMalloc and cudaMemcpy operations
replace the cuda kernel code with a similar, ordinary c++ function
fixup the kernel invocation in main to be an ordinary c++ function call
Which gives us:
#include <stdio.h>
#define N 512
// perform c = a + b vector add
void vector_add(const float *a, const float *b, float *c){
for (int idx = 0; idx < N; idx++)
c[idx]=a[idx]+b[idx];
}
int main(){
float a[N] = {1};
float b[N] = {2};
float c[N] = {0};
vector_add(a, b, c);
for (int i = 0; i < N; i++) if (c[i] != a[i]+b[i]) {printf("Fail!\n"); return 1;}
printf("Success!\n");
return 0;
}
The point of working through this example is not to suggest the process will be in general this trivially simple. But hopefully it is evident that the process is not a purely mechanical one, but depends on some knowledge of CUDA and also requires some actual code refactoring; it cannot be done simply by changing file extensions and modifying a few function calls.
A few other comments:
Many laptops are available which have CUDA-capable (i.e. NVIDIA) GPUs in them. If you have one of these (I realize you don't but I'm including this for others who may read this), you can probably run CUDA codes on it.
If you have an available desktop PC, it's likely that for less than $100 you could add a CUDA-capable GPU to it.
Trying to leverage emulation technology IMO is not the way to go here, unless you can use it in a turnkey fashion. Cobbling bits and pieces from an emulator into an application of your own is a non-trivial exercise, in my opinion.
I believe in the general case, conversion of a CUDA code to a corresponding OpenCL code will not be trivial either. (The motivation here is that there is a lot of similarity between CUDA and OpenCL, and an OpenCL code could probably be made to run on your laptop, as OpenCL codes can usually be run on a variety of targets, including CPUs and GPUs). There are enough differences between the two technologies that it requires some effort, and this brings the added burden of requiring some level of familiarity with both OpenCL and CUDA, and the thrust of your question seems to be wanting to avoid those learning curves.