I have been trying to implement a simple parallel algorithm using OpenCL c++ bindings (version 1.2).
Roughly here is the c code (no OpenCL):
typedef struct coord{
double _x;
double _y;
double _z;
}__coord;
typedef struct node{
__coord _coord;
double _dist;
} __node;
double input[3] = {-1.0, -2, 3.5};
//nodeVector1D is a 1Dim random array of struct __node
//nodeVectorSize is the Size of the above array (>1,000)
double d = 0.0;
for(int i=0; i < nodeVectorSize; i++){
__node n = nodeVector1D[i];
d += (input[0] - n._coord._x)*(input[0] - n._coord._x);
d += (input[1] - n._coord._y)*(input[1] - n._coord._y);
d += (input[2] - n._coord._z)*(input[2] - n._coord._z);
n._dist = d;
}
I use a MacBook Pro 13" Late 2013, running on Mac Os X Lion.
OpenCL only detects the CPU.
The CPU: an Intel Ivy i5 2.6GHz, has an integrated GPU of 1Gb at 1.6Ghz (Intel HD Graphics 4000).
The maximum detected Group Item Size is 1024 bytes.
When I run the flat code above (with 1024 nodes), it takes around 17 micro seconds.+
When I run its parallel version using OpenCL, C++ library, it takes 10 times as long, around 87 micro seconds
(excluding the program creation, buffer allocation and writing).
What am I doing wrong here?
NB: the OpenCL kernel for this algorithm is obvious to guess, but I can post it if needed.
Thanks in advance.
EDIT N#1: THE KERNEL CODE
__kernel void _computeDist(
__global void* nodeVector1D,
const unsigned int nodeVectorSize,
const unsigned int itemsize,
__global const double* input){
double d = 0.;
int i,c;
double* n;
i = get_global_id(0);
if (i >= nodeVectorSize) return;
n = (double*)(nodeVector1D + i*itemsize);
for (c=0; c<3;c++){
d += (input[c] - n[c])*(input[c] - n[c]);
}
n[3] = d;
}
Sorry for the void pointer arithmetic, but it works (no seg default).
I can also post the OpenCL initialization routine, but I think it's all over the Internet. However, I will post it, if someone asks.
#pmdj: As I said above OpenCL recognizes my CPU, otherwise I wouldn't have been able to run the tests and get the performance results presented above.
#pmdj: OpenCL kernel code, to my knowledge are always written in C. However, I tagged C++ because (as I said above), I'm using the OpenCL C++ bindings.
I finally found the issue.
The problem was that OpenCL on Mac OS X returns the wrong maximum device work group size of 1024.
I tested with various work group sizes and ended up having 200% performance gains when using a work group size of 128 work items per group.
Here is a clearer benchmark picture. IGPU stands for Integrated GPU.
(X-Axis: the array size, Y-Axis: The Time Duration in microseconds)
Related
I was trying to run an OpenCL kernel on both Adreno 630 and my laptop, it turns out that the kernel runs perfectly on mobile but crashes my laptop every single time. I am still trying to figure out the reason by myself. Here's my kernel. I hope you could help me with it, thanks.
__kernel void gen_mapxy( __read_only image2d_t _disp, const float offsetX, __write_only image2d_t _mapxy )
{
const int y = get_global_id(0);
const int local_y = get_local_id(0);
__local short temp[24][1080];
const int imageWidth = get_image_width(_disp);
for(int x = 0; x < imageWidth; ++x)
temp[local_y][x] = 0;
for(int x = imageWidth - 1; x >= 0; --x){
int tempDisp = read_imagei(_disp, sampler_nearest, (int2)(x, y)).x;
int newPos = clamp((int)(x + offsetX * (tempDisp) / 255), 0, imageWidth - 1);
temp[local_y][newPos] = tempDisp;
write_imagef(_mapxy, (int2)(newPos, y), (float4)(x, y, 0, 0));
}
You are using a big local array.
__local short temp[24][1080]
2 byte * 24 * 1080 = 50.6kB. Some desktop GPUs(and their notebook counterparts) have less available local memory limits. For example, GTX 1060 supports the value CL_DEVICE_LOCAL_MEM_SIZE 49152 bytes. But adreno 620, either it is ignoring the array usage silently or supporting larger local arrays because there is a possilibity that local arrays are emulated inside global arrays (limited in hundreds of megabytes) for those chips. If they do support in-chip fast local memory, then there is more possibility of "ignoring" issue or they really doubled local memory limits from last generation of Adrenos.
Even when GPU supports exact value, using all of it will limit thread-level-parallelism on each pipeline, severely reducing potential performance gains, generally.
If last generation of Adreno GPUs are same,
https://compubench.com/device.jsp?benchmark=compu15m&os=Android&api=cs&D=Samsung+Galaxy+S7+%28SM-G930x%29&testgroup=info
this page says
CL_DEVICE_LOCAL_MEM_SIZE
32768
CL_DEVICE_LOCAL_MEM_TYPE
CL_LOCAL
it is fast but it is 32kB so it is ignoring the error or you've missed adding necessary error catching logic in there, or both.
I'm looking to upgrade my graphics card to be able to process the following task in parallel.
As I have no experience in GPU computing, would this task be suitable, and is it possible to estimate the rate at which the processing could be done before I buy?
My project is publicly funded but has a limited budget, so I need to make the right choice.
I have an in-house build camera chip that produces 4x 256x256 images at 100fps. The data is accessed by calling a c function, passing a pointer to an array data of type unsigned short. I can read out the data fast enough into a memory buffer.
Currently the raw data is saved to disk and then processed offline later, but for future lab experiments with this camera I wish to access data derived from the images as the experiment runs.
I have written in c++ using valarray, methods to calculate the derived data, but it is too slow on my current hardware at about 40ms per frame . (I have experimented with optimisation and I have cut the time considerably from >100ms)
If a Frame is denoted by S, the four subframes (in time) are S1,S2,S3,S4.
I must calculate the following images and the image averages, (S1+S2+S3+S4)/4,
Sqrt((S3-S1)^2 + (S4-S2)^2),
arctan(S3-S1/S2-S4)
It seems like a good fit for an operation to be carried out by a GPU. GPUs are better suited than CPUs to performing massive amounts of relatively simple calculations. They are not as efficient when there is logic, or interdependencies between 'threads'. Although this kind of wanders into the 'opinion' area, I'll try to back up my answer with some numbers.
As a quick estimation of the performance you can expect, I made a quick HLSL pixel shader which does your proposed operations (untested - no guarantee of functionality!):
Texture2D S[4] : register(t0);
SamplerState mySampler : register(s0);
struct PS_OUT
{
float4 average : SV_Target0;
float4 sqrt : SV_Target1;
float4 arctan : SV_Target2;
};
PS_OUT main(float2 UV: TEXCOORD0)
{
PS_OUT output;
float4 SSamples[4];
int i;
for (i = 0; i < 4; i++)
{
SSamples[i] = S[i].Sample(mySampler, UV);
}
float4 s3ms1 = SSamples[2] - SSamples[0];
float4 s4ms2 = SSamples[3] - SSamples[1];
output.average = (SSamples[0] + SSamples[1] + SSamples[2] + SSamples[3]) / 4.0;
output.sqrt = sqrt(s3ms1*s3ms1 + s4ms2*s4ms2);
output.arctan = atan(s3ms1 / s4ms2);
return output;
}
When compiling this (fxc /T ps_4_0 example.ps), it gives the estimation of: Approximately 32 instruction slots used.
If you are processing 256x256 (64k pixels) per-frame, that works out to be about 2.1m/frame, or 210m/s, at 100fps. Looking at a chart of GPU performance (Nvidia for example: http://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units), all their GPUs past Geforce 4 (circa 2005), have sufficient speed to acheive this.
Note that, this shader performance is only an estimation, and the listed rates are theoretical maximums, and I'm only accounting for the pixel unit work (although it will be doing the majority of the work). However, with any sufficiently recent video card the FLOPS will far exceed your needs, so you should be able to easily do this on the GPU at 100fps. Assuming you have a PC newer than 2005, you probably already have a video card powerful enough.
In addition to what #MuertoExcobito already wrote you must also account for copying the data to and from the GPU, however in your case this is not much data.
I created a simple thrust-based implementation which can be compiled and run using CUDA 7 like this:
nvcc -std=c++11 main.cu && ./a.out
Averaged over 10000 runs one iteration which includes copying to the GPU, calculating the three result images and copying the results back from the GPU takes 1.79 ms on my computer (Ubuntu 14.04 x64, Intel Xeon#3.6 Ghz, Geforce GTX 680).
The file "helper_math.h" is adapted from the CUDA SDK and can be found here:
https://gist.github.com/dachziegel/70e008dee7e3f0c18656
#include <thrust/device_vector.h>
#include <thrust/iterator/zip_iterator.h>
#include <thrust/transform.h>
#include <vector_types.h>
#include <iostream>
#include <chrono>
#include "helper_math.h"
template<typename T>
struct QuadVec
{
T S1, S2, S3, S4;
QuadVec(const int N) : S1(N), S2(N), S3(N), S4(N){}
};
template<typename T>
struct Result
{
T average, sqrt, arctan;
Result(const int N) : average(N), sqrt(N), arctan(N){}
};
typedef thrust::tuple<float4,float4,float4,float4> QuadInput;
typedef thrust::tuple<float4,float4,float4> TripleOutput;
struct CalcResult : public thrust::unary_function<QuadInput,TripleOutput>
{
__host__ __device__
TripleOutput operator()(const QuadInput& f) const
{
const float4 s3ms1 = thrust::get<2>(f) - thrust::get<0>(f);
const float4 s4ms2 = thrust::get<3>(f) - thrust::get<1>(f);
const float4 sqrtArg = s3ms1*s3ms1 + s4ms2*s4ms2;
const float4 atanArg = s3ms1 / s4ms2;
return thrust::make_tuple((thrust::get<0>(f) + thrust::get<1>(f) + thrust::get<2>(f) + thrust::get<3>(f)) / 4.0f,
make_float4(sqrtf(sqrtArg.x), sqrtf(sqrtArg.y), sqrtf(sqrtArg.z), sqrtf(sqrtArg.w)),
make_float4(atanf(atanArg.x), atanf(atanArg.y), atanf(atanArg.z), atanf(atanArg.w))
);
}
};
int main()
{
typedef thrust::host_vector<float4> HostVec;
typedef thrust::device_vector<float4> DevVec;
const int N = 256;
QuadVec<HostVec> hostFrame(N*N);
QuadVec<DevVec> devFrame(N*N);
Result<HostVec> hostResult(N*N);
Result<DevVec> devResult(N*N);
const int runs = 10000;
int accumulatedDuration = 0;
for (int i = 0; i < runs; ++i)
{
auto start = std::chrono::system_clock::now();
thrust::copy(hostFrame.S1.begin(), hostFrame.S1.end(), devFrame.S1.begin());
thrust::copy(hostFrame.S2.begin(), hostFrame.S2.end(), devFrame.S2.begin());
thrust::copy(hostFrame.S3.begin(), hostFrame.S3.end(), devFrame.S3.begin());
thrust::copy(hostFrame.S4.begin(), hostFrame.S4.end(), devFrame.S4.begin());
thrust::transform(thrust::make_zip_iterator(make_tuple(devFrame.S1.begin(), devFrame.S2.begin(), devFrame.S3.begin(), devFrame.S4.begin())),
thrust::make_zip_iterator(make_tuple(devFrame.S1.end(), devFrame.S2.end(), devFrame.S3.end(), devFrame.S4.end())),
thrust::make_zip_iterator(make_tuple(devResult.average.begin(), devResult.sqrt.begin(), devResult.arctan.begin())),
CalcResult() );
thrust::copy(devResult.average.begin(), devResult.average.end(), hostResult.average.begin());
thrust::copy(devResult.sqrt.begin(), devResult.sqrt.end(), hostResult.sqrt.begin());
thrust::copy(devResult.arctan.begin(), devResult.arctan.end(), hostResult.arctan.begin());
auto duration = std::chrono::duration_cast<std::chrono::microseconds>(std::chrono::system_clock::now() - start);
accumulatedDuration += duration.count();
}
std::cout << accumulatedDuration/runs << std::endl;
return 0;
}
I am learning CUDA from the book "CUDA by example".In the chapter 4 there is a demo to generate Julia fractals.The showcase demonstrates both CPU and GPU versions.I decided to add a time to see the execution speed for both cases and to my great surprise found that the CPU version executes 3 times faster than GPU.
CPU Julia generation total time:
745 milliseconds .
GPU Julia generation total time:
2456 milliseconds .
So what is going on ? It is clear ,at least from the CUDA kernel code that the execution is parallel as is distributed among 1000 block each of which calculates a pixel for 1000x1000 resolution final image.
Here is the source code of the implementation:
#define N 10
#define DIM 1000
typedef unsigned char byte;
struct cuComplex {
float r;
float i;
__host__ __device__ cuComplex( float a, float b ) : r(a), i(b) {}
__host__ __device__ float magnitude2( void ) {
return r * r + i * i;
}
__host__ __device__ cuComplex operator*(const cuComplex& a) {
return cuComplex(r*a.r - i*a.i, i*a.r + r*a.i);
}
__host__ __device__ cuComplex operator+(const cuComplex& a) {
return cuComplex(r+a.r, i+a.i);
}
};
__device__ int juliaGPU(int x , int y){
const float scale =1.3;
float jx = scale * (float)(DIM/2 -x)/(DIM/2);
float jy= scale *(float)(DIM/2 -y)/(DIM/2);
cuComplex c(-0.8 ,0.156);
cuComplex a(jx ,jy);
int i = 0;
for(i=0; i <200;i++){
a = a * a +c;
if(a.magnitude2() >1000){
return 0;
}
}
return 1;
}
__global__ void kernelGPU(byte *ptr){
int x = blockIdx.x;
int y = blockIdx.y;
int offset =x + y * gridDim.x;
int juliaValue =juliaGPU(x , y);
ptr[offset * 4 + 0]=255 * juliaValue;
ptr[offset * 4 + 1]=0;
ptr[offset * 4 + 2]=0;
ptr[offset * 4 + 3]=255 ;
}
struct DataBlock {
unsigned char *dev_bitmap;
};
void juliaGPUTestSample(){
DataBlock data;
CPUBitmap bitmap(DIM,DIM);
byte *dev_bitmap; //memory on GPU
HANDLE_ERROR(cudaMalloc((void**)&dev_bitmap , bitmap.image_size()) );
data.dev_bitmap =dev_bitmap;
dim3 grid(DIM,DIM);
int starTime=glutGet(GLUT_ELAPSED_TIME);
kernelGPU<<<grid ,1 >>>(dev_bitmap);
HANDLE_ERROR(cudaMemcpy(bitmap.get_ptr() , dev_bitmap ,bitmap.image_size() ,cudaMemcpyDeviceToHost ) );
int endTime=glutGet(GLUT_ELAPSED_TIME)-starTime;
printf("Total time %d\n:" ,endTime);
HANDLE_ERROR(cudaFree(dev_bitmap));
bitmap.display_and_exit();
}
int main(void){
juliaGPUTestSample();
return 1;
}
Here is the CPU version :
/// the "cuComplex " struct is the same from above.
int julia (int x , int y){
const float scale = 1.3;
float jx = scale * (float)(DIM/2 -x)/(DIM/2);
float jy = scale * (float)(DIM/2 -y)/(DIM/2);
cuComplex c(-0.8 ,0.156);
cuComplex a(jx ,jy);
int i = 0;
for(i=0; i <200;i++){
a = a * a +c;
if(a.magnitude2() >1000){
return 0;
}
}
return 1;
}
void kernel(unsigned char *ptr){
for(int y = 0 ; y <DIM ;++y){
for(int x = 0 ; x <DIM ; ++x){
int offset =x + y * DIM;
int juliaValue = julia(x , y);
ptr[offset * 4 + 0 ] = juliaValue * 125;
ptr[offset * 4 + 1 ] = juliaValue * x;
ptr[offset * 4 + 2 ] = juliaValue * y;
ptr[offset * 4 + 3 ] = 255 ;
}
}
}
void juliaCPUTestSample(){
CPUBitmap bitmap(DIM ,DIM);
unsigned char *ptr = bitmap.get_ptr();
int starTime=glutGet(GLUT_ELAPSED_TIME);
kernel(ptr);
int endTime=glutGet(GLUT_ELAPSED_TIME)-starTime;
printf("Total time %d\n:" ,endTime);
bitmap.display_and_exit();
}
Update -system configurations :
Windows 7 64bit
CPU - Intel i7 -3770CPU 3.40GHz ,16GB RAM
GPU - NVidia Quadro 4000
Others have noticed this.
First of all, when talking about perf comparisons between CPU and GPU it's a good idea to mention the system configuration including hw platform and software. For example, I ran your code on an HP laptop with a core i7 2.60GHz quad core CPU and a quadro1000M GPU, running RHEL 6.2, and cuda 5.0, and I got a score of 438 for the GPU and 441 for the CPU.
Second, and more importantly, the julia sample in that book is a relatively early example of CUDA coding, and so it's not really oriented towards max performance, but rather to illustrate the concepts that have been discussed so far. That book and various other CUDA tutorial material starts by introducing parallel programming using CUDA at the block level. The indication of this is here:
kernelGPU<<<grid ,1 >>>(dev_bitmap);
The kernel launch parameters <<<grid, 1>>> indicate that a grid of some number (grid, which is 1 million total blocks in this case) blocks will be launched, with each block having a single thread. This immediately reduces the power of a Fermi-class GPU, for example, by a factor of 1/32 compared with launching a grid with a full complement of threads per threadblock. Each SM in a Fermi-class GPU has 32 thread processors, all executing in lockstep. If you launch a block with only 16 threads in it, then 16 thread processors will execute your code and the other 16 thread processors will do nothing (i.e. nothing useful). A threadblock containing only 1 thread will therefore use only 1 out of 32 thread processors, the other 31 being idle.
Therefore this particular code sample is not well-designed to utilize the full parallel capability of the GPU. Given that it is relatively early in the exposition of CUDA concepts in the book, this is understandable; I don't believe it was the authors intent to have this code benchmarked or used as a legitimate representation of how to write fast code on the GPU.
In light of this factor of 1/32, the idea that on your system the CPU is only 3 times faster, and on my system the CPU and GPU have comparable throughput (niether of these being particularly high-performance CUDA GPUs, most likely) I think it shows the GPU in reasonably good light. The GPU is fighting this battle with about 97% of it's capability unused.
I'm writing a simple monte carlo simulation for particle transport. My approach is writing a kernel for CUDA and execute it as a Mathematica function.
Kernel:
#include "curand_kernel.h"
#include "math.h"
extern "C" __global__ void monteCarlo(Real_t *transmission, mint seed, mint pathN) {
curandState rngState;
int index = threadIdx.x + blockIdx.x*blockDim.x;
curand_init(seed, index, 0, &rngState);
if (index < pathN) {
//-------------start one packet run----------------------
float packetWeight = 1.0;
int m = 0;
while(packetWeight > 0.0){
//MONTE CARLO CODE
// Test: still in the sample?
if(z_coordinate > sampleThickness){
packetWeight = 0;
z_coordinate = sampleThickness;
transmission[index]=1;
}
}
}
//-------------end one packet run------------------------
}
}
Mathematica code:
Needs["CUDALink`"];
cudaBM = CUDAFunctionLoad[code,
"monteCarlo", {{_Real, "Output"}, _Integer, _Integer}, 256,
"UnmangleCode" -> False];
pathN = 100000;
result = 0; (*count for transmitted particles*)
For[j = 0, j < 10, j++,
buffer = CUDAMemoryAllocate["Float", 100000];
cudaBM[buffer, 1490, pathN];
resultOneRun = Total[CUDAMemoryGet[buffer]];
result = result + resultOneRun;
];
Everything seems to work so far, but the speed improvement compared to the pure C code without CUDA is neglible. I have two problems:
the curand_init() function is executed by all threads at the beginning of every sumulation step -> can I call this function once for all threads?
the kernel returns to Mathematica a very large array of reals (100 000). I know, that the bottleneck of CUDA is the channel bandwidth between GPU and CPU. I need only the sum of all elements of the list, so it would be more efficient to calculate the sum of the list elements in the GPU and send only one real number to the CPU.
1) If you need to execute curand_init() once for all threads, can you just do that in the CPU and pass that as an argument to CUDA?
2) How about a "device float sumTotal" function which sums and returns your values? Have you copied as much *transmission data into a shared memory buffer?
As per CURAND docs,
"Calls to curand_init() are slower than calls to curand() or curand_uniform(). Large offsets to curand_init() take more time than smaller offsets. It is much faster to save and restore random generator state than to recalculate the starting state repeatedly."
http://docs.nvidia.com/cuda/curand/index.html#topic_1_3_4
Also please look into this thread for more details
CUDA program causes nvidia driver to crash
I'm working on a statistical application containing approximately 10 - 30 million floating point values in an array.
Several methods performing different, but independent, calculations on the array in nested loops, for example:
Dictionary<float, int> noOfNumbers = new Dictionary<float, int>();
for (float x = 0f; x < 100f; x += 0.0001f) {
int noOfOccurrences = 0;
foreach (float y in largeFloatingPointArray) {
if (x == y) {
noOfOccurrences++;
}
}
noOfNumbers.Add(x, noOfOccurrences);
}
The current application is written in C#, runs on an Intel CPU and needs several hours to complete. I have no knowledge of GPU programming concepts and APIs, so my questions are:
Is it possible (and does it make sense) to utilize a GPU to speed up such calculations?
If yes: Does anyone know any tutorial or got any sample code (programming language doesn't matter)?
UPDATE GPU Version
__global__ void hash (float *largeFloatingPointArray,int largeFloatingPointArraySize, int *dictionary, int size, int num_blocks)
{
int x = (threadIdx.x + blockIdx.x * blockDim.x); // Each thread of each block will
float y; // compute one (or more) floats
int noOfOccurrences = 0;
int a;
while( x < size ) // While there is work to do each thread will:
{
dictionary[x] = 0; // Initialize the position in each it will work
noOfOccurrences = 0;
for(int j = 0 ;j < largeFloatingPointArraySize; j ++) // Search for floats
{ // that are equal
// to it assign float
y = largeFloatingPointArray[j]; // Take a candidate from the floats array
y *= 10000; // e.g if y = 0.0001f;
a = y + 0.5; // a = 1 + 0.5 = 1;
if (a == x) noOfOccurrences++;
}
dictionary[x] += noOfOccurrences; // Update in the dictionary
// the number of times that the float appears
x += blockDim.x * gridDim.x; // Update the position here the thread will work
}
}
This one I just tested for smaller inputs, because I am testing in my laptop. Nevertheless, it is working, but more tests are needed.
UPDATE Sequential Version
I just did this naive version that executes your algorithm for an array with 30,000,000 element in less than 20 seconds (including the time taken by function that generates the data).
This naive version first sorts your array of floats. Afterward, will go through the sorted array and check the number of times a given value appears in the array and then puts this value in a dictionary along with the number of times it has appeared.
You can use sorted map, instead of the unordered_map that I used.
Heres the code:
#include <stdio.h>
#include <stdlib.h>
#include "cuda.h"
#include <algorithm>
#include <string>
#include <iostream>
#include <tr1/unordered_map>
typedef std::tr1::unordered_map<float, int> Mymap;
void generator(float *data, long int size)
{
float LO = 0.0;
float HI = 100.0;
for(long int i = 0; i < size; i++)
data[i] = LO + (float)rand()/((float)RAND_MAX/(HI-LO));
}
void print_array(float *data, long int size)
{
for(long int i = 2; i < size; i++)
printf("%f\n",data[i]);
}
std::tr1::unordered_map<float, int> fill_dict(float *data, int size)
{
float previous = data[0];
int count = 1;
std::tr1::unordered_map<float, int> dict;
for(long int i = 1; i < size; i++)
{
if(previous == data[i])
count++;
else
{
dict.insert(Mymap::value_type(previous,count));
previous = data[i];
count = 1;
}
}
dict.insert(Mymap::value_type(previous,count)); // add the last member
return dict;
}
void printMAP(std::tr1::unordered_map<float, int> dict)
{
for(std::tr1::unordered_map<float, int>::iterator i = dict.begin(); i != dict.end(); i++)
{
std::cout << "key(string): " << i->first << ", value(int): " << i->second << std::endl;
}
}
int main(int argc, char** argv)
{
int size = 1000000;
if(argc > 1) size = atoi(argv[1]);
printf("Size = %d",size);
float data[size];
using namespace __gnu_cxx;
std::tr1::unordered_map<float, int> dict;
generator(data,size);
sort(data, data + size);
dict = fill_dict(data,size);
return 0;
}
If you have the library thrust installed in you machine your should use this:
#include <thrust/sort.h>
thrust::sort(data, data + size);
instead of this
sort(data, data + size);
For sure it will be faster.
Original Post
I'm working on a statistical application which has a large array
containing 10 - 30 millions of floating point values.
Is it possible (and does it make sense) to utilize a GPU to speed up
such calculations?
Yes, it is. A month ago, I ran an entirely Molecular Dynamic simulation on a GPU. One of the kernels, which calculated the force between pairs of particles, received as parameter 6 array each one with 500,000 doubles, for a total of 3 Millions doubles (22 MB).
So if you are planning to put 30 Million floating points, which is about 114 MB of global Memory, it will not be a problem.
In your case, can the number of calculations be an issue? Based on my experience with the Molecular Dynamic (MD), I would say no. The sequential MD version takes about 25 hours to complete while the GPU version took 45 Minutes. You said your application took a couple hours, also based in your code example it looks softer than the MD.
Here's the force calculation example:
__global__ void add(double *fx, double *fy, double *fz,
double *x, double *y, double *z,...){
int pos = (threadIdx.x + blockIdx.x * blockDim.x);
...
while(pos < particles)
{
for (i = 0; i < particles; i++)
{
if(//inside of the same radius)
{
// calculate force
}
}
pos += blockDim.x * gridDim.x;
}
}
A simple example of a code in CUDA could be the sum of two 2D arrays:
In C:
for(int i = 0; i < N; i++)
c[i] = a[i] + b[i];
In CUDA:
__global__ add(int *c, int *a, int*b, int N)
{
int pos = (threadIdx.x + blockIdx.x)
for(; i < N; pos +=blockDim.x)
c[pos] = a[pos] + b[pos];
}
In CUDA you basically took each for iteration and assigned to each thread,
1) threadIdx.x + blockIdx.x*blockDim.x;
Each block has an ID from 0 to N-1 (N the number maximum of blocks) and each block has a 'X' number of threads with an ID from 0 to X-1.
Gives you the for loop iteration that each thread will compute based on its ID and the block ID which the thread is in; the blockDim.x is the number of threads that a block has.
So if you have 2 blocks each one with 10 threads and N=40, the:
Thread 0 Block 0 will execute pos 0
Thread 1 Block 0 will execute pos 1
...
Thread 9 Block 0 will execute pos 9
Thread 0 Block 1 will execute pos 10
....
Thread 9 Block 1 will execute pos 19
Thread 0 Block 0 will execute pos 20
...
Thread 0 Block 1 will execute pos 30
Thread 9 Block 1 will execute pos 39
Looking at your current code, I have made this draft of what your code could look like in CUDA:
__global__ hash (float *largeFloatingPointArray, int *dictionary)
// You can turn the dictionary in one array of int
// here each position will represent the float
// Since x = 0f; x < 100f; x += 0.0001f
// you can associate each x to different position
// in the dictionary:
// pos 0 have the same meaning as 0f;
// pos 1 means float 0.0001f
// pos 2 means float 0.0002f ect.
// Then you use the int of each position
// to count how many times that "float" had appeared
int x = blockIdx.x; // Each block will take a different x to work
float y;
while( x < 1000000) // x < 100f (for incremental step of 0.0001f)
{
int noOfOccurrences = 0;
float z = converting_int_to_float(x); // This function will convert the x to the
// float like you use (x / 0.0001)
// each thread of each block
// will takes the y from the array of largeFloatingPointArray
for(j = threadIdx.x; j < largeFloatingPointArraySize; j += blockDim.x)
{
y = largeFloatingPointArray[j];
if (z == y)
{
noOfOccurrences++;
}
}
if(threadIdx.x == 0) // Thread master will update the values
atomicAdd(&dictionary[x], noOfOccurrences);
__syncthreads();
}
You have to use atomicAdd because different threads from different blocks may write/read noOfOccurrences concurrently, so you have to ensure mutual exclusion.
This is just one approach; you can even assign the iterations of the outer loop to the threads instead of the blocks.
Tutorials
The Dr Dobbs Journal series CUDA: Supercomputing for the masses by Rob Farmer is excellent and covers just about everything in its fourteen installments. It also starts rather gently and is therefore fairly beginner-friendly.
and anothers:
Volume I: Introduction to CUDA Programming
Getting started with CUDA
CUDA Resources List
Take a look on the last item, you will find many link to learn CUDA.
OpenCL: OpenCL Tutorials | MacResearch
I don't know much of anything about parallel processing or GPGPU, but for this specific example, you could save a lot of time by making a single pass over the input array rather than looping over it a million times. With large data sets you will usually want to do things in a single pass if possible. Even if you're doing multiple independent computations, if it's over the same data set you might get better speed doing them all in the same pass, as you'll get better locality of reference that way. But it may not be worth it for the increased complexity in your code.
In addition, you really don't want to add a small amount to a floating point number repetitively like that, the rounding error will add up and you won't get what you intended. I've added an if statement to my below sample to check if inputs match your pattern of iteration, but omit it if you don't actually need that.
I don't know any C#, but a single pass implementation of your sample would look something like this:
Dictionary<float, int> noOfNumbers = new Dictionary<float, int>();
foreach (float x in largeFloatingPointArray)
{
if (math.Truncate(x/0.0001f)*0.0001f == x)
{
if (noOfNumbers.ContainsKey(x))
noOfNumbers.Add(x, noOfNumbers[x]+1);
else
noOfNumbers.Add(x, 1);
}
}
Hope this helps.
Is it possible (and does it make sense) to utilize a GPU to speed up
such calculations?
Definitely YES, this kind of algorithm is typically the ideal candidate for massive data-parallelism processing, the thing GPUs are so good at.
If yes: Does anyone know any tutorial or got any sample code
(programming language doesn't matter)?
When you want to go the GPGPU way you have two alternatives : CUDA or OpenCL.
CUDA is mature with a lot of tools but is NVidia GPUs centric.
OpenCL is a standard running on NVidia and AMD GPUs, and CPUs too. So you should really favour it.
For tutorial you have an excellent series on CodeProject by Rob Farber : http://www.codeproject.com/Articles/Rob-Farber#Articles
For your specific use-case there is a lot of samples for histograms buiding with OpenCL (note that many are image histograms but the principles are the same).
As you use C# you can use bindings like OpenCL.Net or Cloo.
If your array is too big to be stored in the GPU memory, you can block-partition it and rerun your OpenCL kernel for each part easily.
In addition to the suggestion by the above poster use the TPL (task parallel library) when appropriate to run in parallel on multiple cores.
The example above could use Parallel.Foreach and ConcurrentDictionary, but a more complex map-reduce setup where the array is split into chunks each generating an dictionary which would then be reduced to a single dictionary would give you better results.
I don't know whether all your computations map correctly to the GPU capabilities, but you'll have to use a map-reduce algorithm anyway to map the calculations to the GPU cores and then reduce the partial results to a single result, so you might as well do that on the CPU before moving on to a less familiar platform.
I am not sure whether using GPUs would be a good match given that
'largerFloatingPointArray' values need to be retrieved from memory. My understanding is that GPUs are better suited for self contained calculations.
I think turning this single process application into a distributed application running on many systems and tweaking the algorithm should speed things up considerably, depending how many systems are available.
You can use the classic 'divide and conquer' approach. The general approach I would take is as follows.
Use one system to preprocess 'largeFloatingPointArray' into a hash table or a database. This would be done in a single pass. It would use floating point value as the key, and the number of occurrences in the array as the value. Worst case scenario is that each value only occurs once, but that is unlikely. If largeFloatingPointArray keeps changing each time the application is run then in-memory hash table makes sense. If it is static, then the table could be saved in a key-value database such as Berkeley DB. Let's call this a 'lookup' system.
On another system, let's call it 'main', create chunks of work and 'scatter' the work items across N systems, and 'gather' the results as they become available. E.g a work item could be as simple as two numbers indicating the range that a system should work on. When a system completes the work, it sends back array of occurrences and it's ready to work on another chunk of work.
The performance is improved because we do not keep iterating over largeFloatingPointArray. If lookup system becomes a bottleneck, then it could be replicated on as many systems as needed.
With large enough number of systems working in parallel, it should be possible to reduce the processing time down to minutes.
I am working on a compiler for parallel programming in C targeted for many-core based systems, often referred to as microservers, that are/or will be built using multiple 'system-on-a-chip' modules within a system. ARM module vendors include Calxeda, AMD, AMCC, etc. Intel will probably also have a similar offering.
I have a version of the compiler working, which could be used for such an application. The compiler, based on C function prototypes, generates C networking code that implements inter-process communication code (IPC) across systems. One of the IPC mechanism available is socket/tcp/ip.
If you need help in implementing a distributed solution, I'd be happy to discuss it with you.
Added Nov 16, 2012.
I thought a little bit more about the algorithm and I think this should do it in a single pass. It's written in C and it should be very fast compared with what you have.
/*
* Convert the X range from 0f to 100f in steps of 0.0001f
* into a range of integers 0 to 1 + (100 * 10000) to use as an
* index into an array.
*/
#define X_MAX (1 + (100 * 10000))
/*
* Number of floats in largeFloatingPointArray needs to be defined
* below to be whatever your value is.
*/
#define LARGE_ARRAY_MAX (1000)
main()
{
int j, y, *noOfOccurances;
float *largeFloatingPointArray;
/*
* Allocate memory for largeFloatingPointArray and populate it.
*/
largeFloatingPointArray = (float *)malloc(LARGE_ARRAY_MAX * sizeof(float));
if (largeFloatingPointArray == 0) {
printf("out of memory\n");
exit(1);
}
/*
* Allocate memory to hold noOfOccurances. The index/10000 is the
* the floating point number. The contents is the count.
*
* E.g. noOfOccurances[12345] = 20, means 1.2345f occurs 20 times
* in largeFloatingPointArray.
*/
noOfOccurances = (int *)calloc(X_MAX, sizeof(int));
if (noOfOccurances == 0) {
printf("out of memory\n");
exit(1);
}
for (j = 0; j < LARGE_ARRAY_MAX; j++) {
y = (int)(largeFloatingPointArray[j] * 10000);
if (y >= 0 && y <= X_MAX) {
noOfOccurances[y]++;
}
}
}