Related
Consider the following C++ function in R using Rcpp:
cppFunction('long double statZn_cpp(NumericVector dat, double kn) {
double n = dat.size();
// Get total sum and sum of squares; this will be the "upper sum"
// (i.e. the sum above k)
long double s_upper, s_square_upper;
// The "lower sums" (i.e. those below k)
long double s_lower, s_square_lower;
// Get lower sums
// Go to kn - 1 to prevent double-counting in main
// loop
for (int i = 0; i < kn - 1; ++i) {
s_lower += dat[i];
s_square_lower += dat[i] * dat[i];
}
// Get upper sum
for (int i = kn - 1; i < n; ++i) {
s_upper += dat[i];
s_square_upper += dat[i] * dat[i];
}
// The maximum, which will be returned
long double M = 0;
// A candidate for the new maximum, used in a loop
long double M_candidate;
// Compute the test statistic
for (int k = kn; k <= (n - kn); ++k) {
// Update s and s_square for both lower and upper
s_lower += dat[k-1];
s_square_lower += dat[k-1] * dat[k-1];
s_upper -= dat[k-1];
s_square_upper -= dat[k-1] * dat[k-1];
// Get estimate of sd for this k
long double sdk = sqrt((s_square_lower - pow(s_lower, 2.0) / k +
s_square_upper -
pow(s_upper, 2.0) / (n - k))/n);
M_candidate = abs(s_lower / k - s_upper / (n - k)) / sdk;
// Choose new maximum
if (M_candidate > M) {
M = M_candidate;
}
}
return M * sqrt(kn);
}')
Try the command statZn_cpp(1:20,4), and you will get 6.963106, which is the correct answer. Scaling should not matter; statZn_cpp(1:20*10,4) will also yield the correct answer of 6.963106. But statZn_cpp(1:20/10,4) yields the wrong answer of 6.575959, and statZn_cpp(1:20/100,4) again gives you the obviously wrong answer of 0. More to the point (and relevant to my research, which involves simulation studies), when I try statZn_cpp(rnorm(20),4), the answer is almost always 0, which is wrong.
Clearly the problem has to do with rounding errors, but I don't know where they are or how to fix them (I am brand new to C++). I've tried to expand precision as much as possible. Is there a way to fix the rounding problem? (An R wrapper function is permissible if I should be attempting what amounts to a preprocessing step, but it needs to be robust, working for general levels of precision.)
EDIT: Here is some "equivalent" R code:
statZn <- function(dat, kn = function(n) {floor(sqrt(n))}) {
n = length(dat)
return(sqrt(kn(n))*max(sapply(
floor(kn(n)):(n - floor(kn(n))), function(k)
abs(1/k*sum(dat[1:k]) -
1/(n-k)*sum(dat[(k+1):n]))/sqrt((sum((dat[1:k] -
mean(dat[1:k]))^2)+sum((dat[(k+1):n] -
mean(dat[(k+1):n]))^2))/n))))
}
Also, the R code below basically replicates the method that should be used by the C++ code. It is capable of reaching the correct answer.
n = length(dat)
s_lower = 0
s_square_lower = 0
s_upper = 0
s_square_upper = 0
for (i in 1:(kn-1)) {
s_lower = s_lower + dat[i]
s_square_lower = s_square_lower + dat[i] * dat[i]
}
for (i in kn:n) {
s_upper = s_upper + dat[i]
s_square_upper = s_square_upper + dat[i] * dat[i]
}
M = 0
for (k in kn:(n-kn)) {
s_lower = s_lower + dat[k]
s_square_lower = s_square_lower + dat[k] * dat[k]
s_upper = s_upper - dat[k]
s_square_upper = s_square_upper - dat[k] * dat[k]
sdk = sqrt((s_square_lower - (s_lower)^2/k +
s_square_upper -
(s_upper)^2/(n-k))/n)
M_candidate = sqrt(kn) * abs(s_lower / k - s_upper / (n - k)) / sdk
cat('k', k, '\n',
"s_lower", s_lower, '\n',
's_square_lower', s_square_lower, '\n',
's_upper', s_upper, '\n',
's_square_upper', s_square_upper, '\n',
'sdk', sdk, '\n',
'M_candidate', M_candidate, '\n\n')
if (M_candidate > M) {
M = M_candidate
}
}
1: You should not be using long double, since R represents all numeric values in the double type. Using a more precise type for intermediate calculations is extremely unlikely to provide any benefit, and is more likely to result in strange inconsistencies between platforms.
2: You're not initializing s_upper, s_square_upper, s_lower, and s_square_lower. (You actually are initializing them in the R implementation, but you forgot in the C++ implementation.)
3: Minor point, but I would replace the pow(x,2.0) calls with x*x. Although this doesn't really matter.
4: This is what fixed it for me: You need to qualify calls to C++ standard library functions with their containing namespace. IOW, std::sqrt() instead of just sqrt(), std::abs() instead of just abs(), and std::pow() instead of just pow() if you continue to use it.
cppFunction('double statZn_cpp(NumericVector dat, double kn) {
int n = dat.size();
double s_upper = 0, s_square_upper = 0; // Get total sum and sum of squares; this will be the "upper sum" (i.e. the sum above k)
double s_lower = 0, s_square_lower = 0; // The "lower sums" (i.e. those below k)
for (int i = 0; i < kn - 1; ++i) { s_lower += dat[i]; s_square_lower += dat[i] * dat[i]; } // Get lower sums; Go to kn - 1 to prevent double-counting in main
for (int i = kn - 1; i < n; ++i) { s_upper += dat[i]; s_square_upper += dat[i] * dat[i]; } // Get upper sum
double M = 0; // The maximum, which will be returned
double M_candidate; // A candidate for the new maximum, used in a loop
// Compute the test statistic
for (int k = kn; k <= (n - kn); ++k) {
// Update s and s_square for both lower and upper
s_lower += dat[k-1];
s_square_lower += dat[k-1] * dat[k-1];
s_upper -= dat[k-1];
s_square_upper -= dat[k-1] * dat[k-1];
// Get estimate of sd for this k
double sdk = std::sqrt((s_square_lower - s_lower*s_lower / k + s_square_upper - s_upper*s_upper / (n - k))/n);
M_candidate = std::abs(s_lower / k - s_upper / (n - k)) / sdk;
if (M_candidate > M) M = M_candidate; // Choose new maximum
}
return std::sqrt(kn) * M;
}');
statZn_cpp(1:20,4); ## you will get 6.963106, which is the correct answer
## [1] 6.963106
statZn_cpp(1:20*10,4); ## Scaling should not matter; will also yield the correct answer of 6.963106
## [1] 6.963106
statZn_cpp(1:20/10,4); ## yields the wrong answer of 6.575959
## [1] 6.963106
statZn_cpp(1:20/100,4); ## again gives you the obviously wrong answer of 0.
## [1] 6.963106
set.seed(1L); statZn_cpp(rnorm(20),4); ## More to the point (and relevant to my research, which involves simulation studies), the answer is almost always 0, which is wrong.
## [1] 1.270117
This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 10 years ago.
I hesitate to ask this question because there is probably something wrong with my C++ template program but this problem has been bugging me for the past couple of hours. I am running the exact same program on Visual C++ and Mingw-g++ compilers but only VC2010 is giving me the expected results. I am not proficient C++ programmer by any means so not getting any error messages from either compilers is even more frustrating.
Edit : I did mingw-get upgrade after failing to resolve the error. I was running g++ 4.5.2 and now I have version 4.7.2 but the problem persists.
Late Update - I did a complete uninstall of MinGW platform, manually removed every folder and then installed TDM-GCC but the problem persists. Maybe there is some conflict with my Windows Installation. I have installed Cygwin and g++ 4.5.3 for the time being (It is working) as OS reinstallation isn't really an option right now. Thanks for all the help.
Here is my code. (Header File itertest.h)
#ifndef ITERTEST_H
#define ITERTEST_H
#include <iostream>
#include <cmath>
#include <vector>
#include <string>
#include <algorithm>
using namespace std;
template <typename T>
class fft_data{
public:
vector<T> re;
vector<T> im;
};
template <typename T>
void inline twiddle(fft_data<T> &vec,int N,int radix){
// Calculates twiddle factors for radix-2
T PI2 = (T) 6.28318530717958647692528676655900577;
T theta = (T) PI2/N;
vec.re.resize(N/radix,(T) 0.0);
vec.im.resize(N/radix,(T) 0.0);
vec.re[0] = (T) 1.0;
for (int K = 1; K < N/radix; K++) {
vec.re[K] = (T) cos(theta * K);
vec.im[K] = (T) sin(theta * K);
}
}
template <typename T>
void inline sh_radix5_dif(fft_data<T> &x,fft_data<T> &wl, int q, int sgn) {
int n = x.re.size();
int L = (int) pow(5.0, (double)q);
int Ls = L / 5;
int r = n / L;
T c1 = 0.30901699437;
T c2 = -0.80901699437;
T s1 = 0.95105651629;
T s2 = 0.58778525229;
T tau0r,tau0i,tau1r,tau1i,tau2r,tau2i,tau3r,tau3i;
T tau4r,tau4i,tau5r,tau5i;
T br,bi,cr,ci,dr,di,er,ei;
fft_data<T> y = x;
T wlr,wli,wl2r,wl2i,wl3r,wl3i,wl4r,wl4i;
int lsr = Ls*r;
for (int j = 0; j < Ls; j++) {
int ind = j*r;
wlr = wl.re[ind];
wli = wl.im[ind];
wl2r = wlr*wlr - wli*wli;
wl2i = 2.0*wlr*wli;
wl3r = wl2r*wlr - wli*wl2i;
wl3i= wl2r*wli + wl2i*wlr;
wl4r = wl2r*wl2r - wl2i*wl2i;
wl4i = 2.0*wl2r*wl2i;
for (int k =0; k < r; k++) {
int index = k*L+j;
int index1 = index+Ls;
int index2 = index1+Ls;
int index3 = index2+Ls;
int index4 = index3+Ls;
tau0r = y.re[index1] + y.re[index4];
tau0i = y.im[index1] + y.im[index4];
tau1r = y.re[index2] + y.re[index3];
tau1i = y.im[index2] + y.im[index3];
tau2r = y.re[index1] - y.re[index4];
tau2i = y.im[index1] - y.im[index4];
tau3r = y.re[index2] - y.re[index3];
tau3i = y.im[index2] - y.im[index3];
tau4r = c1 * tau0r + c2 * tau1r;
tau4i = c1 * tau0i + c2 * tau1i;
tau5r = sgn * ( s1 * tau2r + s2 * tau3r);
tau5i = sgn * ( s1 * tau2i + s2 * tau3i);
br = y.re[index] + tau4r + tau5i;
bi = y.im[index] + tau4i - tau5r;
er = y.re[index] + tau4r - tau5i;
ei = y.im[index] + tau4i + tau5r;
tau4r = c2 * tau0r + c1 * tau1r;
tau4i = c2 * tau0i + c1 * tau1i;
tau5r = sgn * ( s2 * tau2r - s1 * tau3r);
tau5i = sgn * ( s2 * tau2i - s1 * tau3i);
cr = y.re[index] + tau4r + tau5i;
ci = y.im[index] + tau4i - tau5r;
dr = y.re[index] + tau4r - tau5i;
di = y.im[index] + tau4i + tau5r;
int indexo = k*Ls+j;
int indexo1 = indexo+lsr;
int indexo2 = indexo1+lsr;
int indexo3 = indexo2+lsr;
int indexo4 = indexo3+lsr;
x.re[indexo]= y.re[index] + tau0r + tau1r;
x.im[indexo]= y.im[index] + tau0i + tau1i;
x.re[indexo1] = wlr*br - wli*bi;
x.im[indexo1] = wlr*bi + wli*br;
x.re[indexo2] = wl2r*cr - wl2i*ci;
x.im[indexo2] = wl2r*ci + wl2i*cr;
x.re[indexo3] = wl3r*dr - wl3i*di;
x.im[indexo3] = wl3r*di + wl3i*dr;
x.re[indexo4] = wl4r*er - wl4i*ei;
x.im[indexo4] = wl4r*ei + wl4i*er;
}
}
}
template <typename T>
void inline fftsh_radix5_dif(fft_data<T> &data,int sgn, unsigned int N) {
//unsigned int len = data.re.size();
int num = (int) ceil(log10(static_cast<double>(N))/log10(5.0));
//indrev(data,index);
fft_data<T> twi;
twiddle(twi,N,5);
if (sgn == 1) {
transform(twi.im.begin(), twi.im.end(),twi.im.begin(),bind1st(multiplies<T>(),(T) -1.0));
}
for (int i=num; i > 0; i--) {
sh_radix5_dif(data,twi,i,sgn);
}
}
#endif
main.cpp
#include "itertest.h"
using namespace std;
int main(int argc, char **argv)
{
int N = 25;
//vector<complex<double> > sig1;
fft_data<double> sig1;
for (int i =0; i < N; i++){
//sig1.push_back(complex<double>((double)1.0, 0.0));
//sig2.re.push_back((double) i);
//sig2.im.push_back((double) i+2);
sig1.re.push_back((double) 1);
sig1.im.push_back((double) 0);
}
fftsh_radix5_dif(sig1,1,N);
for (int i =0; i < N; i++){
cout << sig1.re[i] << " " << sig1.im[i] << endl;
}
cin.get();
return 0;
}
The expected Output (which I am getting from VC2010)
25 0
4.56267e-016 -2.50835e-016
2.27501e-016 -3.58484e-016
1.80101e-017 -2.86262e-016
... rest 21 rows same as the last three rows ( < 1e-015)
The Output from Mingw-g++
20 0
4.94068e-016 -2.10581e-016
2.65385e-016 -3.91346e-016
-5.76751e-017 -2.93654e-016
5 0
-1.54508 -4.75528
-3.23032e-017 1.85061e-017
-4.68253e-017 -1.18421e-016
-6.32003e-017 -2.05833e-016
1.11022e-016 0
4.04508 -2.93893
8.17138e-017 6.82799e-018
3.5246e-017 9.06767e-017
-6.59101e-017 -1.62762e-016
1.11022e-016 0
4.04508 2.93893
-6.28467e-017 6.40636e-017
1.79807e-016 3.34411e-017
-6.94919e-017 -1.05831e-016
1.11022e-016 0
-1.54508 4.75528
5.70402e-017 -1.68674e-017
-1.36169e-016 -8.30473e-017
-9.75639e-017 3.40359e-016
1.11022e-016 0
There must be something wrong with your MinGW installation. You might have an out-of-date, buggy version of GCC. The unofficial TDM-GCC distribution usually has a more up-to-date version: http://tdm-gcc.tdragon.net/
When I compile your code with GCC 4.6.3 on Ubuntu, it produces the output below, which appears to match the VC2010 output exactly (but I can't verify this, since you didn't provide it in full). Adding the options -O3 -ffast-math -march=native doesn't seem to change anything.
Note that I had to fix an obvious typo in fftsh_radix5_dif (missing closing angle bracket in the list of template arguments to multiply), but I assume you do not have it in your code, since it wouldn't compile at all.
25 0
4.56267e-16 -2.50835e-16
2.27501e-16 -3.58484e-16
1.80101e-17 -2.86262e-16
-5.76751e-17 -1.22566e-16
8.88178e-16 0
9.45774e-17 1.19479e-17
1.27413e-16 -5.04465e-17
7.97139e-17 -9.63575e-17
1.35142e-17 -7.08438e-17
8.88178e-16 0
4.84283e-17 4.54772e-17
1.02473e-16 2.63107e-17
1.02473e-16 -2.63107e-17
4.84283e-17 -4.54772e-17
8.88178e-16 0
1.35142e-17 7.08438e-17
7.97139e-17 9.63575e-17
1.27413e-16 5.04465e-17
9.45774e-17 -1.19479e-17
8.88178e-16 0
-5.76751e-17 1.22566e-16
1.80101e-17 2.86262e-16
2.27501e-16 3.58484e-16
4.56267e-16 2.50835e-16
Check the creation date of the executable you're running.
You may be running an earlier draft of your program.
I am trying to write an android app which needs to calculate gaussian and laplacian pyramids for multiple full resolution images, i wrote this it on C++ with NDK, the most critical part of the code is applying gaussian filter to images abd i am applying this filter with horizontally and vertically.
The filter is (0.0625, 0.25, 0.375, 0.25, 0.0625)
Since i am working on integers i am calculating (1, 4, 6, 4, 1)/16
dst[index] = ( src[index-2] + src[index-1]*4 + src[index]*6+src[index+1]*4+src[index+2])/16;
I have made a few simple optimization however it still is working slow than expected and i was wondering if there are any other optimization options that i am missing.
PS: I should mention that i have tried to write this filter part with inline arm assembly however it give 2x slower results.
//horizontal filter
for(unsigned y = 0; y < height; y++) {
for(unsigned x = 2; x < width-2; x++) {
int index = y*width+x;
dst[index].r = (src[index-2].r+ src[index+2].r + (src[index-1].r + src[index+1].r)*4 + src[index].r*6)>>4;
dst[index].g = (src[index-2].g+ src[index+2].g + (src[index-1].g + src[index+1].g)*4 + src[index].g*6)>>4;
dst[index].b = (src[index-2].b+ src[index+2].b + (src[index-1].b + src[index+1].b)*4 + src[index].b*6)>>4;
}
}
//vertical filter
for(unsigned y = 2; y < height-2; y++) {
for(unsigned x = 0; x < width; x++) {
int index = y*width+x;
dst[index].r = (src[index-2*width].r + src[index+2*width].r + (src[index-width].r + src[index+width].r)*4 + src[index].r*6)>>4;
dst[index].g = (src[index-2*width].g + src[index+2*width].g + (src[index-width].g + src[index+width].g)*4 + src[index].g*6)>>4;
dst[index].b = (src[index-2*width].b + src[index+2*width].b + (src[index-width].b + src[index+width].b)*4 + src[index].b*6)>>4;
}
}
The index multiplication can be factored out of the inner loop since the mulitplicatation only occurs when y is changed:
for (unsigned y ...
{
int index = y * width;
for (unsigned int x...
You may gain some speed by loading variables before you use them. This would make the processor load them in the cache:
for (unsigned x = ...
{
register YOUR_DATA_TYPE a, b, c, d, e;
a = src[index - 2].r;
b = src[index - 1].r;
c = src[index + 0].r; // The " + 0" is to show a pattern.
d = src[index + 1].r;
e = src[index + 2].r;
dest[index].r = (a + e + (b + d) * 4 + c * 6) >> 4;
// ...
Another trick would be to "cache" the values of the src so that only a new one is added each time because the value in src[index+2] may be used up to 5 times.
So here is a example of the concepts:
//horizontal filter
for(unsigned y = 0; y < height; y++)
{
int index = y*width + 2;
register YOUR_DATA_TYPE a, b, c, d, e;
a = src[index - 2].r;
b = src[index - 1].r;
c = src[index + 0].r; // The " + 0" is to show a pattern.
d = src[index + 1].r;
e = src[index + 2].r;
for(unsigned x = 2; x < width-2; x++)
{
dest[index - 2 + x].r = (a + e + (b + d) * 4 + c * 6) >> 4;
a = b;
b = c;
c = d;
d = e;
e = src[index + x].r;
I'm not sure how your compiler would optimize all this, but I tend to work in pointers. Assuming your struct is 3 bytes... You can start with pointers in the right places (the edge of the filter for source, and the destination for target), and just move them through using constant array offsets. I've also put in an optional OpenMP directive on the outer loop, as this can also improve things.
#pragma omp parallel for
for(unsigned y = 0; y < height; y++) {
const int rowindex = y * width;
char * dpos = (char*)&dest[rowindex+2];
char * spos = (char*)&src[rowindex];
const char *end = (char*)&src[rowindex+width-2];
for( ; spos != end; spos++, dpos++) {
*dpos = (spos[0] + spos[4] + ((spos[1] + src[3])<<2) + spos[2]*6) >> 4;
}
}
Similarly for the vertical loop.
const int scanwidth = width * 3;
const int row1 = scanwidth;
const int row2 = row1+scanwidth;
const int row3 = row2+scanwidth;
const int row4 = row3+scanwidth;
#pragma omp parallel for
for(unsigned y = 2; y < height-2; y++) {
const int rowindex = y * width;
char * dpos = (char*)&dest[rowindex];
char * spos = (char*)&src[rowindex-row2];
const char *end = spos + scanwidth;
for( ; spos != end; spos++, dpos++) {
*dpos = (spos[0] + spos[row4] + ((spos[row1] + src[row3])<<2) + spos[row2]*6) >> 4;
}
}
This is how I do convolutions, anyway. It sacrifices readability a little, and I've never tried measuring the difference. I just tend to write them that way from the outset. See if that gives you a speed-up. The OpenMP definitely will if you have a multicore machine, and the pointer stuff might.
I like the comment about using SSE for these operations.
Some of the more obvious optimizations are exploiting the symmetry of the kernel:
a=*src++; b=*src++; c=*src++; d=*src++; e=*src++; // init
LOOP (n/5) times:
z=(a+e)+(b+d)<<2+c*6; *dst++=z>>4; // then reuse the local variables
a=*src++;
z=(b+a)+(c+e)<<2+d*6; *dst++=z>>4; // registers have been read only once...
b=*src++;
z=(c+b)+(d+a)<<2+e*6; *dst++=z>>4;
e=*src++;
The second thing is that one can perform multiple additions using a single integer. When the values to be filtered are unsigned, one can fit two channels in a single 32-bit integer (or 4 channels in a 64-bit integer); it's the poor mans SIMD.
a= 0x[0011][0034] <-- split to two
b= 0x[0031][008a]
----------------------
sum 0042 00b0
>>4 0004 200b0 <-- mask off
mask 00ff 00ff
-------------------
0004 000b <-- result
(The Simulated SIMD shows one addition followed by a shift by 4)
Here's a kernel that calculates 3 rgb operations in parallel (easy to modify for 6 rgb operations in 64-bit architectures...)
#define MASK (255+(255<<10)+(255<<20))
#define KERNEL(a,b,c,d,e) { \
a=((a+e+(c<<1))>>2) & MASK; a=(a+b+c+d)>>2 & MASK; *DATA++ = a; a=DATA[4]; }
void calc_5_rgbs(unsigned int *DATA)
{
register unsigned int a = DATA[0], b=DATA[1], c=DATA[2], d=DATA[3], e=DATA[4];
KERNEL(a,b,c,d,e);
KERNEL(b,c,d,e,a);
KERNEL(c,d,e,a,b);
KERNEL(d,e,a,b,c);
KERNEL(e,a,b,c,d);
}
Works best on ARM and on 64-bit IA with 16 registers... Needs heavy assembler optimizations to overcome register shortage in 32-bit IA (e.g. use ebp as GPR). And just because of that it's an inplace algorithm...
There are just 2 guardian bits between every 8 bits of data, which is just enough to get exactly the same result as in integer calculation.
And BTW: it's faster to just run through the array byte per byte than by r,g,b elements
unsigned char *s=(unsigned char *) source_array;
unsigned char *d=(unsigned char *) dest_array;
for (j=0;j<3*N;j++) d[j]=(s[j]+s[j+16]+s[j+8]*6+s[j+4]*4+s[j+12]*4)>>4;
BYTE * srcData;
BYTE * pData;
int i,j;
int srcPadding;
//some variable initialization
for (int r = 0;r < h;r++,srcData+= srcPadding)
{
for (int col = 0;col < w;col++,pData += 4,srcData += 3)
{
memcpy(pData,srcData,3);
}
}
I've tried loop unrolling, but it helps little.
int segs = w / 4;
int remain = w - segs * 4;
for (int r = 0;r < h;r++,srcData+= srcPadding)
{
int idx = 0;
for (idx = 0;idx < segs;idx++,pData += 16,srcData += 12)
{
memcpy(pData,srcData,3);
*(pData + 3) = 0xFF;
memcpy(pData + 4,srcData + 3,3);
*(pData + 7) = 0xFF;
memcpy(pData + 8,srcData + 6,3);
*(pData + 11) = 0xFF;
memcpy(pData + 12,srcData + 9,3);
*(pData + 15) = 0xFF;
}
for (idx = 0;idx < remain;idx++,pData += 4,srcData += 3)
{
memcpy(pData,srcData,3);
*(pData + 3) = 0xFF;
}
}
Depending on your compiler, you may not want memcpy at all for such a small copy. Here is a variant version for the body of your unrolled loop; see if it's faster:
uint32_t in0 = *(uint32_t*)(srcData);
uint32_t in1 = *(uint32_t*)(srcData + 4);
uint32_t in2 = *(uint32_t*)(srcData + 8);
uint32_t out0 = UINT32_C(0xFF000000) | (in0 & UINT32_C(0x00FFFFFF));
uint32_t out1 = UINT32_C(0xFF000000) | (in0 >> 24) | ((in1 & 0xFFFF) << 8);
uint32_t out2 = UINT32_C(0xFF000000) | (in1 >> 16) | ((in2 & 0xFF) << 16);
uint32_t out3 = UINT32_C(0xFF000000) | (in2 >> 8);
*(uint32_t*)(pData) = out0;
*(uint32_t*)(pData + 4) = out1;
*(uint32_t*)(pData + 8) = out2;
*(uint32_t*)(pData + 12) = out3;
You should also declare srcData and pData as BYTE * restrict pointers so the compiler will know they don't alias.
I don't see much that you're doing that isn't necessary. You could change the post-increments to pre-increments (idx++ to ++idx, for instance), but that won't have a measurable effect.
Additionally, you could use std::copy instead of memcpy. std::copy has more information available to it and in theory can pick the most efficient way to copy things. Unfortunately I don't believe that many STL implementations actually take advantage of the extra information.
The only thing that I expect would make a difference is that there's no reason to wait for one memcpy to finish before starting the next. You could use OpenMP or Intel Threading Building Blocks (or a thread queue of some kind) to parallelize the loops.
Don't call memcpy, just do the copy by hand. The function call overhead isn't worth it unless you can copy more than 3 bytes at a time.
As far as this particular loop goes, you may want to look at a technique called Duff's device, which is a loop-unrolling technique that takes advantage of the switch construct.
Maybe changing to a while loop instead of nested for loops:
BYTE *src = srcData;
BYTE *dest = pData;
int maxsrc = h*(w*3+srcPadding);
int offset = 0;
int maxoffset = w*3;
while (src+offset < maxsrc) {
*dest++ = *(src+offset++);
*dest++ = *(src+offset++);
*dest++ = *(src+offset++);
dest++;
if (offset > maxoffset) {
src += srcPadding;
offset = 0;
}
}
I'm writing a sparse matrix solver using the Gauss-Seidel method. By profiling, I've determined that about half of my program's time is spent inside the solver. The performance-critical part is as follows:
size_t ic = d_ny + 1, iw = d_ny, ie = d_ny + 2, is = 1, in = 2 * d_ny + 1;
for (size_t y = 1; y < d_ny - 1; ++y) {
for (size_t x = 1; x < d_nx - 1; ++x) {
d_x[ic] = d_b[ic]
- d_w[ic] * d_x[iw] - d_e[ic] * d_x[ie]
- d_s[ic] * d_x[is] - d_n[ic] * d_x[in];
++ic; ++iw; ++ie; ++is; ++in;
}
ic += 2; iw += 2; ie += 2; is += 2; in += 2;
}
All arrays involved are of float type. Actually, they are not arrays but objects with an overloaded [] operator, which (I think) should be optimized away, but is defined as follows:
inline float &operator[](size_t i) { return d_cells[i]; }
inline float const &operator[](size_t i) const { return d_cells[i]; }
For d_nx = d_ny = 128, this can be run about 3500 times per second on an Intel i7 920. This means that the inner loop body runs 3500 * 128 * 128 = 57 million times per second. Since only some simple arithmetic is involved, that strikes me as a low number for a 2.66 GHz processor.
Maybe it's not limited by CPU power, but by memory bandwidth? Well, one 128 * 128 float array eats 65 kB, so all 6 arrays should easily fit into the CPU's L3 cache (which is 8 MB). Assuming that nothing is cached in registers, I count 15 memory accesses in the inner loop body. On a 64-bits system this is 120 bytes per iteration, so 57 million * 120 bytes = 6.8 GB/s. The L3 cache runs at 2.66 GHz, so it's the same order of magnitude. My guess is that memory is indeed the bottleneck.
To speed this up, I've attempted the following:
Compile with g++ -O3. (Well, I'd been doing this from the beginning.)
Parallelizing over 4 cores using OpenMP pragmas. I have to change to the Jacobi algorithm to avoid reads from and writes to the same array. This requires that I do twice as many iterations, leading to a net result of about the same speed.
Fiddling with implementation details of the loop body, such as using pointers instead of indices. No effect.
What's the best approach to speed this guy up? Would it help to rewrite the inner body in assembly (I'd have to learn that first)? Should I run this on the GPU instead (which I know how to do, but it's such a hassle)? Any other bright ideas?
(N.B. I do take "no" for an answer, as in: "it can't be done significantly faster, because...")
Update: as requested, here's a full program:
#include <iostream>
#include <cstdlib>
#include <cstring>
using namespace std;
size_t d_nx = 128, d_ny = 128;
float *d_x, *d_b, *d_w, *d_e, *d_s, *d_n;
void step() {
size_t ic = d_ny + 1, iw = d_ny, ie = d_ny + 2, is = 1, in = 2 * d_ny + 1;
for (size_t y = 1; y < d_ny - 1; ++y) {
for (size_t x = 1; x < d_nx - 1; ++x) {
d_x[ic] = d_b[ic]
- d_w[ic] * d_x[iw] - d_e[ic] * d_x[ie]
- d_s[ic] * d_x[is] - d_n[ic] * d_x[in];
++ic; ++iw; ++ie; ++is; ++in;
}
ic += 2; iw += 2; ie += 2; is += 2; in += 2;
}
}
void solve(size_t iters) {
for (size_t i = 0; i < iters; ++i) {
step();
}
}
void clear(float *a) {
memset(a, 0, d_nx * d_ny * sizeof(float));
}
int main(int argc, char **argv) {
size_t n = d_nx * d_ny;
d_x = new float[n]; clear(d_x);
d_b = new float[n]; clear(d_b);
d_w = new float[n]; clear(d_w);
d_e = new float[n]; clear(d_e);
d_s = new float[n]; clear(d_s);
d_n = new float[n]; clear(d_n);
solve(atoi(argv[1]));
cout << d_x[0] << endl; // prevent the thing from being optimized away
}
I compile and run it as follows:
$ g++ -o gstest -O3 gstest.cpp
$ time ./gstest 8000
0
real 0m1.052s
user 0m1.050s
sys 0m0.010s
(It does 8000 instead of 3500 iterations per second because my "real" program does a lot of other stuff too. But it's representative.)
Update 2: I've been told that unititialized values may not be representative because NaN and Inf values may slow things down. Now clearing the memory in the example code. It makes no difference for me in execution speed, though.
Couple of ideas:
Use SIMD. You could load 4 floats at a time from each array into a SIMD register (e.g. SSE on Intel, VMX on PowerPC). The disadvantage of this is that some of the d_x values will be "stale" so your convergence rate will suffer (but not as bad as a jacobi iteration); it's hard to say whether the speedup offsets it.
Use SOR. It's simple, doesn't add much computation, and can improve your convergence rate quite well, even for a relatively conservative relaxation value (say 1.5).
Use conjugate gradient. If this is for the projection step of a fluid simulation (i.e. enforcing non-compressability), you should be able to apply CG and get a much better convergence rate. A good preconditioner helps even more.
Use a specialized solver. If the linear system arises from the Poisson equation, you can do even better than conjugate gradient using an FFT-based methods.
If you can explain more about what the system you're trying to solve looks like, I can probably give some more advice on #3 and #4.
I think I've managed to optimize it, here's a code, create a new project in VC++, add this code and simply compile under "Release".
#include <iostream>
#include <cstdlib>
#include <cstring>
#define _WIN32_WINNT 0x0400
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <conio.h>
using namespace std;
size_t d_nx = 128, d_ny = 128;
float *d_x, *d_b, *d_w, *d_e, *d_s, *d_n;
void step_original() {
size_t ic = d_ny + 1, iw = d_ny, ie = d_ny + 2, is = 1, in = 2 * d_ny + 1;
for (size_t y = 1; y < d_ny - 1; ++y) {
for (size_t x = 1; x < d_nx - 1; ++x) {
d_x[ic] = d_b[ic]
- d_w[ic] * d_x[iw] - d_e[ic] * d_x[ie]
- d_s[ic] * d_x[is] - d_n[ic] * d_x[in];
++ic; ++iw; ++ie; ++is; ++in;
}
ic += 2; iw += 2; ie += 2; is += 2; in += 2;
}
}
void step_new() {
//size_t ic = d_ny + 1, iw = d_ny, ie = d_ny + 2, is = 1, in = 2 * d_ny + 1;
float
*d_b_ic,
*d_w_ic,
*d_e_ic,
*d_x_ic,
*d_x_iw,
*d_x_ie,
*d_x_is,
*d_x_in,
*d_n_ic,
*d_s_ic;
d_b_ic = d_b;
d_w_ic = d_w;
d_e_ic = d_e;
d_x_ic = d_x;
d_x_iw = d_x;
d_x_ie = d_x;
d_x_is = d_x;
d_x_in = d_x;
d_n_ic = d_n;
d_s_ic = d_s;
for (size_t y = 1; y < d_ny - 1; ++y)
{
for (size_t x = 1; x < d_nx - 1; ++x)
{
/*d_x[ic] = d_b[ic]
- d_w[ic] * d_x[iw] - d_e[ic] * d_x[ie]
- d_s[ic] * d_x[is] - d_n[ic] * d_x[in];*/
*d_x_ic = *d_b_ic
- *d_w_ic * *d_x_iw - *d_e_ic * *d_x_ie
- *d_s_ic * *d_x_is - *d_n_ic * *d_x_in;
//++ic; ++iw; ++ie; ++is; ++in;
d_b_ic++;
d_w_ic++;
d_e_ic++;
d_x_ic++;
d_x_iw++;
d_x_ie++;
d_x_is++;
d_x_in++;
d_n_ic++;
d_s_ic++;
}
//ic += 2; iw += 2; ie += 2; is += 2; in += 2;
d_b_ic += 2;
d_w_ic += 2;
d_e_ic += 2;
d_x_ic += 2;
d_x_iw += 2;
d_x_ie += 2;
d_x_is += 2;
d_x_in += 2;
d_n_ic += 2;
d_s_ic += 2;
}
}
void solve_original(size_t iters) {
for (size_t i = 0; i < iters; ++i) {
step_original();
}
}
void solve_new(size_t iters) {
for (size_t i = 0; i < iters; ++i) {
step_new();
}
}
void clear(float *a) {
memset(a, 0, d_nx * d_ny * sizeof(float));
}
int main(int argc, char **argv) {
size_t n = d_nx * d_ny;
d_x = new float[n]; clear(d_x);
d_b = new float[n]; clear(d_b);
d_w = new float[n]; clear(d_w);
d_e = new float[n]; clear(d_e);
d_s = new float[n]; clear(d_s);
d_n = new float[n]; clear(d_n);
if(argc < 3)
printf("app.exe (x)iters (o/n)algo\n");
bool bOriginalStep = (argv[2][0] == 'o');
size_t iters = atoi(argv[1]);
/*printf("Press any key to start!");
_getch();
printf(" Running speed test..\n");*/
__int64 freq, start, end, diff;
if(!::QueryPerformanceFrequency((LARGE_INTEGER*)&freq))
throw "Not supported!";
freq /= 1000000; // microseconds!
{
::QueryPerformanceCounter((LARGE_INTEGER*)&start);
if(bOriginalStep)
solve_original(iters);
else
solve_new(iters);
::QueryPerformanceCounter((LARGE_INTEGER*)&end);
diff = (end - start) / freq;
}
printf("Speed (%s)\t\t: %u\n", (bOriginalStep ? "original" : "new"), diff);
//_getch();
//cout << d_x[0] << endl; // prevent the thing from being optimized away
}
Run it like this:
app.exe 10000 o
app.exe 10000 n
"o" means old code, yours.
"n" is mine, the new one.
My results:
Speed (original):
1515028
1523171
1495988
Speed (new):
966012
984110
1006045
Improvement of about 30%.
The logic behind:
You've been using index counters to access/manipulate.
I use pointers.
While running, breakpoint at a certain calculation code line in VC++'s debugger, and press F8. You'll get the disassembler window.
The you'll see the produced opcodes (assembly code).
Anyway, look:
int *x = ...;
x[3] = 123;
This tells the PC to put the pointer x at a register (say EAX).
The add it (3 * sizeof(int)).
Only then, set the value to 123.
The pointers approach is much better as you can understand, because we cut the adding process, actually we handle it ourselves, thus able to optimize as needed.
I hope this helps.
Sidenote to stackoverflow.com's staff:
Great website, I hope I've heard of it long ago!
For one thing, there seems to be a pipelining issue here. The loop reads from the value in d_x that has just been written to, but apparently it has to wait for that write to complete. Just rearranging the order of the computation, doing something useful while it's waiting, makes it almost twice as fast:
d_x[ic] = d_b[ic]
- d_e[ic] * d_x[ie]
- d_s[ic] * d_x[is] - d_n[ic] * d_x[in]
- d_w[ic] * d_x[iw] /* d_x[iw] has just been written to, process this last */;
It was Eamon Nerbonne who figured this out. Many upvotes to him! I would never have guessed.
Poni's answer looks like the right one to me.
I just want to point out that in this type of problem, you often gain benefits from memory locality. Right now, the b,w,e,s,n arrays are all at separate locations in memory. If you could not fit the problem in L3 cache (mostly in L2), then this would be bad, and a solution of this sort would be helpful:
size_t d_nx = 128, d_ny = 128;
float *d_x;
struct D { float b,w,e,s,n; };
D *d;
void step() {
size_t ic = d_ny + 1, iw = d_ny, ie = d_ny + 2, is = 1, in = 2 * d_ny + 1;
for (size_t y = 1; y < d_ny - 1; ++y) {
for (size_t x = 1; x < d_nx - 1; ++x) {
d_x[ic] = d[ic].b
- d[ic].w * d_x[iw] - d[ic].e * d_x[ie]
- d[ic].s * d_x[is] - d[ic].n * d_x[in];
++ic; ++iw; ++ie; ++is; ++in;
}
ic += 2; iw += 2; ie += 2; is += 2; in += 2;
}
}
void solve(size_t iters) { for (size_t i = 0; i < iters; ++i) step(); }
void clear(float *a) { memset(a, 0, d_nx * d_ny * sizeof(float)); }
int main(int argc, char **argv) {
size_t n = d_nx * d_ny;
d_x = new float[n]; clear(d_x);
d = new D[n]; memset(d,0,n * sizeof(D));
solve(atoi(argv[1]));
cout << d_x[0] << endl; // prevent the thing from being optimized away
}
For example, this solution at 1280x1280 is a little less than 2x faster than Poni's solution (13s vs 23s in my test--your original implementation is then 22s), while at 128x128 it's 30% slower (7s vs. 10s--your original is 10s).
(Iterations were scaled up to 80000 for the base case, and 800 for the 100x larger case of 1280x1280.)
I think you're right about memory being a bottleneck. It's a pretty simple loop with just some simple arithmetic per iteration. the ic, iw, ie, is, and in indices seem to be on opposite sides of the matrix so i'm guessing that there's a bunch of cache misses there.
I'm no expert on the subject, but I've seen that there are several academic papers on improving the cache usage of the Gauss-Seidel method.
Another possible optimization is the use of the red-black variant, where points are updated in two sweeps in a chessboard-like pattern. In this way, all updates in a sweep are independent and can be parallelized.
I suggest putting in some prefetch statements and also researching "data oriented design":
void step_original() {
size_t ic = d_ny + 1, iw = d_ny, ie = d_ny + 2, is = 1, in = 2 * d_ny + 1;
float dw_ic, dx_ic, db_ic, de_ic, dn_ic, ds_ic;
float dx_iw, dx_is, dx_ie, dx_in, de_ic, db_ic;
for (size_t y = 1; y < d_ny - 1; ++y) {
for (size_t x = 1; x < d_nx - 1; ++x) {
// Perform the prefetch
// Sorting these statements by array may increase speed;
// although sorting by index name may increase speed too.
db_ic = d_b[ic];
dw_ic = d_w[ic];
dx_iw = d_x[iw];
de_ic = d_e[ic];
dx_ie = d_x[ie];
ds_ic = d_s[ic];
dx_is = d_x[is];
dn_ic = d_n[ic];
dx_in = d_x[in];
// Calculate
d_x[ic] = db_ic
- dw_ic * dx_iw - de_ic * dx_ie
- ds_ic * dx_is - dn_ic * dx_in;
++ic; ++iw; ++ie; ++is; ++in;
}
ic += 2; iw += 2; ie += 2; is += 2; in += 2;
}
}
This differs from your second method since the values are copied to local temporary variables before the calculation is performed.