Performance difference between member function and global function in release version - c++

Question

I have implemented two functions to perform the cross product of two Vectors (not std::vector), one is a member function and another is a global one, here is the key codes(additional parts are ommitted)
//for member function
template <typename Scalar>
SquareMatrix<Scalar,3> Vector<Scalar,3>::outerProduct(const Vector<Scalar,3> &vec3) const
{
SquareMatrix<Scalar,3> result;
for(unsigned int i = 0; i < 3; ++i)
for(unsigned int j = 0; j < 3; ++j)
result(i,j) = (*this)[i]*vec3[j];
return result;
}
//for global function: Dim = 3
template<typename Scalar, int Dim>
void outerProduct(const Vector<Scalar, Dim> & v1 , const Vector<Scalar, Dim> & v2, SquareMatrix<Scalar, Dim> & m)
{
for (unsigned int i=0; i<Dim; i++)
for (unsigned int j=0; j<Dim; j++)
{
m(i,j) = v1[i]*v2[j];
}
}
They are almost the same except that one is a member function having a return value and another is a global function where the values calculated are straightforwardly assigned to a square matrix, thus requiring no return value.
Actually, I was meant to replace the member one by the global one to improve the performance, since the first one involes copy operations. The strange thing, however, is that the time cost by the global function is almost two times longer than the member one. Furthermore, I find that the execution of
m(i,j) = v1[i]*v2[j]; // in global function
requires much more time than that of
result(i,j) = (*this)[i]*vec3[j]; // in member function
So the question is, how does this performance difference between member and global function arise?
Anyone can tell the reasons?
Hope I have presented my question clearly, and sorry to my poor english!
//----------------------------------------------------------------------------------------
More information added:
The following is the codes I use to test the performance:
//the codes below is in a loop
Vector<double, 3> vec1;
Vector<double, 3> vec2;
Timer timer;
timer.startTimer();
for (unsigned int i=0; i<100000; i++)
{
SquareMatrix<double,3> m = vec1.outerProduct(vec2);
}
timer.stopTimer();
std::cout<<"time cost for member function: "<< timer.getElapsedTime()<<std::endl;
timer.startTimer();
SquareMatrix<double,3> m;
for (unsigned int i=0; i<100000; i++)
{
outerProduct(vec1, vec2, m);
}
timer.stopTimer();
std::cout<<"time cost for global function: "<< timer.getElapsedTime()<<std::endl;
std::system("pause");
and the result captured:
You can see that the member funtion is almost twice faster than the global one.
Additionally, my project is built upon a 64bit windows system, and the codes are in fact used to generate the static lib files based on the Scons construction tools, along with vs2010 project files produced.
I have to remind that the strange performance difference only occurs in a release version, while in a debug build type, the global function is almost five times faster than the member one.(about 0.10s vs 0.02s)

Answer 1

One possible explanation:
With inlining, in the first case, compiler may knows that result(i, j) (from local variable) doesn't alias this[i] or vec3[j], and so neither of the Scalar array of this nor vec3 are modified.
In the second case, from the function point of view, the variables may alias, so each write into m might modify Scalars of v1 or v2, so neither of v1[i] nor v2[j] can be cached.
You may try the restrict keyword extension to check if my hypothesis is correct.

Answer 2

EDIT: loop elision in the original assembly has been corrected
[paraphrased] Why is the performance different between the member function and the static function?
I'll start with the simplest things mentioned in your question, and progress to the more nuanced points of performance testing / analysis.
It is a bad idea to measure performance of debug builds. Compilers take liberties in many places, such as zeroing arrays that are uninitialized, generating extra code that isn't strictly necessary, and (obviously) not performing any optimization past the trivial ones such as constant propagation. This leads to the next point...
Always look at the assembly. C and C++ are high level languages when it comes to the subtleties of performance. Many people even consider x86 assembly a high level language since each instruction is decomposed into possibly several micro-ops during decoding. You cannot tell what the computer is doing just by looking at C++ code. For example, depending on how you implemented SquareMatrix, the compiler may or may not be able to perform copy elision during optimization.
Entering the somewhat more nuanced topics when testing for performance...
Make sure the compiler is actually generating loops. Using your example test code, g++ 4.7.2 doesn't actually generate loops with my implementation of SquareMatrix and Vector. I implemented them to initialize all components to 0.0, so the compiler can statically determine that the values never change, and so only generates a single set of mov instructions instead of a loop. In my example code, I use COMPILER_NOP which (with gcc) is __asm__ __volatile__("":::) inside the loop to prevent this (as compilers cannot predict side-effects from manual assembly, and so cannot elide the loop). Edit: I DO use COMPILER_NOP but since the output values from the functions are never used, the compiler is still able to remove the bulk of the work from the loop, and reduce the loop to this:
.L7
subl $1, %eax
jne .L7
I have corrected this by performing additional operations inside the loop. The loop now assigns a value from the output to the inputs, preventing this optimization and forcing the loop to cover what was originally intended.
To (finally) get around to answering your question: When I implemented the rest of what is needed to get your code to run, and verified by checking the assembly that loops are actually generated, the two functions execute in the same amount of time. They even have nearly identical implementations in assembly.
Here's the assembly for the member function:
movsd 32(%rsp), %xmm7
movl $100000, %eax
movsd 24(%rsp), %xmm5
movsd 8(%rsp), %xmm6
movapd %xmm7, %xmm12
movsd (%rsp), %xmm4
movapd %xmm7, %xmm11
movapd %xmm5, %xmm10
movapd %xmm5, %xmm9
mulsd %xmm6, %xmm12
mulsd %xmm4, %xmm11
mulsd %xmm6, %xmm10
mulsd %xmm4, %xmm9
movsd 40(%rsp), %xmm1
movsd 16(%rsp), %xmm0
jmp .L7
.p2align 4,,10
.p2align 3
.L12:
movapd %xmm3, %xmm1
movapd %xmm2, %xmm0
.L7:
movapd %xmm0, %xmm8
movapd %xmm1, %xmm3
movapd %xmm1, %xmm2
mulsd %xmm1, %xmm8
movapd %xmm0, %xmm1
mulsd %xmm6, %xmm3
mulsd %xmm4, %xmm2
mulsd %xmm7, %xmm1
mulsd %xmm5, %xmm0
subl $1, %eax
jne .L12
and the assembly for the static function:
movsd 32(%rsp), %xmm7
movl $100000, %eax
movsd 24(%rsp), %xmm5
movsd 8(%rsp), %xmm6
movapd %xmm7, %xmm12
movsd (%rsp), %xmm4
movapd %xmm7, %xmm11
movapd %xmm5, %xmm10
movapd %xmm5, %xmm9
mulsd %xmm6, %xmm12
mulsd %xmm4, %xmm11
mulsd %xmm6, %xmm10
mulsd %xmm4, %xmm9
movsd 40(%rsp), %xmm1
movsd 16(%rsp), %xmm0
jmp .L9
.p2align 4,,10
.p2align 3
.L13:
movapd %xmm3, %xmm1
movapd %xmm2, %xmm0
.L9:
movapd %xmm0, %xmm8
movapd %xmm1, %xmm3
movapd %xmm1, %xmm2
mulsd %xmm1, %xmm8
movapd %xmm0, %xmm1
mulsd %xmm6, %xmm3
mulsd %xmm4, %xmm2
mulsd %xmm7, %xmm1
mulsd %xmm5, %xmm0
subl $1, %eax
jne .L13
In conclusion: You probably need to tighten your code up a bit before you can tell whether the implementations differ on your system. Make sure your loops are actually being generated (look at the assembly) and see whether the compiler was able to elide the return value from the member function.
If those things are true and you still see differences, can you post the implementations here for SquareMatrix and Vector so we can give you some more info?
Full code, a makefile, and the generated assembly for my working example is available as a GitHub gist.

Answer 3

Explicit instantiations of template function produce performance difference?
Some experiments I have done to look for the performance difference:
1.
Firstly I suspected that the performance difference may be caused by the implementation itself. In fact, we have two sets of implementation, one is implemented by ourself(this one is quite similar to codes by #black), and another is implemented to serve as a wrapper of Eigen::Matrix, which is controlled by a macro on-off, But switch between these two implementation does not make any change, the global one is still slower than the member one.
2.
Since these codes(classVector<Scalar, Dim> & SquareMatrix<Scalar, Dim>) are implemented in a large project, then I guess that the performance difference may probably be influenced by other codes(though I think it impossible, but still worth a try). So I extract all necessary codes(implementation by ourself used), and put them in my manually-generated VS2010 project. Surprisingly but also normally, I find that the global one is slightly faster than the member one, which is the same result as #black #Myles Hathcock, even though I leave the implementation of codes unchanged.
3.
Because in our project, outerProduct are put into a release lib files, while in my manually-generate project, it straightforwardly produce the .obj files, and be link to .exe files. To exclude this issue, I use the codes extracted and produce the lib file through VS2010, and apply this lib file to another VS project to test the performance difference, but still the global one is slight faster than the member one. So, both codes have the same implementation and both of them are put into lib files, though one is produced by Scons and the other is generated by VS project, but they have different performance. Is Scons causing this problem?
4.
For the codes shown in my question, global function outerProduct is declared and defined in .h file, then #include by a .cpp file. So when compiling this .cpp file, outerProduct will be instantiated. But if I change this to another manner:(I have to remind that these codes are now compiled by Scons to product lib file, not manually-generated VS2010 project)
First, I declare the global function outerProduct in .h file:
\\outProduct.h
template<typename Scalar, int Dim>
void outerProduct(const Vector<Scalar, Dim> & v1 , const Vector<Scalar, Dim> & v2, SquareMatrix<Scalar, Dim> & m);
then in .cpp file,
\\outerProduct.cpp
template<typename Scalar, int Dim>
void outerProduct(const Vector<Scalar, Dim> & v1 , const Vector<Scalar, Dim> & v2, SquareMatrix<Scalar, Dim> & m)
{
for (unsigned int i=0; i<Dim; i++)
for (unsigned int j=0; j<Dim; j++)
{
m(i,j) = v1[i]*v2[j];
}
}
Since it a template function, It requires some explicit instantiations:
\\outerProduct.cpp
template void outerProduct<double, 3>(const Vector<double, 3> &, const Vector<double, 3> &, SquareMatrix<double, 3> &);
template void outerProduct<float, 3>(const Vector<float, 3> &, const Vector<float, 3> &, SquareMatrix<float, 3> &);
Finally, in .cpp file calling this function:
\\use_outerProduct.cpp
#include "outerProduct.h" //note: outerProduct.cpp is not needful.
...
outerProduct(v1, v2, m)
...
The strange thing, now, is that the global one finally be slightly faster than the member one, shown in the following picture:
But this only happens in a Scons environment. In mannually-generated VS2010 project, global one will always be slightly faster than the member one. So this performance difference only results from a Scons environment? and if template function being explicitly instantiated, it will become normal?
Things are still strange ! It seems that Scons would have done something I didn't expected.
//------------------------------------------------------------------------
Additionally, test codes are now changed to the followings to avoid the loop elision:
Vector<double, 3> vec1(0.0);
Vector<double, 3> vec2(1.0);
Timer timer;
while(true)
{
timer.startTimer();
for (unsigned int i=0; i<100000; i++)
{
vec1 = Vector<double, 3>(i);
SquareMatrix<double,3> m = vec1.outerProduct(vec2);
}
timer.stopTimer();
cout<<"time cost for member function: "<< timer.getElapsedTime()<<endl;
timer.startTimer();
SquareMatrix<double,3> m;
for (unsigned int i=0; i<100000; i++)
{
vec1 = Vector<double, 3>(i);
outerProduct(vec1, vec2, m);
}
timer.stopTimer();
cout<<"time cost for global function: "<< timer.getElapsedTime()<<endl;
system("pause");
}
#black #Myles Hathcock, Great thanks to warm-hearted people!
#Myles Hathcock, your explanation is really a subtle and abstruse one, but I think I would benefit a lot from it.
Finally, the entire implementation is on
https://github.com/FeiZhu/Physika
Which is a physical engine we are developing, and from which you can find more info including the whole source codes. Vector and SquareMatrix are defined in Physika_Src/Physika_Core folder! But global function outerProduct is not uploaded, you can add it appropriately to somewhere.