The const and volatile chapter on the 'Surviving the Release Version' Article gave me the idea that the compiler can use the const keyword as hint for its optimization job.
Do you know some other optimization-hints for the compiler or design principles for functions so that the compiler can make them inline?
By the way, do you declare primitive-type function parameters as const or const reference (like void foo(const int i) or void foo(const int& i))?
Thanks!
It is rare that const qualification can help the compiler to optimize your code. You can read more about why this is the case in Herb Sutter's "Constant Optimization?"
Concerning your last question: in general, you should prefer to pass by value things that are cheap to copy (like fundamental type objects--ints and floats and such--and small class type objects) and pass other types by const reference. This is a very general rule and there are lots of caveats and exceptions.
As soon as you enable some optimization the compiler will notice that the parameter i is never modified, so whether you declare it as int or as const int doesn't matter for the generated code.
The point of passing parameters by const & is to avoid needless copying. In case of small parameters (one machine word or less) this doesn't lead to better performance, so you shouldn't do that. foo(int) is more efficient than foo(const int&).
There's no practical benefit to either form. If the type is less than a single machine word, take it by value. The other thing is that a modern compiler's semantic analysis is way above what const can and can't do, you could only apply optimizations if it was pre-compiled or your code was VERY complex. The article you linked to is several years old and the compiler has done nothing but improve massively since then.
I don't think the compiler can use const keyword for optimization since at any point the constness can be casted away.
It is more for correctness than optimization.
A few "general compiler" things off the top of my head.
const as a hint that a variable will never change
volatile as a hint that a variable can change at any point
restrict keyword
memory barriers (to hint to the compiler a specific ordering) - probably not so much an "optimisation" mind you.
inline keyword (use very carefully)
All that however should only come from an extensive profiling routine so you know what actually needs to be optimised. Compilers in general are pretty good at optimising without much in the way of hints from the programmer.
If you look into the sources of Linux kernel or some such similar projects, you will find all the optimisation clues that are passed on to gcc (or whichever compiler is used). Linux kernel uses every feature that gcc offers even if it is not in the standard.
This page sums up gcc's extensions to the C language. I referred C here because const and volatile are used in C as well. More than C or C++, compiler optimization appears the focus of the question here.
I don't think the real purpose of const has much to do with optimization, though it helps.
Isn't the real value in compile-time checking, to prevent you from modifying things you shouldn't modify, i.e. preventing bugs?
For small arguments that you are not going to modify, use call-by-value.
For large arguments that you are not going to modify, use either call-by-reference or passing the address (which are basically the same thing), along with const.
For large or small arguments that you are going to modify, drop the const.
BTW: In case it's news, for real performance, you need to know how to find the problems you actually have, by profiling. No compiler can do that for you.
Related
If I have a member function declared like so:
double* restrict data(){
return m_data; // array member variable
}
can the restrict keyword do anything?
Apparently, with g++ (x86 architecture) it cannot, but are there other compilers/architectures where this type of construction makes sense, and would allow for optimized machine code generation?
I'm asking because the Blitz library (Blitz++) has a whole slew of functions declared in this manner, and it doesn't make sense that someone would go in and add the restrict keyword unless it actually does something. So before I go in and remove the restrict's (to get rid of compiler warnings) I'd like to know how I'm abusing the code.
WHAT restrict ARE WE TALKING ABOUT?
restrict is, as it currently stands, non-standard.. which means that it's a compiler extension; it's non-portable in the sense that the C++ Standard doesn't mandate its existance, nor is there any formal text in it that tells us what it is supposed to do.
restrict is currently compiler specific in C++, and one has to resort to the compiler documentation of their choice to see exactly what it is doing.
SOME THOUGHTS
There are many papers about the usage of restrict, among them:
Restricted Pointers - Using the GNU Compiler Collection
restrict - wikipedia.org
Demystifying The Restrict Keyword - CellPerformance
It's hinted at several places that the purpose of restrict is to qualify pointers so that the compiler knows that two pointers in the same scope doesn't refer to the same memory location.
With this in mind we can easily see that the return-type has no potential collision with other pointers, so using it in such context will generally not gain any optimization opportunities. However; one must refer to the documented behaviour of the used implementation to know for sure.. as stated: restrict is not standard, yet.
I also found the following thread where the developers of Blitz++ discusses the removal of strict applied to the return-type of a function, since it doesn't do anything:
Re: [Blitz-devel] type qualifiers ignored on function return type
A LITTLE NOTE
As a further note, here's what the LLVM Documentation says about noalias vs restrict:
For function return values, C99’s restrict is not meaningful, while LLVM’s noalias is.
Generaly restrict qualifier can only help to better optimize code. By removing 'restrict' you don't break anything, but when you add it without care you can get some errors. A great example is the difference between memcpy and memmove. You can always use slower memmove, but you can use faster memcpy only if you know that src and dst aren't overlaping.
I'm interested in writing good code from the beginning instead of optimizing the code later. Sorry for not providing benchmark I don't have a working scenario at the moment. Thanks for your attention!
What are the performance gains of using FunctionY over FunctionX?
There is a lot of discussion on stackoverflow about this already but I'm in doubts in the case when accessing sub-members (recursive) as shown below. Will the compiler (say VS2008) optimize FunctionX into something like FunctionY?
void FunctionX(Obj * pObj)
{
pObj->MemberQ->MemberW->MemberA.function1();
pObj->MemberQ->MemberW->MemberA.function2();
pObj->MemberQ->MemberW->MemberB.function1();
pObj->MemberQ->MemberW->MemberB.function2();
..
pObj->MemberQ->MemberW->MemberZ.function1();
pObj->MemberQ->MemberW->MemberZ.function2();
}
void FunctionY(Obj * pObj)
{
W * localPtr = pObj->MemberQ->MemberW;
localPtr->MemberA.function1();
localPtr->MemberA.function2();
localPtr->MemberB.function1();
localPtr->MemberB.function2();
...
localPtr->MemberZ.function1();
localPtr->MemberZ.function2();
}
In case none of the member pointers are volatile or pointers to volatile and you don't have the operator -> overloaded for any members in a chain both functions are the same.
The optimization rule you suggested is widely known as Common Expression Elimination and is supported by vast majority of compilers for many decades.
In theory, you save on the extra pointer dereferences, HOWEVER, in the real world, the compiler will probably optimize it out for you, so it's a useless optimization.
This is why it's important to profile first, and then optimize later. The compiler is doing everything it can to help you, you might as well make sure you're not just doing something it's already doing.
if the compiler is good enough, it should translate functionX into something similar to functionY.
But you can have different result on different compiler and on the same compiler with different optimization flag.
Using a "dumb" compiler functionY should be faster, and IMHO it is more readable and faster to code. So stick with functionY
ps. you should take a look at some code style guide, normally member and function name should always start with a low-case letter
Is there a difference if it is the first use of the variable or not. For example are a and b treated differently?
void f(bool&a, bool& b)
{
...
a=false;
boost::this_thread::sleep...//1 sec sleep
a=true;
b=true;
...
}
EDIT: people asked why I want to know this.
1. I would like to have some way to tell the compiler not to optimize(swap the order of the execution of the instructions) in some function, and using atomic and or mutexes is much more complicated than using sleep(and in my case sleeping is not a performance problem).
2. Like I said this is generally important to know.
We can't really tell. On scenario could be that the compiler has full introspection to your function at the calling site (and possibly does inline it), in which case it can jumble your function with the caller, and then do optimizations appropriately.
It could then e.g. completely optimize away a and b because there is no code that depends on a and b. Or it might see that you violate aliasing rules so that a and b refer to the same entity, and then merge them according to your program flow.
But it could also be that you tell the compiler to not optimize at all, e.g. with g++'s -O0 flag, in which case not much will happen.
The only proof for your particular platform *, can be made by looking at the generated assembly, or by telling the compiler to please output some log about what it optimizes (g++ has many flags for that).
* compiler+flags used to compile compiler+version+add-ons, hardware, operating system; even the weather might be relevant if your compiler omits some optimizations if it takes to long [which would actually be cool feature for debug builds, imho]
They are not local (because they are references), so it can't, because it can't tell whether the called function sees them or not and has to assume that it does. If they were local variables, it could, because local variables are not visible to the called function unless pointer or reference to them was created.
This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
Constants and compiler optimization in C++
Let the holy wars begin:
I've heard a number of differing opinions on the usefulness of const in C++. Of course it has uses in member function declarations, etc. But how useful is it as a modifier on variables (or rather, constants)? Does it indeed help the optimizer, if the rest of the code is left the same?
There are a lot of cases where the const modifier won't help the optimizer, for the simple fact that the compiler can already tell if you modified a variable or not. The biggest benefit of const, in my opinion, is that it tells the compiler whether the programmer intended to modify that variable, which is useful in finding certain types of semantic errors at compile time instead of run time. Any error you can move to compile time is a huge boost in programmer productivity.
const does not help the optimizer.
Since const can be cast away with const_cast, it's possible to write programs that use const in a number of places, then cast it away and modify variables anyway, with defined behavior according to the standard. The compiler therefore must look at the actual code of the program to determine which variables are modified when, and it's probably pretty good at this anyway (for example it might determine a non-const variable is invariant over a certain block of code and optimize accordingly).
If the compiler blindly treated const as a guarantee that something won't change, the optimizer would break some well-formed programs.
const is a compile-time feature to help programmers write correct code, by adding some compile-time constraints, and indicating a code contract (eg. 'I promise not to change this parameter'). It has nothing to do with optimization. While invariants are important to optimizers, this has nothing to do with the const keyword.
There is one exception: objects declared with const. These cannot be modified; even if they are via casting, the behavior is undefined. There's some subtlety here:
const int ci = 5;
const_cast<int&>(ci) = 5; // undefined behavior, original object declared const
int i = 5;
const int& ci2 = i; // cannot modify i through ci2, const reference
const_cast<int&>(ci2) = 5; // OK, original object not declared const
So when the compiler sees const int ci it probably does assume it will never, ever change, because modifying it is undefined behavior. However, chances are this isn't the bottleneck in your program, it's just a more sophisticated #define. Apart from that, const is weak - just a keyword for the type system.
In general, no, it will not help the compiler. Since the const-ness can be casted away in a second in both C and C++, it'd be hard for the compiler to make the necessary assumptions about fulfilled code requirements for optimizations.
That said, const-correctness should always be used for its other benefits.
It can't hurt and in theory could allow for some optimizations so you might as well use it - don't know if any production compilers do.
Visual Studio includes support for __forceinline. The Microsoft Visual Studio 2005 documentation states:
The __forceinline keyword overrides
the cost/benefit analysis and relies
on the judgment of the programmer
instead.
This raises the question: When is the compiler's cost/benefit analysis wrong? And, how am I supposed to know that it's wrong?
In what scenario is it assumed that I know better than my compiler on this issue?
You know better than the compiler only when your profiling data tells you so.
The one place I am using it is licence verification.
One important factor to protect against easy* cracking is to verify being licenced in multiple places rather than only one, and you don't want these places to be the same function call.
*) Please don't turn this in a discussion that everything can be cracked - I know. Also, this alone does not help much.
The compiler is making its decisions based on static code analysis, whereas if you profile as don says, you are carrying out a dynamic analysis that can be much farther reaching. The number of calls to a specific piece of code is often largely determined by the context in which it is used, e.g. the data. Profiling a typical set of use cases will do this. Personally, I gather this information by enabling profiling on my automated regression tests. In addition to forcing inlines, I have unrolled loops and carried out other manual optimizations on the basis of such data, to good effect. It is also imperative to profile again afterwards, as sometimes your best efforts can actually lead to decreased performance. Again, automation makes this a lot less painful.
More often than not though, in my experience, tweaking alogorithms gives much better results than straight code optimization.
I've developed software for limited resource devices for 9 years or so and the only time I've ever seen the need to use __forceinline was in a tight loop where a camera driver needed to copy pixel data from a capture buffer to the device screen. There we could clearly see that the cost of a specific function call really hogged the overlay drawing performance.
The only way to be sure is to measure performance with and without. Unless you are writing highly performance critical code, this will usually be unnecessary.
For SIMD code.
SIMD code often uses constants/magic numbers. In a regular function, every const __m128 c = _mm_setr_ps(1,2,3,4); becomes a memory reference.
With __forceinline, compiler can load it once and reuse the value, unless your code exhausts registers (usually 16).
CPU caches are great but registers are still faster.
P.S. Just got 12% performance improvement by __forceinline alone.
The inline directive will be totally of no use when used for functions which are:
recursive,
long,
composed of loops,
If you want to force this decision using __forceinline
Actually, even with the __forceinline keyword. Visual C++ sometimes chooses not to inline the code. (Source: Resulting assembly source code.)
Always look at the resulting assembly code where speed is of importance (such as tight inner loops needed to be run on each frame).
Sometimes using #define instead of inline will do the trick. (of course you loose a lot of checking by using #define, so use it only when and where it really matters).
Actually, boost is loaded with it.
For example
BOOST_CONTAINER_FORCEINLINE flat_tree& operator=(BOOST_RV_REF(flat_tree) x)
BOOST_NOEXCEPT_IF( (allocator_traits_type::propagate_on_container_move_assignment::value ||
allocator_traits_type::is_always_equal::value) &&
boost::container::container_detail::is_nothrow_move_assignable<Compare>::value)
{ m_data = boost::move(x.m_data); return *this; }
BOOST_CONTAINER_FORCEINLINE const value_compare &priv_value_comp() const
{ return static_cast<const value_compare &>(this->m_data); }
BOOST_CONTAINER_FORCEINLINE value_compare &priv_value_comp()
{ return static_cast<value_compare &>(this->m_data); }
BOOST_CONTAINER_FORCEINLINE const key_compare &priv_key_comp() const
{ return this->priv_value_comp().get_comp(); }
BOOST_CONTAINER_FORCEINLINE key_compare &priv_key_comp()
{ return this->priv_value_comp().get_comp(); }
public:
// accessors:
BOOST_CONTAINER_FORCEINLINE Compare key_comp() const
{ return this->m_data.get_comp(); }
BOOST_CONTAINER_FORCEINLINE value_compare value_comp() const
{ return this->m_data; }
BOOST_CONTAINER_FORCEINLINE allocator_type get_allocator() const
{ return this->m_data.m_vect.get_allocator(); }
BOOST_CONTAINER_FORCEINLINE const stored_allocator_type &get_stored_allocator() const
{ return this->m_data.m_vect.get_stored_allocator(); }
BOOST_CONTAINER_FORCEINLINE stored_allocator_type &get_stored_allocator()
{ return this->m_data.m_vect.get_stored_allocator(); }
BOOST_CONTAINER_FORCEINLINE iterator begin()
{ return this->m_data.m_vect.begin(); }
BOOST_CONTAINER_FORCEINLINE const_iterator begin() const
{ return this->cbegin(); }
BOOST_CONTAINER_FORCEINLINE const_iterator cbegin() const
{ return this->m_data.m_vect.begin(); }
There are several situations where the compiler is not able to determine categorically whether it is appropriate or beneficial to inline a function. Inlining may involve trade-off's that the compiler is unwilling to make, but you are (e.g,, code bloat).
In general, modern compilers are actually pretty good at making this decision.
When you know that the function is going to be called in one place several times for a complicated calculation, then it is a good idea to use __forceinline. For instance, a matrix multiplication for animation may need to be called so many times that the calls to the function will start to be noticed by your profiler. As said by the others, the compiler can't really know about that, especially in a dynamic situation where the execution of the code is unknown at compile time.
wA Case For noinline
I wanted to pitch in with an unusual suggestion and actually vouch for __noinline in MSVC or the noinline attribute/pragma in GCC and ICC as an alternative to try out first over __forceinline and its equivalents when staring at profiler hotspots. YMMV but I've gotten so much more mileage (measured improvements) out of telling the compiler what to never inline than what to always inline. It also tends to be far less invasive and can produce much more predictable and understandable hotspots when profiling the changes.
While it might seem very counter-intuitive and somewhat backward to try to improve performance by telling the compiler what not to inline, I'd claim based on my experience that it's much more harmonious with how optimizing compilers work and far less invasive to their code generation. A detail to keep in mind that's easy to forget is this:
Inlining a callee can often result in the caller, or the caller of the caller, to cease to be inlined.
This is what makes force inlining a rather invasive change to the code generation that can have chaotic results on your profiling sessions. I've even had cases where force inlining a function reused in several places completely reshuffled all top ten hotspots with the highest self-samples all over the place in very confusing ways. Sometimes it got to the point where I felt like I'm fighting with the optimizer making one thing faster here only to exchange a slowdown elsewhere in an equally common use case, especially in tricky cases for optimizers like bytecode interpretation. I've found noinline approaches so much easier to use successfully to eradicate a hotspot without exchanging one for another elsewhere.
It would be possible to inline functions much less invasively if we
could inline at the call site instead of determining whether or not
every single call to a function should be inlined. Unfortunately, I've
not found many compilers supporting such a feature besides ICC. It
makes much more sense to me if we are reacting to a hotspot to respond
by inlining at the call site instead of making every single call of a
particular function forcefully inlined. Lacking this wide support
among most compilers, I've gotten far more successful results with
noinline.
Optimizing With noinline
So the idea of optimizing with noinline is still with the same goal in mind: to help the optimizer inline our most critical functions. The difference is that instead of trying to tell the compiler what they are by forcefully inlining them, we are doing the opposite and telling the compiler what functions definitely aren't part of the critical execution path by forcefully preventing them from being inlined. We are focusing on identifying the rare-case non-critical paths while leaving the compiler still free to determine what to inline in the critical paths.
Say you have a loop that executes for a million iterations, and there is a function called baz which is only very rarely called in that loop once every few thousand iterations on average in response to very unusual user inputs even though it only has 5 lines of code and no complex expressions. You've already profiled this code and the profiler shows in the disassembly that calling a function foo which then calls baz has the largest number of samples with lots of samples distributed around calling instructions. The natural temptation might be to force inline foo. I would suggest instead to try marking baz as noinline and time the results. I've managed to make certain critical loops execute 3 times faster this way.
Analyzing the resulting assembly, the speedup came from the foo function now being inlined as a result of no longer inlining baz calls into its body.
I've often found in cases like these that marking the analogical baz with noinline produces even bigger improvements than force inlining foo. I'm not a computer architecture wizard to understand precisely why but glancing at the disassembly and the distribution of samples in the profiler in such cases, the result of force inlining foo was that the compiler was still inlining the rarely-executed baz on top of foo, making foo more bloated than necessary by still inlining rare-case function calls. By simply marking baz with noinline, we allow foo to be inlined when it wasn't before without actually also inlining baz. Why the extra code resulting from inlining baz as well slowed down the overall function is still not something I understand precisely; in my experience, jump instructions to more distant paths of code always seemed to take more time than closer jumps, but I'm at a loss as to why (maybe something to do with the jump instructions taking more time with larger operands or something to do with the instruction cache). What I can definitely say for sure is that favoring noinline in such cases offered superior performance to force inlining and also didn't have such disruptive results on the subsequent profiling sessions.
So anyway, I'd suggest to give noinline a try instead and reach for it first before force inlining.
Human vs. Optimizer
In what scenario is it assumed that I know better than my compiler on
this issue?
I'd refrain from being so bold as to assume. At least I'm not good enough to do that. If anything, I've learned over the years the humbling fact that my assumptions are often wrong once I check and measure things I try with the profiler. I have gotten past the stage (over a couple of decades of making my profiler my best friend) to avoid completely blind stabs at the dark only to face humbling defeat and revert my changes, but at my best, I'm still making, at most, educated guesses. Still, I've always known better than my compiler, and hopefully, most of us programmers have always known this better than our compilers, how our product is supposed to be designed and how it is is going to most likely be used by our customers. That at least gives us some edge in the understanding of common-case and rare-case branches of code that compilers don't possess (at least without PGO and I've never gotten the best results with PGO). Compilers don't possess this type of runtime information and foresight of common-case user inputs. It is when I combine this user-end knowledge, and with a profiler in hand, that I've found the biggest improvements nudging the optimizer here and there in teaching it things like what to inline or, more commonly in my case, what to never inline.