Consider this scenario:
class A
{
std::shared_ptr<B> _b;
B* _raw;
A(std::shared_ptr<B> b)
{
_b = b;
_raw = b.get();
}
void foo()
{
// Use _raw instead of _b
// avoid one extra indirection / memory jump
// and also avoid polluting cache
}
};
I know that technically it works and offers a slight performance advantage (I tried it). (EDIT: False conclusion).
But my question is: Is it conceptually wrong? Is it bad practice? And why?
And if not why is this hack not more commonly used?
Here is a minimal reproducible example comparing raw pointer access to shared_ptr access:
#include <chrono>
#include <iostream>
#include <memory>
struct timer final
{
timer()
: start{std::chrono::system_clock::now()}
{ }
void elapsed()
{
auto now = std::chrono::system_clock::now();
std::cout << std::chrono::duration<double>(now - start).count() << " seconds" << std::endl;
}
private:
std::chrono::time_point<std::chrono::system_clock> start;
};
struct A
{
size_t a [2097152];
};
int main()
{
size_t data_size = 2097152;
size_t count = 10000000000;
// Using Raw pointer
A * pa = new A();
timer t0;
for(size_t i = 0; i < count; i++)
pa->a[i % data_size] = i;
t0.elapsed();
// Using shared_ptr
std::shared_ptr<A> sa = std::make_shared<A>();
timer t1;
for(size_t i = 0; i < count; i++)
sa->a[i % data_size] = i;
t1.elapsed();
}
Output:
3.98586 seconds
4.10491 seconds
I ran this multiple times and the results are consistent.
EDIT: As per the consensus in the answers, the above experiment is invalid. Compilers are way smarter than they appear.
This answer proves your test is invalid (correct performance measurements in C++ are quite hard since there are lots of pitfalls) and as a result you come to invalid conclusions.
Take a look on this godbolt.
for loop for first version:
.L39:
mov rdx, rax
and edx, 2097151
mov QWORD PTR [rbp+0+rdx*8], rax
add rax, 1
cmp rax, rcx
jne .L39
for loop for second version:
.L40:
mov rdx, rax
and edx, 2097151
mov QWORD PTR [rbp+16+rdx*8], rax
add rax, 1
cmp rax, rcx
jne .L40
I do not see a difference! Results should be exactly same.
So I suspect that you have done measurements when building in Debug configuration.
Here is version where you can compare this
What is more interesting clang is able to optimize away for loop if shared pointer is not used. It noticed there is no viable results of this loop and just remove it. So if you used release configuration compiler just outsmart you.
Bottom line:
shared_ptr do not provide overhead
when checking performance you must compile with optimizations enabled
you must also ensure if test code was not optimized away to be sure that results are valid.
Here is proper test written using google benchmark and test results for both cases are exactly the same.
A shared_ptr internally looks roughly like this:
template <typename _Tp>
struct shared_ptr {
T *_M_ptr;
control_block *cb;
}
So one member (_M_ptr) points to the managed object and one pointer to the control block (that is used for reference counting, and locking).
The oeprator-> will look something like this:
_Tp* operator->() const {
return _M_ptr;
}
Because the location of _M_ptr within shared_ptr is known, the compiler can directly retrieve the memory location from where to read the memory address stored in _M_ptr without the indirection.
If T *_M_ptr is the first member of shared_ptr(Which is indeed the case for libstdc++) a compiler could change this code:
std::shared_ptr<A> sa = std::make_shared<A>();
sa->a[0] = 1;
To something that would be similar to this:
std::shared_ptr<A> sa = std::make_shared<A>();
(*reinterpret_cast<A**>(&sa))->a[0] = 1;
And reinterpret_cast is a no-opt (won't create any machine code).
So in most cases, there won't be a difference between dereferencing a shared_ptr or a raw pointer. You might be able to construct a case where the memory address of a raw pointer is sorely stored in a register, and the same thing might not be possible for _M_ptr of shared_ptr, but that would IMHO be a really artificial example.
Regarding your performance test:
The shown code will be slightly slower for shared_ptr with no optimizations active because there the indirection over operator->() won't be optimized away. With optimizations on, the compiler might optimize - for the given example - everything away and your measurement can be meaningless.
And if not why is this hack not more commonly used?
In many cases, you use shared_ptr only to manage the ownership. When working on the object that is managed by a shared pointer, you often pass the object itself (by reference, or pointer) to another function that does not need ownership (void do_something_with_object(A *ptr) {…} and a call do_something_with_object(sa.get())) so even if there is a performance impact for a certain stdlib implementation, this won't manifest in most of the cases.
Is it conceptually wrong?
In general, it's not that wrong a priori to have internal supporting objects/helpers in order to achieve required performance goals. Caches for instance are the most popular example I guess. But here for your particular example, I tend to say, it's conceptually wrong even for the case of a slightly better performance (which I doubt at latest when significance is the keyword), since it's not really an internal quality since the raw pointer you use is not an internal of the shared_ptr. The issue I see here especially is the fact, that you duplicate responsibilites since you don't trust a well established standard class here for the sake of a minimal better performance. In software design, the keyword here besides single responsibility is proportionality. You have to duplicate semantics here for all relevant places (Copy/Move constructors), you have to think twice in terms of exception safety, you have to think about that aspect again and again if your class expands and so on. And it will become quite unintelligibly for other developers being faced with your code due to the entangled responsibilites.
A word about shared_ptr's performance:
SharedPtr-dereferencing within memory.h for my current VS 2019 looks like this:
template<class _Ty2 = _Ty,
enable_if_t<!is_array_v<_Ty2>, int> = 0>
_NODISCARD _Ty2 * operator->() const noexcept
{ // return pointer to resource
return (get());
}
And get() directly returns the raw pointer. So why should any common compiler's optimizer not be able to inline the hell out of this?
What assurances do I have that a core constant expression (as in [expr.const].2) possibly containing constexpr function calls will actually be evaluated at compile time and on which conditions does this depend?
The introduction of constexpr implicitly promises runtime performance improvements by moving computations into the translation stage (compile time).
However, the standard does not (and presumably cannot) mandate what code a compiler produces. (See [expr.const] and [dcl.constexpr]).
These two points appear to be at odds with each other.
Under which circumstances can one rely on the compiler resolving a core constant expression (which might contain an arbitrarily complicated computation) at compile time rather than deferring it to runtime?
At least under -O0 gcc appears to actually emit code and call for a constexpr function. Under -O1 and up it doesn't.
Do we have to resort to trickery such as this, that forces the constexpr through the template system:
template <auto V>
struct compile_time_h { static constexpr auto value = V; };
template <auto V>
inline constexpr auto compile_time = compile_time_h<V>::value;
constexpr int f(int x) { return x; }
int main() {
for (int x = 0; x < compile_time<f(42)>; ++x) {}
}
When a constexpr function is called and the output is assigned to a constexpr variable, it will always be run at compiletime.
Here's a minimal example:
// Compile with -std=c++14 or later
constexpr int fib(int n) {
int f0 = 0;
int f1 = 1;
for(int i = 0; i < n; i++) {
int hold = f0 + f1;
f0 = f1;
f1 = hold;
}
return f0;
}
int main() {
constexpr int blarg = fib(10);
return blarg;
}
When compiled at -O0, gcc outputs the following assembly for main:
main:
push rbp
mov rbp, rsp
mov DWORD PTR [rbp-4], 55
mov eax, 55
pop rbp
ret
Despite all optimization being turned off, there's never any call to fib in the main function itself.
This applies going all the way back to C++11, however in C++11 the fib function would have to be re-written to use conversion to avoid the use of mutable variables.
Why does the compiler include the assembly for fib in the executable sometimes? A constexpr function can be used at runtime, and when invoked at runtime it will behave like a regular function.
Used properly, constexpr can provide some performance benefits in specific cases, but the push to make everything constexpr is more about writing code that the compiler can check for Undefined Behavior.
What's an example of constexpr providing performance benefits? When implementing a function like std::visit, you need to create a lookup table of function pointers. Creating the lookup table every time std::visit is called would be costly, and assigning the lookup table to a static local variable would still result in measurable overhead because the program has to check if that variable's been initialized every time the function is run.
Thankfully, you can make the lookup table constexpr, and the compiler will actually inline the lookup table into the assembly code for the function so that the contents of the lookup table is significantly more likely to be inside the instruction cache when std::visit is run.
Does C++20 provide any mechanisms for guaranteeing that something runs at compiletime?
If a function is consteval, then the standard specifies that every call to the function must produce a compile-time constant.
This can be trivially used to force the compile-time evaluation of any constexpr function:
template<class T>
consteval T run_at_compiletime(T value) {
return value;
}
Anything given as a parameter to run_at_compiletime must be evaluated at compile-time:
constexpr int fib(int n) {
int f0 = 0;
int f1 = 1;
for(int i = 0; i < n; i++) {
int hold = f0 + f1;
f0 = f1;
f1 = hold;
}
return f0;
}
int main() {
// fib(10) will definitely run at compile time
return run_at_compiletime(fib(10));
}
Never; the C++ standard permits almost the entire compilation to occur at "runtime". Some diagnostics have to be done at compile time, but nothing prevents insanity on the part of the compiler.
Your binary could be a copy of the compiler with your source code appended, and C++ wouldn't say the compiler did anything wrong.
What you are looking at is a QoI - Quality of Implrmentation - issue.
In practice, constexpr variables tend to be compile time computed, and template parameters are always compile time computed.
consteval can also be used to markup functions.
I want to use vector<char> as a buffer. The interface is perfect for my needs, but there's a performance penalty when resizing it beyond its current size, since the memory is initialized. I don't need the initialization, since the data will be overwritten in any case by some third-party C functions. Is there a way or a specific allocator to avoid the initialization step? Note that I do want to use resize(), not other tricks like reserve() and capacity(), because I need size() to always represent the significative size of my "buffer" at any moment, while capacity() might be greater than its size after a resize(), so, again, I cannot rely on capacity() as a significative information for my application. Furthemore, the (new) size of the vector is never known in advance, so I cannot use std::array. If vector cannot be configured that way, I'd like to know what kind of container or allocator I could use instead of vector<char, std::alloc>. The only requirement is that the alternative to vector must at most be based on STL or Boost. I have access to C++11.
It is a known issue that initialization can not be turned off even explicitly for std::vector.
People normally implement their own pod_vector<> that does not do any initialization of the elements.
Another way is to create a type which is layout-compatible with char, whose constructor does nothing:
struct NoInitChar
{
char value;
NoInitChar() noexcept {
// do nothing
static_assert(sizeof *this == sizeof value, "invalid size");
static_assert(__alignof *this == __alignof value, "invalid alignment");
}
};
int main() {
std::vector<NoInitChar> v;
v.resize(10); // calls NoInitChar() which does not initialize
// Look ma, no reinterpret_cast<>!
char* beg = &v.front().value;
char* end = beg + v.size();
}
There's nothing in the standard library that meets your requirements, and nothing I know of in boost either.
There are three reasonable options I can think of:
Stick with std::vector for now, leave a comment in the code and come back to it if this ever causes a bottleneck in your application.
Use a custom allocator with empty construct/destroy methods - and hope your optimiser will be smart enough to remove any calls to them.
Create a wrapper around a a dynamically allocated array, implementing only the minimal functionality that you require.
As an alternative solution that works with vectors of different pod types:
template<typename V>
void resize(V& v, size_t newSize)
{
struct vt { typename V::value_type v; vt() {}};
static_assert(sizeof(vt[10]) == sizeof(typename V::value_type[10]), "alignment error");
typedef std::vector<vt, typename std::allocator_traits<typename V::allocator_type>::template rebind_alloc<vt>> V2;
reinterpret_cast<V2&>(v).resize(newSize);
}
And then you can:
std::vector<char> v;
resize(v, 1000); // instead of v.resize(1000);
This is most likely UB, even though it works properly for me for cases where I care more about performance. Difference in generated assembly as produced by clang:
test():
push rbx
mov edi, 1000
call operator new(unsigned long)
mov rbx, rax
mov edx, 1000
mov rdi, rax
xor esi, esi
call memset
mov rdi, rbx
pop rbx
jmp operator delete(void*)
test_noinit():
push rax
mov edi, 1000
call operator new(unsigned long)
mov rdi, rax
pop rax
jmp operator delete(void*)
So to summarize the various solutions found on stackoverflow:
use a special default-init allocator. (https://stackoverflow.com/a/21028912/1984766)
drawback: changes the vector-type to std::vector<char, default_init_allocator<char>> vec;
use a wrapper-struct struct NoInitChar around a char that has an empty constructor and therefore skips the value-initialization (https://stackoverflow.com/a/15220853/1984766)
drawback: changes the vector-type to std::vector<NoInitChar> vec;
temporarily cast the vector<char> to vector<NoInitChar> and resize it (https://stackoverflow.com/a/57053750/1984766)
drawback: does not change the type of the vector but you need to call your_resize_function (vec, x) instead of vec.resize (x).
With this post I wanted to point out that all of these methods need to be optimized by the compiler in order to speed up your program. I can confirm that the initialization of the new chars when resizing is indeed optimized away with every compiler I tested. So everything looks good ...
But --> since method 1 & 2 change the type of the vector, what happens when you use these vectors under more "complex" circumstances.
Consider this example:
#include <time.h>
#include <vector>
#include <string_view>
#include <iostream>
//high precision-timer
double get_time () {
struct timespec timespec;
::clock_gettime (CLOCK_MONOTONIC_RAW, ×pec);
return timespec.tv_sec + timespec.tv_nsec / (1000.0 * 1000.0 * 1000.0);
}
//method 1 --> special allocator
//reformated to make it readable
template <typename T, typename A = std::allocator<T>>
class default_init_allocator : public A {
private:
typedef std::allocator_traits<A> a_t;
public:
template<typename U>
struct rebind {
using other = default_init_allocator<U, typename a_t::template rebind_alloc<U>>;
};
using A::A;
template <typename U>
void construct (U* ptr) noexcept (std::is_nothrow_default_constructible<U>::value) {
::new (static_cast<void*>(ptr)) U;
}
template <typename U, typename...Args>
void construct (U* ptr, Args&&... args) {
a_t::construct (static_cast<A&>(*this), ptr, std::forward<Args>(args)...);
}
};
//method 2 --> wrapper struct
struct NoInitChar {
public:
NoInitChar () noexcept { }
NoInitChar (char c) noexcept : value (c) { }
public:
char value;
};
//some work to waste time
template<typename T>
void do_something (T& vec, std::string_view str) {
vec.push_back ('"');
vec.insert (vec.end (), str.begin (), str.end ());
vec.push_back ('"');
vec.push_back (',');
}
int main (int argc, char** argv) {
double timeBegin = get_time ();
std::vector<char> vec; //normal case
//std::vector<char, default_init_allocator<char>> vec; //method 1
//std::vector<NoInitChar> vec; //method 2
vec.reserve (256 * 1024 * 1024);
for (int i = 0; i < 1024 * 1024; ++i) {
do_something (vec, "foobar1");
do_something (vec, "foobar2");
do_something (vec, "foobar3");
do_something (vec, "foobar4");
do_something (vec, "foobar5");
do_something (vec, "foobar6");
do_something (vec, "foobar7");
do_something (vec, "foobar8");
vec.resize (vec.size () + 64);
}
double timeEnd = get_time ();
std::cout << (timeEnd - timeBegin) * 1000 << "ms" << std::endl;
return 0;
}
You would expect that method 1 & 2 outperform the normal vector with every "recent" compiler since the resize is free and the other operations are the same. Well think again:
g++ 7.5.0 g++ 8.4.0 g++ 9.3.0 clang++ 9.0.0
vector<char> 95ms 134ms 133ms 97ms
method 1 130ms 159ms 166ms 91ms
method 2 166ms 160ms 159ms 89ms
All test-applications are compiled like this and executed 50times taking the lowest measurement:
$(cc) -O3 -flto -std=c++17 sample.cpp
Encapsulate it.
Initialise it to the maximum size (not reserve).
Keep a reference to the iterator representing the end of the real size, as you put it.
Use begin and real end, instead of end, for your algorithms.
It's very rare that you would need to do this; I strongly encourage you to benchmark your situation to be absolutely sure this hack is needed.
Even then, I prefer the NoInitChar solution. (See Maxim's answer)
But if you're sure you would benefit from this, and NoInitChar doesn't work for you, and you're using clang, or gcc, or MSVC as your compiler, consider using folly's routine for this purpose.
See https://github.com/facebook/folly/blob/master/folly/memory/UninitializedMemoryHacks.h
The basic idea is that each of these library implementations has a routine for doing uninitialized resizing; you just need to call it.
While hacky, you can at least console yourself with knowing that facebook's C++ code relies on this hack working properly, so they'll be updating it if new versions of these library implementations require it.
Thanks to C++11 we received the std::function family of functor wrappers. Unfortunately, I keep hearing only bad things about these new additions. The most popular is that they are horribly slow. I tested it and they truly suck in comparison with templates.
#include <iostream>
#include <functional>
#include <string>
#include <chrono>
template <typename F>
float calc1(F f) { return -1.0f * f(3.3f) + 666.0f; }
float calc2(std::function<float(float)> f) { return -1.0f * f(3.3f) + 666.0f; }
int main() {
using namespace std::chrono;
const auto tp1 = system_clock::now();
for (int i = 0; i < 1e8; ++i) {
calc1([](float arg){ return arg * 0.5f; });
}
const auto tp2 = high_resolution_clock::now();
const auto d = duration_cast<milliseconds>(tp2 - tp1);
std::cout << d.count() << std::endl;
return 0;
}
111 ms vs 1241 ms. I assume this is because templates can be nicely inlined, while functions cover the internals via virtual calls.
Obviously templates have their issues as I see them:
they have to be provided as headers which is not something you might not wish to do when releasing your library as a closed code,
they may make the compilation time much longer unless extern template-like policy is introduced,
there is no (at least known to me) clean way of representing requirements (concepts, anyone?) of a template, bar a comment describing what kind of functor is expected.
Can I thus assume that functions can be used as de facto standard of passing functors, and in places where high performance is expected templates should be used?
Edit:
My compiler is the Visual Studio 2012 without CTP.
In general, if you are facing a design situation that gives you a choice, use templates. I stressed the word design because I think what you need to focus on is the distinction between the use cases of std::function and templates, which are pretty different.
In general, the choice of templates is just an instance of a wider principle: try to specify as many constraints as possible at compile-time. The rationale is simple: if you can catch an error, or a type mismatch, even before your program is generated, you won't ship a buggy program to your customer.
Moreover, as you correctly pointed out, calls to template functions are resolved statically (i.e. at compile time), so the compiler has all the necessary information to optimize and possibly inline the code (which would not be possible if the call were performed through a vtable).
Yes, it is true that template support is not perfect, and C++11 is still lacking a support for concepts; however, I don't see how std::function would save you in that respect. std::function is not an alternative to templates, but rather a tool for design situations where templates cannot be used.
One such use case arises when you need to resolve a call at run-time by invoking a callable object that adheres to a specific signature, but whose concrete type is unknown at compile-time. This is typically the case when you have a collection of callbacks of potentially different types, but which you need to invoke uniformly; the type and number of the registered callbacks is determined at run-time based on the state of your program and the application logic. Some of those callbacks could be functors, some could be plain functions, some could be the result of binding other functions to certain arguments.
std::function and std::bind also offer a natural idiom for enabling functional programming in C++, where functions are treated as objects and get naturally curried and combined to generate other functions. Although this kind of combination can be achieved with templates as well, a similar design situation normally comes together with use cases that require to determine the type of the combined callable objects at run-time.
Finally, there are other situations where std::function is unavoidable, e.g. if you want to write recursive lambdas; however, these restrictions are more dictated by technological limitations than by conceptual distinctions I believe.
To sum up, focus on design and try to understand what are the conceptual use cases for these two constructs. If you put them into comparison the way you did, you are forcing them into an arena they likely don't belong to.
Andy Prowl has nicely covered design issues. This is, of course, very important, but I believe the original question concerns more performance issues related to std::function.
First of all, a quick remark on the measurement technique: The 11ms obtained for calc1 has no meaning at all. Indeed, looking at the generated assembly (or debugging the assembly code), one can see that VS2012's optimizer is clever enough to realize that the result of calling calc1 is independent of the iteration and moves the call out of the loop:
for (int i = 0; i < 1e8; ++i) {
}
calc1([](float arg){ return arg * 0.5f; });
Furthermore, it realises that calling calc1 has no visible effect and drops the call altogether. Therefore, the 111ms is the time that the empty loop takes to run. (I'm surprised that the optimizer has kept the loop.) So, be careful with time measurements in loops. This is not as simple as it might seem.
As it has been pointed out, the optimizer has more troubles to understand std::function and doesn't move the call out of the loop. So 1241ms is a fair measurement for calc2.
Notice that, std::function is able to store different types of callable objects. Hence, it must perform some type-erasure magic for the storage. Generally, this implies a dynamic memory allocation (by default through a call to new). It's well known that this is a quite costly operation.
The standard (20.8.11.2.1/5) encorages implementations to avoid the dynamic memory allocation for small objects which, thankfully, VS2012 does (in particular, for the original code).
To get an idea of how much slower it can get when memory allocation is involved, I've changed the lambda expression to capture three floats. This makes the callable object too big to apply the small object optimization:
float a, b, c; // never mind the values
// ...
calc2([a,b,c](float arg){ return arg * 0.5f; });
For this version, the time is approximately 16000ms (compared to 1241ms for the original code).
Finally, notice that the lifetime of the lambda encloses that of the std::function. In this case, rather than storing a copy of the lambda, std::function could store a "reference" to it. By "reference" I mean a std::reference_wrapper which is easily build by functions std::ref and std::cref. More precisely, by using:
auto func = [a,b,c](float arg){ return arg * 0.5f; };
calc2(std::cref(func));
the time decreases to approximately 1860ms.
I wrote about that a while ago:
http://www.drdobbs.com/cpp/efficient-use-of-lambda-expressions-and/232500059
As I said in the article, the arguments don't quite apply for VS2010 due to its poor support to C++11. At the time of the writing, only a beta version of VS2012 was available but its support for C++11 was already good enough for this matter.
With Clang there's no performance difference between the two
Using clang (3.2, trunk 166872) (-O2 on Linux), the binaries from the two cases are actually identical.
-I'll come back to clang at the end of the post. But first, gcc 4.7.2:
There's already a lot of insight going on, but I want to point out that the result of the calculations of calc1 and calc2 are not the same, due to in-lining etc. Compare for example the sum of all results:
float result=0;
for (int i = 0; i < 1e8; ++i) {
result+=calc2([](float arg){ return arg * 0.5f; });
}
with calc2 that becomes
1.71799e+10, time spent 0.14 sec
while with calc1 it becomes
6.6435e+10, time spent 5.772 sec
that's a factor of ~40 in speed difference, and a factor of ~4 in the values. The first is a much bigger difference than what OP posted (using visual studio). Actually printing out the value a the end is also a good idea to prevent the compiler to removing code with no visible result (as-if rule). Cassio Neri already said this in his answer. Note how different the results are -- One should be careful when comparing speed factors of codes that perform different calculations.
Also, to be fair, comparing various ways of repeatedly calculating f(3.3) is perhaps not that interesting. If the input is constant it should not be in a loop. (It's easy for the optimizer to notice)
If I add a user supplied value argument to calc1 and 2 the speed factor between calc1 and calc2 comes down to a factor of 5, from 40! With visual studio the difference is close to a factor of 2, and with clang there is no difference (see below).
Also, as multiplications are fast, talking about factors of slow-down is often not that interesting. A more interesting question is, how small are your functions, and are these calls the bottleneck in a real program?
Clang:
Clang (I used 3.2) actually produced identical binaries when I flip between calc1 and calc2 for the example code (posted below). With the original example posted in the question both are also identical but take no time at all (the loops are just completely removed as described above). With my modified example, with -O2:
Number of seconds to execute (best of 3):
clang: calc1: 1.4 seconds
clang: calc2: 1.4 seconds (identical binary)
gcc 4.7.2: calc1: 1.1 seconds
gcc 4.7.2: calc2: 6.0 seconds
VS2012 CTPNov calc1: 0.8 seconds
VS2012 CTPNov calc2: 2.0 seconds
VS2015 (14.0.23.107) calc1: 1.1 seconds
VS2015 (14.0.23.107) calc2: 1.5 seconds
MinGW (4.7.2) calc1: 0.9 seconds
MinGW (4.7.2) calc2: 20.5 seconds
The calculated results of all binaries are the same, and all tests were executed on the same machine. It would be interesting if someone with deeper clang or VS knowledge could comment on what optimizations may have been done.
My modified test code:
#include <functional>
#include <chrono>
#include <iostream>
template <typename F>
float calc1(F f, float x) {
return 1.0f + 0.002*x+f(x*1.223) ;
}
float calc2(std::function<float(float)> f,float x) {
return 1.0f + 0.002*x+f(x*1.223) ;
}
int main() {
using namespace std::chrono;
const auto tp1 = high_resolution_clock::now();
float result=0;
for (int i = 0; i < 1e8; ++i) {
result=calc1([](float arg){
return arg * 0.5f;
},result);
}
const auto tp2 = high_resolution_clock::now();
const auto d = duration_cast<milliseconds>(tp2 - tp1);
std::cout << d.count() << std::endl;
std::cout << result<< std::endl;
return 0;
}
Update:
Added vs2015. I also noticed that there are double->float conversions in calc1,calc2. Removing them does not change the conclusion for visual studio (both are a lot faster but the ratio is about the same).
Different isn't the same.
It's slower because it does things that a template can't do. In particular, it lets you call any function that can be called with the given argument types and whose return type is convertible to the given return type from the same code.
void eval(const std::function<int(int)>& f) {
std::cout << f(3);
}
int f1(int i) {
return i;
}
float f2(double d) {
return d;
}
int main() {
std::function<int(int)> fun(f1);
eval(fun);
fun = f2;
eval(fun);
return 0;
}
Note that the same function object, fun, is being passed to both calls to eval. It holds two different functions.
If you don't need to do that, then you should not use std::function.
You already have some good answers here, so I'm not going to contradict them, in short comparing std::function to templates is like comparing virtual functions to functions.
You never should "prefer" virtual functions to functions, but rather you use virtual functions when it fits the problem, moving decisions from compile time to run time. The idea is that rather than having to solve the problem using a bespoke solution (like a jump-table) you use something that gives the compiler a better chance of optimizing for you. It also helps other programmers, if you use a standard solution.
This answer is intended to contribute, to the set of existing answers, what I believe to be a more meaningful benchmark for the runtime cost of std::function calls.
The std::function mechanism should be recognized for what it provides: Any callable entity can be converted to a std::function of appropriate signature. Suppose you have a library that fits a surface to a function defined by z = f(x,y), you can write it to accept a std::function<double(double,double)>, and the user of the library can easily convert any callable entity to that; be it an ordinary function, a method of a class instance, or a lambda, or anything that is supported by std::bind.
Unlike template approaches, this works without having to recompile the library function for different cases; accordingly, little extra compiled code is needed for each additional case. It has always been possible to make this happen, but it used to require some awkward mechanisms, and the user of the library would likely need to construct an adapter around their function to make it work. std::function automatically constructs whatever adapter is needed to get a common runtime call interface for all the cases, which is a new and very powerful feature.
To my view, this is the most important use case for std::function as far as performance is concerned: I'm interested in the cost of calling a std::function many times after it has been constructed once, and it needs to be a situation where the compiler is unable to optimize the call by knowing the function actually being called (i.e. you need to hide the implementation in another source file to get a proper benchmark).
I made the test below, similar to the OP's; but the main changes are:
Each case loops 1 billion times, but the std::function objects are constructed only once. I've found by looking at the output code that 'operator new' is called when constructing actual std::function calls (maybe not when they are optimized out).
Test is split into two files to prevent undesired optimization
My cases are: (a) function is inlined (b) function is passed by an ordinary function pointer (c) function is a compatible function wrapped as std::function (d) function is an incompatible function made compatible with a std::bind, wrapped as std::function
The results I get are:
case (a) (inline) 1.3 nsec
all other cases: 3.3 nsec.
Case (d) tends to be slightly slower, but the difference (about 0.05 nsec) is absorbed in the noise.
Conclusion is that the std::function is comparable overhead (at call time) to using a function pointer, even when there's simple 'bind' adaptation to the actual function. The inline is 2 ns faster than the others but that's an expected tradeoff since the inline is the only case which is 'hard-wired' at run time.
When I run johan-lundberg's code on the same machine, I'm seeing about 39 nsec per loop, but there's a lot more in the loop there, including the actual constructor and destructor of the std::function, which is probably fairly high since it involves a new and delete.
-O2 gcc 4.8.1, to x86_64 target (core i5).
Note, the code is broken up into two files, to prevent the compiler from expanding the functions where they are called (except in the one case where it's intended to).
----- first source file --------------
#include <functional>
// simple funct
float func_half( float x ) { return x * 0.5; }
// func we can bind
float mul_by( float x, float scale ) { return x * scale; }
//
// func to call another func a zillion times.
//
float test_stdfunc( std::function<float(float)> const & func, int nloops ) {
float x = 1.0;
float y = 0.0;
for(int i =0; i < nloops; i++ ){
y += x;
x = func(x);
}
return y;
}
// same thing with a function pointer
float test_funcptr( float (*func)(float), int nloops ) {
float x = 1.0;
float y = 0.0;
for(int i =0; i < nloops; i++ ){
y += x;
x = func(x);
}
return y;
}
// same thing with inline function
float test_inline( int nloops ) {
float x = 1.0;
float y = 0.0;
for(int i =0; i < nloops; i++ ){
y += x;
x = func_half(x);
}
return y;
}
----- second source file -------------
#include <iostream>
#include <functional>
#include <chrono>
extern float func_half( float x );
extern float mul_by( float x, float scale );
extern float test_inline( int nloops );
extern float test_stdfunc( std::function<float(float)> const & func, int nloops );
extern float test_funcptr( float (*func)(float), int nloops );
int main() {
using namespace std::chrono;
for(int icase = 0; icase < 4; icase ++ ){
const auto tp1 = system_clock::now();
float result;
switch( icase ){
case 0:
result = test_inline( 1e9);
break;
case 1:
result = test_funcptr( func_half, 1e9);
break;
case 2:
result = test_stdfunc( func_half, 1e9);
break;
case 3:
result = test_stdfunc( std::bind( mul_by, std::placeholders::_1, 0.5), 1e9);
break;
}
const auto tp2 = high_resolution_clock::now();
const auto d = duration_cast<milliseconds>(tp2 - tp1);
std::cout << d.count() << std::endl;
std::cout << result<< std::endl;
}
return 0;
}
For those interested, here's the adaptor the compiler built to make 'mul_by' look like a float(float) - this is 'called' when the function created as bind(mul_by,_1,0.5) is called:
movq (%rdi), %rax ; get the std::func data
movsd 8(%rax), %xmm1 ; get the bound value (0.5)
movq (%rax), %rdx ; get the function to call (mul_by)
cvtpd2ps %xmm1, %xmm1 ; convert 0.5 to 0.5f
jmp *%rdx ; jump to the func
(so it might have been a bit faster if I'd written 0.5f in the bind...)
Note that the 'x' parameter arrives in %xmm0 and just stays there.
Here's the code in the area where the function is constructed, prior to calling test_stdfunc - run through c++filt :
movl $16, %edi
movq $0, 32(%rsp)
call operator new(unsigned long) ; get 16 bytes for std::function
movsd .LC0(%rip), %xmm1 ; get 0.5
leaq 16(%rsp), %rdi ; (1st parm to test_stdfunc)
movq mul_by(float, float), (%rax) ; store &mul_by in std::function
movl $1000000000, %esi ; (2nd parm to test_stdfunc)
movsd %xmm1, 8(%rax) ; store 0.5 in std::function
movq %rax, 16(%rsp) ; save ptr to allocated mem
;; the next two ops store pointers to generated code related to the std::function.
;; the first one points to the adaptor I showed above.
movq std::_Function_handler<float (float), std::_Bind<float (*(std::_Placeholder<1>, double))(float, float)> >::_M_invoke(std::_Any_data const&, float), 40(%rsp)
movq std::_Function_base::_Base_manager<std::_Bind<float (*(std::_Placeholder<1>, double))(float, float)> >::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation), 32(%rsp)
call test_stdfunc(std::function<float (float)> const&, int)
In case you use a template instead of std::function in C++20 you can actually write your own concept with variadic templates for it (inspired by Hendrik Niemeyer's talk about C++20 concepts):
template<class Func, typename Ret, typename... Args>
concept functor = std::regular_invocable<Func, Args...> &&
std::same_as<std::invoke_result_t<Func, Args...>, Ret>;
You can then use it as functor<Ret, Args...> F> where Ret is the return value and Args... are the variadic input arguments. E.g. functor<double,int> F such as
template <functor<double,int> F>
auto CalculateSomething(F&& f, int const arg) {
return f(arg)*f(arg);
}
requires a functor as template argument which has to overload the () operator and has a double return value and a single input argument of type int. Similarly functor<double> would be a functor with double return type which does not take any input arguments.
Try it here!
You can also use it with variadic functions such as
template <typename... Args, functor<double, Args...> F>
auto CalculateSomething(F&& f, Args... args) {
return f(args...)*f(args...);
}
Try it here!
I found your results very interesting so I did a bit of digging to understand what is going on. First off as many others have said with out having the results of the computation effect the state of the program the compiler will just optimize this away. Secondly having a constant 3.3 given as an armament to the callback I suspect that there will be other optimizations going on. With that in mind I changed your benchmark code a little bit.
template <typename F>
float calc1(F f, float i) { return -1.0f * f(i) + 666.0f; }
float calc2(std::function<float(float)> f, float i) { return -1.0f * f(i) + 666.0f; }
int main() {
const auto tp1 = system_clock::now();
for (int i = 0; i < 1e8; ++i) {
t += calc2([&](float arg){ return arg * 0.5f + t; }, i);
}
const auto tp2 = high_resolution_clock::now();
}
Given this change to the code I compiled with gcc 4.8 -O3 and got a time of 330ms for calc1 and 2702 for calc2. So using the template was 8 times faster, this number looked suspects to me, speed of a power of 8 often indicates that the compiler has vectorized something. when I looked at the generated code for the templates version it was clearly vectoreized
.L34:
cvtsi2ss %edx, %xmm0
addl $1, %edx
movaps %xmm3, %xmm5
mulss %xmm4, %xmm0
addss %xmm1, %xmm0
subss %xmm0, %xmm5
movaps %xmm5, %xmm0
addss %xmm1, %xmm0
cvtsi2sd %edx, %xmm1
ucomisd %xmm1, %xmm2
ja .L37
movss %xmm0, 16(%rsp)
Where as the std::function version was not. This makes sense to me, since with the template the compiler knows for sure that the function will never change throughout the loop but with the std::function being passed in it could change, therefor can not be vectorized.
This led me to try something else to see if I could get the compiler to perform the same optimization on the std::function version. Instead of passing in a function I make a std::function as a global var, and have this called.
float calc3(float i) { return -1.0f * f2(i) + 666.0f; }
std::function<float(float)> f2 = [](float arg){ return arg * 0.5f; };
int main() {
const auto tp1 = system_clock::now();
for (int i = 0; i < 1e8; ++i) {
t += calc3([&](float arg){ return arg * 0.5f + t; }, i);
}
const auto tp2 = high_resolution_clock::now();
}
With this version we see that the compiler has now vectorized the code in the same way and I get the same benchmark results.
template : 330ms
std::function : 2702ms
global std::function: 330ms
So my conclusion is the raw speed of a std::function vs a template functor is pretty much the same. However it makes the job of the optimizer much more difficult.