Does passing fundamental values by const reference really hurt performance? - c++

Question

I'm writing a library which does numeric computations. I'm using templates so that the end-user can pick the precision they want. I would like this to work both with fundamental types (double, float) and high precision class types (e.g. boost::multiprecision). I'm wondering then if the argument types should be T or const & T.
On SO/google there are many posts about passing by value vs by reference. One of the "rules-of-thumb" seems to be:
Pass fundamental types by value
Pass everything else by const reference
However, this gets muddy if you have a template:
template<typename T>
T doSomething(T x, T y)
{
return x + y;
}
vs.
template<typename T>
T doSomething(const T & x, const T & y)
{
return x + y;
}
For boost::multiprecision you almost certainly want to pass by const reference. The question is whether passing double by const & is worse than by value. Many SO answers say const & is "no better and maybe worse"...but I couldn't find any good hard references.
I did the following benchmark
which seems to indicate there's no difference although maybe it depends on the simplicity of the function and inlining behavior.
There's the possibility of doing something like:
#include <type_traits>
template<typename T>
using choose_arg_type =
typename std::conditional<std::is_fundamental<T>::value,
T,
const T &>::type;
template <typename T>
T someFunc(choose_arg_type<T> arg)
{
return arg + arg;
}
int main()
{
auto result = someFunc<double>(0.0);
return 0;
}
But if it brings no benefit, it's added complexity and you lose type deduction (Any way to fix this type deduction?)
One reason I can think that pass by const reference is slower is, if it truly is using a reference, there may be cache locality issues. But if the compiler just optimizes to value...this won't matter.
What's the best way to handle this?

Answer 1

There is at least one circumstance where passing by const reference might disable optimizations. However, the most popular compilers provide a way to re-enable them.
Let’s look at this function:
int cryptographicHash( int& salt, const int& plaintext )
{
salt = 4; // Chosen by fair dice roll
// guaranteed to be random
return plaintext; // If we tell them there's a salt,
// this is the last hash function they'll
// ever suspect!
}
Looks pretty secure, right? But, since we’re writing in C++, is it as fast as it could possibly be? (Definitely what we want in a cryptographic hash.)
No, because what if you call it with:
int x = 0xFEED;
const int y = cryptographicHash( x, x );
Now the parameters passed by reference alias the same object, so the function should, as written, return 4, not 0xFEED. This means that, disastrously, the compiler can no longer optimize away the & in its const int& parameter.
However, the most popular compilers (including GCC, clang, Intel C++ and Visual C++ 2015 and up) all support the __restrict extension. So, change the function signature to int cryptographicHash( int& salt, const int& __restrict plaintext ) and all problems with it are solved forever.
Since this extension is not part of the C++ standard, you can improve portability with something like the following:
#if ( __GNUC__ || __clang__ || __INTEL_COMPILER || __ICL || _MSC_VER >= 1900 )
# define RESTRICT __restrict
#else
# define RESTRICT /**/
#endif
int cryptographicHash( int& salt, const int& RESTRICT plaintext );
(In GCC and clang, this does not appear to change the generated code.)

Answer 2

On platforms where the fundamental type in question fits into a register, a decent compiler should eliminate const references from parameters if it can see both sides of the call. For templates that is usually a given (unless they were explicitly instantiated somewhere). Since your library presumably has to be templated all the way down, this will apply to your case.
It's possible that your end users will have bad compilers or platforms where e.g. a double does not fit into a register. I don't see why you'd be incentivized to make micro-optimizations for these particular users, but maybe you do.
It's also possible that you want to explicitly instantiate all templates in your library for some set of types and provide implementation-less header files. In that case the user's compiler must obey whatever calling conventions exist on that platform and will probably pass fundamental types by reference.
Ultimately, the answer is "profile the relevant and representative use cases" if you don't have faith in the compiler(s).
Edit (removed macro solution): As suggested by Jarod42, the C++ way would be using an alias template. This also avoids the lack of deduction that the asker was running into with their original approach:
template<class T>
using CONSTREF = const T&; // Or just T for benchmarking.
https://godbolt.org/z/mopZ6B
As cppreference says:
Alias templates are never deduced by template argument deduction when deducing a template template parameter.

Answer 3

Passing something like int by reference (basically a pointer) is clearly sub-optimal since the extra indirection through the pointer can incur a cache miss and it may also prevent compiler optimizations since the compiler can't always know that the pointed-to variable cannot be changed by other entities, so it may in some cases be forced to do additional loads from memory. Passing by value removes the indirection and lets the compiler assume that noone else is changing the value.

Answer 4

This is a complex question that depends on architecture, compiler optimizations and many other specifics as the variation in answers shows. Since the OP is about writing template functions there is also the option of controlling which function is called by using SFINAE.
#include <iostream>
template <typename T, typename = typename std::enable_if_t<std::is_fundamental_v<T>> >
void f(T t) {
std::cout << "Pass by value\n";
}
template <typename T, typename = typename std::enable_if_t<not std::is_fundamental_v<T>> >
void f(T const &t) {
std::cout << "Pass by const ref.\n";
}
class myclass {};
int main() {
float x;
int i;
myclass c;
std::cout << "float: ";
f(x);
std::cout << "int: ";
f(i);
std::cout << "myclass: ";
f(c);
return 0;
}
Output:
float: Pass by value
int: Pass by value
myclass: Pass by const ref.

Answer 5

If an argument is trivially constructible and isn't modified, pass by value. The calling convention will automatically pass large structs by reference.
struct alignas(4096) page {unsigned char bytes[4096];};
[[nodiscard]] constexpr page operator^(page l, page r) noexcept {
for (int i = 0; i < 4096; ++i)
l.bytes[i] = l.bytes[i] ^ r.bytes[i];
return l;
}
Arguments that are modified and/or returned by nonconst reference must be passed by nonconst reference.
constexpr page& operator^=(page& l, page r) noexcept {return l = l ^ r;}
Pass any argument returned with const reference semantics by const reference.
using buffer = std::vector<unsigned char>;
[[nodiscard]] std::string_view to_string_view(const buffer& b) noexcept {
return {reinterpret_cast<const char*>(b.data()), b.size()};
}
Pass any argument deep copied to a different type by const reference.
[[nodiscard]] std::string to_string(const buffer& b) {
return std::string{to_string_view(b)};
}
Pass any non-trivially constructible, unmodified and non-deep copied argument by const reference.
std::ostream& operator<<(std::ostream& os, const buffer& b) {
os << std::hex;
for (const unsigned short u8 : b)
os << u8 << ',';
return os << std::dec;
}
Pass any argument deep copied to a value of the same type by value. There's no point in passing by reference an argument that's copied anyway, and the constructor for the returned copy is optimized away. See https://en.cppreference.com/w/cpp/language/copy_elision
[[nodiscard]] buffer operator^(buffer l, const buffer& r) {
const auto lsize = l.size();
const auto rsize = r.size();
const auto minsize = std::min(lsize, rsize);
for (buffer::size_type i = 0; i < minsize; ++i)
l[i] = l[i] ^ r[i];
if (lsize < rsize)
l.insert(l.end(), r.begin() + minsize, r.end());
return l;
}
This includes template functions as well.
template<typename T>
[[nodiscard]]
constexpr T clone(T t) noexcept(std::is_nothrow_constructible_v<T, T>) {
return t;
}
Otherwise take arguments of template parameter type by forwarding reference (&&). Note: && only has forwarding (universal) reference semantics in arguments of template parameter type, and/or for auto&& or decltype(auto)&&.
template<typename T>
constexpr bool nt = noexcept(std::is_nothrow_constructible_v<int, T&&>);
template<typename T>
[[nodiscard]]
constexpr int to_int(T&& t) noexcept(nt<T>) {return static_cast<int>(t);}
const auto to_int_lambda = [](auto&& t) noexcept(noexcept(to_int(t))) {
return to_int(t);
};