Lets say that you have a function which generates some security token for your application, such as some hash salt, or maybe a symetric or asymetric key.
Now lets say that you have this function in your C++ as a constexpr and that you generate keys for your build based on some information (like, the build number, a timestamp, something else).
You being a diligent programmer make sure and call this in the appropriate ways to ensure it's only called at compile time, and thus the dead stripper removes the code from the final executable.
However, you can't ever be sure that someone else isn't going to call it in an unsafe way, or that maybe the compiler won't strip the function out, and then your security token algorithm will become public knowledge, making it more easy for would be attackers to guess future tokens.
Or, security aside, let's say the function takes a long time to execute and you want to make sure it never happens during runtime and causes a bad user experience for your end users.
Are there any ways to ensure that a constexpr function can never be called at runtime? Or alternately, throwing an assert or similar at runtime would be ok, but not as ideal obviously as a compile error would be.
I've heard that there is some way involving throwing an exception type that doesn't exist, so that if the constexpr function is not deadstripped out, you'll get a linker error, but have heard that this only works on some compilers.
Distantly related question: Force constexpr to be evaluated at compile time
In C++20 you can just replace constexpr by consteval to enforce a function to be always evaluated at compile time.
Example:
int rt_function(int v){ return v; }
constexpr int rt_ct_function(int v){ return v; }
consteval int ct_function(int v){ return v; }
int main(){
constexpr int ct_value = 1; // compile value
int rt_value = 2; // runtime value
int a = rt_function(ct_value);
int b = rt_ct_function(ct_value);
int c = ct_function(ct_value);
int d = rt_function(rt_value);
int e = rt_ct_function(rt_value);
int f = ct_function(rt_value); // ERROR: runtime value
constexpr int g = rt_function(ct_value); // ERROR: runtime function
constexpr int h = rt_ct_function(ct_value);
constexpr int i = ct_function(ct_value);
}
Pre C++20 workaround
You can enforce the use of it in a constant expression:
#include<utility>
template<typename T, T V>
constexpr auto ct() { return V; }
template<typename T>
constexpr auto func() {
return ct<decltype(std::declval<T>().value()), T{}.value()>();
}
template<typename T>
struct S {
constexpr S() {}
constexpr T value() { return T{}; }
};
template<typename T>
struct U {
U() {}
T value() { return T{}; }
};
int main() {
func<S<int>>();
// won't work
//func<U<int>>();
}
By using the result of the function as a template argument, you got an error if it can't be solved at compile-time.
A theoretical solution (as templates should be Turing complete) - don't use constexpr functions and fall back onto the good-old std=c++0x style of computing using exclusively struct template with values. For example, don't do
constexpr uintmax_t fact(uint n) {
return n>1 ? n*fact(n-1) : (n==1 ? 1 : 0);
}
but
template <uint N> struct fact {
uintmax_t value=N*fact<N-1>::value;
}
template <> struct fact<1>
uintmax_t value=1;
}
template <> struct fact<0>
uintmax_t value=0;
}
The struct approach is guaranteed to be evaluated exclusively at compile time.
The fact the guys at boost managed to do a compile time parser is a strong signal that, albeit tedious, this approach should be feasible - it's a one-off cost, maybe one can consider it an investment.
For example:
to power struct:
// ***Warning: note the unusual order of (power, base) for the parameters
// *** due to the default val for the base
template <unsigned long exponent, std::uintmax_t base=10>
struct pow_struct
{
private:
static constexpr uintmax_t at_half_pow=pow_struct<exponent / 2, base>::value;
public:
static constexpr uintmax_t value=
at_half_pow*at_half_pow*(exponent % 2 ? base : 1)
;
};
// not necessary, but will cut the recursion one step
template <std::uintmax_t base>
struct pow_struct<1, base>
{
static constexpr uintmax_t value=base;
};
template <std::uintmax_t base>
struct pow_struct<0,base>
{
static constexpr uintmax_t value=1;
};
The build token
template <uint vmajor, uint vminor, uint build>
struct build_token {
constexpr uintmax_t value=
vmajor*pow_struct<9>::value
+ vminor*pow_struct<6>::value
+ build_number
;
}
In the upcoming C++20 there will be consteval specifier.
consteval - specifies that a function is an immediate function, that is, every call to the function must produce a compile-time constant
Since now we have C++17, there is an easier solution:
template <auto V>
struct constant {
constexpr static decltype(V) value = V;
};
The key is that non-type arguments can be declared as auto. If you are using standards before C++17 you may have to use std::integral_constant. There is also a proposal about the constant helper class.
An example:
template <auto V>
struct constant {
constexpr static decltype(V) value = V;
};
constexpr uint64_t factorial(int n) {
if (n <= 0) {
return 1;
}
return n * factorial(n - 1);
}
int main() {
std::cout << "20! = " << constant<factorial(20)>::value << std::endl;
return 0;
}
Have your function take template parameters instead of arguments and implement your logic in a lambda.
#include <iostream>
template< uint64_t N >
constexpr uint64_t factorial() {
// note that we need to pass the lambda to itself to make the recursive call
auto f = []( uint64_t n, auto& f ) -> uint64_t {
if ( n < 2 ) return 1;
return n * f( n - 1, f );
};
return f( N, f );
}
using namespace std;
int main() {
cout << factorial<5>() << std::endl;
}
Related
Here is my code:
template<int... I>
class MetaString1
{
public:
constexpr MetaString1(constexpr char* str)
: buffer_{ encrypt(str[I])... } { }
const char* decrypt()
{
for (int i = 0; i < sizeof...(I); ++i)
buffer_[i] = decrypt1(buffer_[i]);
buffer_[sizeof...(I)] = 0;
return buffer_;
}
private:
constexpr char encrypt(constexpr char c) const { return c ^ 0x55; }
constexpr char decrypt1(constexpr char c) const { return encrypt(c); }
private:
char buffer_[sizeof...(I)+1];
};
#define OBFUSCATED1(str) (MetaString1<0, 1, 2, 3, 4, 5>(str).decrypt())
int main()
{
constexpr char *var = OBFUSCATED1("Post Malone");
std::cout << var << std::endl;
return 1;
}
This is the code from the paper that I'm reading Here. The Idea is simple, to XOR the argument of OBFUSCATED1 and then decrypt back to original value.
The problem that I'm having is that VS 2017 gives me error saying function call must have a constant value in constant expression.
If I only leave OBFUSCATED1("Post Malone");, I have no errors and program is run, but I've noticed that if I have breakpoints in constexpr MetaString1 constructor, the breakpoint is hit, which means that constexpr is not evaluated during compile time. As I understand it's because I don't "force" compiler to evaluate it during compilation by assigning the result to a constexpr variable.
So I have two questions:
Why do I have error function call must have a constant value in constant expression?
Why do people use template classes when they use constexpr functions? As I know template classes get evaluated during compilation, so using template class with constexpr is just a way to push compiler to evaluate those functions during compilation?
You try to assign a non constexpr type to a constexpr type variable,
what's not possible
constexpr char *var = OBFUSCATED1("Post Malone")
// ^^^ ^^^^^^^^^^^
// type of var is constexpr, return type of OBFUSCATED1 is const char*
The constexpr keyword was introduced in C++11, so before you had this keyword you had to write complicated TMP stuff to make the compiler do stuff at compile time. Since TMP is turing complete you theoretically don't need something more than TMP, but since TMP is slow to compile and ugly to ready, you are able to use constexpr to express things you want evaluate at compile time in a more readable way. Although there is no correlation between TMP and constexpr, what means, you are free to use constexpr without template classes.
To achieve what you want, you could save both versions of the string:
template <class T>
constexpr T encrypt(T l, T r)
{
return l ^ r;
}
template <std::size_t S, class U>
struct in;
template <std::size_t S, std::size_t... I>
struct in<S, std::index_sequence<I...>>
{
constexpr in(const char str[S])
: str_{str[I]...}
, enc_{encrypt(str[I], char{0x12})...}
{}
constexpr const char* dec() const
{
return str_;
}
constexpr const char* enc() const
{
return enc_;
}
protected:
char str_[S];
char enc_[S];
};
template <std::size_t S>
class MetaString1
: public in<S, std::make_index_sequence<S - 1>>
{
public:
using base1_t = in<S, std::make_index_sequence<S - 1>>;
using base1_t::base1_t;
constexpr MetaString1(const char str[S])
: base1_t{str}
{}
};
And use it like this:
int main()
{
constexpr char str[] = "asdffasegeasf";
constexpr MetaString1<sizeof(str)> enc{str};
std::cout << enc.dec() << std::endl;
std::cout << enc.enc() << std::endl;
}
Consider the following code that implements a compile time counter.
#include <iostream>
template<int>
struct Flag { friend constexpr int flag(Flag); };
template<int N>
struct Writer
{
friend constexpr int flag(Flag<N>) { return 0; }
};
template<int N>
constexpr int reader(float, Flag<N>) { return N; }
template<int N, int = flag(Flag<N>{})>
constexpr int reader(int, Flag<N>, int value = reader(0, Flag<N + 1>{}))
{
return value;
}
template<int N = reader(0, Flag<0>{}), int = sizeof(Writer<N>) >
constexpr int next() { return N; }
int main() {
constexpr int a = next();
constexpr int b = next();
constexpr int c = next();
constexpr int d = next();
std::cout << a << b << c << d << '\n'; // 0123
}
For the second reader overload, if I put the default parameter inside the body of the function, like so:
template<int N, int = flag(Flag<N>{})>
constexpr int reader(int, Flag<N>)
{
return reader(0, Flag<N + 1>{});
}
Then the output will become:
0111
Why does this happen? What makes the second version not work anymore?
If it matters, I'm using Visual Studio 2015.2.
Without value being passed as a parameter, nothing stops the compiler from caching the call to reader(0, Flag<1>).
In both cases first next() call will work as expected since it will immediately result in SFINAEing to reader(float, Flag<0>).
The second next() will evaluate reader<0,0>(int, ...), which depends on reader<1>(float, ...) that can be cached if it does not depend on a value parameter.
Unfortunately (and ironically) the best source I found that confirms that constexpr calls can be cached is #MSalters comment to this question.
To check if your particular compiler caches/memoizes, consider calling
constexpr int next_c() { return next(); }
instead of next(). In my case (VS2017) the output turns into 0000.
next() is protected from caching by the fact that its default template arguments actually change with every instantiation, so it's a new separate function every time. next_c() is not a template at all, so it can be cached, and so is reader<1>(float, ...).
I do believe that this is not a bug and compiler can legitimately expect constexprs in compile-time context to be pure functions.
Instead it is this code that should be considered ill-formed - and it soon will be, as others noted.
The relevance of value is that it participates in overload resolution. Under SFINAE rules, template instantiation errors silently exclude candidates from overload resolution. But it does instantiate Flag<N+1>, which causes the overload resolution to become viable the next time (!). So in effect you're counting successful instantiations.
Why does your version behave differently? You still reference Flag<N+1>, but in the implementation of the function. This is important. With function templates, the declaration must be considered for SFINAE, but only the chosen overload is then instantiated. Your declaration is just template<int N, int = flag(Flag<N>{})> constexpr int reader(int, Flag<N>); and does not depend on Flag<N+1>.
As noted in the comments, don't count on this counter ;)
Lets say that you have a function which generates some security token for your application, such as some hash salt, or maybe a symetric or asymetric key.
Now lets say that you have this function in your C++ as a constexpr and that you generate keys for your build based on some information (like, the build number, a timestamp, something else).
You being a diligent programmer make sure and call this in the appropriate ways to ensure it's only called at compile time, and thus the dead stripper removes the code from the final executable.
However, you can't ever be sure that someone else isn't going to call it in an unsafe way, or that maybe the compiler won't strip the function out, and then your security token algorithm will become public knowledge, making it more easy for would be attackers to guess future tokens.
Or, security aside, let's say the function takes a long time to execute and you want to make sure it never happens during runtime and causes a bad user experience for your end users.
Are there any ways to ensure that a constexpr function can never be called at runtime? Or alternately, throwing an assert or similar at runtime would be ok, but not as ideal obviously as a compile error would be.
I've heard that there is some way involving throwing an exception type that doesn't exist, so that if the constexpr function is not deadstripped out, you'll get a linker error, but have heard that this only works on some compilers.
Distantly related question: Force constexpr to be evaluated at compile time
In C++20 you can just replace constexpr by consteval to enforce a function to be always evaluated at compile time.
Example:
int rt_function(int v){ return v; }
constexpr int rt_ct_function(int v){ return v; }
consteval int ct_function(int v){ return v; }
int main(){
constexpr int ct_value = 1; // compile value
int rt_value = 2; // runtime value
int a = rt_function(ct_value);
int b = rt_ct_function(ct_value);
int c = ct_function(ct_value);
int d = rt_function(rt_value);
int e = rt_ct_function(rt_value);
int f = ct_function(rt_value); // ERROR: runtime value
constexpr int g = rt_function(ct_value); // ERROR: runtime function
constexpr int h = rt_ct_function(ct_value);
constexpr int i = ct_function(ct_value);
}
Pre C++20 workaround
You can enforce the use of it in a constant expression:
#include<utility>
template<typename T, T V>
constexpr auto ct() { return V; }
template<typename T>
constexpr auto func() {
return ct<decltype(std::declval<T>().value()), T{}.value()>();
}
template<typename T>
struct S {
constexpr S() {}
constexpr T value() { return T{}; }
};
template<typename T>
struct U {
U() {}
T value() { return T{}; }
};
int main() {
func<S<int>>();
// won't work
//func<U<int>>();
}
By using the result of the function as a template argument, you got an error if it can't be solved at compile-time.
A theoretical solution (as templates should be Turing complete) - don't use constexpr functions and fall back onto the good-old std=c++0x style of computing using exclusively struct template with values. For example, don't do
constexpr uintmax_t fact(uint n) {
return n>1 ? n*fact(n-1) : (n==1 ? 1 : 0);
}
but
template <uint N> struct fact {
uintmax_t value=N*fact<N-1>::value;
}
template <> struct fact<1>
uintmax_t value=1;
}
template <> struct fact<0>
uintmax_t value=0;
}
The struct approach is guaranteed to be evaluated exclusively at compile time.
The fact the guys at boost managed to do a compile time parser is a strong signal that, albeit tedious, this approach should be feasible - it's a one-off cost, maybe one can consider it an investment.
For example:
to power struct:
// ***Warning: note the unusual order of (power, base) for the parameters
// *** due to the default val for the base
template <unsigned long exponent, std::uintmax_t base=10>
struct pow_struct
{
private:
static constexpr uintmax_t at_half_pow=pow_struct<exponent / 2, base>::value;
public:
static constexpr uintmax_t value=
at_half_pow*at_half_pow*(exponent % 2 ? base : 1)
;
};
// not necessary, but will cut the recursion one step
template <std::uintmax_t base>
struct pow_struct<1, base>
{
static constexpr uintmax_t value=base;
};
template <std::uintmax_t base>
struct pow_struct<0,base>
{
static constexpr uintmax_t value=1;
};
The build token
template <uint vmajor, uint vminor, uint build>
struct build_token {
constexpr uintmax_t value=
vmajor*pow_struct<9>::value
+ vminor*pow_struct<6>::value
+ build_number
;
}
In the upcoming C++20 there will be consteval specifier.
consteval - specifies that a function is an immediate function, that is, every call to the function must produce a compile-time constant
Since now we have C++17, there is an easier solution:
template <auto V>
struct constant {
constexpr static decltype(V) value = V;
};
The key is that non-type arguments can be declared as auto. If you are using standards before C++17 you may have to use std::integral_constant. There is also a proposal about the constant helper class.
An example:
template <auto V>
struct constant {
constexpr static decltype(V) value = V;
};
constexpr uint64_t factorial(int n) {
if (n <= 0) {
return 1;
}
return n * factorial(n - 1);
}
int main() {
std::cout << "20! = " << constant<factorial(20)>::value << std::endl;
return 0;
}
Have your function take template parameters instead of arguments and implement your logic in a lambda.
#include <iostream>
template< uint64_t N >
constexpr uint64_t factorial() {
// note that we need to pass the lambda to itself to make the recursive call
auto f = []( uint64_t n, auto& f ) -> uint64_t {
if ( n < 2 ) return 1;
return n * f( n - 1, f );
};
return f( N, f );
}
using namespace std;
int main() {
cout << factorial<5>() << std::endl;
}
What I want to do is:
int const bitsPerInt = log2(X);
bitset<bitsPerInt> bits(a random number...);
but I get this error:
'bitsPerInt' cannot appear in a constant expression
error: template argument 1 is invalid
If you really need this to work, make your own log2 that works in compile-time and pass it to bitset's template argument.
constexpr unsigned Log2(unsigned n, unsigned p = 0) {
return (n <= 1) ? p : Log2(n / 2, p + 1);
}
constexpr size_t bitCount = Log2(X);
std::bitset<bitCount> bits;
Live example.
Here's the solution using template meta-programming i.e. without using constexpr:
template<int N,unsigned int P=0>
struct Log2 { enum { value = Log2<N/2,P+1>::value }; };
template <unsigned p>
struct Log2<0, p> { enum { value = p }; };
template <unsigned p>
struct Log2<1, p> { enum { value = p }; };
std::bitset<Log2<4>::value> bits;
Live example.
This version should work in both C++03 and C++11; however, if you've access to C++11, I'd still recommend the constexpr way since it's cleaner (easier to understand).
Template parameter needs to be known(and constant if it is a value and not a type) at compile time. This is how templates work in C++. Templates actually generate real code for each specific version of the generic code.
I would like to know if it is possible to have sort of compile time loops.
For example, I have the following templated class:
template<class C, int T=10, int B=10>
class CountSketch
{
public:
CountSketch()
{
hashfuncs[0] = &CountSketch<C>::hash<0>;
hashfuncs[1] = &CountSketch<C>::hash<1>;
// ... for all i until i==T which is known at compile time
};
private:
template<int offset>
size_t hash(C &c)
{
return (reinterpret_cast<int>(&c)+offset)%B;
}
size_t (CountSketch::*hashfuncs[T])(C &c);
};
I would thus like to know if I can do a loop to initialize the T hash functions using a loop. The bounds of the loops are known at compile time, so, in principle, I don't see any reason why it couldn't be done (especially since it works if I unroll the loop manually).
Of course, in this specific example, I could just have made a single hash function with 2 parameters (although it would be less efficient I guess). I am thus not interested in solving this specific problem, but rather knowing if "compile time loops" existed for similar cases.
Thanks!
Nope, it's not directly possible. Template metaprogramming is a pure functional language. Every value or type defined through it are immutable. A loop inherently requires mutable variables (Repeatedly test some condition until X happens, then exit the loop).
Instead, you would typically rely on recursion. (Instantiate this template with a different template parameter each time, until you reach some terminating condition).
However, that can solve all the same problems as a loop could.
Edit: Here's a quick example, computing the factorial of N using recursion at compile-time:
template <int N>
struct fac {
enum { value = N * fac<N-1>::value };
};
template <>
struct fac<0> {
enum { value = 1 };
};
int main() {
assert(fac<4>::value == 24);
}
Template metaprogramming in C++ is a Turing-complete language, so as long as you don't run into various internal compiler limits, you can solve basically any problem with it.
However, for practical purposes, it may be worth investigating libraries like Boost.MPL, which contains a large number of data structures and algorithms which simplify a lot of metaprogramming tasks.
Yes. Possible using compile time recursion.
I was trying with your code but since it was not compilable here is a modified and compiling exmaple:
template<class C, int T=10>
class CountSketch
{
template<int N>
void Init ()
{
Init<N-1>();
hashfuncs[N] = &CountSketch<C>::template hash<N>;
cout<<"Initializing "<<N<<"th element\n";
}
public:
CountSketch()
{
Init<T>();
}
private:
template<int offset>
size_t hash(C &c)
{
return 0;
}
size_t (CountSketch::*hashfuncs[T])(C &c);
};
template<>
template<>
void CountSketch<int,10>::Init<0> ()
{
hashfuncs[0] = &CountSketch<int,10>::hash<0>;
cout<<"Initializing "<<0<<"th element\n";
}
Demo. The only constraint of this solution is that you have to provide the final specialized version as, CountSketch<int,10>::Init<0> for whatever type and size.
You need a combination of boost::mpl::for_each and boost::mpl::range_c.
Note: This will result in run-time code and this is what you actually need. Because there is no way to know the result of operator& at compile time. At least none that I'm aware of.
The actual difficulty with this is to build a struct that is templated on an int parameter (mpl::int_ in our case) and that does the assignment when operator() is called and we also need a functor to actually capture the this pointer.
This is somewhat more complicated than I anticipated but it's fun.
#include <boost/mpl/range_c.hpp>
#include <boost/mpl/vector.hpp>
#include <boost/mpl/for_each.hpp>
#include <boost/mpl/transform.hpp>
#include <boost/mpl/copy.hpp>
// aforementioned struct
template<class C, class I>
struct assign_hash;
// this actually evaluates the functor and captures the this pointer
// T is the argument for the functor U
template<typename T>
struct my_apply {
T* t;
template<typename U>
void operator()(U u) {
u(t);
}
};
template<class C, int T=10, int B=10>
class CountSketch
{
public:
CountSketch()
{
using namespace boost::mpl;
// we need to do this because range_c is not an ExtensibleSequence
typedef typename copy< range_c<int, 0, T>,
back_inserter< vector<> > >::type r;
// fiddle together a vector of the correct types
typedef typename transform<r, typename lambda< assign_hash<C, _1 > >::type >
::type assignees;
// now we need to unfold the type list into a run-time construct
// capture this
my_apply< CountSketch<C, T, B> > apply = { this };
// this is a compile-time loop which actually does something at run-time
for_each<assignees>(apply);
};
// no way around
template<typename TT, typename I>
friend struct assign_hash;
private:
template<int offset>
size_t hash(C& c)
{
return c;
// return (reinterpret_cast<int>(&c)+offset)%B;
}
size_t (CountSketch::*hashfuncs[T])(C &c);
};
// mpl uses int_ so we don't use a non-type template parameter
// but get a compile time value through the value member
template<class C, class I>
struct assign_hash {
template<typename T>
void operator()(T* t) {
t->hashfuncs[I::value] = &CountSketch<C>::template hash<I::value>;
}
};
int main()
{
CountSketch<int> a;
}
with C++20 and consteval compile time loops became possible without doing template hell unless the value can have multiple types:
consteval int func() {
int out = 0;
for(int i = 10; i--;) out += i;
return out;
}
int main() {
std::cout << func(); // outputs 45
}
There are compilers that will see the loop and unroll it. But it's not part of the language specification that it must be done (and, in fact, the language specification throws all sorts of barriers in the way of doing it), and there's no guarantee that it will be done, in a particular case, even on a compiler that "knows how".
There are a few languages that explicitly do this, but they are highly specialized.
(BTW, there's no guarantee that the "unrolled" version of your initializations would be done "at compile time" in a reasonably efficient fashion. But most compilers will, when not compiling to a debug target.)
Here is, I think, a better version of the solution given above.
You can see that we use the compile-time recursive on the function params.
This enables putting all the logic inside your class, and the base case of Init(int_<0>) is very clear - just do nothing :)
Just so you won't fear performance penalty, know that the optimizer will throw away these unused parameters.
As a matter of fact, all these function calls will be inlined anyway. that's the whole point here.
#include <string.h>
#include <stdio.h>
#include <algorithm>
#include <iostream>
using namespace std;
template <class C, int N = 10, int B = 10>
class CountSketch {
public:
CountSketch() {
memset(&_hashFunctions, sizeof(_hashFunctions), 0); // for safety
Init(int_<N>());
}
size_t HashAll(C& c)
{
size_t v = 0;
for(const auto& h : _hashFunctions)
{
v += (this->*h)(c); // call through member pointer
}
return v;
}
private:
template<int offset>
size_t hash(C &c)
{
return (reinterpret_cast<size_t>(&c)+offset)%B;
}
size_t (CountSketch::*_hashFunctions[N])(C &c);
private: // implementation detail
// Notice: better approach.
// use parameters for compile-time recursive call.
// you can just override for the base case, as seen for N-1 below
template <int M>
struct int_ {};
template <int M>
void Init(int_<M>) {
Init(int_<M - 1>());
_hashFunctions[M - 1] = &CountSketch<C, N, B>::template hash<M>;
printf("Initializing %dth element\n", M - 1);
}
void Init(int_<0>) {}
};
int main() {
int c;
CountSketch<int, 10> cs;
int i;
cin >> i;
printf("HashAll: %d", cs.HashAll(c));
return 0;
}
Compiler Explorer