I have the function hash_constexpr that takes in a const char* and returns a hash using a novel algorithm. The hash_constexpr function should be generating the hash at compile time.
namespace detail
{
template<size_t Count>
inline constexpr size_t countof(const char(&string)[Count])
{
return Count - 1;
}
template<typename T>
struct ascii_hash_t
{
template<typename L>
static constexpr T f(L const& data, T hash, size_t i = 0)
{
return i < countof(data) ? f(data, (hash & (~0u)) ^ (hash << 7) ^ T(data[i]), i + 1) : hash;
}
};
template<typename T, typename L>
inline constexpr T generate_ascii_hash(L const& data)
{
return detail::ascii_hash_t<T>::f(data, 0);
}
};
template<size_t Count>
inline constexpr uint32_t hash_constexpr(const char(&string)[Count])
{
return detail::generate_ascii_hash<uint32_t>(string);
}
My issue is that it appears that the hash_constexpr function doesn't appear to actually be returning a constexpr value. When I invoke it like so:
constexpr uint32_t asd = hash_constexpr("asdasd");
I get the following error:
Constexpr variable 'asd' must be initialized by a constant expression
What am I doing wrong?
EDIT #1:
Note that this call is working correctly:
constexpr int32_t countof_test = detail::countof("hello");
EDIT #2:
It appears that this call is working correctly as well:
constexpr int32_t generate_ascii_hash_test = detail::generate_ascii_hash<int32_t>("asd");
The issue is that the function ascii_hash_t::f, for the specific template instantiation is not a constexpr. This prevents you from performing the operation as a constexpr.
See http://ideone.com/heFuFP for an example if it working as you expect.
Related
We have a formatting library similar to fmtlib which allows us to format strings like this:
int foo = 1;
double bar = 2.3;
FMT("foo={} bar={}", foo, bar);
We recently added parsing of the format string at compile time, allowing us to specify, eg width,
precision etc directly in the format string
// print foo with width=5, pad with 0; print bar with width 6, precision 2
// as decimal floating point
FMT("foo={:05} bar={:6.2f}", foo, bar);
The way we do this is as follows:
We have a struct which I can use to pass the format string as a template parameter
struct FmtString
{
template<std::size_t N>
constexpr FmtString(const char (&s)[N])
: str(s)
{}
const char* const str;
};
Together with this FmtString and the arguments to be formatted, we use the following somewhat convoluted dance
to generate a std::tuple of FmtSpecT format specifications.
We use the variadic template parameter pack to generate a std::index_sequence:
// use Ts... to create a tuple of the argument types and an std::index_sequence
template<FmtString S, typename... Ts>
struct CompiledFmtSpec
{
using type = decltype(compileFmtSpec<S, std::tuple<Ts...>>(std::index_sequence_for<Ts...> {}));
};
// pull the indices out of the std::index_sequence
template<FmtString S, typename Tuple, std::size_t... Is>
consteval auto compileFmtSpec(std::index_sequence<Is...>)
{
return typename FmtSpecs<S, Tuple, Is...>::type {};
}
We use the std::index_sequence to generate a variadic std::size_t sequence which we can use to
extract each argument from the tuple of arguments, and its respective sequence value
template<FmtString S, typename Tuple, std::size_t... Is>
struct FmtSpecs
{
using type = std::tuple<FmtSpecWrapper<S, std::tuple_element_t<Is, Tuple>, Is>...>;
};
This results in the following tuple:
std::tuple<
FmtSpecWrapper<S, T0, 0>,
FmtSpecWrapper<S, T1, 1>,
FmtSpecWrapper<S, T2, 2>,
...
FmtSpecWrapper<S, Tn, n>>
We can then specialise FmtSpecWrapper for the particular type and parse its supported format spec
Here is the default implementation
template<FmtString S, typename T, std::size_t I>
struct FmtSpecWrapper
{
using type = decltype(parseFmtString<S, I>());
};
template<FmtString S, std::size_t I>
consteval auto parseFmtString()
{
constexpr const char* pos = findFmtSpec(S, I);
constexpr FmtSpec fmt = parseFmtSpec(pos);
return FmtSpecT<fmt.pad, fmt.align_right, fmt.sign, fmt.width, fmt.precision, fmt.type, fmt.spec_len> {};
}
This results in a FmtSpecT specialisation, which has the following NTTPs:
template<char Pad, bool AlignRight, bool Sign, int Width, int Precision, char Type, int SpecLen>
struct FmtSpecT
{
static constexpr char pad = Pad;
static constexpr bool align_right = AlignRight;
static constexpr bool sign = Sign;
static constexpr int width = Width;
static constexpr int precision = Precision;
static constexpr char type = Type;
static constexpr int spec_len = SpecLen;
};
During formatting we can then pull out each FmtSpecT and use the members to format the argument.
The issue I'd like to try and solve is findFmtSpec(S, I).
Its job is to find the I'th format spec from the format string S.
constexpr const char* findFmtSpec(const FmtString& S, std::size_t i)
{
std::size_t curr = 0;
const char* next = S.str;
while (*next)
{
const char c = *next++;
if (c == '{')
{
if (c == *next) // found an escaped brace; {{ or }}
{
++next;
}
else // found the start of the fmt spec
{
if (curr == i)
return next;
++curr;
++next;
}
}
}
throw std::domain_error("too many arguments for format string");
}
My issue is that it is Big O N^2, as it starts from 0 for every argument.
Ideally I would be able to start from the end of the previous format spec and seek forwards to find the next, turning the complexity into O(N).
Any ideas on how I can modify findFmtSpec so that for each i it starts from the return value of the previouc call?
FmtString S is a template parameter, so it is usable in constant expressions.
Therefore all you need to do is write a constexpr function that extracts the specifiers up-front into a suitable container type, for example std::array<std::string_view, N>. You can either enforce directly in the function that the number of specifiers matches sizeof...(Ts), so that N can just be that, or you can write another constexpr function which first counts the number of arguments:
struct FmtString
{
template<std::size_t N>
constexpr FmtString(const char (&s)[N])
: str(s)
{}
const char* const str;
template<FmtString S>
static constexpr auto findFmtSpec = []{
constexpr auto N = []{
// iterate `S.str` here and return number of specifiers
}();
std::array<const char*, N> result;
// iterate `S.str` again and store beginning of format specifiers consecutively in `result`.
return result;
}();
};
Then parseFmtString can just use FmtString::findFmtSpec<S>[I]. I am using a static member variable template instead of a static function template to make sure that the compiler doesn't keep reevaluating an equivalent function call, although I think the compiler should memoize calls with the same arguments. I used nested lambdas for brevity, but you might want to put these into separate functions. There is also no particular reason to have the variable template be a member. It works outside the class as well.
You can also just generate the whole tuple this way:
template<FmtString S, typename... Ts>
static constexpr auto specs = []{
// number of specifiers, can be `sizeof...(Ts)`
// if no specific error handling is required
constexpr auto N = []{
// iterate `S.str` here and return number of specifiers
}();
if constexpr(sizeof...(Ts) != N) {
// handle argument number mismatch
} else {
// specs as values
constexpr auto specs = []{
std::array<FmtSpec, N> specs;
// iterate `S.str` again and store `FmtSpec` for each specifier consecutively in `specs`
return specs;
}();
// lift specs into type (template arguments)
return []<std::size_t... Is>(std::index_sequence<Is...>){
return std::tuple<FmtSpecWrapper<S, Ts, FmtSpecT<specs[Is].pad, /*...*/>>...>{};
}(std::make_index_sequence<N>{});
}
}();
Here is my code:
template<int... I>
class MetaString1
{
public:
constexpr MetaString1(constexpr char* str)
: buffer_{ encrypt(str[I])... } { }
const char* decrypt()
{
for (int i = 0; i < sizeof...(I); ++i)
buffer_[i] = decrypt1(buffer_[i]);
buffer_[sizeof...(I)] = 0;
return buffer_;
}
private:
constexpr char encrypt(constexpr char c) const { return c ^ 0x55; }
constexpr char decrypt1(constexpr char c) const { return encrypt(c); }
private:
char buffer_[sizeof...(I)+1];
};
#define OBFUSCATED1(str) (MetaString1<0, 1, 2, 3, 4, 5>(str).decrypt())
int main()
{
constexpr char *var = OBFUSCATED1("Post Malone");
std::cout << var << std::endl;
return 1;
}
This is the code from the paper that I'm reading Here. The Idea is simple, to XOR the argument of OBFUSCATED1 and then decrypt back to original value.
The problem that I'm having is that VS 2017 gives me error saying function call must have a constant value in constant expression.
If I only leave OBFUSCATED1("Post Malone");, I have no errors and program is run, but I've noticed that if I have breakpoints in constexpr MetaString1 constructor, the breakpoint is hit, which means that constexpr is not evaluated during compile time. As I understand it's because I don't "force" compiler to evaluate it during compilation by assigning the result to a constexpr variable.
So I have two questions:
Why do I have error function call must have a constant value in constant expression?
Why do people use template classes when they use constexpr functions? As I know template classes get evaluated during compilation, so using template class with constexpr is just a way to push compiler to evaluate those functions during compilation?
You try to assign a non constexpr type to a constexpr type variable,
what's not possible
constexpr char *var = OBFUSCATED1("Post Malone")
// ^^^ ^^^^^^^^^^^
// type of var is constexpr, return type of OBFUSCATED1 is const char*
The constexpr keyword was introduced in C++11, so before you had this keyword you had to write complicated TMP stuff to make the compiler do stuff at compile time. Since TMP is turing complete you theoretically don't need something more than TMP, but since TMP is slow to compile and ugly to ready, you are able to use constexpr to express things you want evaluate at compile time in a more readable way. Although there is no correlation between TMP and constexpr, what means, you are free to use constexpr without template classes.
To achieve what you want, you could save both versions of the string:
template <class T>
constexpr T encrypt(T l, T r)
{
return l ^ r;
}
template <std::size_t S, class U>
struct in;
template <std::size_t S, std::size_t... I>
struct in<S, std::index_sequence<I...>>
{
constexpr in(const char str[S])
: str_{str[I]...}
, enc_{encrypt(str[I], char{0x12})...}
{}
constexpr const char* dec() const
{
return str_;
}
constexpr const char* enc() const
{
return enc_;
}
protected:
char str_[S];
char enc_[S];
};
template <std::size_t S>
class MetaString1
: public in<S, std::make_index_sequence<S - 1>>
{
public:
using base1_t = in<S, std::make_index_sequence<S - 1>>;
using base1_t::base1_t;
constexpr MetaString1(const char str[S])
: base1_t{str}
{}
};
And use it like this:
int main()
{
constexpr char str[] = "asdffasegeasf";
constexpr MetaString1<sizeof(str)> enc{str};
std::cout << enc.dec() << std::endl;
std::cout << enc.enc() << std::endl;
}
Please take a look at the code below, sorry that is a bit lengthy, but I did my best to reproduce the problem with a minimum example (there is also a live copy of it). There I basically have a metafunction which returns the size of string literal, and constexpr function which wraps it. Then when I call those functions in a template parameter gcc (5.4, 6.2) is happy with it, but clang (3.8, 3.9) barfs with "non-type template argument is not a constant expression" in test body on strsize(s). If I replace with a str_size<S> both compilers are happy. So the questions are:
whether that is a problem with clang, or my code?
What is the way to make it compile on both clang and gcc with constexpr function?
template<size_t N> using string_literal_t = char[N];
template<class T> struct StrSize; ///< metafunction to get the size of string literal alikes
/// specialize StrSize for string literals
template<size_t N>
struct StrSize <string_literal_t<N>>{ static constexpr size_t value = N-1; };
/// template variable, just for convenience
template <class T>
constexpr size_t str_size = StrSize<T>::value;
/// now do the same but with constexpr function
template<class T>
constexpr auto strsize(const T&) noexcept-> decltype(str_size<T>) {
return str_size<T>;
}
template<class S, size_t... Is>
constexpr auto test_helper(const S& s, index_sequence<Is...>) noexcept-> array<char, str_size<S>> {
return {s[Is]...};
}
template<class S>
constexpr auto test(const S& s) noexcept-> decltype(auto) {
// return test_helper(s, make_index_sequence<str_size<S>>{}); // this work in both clang and gcc
return test_helper(s, make_index_sequence<strsize(s)>{}); // this works only in gcc
}
auto main(int argc, char *argv[])-> int {
static_assert(strsize("qwe") == 3, "");
static_assert(noexcept(test("qwe")) == true, "");
return 0;
}
Clang is correct here. The problem is in the code and in GCC, which erroneously accepted it. This was fixed in GCC 10: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66477
According to the standard expr.const#5.12:
An expression E is a core constant expression unless the evaluation of E, following the rules of the abstract machine, would evaluate one of the following:
...
an id-expression that refers to a variable or data member of reference type unless the reference has a preceding initialization and either
it is usable in constant expressions or
its lifetime began within the evaluation of E;
And here the compiler is unable to verify the validity of reference in test(const S& s).
Actually there is a nice article to read here: https://brevzin.github.io/c++/2020/02/05/constexpr-array-size/
As to your other question:
What is the way to make it compile on both clang and gcc with constexpr function?
You can replace references with std::array passed by value:
#include <array>
using namespace std;
template<class T> struct StrSize;
template<size_t N>
struct StrSize <array<char,N>>{ static constexpr size_t value = N-1; };
template <class T>
constexpr size_t str_size = StrSize<T>::value;
template<class T>
constexpr auto strsize(const T&) noexcept-> decltype(str_size<T>) {
return str_size<T>;
}
template<class S, size_t... Is>
constexpr auto test_helper(const S& s, index_sequence<Is...>) noexcept-> array<char, str_size<S>> {
return {s[Is]...};
}
constexpr auto test(array<char,4> s) noexcept-> decltype(auto) {
return test_helper(s, make_index_sequence<strsize(s)>{});
}
int main() {
static_assert(noexcept(test({"qwe"})) == true, "");
}
Demo: https://gcc.godbolt.org/z/G8zof38b1
Lets say that you have a function which generates some security token for your application, such as some hash salt, or maybe a symetric or asymetric key.
Now lets say that you have this function in your C++ as a constexpr and that you generate keys for your build based on some information (like, the build number, a timestamp, something else).
You being a diligent programmer make sure and call this in the appropriate ways to ensure it's only called at compile time, and thus the dead stripper removes the code from the final executable.
However, you can't ever be sure that someone else isn't going to call it in an unsafe way, or that maybe the compiler won't strip the function out, and then your security token algorithm will become public knowledge, making it more easy for would be attackers to guess future tokens.
Or, security aside, let's say the function takes a long time to execute and you want to make sure it never happens during runtime and causes a bad user experience for your end users.
Are there any ways to ensure that a constexpr function can never be called at runtime? Or alternately, throwing an assert or similar at runtime would be ok, but not as ideal obviously as a compile error would be.
I've heard that there is some way involving throwing an exception type that doesn't exist, so that if the constexpr function is not deadstripped out, you'll get a linker error, but have heard that this only works on some compilers.
Distantly related question: Force constexpr to be evaluated at compile time
In C++20 you can just replace constexpr by consteval to enforce a function to be always evaluated at compile time.
Example:
int rt_function(int v){ return v; }
constexpr int rt_ct_function(int v){ return v; }
consteval int ct_function(int v){ return v; }
int main(){
constexpr int ct_value = 1; // compile value
int rt_value = 2; // runtime value
int a = rt_function(ct_value);
int b = rt_ct_function(ct_value);
int c = ct_function(ct_value);
int d = rt_function(rt_value);
int e = rt_ct_function(rt_value);
int f = ct_function(rt_value); // ERROR: runtime value
constexpr int g = rt_function(ct_value); // ERROR: runtime function
constexpr int h = rt_ct_function(ct_value);
constexpr int i = ct_function(ct_value);
}
Pre C++20 workaround
You can enforce the use of it in a constant expression:
#include<utility>
template<typename T, T V>
constexpr auto ct() { return V; }
template<typename T>
constexpr auto func() {
return ct<decltype(std::declval<T>().value()), T{}.value()>();
}
template<typename T>
struct S {
constexpr S() {}
constexpr T value() { return T{}; }
};
template<typename T>
struct U {
U() {}
T value() { return T{}; }
};
int main() {
func<S<int>>();
// won't work
//func<U<int>>();
}
By using the result of the function as a template argument, you got an error if it can't be solved at compile-time.
A theoretical solution (as templates should be Turing complete) - don't use constexpr functions and fall back onto the good-old std=c++0x style of computing using exclusively struct template with values. For example, don't do
constexpr uintmax_t fact(uint n) {
return n>1 ? n*fact(n-1) : (n==1 ? 1 : 0);
}
but
template <uint N> struct fact {
uintmax_t value=N*fact<N-1>::value;
}
template <> struct fact<1>
uintmax_t value=1;
}
template <> struct fact<0>
uintmax_t value=0;
}
The struct approach is guaranteed to be evaluated exclusively at compile time.
The fact the guys at boost managed to do a compile time parser is a strong signal that, albeit tedious, this approach should be feasible - it's a one-off cost, maybe one can consider it an investment.
For example:
to power struct:
// ***Warning: note the unusual order of (power, base) for the parameters
// *** due to the default val for the base
template <unsigned long exponent, std::uintmax_t base=10>
struct pow_struct
{
private:
static constexpr uintmax_t at_half_pow=pow_struct<exponent / 2, base>::value;
public:
static constexpr uintmax_t value=
at_half_pow*at_half_pow*(exponent % 2 ? base : 1)
;
};
// not necessary, but will cut the recursion one step
template <std::uintmax_t base>
struct pow_struct<1, base>
{
static constexpr uintmax_t value=base;
};
template <std::uintmax_t base>
struct pow_struct<0,base>
{
static constexpr uintmax_t value=1;
};
The build token
template <uint vmajor, uint vminor, uint build>
struct build_token {
constexpr uintmax_t value=
vmajor*pow_struct<9>::value
+ vminor*pow_struct<6>::value
+ build_number
;
}
In the upcoming C++20 there will be consteval specifier.
consteval - specifies that a function is an immediate function, that is, every call to the function must produce a compile-time constant
Since now we have C++17, there is an easier solution:
template <auto V>
struct constant {
constexpr static decltype(V) value = V;
};
The key is that non-type arguments can be declared as auto. If you are using standards before C++17 you may have to use std::integral_constant. There is also a proposal about the constant helper class.
An example:
template <auto V>
struct constant {
constexpr static decltype(V) value = V;
};
constexpr uint64_t factorial(int n) {
if (n <= 0) {
return 1;
}
return n * factorial(n - 1);
}
int main() {
std::cout << "20! = " << constant<factorial(20)>::value << std::endl;
return 0;
}
Have your function take template parameters instead of arguments and implement your logic in a lambda.
#include <iostream>
template< uint64_t N >
constexpr uint64_t factorial() {
// note that we need to pass the lambda to itself to make the recursive call
auto f = []( uint64_t n, auto& f ) -> uint64_t {
if ( n < 2 ) return 1;
return n * f( n - 1, f );
};
return f( N, f );
}
using namespace std;
int main() {
cout << factorial<5>() << std::endl;
}
Lets say that you have a function which generates some security token for your application, such as some hash salt, or maybe a symetric or asymetric key.
Now lets say that you have this function in your C++ as a constexpr and that you generate keys for your build based on some information (like, the build number, a timestamp, something else).
You being a diligent programmer make sure and call this in the appropriate ways to ensure it's only called at compile time, and thus the dead stripper removes the code from the final executable.
However, you can't ever be sure that someone else isn't going to call it in an unsafe way, or that maybe the compiler won't strip the function out, and then your security token algorithm will become public knowledge, making it more easy for would be attackers to guess future tokens.
Or, security aside, let's say the function takes a long time to execute and you want to make sure it never happens during runtime and causes a bad user experience for your end users.
Are there any ways to ensure that a constexpr function can never be called at runtime? Or alternately, throwing an assert or similar at runtime would be ok, but not as ideal obviously as a compile error would be.
I've heard that there is some way involving throwing an exception type that doesn't exist, so that if the constexpr function is not deadstripped out, you'll get a linker error, but have heard that this only works on some compilers.
Distantly related question: Force constexpr to be evaluated at compile time
In C++20 you can just replace constexpr by consteval to enforce a function to be always evaluated at compile time.
Example:
int rt_function(int v){ return v; }
constexpr int rt_ct_function(int v){ return v; }
consteval int ct_function(int v){ return v; }
int main(){
constexpr int ct_value = 1; // compile value
int rt_value = 2; // runtime value
int a = rt_function(ct_value);
int b = rt_ct_function(ct_value);
int c = ct_function(ct_value);
int d = rt_function(rt_value);
int e = rt_ct_function(rt_value);
int f = ct_function(rt_value); // ERROR: runtime value
constexpr int g = rt_function(ct_value); // ERROR: runtime function
constexpr int h = rt_ct_function(ct_value);
constexpr int i = ct_function(ct_value);
}
Pre C++20 workaround
You can enforce the use of it in a constant expression:
#include<utility>
template<typename T, T V>
constexpr auto ct() { return V; }
template<typename T>
constexpr auto func() {
return ct<decltype(std::declval<T>().value()), T{}.value()>();
}
template<typename T>
struct S {
constexpr S() {}
constexpr T value() { return T{}; }
};
template<typename T>
struct U {
U() {}
T value() { return T{}; }
};
int main() {
func<S<int>>();
// won't work
//func<U<int>>();
}
By using the result of the function as a template argument, you got an error if it can't be solved at compile-time.
A theoretical solution (as templates should be Turing complete) - don't use constexpr functions and fall back onto the good-old std=c++0x style of computing using exclusively struct template with values. For example, don't do
constexpr uintmax_t fact(uint n) {
return n>1 ? n*fact(n-1) : (n==1 ? 1 : 0);
}
but
template <uint N> struct fact {
uintmax_t value=N*fact<N-1>::value;
}
template <> struct fact<1>
uintmax_t value=1;
}
template <> struct fact<0>
uintmax_t value=0;
}
The struct approach is guaranteed to be evaluated exclusively at compile time.
The fact the guys at boost managed to do a compile time parser is a strong signal that, albeit tedious, this approach should be feasible - it's a one-off cost, maybe one can consider it an investment.
For example:
to power struct:
// ***Warning: note the unusual order of (power, base) for the parameters
// *** due to the default val for the base
template <unsigned long exponent, std::uintmax_t base=10>
struct pow_struct
{
private:
static constexpr uintmax_t at_half_pow=pow_struct<exponent / 2, base>::value;
public:
static constexpr uintmax_t value=
at_half_pow*at_half_pow*(exponent % 2 ? base : 1)
;
};
// not necessary, but will cut the recursion one step
template <std::uintmax_t base>
struct pow_struct<1, base>
{
static constexpr uintmax_t value=base;
};
template <std::uintmax_t base>
struct pow_struct<0,base>
{
static constexpr uintmax_t value=1;
};
The build token
template <uint vmajor, uint vminor, uint build>
struct build_token {
constexpr uintmax_t value=
vmajor*pow_struct<9>::value
+ vminor*pow_struct<6>::value
+ build_number
;
}
In the upcoming C++20 there will be consteval specifier.
consteval - specifies that a function is an immediate function, that is, every call to the function must produce a compile-time constant
Since now we have C++17, there is an easier solution:
template <auto V>
struct constant {
constexpr static decltype(V) value = V;
};
The key is that non-type arguments can be declared as auto. If you are using standards before C++17 you may have to use std::integral_constant. There is also a proposal about the constant helper class.
An example:
template <auto V>
struct constant {
constexpr static decltype(V) value = V;
};
constexpr uint64_t factorial(int n) {
if (n <= 0) {
return 1;
}
return n * factorial(n - 1);
}
int main() {
std::cout << "20! = " << constant<factorial(20)>::value << std::endl;
return 0;
}
Have your function take template parameters instead of arguments and implement your logic in a lambda.
#include <iostream>
template< uint64_t N >
constexpr uint64_t factorial() {
// note that we need to pass the lambda to itself to make the recursive call
auto f = []( uint64_t n, auto& f ) -> uint64_t {
if ( n < 2 ) return 1;
return n * f( n - 1, f );
};
return f( N, f );
}
using namespace std;
int main() {
cout << factorial<5>() << std::endl;
}