String-literals without escaping null-character - c++

When I built a function which gives the hexadecimal representation of a nibble (4 bits) and I looked at the binary file, for the lookuptable of the digits, there was an additional 0-char even if it was not used.
const char digits[] = "0123456789abcdef";
I know that you can write it in form of an array:
const char digits[] = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'};
But that would take a while to write and use more disk-space for other numerical-systems with more digits.
But is there any way to write it as a literal, but without the null-character at the end?
(I am using Clang with -std=c++14)

In C, you could write
const char digits[16] = "0123456789abcdef";
This is actually supported in C as defined in array initialization:
If the size of the array is known, it may be one less than the size of
the string literal, in which case the terminating null character is
ignored: char str[3] = "abc"; // str has type char[3] and holds 'a',
'b', 'c'
For C++, I see no direct way; but one could come around this as follows:
char digits_t[16] = "0123456789abcde";
digits_t[15]='f';
const char* digits = digits_t;

I have no idea why you'd want to do such a thing, but if you can cope with using a compiler extension, Clang and GCC will let you write a templated user-defined literal operator which will chop off the trailing null:
template <typename CharT, std::size_t N>
struct string_literal {
static constexpr std::size_t size = N;
const CharT str[N];
};
template <typename CharT, CharT... Str>
constexpr string_literal<CharT, sizeof...(Str)> operator"" _lit()
{
return {{ Str... }};
}
int main()
{
constexpr auto s = "test"_lit;
static_assert(s.size == 4);
static_assert(s.str[0] == 't'); // etc
}
(Returning a std::array<const char, N> is another option.)
I've no idea whether this is what you're after, but then I don't really understand the motivation to be honest -- even back in the 70s the designers of C weren't too worried about "wasting" a single byte in string literal.

This might be overkill, but if it’s acceptable to use std::array, then you can statically build one from a string literal using constexpr functions:
#include <array>
#include <iostream>
// A type-level list of indices.
template <size_t... Xs>
struct indices { using type = indices<Xs..., sizeof...(Xs)>; };
// Generate indices from 0 to N.
template <std::size_t N>
struct upto { using type = typename upto<N - 1>::type::type; };
template <>
struct upto<0> { using type = indices<>; };
// Make an array by assigning each character
// from the corresponding index in the source.
template <std::size_t... X, typename A = std::array<char, sizeof...(X)>>
constexpr A make_array(char const *const source, const indices<X...>&) {
return A{{source[X]...}};
}
// A convenience function that deduces the size.
template <std::size_t N>
constexpr std::array<char, N - 1> string_constant(char const (&data)[N]) {
return make_array(&data[0], typename upto<N - 1>::type{});
}
int main() {
constexpr auto s = string_constant("123456");
std::cout << sizeof(s) << '\n';
}
Now sizeof(s) is 6, and if you look at the generated assembly, the string literal is stored as .ascii, not .asciz, so there is no trailing null character. You can use the member functions of std::array such as size(), begin(), and end(), and can cast &s[0] to const char * to access character data directly.

Related

If constexpr ternary operator

I have a simple snipped below:
template<typename>
struct function_traits;
template<typename Rv, typename... Args>
struct function_traits<Rv(Args...)>
{
static constexpr const char _signature[] = {GetTypeChar<Rv>, '(', GetTypeChar<Args>()..., ')', '\0'};
}
where GetTypeChar returns character representing a type (i for integer, d for double et cetra).
This works really well for simple types, eg function_traits<int(float,double)>::_signature returns i(f,d) as expected.
Now Imagine I would have the ability to see whether it is an array or not by adding '[]' after the array, eg double[] would result in d[].
I tried changing the signature to
static constexpr const char _signature[] = {GetTypeChar<Rv>(), '(', (std::is_array_v<Args> ? GetTypeChar<Args>(), '[', ']' : GetTypeChar<Args>())..., ')', '\0'};
but that seems to be completely ignored, and only things after : are evaluated.
Is something like this possible?
Well, Initially I though I will be able to do this in a single array initialization, but the constexpr getter works well.
This is what I ended up with:
template<typename Tp>
constexpr void SetParamType(char*& dst)
{
if constexpr (std::is_pointer_v<Tp>)
{
*dst++ = '[';
}
*dst++ = GetTypeChar<Tp>();
}
[[nodiscard]]
constexpr auto signature() const
{
std::array<char, sizeof...(Args) * 2 + 4> out = {0};
char * ptr = out.data();
*ptr++ = GetTypeChar<return_type>();
*ptr++ = '(';
(SetParamType<Args>(ptr),...);
*ptr++ = ')';
*ptr++ = '\0';
return out;
}
Please note that it would be wise to calculate the size of an array to exactly match the required length (which would not be hard after all, but for sake of simplicity I'll leave it as it is for now)

How to define string literal with character type that depends on template parameter?

template<typename CharType>
class StringTraits {
public:
static const CharType NULL_CHAR = '\0';
static constexpr CharType* WHITESPACE_STR = " ";
};
typedef StringTraits<char> AStringTraits;
typedef StringTraits<wchar_t> WStringTraits;
I know I could do it with template specialization, but this would require some duplication (by defining string literals with and without L prefix).
Is there a simpler way to define const/constexpr char/wchar_t and char*/wchar_t* with same string literal in a template class?
There are several ways to do this, depending on the available version of the C++ standard.
If you have C++17 available, you can scroll down to Method 3, which is the most elegant solution in my opinion.
Note: Methods 1 and 3 assume that the characters of the string literal will be restricted to 7-bit ASCII. This requires that characters are in the range [0..127] and the execution character set is compatible with 7-bit ASCII (e. g. Windows-1252 or UTF-8). Otherwise the simple casting of char values to wchar_t used by these methods won't give the correct result.
Method 1 - aggregate initialization (C++03)
The simplest way is to define an array using aggregate initialization:
template<typename CharType>
class StringTraits {
public:
static const CharType NULL_CHAR = '\0';
static constexpr CharType WHITESPACE_STR[] = {'a','b','c',0};
};
Method 2 - template specialization and macro (C++03)
(Another variant is shown in this answer.)
The aggregate initialization method can be cumbersome for long strings. For more comfort, we can use a combination of template specialization and macros:
template< typename CharT > constexpr CharT const* NarrowOrWide( char const*, wchar_t const* );
template<> constexpr char const* NarrowOrWide< char >( char const* c, wchar_t const* )
{ return c; }
template<> constexpr wchar_t const* NarrowOrWide< wchar_t >( char const*, wchar_t const* w )
{ return w; }
#define TOWSTRING1(x) L##x
#define TOWSTRING(x) TOWSTRING1(x)
#define NARROW_OR_WIDE( C, STR ) NarrowOrWide< C >( ( STR ), TOWSTRING( STR ) )
Usage:
template<typename CharType>
class StringTraits {
public:
static constexpr CharType const* WHITESPACE_STR = NARROW_OR_WIDE( CharType, " " );
};
Live Demo at Coliru
Explanation:
The template function NarrowOrWide() returns either the first (char const*) or the second (wchar_t const*) argument, depending on template parameter CharT.
The macro NARROW_OR_WIDE is used to avoid having to write both the narrow and the wide string literal. The macro TOWSTRING simply prepends the L prefix to the given string literal.
Of course the macro will only work if the range of characters is limited to basic ASCII, but this is usually sufficient. Otherwise one can use the NarrowOrWide() template function to define narrow and wide string literals separately.
Notes:
I would add a "unique" prefix to the macro names, something like the name of your library, to avoid conflicts with similar macros defined elsewhere.
Method 3 - array initialized via template parameter pack (C++17)
C++17 finally allows us to get rid of the macro and use a pure C++ solution. The solution uses template parameter pack expansion to initialize an array from a string literal while static_casting the individual characters to the desired type.
First we declare a str_array class, which is similar to std::array but tailored for constant null-terminated string (e. g. str_array::size() returns number of characters without '\0', instead of buffer size). This wrapper class is necessary, because a plain array cannot be returned from a function. It must be wrapped in a struct or class.
template< typename CharT, std::size_t Length >
struct str_array
{
constexpr CharT const* c_str() const { return data_; }
constexpr CharT const* data() const { return data_; }
constexpr CharT operator[]( std::size_t i ) const { return data_[ i ]; }
constexpr CharT const* begin() const { return data_; }
constexpr CharT const* end() const { return data_ + Length; }
constexpr std::size_t size() const { return Length; }
// TODO: add more members of std::basic_string
CharT data_[ Length + 1 ]; // +1 for null-terminator
};
So far, nothing special. The real trickery is done by the following str_array_cast() function, which initializes the str_array from a string literal while static_casting the individual characters to the desired type:
#include <utility>
namespace detail {
template< typename ResT, typename SrcT >
constexpr ResT static_cast_ascii( SrcT x )
{
if( !( x >= 0 && x <= 127 ) )
throw std::out_of_range( "Character value must be in basic ASCII range (0..127)" );
return static_cast<ResT>( x );
}
template< typename ResElemT, typename SrcElemT, std::size_t N, std::size_t... I >
constexpr str_array< ResElemT, N - 1 > do_str_array_cast( const SrcElemT(&a)[N], std::index_sequence<I...> )
{
return { static_cast_ascii<ResElemT>( a[I] )..., 0 };
}
} //namespace detail
template< typename ResElemT, typename SrcElemT, std::size_t N, typename Indices = std::make_index_sequence< N - 1 > >
constexpr str_array< ResElemT, N - 1 > str_array_cast( const SrcElemT(&a)[N] )
{
return detail::do_str_array_cast< ResElemT >( a, Indices{} );
}
The template parameter pack expansion trickery is required, because constant arrays can only be initialized via aggregate initialization (e. g. const str_array<char,3> = {'a','b','c',0};), so we have to "convert" the string literal to such an initializer list.
The code triggers a compile time error if any character is outside of basic ASCII range (0..127), for the reasons given at the beginning of this answer. There are code pages where 0..127 doesn't map to ASCII, so this check does not give 100% safety though.
Usage:
template< typename CharT >
struct StringTraits
{
static constexpr auto WHITESPACE_STR = str_array_cast<CharT>( "abc" );
// Fails to compile (as intended), because characters are not basic ASCII.
//static constexpr auto WHITESPACE_STR1 = str_array_cast<CharT>( "Àâü" );
};
Live Demo at Coliru
Here is a refinement of the now-common template-based solution which
preserves the array[len] C++ type of the C strings rather than decaying them to pointers, which means you can call sizeof() on the result and get the size of the string+NUL, not the size of a pointer, just as if you had the original string there.
Works even if the strings in different encodings have different length in code units (which is virtually guaranteed if the strings have non-ASCII text).
Does not incur any runtime overhead nor does it attempt/need to do encoding conversion at runtime.
Credit: This refinement starts with the original template idea from Mark Ransom and #2 from zett42 and borrows some ideas from, but fixes the size limitations of, Chris Kushnir's answer.
This code does char and wchar_t but it is trivial to extend it to char8_t+char16_t+char32_t
// generic utility for C++ pre-processor concatenation
// - avoids a pre-processor issue if x and y have macros inside
#define _CPP_CONCAT(x, y) x ## y
#define CPP_CONCAT(x, y) _CPP_CONCAT(x, y)
// now onto stringlit()
template<size_t SZ0, size_t SZ1>
constexpr
auto _stringlit(char c,
const char (&s0) [SZ0],
const wchar_t (&s1) [SZ1]) -> const char(&)[SZ0]
{
return s0;
}
template<size_t SZ0, size_t SZ1>
constexpr
auto _stringlit(wchar_t c,
const char (&s0) [SZ0],
const wchar_t (&s1) [SZ1]) -> const wchar_t(&)[SZ1]
{
return s1;
}
#define stringlit(code_unit, lit) \
_stringlit(code_unit (), lit, CPP_CONCAT(L, lit))
Here we are not using C++ overloading but rather defining one function per char encoding, each function with different signatures. Each function returns the original array type with the original bounds. The selector that chooses the appropriate function is a single character in the desired encoding (value of that character not important). We cannot use the type itself in a template parameter to select because then we'd be overloading and have conflicting return types. This code also works without the constexpr. Note we are returning a reference to an array (which is possible in C++) not an array (which is not allowed in C++). The use of trailing return type syntax here is optional, but a heck of a lot more readable than the alternative, something like const char (&stringlit(...params here...))[SZ0] ugh.
I compiled this with clang 9.0.8 and MSVC++ from Visual Studio 2019 16.7 (aka _MSC_VER 1927 aka pdb ver 14.27). I had c++2a/c++latest enabled, but I think C++14 or 17 is sufficient for this code.
Enjoy!
Here's an alternative implementation based on #zett42 's answer. Please advise me.
#include <iostream>
#include <tuple>
#define TOWSTRING_(x) L##x
#define TOWSTRING(x) TOWSTRING_(x)
#define MAKE_LPCTSTR(C, STR) (std::get<const C*>(std::tuple<const char*, const wchar_t*>(STR, TOWSTRING(STR))))
template<typename CharType>
class StringTraits {
public:
static constexpr const CharType* WHITESPACE_STR = MAKE_LPCTSTR(CharType, "abc");
};
typedef StringTraits<char> AStringTraits;
typedef StringTraits<wchar_t> WStringTraits;
int main(int argc, char** argv) {
std::cout << "Narrow string literal: " << AStringTraits::WHITESPACE_STR << std::endl;
std::wcout << "Wide string literal : " << WStringTraits::WHITESPACE_STR << std::endl;
return 0;
}
I've just came up with a compact answer, which is similar to other C++17 versions. Similarly, it relies on implementation defined behavior, specifically on the environment character types. It supports converting ASCII and ISO-8859-1 to UTF-16 wchar_t, UTF-32 wchar_t, UTF-16 char16_t and UTF-32 char32_t. UTF-8 input is not supported, but more elaborate conversion code is feasible.
template <typename Ch, size_t S>
constexpr auto any_string(const char (&literal)[S]) -> const array<Ch, S> {
array<Ch, S> r = {};
for (size_t i = 0; i < S; i++)
r[i] = literal[i];
return r;
}
Full example follows:
$ cat any_string.cpp
#include <array>
#include <fstream>
using namespace std;
template <typename Ch, size_t S>
constexpr auto any_string(const char (&literal)[S]) -> const array<Ch, S> {
array<Ch, S> r = {};
for (size_t i = 0; i < S; i++)
r[i] = literal[i];
return r;
}
int main(void)
{
auto s = any_string<char>("Hello");
auto ws = any_string<wchar_t>(", ");
auto s16 = any_string<char16_t>("World");
auto s32 = any_string<char32_t>("!\n");
ofstream f("s.txt");
f << s.data();
f.close();
wofstream wf("ws.txt");
wf << ws.data();
wf.close();
basic_ofstream<char16_t> f16("s16.txt");
f16 << s16.data();
f16.close();
basic_ofstream<char32_t> f32("s32.txt");
f32 << s32.data();
f32.close();
return 0;
}
$ c++ -o any_string any_string.cpp -std=c++17
$ ./any_string
$ cat s.txt ws.txt s16.txt s32.txt
Hello, World!
A variation of zett42 Method 2 above.
Has the advantage of supporting all char types (for literals that can be represented as char[]) and preserving the proper string literal array type.
First the template functions:
template<typename CHAR_T>
constexpr
auto LiteralChar(
char A,
wchar_t W,
char8_t U8,
char16_t U16,
char32_t U32
) -> CHAR_T
{
if constexpr( std::is_same_v<CHAR_T, char> ) return A;
else if constexpr( std::is_same_v<CHAR_T, wchar_t> ) return W;
else if constexpr( std::is_same_v<CHAR_T, char8_t> ) return U8;
else if constexpr( std::is_same_v<CHAR_T, char16_t> ) return U16;
else if constexpr( std::is_same_v<CHAR_T, char32_t> ) return U32;
}
template<typename CHAR_T, size_t SIZE>
constexpr
auto LiteralStr(
const char (&A) [SIZE],
const wchar_t (&W) [SIZE],
const char8_t (&U8) [SIZE],
const char16_t (&U16)[SIZE],
const char32_t (&U32)[SIZE]
) -> const CHAR_T(&)[SIZE]
{
if constexpr( std::is_same_v<CHAR_T, char> ) return A;
else if constexpr( std::is_same_v<CHAR_T, wchar_t> ) return W;
else if constexpr( std::is_same_v<CHAR_T, char8_t> ) return U8;
else if constexpr( std::is_same_v<CHAR_T, char16_t> ) return U16;
else if constexpr( std::is_same_v<CHAR_T, char32_t> ) return U32;
}
Then the macros:
#define CMK_LC(CHAR_T, LITERAL) \
LiteralChar<CHAR_T>( LITERAL, L ## LITERAL, u8 ## LITERAL, u ## LITERAL, U ## LITERAL )
#define CMK_LS(CHAR_T, LITERAL) \
LiteralStr<CHAR_T>( LITERAL, L ## LITERAL, u8 ## LITERAL, u ## LITERAL, U ## LITERAL )
Then use:
template<typename CHAR_T>
class StringTraits {
public:
struct LC { // literal character
static constexpr CHAR_T Null = CMK_LC(CHAR_T, '\0');
static constexpr CHAR_T Space = CMK_LC(CHAR_T, ' ');
};
struct LS { // literal string
// can't seem to avoid having to specify the size
static constexpr CHAR_T Space [2] = CMK_LS(CHAR_T, " ");
static constexpr CHAR_T Ellipsis [4] = CMK_LS(CHAR_T, "...");
};
};
auto char_space { StringTraits<char>::LC::Space };
auto wchar_space { StringTraits<wchar_t>::LC::Space };
auto char_ellipsis { StringTraits<char>::LS::Ellipsis }; // note: const char*
auto wchar_ellipsis { StringTraits<wchar_t>::LS::Ellipsis }; // note: const wchar_t*
auto (& char_space_array) [4] { StringTraits<char>::LS::Ellipsis };
auto (&wchar_space_array) [4] { StringTraits<wchar_t>::LS::Ellipsis };
? syntax to get a local copy ?
Admittedly, the syntax to preserve the string literal array type is a bit of a burden, but not overly so.
Again, only works for literals that have the same # of code units in all char type representations.
If you want LiteralStr to support all literals for all types would likely need to pass pointers as param and return CHAR_T* instead of CHAR_T(&)[SIZE]. Don't think can get LiteralChar to support multibyte char.
[EDIT]
Applying Louis Semprini SIZE support to LiteralStr gives:
template<typename CHAR_T,
size_t SIZE_A, size_t SIZE_W, size_t SIZE_U8, size_t SIZE_U16, size_t SIZE_U32,
size_t SIZE_R =
std::is_same_v<CHAR_T, char> ? SIZE_A :
std::is_same_v<CHAR_T, wchar_t> ? SIZE_W :
std::is_same_v<CHAR_T, char8_t> ? SIZE_U8 :
std::is_same_v<CHAR_T, char16_t> ? SIZE_U16 :
std::is_same_v<CHAR_T, char32_t> ? SIZE_U32 : 0
>
constexpr
auto LiteralStr(
const char (&A) [SIZE_A],
const wchar_t (&W) [SIZE_W],
const char8_t (&U8) [SIZE_U8],
const char16_t (&U16) [SIZE_U16],
const char32_t (&U32) [SIZE_U32]
) -> const CHAR_T(&)[SIZE_R]
{
if constexpr( std::is_same_v<CHAR_T, char> ) return A;
else if constexpr( std::is_same_v<CHAR_T, wchar_t> ) return W;
else if constexpr( std::is_same_v<CHAR_T, char8_t> ) return U8;
else if constexpr( std::is_same_v<CHAR_T, char16_t> ) return U16;
else if constexpr( std::is_same_v<CHAR_T, char32_t> ) return U32;
}
It is also possible to use a simpler syntax to create variables;
for example, in StringTraits::LS can change to constexpr auto &
so
static constexpr CHAR_T Ellipsis [4] = CMK_LS(CHAR_T, "...");
becomes
static constexpr auto & Ellipsis { CMK_LS(CHAR_T, "...") };
When using CMK_LS(char, "literal") any invalid char in literal are converted to '?' by VS 2019, not sure what other compilers do.

Template code and literal strings

I am currently writing some template code where the template parameter is the char type to use. This causes a problem when referring to literal strings. I can, of course, make a struct with the strings I use but I was thinking if it would be possible to make something like:
template<typename chartype, char32_t... chars>
struct tr {
/* some magic here */
};
so that tr<char32_t,U"hello world"> would result in U"hello world"
and tr<char16_t,U"Hello world"> would result in u"hello world"
and tr<char,U"hello world"> would result in "hello world" (in UTF-8).
The magic should of course correctly translate codes above 0x10000 to lead code and follow code for char16_t and to proper 2, 3 and 4 byte codes for UTF-8
at compile time.
Problem is: how do you define a constant C-style string of a given char type using the char32_t... chars template argument? You can extract the characters
from it but how do you rebuild a new string based on the chars of the input string in template code?
Note, the preprocessor can correctly define a string such as "hello world" with suitable prefix u or U if you like but it cannot access the individual characters of the string to properly translate it.
EDIT:
Strings as template arguments are definitely possible in new C++, however,
the template argument is NOT declared as const char * or something like that:
template <char... txt>
struct foo { ... }
allows you to write foo<"hello"> as a type with the string "hello" as template argument. The problem is how to build the string from those characters.
I mean at some point you want the struct to contain a string value to return:
template <char32_t... txt>
struct foo;
template <>
struct foo<> {
static const char16_t * txt() { return ""; }
};
template <char32_t a, char32_t... rest>
struct foo<a, rest...> {
static const char16_t * txt()
{
char16_t buf[100];
int k = 0;
if (a < 0x10000) buf[k++] = a;
else {
buf[k++] = 0xd800 | ((a - 0x10000) >> 10);
buf[k++] = 0xdc00 | ((a-0x10000) & 0x3ff);
}
// copy rest of string into buf + 2..99
u16strcpy(buf + k, foo<rest...>::txt());
return buf;
}
}
Several obvious problems with this "solution", one problem is that buf only have room for 100 characters which will fail if the string is longer. but the main problem is that I wanted this to happen in compile time and this looks very much like run time code to me and is not at all what I wanted to do.
Basically I wanted something that works this way:
foo<char, "hello"> results in something that is effectively a string literal
"hello" or u8"hello".
foo<char16_t, "hello"> results in something that is effectively a string literal u"hello" and foo<char32_t, "hello"> results in something that is effectively a string literal U"hello".
The problem is when writing template code to handle various character formats and then having string literals involved. Yes, you can write a simple struct:
template <typename ChT> struct bar;
template <>
struct bar<char32_t> {
static const char32_t * txta = U"AAAA";
static const char32_t * txtb = U"BBBB";
};
and so on and bar<char16_t> has txta = u"AAAA" etc. Then refer to the strings
in your templated code by bar<T>::txta etc. However, I wish there was a way that you could specify those strings directly in templated code and the compiler would do the right thing. A templated string literal in other words.
Maybe it should be added as a feature to the language that
T<char32_t> string-literal is the same as U string-literal etc
so that you could write
template <typename ChT>
struct foo {
static const ChT * txta = T<ChT> "AAAAA";
};
and the compiler would do the right thing.
This would appear to simply not be legal, even the following is rejected (vs2017, with standard set to latest):
template<char const * ptr>
struct test
{};
void bar()
{
test<"testing"> t;
}
with the error: invalid expression as a template argument for 'ptr', and if that's not going to work trying to convert it at compile-time is a non-starter. And honestly this doesn't seem all that surprising that a pointer-to-data isn't constant enough. to be a template argument.
Here are some tools to make it work in C++17 (might be portable to C++11 and C++14):
static constexpr data member of templated class
The output literal you wish to work with needs some "storage". I suggest to instantiate a unique class template for each literal, e.g., Literal<char, 'f', 'o', 'o', '\0'>. That class can hold the data as astatic constexpr` member.
template<class C, C... cs>
struct Literal {
static_assert(sizeof...(cs) >= 1);
static constexpr C data[] = {cs...};// or use `std::array<C, sizeof...(cs)>`
};
template<class C, C... cs>
constexpr C Literal<C, cs...>::data[];
user-defined string literal to simplify syntax
Of course you wish to avoid typing, e.g., Literal<char, 'f', 'o', 'o', '\0'>. A useful tool to achieve that is the following overload for user-defined string literals.
template<class C, C... cs>
constexpr Literal<C, cs..., C('\0')> operator""_c() {// or use `auto`
return Literal<C, cs..., C('\0')>{};
}
Note how the input literal is passed as non-type template parameters to that overload. That way, it is possible to "carry the value as a type".
constexpr algorithms for re-encoding
So far, you can type "foo"_c to obtain a Literal<char, 'f', 'o', 'o', '\0'> which has a static constexpr data member yielding the same as "foo". Next you can pass that Literal<char, 'f', 'o', 'o', '\0'> to a function which returns a const char16_t(&)[4] to data of the corresponding Literal<char16_t, ..., '\0'>. The syntax could read tr<char16_t>("foo"_c).
The code that transforms a Literal<char, ...> into the corresponding Literal<char16_t, ...> may be based on constexpr algorithms as shown below.
template<
class OutChar, class InChar, InChar... cs,
std::size_t... input_indices, std::size_t... output_indices
>
constexpr auto& tr_impl(// called by `tr` with appropriate `index_sequence`s
Literal<InChar, cs...>,
std::index_sequence<input_indices...>,
std::index_sequence<output_indices...>
) {
constexpr std::size_t outsize = sizeof...(output_indices);
using Buffer = std::array<OutChar, outsize>;
constexpr Buffer buf = encode_as<OutChar, outsize>({cs...});
return Literal<OutChar, buf[output_indices]...>::data;
}
template<class OutChar, class InChar, InChar... cs>
constexpr auto& tr(Literal<InChar, cs...> literal) {
constexpr std::size_t outsize = count_as<OutChar>({cs...});
return tr_impl<OutChar>(
literal,
std::make_index_sequence<sizeof...(cs)>{},// input indices
std::make_index_sequence<outsize>{}// output indices
);
}
The remaining part would be to implement those functions count_as and encode_as.
assign to constexpr auto& in final usage
Finally you can assign to constexpr auto& to verify that type and value are equivalent to the plain string literal based on the desired character type.
static_assert(std::size(U"π„žπ„ž") == 3);
static_assert(std::size(u"π„žπ„ž") == 5);
constexpr auto& test = tr<char16_t>(U"π„žπ„ž"_c);
static_assert(std::is_same<decltype(test), const char16_t(&)[5]>{});
for(std::size_t i=0; i<std::size(u"π„žπ„ž"); ++i) {
assert(test[i] == u"π„žπ„ž"[i]);
std::cout << i << ": " << test[i] << std::endl;
}

trying to distinguish between different kinds of rvalues - literals and non-literals

I want to have a method that can be called only with lvalues so I've done the following:
template <typename T>
MyClass& cache(T&) {}
template <typename T>
MyClass& cache(const T&&) = delete;
And this works fine - I can even pass C string literals since they are also lvalues.
The reason I need lvalues is because I'm caching pointers to the passed objects - and that means they cannot be temporaries.
The following code works:
MyClass a;
a.cache("string literal");
int temp = 6;
a.cache(temp);
And the following code (as desired) does not work:
int getInt(); // fwd decl
a.cache(getInt());
But I also want to be able to pass other literals - but they all seem to be rvalues...
The following code does not work (but I wish it could):
MyClass a;
a.cache(6);
Is there a way to distinguish between such literals and non-literal rvalues?
Is there a way to easily turn such literals into lvalues? I found this answer on Stack Overflow mentioning something like unless you write 1L but even if I give an L suffix to a literal it is still a temporary.
Even a macro is OK - something like this: a.cache(TURN_TO_LVALUE(6)); (or CONSTANTIZE(6))
You may be confused by "hello"; it is a literal, but it (in a sense) is also an lvalue (if a const one). "hello" creates an object that doesn't go away when the line ends, an array of const characters consisting of {'h', 'e', 'l', 'l', 'o', '\0'}. Two different "hello" could refer to the same object or not.
6 doesn't do the same thing; there is no persistant 6 in a C++ program with the constant 6 in it, there is a persistant "hello" in a C++ program with the string constant "hello" in it.
The literal 6 can cause a temporary to be instantiated. The lifetime of this temporary is until the end of the expression it is in (the "end of the line" as it where).
You cannot distinguish between the temporary created by 6 and the temporary returned from a function during a function call. This is fortunate, because both are temporaries with the same advantages and disadvantages.
A pointer to that temporary is going to be invalid at that point; even == or < on that pointer is undefined behavior.
So a.cache(6) is a bad plan.
Now, we could do something horrible.
unsigned long long const& operator""_lvalue(unsigned long long x) {
thread_local unsigned long long value;
value = x;
return value;
}
live example.
This creates a static storage duration lvalue value and copies x into it. So 6_lvalue + 4_lvalue is going to be either 8 or 12, never 10, as the single lvalue is overwritten.
We can remove that overwriting problem with more template abuse.
template<int P>
constexpr unsigned long long pow_( unsigned x, std::size_t tens ) {
if (tens == 0) return x;
return P*pow_<P>(x, tens-1);
}
template<int base>
constexpr unsigned long long ucalc(std::integer_sequence<char>) {
return 0;
}
constexpr unsigned digit( char c ) {
if (c >= '0' && c <= '9') return c-'0';
if (c >= 'a' && c <= 'z') return c-'a'+10;
if (c >= 'A' && c <= 'Z') return c-'A'+10;
exit(-1);
}
template<int base, char c0, char...chars>
constexpr unsigned long long ucalc(std::integer_sequence<char, c0, chars...>) {
return pow_<base>( digit(c0), sizeof...(chars) ) + ucalc<base>( std::integer_sequence<char, chars...>{} );
}
template <char... chars>
constexpr unsigned long long calc(std::integer_sequence<char, chars...>) {
return ucalc<10>(std::integer_sequence<char, chars...>{});
}
template <char... chars>
constexpr unsigned long long calc(std::integer_sequence<char, '0', 'x', chars...>) {
return ucalc<16>(std::integer_sequence<char, chars...>{});
}
template <char... chars>
constexpr unsigned long long calc(std::integer_sequence<char, '0', 'X', chars...>) {
return ucalc<16>(std::integer_sequence<char, chars...>{});
}
template <char... chars>
constexpr unsigned long long calc(std::integer_sequence<char, '0', 'b', chars...>) {
return ucalc<2>(std::integer_sequence<char, chars...>{});
}
template <char... chars>
constexpr unsigned long long calc(std::integer_sequence<char, '0', 'B', chars...>) {
return ucalc<2>(std::integer_sequence<char, chars...>{});
}
template <char... chars>
constexpr unsigned long long calc(std::integer_sequence<char, '0', chars...>) {
return ucalc<8>(std::integer_sequence<char, chars...>{});
}
template <char... chars>
constexpr unsigned long long calc() {
return calc( std::integer_sequence<char, chars...>{} );
}
template<class T, T x>
constexpr T lvalue = x;
template <char... chars>
unsigned long long const& operator "" _lvalue() {
return lvalue<unsigned long long, calc<chars...>()>;
}
live example. About half of this is 0b and 0x and 0 binary/hex/octal support.
Use:
a.cache(6_lvalue);
Alternatively, in C++17 you can do this:
template<auto x>
constexpr auto lvalue = x;
or in C++14
template<class T, T x>
constexpr T lvalue = x;
with
#define LVALUE(...) lvalue<std::decay_t<decltype(__VA_ARGS__)>, __VA_ARGS__>
In C++17 it is lvalue<7> and in C++14 it is LVALUE(7).

std::end with raw arrays of char

By default, the std::end overload for raw arrays looks something like this:
template<class T, std::size_t N>
T* end(T (&array)[N])
{ return array + N; }
However, this overload was undesirable for me when passing a string literal or a char array, as they both have an implicit \0 at the end which gets counted.
I thought as a workaround I could overload end in my own namespace:
Example:
namespace unique
{
const char* end(const char (&array)[5])
{
return array + 4;
}
const char* end(const char (&array)[11])
{
return array + 10;
}
}
int main()
{
using std::begin;
using std::end;
using unique::end;
const char str1[] = "XXXTEXTXXX";
const char str2[] = "TEXT";
auto iter = std::search(begin(str1), end(str1), begin(str2), end(str2));
//...
}
However, that would require a lot of overloads to write.
QUESTION
I realize using std::string or another container would solve my problem. However, I wanted to be able to call end unqualified with a string literal or raw array and have it omit the null terminator as above. Is there a better approach that would avoid writing the overloads?
The obvious (if you're sure it'll only be used under the right circumstances) would be a variant of the original:
namespace unique {
template<class T, std::size_t N>
T* end(T (&array)[N])
{ return array + N-1; }
}
Just be sure to only use it in the right situations, or you'll end up with end that points one before the end.
If you want to restrict it to character types, you can use a couple of overloads that only work for character types:
namespace unique {
template <std::size_t N>
char *end(char (&array)[N])
{
return array + N - 1;
}
template <std::size_t N>
wchar_t *end(wchar_t (&array)[N])
{
return array + N - 1;
}
}
With this, an array of char will use the version that assumes a NUL terminator, but an array of int will use std::end so it refers to the entire array:
int main()
{
using std::begin;
using std::end;
using unique::end;
char str1 [] = "12345";
wchar_t str2 [] = L"12345";
int i4 [] = { 1, 2, 3, 4, 5, 6 };
std::cout << std::distance(begin(str1), end(str1)) << "\n";
std::cout << std::distance(begin(str2), end(str2)) << "\n";
std::cout << std::distance(begin(i4), end(i4)) << "\n";
}
Do note, however, that since there's an existing template named begin, these overloads will only match exact types, so if you want them to work with const char and const wchar_t (for example) those will require separate overloads from the ones above that work with non-const types.
Also note that these will still apply to typedefs, so (for example) the fairly common:
typedef char small_int;
...can/will lead to problems -- the real type is still char, so end(my_small_int_array) will use the char overload instead of the base template.