How to construct a tuple from an array - c++

I am designing a C++ library that reads a CSV file of reported data from some experiment and does some aggregation and outputs a pgfplots code. I want to make the library as generic and easy to use as possible. I also want to isolate it from the data types that are represented in the CSV file and leave the option to user to parse each column as she desires. I also want to avoid Boost Spirit Qi or other heavy duty parser.
The simple solution I have is for the user to create a type for each column, with a constructor that takes "char *". The constructor does its own parsing for the value it is given, which is one cell from the data. The user then passes me a list of types; the schema, representing the types in a line of data. I use this type list to create a tuple, in which every member of the tuple is responsible for parsing itself.
The problem now is how to initialise (construct) this tuple. Dealing with tuples is of course not straightforward since iterating over their elements is mostly a compile-time operation. I used Boost Fusion at first to achieve this task. However, the function I used (transform) although might take a tuple as input (with the appropriate adapter), it does not seem to return a tuple. I need the return value to be a tuple so some other code can use it as an associative type-to-value container (access it by type via std::get<T>), while using only standard tools, that is, without using Boost. So I had to convert whatever Fusion's transform returned into std::tuple.
My question is how to avoid this conversion, and better yet how to avoid Boost Fusion completely.
A simple solution that comes to mind is to use the constructor of std::tuple, and somehow pass each element its respective "const *" that it needs to construct. However, while this is possible using some complicated template-based enumeration techniques, I am wondering if there is a straightforward "parameter-pack"-like approach, or an even simpler way to pass the values to the constructors of the individual elements of a tuple.
To clarify what I am seeking, kindly take a look at this following code.
#include <cstdio>
#include <array>
template <typename...> struct format {};
template <typename...> struct file_loader {};
template <typename... Format>
struct
file_loader<format<Format...> > {
void load_file() {
size_t strsize = 500u;
char *str = new char[strsize]();
auto is = fopen("RESULT","r");
/* example of RESULT:
dataset2,0.1004,524288
dataset1,0.3253,4194304
*/
while(getline(&str, &strsize, is) >= 0) {
std::array<char*, 3> toks{};
auto s = str;
int i = 2;
while(i --> 0)
toks[i] = strsep (&s, ",");
toks[2] = strsep (&s, ",\n");
std::tuple<Format...> the_line{ /* toks */ } ; // <-- HERE
//// current solution:
// auto the_line{
// as_std_tuple( // <-- unnecessary conversion I'd like to avoid
// boost::fusion::transform(boost::fusion::zip(types, toks), boost::fusion::make_fused( CAST() ))
// )};
// do something with the_line
}
}
};
#include <string>
class double_type {
public:
double_type() {}
double_type(char const *token) { } // strtod
};
class int_type {
public:
int_type() {}
int_type(char const *token) { } // strtoul
};
int main(int argc, char *argv[]) {
file_loader< format< std::string,
double_type,
int_type > >
{}.load_file();
return 0;
}
I've highlighted the interesting line as "HERE" in a comment.
My question precisely is:
Is there a way to construct a std::tuple instance (of heterogeneous
types, each of which is implicitly convertible from "char *") with
automatic storage duration (on the stack) from a std::array<char *, N>,
where N equals the size of that tuple?
The answer I am seeking should
Avoid Boost Fusion
(Simplicity condition) Avoid using more than 5 lines of boilerplate template-based enumeration code
Alternatively, shows why this is not possible to do in the C++14 standard
The answer can use C++17 constructs, I wouldn't mind.
Thank you,

As with all questions involving std::tuple, use index_sequence to give you a parameter pack to index the array with:
template <class... Formats, size_t N, size_t... Is>
std::tuple<Formats...> as_tuple(std::array<char*, N> const& arr,
std::index_sequence<Is...>)
{
return std::make_tuple(Formats{arr[Is]}...);
}
template <class... Formats, size_t N,
class = std::enable_if_t<(N == sizeof...(Formats))>>
std::tuple<Formats...> as_tuple(std::array<char*, N> const& arr)
{
return as_tuple<Formats...>(arr, std::make_index_sequence<N>{});
}
Which you would use as:
std::tuple<Format...> the_line = as_tuple<Format...>(toks);

Related

Constexpr: Convert a list of string views into a list of char arrays

I came across an interesting constexpr problem that haven't been able to fully crack. The last piece of the puzzle that I'm missing is the following
// Given a constexpr array of string views
constexpr std::array<std::string_view, X> views = ...;
// Convert it to a list of char arrays
constexpr std::tuple<std::array<char, Xs>...> buffers = ...;
My problem here is finding the right size for each array. How can I extract the sizes of the string_views in views and pass them as template parameters to another function?
I could just use the same size for every buffer, big enough to hold every argument, but I'm wondering if there is a way of optimizing their sizes since the info is known at compile time.
Full description of the problem I'm trying to solve.
(in case someone can come up with a better approach, also cause I think it's interesting...)
I want to create a macro that converts a list of arguments into a list of name-value pairs. For instance
// Adding types explicitly to show what I want to achieve.
int x;
float y;
double z;
using named_values_t = std::tuple<
std::pair<const char*, int&>,
std::pair<const char*, float&>,
std::pair<const char*, double&>>;
named_values_t nv = MY_MACRO(x, y, z);
The const char* adds significant difficulty but it's a requirement for a third-party lib.
Now I know that can be done with Boost.Preprocessor but I'd like to do it just using the STL and constexpr methods to avoid adding boost just for this. Also I know this would be trivial on a compiler with support for constexpr std::string but I'm using C++17.
The string processing can be easily done with constexpr functions,
// Split "one, two, three" into {"one", "two", "three"}.
template <size_t Count>
constexpr std::array<std::string_view, Count> split_arguments(std::string_view);
However, I cannot pass these string_views directly as char pointers, since at this point they're just pointers to a bigger array in memory (the complete "one, two, three"). To pass as const char* each element needs to be null-terminated.
But we can build an std::array for each std::string_view and copy its contents, this way we have a char array for each argument name, that will generate a null-terminated segment of memory for each name.
constexpr std::string_view args = "one, two, three";
constexpr std::array<std::string_view, 3> views = split_arguments<3>(args); // {"one", "two", "three"}
constexpr std::tuple<std::array<char, Xs>...> buffers = make_buffers<Xs...>(views);
Here I'm not able to figure out how pass the lengths of the views as template arguments to the next function.
Working solution here (using bigger fixed-sized buffers): https://gcc.godbolt.org/z/WKsbvb
The fixed-size buffer solution is ok, but it would be great to go that extra step to fit the buffers to their actual size.
So long as views is a variable with static storage duration (rather than a prvalue created by a constexpr function call), you can implement this with the usual auto& template parameter and std::index_sequence tricks:
#include<array>
#include<string_view>
#include<tuple>
#include<utility>
#include<type_traits>
#include<cstddef>
namespace detail {
template<std::size_t N>
constexpr auto copy_string(std::string_view s) {
std::array<char,N+1> ret{}; // zero-initialize
for(std::size_t i=N;i--;) ret[i]=s[i];
return ret;
}
template<auto &V,std::size_t ...II>
constexpr auto buffers(std::index_sequence<II...>) {
return std::make_tuple(copy_string<V[II].size()>(V[II])...);
}
}
template<auto &V> constexpr auto buffers=
detail::buffers<V>
(std::make_index_sequence
<std::tuple_size_v<std::remove_reference_t<decltype(V)>>>());
constexpr std::array<std::string_view, 3> views = {"C","++",""};
static_assert(std::is_same_v
<decltype(buffers<views>),
const std::tuple<std::array<char,2>,
std::array<char,3>,
std::array<char,1>>>);
static_assert(std::get<0>(buffers<views>)[0]=='C');
static_assert(std::get<1>(buffers<views>)[1]=='+');
static_assert(std::get<2>(buffers<views>)[0]=='\0');

Make a user defined variable array

I have to write a simple code that takes a char as input or a string with an integer. Then makes a std vector, depending on the input. If the text says int I have an int vector.
The only problem is that I don't want to declare for each variable type a vector even if empty, and I want to make it scalable so if someday I want to put a struct or something else in it i can.
dvec::dvec( char t){
if ( t=='i')
vector<int> a;
else if( t=='f')
vector<float> a;
}
If all you want is primitive types or pointers, you can make a union (8 bytes in size) and a vector of that union. It's a bit C'ish, but template is a compile time solution, so for a linkable solution, you need space for every type. You can have getters, setters, constructors for each type.
You can implement this thing in two ways:
1.
using element_type = std::variant<char, int, float /* ... other types */>;
using dvec = std::vector<element_type>;
This will be type safe, albeit the disadvantage is that every element of the vector is a variant, which might not be what you want.
2.
using dvec = std::variant<std::vector<char>,
std::vector<int>,
std::vector<float> /* ... other vector types */>;
This will give you a variant of vectors, where all vector elements are homogeneous.
This cumbersome expression could be simplified by using Boost.Mp11
template<class T> using vectorize_t = std::vector<T>;
template<typename ...T>
using dvec = std::variant<mp_transform<vectorize_t, T...>>;
which basically puts all respective types in T... to an std::vector.
Write a helper function to do the real work.
template <typename T>
void do_work() {
std::vector<T> a;
// do stuff
}
// ...
if(t == 'i') {
do_work<int>();
}
else if(t == 'f') {
do_work<float>();
}
else if(t == 's') {
do_work<your_struct>();
}
Depending on what your specific uses are you might need (or at least want) to have do_work just call multiple functions, that way you can specialize if needed.

Convert from Boost Hana Tuple to Std Vector

I am trying to create a std::vector from a boost::hana::tuple at compile-time like so:
boost::hana::tuple<std::string> namesString{ "Hello", "World" };
std::vector<std::string> namesStringVector{};
while(!(hana::is_empty(namesString)))
{
namesStringVector.emplace_back(hana::front(namesString));
hana::drop_front(namesString);
}
This clearly doesn't work because the while loop is not run at compile-time.
How do we achieve this effect in Boost::Hana? I.e. what compile-time Hana construct would allow us to perform this cast? I tried doing
namesStringVector = (std::vector<std::string>)namesString;
and
hana::to < std::vector < std::string > >(namesString);
But it tells me there does not exist such a cast in both cases.
In addition to the concerns that Louis addressed there are some other issues with how you are trying to use a tuple. Note that a tuple can not have its length or the types within it changed as types are completely immutable in C++ so you have to think in terms of pure functional programming.
Your call to is_empty will always return true, and drop_front can not change the input value, it only returns a new tuple with the front element removed. (in your case it would be tuple<> which is empty).
You might want to rethink your use case for wanting to convert a tuple to vector, but here is an example to hopefully get you started.
#include <boost/hana.hpp>
#include <iostream>
#include <string>
#include <vector>
namespace hana = boost::hana;
template <typename T>
constexpr auto to_vector = [](auto&& ...x) {
return std::vector<T>{std::forward<decltype(x)>(x)...};
};
int main() {
auto xs = hana::make_tuple("Hello", "World");
auto vec = hana::unpack(xs, to_vector<std::string>);
for (auto const& x : vec) {
std::cout << x << ' ';
}
std::cout << '\n';
}
Notes about list types:
std::vector has a run-time length and a single type for all
elements.
std::array has a compile-time length and a single type for all
elements.
hana::tuple has a compile-time length and any element can be any
type.
For starters, boost::hana::tuple<std::string> namesString{ "Hello", "World" }; doesn't make sense because your tuple only has one element but you try to initialize it with two.
Second, it doesn't really make sense to initialize a std::vector from a hana::tuple, since it implies that all the elements of the tuple have the same type. Instead, you should probably be using std::array. Hana is useful when you need heterogeneity (elements with different types), which you don't seem to need here. When you don't need heterogeneity, using std::array or similar tools will be much easier and natural than using Hana.

Removing potentially redundant template parameters when loading a binary file into Eigen matrix

I want to read a plain binary file containing a number of unsigned 16-bit integers into an Eigen matrix, and I wrote a templated utility to do this. This is what the caller looks like:
Matrix<uint16_t, Dynamic, Dynamic> data;
int retval = read_data<Matrix<uint16_t, Dynamic, Dynamic>, uint16_t>(
argv[1], data);
And here's what read_data looks like:
template <typename Derived, typename Scalar> // Per #Jarod42, get rid of Scalar here (*)
int read_data(const char* const fname, MatrixBase<Derived>& data) {
// (*) If we don't have Scalar as a template, just uncomment this:
// typedef typename Derived::Scalar Scalar;
ifstream fin(fname, ios::binary);
if (!fin) {
return 2;
}
fin.seekg(0, fin.end);
long long bytes = fin.tellg();
if (bytes % sizeof(Scalar) != 0) {
// The available number of bytes won't fill an even number of Scalar
// values
return 3;
}
long long nscalars = bytes / sizeof(Scalar);
// See http://forum.kde.org/viewtopic.php?f=74&t=107551
MatrixBase<Derived>& data_edit = const_cast<MatrixBase<Derived>&>(data);
data_edit.derived().resize(nscalars, 1);
Scalar* buffer = new Scalar[nscalars]; // Switched to vector per #Casey
fin.seekg(0, fin.beg);
fin.read(reinterpret_cast<char*>(buffer), bytes);
if (!fin) {
// All data not read. fin.gcount() will indicate bytes read.
return 4;
}
for (long long idx = 0; idx < nscalars; ++idx) {
data_edit(idx) = buffer[idx];
}
return 0;
}
In brief,
the file is opened,
its size is obtained,
an array is dynamically allocated to store all the data,
the file is read into the array,
the array's contents are copied into the matrix.
This is reasonable and it works (though I'm open to suggestions for improvements), but I think the function has one too many template parameters, and the function call in the caller is just too verbose. I think there should be a way to eliminate the second template parameter, which only serves to tell read_data the number of bytes per scalar (2 in the case of uint16_t), and which I believe should be inferrable using the first template parameter.
Questions Is there a way to eliminate the seemingly redundant second template parameter to read_data?
Also, is my approach of passing in a matrix reference only to resize it in the read_data function (using the verbose and confusing idiom of creating a modifiable reference to the matrix in order to resize it via derived()) the right way to proceed? I realize this dynamically allocates memory, which is fine, but I think it is not doing anything wasteful---correct?.
(Discussion Is there other improvements to this code one would like to see? I'm a C or Python numerical coder; in C, I'd just deal with void* arrays and pass an extra function argument telling the function the size of each scalar; with Python I'd just do numpy.fromfile('path/to/file.bin', dtype=numpy.uint16) and be done with it. But I'd like to do it right by Eigen and C++.)
NB. I use matrixes instead of vectors because I'll be resizing them into rectangular matrixes later.
NB2. In Fixed Sized Eigen types as parameters the notion of templating the function using the scalar type is promoted. I am not averse to this approach, I chose to pass read_data a matrix reference instead of making it return a Matrix object because I wanted integer return values indicating errors---though now I realize I ought to make those exceptions.
NB3. In c++ check for nested typedef of a template parameter to get its scalar base type an elaborate set of helper classes is used to, I think, achieve a similar effect for templated classes. I'm curious if a simpler approach can be used here, for a templated function.
NB4. A simple improvement that I'm aware of is typedeffing Matrix<uint16_t, Dynamic, Dynamic> to reduce verbosity.
I think that Matrix<uint16_t, Dynamic, Dynamic> should have a typedef to retrieve uint16_t. If so, you may write as first line (assuming it is type):
typedef typename Derived::type Scalar;
If there is no typedef, you may write a traits for that:
template <typename> struct matrix_type;
template <typename T, typename U1, typename U2>
struct matrix_type<Matrix<T, U1, U2> >
{
typedef T type;
typedef U1 type1;
typedef U2 type2;
};
And then write
typedef typename matrix_type<Derived>::type Scalar;
Note:
If you have written the parameters in the other order
template <typename Derived, typename Scalar>
int read_data(const char* const fname, MatrixBase<Derived>& data);
you may write (as the last parameter may be deduced):
Matrix<uint16_t, Dynamic, Dynamic> data;
int retval = read_data<uint16_t>(argv[1], data);
And now that the parameter is obsolete, you may write directly:
Matrix<uint16_t, Dynamic, Dynamic> data;
int retval = read_data(argv[1], data);
You may want to look at map in eigen. It takes a chuck a memory directly.
TutorialMapClass.html">http://eigen.tuxfamily.org/dox/group_TutorialMapClass.html
To construct a Map variable, you need two other pieces of information: a pointer to the region of memory defining the array of coefficients, and the desired shape of the matrix or vector. For example, to define a matrix of float with sizes determined at compile time, you might do the following:
Map mf(pf,rows,columns);

Populate An Array Using Constexpr at Compile-time

I would like to populate an array of enum using constexpr.
The content of the array follows a certain pattern.
I have an enum separating ASCII character set into four categories.
enum Type {
Alphabet,
Number,
Symbol,
Other,
};
constexpr Type table[128] = /* blah blah */;
I would like to have an array of 128 Type. They can be in a structure.
The index of the array will be corresponding to the ASCII characters and the value will be the Type of each character.
So I can query this array to find out which category an ASCII character belongs to. Something like
char c = RandomFunction();
if (table[c] == Alphabet)
DoSomething();
I would like to know if this is possible without some lengthy macro hacks.
Currently, I initialize the table by doing the following.
constexpr bool IsAlphabet (char c) {
return ((c >= 0x41 && c <= 0x5A) ||
(c >= 0x61 && c <= 0x7A));
}
constexpr bool IsNumber (char c) { /* blah blah */ }
constexpr bool IsSymbol (char c) { /* blah blah */ }
constexpr Type whichCategory (char c) { /* blah blah */ }
constexpr Type table[128] = { INITIALIZE };
where INITIALIZE is the entry point of some very lengthy macro hacks.
Something like
#define INITIALIZE INIT(0)
#define INIT(N) INIT_##N
#define INIT_0 whichCategory(0), INIT_1
#define INIT_1 whichCategory(1), INIT_2
//...
#define INIT_127 whichCategory(127)
I would like a way to populate this array or a structure containing the array without the need for this macro hack...
Maybe something like
struct Table {
Type _[128];
};
constexpr Table table = MagicFunction();
So, the question is how to write this MagicFunction?
Note: I am aware of cctype and likes, this question is more of a Is this possible? rather than Is this the best way to do it?.
Any help would be appreciated.
Thanks,
Ignoring ALL the issues, indices to the rescue:
template<unsigned... Is> struct seq{};
template<unsigned N, unsigned... Is>
struct gen_seq : gen_seq<N-1, N-1, Is...>{};
template<unsigned... Is>
struct gen_seq<0, Is...> : seq<Is...>{};
template<unsigned... Is>
constexpr Table MagicFunction(seq<Is...>){
return {{ whichCategory(Is)... }};
}
constexpr Table MagicFunction(){
return MagicFunction(gen_seq<128>{});
}
Live example.
In C++17 ::std::array has been updated to be more constexpr friendly and you can do the same as in C++14, but without some of the scary looking hacks to get around the lack of constexpr in crucial places. Here is what the code would look like there:
#include <array>
enum Type {
Alphabet,
Number,
Symbol,
Other,
};
constexpr ::std::array<Type, 128> MagicFunction()
{
using result_t = ::std::array<Type, 128>;
result_t result = {Other};
result[65] = Alphabet;
//....
return result;
}
const ::std::array<Type, 128> table = MagicFunction();
Again MagicFunction still needs to obey the rather loose constexpr rules. Mainly, it may not modify any global variables or use new (which implies modifying global state, namely the heap) or other such things.
IMHO the best way to do this is simply write a tiny setup program that will generate table for you. And then you can either throw out the setup program, or check it in alongside the generated source code.
The tricky part of this question is just a duplicate of this other one: Is it possible to create and initialize an array of values using template metaprogramming?
The trick is, it's impossible to write anything like
Type table[256] = some_expression();
at file scope, because global arrays can be initialized only with literal (source-level) initializer-lists. You can't initialize a global array with the result of a constexpr function, even if you could somehow get that function to return a std::initializer_list, which you can't because its constructor isn't declared constexpr.
So what you have to do is get the compiler to generate the array for you, by making it a static const data member of a template class. After one or two levels of metaprogramming that I'm too confused to write out, you'll bottom out in a line that looks something like
template <int... Indices>
Type DummyStruct<Indices...>::table[] = { whichCategory(Indices)... };
where Indices is a parameter-pack that looks like 0,1,2,... 254,255. You construct that parameter-pack using a recursive helper template, or maybe just using something out of Boost. And then you can write
constexpr Type (&table)[] = IndexHelperTemplate<256>::table;
...But why would you do all that, when the table is only 256 entries that will never change unless ASCII itself changes? The right way is the simplest way: precompute all 256 entries and write out the table explicitly, with no templates, constexpr, or any other magic.
The way to do this in C++14 looks like this:
#include <array>
enum Type {
Alphabet,
Number,
Symbol,
Other,
};
constexpr ::std::array<Type, 128> MagicFunction()
{
using result_t = ::std::array<Type, 128>;
result_t result = {Other};
const result_t &fake_const_result = result;
const_cast<result_t::reference>(fake_const_result[65]) = Alphabet;
//....
return result;
}
const ::std::array<Type, 128> table = MagicFunction();
No clever template hackery required any longer. Though, because C++14 didn't really undergo a thorough enough review of what did and didn't have to be constexpr in the standard library, a horrible hack involving const_cast has to be used.
And, of course, MagicFunction had better not modify any global variables or otherwise violate the constexpr rules. But those rules are pretty liberal nowadays. You can, for example, modify all the local variables you want, though passing them by reference or taking their addresses may not work out so well.
See my other answer for C++17, which allows you to drop some of the ugly-looking hacks.