Parsing a C++ string into a tuple - c++

Question

I am working on a simple CSV parser that would store the lines of a file in a tuple. This would be an easy task if it wasn't for the fact that the number of entries on the lines inside the file is a variable, as well as their type. Thus, the lines could be like that:
1,2.2,hello,18,world
The parser should be able to work like this:
ifstream file("input.csv");
SimpleCSVParser<int, float, string, int, string> parser(file);
Things get complex when I try to implement a function to parse the actual line. I still have not found a way to extract the next type from the parameter list to declare the variable before calling file >> var on it. I would also need to do this in a loop, somehow constructing a tuple from the results of each iteration.
So how do I parse the string into a tuple using plain C++11?
I tried this:
template <typename ...Targs>
tuple<Targs...> SimpleCSVParser<Targs...>::iterator::operator*() {
istringstream in(cur);
in.imbue(locale(locale(), new commasep)); // for comma separation
tuple<Targs...> t;
for (size_t i = 0; i < sizeof...(Targs); ++i) {
tuple_element<i,decltype(t)>::type first;
in >> first;
auto newt = make_tuple(first);
// what do I do here?
}
}
But it doesn't work since the tuple I use to extract types is empty.

Answer 1

It seems, you try to iterate over tuple indices/types which doesn't work, I think. What you can do, however, is to just call a read function for each member. The idea is to delegate processing of the tuple to a function which uses a parameter pack to expand an operation to an operation on each element. std::index_sequence<...> can be used to get the sequence of integers.
Something like this:
template <typename T>
bool read_tuple_element(std::istream& in, T& value) {
in >> value;
return true;
}
template <typename Tuple, std::size_t... I>
void read_tuple_elements(std::istream& in, Tuple& value, std::index_sequence<I...>) {
std::initializer_list<bool>{ read_tuple_element(in, std::get<I>(value))... });
}
template <typename ...Targs>
tuple<Targs...> SimpleCSVParser<Targs...>::iterator::operator*() {
std::istringstream in(cur);
in.imbue(std::locale(std::locale(), new commasep)); // for comma separation
std::tuple<Targs...> t;
read_tuple_elements(in, t, std::make_index_sequence<sizeof...(Targs)>{});
if (in) { // you may want to check if all data was consumed by adding && in.eof()
// now do something with the filled t;
}
else {
// the value could *not* successfully be read: somehow deal with that
}
}
The basic idea of the above code is simply to create a suitable sequence of calls to read_tuple_element(). Before jumping into the generic code, assume we'd want to implement reading of a std::tuple<T0, T1, T2> value with just three elements. We could implement the read using (using rte() instead of read_tuple_element() for brevity):
rte(get<0>(value)), rte(get<1>(value)), rte(get<2>(value));
Now, instead of writing this out for each number of elements, if we had an index sequence std::size_t... I we could get this sequence [nearly] using
rte(get<I>(value))...;
It isn't allowed to expand a parameter pack like this, though. Instead, the parameter pack needs to be put into some context. The code above uses a std::initializer_list<bool> for this purpose: the elements of a std::initializer_list<T> are constructed in the order listed. That is, we got
std::initializer_list<bool>{ rte(get<I>(value))... };
The missing bit is how to create the parameter pack I evaluating to a sequence of suitable indices. Conveniently, the standard library defines std::make_index_sequence<Size> which creates a std::index_sequence<I...> with a sequence of values for I as 0, 1, 2, ..., Size-1. So, calling read_tuple_elements() with std::make_index_sequence<sizeof...(Targs){} creates an object with a suitable list of arguments which can be deduced and then used to expand the tuple into a sequence of elements passed to read_tuple_element().

Answer 2

You cannot use tuples like this.
This would be an easy task if it wasn't for the fact that the number
of entries on the lines inside the file is a variable, as well as
their type.
If I understand, you only know your wanted tuples size and types at run-time while processing your file. Unfortunately this must be known at compile-time...
If you really want to use tuples, you have to make a pre treatment on your file to determine the data size and types. Then you can use the right tuples accordingly. But you cannot do that directly.

Answer 3

The usual method to do something like this via type erasure, for example using a union of all possible value types plus a flag indicating which is the actual entry
namespace generic_type {
struct generic
{
enum type { Void=0, Bool=1, Int=2, String=3, Float=4 };
type Type=Void;
union {
bool B;
std::int64_t I;
std::string S;
double X;
}
generic() = default;
generic(generic&&) = default;
generic(generic const&) = default;
generic(bool b) : Type(Bool), B(b) {}
generic(std::int64_t i) : Type(Int), I(i) {}
generic(std::uint64_t i) : Type(Int), I(i) {}
generic(std::string const&s) : Type(String), S(s) {}
generic(std::string &&s) : Type(String), S(std::move(s)) {}
generic(double x) : Type(Float), X(x) {}
};
namespace details {// auxiliary stuff
template<typename T, typename E=void>
struct traits
{
static constexpr generic::type Type=generic::Void;
static void get(generic const&) {}
};
template<>
struct traits<bool,void>
{
static constexpr generic::type Type=generic::Bool;
static bool get(generic const&x) { return x.B; }
};
template<typename T>
struct traits<T,enable_if_t<std::is_integral<T>::value>
{
static constexpr generic::type Type=generic::Int;
static T get(generic const&x) { return x.I; }
};
template<>
struct traits<std::string,void>
{
static constexpr generic::type Type=generic::Str;
static std::string const& get(generic const&x) { return x.S; }
static std::string&& get(generic&&x) { return std::move(x.S); }
};
template<T>
struct traits<float,enable_if<std::is_same<T,float>::value ||
std::is_same<T,double>::value>
{
static constexpr generic::type Type=generic::Float; };
static T get(generic const&x) { return x.X; }
}
}
template<typename T>
auto unsafe_extract(generic const&x)
-> decltype(details::traits<T>::get(x))
{ return details::traits<T>::get(x); }
template<typename T>
auto unsafe_extract(generic&&x)
-> decltype(details::traits<T>::get(std::move(x)))
{ return details::traits<T>::get(std::move(x)); }
template<typename T>
auto extract(generic const&x)
-> decltype(unsafe_extract(x))
{
if(details::traits<T>::Type != x.Type)
throw std::runtime_error("type mismatch in extract(generic)");
return unsafe_extract(x);
}
template<typename T>
auto extract(generic&&x)
-> decltype(unsafe_extract(std::move(x)))
{
if(details::traits<T>::Type != x.Type)
throw std::runtime_error("type mismatch in extract(generic&&)");
return unsafe_extract(std::move(x));
}
}
using generic_type::generic;
and then you can store your data in a std::vector<generic>.

Answer 4

for (size_t i = 0; i < sizeof...(Targs); ++i) {
tuple_element<i,decltype(t)>::type first;
in >> first;
auto newt = make_tuple(first);
// what do I do here?
}
This is runtime. You should consider using recursive functions using variadic templates. That would be compile time.

Answer 5

If you use std::tuple_cat you should be able to add each subsequent value to the tuple. I would also reccommend using C++14 return type deduction if I were you it eliminates the need to know the return type.