We are using flatbuffers with size prefixed buffers and want to mutate a flatbuffer but there is no GetMutableSizePrefixedRoot or GetSizePrefixedMutableRoot.
I could make a PR and add one like that.
template<typename T> T *GetMutableSizePrefixedRoot(void *buf) {
return GetMutableRoot<T>(reinterpret_cast<uint8_t *>(buf) +
sizeof(uoffset_t));
}
I could also change the code generator so that a GetMutableSizePrefixedXXX (e.g. GetMutableSizePrefixedMonster) will be generated .
Related
I have 3 structs : Student, Citizen, Employee. I want user to be able to choose what struct they want to work with (std::vector of structs, actually). Since there's no way to define type at runtime, I created all 3 vectors, but will use only one of them (depending on the user's choice), others will stay empty:
std::vector<Student> container_student;
std::vector<Citizen> container_citizen;
std::vector<Employee> container_employee;
auto containers = make_tuple(container_student, container_citizen, container_employee);
std::cout << "Enter:\n0 to operate \"Student\" struct\n1 to operate \"Citizen\" struct\n2 to operate \"Employee\" struct\n";
std::cin >> container_type;
auto container = std::get<container_type>(containers);
But I get No matching function for call to 'get', even though container_type is an int and containers is a tuple.
Edit: understandable, auto can't make magic and I still try to make container's type to depend on runtime. But even if I try to use std::get<container_type>(containers) (probably define would help) instead of container in functions etc., I get the same error, which is not understandable.
Unfortunately, what you're proposing isn't possible in C++. The C++ typing and template system works at compile-time, where information read in from the user isn't available. As a result, anything passed into a template's angle braces needs to be determinable at compile-time. In your case, the number the user enters, indicating which option they want to select, is only knowable at runtime.
There are some routes you could take to achieve the same result, though. For example, one option would be to do something like this:
if (container_type == 0) {
auto container = std::get<0>(containers);
/* ... */
} else if (container_type == 1) {
auto container = std::get<1>(containers);
/* ... */
} /* etc */
Here, all the template angle braces are worked out at compile-time. (Then again, if this is what you're going to be doing, you wouldn't need the tuple at all. ^_^)
Another option would be to use templates, like this:
template <typename T> void doSomething(std::vector<T>& container) {
/* Put your code here */
}
/* Then, back in main... */
if (container_type == 0) {
doSomething(container_student);
} else if (container_type == 1) {
doSomething(container_citizen);
} /* etc */
This still requires you to insert some code to map from integer types to the functions you want to call, but it leaves you the freedom to have a container variable (the one in doSomething) that you can treat generically at that point.
It's basically the Fundamental Theorem of Software Engineering in action - all problems can be solved by adding another layer of indirection. :-)
Hope this helps!
I am writing code generator and using flatbuffers for generating classes. The rest of the code generator will work with these classes in C++.
I have not been able to figure out how to keep the API consistent for reading data for two different types of classes that flatbuffer may generate. I am using the object api (testRecordT) in the example for whenever an object needs to be written to (and can be read back as well) and flatbuffer overlay for when the data can only be read from.
I have not been able to get any template or free functions to work to give me a consistent api that would work in both the cases.
Below is a snippet of what I am trying to get to work.
struct testRecordT {
int32_t field1;
std::string field2;
};
struct testRecord {
int32_t field1() const {
return 0;
// flatbuffer generated - return GetField<int32_t>(VT_FIELD1, 0);
}
const flatbuffers::String *field2() const {
return nullptr;
// flatbuffer generated - return GetPointer<const flatbuffers::String *>(VT_FIELD3);
}
};
void Test() {
testRecordT * members; // assume pointers are valid
testRecord * memberFunctions;
// Need to be able to create a read function/template that would work. This would simplify the code generation a lot. I can generate either one below, as long as it is consistent in both cases.
auto r = read(members->field1); // or read(members,field1)
auto v = read(memberFunctions->field1); // or read(memberFunctions,field1)
}
The read functions or template functions should be consistent. Any pointers or thoughts would be helpful. I am using C++17 with gcc 7.3.1 .
You can use std::invoke for this. It can both call functions or access members.
auto r = std::invoke(&testRecordT::field1, members);
auto v = std::invoke(&testRecord::field1, memberFunctions);
You can use std::invoke() for this problem.
I need to represent in a AST a structure like this:
struct {
int data;
double doubleDataArray[10];
struct {
int nestedData;
};
};
I'm creating an AST like this one:
I need to retrieve data from leaves. The problem that I have is that leaves contains heterogenous data. A leaf can represent an integer value, a double, a string and so on.
I can create classes like IntValue, DoubleValue that inherit from Value and store respective data, perform a dynamic_cast to convert Value to the type referred in its type attribute. Something like
switch (value->getType()) {
case Type::Int: {
auto iv = dynamic_cast<IntValue>(value);
int value = iv->getValue();
} break;
case Type::Double() {
auto dv = dynamic_cast<DoubleValue>(value);
double value = dv->getValue();
} break;
//…
}
but I'd like to know if there's a better way, because a switch like that one it's not easy maintainable and readable.
I've seen some example, like in boost::program_options, something like:
int value = value->getValue().as<int>();
It's a better way? How can I reproduce this?
You could do something like this using c++17
struct node {
//... other stuff
std::variant</*your types of nodes here*/> type;
}
then call this visitor on your nodes
std::visit([](auto&& node) {
if constexpr(std::is_same_v<std::decay_t<decltype(node)>, /* your type here */>) {
// ...
}
else if constexpr(/* ... */) {
// ...
}
}, node0.type);
Going on a tangent for a slightly different flavor of a solution, how about doing it the way capnproto does it? Capnproto's own schema compiler represents the AST in memory using the Capnproto wire encoding. The schema supports tagged unions. The lexer and parser for the schema are built using combinators (although I presume that you already have a good parser in place that produces the AST).
The structure could be expressed as follows using capnp schema:
# MyAst.capnp
struct Struct {
fields #0 :List(Field);
}
struct Field {
name #4 :Text;
union {
integer #0 :List(Int32);
fpoint #1 :List(Double);
text #2 :List(Text);
structure #3 :Struct;
}
}
The schema compiler would generate C++ code for this, with the following important classes Struct::Reader, Struct::Builder, Field::Reader and Field::Builder. Whatever makes the AST would use the Struct::Builder type to make a structure instance, with its data. Then, you'd traverse the structure as follows:
void processData(Struct::Reader reader) {
auto fields = reader.getFields();
for (auto &field : fields) {
if (field.hasInteger()) {
int32_t val = field.getInteger();
...
} else if (field.hasFpoint()) {
double val = field.getFpoint();
...
} else if (field.hasText()) {
kj::StringPtr val = field.getText();
...
} else if (field.hasStructure()) {
processData(field.getStructure());
}
}
}
The kj framework (included in capnproto) has quite a few compiler-building goodies, such as memory arenas. A Foo::Builder would then be obtained from an Orphan<Foo>, and the orphan is produced by an orphanage that carves out memory from an arena allocator. With your entire AST built in an arena with one or few large, contiguous segments, this would perform better than allocating all those types on general-purpose heap (assuming that your AST is not tiny). This representation also serializes directly to disk or network with no transcoding: you can do a binary dump of an orphanage's arena, then later load it directly and you get all your data back with zero effort and zero transcoding. The Foo::Reader and Foo::Builder types provide very fast accessors that don't do any data decoding nor translation - that's the advantage of the capnproto encoding. If you modify the data in the AST, the orphanage may grow, but it also provides a copy operation that copies only the referenced areas (a copying GC, if you will) - and that's blazing fast, too, since no transcoding is done. Chunks of verbatim binary data are copied with very little traversal overhead.
I am processing large files consisting of many redundant values (using YAML's anchors and references). The processing I do on each structure is expensive, and I would like to detect whether I'm looking at a reference to an anchor I've already processed. In Python (with python-yaml), I did this by simply building a dictionary keyed by id(node). Since yaml-cpp uses Node as a reference type, however, this does not seem to work here. Any suggestions?
This is similar to Retrieve anchor & alias string in yaml-cpp from document, but although that feature would be sufficient to solve my problem, it is not neccessary -- if I could get somehow a hash based on the internal address of the node, for example, that would be fine.
The expensive thing I'm doing is computing a hash of each node including itself and its children.
Here is a patch that seems to do what I need. Proceed with caution.
diff -nr include/yaml-cpp/node/detail/node.h new/yaml-cpp-0.5.1/include/yaml-cpp/node/detail/node.h
a13 1
#include <boost/functional/hash.hpp>
a24 1
std::size_t identity_hash() const { return boost::hash<node_ref*>()(m_pRef.get()); }
diff -nr /include/yaml-cpp/node/impl.h new/yaml-cpp-0.5.1/include/yaml-cpp/node/impl.h
a175 5
inline std::size_t Node::identity_hash() const
{
return m_pNode->identity_hash();
}
diff -nr include/yaml-cpp/node/node.h new/yaml-cpp-0.5.1/include/yaml-cpp/node/node.h
a55 2
std::size_t identity_hash() const;
I can then use the below to make a unordered_map using YAML::Node as key.
namespace std {
template <>
struct hash<YAML::Node> {
size_t operator()(const YAML::Node& ss) const {
return ss.identity_hash();
}
};
}
You can check node identity by operator == or Node::is, e.g.:
Node a = ...;
process(a);
Node b = ...;
if (!a.is(b)) {
process(b);
}
I suppose this isn't perfect - if you're trying to do this on a large list of nodes, the checking will have to be O(n).
If you want more than this, please file an issue on the project page.
I have a header file like:
#include <vector>
inline std::vector<uint8_t>& vec() {
static std::vector<uint8_t> v { 'a', 'b', 'c', 'd' };
return v;
}
inline const std::vector<uint8_t>& cvec() {
return vec();
}
I can wrap it in SWIG using std_vector.i and pyabc.i but that is quite inefficient (there's a jump between C++ and Python code for every access) and given that these are literally just a bunch of bytes I ought to be able to wrap them with Python's memoryview interface.
How can I expose my std::vector<uint8_t> as a Python memoryview?
Exposing it as a memoryview requires creating a Py_buffer first. In Python 3.3+ there is a convenient helper function, PyMemoryView_FromMemory that does a lot of the work for us. In earlier versions though we'll need to take a few extra steps, so our basic out typemap looks like:
%typemap(out) std::vector<uint8_t>&, const std::vector<uint8_t>& {
Py_buffer *buf=(Py_buffer*)malloc(sizeof *buf);
const bool ro = info<$1_type>::is_readonly();
if (PyBuffer_FillInfo(buf, NULL, &((*$1)[0]), (*$1).size(), ro, PyBUF_ND)) {
// error, handle
}
$result = PyMemoryView_FromBuffer(buf);
}
Here we're basically allocating some memory for the Py_buffer. This just contains the details of the buffer internally for Python. The memory we allocate will be owned by the memoryview object once it's created. Unfortunately since it's going to be released with a call to free() we need to allocate it with malloc(), even though it's C++ code.
Besides the Py_buffer and an optional Py_Object PyBuffer_FillInfo takes a void* (the buffer itself), the size of the buffer, a boolean indicating if it's writeable and a flag. In this case our flag simply indicates that we have provided C-style contiguous memory for the buffer.
For deciding if it is readonly or not we used SWIG's built in $n_type variable and a helper (which could be a C++11 type trait if we wanted).
To complete our SWIG interface we need to provide that helper and include the header file, so the whole thing becomes:
%module test
%{
#include "test.hh"
namespace {
template <typename T>
struct info {
static bool is_readonly() {
return false;
}
};
template <typename T>
struct info<const T&> {
static bool is_readonly() {
return true;
}
};
}
%}
%typemap(out) std::vector<uint8_t>&, const std::vector<uint8_t>& {
Py_buffer *buf=(Py_buffer*)malloc(sizeof *buf);
const bool ro = info<$1_type>::is_readonly();
if (PyBuffer_FillInfo(buf, NULL, &((*$1)[0]), (*$1).size(), ro, PyBUF_ND)) {
// error, handle
}
$result = PyMemoryView_FromBuffer(buf);
}
%include "test.hh"
We can then test it with:
import test
print test.vec()
print len(test.vec())
print test.vec()[0]
print test.vec().readonly
test.vec()[0]='z'
print test.vec()[0]
print "This should fail:"
test.cvec()[0] = 0
Which worked as expected, tested using Python 2.7.
Compared to just wrapping it using std_vector.i this approach does have some drawbacks. The biggest being that we can't resize the vector, or convert it back to a vector later trivially. We could work around that, at least partially by creating a SWIG proxy for the vector like normal and using the second parameter of PyBuffer_FillInfo to store it internally. (This would also be needed if we had to manage the ownership of the vector for instance).