Extending a thrift generated object in C++ - c++

Using the following .thrift file
struct myElement {
1: required i32 num,
}
struct stuff {
1: optional map<i32,myElement> mymap,
}
I get thrift-generated class with an STL map. The instance of this class is long-lived
(I append and remove from it as well as write it to disk using TSimpleFileTransport).
I would like to extend myElement in C++, the extenstions should not affect
the serialized version of this object (and this object is not used in any
other language). Whats a clean way to acomplish that?
I contemplated the following, but they didn't seem clean:
Make a second, non thrift map that is indexed with the same key
keeping both in sync could prove to be a pain
Modify the generated code either by post-processing of the generated
header (incl. proprocessor hackery).
Similar to #2, but modify the generation side to include the following in the generated struct and then define NAME_CXX_EXT in a forced-included header
#ifdef NAME_CXX_EXT
NAME_CXX_EXT ...
#endif
All of the above seem rather nasty
The solution I am going to go with for now:
[This is all pseudo code, didn't check this copy for compilation]
The following generated code, which I cannot modify
(though I can change the map to a set)
class GeneratedElement {
public:
// ...
int32_t num;
// ...
};
class GeneratedMap {
public:
// ...
std::map<int32_t, GeneratedElement> myGeneratedMap;
// ...
};
// End of generated code
Elsewhere in the app:
class Element {
public:
GeneratedElement* pGenerated; // <<== ptr into element of another std::map!
time_t lastAccessTime;
};
class MapWrapper {
private:
GeneratedMap theGenerated;
public:
// ...
std::map<int32_t, Element> myMap;
// ...
void doStuffWIthBoth(int32_t key)
{
// instead of
// theGenerated.myGeneratedMap[key].num++; [lookup in map #1]
// time(&myMap[key].lastAccessTime); [lookup in map #2]
Element& el=myMap[key];
el.pGenerated->num++;
time(&el.lastAccessTime);
}
};
I wanted to avoid the double map lookup for every access
(though I know that the complexity remains the same, it is still two lookups ).
I figured I can guarantee that all insertions and removals to/from the theGenerated)
are done in a single spot, and in that same spot is where I populate/remove
the corresponding entry in myMap, I would then be able to initialize
Element::pGenerated to its corresponding element in theGenerated.myGeneratedMap
Not only will this let me save half of the lookup time, I may even change
myMap to a better container type for my keytype (say a hash_map or even a boost
multi index map)
At first this sounded to me like a bad idea. With std::vector and std::dqueue I can
see how this can be a problem as the values will be moved around,
invalidating the pointers. Given that std::map is implemented with a tree
structure, is there really a time where a map element will be relocated?
(my above assumptions were confirmed by the discussion in enter link description here)
While I probably won't provide an access method to each member of myElement or any syntactic sugar (like overloading [] () etc), this lets me treat these elements almost a consistent manner. The only key is that (aside for insertion) I never look for members of mymap directly.

Have you considered just using simple containership?
You're using C++, so you can just wrap the struct(s) in some class or other struct, and provide wrapper methods to do whatever you want.

Related

What's the best way to store a maya api object inside an std::map that will never become stale? c++

I have an std::map that I'm using to cache some objects info and I will be using that later, I'd like to have my object as a key so it takes as little time as possible to access it.
I was thinking of converting the UUID to string but I realised you can actually end up with duplicated uuids if the object gets brought to the scene as a reference multiple times.
I've also tried with adding an MDagPath but it won't allow me to store that into an std::map. I imagine I'd have to make it hashable but I can't think of a way to do that safely. Using a name I also think it's a big nono since it can be renamed.
Thanks for the help. I hope I was clear enough with my problem.
std::map isn't a hash map (you'd want unordered_map for that). As for adding an MDagPath to a std::map, you could do it via something along these lines:
// I don't think this is defined for dagpath?
// So you should be able to define it for MDagPath.
static inline bool operator < (const MDagPath& a, const MDagPath& b) {
return a.fullPathName() < b.fullPathName();
}
std::map<MDagPath, MyObjectInfo> myMap;
Alternatively, if you want to generate a lookup based on the object, then you can use MObjectHandle. It may still go stale, but at least it has the isAlive() and isValid() methods to tell you when it is stale.
You could insert that into an unordered_map
namespace std {
// override the std::hash for the MObjectHandle
template <>
struct hash< MObjectHandle >
{
std::size_t operator()(const MObjectHandle& k) const
{
return k.hashCode();
}
};
}
std::unordered_map<MObjectHandle, MyObjectInfo> myMap;
(code is untested - I don't have access to Maya at the moment)

Using different array-types interchangeably

I'm sure this question has been answered somewhere, but I'm not sure, what to even search for:
Say I have a class that stores some data in a std::vector, but I want to be able to use two differenct types interchangeably (for example to save memory) depending on the "mode" I set the class to. Something like this:
class MyClass
{
int Mode; // Could be, say, 0 or 1
std::vector<float> MyData; // Use this when Mode is 0
std::vector<short> MyData; // Use this when Mode is 1
void DoStuffWithData()
{
// ...
}
}
I know, this could probably be done with template, but I have not yet managed to wrap my head around this (if anybody could give me an example, please do so). Of course I could work with overloaded functions and use a lot of if-statements to switch between the cases or use a void-pointer that is set to one of the vectors and then cast to its type, but I think, this would make pretty messy code.
What would be a nice and clean way to use (mostly) the same code by just referring to MyData?
Edit
Mode would change during runtime. Basically the class would record and work with data it collects in the background. I want to be able to just change Mode for a new "recording" (of which I might start multiple during runtime). This would then set both vectors to size zero and use the one selected. Again - I know, I could overload functions and just use some if-statements, but I would like to keep the code short and clean (and maybe learn some things along the way).
You can't have two members called MyData. You can use templates instead, as that's essentially what they're meant for. This would be a simple example:
template<class T>
class MyClass {
std::vector<T> MyData;
void DoStuffWithData()
{
// ...
}
};
int main() {
MyClass<short> shortData;
MyClass<float> floatData;
MyClass<long> longData; // you're no longer limited to just two types
// ...
return 0;
}
Notice there's no Mode anymore, as you'll just choose the desired type when declaring each variable.

Data Structure that stores object with names

I want to store object that are given a certain name.
I wanted to use struct and then store them in a vector, but it was suggested to me that I should rather use a different data structure, a little more simple, but I cant seem to find one.
My current ("complex") solution:
//in header file
struct objStorage{
Classname obj;
string name;
};
vector<objStorage> vec;
//in constructor
objStorage firstObj;
firstObj.obj = new Classname();
firstObj.name = "foo";
vec.push_back(firstObj);
Is there a more simple solution (Data structure)?
I should add that I don't need the structure once I stored (copied?) it in the vector, because this is all happening in another class (constructor) so I don't want any problems when calling the constructor multiple times.
If you want to lookup items by some key, for example a string, the classic thing to use is a map:
std::map<std::string, Classname> items;
std::pair<std::map<std::string, Classname>::iterator, bool> inserted =
items.insert(std::make_pair(std::string("foo"), Classname()));
items["bar"] = Classname();
In this set up, if you really think you want to use pointers, you should consider some form of smart pointer.
There are other options, for example, C++11 introduces other lookup structures - e.g. unordered maps.

Dynamically storing an internal configuration

I've been thinking of The Right Way (R) to store my program's internal configuration.
Here's the details:
The configuration is runtime only, so generated each run.
It can be adapted (and should) through directives in a "project" file (the reading of that file is not in the scope of this question)
It needs to be extensible, ie there should be a way to add new "variables" with assignes values.
My questions about this:
How should I begin with this? Is a
class with accessors and setters
with an internal std::map for
custom variables a good option?
Are there any known and "good" ways
of doing this?
Should there be a difference between
integer, boolean and string
configuration variables?
Should there be a difference at all
between user and built-in
(pre-existing as in I already
thought of them) variables?
Thanks!
PS: If the question isn't clear, feel free to ask for more info.
UPDATE: Wow, every answer seems to have implicitely or explicitly used boost. I should have mentioned I'd like to avoid boost (I want to explore the Standard libraries' capabilities as is for now).
You could use Boost.PropertyTree for this.
Property trees are versatile data
structures, but are particularly
suited for holding configuration data.
The tree provides its own,
tree-specific interface, and each node
is also an STL-compatible Sequence for
its child nodes.
You could do worse than some kind of a property map (StringMap is just a typedef'd std::map)
class PropertyMap
{
private:
StringMap m_Map;
public:
PropertyMap() { };
~PropertyMap() { };
// properties
template<class T>
T get(const String& _key, const T& _default = T()) const
{
StringMap_cit cit(m_Map.find(_key));
return (cit != m_Map.end()) ? boost::lexical_cast<T>(cit->second) : _default;
}; // eo get
// methods
void set(const String& _cap, const String& _value)
{
m_Map[_cap] = _value;
}; // eo set
template<class T>
void set(const String& _key, const T& _val)
{
set(_key, boost::lexical_cast<String>(_val));
}; // eo set
};
It is very useful to support nesting in configuration files. Something like JSON.
As parameter values can be scalars, arrays and nested groups of parameters, it could be stored in a std::map of boost::variant's, whose value can be a scalar, array or other std::map recursively. Note that std::map sorts by name, so if the original config file order of parameters is important there should be a sequential index of parameters as well. This can be achieved by using boost::multi_index with an ordered or hashed index for fast lookup and a sequential index for traversing the parameters in the original config file order.
I haven't checked, that boost property map could do that from what I've heard.
It is possible to store all values as strings (or arrays of strings for array values) converting them to the destination type only when accessing it.

Is there a way to apply an action to N C++ class members in a loop over member names (probably via pre-processor)?

The problem:
I have a C++ class with gajillion (>100) members that behave nearly identically:
same type
in a function, each member has the same exact code done to it as other members, e.g. assignment from a map in a constructor where map key is same as member key
This identicality of behavior is repeated across many-many functions (>20), of course the behavior in each function is different so there's no way to factor things out.
The list of members is very fluid, with constant additions and sometimes deletions, some (but not all) driven by changing columns in a DB table.
As you can imagine, this presents a big pain-in-the-behind as far as code creation and maintenance, since to add a new member you have to add code to every function
where analogous members are used.
Example of a solution I'd like
Actual C++ code I need (say, in constructor):
MyClass::MyClass(SomeMap & map) { // construct an object from a map
intMember1 = map["intMember1"];
intMember2 = map["intMember2"];
... // Up to
intMemberN = map["intMemberN"];
}
C++ code I want to be able to write:
MyClass::MyClass(SomeMap & map) { // construct an object from a map
#FOR_EACH_WORD Label ("intMember1", "intMember2", ... "intMemberN")
$Label = map["$Label"];
#END_FOR_EACH_WORD
}
Requirements
The solution must be compatible with GCC (with Nmake as make system, if that matters).
Don't care about other compilers.
The solution can be on a pre-processor level, or something compilable. I'm fine with either one; but so far, all of my research pointed me to the conclusion that the latter is just plain out impossible in C++ (I so miss Perl now that I'm forced to do C++ !)
The solution must be to at least some extent "industry standard" (e.g. Boost is great, but a custom Perl script that Joe-Quick-Fingers created once and posted on his blog is not. Heck, I can easily write that Perl script, being much more of a Perl expert than a C++ one - I just can't get bigwigs in Software Engineering at my BigCompany to buy into using it :) )
The solution should allow me to declare a list of IDs (ideally, in only one header file instead of in every "#FOR_EACH_WORD" directive as I did in the example above)
The solution must not be limited to "create an object from a DB table" constructor. There are many functions, most of them not constructors, that need this.
A solution of "Make them all values in a single vector, and then run a 'for' loop across the vector" is an obvious one, and can not be used - the code's in a library used by many apps, the members are public, and re-writing those apps to use vector members instead of named members is out of the question, sadly.
Boost includes a great preprocessor library that you can use to generate such code:
#include <boost/preprocessor/repetition.hpp>
#include <boost/preprocessor/stringize.hpp>
#include <boost/preprocessor/cat.hpp>
typedef std::map<std::string, int> SomeMap;
class MyClass
{
public:
int intMember1, intMember2, intMember3;
MyClass(SomeMap & map)
{
#define ASSIGN(z,n,_) BOOST_PP_CAT(intMember, n) = map[ BOOST_PP_STRINGIZE(BOOST_PP_CAT(intMember, n))];
BOOST_PP_REPEAT_FROM_TO(1, 4, ASSIGN, nil)
}
};
Boost.Preprocessor proposes many convenient macros to perform such operations. Bojan Resnik already provided a solution using this library, but it assumes that every member name is constructed the same way.
Since you explicitely required the possibily to declare a list of IDs, here is a solution that should better fulfill your needs.
#include <boost/preprocessor/seq/for_each.hpp>
#include <boost/preprocessor/stringize.hpp>
// sequence of member names (can be declared in a separate header file)
#define MEMBERS (foo)(bar)
// macro for the map example
#define GET_FROM_MAP(r, map, member) member = map[BOOST_PP_STRINGIZE(member)];
BOOST_PP_SEQ_FOR_EACH(GET_FROM_MAP, mymap, MEMBERS)
// generates
// foo = mymap["foo"]; bar = mymap["bar];
-------
//Somewhere else, we need to print all the values on the standard output:
#define PRINT(r, ostream, member) ostream << member << std::endl;
BOOST_PP_SEQ_FOR_EACH(PRINT, std::cout, MEMBERS)
As you can see, you just need to write a macro representing the pattern you want to repeat, and pass it to the BOOST_PP_SEQ_FOR_EACH macro.
You could do something like this: create an adapter class or modify the existing class to have a vector of pointers to those fields, add the addresses of all member variables in question to that vector in the class constructor, then when needed run the for-loop on that vector. This way you don't (or almost don't) change the class for external users and have a nice for-loop capability.
Of course, the obvious question is: Why do you have a class with 100 members? It doesn't really seem sane.
Assuming it is sane nevertheless -- have you looked at boost preprocessor library? I have never used it myself (as one friend used to say: doing so leads to the dark side), but from what I heard it should be the tool for the job.
Surreptitiously use perl on your own machine to create the constructor. Then ask to increase your salary since you're succesfully maintaining such a huge chunk of code.
You could use the preprocessor to define the members, and later use the same definition to access them:
#define MEMBERS\
MEMBER( int, value )\
SEP MEMBER( double, value2 )\
SEP MEMBER( std::string, value3 )\
struct FluctuatingMembers {
#define SEP ;
#define MEMBER( type, name ) type name
MEMBERS
#undef MEMBER
#undef SEP
};
.. client code:
FluctuatingMembers f = { 1,2., "valuesofstringtype" };
std::cout <<
#define SEP <<
#define MEMBER( type, name ) #name << ":" << f.##name
MEMBERS;
#undef MEMBER
#undef SEP
It worked for me, but is hard to debug.
You can also implement a visitor pattern based on pointer-to-members. After the preprocessor solution, this one turns out way more debuggeable.
struct FluctuatingMembers {
int v1;
double v2;
std::string v3;
template<typename Visitor> static void each_member( Visitor& v );
};
template<typename Visitor> void FluctuatingMembers::each_member( Visitor& v ) {
v.accept( &FluctuatingMembers::v1 );
v.accept( &FluctuatingMembers::v2 );
v.accept( &FluctuatingMembers::v3 );
}
struct Printer {
FluctuatingMembers& f;
template< typename pt_member > void accept( pt_member m ) const {
std::cout << (f::*m) << "\n";
}
};
// you can even use this approach for visiting
// multiple objects simultaneously
struct MemberComparer {
FluctuatingMembers& f1, &f2;
bool different;
MemberComparer( FluctuatingMembers& f1, FluctuatingMembers& f2 )
: f1(f1),f2(f2)
,different(false)
{}
template< typename pt_member > void accept( pt_member m ) {
if( (f1::*m) != (f2::*m) ) different = true;
}
};
... client code:
FluctuatingMembers object1 = { 1, 2.2, "value2" }
, object2 = { 1, 2.2, "valuetoo" };
Comparer compare( object1, object2 );
FluctuatingMembers::each_member( compare );
Printer pr = { object1 };
FluctuatingMembers::each_member( pr );
Why not do it at run time? (I really hate macro hackery)
What you really are asking for, in some sense, is class metadata.
So I would try something like:
class AMember{
......
};
class YourClass{
AMember member1;
AMember member2;
....
AMember memberN;
typedef AMember YourClass::* pMember_t;
struct MetaData : public std::vector<std::pair<std::string,pMember_t>>{
MetaData(){
push_back(std::make_pair(std::string("member1"),&YourClass::member1));
...
push_back(std::make_pair(std::string("memberN"),&YourClass::memberN));
}
};
static const MetaData& myMetaData() {
static const MetaData m;//initialized once
return m;
}
YourClass(const std::map<std::string,AMember>& m){
const MetaData& md = myMetaData();
for(MetaData::const_iterator i = md.begin();i!= md.end();++i){
this->*(i->second) = m[i->first];
}
}
YourClass(const std::vector<std::pair<std::string,pMember_t>>& m){
const MetaData& md = myMetaData();
for(MetaData::const_iterator i = md.begin();i!= md.end();++i){
this->*(i->second) = m[i->first];
}
}
};
(pretty sure I've got the syntax right but this is a machinery post not a code post)
RE:
in a function, each member has the same exact code done to it as other members, e.g. assignment from a map in a constructor where map key is same as member key
this is handled above.
RE:
The list of members is very fluid, with constant additions and sometimes deletions, some (but not all) driven by changing columns in a DB table.
When you add a new AMember, say newMember, all you have to do is update the MetaData constructor with an:
push_back(make_pair(std::string("newMember"),&YourClass::newMember));
RE:
This identicality of behavior is repeated across many-many functions (>20), of course the behavior in each function is different so there's no way to factor things out.
You have the machinery to apply this same idiom to build the functions
eg: setAllValuesTo(const AMember& value)
YourClass::setAllValuesTo(const AMember& value){
const MetaData& md = myMetaData();
for(MetaData::const_iterator i = md.begin();i!= md.end();++i){
this->*(i->second) = value;
}
}
If you are a tiny bit creative with function pointers or template functionals you can factor out the mutating operation and do just about anything you want to YourClass' AMember's on a collection basis. Wrap these general functions (that may take a functional or function pointer) to implement your current set of 20 public methods in the interface.
If you need more metadata just augment the codomain of the MetaData map beyond a pointer to member. (Of course the i->second above would change then)
Hope this helps.
You can do something like his:
#define DOTHAT(m) m = map[#m]
DOTHAT(member1); DOTHAT(member2);
#undef DOTHAT
That doesn't fully fit your description, but closest to it that saves you typing.
Probably what I'd look to do would be to make use of runtime polymorphism (dynamic dispatch). Make a parent class for those members with a method that does the common stuff. The members derive their class from that parent class. The ones that need a different implementation of the method implement their own. If they need the common stuff done too, then inside the method they can downcast to the base class and call its version of the method.
Then all you have to do inside your original class is call the member for each method.
I would recommend a small command-line app, written in whatever language you or your team are most proficient in.
Add some kind of template language to your source files. For something like this, you don't need to implement a full-fledged parser or anything fancy like that. Just look for an easily-identified character at the beginning of a line, and some keywords to replace.
Use the command-line app to convert the templated source files into real source files. In most build systems, this should be pretty easy to do automatically by adding a build phase, or simply telling the build system: "use MyParser.exe to handle files of type *.tmp"
Here's an example of what I'm talking about:
MyClass.tmp
MyClass::MyClass(SomeMap & map) { // construct an object from a map
▐REPLACE_EACH, LABEL, "intMember1", "intMember2, ... , "intMemberN"
▐ LABEL = map["$Label"];
}
I've used "▐" as an example, but any character that would otherwise never appear as the first character on a line is perfectly acceptable.
Now, you would treat these .tmp files as your source files, and have the actual C++ code generated automatically.
If you've ever heard the phrase "write code that writes code", this is what it means :)
There are already a lot of good answers and ideas here, but for the sake of diversity I'll present another.
In the code file for MyClass would be:
struct MemberData
{
size_t Offset;
const char* ID;
};
static const MemberData MyClassMembers[] =
{
{ offsetof(MyClass, Member1), "Member1" },
{ offsetof(MyClass, Member2), "Member2" },
{ offsetof(MyClass, Member3), "Member3" },
};
size_t GetMemberCount(void)
{
return sizeof(MyClassMembers)/sizeof(MyClassMembers[0]);
}
const char* GetMemberID(size_t i)
{
return MyClassMembers[i].ID;
}
int* GetMemberPtr(MyClass* p, size_t i) const
{
return (int*)(((char*)p) + MyClassMembers[i].Offset);
}
Which then makes it possible to write the desired constructor as:
MyClass::MyClass(SomeMap& Map)
{
for(size_t i=0; i<GetMemberCount(); ++i)
{
*GetMemberPtr(i) = Map[GetMemberID(i)];
}
}
And of course, for any other functions operating on all the members you would write similar loops.
Now there are a few issues with this technique:
Operations on members use a runtime loop as opposed to other solutions which would yield an unrolled sequence of operations.
This absolutely depends on each member having the same type. While that was allowed by OP, one should still evaluate whether or not that might change in the future. Some of the other solutions don't have this restriction.
If I remember correctly, offsetof is only defined to work on POD types by the C++ standard. In practice, I've never seen it fail. However I haven't used all the C++ compilers out there. In particular, I've never used GCC. So you would need to test this in your environment to ensure it actually works as intended.
Whether or not any of these are problems is something you'll have to evaluate against your own situation.
Now, assuming this technique is usable, there is one nice advantage. Those GetMemberX functions can be turned into public static/member functions of your class, thus providing this generic member access to more places in your code.
class MyClass
{
public:
MyClass(SomeMap& Map);
int Member1;
int Member2;
int Member3;
static size_t GetMemberCount(void);
static const char* GetMemberID(size_t i);
int* GetMemberPtr(size_t i) const;
};
And if useful, you could also add a GetMemberPtrByID function to search for a given string ID and return a pointer to the corresponding member.
One disadvantage with this idea so far is that there is a risk that a member could be added to the class but not to the MyClassMembers array. However, this technique could be combined with xtofl's macro solution so that a single list could populate both the class and the array.
changes in the header:
#define MEMBERS\
MEMBER( Member1 )\
SEP MEMBER( Member2 )\
SEP MEMBER( Member3 )\
class MyClass
{
public:
#define SEP ;
#define MEMBER( name ) int name
MEMBERS;
#undef MEMBER
#undef SEP
// other stuff, member functions, etc
};
and changes in the code file:
const MemberData MyClassMembers[] =
{
#define SEP ,
#define MEMBER( name ) { offsetof(MyClass, name), #name }
MEMBERS
#undef MEMBER
#undef SEP
};
Note: I have left error checking out of my examples here. Depending on how this would be used, you might want to ensure the array bounds are not overrun with debug mode asserts and/or release mode checks that would return NULL pointers for bad indexes. Or some use of exceptions if appropriate.
Of course, if you aren't worried about error checking the array bounds, then GetMemberPtr could actually be changed into something else that would return a reference to the member.