While doing a game engine that uses .lua files in order to read parameter values, I got stuck when I had to read these values and assign them to the parameters of each component in C++. I tried to investigate the way Unity does it, but I didn't find it (and I'm starting to doubt that Unity has to do it at all).
I want the parameters to be initialized automatically, without the user having to do the process of
myComponentParameter = readFromLuaFile("myParameterName")
for each one of the parameters.
My initial idea is to use the std::variant type, and storing an array of variants in order to read them automatically. My problems with this are:
First of all, I don't know how to know the type that std::variant is storing at the moment (tried with std::variant::type, but it didn't work for the template), in order to cast from the untyped .lua value to the C++ value. For reference, my component initialization looks like this:
bool init(luabridge::LuaRef parameterTable)
{
myIntParameter = readVariable<int>(parameterTable, "myIntParameter");
myStringParameter = readVariable<std::string>(parameterTable, "myStringParameter");
return true;
}
(readVariable function is already written in this question, in case you're curious)
The second problem is that the user would have to write std::get(myIntParameter); whenever they want to access to the value stored by the variant, and that sounds like something worse than making the user read the parameter value.
The third problem is that I can't create an array of std::variant<any type>, which is what I would like to do in order to automatically initialize the parameters.
Is there any good solution for this kind of situation where I want the init function to not be necessary, and the user doesn't need to manually set up the parameter values?
Thanks in advance.
Let's expand my comment. In a nutshell, you need to get from
"I have some things entered by the user in some file"
to:
"the client code can read the value without std::get"
…which roughly translates to:
"input validation was done, and values are ready for direct use."
…which implies you do not store your variables in variants.
In the end it is a design question. One module somewhere must have the knowledge of which variable names exist, and the type of each, and the valid values.
The input of that module will be unverified values.
The output of the module will probably be some regular c++ struct.
And the body of that module will likely have a bunch of those:
config.foo = readVariable<int>("foo");
config.bar = readVariable<std::string>("bar");
// you also want to validate values there - all ints may not be valid values for foo,
// maybe bar must follow some specific rules, etc
assuming somewhere else it was defined as:
struct Configuration {
int fooVariable;
std::string bar;
};
Where that module lives depends on your application. If all expected types are known, there is no reason to ever use a variant, just parse right away.
You would read to variants if some things do not make sense until later. For instance if you want to read configuration values that will be used by plugins, so you cannot make sense of them yet.
(actually even then simply re-parsing the file later, or just saving values as text for later parsing would work)
It's my first time working with Django's JSON fields. When going through examples I usually see the field implementations that set the default to an empty object.
However, for my purposes, I'd prefer to allow the field to be None as well. Since all the examples set a default, I'm not sure if allowing null will yield any negative consequences I'm unaware of currently.
I want to set limits on value specified in command line. And it would be great if this range will be automatically printed within description like default value.
Also is is interesting if there is a way to create my own type and do specialization of po::value for my enum type.
For example
enum percent
{
percent0,
percent10,
percent20,
percent30,
percent40,
...
percent100
};
po::value<percent>() <-- gives compile time error
update
I would like to add information about allowed values to the value_semantic object, like it is done for the default value or for required flag.
I noticed that typed_value class is polymorphic and can be extended, so I want to create a new class
class range_int : public po::typed_value<int> {}
and later when iterating description options I can do dynamic_cast<> and check if value is range_int, but I don't like this solution because of cast is required for it, but there is no way to add virtual functions to the base interface.
The purpose of this is to extend program options. I need to write another description output function which will output program options in format that is easy to parse (for example XML). I also need to add range to some of the values, so I need this information be stored in program options.
Any suggestions?
You can try to use custom validator with a custom type to hold your information.
The example provided is self explanatory
I have a TCP client-server setup where I need to be able to pass messages of different formats at different times, using the same transmit/receive infrastructure.
Two different types of messages sent from client to server might be:
TIME_SYNC_REQUEST: Requesting server's game time. Contains no information other than the message type.
UPDATE: Describes all changes to game state that happened since the last update that was posted (if this is not the very first one after connecting), so that the server may update its data model where it sees fit.
(The message type to be included in the header, and any data to be included in the body of the message.)
In dynamic languages, I'd create an AbstractMessage type, and derive two different message types from it, with TimeSyncRequestMessage accommodating no extra data members, and UpdateMessage containing all necessary members (player position etc.), and use reflection to see what I need to actually serialise for socket send(). Since the class name describes the type, I would not even need an additional member for that.
In C++: I do not wish to use dynamic_cast to mirror the approach described above, for performance reasons. Should I use a compositional approach, with dummy members filling in for any possible data, and a char messageType? I guess another possibility is to keep different message types in differently-typed lists. Is this the only choice? Otherwise, what else could I do to store the message info until it's time to serialise it?
Maybe you can let the message class to do the serialization - Define a serialize interface, and each message implements this interface. So at the time you want to serialize and send, you call AbstractMessage::Serialize() to get the serialized data.
Unless you have some very high performance characteristics, I would use a self describing message format. This typically use a common format (say key=value), but no specific structure, instead known attributes would describe the type of the message, and then any other attributes can be extracted from that message using logic specific to that message type.
I find this type of messaging retains better backward compatibility - so if you have new attributes you want to add, you can add away and older clients will simply not see them. Messaging that uses fixed structures tend to fare less well.
EDIT: More information on self describing message formats. Basically the idea here is that you define a dictionary of fields - this is the universe of fields that your generic message contains. Now a message be default must contain some mandatory fields, and then it's up to you what other fields are added to the message. The serialization/deserialization is pretty straightforward, you end up constructing a blob which has all the fields you want to add, and at the other end, you construct a container which has all the attributes (imagine a map). The mandatory fields can describe the type, for example you can have a field in your dictionary which is the message type, and this is set for all messages. You interrogate this field to determine how to handle this message. Once you are in the handling logic, you simply extract the other attributes the logic needs from the container (map) and process them.
This approach affords the best flexibility, allows you to do things like only transmit fields that have really changed. Now how you keep this state on either side is up to you - but given you have a one-to-one mapping between message and the handling logic - you need neither inheritance or composition. The smartness in this type of system stems from how you serialize the fields (and deserialize so that you know what attribute in the dictionary the field is). For an example of such a format look at the FIX protocol - now I wouldn't advocate this for gaming, but the idea should demonstrate what a self describing message is.
EDIT2: I cannot provide a full implementation, but here is a sketch.
Firstly let me define a value type - this is the typical type of values which can exist for a field:
typedef boost::variant<int32, int64, double, std::string> value_type;
Now I describe a field
struct field
{
int field_key;
value_type field_value;
};
Now here is my message container
struct Message
{
field type;
field size;
container<field> fields; // I use a generic "container", you can use whatever you want (map/vector etc. depending on how you want to handle repeating fields etc.)
};
Now let's say that I want to construct a message which is the TIME_SYNC update, use a factory to generate me an appropriate skeleton
boost::unique_ptr<Message> getTimeSyncMessage()
{
boost::unique_ptr<Message> msg(new Message);
msg->type = { dict::field_type, TIME_SYNC }; // set the type
// set other default attributes for this message type
return msg;
}
Now, I want to set more attributes, and this is where I need a dictionary of supported fields for example...
namespace dict
{
static const int field_type = 1; // message type field id
// fields that you want
static const int field_time = 2;
:
}
So now I can say,
boost::unique_ptr<Message> msg = getTimeSyncMessage();
msg->setField(field_time, some_value);
msg->setField(field_other, some_other_value);
: // etc.
Now the serialization of this message when you are ready to send is simply stepping through the container and adding to the blob. You can use ASCII encoding or binary encoding (I would start with former first and then move to latter - depending on requirements). So an ASCII encoded version of the above could be something like:
1=1|2=10:00:00.000|3=foo
Here for arguments sake, I use a | to separate the fields, you can use something else that you can guarantee doesn't occur in your values. With a binary format - this is not relevant, the size of each field can be embedded in the data.
The deserialization would step through the blob, extract each field appropriately (so by seperating by | for example), use the factory methods to generate a skeleton (once you've got the type - field 1), then fill in all the attributes in the container. Later when you want to get a specific attribute - you can do something like:
msg->getField(field_time); // this will return the variant - and you can use boost::get for the specific type.
I know this is only a sketch, but hopefully it conveys the idea behind a self describing format. Once you've got the basic idea, there are lots of optimizations that can be done - but that's a whole another thing...
A common approach is to simply have a header on all of your messages. for example, you might have a header structure that looks like this:
struct header
{
int msgid;
int len;
};
Then the stream would contain both the header and the message data. You could use the information in the header to read the correct amount of data from the stream and to determine which type it is.
How the rest of the data is encoded, and how the class structure is setup, greatly depends on your architecture. If you are using a private network where each host is the same and runs identical code, you can use a binary dump of a structure. Otherwise, the more likely case, you'll have a variable length data structure for each type, serialized perhaps using Google Protobuf, or Boost serialization.
In pseudo-code, the receiving end of a message looks like:
read_header( header );
switch( header.msgid )
{
case TIME_SYNC:
read_time_sync( ts );
process_time_sync( ts );
break;
case UPDATE:
read_update( up );
process_update( up );
break;
default:
emit error
skip header.len;
break;
}
What the "read" functions look like depends on your serialization. Google protobuf is pretty decent if you have basic data structures and need to work in a variety of languages. Boost serialization is good if you use only C++ and all code can share the same data structure headers.
A normal approach is to send the message type and then send the serialized data.
On the receiving side, you receive the message type and based on that type, you instantiate the class via a factory method (using a map or a switch-case), and then let the object deserialize the data.
Your performance requirements are strong enough to rule out dynamic_cast? I do not see how testing a field on a general structure can possibly be faster than that, so that leaves only the different lists for different messages: you have to know by some other means the type of your object on every case. But then you can have pointers to an abstract class and do a static cast over those pointers.
I recommend that you re-assess the usage of dynamic_cast, I do not think that it be deadly slow for network applications.
On the sending end of the connection, in order to construct our message, we keep the message ID and header separate from the message data:
Message is a type that holds only the messageCategory and messageID.
Each such Message is pushed onto a unified messageQueue.
Seperate hashes are kept for data pertaining to each of the messageCategorys. In these, there is a record of data for each message of that type, keyed by messageID. The value type depends on the message category, so for a TIME_SYNC message we'd have a struct TimeSyncMessageData, for instance.
Serialisation:
Pop the message from the messageQueue, reference the appropriate hash for that message type, by messageID, to retrieve the data we want to serialise & send.
Serialise & send the data.
Advantages:
No potentially unused data members in a single, generic Message object.
An intuitive setup for data retrieval when the time comes for serialisation.
Suppose I am recording data and want to associate some number of data elements, such that each recorded set always has a fixed composition, i.e. no missing fields.
Most of my experience as a programmer is with Ada or C/C++ variants. In Ada, I would use a record type and aggregate assignment, so that when the record type was updated with new fields, anyone using the record would be notified by the compiler. In C++, chances are I would use a storage class and constructor to do something similar.
What is the appropriate way to handle a similar situation in Python? Is this a case where classes are the proper answer, or is there a lighter weight analog to the Ada record?
An additional thought, both Ada records and C++ constructors allow for default initialization values. Is there a Python solution to the above question which provides that capability as well?
A namedtuple (from the collections library) may suit your purposes. It is basically a tuple which allows reference to fields by name as well as index position. So it's a fixed structure of ordered named fields. It's also lightweight in that it uses slots to define field names thus eliminating the need to carry a dictionary in every instance.
A typical use case is to define a point:
from collections import namedtuple
Point = namedtuple("Point", "x y")
p1 = Point(x=11, y=22)
It's main drawback is that, being a tuple, it is immutable. But there is a method, replace which allows you to replace one or more fields with new values, but a new instance is created in the process.
There is also a mutable version of namedtuple available at ActiveState Python Recipes 576555 called records which permits direct field changes. I've used it and can vouch that it works well.
A dictionary is the classical way to do this in Python. It can't enforce that a value must exist though, and doesn't do initial values.
config = {'maxusers': 20, 'port': 2345, 'quota': 20480000}
collections.namedtuple() is another option in versions of Python that support it.