I have a collection of objects, some of which reference others. The referencing is implemented using pointers, however pointers do not persist between application runs - the data structures are serialized and deserialized, but each time their memory addresses of the actual objects are different.
I have implemented a hash table, using integer ids as keys and pointers as values, storing the ids and using them to lookup the object pointer for each id. Sometimes references to objects are created before the actual referenced object is created, so in such cases I have to implement a check for this id, and inform the objects, referencing that id the moment an object with such id is created.
It works, but I feel like it is clumsy and inefficient. So my question is whether there is established design pattern to deal with this problem.
Related
I'm developing a game server for a video game called Tibia.
Basically, there can be up to millions of objects, of which there can be up to thousands of deletes and re-creations as players interact with the game world.
The thing is, the original creators used a Slot Map / Object Pool on which pointers are re-used when an object is removed. This is a huge performance boost since there's no need to do much memory reallocation unless needed.
And of course, I'm trying to accomplish that myself, but I've come into one huge problem with my Slot Map:
Here's just a few explanation of how Slot Map works according to a source I found online:
Object class is the base class for every game object, my Slot Map / object Pool is using this Object class to save every allocated object.
Example:
struct TObjectBlock
{
Object Object[36768];
};
The way the slot map works is that, the server first allocates, say, 36768 objects in a list of TObjectBlock and gives them a unique ID ObjectID for each Object which can be re-used in a free object list when the server needs to create a new object.
Example:
Object 1 (ID: 555) is deleted, it's ID 555 is put in a free object ID
list, an Item creation is requested, ID 555 is reused since it's on
the free object list, and there is no need to reallocate another
TObjectBlock in the array for further objects.
My problem: How can I use "Player" "Creature" "Item" "Tile" to support this Slot Map? I don't seem to come up with a solution into this logic problem.
I am using a virtual class to manage all objects:
struct Object
{
uint32_t ObjectID;
int32_t posx;
int32_t posy;
int32_t posz;
};
Then, I'd create the objects themselves:
struct Creature : Object
{
char Name[31];
};
struct Player : Creature
{
};
struct Item : Object
{
uint16_t Attack;
};
struct Tile : Object
{
};
But now if I was to make use of the slot map, I'd have to do something like this:
Object allocatedObject;
allocatedObject.ObjectID = CreateObject(); // Get a free object ID to use
if (allocatedObject.ObjectID != INVALIDOBJECT.ObjectID)
{
Creature* monster = new Creature();
// This doesn't make much sense, since I'd have this creature pointer floating around!
monster.ObjectID = allocatedObject.ObjectID;
}
It pretty much doesn't make much sense to set a whole new object pointer the already allocated object unique ID.
What are my options with this logic?
I believe you have a lot of tangled concepts here, and you need to detangle them to make this work.
First, you are actually defeating the primary purpose of this model. What you showed smells badly of cargo cult programming. You should not be newing objects, at least without overloading, if you are serious about this. You should allocate a single large block of memory for a given object type and draw from that on "allocation" - be it from an overloaded new or creation via a memory manager class. That means you need separate blocks of memory for each object type, not a single "objects" block.
The whole idea is that if you want to avoid allocation-deallocation of actual memory, you need to reuse the memory. To construct an object, you need enough memory to fit it, and your types are not the same length. Only Tile in your example is the same size as Object, so only that could share the same memory (but it shouldn't). None of the other types can be placed in the objects memory because they are longer. You need separate pools for each type.
Second, there should be no bearing of the object ID on how things are stored. There cannot be, once you take the first point into consideration, if the IDs are shared and the memory is not. But it must be pointed out explicitly - the position in a memory block is largely arbitrary and the IDs are not.
Why? Let's say you take object 40, "delete" it, then create a new object 40. Now let's say some buggy part of the program referenced the original ID 40. It goes looking for the original 40, which should error, but instead finds the new 40. You just created an entirely untrackable error. While this can happen with pointers, it is far more likely to happen with IDs, because few systems impose checks on ID usage. A main reason for indirecting access with IDs is to make access safer by making it easy to catch bad usage, so by making IDs reusable, you make them just as unsafe as storing pointers.
The actual model for handling this should look like how the operating system does similar operations (see below the divide for more on that...). That is to say, follow a model like this:
Create some sort of array (like a vector) of the type you want to store - the actual type, not pointers to it. Not Object, which is a generic base, but something like Player.
Size that to the size you expect to need.
Create a stack of size_t (for indexes) and push into it every index in the array. If you created 10 objects, you push 0 1 2 3 4 5 6 7 8 9.
Every time you need an object, pop an index from the stack and use the memory in that cell of the array.
If you run out of indexes, increase the size of the vector and push the newly created indexes.
When you use objects, indirect via the index that was popped.
Essentially, you need a class to manage the memory.
An alternative model would be to directly push pointers into a stack with matching pointer type. There are benefits to that, but it is also harder to debug. The primary benefit to that system is that it can easily be integrated into existing systems; however, most compilers do similar already...
That said, I suggest against this. It seems like a good idea on paper, and on very limited systems it is, but modern operating systems are not "limited systems" by that definition. Virtual memory already resolves the biggest reason to do this, memory fragmentation (which you did not mention). Many compiler allocators will attempt to more or less do what you are trying to do here in the standard library containers by drawing from memory pools, and those are far more manageable to use.
I once implemented a system just like this, but for many good reasons have ditched it in favor of a collection of unordered maps of pointers. I have plans to replace allocators if I discover performance or memory problems associated with this model. This lets me offset the concern of managing memory until testing/optimization, and doesn't require quirky system design at every level to handle abstraction.
When I say "quirky", believe me when I say that there are many more annoyances with the indirection-pool-stack design than I have listed.
I needed a program for a school project that allowed for the following functionality:
-dynamic object creation thus used c++ keyword:
new
-display a sorted a bunch of objects based on a different member variables.
-quick access to dynamically created objects.
I order to do this I used a map that stored a string key and a pointer to an object. This allowed for quick access and lookup of individual objects by key.
To solve the on-demand sort functionality required, I iterated through the map and added the second element--which was the object pointer--to a vector. I then sorted the vector using a lambda function based on the sorted-ness the user specified.
First, my actual question
I have a shared pointer in a map, I want to remove that shared pointer from the map when there are no other references besides the shared pointer in the map itself, so the target gets released. How can I do this? Alternatively, what's a better design to solve my problem?
Background below:
Out of nostalgia I've been writing a MUD engine in C++ using a MySQL backend. I'm at the point now of persisting entities (things like accounts, characters and so forth in this context). I'm having a bit of trouble figuring out the best way to implement this and decided to seek some advice.
My current persistence architecture looks like this, taking character for an example
Character entity -> Character storage -> Character persistence -> MySQL
Character entity refers to the character itself and is analogous to a model in MVC. It doesn't have anything but data and only knows about other entities.
Character storage is a container currently responsible for holding instances of the character entity in memory. When a request is made to find a character either by ID or name, it checks it's memory, implemented as a map of std::string containing UUIDs to a std::shared_ptr. If we have an instance already in memory, we pass back a shared pointer, if not, we ask the persistence layer for it, store it in the map and then pass a shared pointer back to it.
The persistence layer is abstract, there's a character_persistence interface which has a concrete implementation of mysql_character_persistence, so I could easily switch to different forms of persistence.
I used reference counting and a templated entity_ptr type.
I created a entity_ptr_provider interface which defined the methods obtain(uuid) and release(uuid). An entity_ptr takes a templated argument of entity type (account, character etc), an entity of that type and an entity_ptr_provider.
On construction or copy of the entity_ptr type, it calls obtain on the entity_ptr_provider with the UUID of the entity so it can increment the references to it, when it's deconstructed, it calls release which allows the entity_ptr_provider to decrement the reference. When references reach 0, it's released from the table.
I'm designing a software tool in which there's an in-memory model, and the API user can get objects of the model, query them and set values.
Since all the model's objects belong to a single model, and most operations must be recorded and tested, etc., each created object must be registered to the Model object. The Model stores all objects as std::unique_ptr since it's the only owner of them. When needed, it passes raw pointers to users.
What makes me worry is the possibility that the user calls delete on these pointers. But if I use std::shared_ptr, the user can still use get() and call delete on that. So it's not much safer.
Another option I though of is to refer to objects by a name string, or pass ObjectReference objects instead of the real objects, and then these ObjectReferences can be destroyed without affecting the actual stored object.
These References work somewhat like a client: You tell them what to do, and they forward the request to the actual object. It's a lot of extra work for the developer, but it protectes the pointers.
Should I be worried about the pointers? Until now I was using smart pointers all the time, but now I need to somehow allow the user to access objects managed by a central model, without allowing the user to delete them.
[Hmmm... maybe make the destructor private, and let only the unique_ptr have access to it through a Deleter?]
You shouldn't bother about users calling delete on your objects. It's one of those things that are perfectly fine as a documented constraint, any programmer violating that only deserves whatever problem he runs into.
If you still really want to explicitly forbid this, you could either write a lightweight facade object that your users will pass by value (but it can be lot of work depending on the number of classes you have to wrap) or, as you said, make their destructor private and have unique_ptr use a friend deleter.
I for one am not fond of working through identifiers only, this can quickly lead to performance issues because of the lookup times (even if you're using a map underneath).
Edit: Now that I think of it, there is a way in between identifiers and raw pointers/references: opaque references.
From the point of view of the users, it acts like an identifier, all they can do is copy/move/assign it or pass it to your model.
Internally, it's just a class with a private pointer to your objects. Your model being a friend of this class, it can create new instances of the opaque reference from a raw pointer (which a user can't do), and use the raw pointer to access the object without any performance loss.
Something along the lines of:
class OpaqueRef
{
// default copy/move/assignment/destructor
private:
friend class Model;
Object* m_obj;
OpaqueRef(Object& obj) : m_obj(&obj) {}
};
Still, not sure if it's worth the trouble (I stand by my first paragraph), but at least you got one more option.
Personally, I'd keep the internal pointer in the model without exposing it and provide an interface via model ids, so all operations go through the interface.
So, you could create a separate interface class that allows modification of model attributes via id. External objects would only request and store the id of the object they want to change.
I am attempting to construct a bunch of objects that can share data, but aren't coupled to each other. What I want is for the fields of each object to be a shared_ptr to whatever data they need. Once all the objects are created, this will fulfill all my needs, but my issue is how to distribute the information needed to create the shared_ptr's, since the types can be anything.
I will be creating all objects that share a set of data at once inside a factory object. Also, each shared field will have an enum entry to signify the attribute type (not data type). For instance the enum will have an entry that is Position, and every object that needs position information will use enum to key into a map to find the information it needs to create the shared_ptr.
What I want from my system is this:
Pass an SetEnum to the factory, which defines which "set" of objects to create, along with an optional map<AttributeEnum, ??> that defines the initial data of set of objects
Create the necessary objects, using a map<AttributeEnum, ??> to create shared_ptr's as fields in each of the objects. If the object needs a field corresponding to a specific value of AttributeEnum, it will pull its value from the map. If that value is not a key in the map, it will create it with a default value, and add it to the map for future objects to use. This step can be done during or after construction if a method with a template type would be helpful to solve the issue.
The issue is that I want errors at compile time, not run time. How can I map AttributeEnum to a type? For example, every field corresponding to an AttributeEnum value of Position will have type shared_ptr<double>, but I want a compile error if I attempt to associate Position with, for instance, shared_ptr<int>. My question is how to go about this?
A couple different systems I have thought of (albeit none of them ensure the errors at compile time):
Pass around a std::map<Enum, shared_ptr<void> > and static cast the shared_ptr's to the appropriate types.
Construct the objects, then iterate through the Enum, checking each object for which attributes it needs, then passing the proper pointers to it.
Having one object "own" each attribute, and force the other to get the information via a message-passing system.
I am considering storing two parallel sets of data, the AttributeEnum having entry Position, and compiler constants that would define the types such as #define POSITION double, then I would simply use POSITION for the type where ever I must, but this would make the code much harder to read.