I'm trying to share an instance of a class between two programs. This is a glorified producers consumers problem; however, for abstraction purposes, I have put a mutex in the class.
I've seen instances of sharing structs between processes, but this generally involved a fork. I want to keep the processes separate, because they will be doing two different things so half of the program\code segment will be wasted on each process.
It might be easier to show than try to explain.
class my_class
{
private:
sem_t mutex;
data_type *my_data; //just some linked list
fstream some_file;
public:
my_class();
data_type* retrieve();
void add(string add);
};
my_class::my_class()
{
my_data = new data_type();
sem_init(&mutex, 0, 1);
my_file.open("log", ios::out);
}
data_type* my_class::retrieve()
{
data_type *temp = NULL;
sem_wait(&mutex);
if(my_data -> next != NULL)
{
temp = my_data;
my_data = my_data -> next;
}
sem_post(&mutex);
return my_data;
}
void my_class::add(string data)
{
data_type *temp = new data_type();
temp -> data = data;
data_type *top;
sem_wait(&mutex);
top = my_data;
while(top -> next -> next) //adds it to the end. The end's next is set to NULL
{
top = top -> next;
}
top -> next = temp;
my_file << name << "\n";
sem_post(&mutex);
}
What I'm really looking for is a way to share an instance of this class as a pointer. This way, I can then have threads that can access this instance. I think because of how much sharing I want to do, it needs to go on the heap and not the stack.
I wouldn't consider making this its own program and then using networking i\o to interact because of how simple it is. Needless to say, this is not exactly what I'm doing; however I think I've made a simplified\generic enough example that if this can be solved then I can easily apply it to my solution and it might help others.
Again, I'm looking for a way to share one instance of this code between two separate processes.
I don't know if this can be done because the class has a linked list in it let alone a file in it. If it can, then whose heap and file table does it fill (both?).
EDIT:
Thanks for the help so far; however, it should be worth pointing out that both processes may not be running at the same time. One acts as a daemon and the other will appears intermittently. Both programs already have threads, so that's why I want to do it on the heap.
You cannot share memory that is on the heap between processes. Using mmap() with MAP_ANON | MAP_SHARED you can share whole pages, but not on the heap. Using shm_open etc. you can share other objects, but again not on the heap.
Perhaps what you want is threads. These will allow you to share items on the heap.
I think your understanding of fork() is garbled. fork() will result in a copy-on-write image of your program in memory. As your code won't be written to, if you don't exec(), it will be only use one copy of physical memory. If you exec() a different version of your program (e.g. if the producer exec()s the consumer), it's likely to be less memory efficient than having it all in one place and fork()ing. And in either case you are going to have the overhead of some sort of IPC. Threads here seem a far better solution.
Related
I believe this will be my first question for the site, so I apologize for any mistakes or errors in this post. I am a beginner C++ programmer as well, so forgive me if my questions come across as “noobish”.
Background: A collection of Parent Entity objects are created at startup (and currently not removed or added-to during runtime), and are then linked to a series of Activator Entity objects (both at the beginning, and during, runtime) through a Child Entity object. When establishing a link, the Parent generates a Child (which is stored in a local vector), and returns a pointer to the Child for the Activator to store.
Activators will “activate” children they are linked with, which will then do jobs based off internal and Parent settings. After being activated, they are also updated periodically by the Parent, continuing until eventually deactivating.
Below is a simplified example of the classes present.
class ParentEntity {
std::vector<ChildEntity> m_Children;
std::vector<ChildEntity*> m_ActiveChildren;
public:
//Funcs
ParentEntity(unsigned expectedChildren) { m_Children.reserve(expectedChildren); }
ChildEntity* AddChild(){
m_Children.push_back(ChildEntity(*this));
return &(m_Children.back());
}
void RemoveChild(unsigned iterator) {
//Can't figure a way to remove from the m_Children list without disrupting all pointers.
//m_Children.erase(m_Children.begin() + iterator); Uses Copy operators, which wont work as Const values will be present in Child
}
void AddActiveChild(ChildEntity* activeChild) {
m_ActiveChildren.push_back(activeChild);
}
bool Update(){ //Checks if Children are active,
if (!m_ActiveChildren.empty()) {
std::vector<ChildEntity*> TempActive;
TempActive.reserve(m_ActiveChildren.size());
for (unsigned i = 0; i < m_ActiveChildren.size(); i++) {
if (m_ActiveChildren[i]->Update()) {
TempActive.push_back(m_ActiveChildren[i]);
}
}
if (!TempActive.empty()) {
m_ActiveChildren = TempActive;
return true;
}
else {
m_ActiveChildren.clear();
return false;
}
}
else {
return false;
}
}
};
class ChildEntity {
public:
ChildEntity(ParentEntity& Origin) //Not const because it will call Origin functions that alter the parent
:
m_Origin(Origin)
{}
void SetActive() {
m_ChildActive = true;
m_Origin.AddActiveChild(this);
}
bool Update() { //Psuedo job which causes state switch
srand(unsigned(time(NULL)));
if ((rand() % 10 + 1) > 5) {
m_ChildActive = false;
}
return m_ChildActive;
}
private:
ParentEntity& m_Origin;
bool m_ChildActive = false;
};
class ActivatorEntity {
std::vector<ChildEntity*> ActivationTargets;
public:
ActivatorEntity(unsigned expectedTargets) { ActivationTargets.reserve(expectedTargets); }
void AddTarget(ParentEntity& Target) {
ActivationTargets.push_back(Target.AddChild());
}
void RemoveTarget(unsigned iterator) {
ActivationTargets.erase(ActivationTargets.begin() + iterator);
}
void Activate(){
for (unsigned i = 0; i < ActivationTargets.size(); i++) {
ActivationTargets[i]->SetActive();
}
}
};
With that all laid out, my three questions are:
Is there a way to update Pointers when a vector resizes?
When a Child is added, if it goes past the expected capacity, the vector creates a new array and moves the original objects to the new location. This breaks all of the Activator pointers, and any m_ActiveChild pointers, as they are pointing to the old location.
Is there a way to remove Child objects from the m_Children vector?
Since ChildEntity objects will host const items within them, copy assignment operations won’t work smoothly, and the Vector’s erase function won’t work. The m_Children vector could be rebuilt without the unwanted object through a temporary vector and copy constructor, but this leads to all of the pointers being wrong again.
Please let me know if there are any other suggested optimizations or corrections I should make!
Thank you all for your help!
Your problem, abstractly seen, is that on one hand you have collections of objects that you want to iterate through, kept in a container; and that on the other hand these objects are linked to each other. Re-ordering the container destroys the links.
Any problem can be solved by an additional indirection: Putting not the objects but object handles in the container would make re-ordering possible without affecting cross-references. The trivial case would be to simply use pointers; modern C++ would use smart pointers.
The disadvantage here is that you'll move to dynamic allocation which usually destroys locality right away (though potentially not if most allocations happen during initialization) and carries the usual run-time overhead. The latter may be prohibitive for simple, short-lived objects.
The advantage is that handling pointers enables you to make your objects polymorphic which is a good thing for "activators" and collections of "children" performing "updates": What you have here is the description of an interface which is typically implemented by various concrete classes. Putting objects in a container instead of pointers prevents such a design because all objects in a container must have the same concrete type.
If you need to store more information you can write your own handle class encapsulating a smart pointer; perhaps that's a good idea from the beginning because it is easily extensible without affecting all client code with only a moderate overhead (both in development and run time).
I have an algorithm - very performance sensitive that goes through a graph and make some decisions. As part of it I have to create a backtrace for the solution I find to be the best. At every step there are multiple choices and they are evaluated again and again until at the end there is only one. At every new step current backtraces could be made a subsolution for zero or more new possible solutions.
The complexity of the algorithm does not allow tricks so what I have so far is smart pointers for the nodes where the backtraces are dynamically hooked.
struct Node;
typedef boost::intrusive_ptr<Node> NodeSPtr;
struct Node
{
Node(NodePool& pool)
: mCount(0)
{
}
~Node()
{
}
size_t ref_count()
{
return mCount;
}
NodeSPtr mPreviousNode;
size_t mCount;
// ... some data
};
The problem with this is that it generates big number of small items and this slows down the algo.
The question is what other options you could suggest where both performance and memory are sensitive matters.
I'm not quite sure that I need an object pool, yet it seems the most viable solution, but has some un-wanted cons associated with it. I am making a game, where entities are stored within an object pool. These entities are not allocated directly with new, instead a std::deque handles the memory for them.
This is what my object pool more or less looks like:
struct Pool
{
Pool()
: _pool(DEFAULT_SIZE)
{}
Entity* create()
{
if(!_destroyedEntitiesIndicies.empty())
{
_nextIndex = _destroyedEntitiesIndicies.front();
_destroyedEntitiesIndicies.pop();
}
Entity* entity = &_pool[_nextIndex];
entity->id = _nextIndex;
return entity;
}
void destroy(Entity* x)
{
_destroyedEntitiesIndicies.emplace(x->id);
x->id = 0;
}
private:
std::deque<Entity> _pool;
std::queue<int> _destroyedEntitiesIndicies;
int _nextIndex = 0;
};
If I destroy an entity, it's ID will be added to the _destroyedEntitiesIndicies queue, which will make it so that the ID will be re-used, and lastly it's ID will be set to 0. Now the only pitfall to this is, if I destroy an entity and then immediately create a new one, the Entity that was previously destroyed will be updated to be the same entity that was just created.
i.e.
Entity* object1 = pool.create(); // create an object
pool.destroy(object1); // destroy it
Entity* object2 = pool.create(); // create another object
// now object1 will be the same as object2
std::cout << (object1 == object2) << '\n'; // this will print out 1
This doesn't seem right to me. How do I avoid this? Obviously the above will probably not happen (as I'll delay object destruction until the next frame). But this may cause some disturbance whilst saving entity states to a file, or something along those lines.
EDIT:
Let's say I did NULL entities to destroy them. What if I was able to get an Entity from the pool, or store a copy of a pointer to the actual entity? How would I NULL all the other duplicate entities when destroyed?
i.e.
Pool pool;
Entity* entity = pool.create();
Entity* theSameEntity = pool.get(entity->getId());
pool.destroy(entity);
// now entity == nullptr, but theSameEntity still points to the original entity
If you want an Entity instance only to be reachable via create, you will have to hide the get function (which did not exist in your original code anyway :) ).
I think adding this kind of security to your game is quite a bit of an overkill but if you really need a mechanism to control access to certain parts in memory, I would consider returning something like a handle or a weak pointer instead of a raw pointer. This weak pointer would contain an index on a vector/map (that you store somewhere unreachable to anything but that weak pointer), which in turn contains the actual Entity pointer, and a small hash value indicating whether the weak pointer is still valid or not.
Here's a bit of code so you see what I mean:
struct WeakEntityPtr; // Forward declaration.
struct WeakRefIndex { unsigned int m_index; unsigned int m_hash; }; // Small helper struct.
class Entity {
friend struct WeakEntityPtr;
private:
static std::vector< Entity* > s_weakTable( 100 );
static std::vector< char > s_hashTable( 100 );
static WeakRefIndex findFreeWeakRefIndex(); // find next free index and change the hash value in the hashTable at that index
struct WeakEntityPtr {
private:
WeakRefIndex m_refIndex;
public:
inline Entity* get() {
Entity* result = nullptr;
// Check if the weak pointer is still valid by comparing the hash values.
if ( m_refIndex.m_hash == Entity::s_hashTable[ m_refIndex.m_index ] )
{
result = WeakReferenced< T >::s_weakTable[ m_refIndex.m_index ];
}
return result;
}
}
This is not a complete example though (you will have to take care of proper (copy) constructors, assignment operations etc etc...) but it should give you the idea what I am talking about.
However, I want to stress that I still think a simple pool is sufficient for what you are trying to do in that context. You will have to make the rest of your code to play nicely with the entities so they don't reuse objects that they're not supposed to reuse, but I think that is easier done and can be maintained more clearly than the whole handle/weak pointer story above.
This question seems to have various parts. Let's see:
(...) If I destroy an entity and then immediately create a new one,
the Entity that was previously destroyed will be updated to be the
same entity that was just created. This doesn't seem right to me. How
do I avoid this?
You could modify this method:
void destroy(Entity* x)
{
_destroyedEntitiesIndicies.emplace(x->id);
x->id = 0;
}
To be:
void destroy(Entity *&x)
{
_destroyedEntitiesIndicies.emplace(x->id);
x->id = 0;
x = NULL;
}
This way, you will avoid the specific problem you are experiencing. However, it won't solve the whole problem, you can always have copies which are not going to be updated to NULL.
Another way is yo use auto_ptr<> (in C++'98, unique_ptr<> in C++-11), which guarantee that their inner pointer will be set to NULL when released. If you combine this with the overloading of operators new and delete in your Entity class (see below), you can have a quite powerful mechanism. There are some variations, such as shared_ptr<>, in the new version of the standard, C++-11, which can be also useful to you. Your specific example:
auto_ptr<Entity> object1( new Entity ); // calls pool.create()
object1.release(); // calls pool.destroy, if needed
auto_ptr<Entity> object2( new Entity ); // create another object
// now object1 will NOT be the same as object2
std::cout << (object1.get() == object2.get()) << '\n'; // this will print out 0
You have various possible sources of information, such as the cplusplus.com, wikipedia, and a very interesting article from Herb Shutter.
Alternatives to an Object Pool?
Object pools are created in order to avoid continuous memory manipulation, which is expensive, in those situations in which the maximum number of objects is known. There are not alternatives to an object pool that I can think of for your case, I think you are trying the correct design. However, If you have a lot of creations and destructions, maybe the best approach is not an object pool. It is impossible to say without experimenting, and measuring times.
About the implementation, there are various options.
In the first place, it is not clear whether you're experiencing performance advantages by avoiding memory allocation, since you are using _destroyedEntitiesIndicies (you are anyway potentially allocating memory each time you destroy an object). You'll have to experiment with your code if this is giving you enough performance gain in contrast to plain allocation. You can try to remove _destroyedEntitiesIndicies altogether, and try to find an empty slot only when you are running out of them (_nextIndice >= DEFAULT_SIZE ). Another thing to try is discard the memory wasted in those free slots and allocate another chunk (DEFAULT_SIZE) instead.
Again, it all depends of the real use you are experiencing. The only way to find out is experimenting and measuring.
Finally, remember that you can modify class Entity in order to transparently support the object pool or not. A benefit of this is that you can experiment whether it is a really better approach or not.
class Entity {
public:
// more things...
void * operator new(size_t size)
{
return pool.create();
}
void operator delete(void * entity)
{
}
private:
Pool pool;
};
Hope this helps.
I have an implementation of a queue, something like template <typename T> queue<T> with a struct QueueItem { T data;} and I have a separate library that times the passage of data across different places (including from one producer thread to consumer thread via this queue). In order to do this, I inserted code from that timing library into the push and pop functions of the queue so that when they assign a BufferItem.data they also assign an extra member i added of type void* to some timing metadata from that library. I.e. what used to be something like:
void push(T t)
{
QueueItem i;
i.data = t;
//insert i into queue
}
became
void push(T t)
{
QueueItem i;
i.data = t;
void* fox = timinglib.getMetadata();
i.timingInfo = fox;
//insert i into queue
}
with QueueItem going from
struct QueueItem
{
T data;
}
to
struct QueueItem
{
T data;
void* timingInfo;
}
What I would like to achieve, however, is the ability to swap out of the latter struct in favor of the lighter weight struct whenever the timing library is not activated. Something like:
if timingLib.isInactive()
;//use the smaller struct QueueItem
else
;//use the larger struct QueueItem
as cheaply as possible. What would be a good way to do this?
You can't have a struct that is big and small at the same time, obviously, so you're going to have to look at some form of inheritance or pointer/reference, or a union.
A union would be ideal for you if there's "spare" data in T that could be occupied by your timingInfo. If not, then it's going to be as 'heavy' as the original.
Using inheritance is also likely to be as big as the original, as it'll add a vtable in there which will pad it out too much.
So, the next option is to store a pointer only, and have that point to the data you want to store, either the data or the data+timing. This kind of pattern is known as 'flyweight' - where common data is stored separately to the object that is manipulated. This might be what you're looking for (depending on what the timing info metadata is).
The other, more complex, alternative is to have 2 queues that you keep in sync. You store data in one, and the other one stores the associated timeing info, if enabled. If not enabled, you ignore the 2nd queue. The trouble with this is ensuring the 2 are kept in sync, but that's a organisational problem rather than a technical challenge. Maybe create a new Queue class that contains the 2 real queues internally.
I'll start by just confirming my assumption that this needs to be a runtime choice and you can't just build two different binaries with timing enabled/disabled. That approach eliminates as much overhead in any approach as possible.
So now let's assume we want different runtime behavior. There will need to be runtime decisions, so there are a couple options. If you can get away with the (relatively small) cost of polymorphism then you could make your queue polymorphic and create the appropriate instance once at startup and then its push for example either will or won't add the extra data.
However if that's not an option I believe you can use templates to help accomplish your end, although there will likely be some up-front work and it will probably increase the size of your binary with the extra code.
You start with a template to add timing to a class:
template <typename Timee>
struct Timed : public Timee
{
void* timingInfo;
};
Then a timed QueueItem would look like:
Timed<QueueItem> timed_item;
To anything that doesn't care about the timing, this class looks exactly like a QueueItem: It will automatically upcast or slice to the parent as appropriate. And if a method needs to know the timing information you either create an overload that knows what to do for a Timed<T> or do a runtime check (for the "is timing enabled" flag) and downcast to the correct type.
Next, you'll need to change your Queue instantiation to know whether it's using the base QueueItem or the Timed version. For example, a very very rough sketch of a possible mechanism:
template <typename Element>
void run()
{
Queue<Element> queue;
queue.setup();
queue.process();
}
int main()
{
if(do_timing)
{
run<Timed<QueueItem> >();
}
else
{
run<QueueItem>();
}
return 0;
}
You would "likely" need a specialization for Queue when used with Timed items unless getting the metadata is stateless in which case the Timed constructor can gather the info and self-populate itself when created. Then Queue just stays the same and relies on which instantiation you're using.
Currently I want to optimize my 3d engine for consoles a bit. More precisely I want to be more cache friendly and align my structures more data oriented, but also want to keep my nice user interface.
For example:
bool Init()
{
// Create a node
ISceneNode* pNode = GetSystem()->GetSceneManager()->AddNode("viewerNode");
// Create a transform component
ITransform* pTrans = m_pNode->CreateTransform("trans");
pTrans->SetTranslation(0,1.0f,-4.0f);
pTrans->SetRotation(0,0,0);
// Create a camera component
ICamera* pCam = m_pNode->CreateCamera("cam", pTrans);
pCam->LookAt(Math::Vec3d(0,0,0));
// And so on...
}
So the user can work with interface pointers in his code.
BUT
In my engine I currently store pointers to scene nodes.
boost::ptr_vector<SceneNode> m_nodes
So in data oriented design it's good practice to have structs of arrays and not arrays of structs. So my node gets from...
class SceneNode
{
private:
Math::Vec3d m_pos;
};
std::vector<SceneNode> m_nodes;
to this...
class SceneNodes
{
std::vector<std::string> m_names;
std::vector<Math::Vec3d> m_positions;
// and so on...
};
So I see two problems here if I want to apply DOP.
Firstly how could I keep my nice user interface without having the user to work with IDs, indexes and so on?
Secondly how do I handle relocations of properties when some vectors resize without letting users interface pointers point to nirvana?
Currently my idea is to implement a kind of handle_vector from which you get a handle for persistent "pointers":
typedef handle<ISceneNodeData> SceneNodeHandle;
SceneNodeHandle nodeHandle = nodeHandleVector.get_handle(idx);
So when the intern std::vector resizes, it updates its handles.
A "handle" stores a pointer to the actual object and the "->" operator is overloaded to achive a nice wrapping. But this approach sounds a bis complicated to me?!
What do you think? How to keep a nice interface, but keep thinks contiguous in memory for better cache usage?
Thanks for any help!
You will need to use smarter handles than raw pointers. There is no way around it with DOP.
This means:
class SceneNode
{
public:
std::string const& getName() const { mManager->getSceneName(mId); }
void setName(std::string const& name) { mManager->setSceneName(mId, name); }
// similar with other data
private:
ISceneManager* mManager;
size_t mId;
};
One very good point though: the user cannot accidently call delete on one of the pointer you returned now. That's why smart handles are always better.
On the other hand: how are you going to deal with the lifetime of the pointee of mManager is another issue :-)
For those interested in a practical example of DOP, have a look at this fantastic presentation from Niklas Frykholm => http://bitsquid.blogspot.com/2010/05/practical-examples-in-data-oriented.html
This helped me to implement my scene graph in a data oriented manner.