C++ Iterating a huge std::multimap while handling many elements going MIA - c++

I have a situation where objects will add events (a struct containing a function pointer to a function like object::do_something) to a "chain of events" (std::multimap) in their constructor. My interpreter reads the chain of events (sorted by depth) every time the game updates and executes each one sequentially. When an object is destroyed, it will remove all its events from the chain in its destructor automatically (to prevent possible leaks of events).
Because events are sorted by depth, it's possible that an object might register multiple events which are "next" to each other in the chain. When an object destroys itself, it unlinks all its events and immediately stops running its share of code (when something is destroyed, it can't do anything). I've cunningly produced a way of doing this; the particular function which deletes an object, instance_destroy() will throw an exception which my event interpreter can catch and continue along with the next event in the chain.
I've come to realize;
Unpredictable amounts of events can be unlinked from the chain, and the current iterator is (likely) to be invalidated when an object destroys itself.
Objects can destroy other objects in their lifetime, as well as themselves. I can't simply keep a copy of the next iterator that doesn't belong to the current object in case of destruction, as it could also be removed!
When control is passed back to the interpreter (say, via exception) and heaps of events have been removed, including possibly the current iterator, I have no way of knowing what to execute next. I can't start the map from the beginning -- that would cause undefined behaviour in the game; things would be executed twice. I also can't copy the map -- it's absolutely HUGE -- it would come at an enormous performance penalty. I can't redesign the way the system should work either, as it's not my protocol.
Consider the following data structure;
typedef std::multimap<real_t, event> events_by_depth_t;
How can I iterate it given my requirements above?
I'm using C++11.

Related

Pointer to STL Container Thread Safety (Queue/Deque)

I currently have a bit of a multi-threading conundrum. I have two threads, one that reads serial data, and another that attempts to extracts packets from the data. The two threads share a queue. The thread that attempts to create packets has a function entitled parse with the following declaration:
Parse(std::queue<uint8_t>* data, pthread_mutex_t* lock);
Essentially it takes a pointer to the STL queue and uses pop() as it goes through the queue looking for a packet. The lock is used since any pop() is locked and this lock is shared between the Parse function and the thread that is pushing data onto the queue. This way, the queue can be parsed while data is being actively added to it.
The code seems to work for the most part, but I'm seeing invalid packets at a somewhat higher rate than I'd expect. My main question is I'm wondering if the pointer is changing while I'm reading data out of the queue. For example, if the first thread pushes a bunch of data, is there a chance that where the queue is found in memory can change? Or am I guaranteed that the pointer to the queue will remain constant, even as data is added? My concern is that the memory for the queue can be reallocated during my Parse() function, and therefore in the middle of my function, the pointer is invalidated.
For example, I understand that certain STL iterators are invalidated for certain operations. However, I am passing a pointer to the container itself. That is, something like this:
// somewhere in my code I create a queue
std::queue<uint8_t> queue;
// elsewhere...
Parse(&queue, &lock_shared_between_the_two_threads);
Does the pointer to the container itself ever get invalidated? And what does it point to? The first element, or ...?
Note that I'm not pointing to any given element, but to the container itself. Also, I never specified which underlying container should be used to implement the queue, so underneath it all, it's just a deque.
Any help will be greatly appreciated.
EDIT 8/1:
I was able to run a few tests on my code. A couple of points:
The pointer for the container itself does not change over the lifecycle of my program. This makes sense since the queue itself is a member variable of a class. That is, while the queue's elements are dynamically allocated, it does not appear to be the case for the queue itself.
The bad packets I was experiencing appear to be a function of the serial data I'm receiving. I dumped all the data to a hex file and was able to find packets that were invalid, and my alogrithm was correctly marking them as such.
As a result, I'm thinking that passing a reference or pointer to an STL container into a function is thread safe, but I'd like to hear some more commentary ensuring that this is the case, or if this is implementation specific (as alot of STL is...).
You are worried that modifying a container (adding/deleting nodes) in one thread will somehow invalidate the pointer to the container in another thread. The container is an abstraction and will remain valid unless you delete the container object itself. The memory for the data maintained by the containers are typically allocated on the heap by stl::allocators.
This is quite different from the memory allocated for the container object itself which can be on the stack, heap etc., based on how the container object itself was created. This separation of the container from the allocator is what's preventing some modification to the data from modifying the container object itself.
To make debugging your problem simpler, like Jonathan Reinhart suggests, make it a single threaded system, that reads the stream AND parses it.
On a side note, have you considered using Boost Lookfree Queues or something similar. They are designed exactly for this type of scenarios. If you were receiving packets/reading them frequently, locking the queue for reading/writing for each packet can become a significant performance overhead.

Exit the entire recursion stack

I'm calling a function fooA from main() that calls another function fooB that is recursive.
When I wish to return, I keep using exit(1) to halt execution. What is the right way to exit when the recursion tree is deep?
Returning through the recursion stack may not be of help because returning usually clears a part solution I build and I don't want to do that. I want to do execute more piece of code from main().
I read Exceptions can be used, it would be nice if I can get a code snippet.
The goto statement won't work to hop from one function back to another; Nikos C. is correct that it wouldn't account for releasing the stack frames of each of the calls you've made, so when you got to the function you goto'ed to, the stack pointer would be pointing to the stack frame of the function you were just in... no, that just won't work. Similarly, you can't simply call (either directly, or indirectly via a function pointer) the function you want to end up in when your algorithm is done. You'd never get back to the context you were in prior to diving into your recursive algorithm. You could conceivably architect a system this way, but in essence each time you did this you'd "leak" what was currently on the stack (not quite the same as leaking heap memory, but a similar effect). And if you were deep into a highly recursive algorithm, that could be a lot of "leaked" stack space.
No, you need to somehow return back to the calling context. There are only three ways to do so in C++:
Exit each function in turn by returning from it to its caller
backing up through the call chain in an orderly fashion.
Throw an exception and catch it at the point right after you
launched into your recursive algorithm (which automatically destroys
any objects created by each function on the stack in an orderly
fashion).
Use setjmp() & longjmp() to do something similar to throwing &
catching an exception, but "throwing" a longjmp() will not destroy
objects on the stack; if any such objects own heap allocations,
those allocations will be leaked.
To do option 1, you have to write your recursive function such that once a solution is reached, it returns some sort of indication that it's complete to its caller (which may be the same function), and its caller sees that fact & relays that fact on to its caller by returning to it (which may be the same function), so on and so on, until finally all stack frames of the recursive algorithm are released and you return to whatever function called the first function in the recursive algorithm.
To do option 2, you wrap the call to your recursive algorithm in a try{...} and immediately after it you catch(){...} the expected thrown object (which could conceivably be the result of the computation, or just some object that lets the caller know "hey, I'm done, you know where to find the result"). Example:
try
{
callMyRecursiveFunction(someArg);
}
catch( whateverTypeYouWantToThrow& result )
{
...do whatever you want to do with the result,
including copy it to somewhere else...
}
...and in your recursive function, when you finish the results, you simply:
throw(whateverTypeYouWantToThrow(anyArgsItsConstructorNeeds));
To do option 3...
#include <setjmp.h>
static jmp_buf jmp; // could be allocated other ways; the longjmp() user just needs to have access to it.
.
.
.
if (!setjmp(jmp)) // setjmp() returns zero 1st time, or whatever int value you send back to it with longjmp()
{
callMyRecursiveFunction(someArg);
}
...and in your recursive function, when you finish the results, you simply:
longjmp(jmp, 1); // this passes 1 back to the setjmp(). If your result is an int, you
// could pass that back to setjmp(), but you can't pass zero back.
The bad thing about using setjmp()/longjmp() is that if there are any stack-allocated objects still "alive" on the stack when you call longjmp(), execution will jump back to the setjmp() point, skipping the destructors for those objects. If your algorithm uses only POD types, that's not an issue. It's also not an issue if the non-POD types your algorithm uses do NOT contain any heap allocations (e.g. from malloc() or new). If your algorithm uses non-POD types that contain heap allocations, then you're only safe with options 1 & 2 above. But if your algorithm meets the criteria of being OK with setjmp()/longjmp(), and if your algorithm is buried under a ton of recursive calls at the point it finishes, setjmp()/longjmp() may be the fastest way back to the initial calling context. If that won't work, option 1 is probably your best bet in terms of speed. Option 2 may seem convenient (and would possibly eliminate a condition check at the start of each recursion call), but the overhead associated with the system automatically unwinding the callstack is somewhat significant.
It's typically said you should reserve exceptions for "exceptional events" (events expected to be very rare), and the overhead associated with unwinding the callstack is why. Older compilers used something akin to setjmp()/longjmp() to implement exceptions (setjmp() at the location of the try & catch, and longjmp() at the location of a throw), but there was of course extra overhead associated with determining what objects on the stack need destroyed, even if there are no such objects. Plus, every time you'd run across a try, it would have to save the context just in case there was a throw, and if exceptions are truly exceptional events, the time spent saving that context was simply wasted. Newer compilers are now more likely to use what are known as "Zero Cost Exceptions" (a.k.a. Table Based Exceptions), which seems like that would solve all the world's problems, but it doesn't.... It makes normal runtime faster because there is no longer a need to save the context every time you run across a try, but in the event that a throw executes, there is now even more overhead associated with decoding information stored in massive tables that the runtime has to process in order to figure out how to unwind the stack based on the location where the throw was encountered and content of the runtime stack. So exceptions aren't free, even though they're very convenient. You'll find a lot of stuff on the internet where people make claims about how unreasonably expensive they are and how much they slow down your code, and you'll also find lots of stuff by people refuting those claims, with both sides presenting hard data to bolster their claims. What you should take away from the arguments is that using exceptions is great if you expect them to rarely occur, because they result in cleaner interfaces & logic that's free of a ton of condition checking for "badness" every time you make a function call. But you shouldn't use exceptions as a means of normal communication between a caller and its callees, because that mode of communication is significantly more expensive than simply using return values.
This happened to me while finding the path from root to node of a binary tree. I was using a stack to store the nodes in preorder and the recursion wouldnt stop until the last node returned NULL. I used a global variable, integer i=1, and when I reached the node I was looking for I set that variable to 0 and used while(i==0) return stack; to allow the program to go back up the memory stack without popping my nodes off.

Qt C++ destructor taking a long time to return

I'm working on a pretty standard Qt mobile app (written in C++, targeted at Symbian devices), and am finding that sometimes when the app is closed (i.e. via a call to QApplication::quit), the final destructor in the app can take a long time to return (30 seconds plus). By this I mean, all clean up operations in the destructor have completed (quickly, all well within a second) and we've reached the point where execution is leaving the destructor and returning to the code that implicitly called it (i.e. when we delete the object).
Obviously at that point I'd expect execution to return to just after the call to delete the object, virtually instantly, but as I say sometimes this is taking an age!
This long closure time happens both in debug and release builds, with logging enabled or disabled, so I don't think that's a factor here. When we reach the end of the destructor I'm pretty certain no file handles are left open, or any other open resources (network connections etc.)...though even if they where surely this wouldn't present itself as a problem on exiting the destructor (?).
This is on deleting the application's QMainWindow object. Currently the call to do this is in a slot connected to QApplication::aboutToQuit, though I've tried deleting that object in the apps "main" function too.
The length of delay we experience seems proportional to the amount of activity in the app before we exit. This sort of makes me think memory leaks may be a problem here, however we're not aware of any (doesn't mean there aren't any of course), and also I've never seen this behaviour before with leaked memory.
Has anyone any ideas what might be going on here?
Cheers
If your final destructor is for a class than inherits QObject then the QObject destructor will be called immediately following the destructor of your final object. Presumably this object is the root of a possibly large object tree which will trigger a number of actions to occur including calling the destructor of all the child QObjects. Since you state that the problem is componded by the amount of activity, there are likely a very large number of children being added to the object tree that are deleted at this time, perhaps more than you intended. Instead of adding all the objects to one giant tree to be deleted all at once. Identify objects that are being created often that don't need to persist through the entire execution. Instead of creating those objects with a parent, start a new tree that can be deleted earlier (parent =0). Look at QObject::deleteLater() which will wait until there is no user interaction to delete the objects in these independant trees.

Cleaning up threads referencing an object when deleting the object (in C++)

I have an object (Client * client) which starts multiple threads to handle various tasks (such as processing incoming data). The threads are started like this:
// Start the thread that will process incoming messages and stuff them into the appropriate queues.
mReceiveMessageThread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)receiveRtpMessageFunction, this, 0, 0);
These threads all have references back to the initial object, like so:
// Thread initialization function for receiving RTP messages from a newly connected client.
static int WINAPI receiveRtpMessageFunction(LPVOID lpClient)
{
LOG_METHOD("receiveRtpMessageFunction");
Client * client = (Client *)lpClient;
while(client ->isConnected())
{
if(client ->receiveMessage() == ERROR)
{
Log::log("receiveRtpMessageFunction Failed to receive message");
}
}
return SUCCESS;
}
Periodically, the Client object gets deleted (for various good and sufficient reasons). But when that happens, the processing threads that still have references to the (now deleted) object throw exceptions of one sort or another when trying to access member functions on that object.
So I'm sure that there's a standard way to handle this situation, but I haven't been able to figure out a clean approach. I don't want to just terminate the thread, as that doesn't allow for cleaning up resources. I can't set a property on the object, as it's precisely properties on the object that become inaccessible.
Thoughts on the best way to handle this?
I would solve this problem by introducing a reference count to your object. The worker thread would hold a reference and so would the creator of the object. Instead of using delete, you decrement from the reference count and whoever drops the last reference is the one that actually calls delete.
You can use existing reference counting mechanisms (shared_ptr etc.), or you can roll your own with the Win32 APIs InterlockedIncrement() and InterlockedDecrement() or similar (maybe the reference count is a volatile DWORD starting out at 1...).
The only other thing that's missing is that when the main thread releases its reference, it should signal to the worker thread to drop its own reference. One way you can do this is by an event; you can rewrite the worker thread's loop as calls to WaitForMultipleObjects(), and when a certain event is signalled, you take that to mean that the worker thread should clean up and drop the reference.
You don't have much leeway because of the running threads.
No combination of shared_ptr + weak_ptr may save you... you may call a method on the object when it's valid and then order its destruction (using only shared_ptr would).
The only thing I can imagine is to first terminate the various processes and then destroy the object. This way you ensure that each process terminate gracefully, cleaning up its own mess if necessary (and it might need the object to do that).
This means that you cannot delete the object out of hand, since you must first resynchronize with those who use it, and that you need some event handling for the synchronization part (since you basically want to tell the threads to stop, and not wait indefinitely for them).
I leave the synchronization part to you, there are many alternatives (events, flags, etc...) and we don't have enough data.
You can deal with the actual cleanup from either the destructor itself or by overloading the various delete operations, whichever suits you.
You'll need to have some other state object the threads can check to verify that the "client" is still valid.
One option is to encapsulate your client reference inside some other object that remains persistent, and provide a reference to that object from your threads.
You could use the observer pattern with proxy objects for the client in the threads. The proxies act like smart pointers, forwarding access to the real client. When you create them, they register themselves with the client, so that it can invalidate them from its destructor. Once they're invalidated, they stop forwarding and just return errors.
This could be handled by passing a (boost) weak pointer to the threads.

Object delete itself from container

So I have a container(any kind, probably std::map or std::vector) which contains objects of a class with some network thing running in a thread that checks if it is still connected (the thread is defined inside that class and launches when constructed).
Is there any way I can make the object delete itself from the container when its disconnected or should I move the thread outside the object and use that class just to store data?
In order for the object to delete itself from the container, it will have to know which container it is in. You will need to maintain a pointer to the container in the object. You will also have to protect the container with a lock to stop multiple threads accessing the container at the same time.
I think I prefer your second solution - some managing object looks after removing dead objects from the collection. If nothing else, this will be quite a bit easier to debug and the locking logic becomes centralised in a single object.
I would have am unload queue.
When a thread notices that the connection is down it registers the object (and continer) with the unload queue tides everything up as much as possible then the thred terminates.
A separate thread is then inside the unload queue. Its sole purpose is to monitor the queue. When it sees a new object on the queue, remove it from the container and then destroy it (syncing with the objects thread as required).
STL containers tend to assume they're storing values; objects that can be copied and where copies are identical. Typically, objects which have threads fit poorly into that model. They have a much stronger sense of identity. In this case, you definitely have indentity - a copy of the object in a container is distinct from a copy outside.
I had a problem very similar to yours, which I solved by emitting a boost::signal from the "network thing" when it detected the disconnection, being caught by the object managing the container. Upon receiving that signal, it would iterate through the container, removing the dead network session from it. It might be worth looking at it here:
How to make a C++ boost::signal be caught from an object which encapsulates the object which emits it?
Cheers,
Claudio