I have a publisher thread and a consumer thread. They share data via a std::stack<Data *>. The publisher simply push() the pointer and consumer simply pop() the pointer, use it and call delete on it. Since there is only single thread publishing pointers one at a time, and one thread consuming pointers, is there any need to synchronize the stack? Keep in mind that stack is only storing pointers. Publisher pushes pointer only when Data() is fully constructed.
Failure to synchronize on non-const methods of containers in std namespace is undefined behavior.
Neither push nor pop is const on the underlying container of a stack. So two threads are both writing to the state of the underlying container of the stack.
A way to think about it is that both are, at the very least, going to have to fight over the count of the number of elements in the stack: one is trying to increase it, the other is trying to decrease it. (There are other problems, but that one should convince you that both are writing to the state of the stack)
The std::stack<Data*> instance will need to have access synchronized as more than one thread can be modifying it (via pop() and push()) but the elements contained in it do not as only a single thread can be operating on an element at any one time.
Yes, there is a need to synchronize access to the stack, because std::stack class does not guarantee that any operation is atomic and it is possible, that push(), top() and pop() will interleave.
Related
When I'm iterating through an std::map, is there a possibility that by for example adding an element to the map in another thread, the objects in it will be removed causing the iteration to be corrupt? (As the iterator will be pointing to a non-existing variable as it's moved)
In theory when you add an element to an std::map, all the iterators in that map should stay valid. But the problem is that the operations are not atomic. If the OS suspends the inserting thread in the middle of the operation and gives control back to the iterating thread, the state of std::map might be invalid.
You need to synchronize access to the map via mutex or something similar. Alternatively you could use concurrency friendly collection from TBB or another similar library. TBB provides concurrent_unordered_map and concurrent_hash_map.
STL containers aren't thread safe. No guarantees at all. So you need to synchronize access to any standard container if they are used by different threads.
Yes--if another thread may be modifying the vector, you'll need to use something like a mutex to assure that only one thread has access to the vector at any given time.
With a map, the effects of a modification are much more limited -- rather than potentially moving the entire contents of a vector, a modification only affects an individual node in the map. Nonetheless, if one thread deletes a node just as another thread is trying to read that node, bad things will happen, so you still need a mutex to assure that only one thread is operating on the map at any given time.
I currently have a bit of a multi-threading conundrum. I have two threads, one that reads serial data, and another that attempts to extracts packets from the data. The two threads share a queue. The thread that attempts to create packets has a function entitled parse with the following declaration:
Parse(std::queue<uint8_t>* data, pthread_mutex_t* lock);
Essentially it takes a pointer to the STL queue and uses pop() as it goes through the queue looking for a packet. The lock is used since any pop() is locked and this lock is shared between the Parse function and the thread that is pushing data onto the queue. This way, the queue can be parsed while data is being actively added to it.
The code seems to work for the most part, but I'm seeing invalid packets at a somewhat higher rate than I'd expect. My main question is I'm wondering if the pointer is changing while I'm reading data out of the queue. For example, if the first thread pushes a bunch of data, is there a chance that where the queue is found in memory can change? Or am I guaranteed that the pointer to the queue will remain constant, even as data is added? My concern is that the memory for the queue can be reallocated during my Parse() function, and therefore in the middle of my function, the pointer is invalidated.
For example, I understand that certain STL iterators are invalidated for certain operations. However, I am passing a pointer to the container itself. That is, something like this:
// somewhere in my code I create a queue
std::queue<uint8_t> queue;
// elsewhere...
Parse(&queue, &lock_shared_between_the_two_threads);
Does the pointer to the container itself ever get invalidated? And what does it point to? The first element, or ...?
Note that I'm not pointing to any given element, but to the container itself. Also, I never specified which underlying container should be used to implement the queue, so underneath it all, it's just a deque.
Any help will be greatly appreciated.
EDIT 8/1:
I was able to run a few tests on my code. A couple of points:
The pointer for the container itself does not change over the lifecycle of my program. This makes sense since the queue itself is a member variable of a class. That is, while the queue's elements are dynamically allocated, it does not appear to be the case for the queue itself.
The bad packets I was experiencing appear to be a function of the serial data I'm receiving. I dumped all the data to a hex file and was able to find packets that were invalid, and my alogrithm was correctly marking them as such.
As a result, I'm thinking that passing a reference or pointer to an STL container into a function is thread safe, but I'd like to hear some more commentary ensuring that this is the case, or if this is implementation specific (as alot of STL is...).
You are worried that modifying a container (adding/deleting nodes) in one thread will somehow invalidate the pointer to the container in another thread. The container is an abstraction and will remain valid unless you delete the container object itself. The memory for the data maintained by the containers are typically allocated on the heap by stl::allocators.
This is quite different from the memory allocated for the container object itself which can be on the stack, heap etc., based on how the container object itself was created. This separation of the container from the allocator is what's preventing some modification to the data from modifying the container object itself.
To make debugging your problem simpler, like Jonathan Reinhart suggests, make it a single threaded system, that reads the stream AND parses it.
On a side note, have you considered using Boost Lookfree Queues or something similar. They are designed exactly for this type of scenarios. If you were receiving packets/reading them frequently, locking the queue for reading/writing for each packet can become a significant performance overhead.
Please note - these builds are for VS2008/VS2010 builds I cannot use any 11 constructs.
Imagine I have subscribers listening to some publisher. My publisher has a container of subscriber pointers. In my void detach(ISubscriber *), instead of locking the subscriber list, I will "NULL" out the pointer, for lack of a better word, for that subscriber.
//My container in the publisher. Inserts to not invalidate, removals only invalidate iterators pointing to the removed element, for this reason we NULL
Container<ISubscriber *> myContainer;
Now in the publisher...
void NotifySubscribers(){
foreach(subscriber in container){
if(subscriber)//This is my problem
subscriber->notify()
}
}
Line 3 - pointer is tested and is pointing to valid object.
Before line 4 is executed, another thread NULLs the subscriber.
Line 4 - Boom.
My Question, is there a way that I can use some sort of Interlocked something such that the test and call is atomic.
e.g. for a reference counted object in the destructor, something like this works
RefCountObject::~RefCountObject(){
if(InterlockedDecrement(&m_count) == 0)
delete m_data;
}
Here, the reference counter is decremented and tested against zero automically, then and only then if equal to zero, the data is released.
Is there a way for me to do this for calling a function based on the validity of a pointer?
Edit 1: I need to clarify a little based on the comments and thank you for your replies. The publisher is not responsible for the "releasing of memory" of the Subscribers, so there will be no leak. After the notify, the publisher will go through a loop that cleans up the container by removing nulled out subscribers.
Now as for the subscribers themselves. When they detach, they are just detaching from listening to the publisher. They themselves will live on in static objects (This is the contract we are requiring). Why? Because we cannot afford to hold a lock during notification. The only other option was to use Share_Ptr, which was decided not to be incorporated into this DLL, due to versioning in the future.
I created a hand written shared_ptr, but then it occurred to me that any reference to an object that was not wrapped in a resource management class would fall into the same pitfall and just push the "requirement" that subscribers would have to make sure to not refer to any dangling references within their implementation of said subscriber.
Which brings us back to just saying, subscribers cannot be "released", and currently all the clients that will use this are static objects. We were just looking towards the future. Some of the users are legacy apps and would not be easy to bring in enabled_shared_from_this etc.
is there a way that I can use some sort of Interlocked something such that the test and call is atomic.
For the test, yes there will be a way. You just want to compare a pointer.
To do the call, i doubt it. You will need a guard around the call, i.e. a Critical Section.
You can use a "smart pointer" strategy to do a deferred nulling of the pointer. As long as someone has a reference to the pointer, as determined by an interlocked reference count, keep the pointer valid; when the count goes to zero it's safe to null.
We've been looking to use a lock-free queue in our code to reduce lock contention between a single producer and consumer in our current implementation. There are plenty of queue implementations out there, but I haven't been too clear on how to best manage memory management of nodes.
For example, the producer looks like this:
queue.Add( new WorkUnit(...) );
And the consumer looks like:
WorkUnit* unit = queue.RemoveFront();
unit->Execute();
delete unit;
We currently use memory pool for allocation. You'll notice that the producer allocates the memory and the consumer deletes it. Since we're using pools, we need to add another lock to the memory pool to properly protect it. This seems to negate the performance advantage of a lock-free queue in the first place.
So far, I think our options are:
Implement a lock-free memory pool.
Dump the memory pool and rely on a threadsafe allocator.
Are there any other options that we can explore? We're trying to avoid implementing a lock-free memory pool, but we may to take that path.
Thanks.
Only producer shall be able to create objects and destroy them when they aren't needed anymore. Consumer is only able to use objects and mark them as used. That's the point. You don't need to share memory in this case. This is the only way I know of efficient lock free queue implementation.
Read this great article that describes such algorithm in details.
One more possibility is to have a separate memory pool for each thread, so only one thread is ever using a heap, and you can avoid locking for the allocation.
That leaves you with managing a lock-free function to free a block. Fortunately, that's pretty easy to manage: you maintain a linked list of free blocks for each heap. You put the block's own address into (the memory you'll use as) the link field for the linked list, and then do an atomic exchange with the pointer holding the head of the linked list.
You should look at Intel's TBB. I don't know how much, if anything, it costs for commercial projects, but they already have concurrent queues, concurrent memory allocators, that sort of thing.
Your queue interface also looks seriously dodgy- for example, your RemoveFront() call- what if the queue is empty? The new and delete calls also look quite redundant. Intel's TBB and Microsoft's PPL (included in VS2010) do not suffer these issues.
I am not sure what exactly your requirements are, but if you have scenario where producer creates data and push this data on queue and you have single consumer who takes this data and uses it and then destroys it, you just need tread safe queue or you can make your own single linked list that is tread safe in this scenario.
create list element
append data in list element
change pointer of next element from null to list element (or equivalent in other languages)
Consumer can be implemented in any way, and most linked lists are by default tread safe for this kind of operation (but check that whit implementation)
In this scenario, consumer should free this memory, or it can return it in same way back to producer setting predefined flag. Producer does not need to check all flags (if there are more than like 1000 of them) to find which bucket is free, but this flags can be implemented like tree, enabling log(n) searching for available pool. I am sure this can be done in O(1) time without locking, but have no idea how
I had the exact same concern, so I wrote my own lock-free queue (single-producer, single-consumer) that manages the memory allocation for you (it allocates a series of contiguous blocks, sort of like std::vector).
I recently released the code on GitHub. (Also, I posted on my blog about it.)
If you create the node on the stack, enqueue it, then dequeue it to another node on the stack, you won't need to use pointers/manual allocation at all. Additionally, if your node implements move semantics for construction and assignment, it will be automatically moved instead of copied :-)
So I have a container(any kind, probably std::map or std::vector) which contains objects of a class with some network thing running in a thread that checks if it is still connected (the thread is defined inside that class and launches when constructed).
Is there any way I can make the object delete itself from the container when its disconnected or should I move the thread outside the object and use that class just to store data?
In order for the object to delete itself from the container, it will have to know which container it is in. You will need to maintain a pointer to the container in the object. You will also have to protect the container with a lock to stop multiple threads accessing the container at the same time.
I think I prefer your second solution - some managing object looks after removing dead objects from the collection. If nothing else, this will be quite a bit easier to debug and the locking logic becomes centralised in a single object.
I would have am unload queue.
When a thread notices that the connection is down it registers the object (and continer) with the unload queue tides everything up as much as possible then the thred terminates.
A separate thread is then inside the unload queue. Its sole purpose is to monitor the queue. When it sees a new object on the queue, remove it from the container and then destroy it (syncing with the objects thread as required).
STL containers tend to assume they're storing values; objects that can be copied and where copies are identical. Typically, objects which have threads fit poorly into that model. They have a much stronger sense of identity. In this case, you definitely have indentity - a copy of the object in a container is distinct from a copy outside.
I had a problem very similar to yours, which I solved by emitting a boost::signal from the "network thing" when it detected the disconnection, being caught by the object managing the container. Upon receiving that signal, it would iterate through the container, removing the dead network session from it. It might be worth looking at it here:
How to make a C++ boost::signal be caught from an object which encapsulates the object which emits it?
Cheers,
Claudio