STL and multithread - c++

Question

I'm aware, that I need to use mutex, when I perform operations on single STL container inside multiple threads. However I want to know if there are any exceptions from this rule. Please consider simplified scenario I'm trying to implement.
I have multiple threads adding elements to container and operation is surrounded with mutex lock/unlock. Then threads notify somehow (e.g. using eventfd on linux) single thread dedicated to dispatch elements in this container. What I want to do is to access first element in container without using mutex. Sample code based on deque but note that I ca use any container with queue capability:
std::mutex locker;
std:deque<int> int_queue;
int fd; // eventfd
eventfd_t buffer;
bool some_condition;
Thread 1, 2, 3, etc.
locker.lock ();
int_queue.push_back (1);
locker.unlock ();
eventfd_write (fd, 1);
Thread dedicated to dispatch elements:
while (true)
{
bool some_condition (true);
locker.lock ();
if (int_quque.empty () == false)
{
locker.unlock ();
}
else
{
locker.unlock ();
eventfd_read (fd, &buffer);
}
while (some_condition)
{
int& data (int_queue.front ());
some_condition = some_operation (data); // [1]
}
locker.lock ();
int_queue.pop ();
locker.unlock ();
}
[1] I will do some_operation() on signle element many times, that's why I want to avoid mutex locking here. It's to expensive.
I want to know if this code can lead to any synchronisation problems or something.

Answer 1

What you need is reference stability. I.e. you can use containers this way if the reference to the first element is not invalidated when the container is push_back'd. And even then, you'll want to obtain the reference to the front element under the lock.
I'm more familiar with std::condition_variable for the event notification, so I'll use that:
#include <mutex>
#include <condition_variable>
#include <deque>
std::mutex locker;
std::deque<int> int_queue;
std::condition_variable cv;
void thread_1_2_3()
{
// use lock_guard instead of explicit lock/unlock
// for exception safety
std::lock_guard<std::mutex> lk(locker);
int_queue_.push_back(1);
cv.notify_one();
}
void dispatch()
{
while (true)
{
bool some_condition = true;
std::unique_lock<std::mutex> lk(locker);
while (int_queue.empty())
cv.wait(lk);
// get reference to front under lock
int& data = int_queue.front();
lk.unlock();
// now use the reference without worry
while (some_condition)
some_condition = some_operation(data);
lk.lock();
int_queue.pop_front();
}
}
23.3.3.4 [deque.modifiers] says this about push_back:
An insertion at either end of the deque invalidates all the iterators
to the deque, but has no effect on the validity of references to
elements of the deque.
That is the key to allowing you to hang onto that reference outside of the lock. If thread_1_2_3 starts inserting or erasing in the middle, then you can no longer hang on to this reference.
You can't use a vector this way. But you could use a list this way. Check each container you want to use this way for reference stability.

Answer 2

I can't really see through your question or your code, but in general, the containers in the standard C++ library offer you a loose guarantee that concurrent access at different elements is thread-safe. Be sure to understand the implications and limitations of that, though: If you have a random-access container, or iterators to elements, and you only use those to read or change an element value, then as long as you're doing that at different elements, the result should be well-defined. What isn't OK is changing the container itself, so any erase or insert operations have to be serialized (e.g. by locking access to the entire container), and be sure to understand your container's iterator and reference invalidation rules when you do that.
For individual containers you might be able to say a bit more - for example, insert/erase in a tree-based container, and insert/erase in the middle of a random-access container almost certainly requires a global lock. In a vector/deque you'll need to reacquire iterators. In a list, you might get away with performing insertions concurrently at distinct locations.
Any global operations like size() and empty() need to be serialized as well.

Answer 3

For this particular example this is not safe
int& data (int_queue.front ());
You take a reference to the first element, it could be moved by another thread adding element adding to the queue forcing it to re allocate (deques are typically implemented as "wrap-around" arrays). If you copy the value as opposed to taking a reference, depending on the implementation you might get away with it. If you want to be able to do this, a std::deque doesn't come with any standard "exceptions" to this rule. It's certainly possible to write a data structure similar to a deque where this would be safe, but a deque is not guaranteed to be written like (and is unlikley to be written like) that.

Answer 4

Why do you want to do this? Why does the consumer thread not extract the object within the lock, and then process it out of band?
Assuming that what you want to avoid is having to copy the object outside of the container, a simpler easier to maintain approach could be dynamically allocating the objects, using a container of (smart) pointers and extracting it within the lock (minimal cost). Then you no longer need to consider thread safety issues.
Note that even if you might be able to pull this off in this particular scenario, you cannot use more than one consumer thread. I would recommend against the approach and just find a different approach where you can meet your requirements without walking over the bleeding edge. Multithreading is hard to do right, and very hard to debug or even detect that there is an issue. By adhering to common patterns you make your code easier to reason about and to maintain.

Answer 5

If you do want to a lock free queue, I also recommend you look at http://drdobbs.com/cpp/210604448?pgno=2