I am aware that std::list is not thread safe. In my application threads keep adding elements to a global list. Another thread get elements from list and processes them one by one. However, I don't want processing thread to lock the list all the time while processing is being done.
So processing thread locks list, gets element, unlocks list and processes the element. While processing in progress, other threads keep adding elements to the list. Once processing is over, processing thread again locks list deletes processed element and unlocks it.
Below is pseudo-code:
std::list<int> mylist ; /* Global list of integers */
void add_thread(int element) /* Threads adding element to the list */
{
write_lock();
mylist.push_back(element);
write_unlock();
return;
}
void list_processing_thread() /* Processes elements from the list */
{
for (std::list<int>::iterator it=mylist.begin(); it!=mylist.end(); ++it)
{
read_lock();
int element = *it;
read_unlock();
process_element(element);
write_lock();
mylist.remove(element);
write_unlock();
}
return;
}
Is it correct approach (to process list elements in efficient manner) ? Will it cause any trouble ?
So processing thread locks list, gets element, unlocks list and processes the element. While processing in progress, other threads keep adding elements to the list. Once processing is over, processing thread again locks list deletes processed element and unlocks it.
Producer threads contend with each other and with the consumer thread for access to the list.
You can eliminate the contention between producers by giving each producer its own queue (std::list + std::mutex or a spinlock). This way:
Producers post items into its own queues. Create a temporary std::list of one element, then lock the mutex, splice that element into the queue, unlock the mutex.
When the consumer is ready, it locks those producer queues one by one and splices the elements into its own queue. Splicing std::list is O(1) operation.
If ordering is required each element should have a timestamp, so that the consumer can sort all elements in its consolidated queue by time.
The above method also makes your critical sections very short because all you do while the mutex is locked is std::list splicing, which is just a few pointer modifications.
Alternatively, just use concurrent_bounded_queue class from Intel® Threading Building Blocks.
No it's not safe. Yes it will cause trouble.
To Elaborate:
There should only be one lock which is the write lock. Readers don't need to defend from each other.
You need to get the lock in the processing thread before you access begin and end iterators as they may be corrupted by the writing thread. After you get the lock, copy the list, and release the lock. When finished processing, get the lock, and edit the original list.
Good luck
Here is possible solution to your problem, in C++11, but you can easily port it to C++-03 + boost or whatever thread library you have:
std::deque<int> queue;
std::mutex mtx;
std::condition_variable ready;
void producer()
{
while (true)
{
int element = produce();
std::unique_lock<std::mutex> lock(mtx);
queue.push_back(element);
lock.unlock();
ready.notify_one();
}
}
void consumer()
{
while (true)
{
std::unique_lock<std::mutex> lock(mtx);
ready.wait(lock, [](){ ! queue.empty() });
int element = queue.front();
queue.pop_front();
lock.unlock();
consume(element);
}
}
There are lots of things you could do with different levels of performance, but two very simple ones:
use std::list::swap in the processing thread to swap the queue with an empty one, that way if lock contention is slowing things down at least you'll be getting all of the backlog of items in one hit, minimising the harm
use a single-writer / many-reader lock (e.g. posix_rwlock_rdlock et al)
First of all, the approach is good, you SHOULD lock for a short period, get the current item, unlock, and process the item while the list is unlocked and accessible for other threads to use.
But, I would recommend that you didn't rely on an list iterator while the list is not locked. Iterators are sneaky, and many data structures may corrupt the iterator when items are added/removed from them.
I would suggest you use the following approach:
while(list.empty() == false)
{
lock();
int element = list.front();
list.pop_front();
unlock();
process(element);
}
best!
Related
I am using a queue shared between 2 threads in my program. One thread keeps pushing data to the queue, and second thread keeps popping data from the queue and writing to a vector.
My question is do I need condition variable along with mutex lock for this scenario when doing enqueue or dequeue operation? How to handle race condition ?
My code is as follows:
void push_data_to_queue(){
mtx.lock();
std::lock_guard<std::mutex> lockGuard(mtx);
for( int i=0; i < 10; i++ ) {
queue.push(i);
}
}
void get_data_from_queue(){
std::vector<int> v;
mtx.lock();
std::lock_guard<std::mutex> lockGuard(mtx);
for(int i=0;i<5;i++) {
v.push_back(queue.front()));
queue.pop();
}
}
int main(){
std::mutex mtx;
std::thread(push_data_to_queue,std::ref(mtx));
std::thread(get_data_from_queue,std::ref(mtx));
return 0;
}
first point is you dont need to call mutex.lock() when you are using std::lock_guard.
second point is this senario there is only one writer thread and one reader thread there is no need for locking mechanism you can implement it with wait free approach or something like busy loop. but you can use just mutex but beware there is no guarantee that if reader or writer thread release mutex it will let other side acquire mutex in another word there is no guarantee ordering of acquisition because of that you need something like condition variable's(but beware condition variable also use for put thread in waiting state and prevent from wasting resources).
as i said you need some method of syncronazation.
imagine this senario:
writer thread write A to queue.
reader thread call back() and get item and start to process.
then writer thread write B to queue.
after that reader thread call pop_back() then it will remove item B from queue without processing.
so because of that you need mutex(no guarantee ordering of acquisition) , or condition variable(which is using mutex inside itself), busy loop(may consume so much resource), spinklock, ...
for better performance you can use atomic variable's.
Conditional variables use a mutex and the .wait() function unlocks the
mutex so another thread can access the shared data. When the condition
variable is notified it tries to lock the mutex again to use the shared
data.
This pattern is used in the following concurrent_queue example from Anthony Williams:
template<typename Data>
class concurrent_queue
{
private:
boost::condition_variable the_condition_variable;
public:
void wait_for_data()
{
boost::mutex::scoped_lock lock(the_mutex);
while(the_queue.empty())
{
the_condition_variable.wait(lock);
}
}
void push(Data const& data)
{
boost::mutex::scoped_lock lock(the_mutex);
bool const was_empty=the_queue.empty();
the_queue.push(data);
if(was_empty)
{
the_condition_variable.notify_one();
}
}
};
Since the code uses std::queue it's clear that the mutex has to be
locked when accessing the queue.
But let's say instead of std::queue one uses Microsofts
Concurrency::concurrent_queue from PPL. Member functions like empty,
push and try_pop are thread safe. Do I still need to lock a mutex in
this case or can the condition variable be used like this, without
creating any possible race conditions.
My code (that seems to work, but what does that mean in multithreading?) looks like this. I have one producer that pushes items into Microsofts concurrent_queue and one background thread that waits for new items in this queue.
The consumer/background thread:
while(runFlag) //atomic
{
while(the_queue.empty() && runFlag) //wait only when thread should still run
{
boost::mutex mtx; //local mutex thats locked afterwards. awkward.
boost::mutex::scoped_lock lock(mtx);
condition.wait(lock);
}
Data d;
while(!the_queue.empty() && the_queue.try_pop(d))
{
//process data
}
}
The producer/main thread:
const bool was_empty = the_queue.empty();
Data d;
the_queue.push(d);
if(was_empty) cond_var.notify_one();
The shutdown procedure:
bool expected_run_state = true;
if(runLoop.compare_exchange_strong(expected_run_state, false))
{
//atomically set our loop flag to false and
//notify all clients of the queue to wake up and exit
cond_var.notify_all();
}
As said above this code seems to work but that doesn't necessarily mean it's correct. Especially the local mutex that is only used because the condition variable interface forces me to use a mutex, seems like a very bad idea. I wanted to use condition variables since the time between data items added to the queue hard to predict and I would have to create to sleep and wake up periodically like this:
if(the_queue.empty()) Sleep(short_amount_of_time);
Are there any other, maybe OS (in my case: Windows) specific tools, that make a background thread sleep until some condition is met without regularly waking up and checking the condition?
The code is not correct in different scenarios, for example. If the queue has a single element when const bool was_empty = the_queue.empty(); is evaluated, but a thread consumes the element and a different thread tries to consume and waits on the condition, the writer will not notify that thread after inserting the element in the queue.
The key issue is that the fact that all of the operations in an interface are thread safe does not necessarily mean that your use of the interface is safe. If you depend on multiple operations being performed atomically, you need to provide synchronization mechanisms externally.
Are there any other, maybe OS (in my case: Windows) specific tools,
that make a background thread sleep until some condition is met
without regularly waking up and checking the condition?
This is exactly what Events are for
But if you are targeting only Windows platform (Vista+) you should check out
Slim Reader/Writer (SRW) Locks
I have a map<int, queue<int>> with one thread writing into it i.e. pushing messages into the queues. They key refers to a client_id, and the queue holds messages for the client. I am looking to make this read-write thread safe.
Currently, the thread that writes into it does something like this
map<int, queue<int>> msg_map;
if (msg_map.find(client_id) != msg_map.end())
{
queue<int> dummy_queue;
dummy_queue.push(msg); //msg is an int
msg_map.insert(make_pair(client_id, dummy_queue);
}
else
{
msg_map[client_id].push(msg);
}
There are many clients reading - and removing - from this map.
if (msg_map.find(client_id) != msg_map.end())
{
if (!msg_map.find(client_id)->second.empty())
{
int msg_rxed = msg_map[client_id].front();
//processing message
msg_map[client_id].pop();
}
}
I am reading this on mutexes (haven't used them before) and I was wondering when and where I ought to lock the mutex. My confusion lies in the fact that they are accessing individual queues (held within the same map). Do I lock the queues, or the map?
Is there a standard/accepted way to do this - and is using a mutex the best way to do this? There are '0s of client threads, and just that 1 single writing thread.
Simplifying and optimizing your code
For now we'll not concern ourselves with mutexes, we'll handle that later when the code is cleaned up a bit (it will be easier then).
First, from the code you showed there seems to be no reason to use an ordered std::map (logarithmic complexity), you could use the much more efficient std::unordered_map (average constant-time complexity). The choice is entirely up to you, if you don't need the container to be ordered you just have to change its declaration:
std::map<int, std::queue<int>> msg_map;
// or
std::unordered_map<int, std::queue<int>> msg_map; // C++11 only though
Now, maps are quite efficient by design but if you insist on doing lookups for each and every operation then you lose all the advantage of maps.
Concerning the writer thread, all your block of code (for the writer) can be efficiently replaced by just this line:
msg_map[client_id].push(msg);
Note that operator[] for both std::map and std::unordered_map is defined as:
Inserts a new element to the container using key as the key and a default constructed mapped value and returns a reference to the newly constructed mapped value. If an element with key key already exists, no insertion is performed and a reference to its mapped value is returned.
Concerning your reader threads, you can't directly use operator[] because it would create a new entry if none currently exists for a specific client_id so instead, you need to cache the iterator returned by find in order to reuse it and thus avoid useless lookups:
auto iter = msg_map.find(client_id);
// iter will be either std::map<int, std::queue<int>>::iterator
// or std::unordered_map<int, std::queue<int>>::iterator
if (iter != msg_map.end()) {
std::queue<int>& q = iter->second;
if (!q.empty()) {
int msg = q.front();
q.pop();
// process msg
}
}
The reason why I pop the message immediately, before processing it, is because it will improve concurrency when we add mutexes (we can unlock the mutex sooner, which is always good).
Making the code thread-safe
#hmjd's idea about multiple locks (one for the map, and one per queue) is interesting, but based on the code you showed us I disagree: any benefit you'll get from the additional concurrency will quite probably be negated by the additional time it takes to lock the queue mutexes (indeed, locking mutexes is a very expensive operation), not to mention the additional code complexity you'll have to handle. I'll bet my money on a single mutex (protecting the map and all the queues at once) being more efficient.
Incidentally, a single mutex solves the iterator invalidation problem if you want to use the more efficient std::unordered_map (std::map doesn't suffer from that problem though).
Assuming C++11, just declare a std::mutex along with your map:
std::mutex msg_map_mutex;
std::map<int, std::queue<int>> msg_map; // or std::unordered_map
Protecting the writer thread is quite straightforward, just lock the mutex before accessing the map:
std::lock_guard<std::mutex> lock(msg_map_mutex);
// the lock is held while the lock_guard object stays in scope
msg_map[client_id].push(msg);
Protecting the reader threads is barely any harder, the only trick is that you'll probably want to unlock the mutex ASAP in order to improve concurrency so you'll have to use std::unique_lock (which can be unlocked early) instead of std::lock_guard (which can only unlock when it goes out of scope):
std::unique_lock<std::mutex> lock(msg_map_mutex);
auto iter = msg_map.find(client_id);
if (iter != msg_map.end()) {
std::queue<int>& q = iter->second;
if (!q.empty()) {
int msg = q.front();
q.pop();
// assuming you don't need to access the map from now on, let's unlock
lock.unlock();
// process msg, other threads can access the map concurrently
}
}
If you can't use C++11, you'll have to replace std::mutex et al. with whatever your platform provides (pthreads, Win32, ...) or with the boost equivalent (which has the advantage of being as portable and as easy to use as the new C++11 classes, unlike the platform-specific primitives).
Read and write access to both the map and the queue need synchronized as both structures are being modified, including the map:
map<int, queue<int>> msg_map;
if (msg_map.find(client_id) != msg_map.end())
{
queue<int> dummy_queue;
dummy_queue.push(msg); //msg is an int
msg_map.insert(make_pair(client_id, dummy_queue);
}
else
{
msg_map[client_id].push(msg); // Modified here.
}
Two options are a mutex that locks both the map and queue or have a mutex for the map and a mutex per queue. The second approach is preferable as it reduces the length of time a single lock is held and means multiple threads can be updating several queues concurrently.
In a worker thread, I iterate over a list of items that need updating. In the main thread, the user often calls a function to add elements to that list. I'm using a mutex to prevent the list from being modified while it's being iterated. Generally, the pseudocode looks like this:
// called by the client to add an element.
void AddElementToList(element)
{
// lock mutex
// add element to the list
// unlock mutex
}
// this function is run by the worker thread
void WorkerThreadUpdate()
{
// lock mutex
// for each element in the list
// update the element
// mark the element for deletion if finished
// for every finished element
// remove element from list
// unlock mutex
}
The problem comes when updating the element. The element's update function can potentially take a long time, up to about one second in some cases. When this occurs, the user's next call to AddElementToList blocks, causing their application to freeze until all the element have been updated. The AddElementToList function is called often enough for this to be a very noticable problem.
So I'm looking for a better method. I want to update the elements while protecting the list but keep the user's main thread responsive. I'm sure this is pretty much multithreading 101, but I can't come up with the correct terms to find examples of what I'm looking for.
I was thinking of a solution that uses two lists, a "request" list and a "work" list.
The user calls AddElementToList which adds the element to the request list. In the worker thread, after the work list has been iterated, it goes through the request list and adds its elements to the work list for the next frame. It only locks when it actually modifies the lists.
// called by the client to add an element.
void AddElementToList(element)
{
// lock mutex
// add element to the request list
// unlock mutex
}
// this function is run by the worker thread
void WorkerThreadUpdate()
{
// for each element in the work list
// update the element
// mark the element for deletion if finished
// for every finished element
// lock mutex
// remove element from work list
// unlock mutex
// lock mutex
// for every element in the request list
// add element to the work list
// clear the request list
// unlock mutex
}
I think this should work, but I'm not entirely sure. Is this an acceptable way? Are there better methods for handling this?
Your plan to queue up the additions should work. Whether it's acceptable or not depends on whether it's acceptable to wait for queue additions until the thread does its next 'update' pass.
Is it even necessary to lock the list during the update? What else is accessing this list, and when? Can you lock the list, make a copy vector/list of the references, unlock the list and then run the update on each of the the copy refs one-by-one?
Let me ask these first:
what actually is a list in your implementation? A dynamic array? A linked list? Some hash table?
-- If it is a list, operation on one element should affect only the next, previous elements and possible the head and tail.
how do you add elements to the list? Always at the end? Or can insertions occur somewhere in the middle?
Let's assume that:
It is a single-or-bi-direction linked list, with additional variables pointed to its head and tail.
You add the elements only at the end.
If that is the case, I would suggest doing it somehow similar to this:
void AddElementToList(element) {
mutex_lock("push");
list.push_back(element);
mutex_release("push");
}
void WorkerThreadUpdate() {
AddElementToList(specialElement);
/*
at this point we are guaranteed that other threads,
which can only add to the list,
will not affect us, because we have a "guard" between us and them
*/
iterator del=NULL;
iterator i=list.begin();
for(; *i!=specialElement; ++i) {
if (del) {
del->remove();
del=NULL;
}
i->update();
if (some_condition)
del=i;
}
//*i == specialElement now
/*
We are now operating on the guard, so we have to be careful.
"specialElement" could still be the last element in the list.
*/
mutex_lock("push");
i->remove();
mutex_release("push");
}
I am not sure if standard STL is thread-safe and if you can just use their implementation of the list. Read their specs. If not - just implement your own list container.
Is there any chance that another thread will try to access the element
you're updating. If not, you can free the lock on the list before
starting the update, and reacquire it when you want to continue
iterating (or to delete the object). Something along the lines of:
void
WorkerThreadUpdate()
{
// lock mutex
ListType::iterator current = list.begin();
while ( current != list.end() ) {
ListType::value_type& obj = *current;
// unlock mutex
// do modifications...
// lock mutex
if ( needsDeleting ) {
current = list.erase( current );
} else {
++ current;
}
}
// unlock mutex
}
The important thing is that you must hold the lock when actually
accessing the iterator (at least with any of the standard containers).
Of course, you'll want to use some sort of scoped lock, just in case.
(Although that might not be necessary; I think all of the code in the
locked region should be no-throw.) I've not tried it, but I think you
could use std::unique_lock if you have C++11.
IMHO, while this works, it's not particularly elegant. I'd probably
keep the list entirely on the worker side, and use a message queue to
pass the objects to be inserted, with the worker thread reading from the
message queue from time to time in order to get the objects to insert.
Is it possible to use mutex to lock only one element of a data structure ?
e.g.
boost::mutex m_mutex;
map<string, int> myMap;
// initialize myMap so that it has 10 elements
// then in thread 1
{
boost::unique_lock<boost::mutex> lock(m_mutex);
myMap[1] = 5 ; // write map[1]
}
// in thread 2
{
boost::unique_lock<boost::mutex> lock(m_mutex);
myMap[2] = 4 ; // write map[1]
}
My question:
When thread 1 is writing map[1], thread 2 can writing map[2] at the same time ?
The thread lock the whole map data structure or only an element, e.g. map[1] or map[2].
thanks
If you can guarantee that nobody is modifying the container itself (via insert and erase etc.), then as long as each thread accesses a different element of the container, you should be fine.
If you need per-element locking, you could modify the element type to something that offers synchronized access. (Worst case a pair of a mutex and the original value.)
You need a different mutex for every element of the map. You can do this with a map of mutex or adding a mutex to the mapped type (in your case it is int, so you can't do it without creating a new class like SharedInt)
Mutexes lock executable regions not objects. I always think about locking any code regions that read/modify thread objects. If an object is locked within a region but that object is accessible within another un-synchronized code region, you are not safe (ofcourse). In your case, I'd lock access to the entire object as insertions and reading from containers can easily experience context switching and thus increase the likelihood of data corruption.
Mutex is all about discipline. One thread can call write and other thread can call write1. C++ runtime will assume it is intentional. But most of the cases it is not the programmer intended. Summary is as long as all threads/methods follow the discipline (understand the the critical section and respect it) there will be consistency.
int i=0;
Write()
{
//Lock
i++;
//Unlock
}
Write1()
{
i++;
}