Mutex when writing to queue held in map for thread safety - c++

Question

I have a map<int, queue<int>> with one thread writing into it i.e. pushing messages into the queues. They key refers to a client_id, and the queue holds messages for the client. I am looking to make this read-write thread safe.
Currently, the thread that writes into it does something like this
map<int, queue<int>> msg_map;
if (msg_map.find(client_id) != msg_map.end())
{
queue<int> dummy_queue;
dummy_queue.push(msg); //msg is an int
msg_map.insert(make_pair(client_id, dummy_queue);
}
else
{
msg_map[client_id].push(msg);
}
There are many clients reading - and removing - from this map.
if (msg_map.find(client_id) != msg_map.end())
{
if (!msg_map.find(client_id)->second.empty())
{
int msg_rxed = msg_map[client_id].front();
//processing message
msg_map[client_id].pop();
}
}
I am reading this on mutexes (haven't used them before) and I was wondering when and where I ought to lock the mutex. My confusion lies in the fact that they are accessing individual queues (held within the same map). Do I lock the queues, or the map?
Is there a standard/accepted way to do this - and is using a mutex the best way to do this? There are '0s of client threads, and just that 1 single writing thread.

Answer 1

Simplifying and optimizing your code
For now we'll not concern ourselves with mutexes, we'll handle that later when the code is cleaned up a bit (it will be easier then).
First, from the code you showed there seems to be no reason to use an ordered std::map (logarithmic complexity), you could use the much more efficient std::unordered_map (average constant-time complexity). The choice is entirely up to you, if you don't need the container to be ordered you just have to change its declaration:
std::map<int, std::queue<int>> msg_map;
// or
std::unordered_map<int, std::queue<int>> msg_map; // C++11 only though
Now, maps are quite efficient by design but if you insist on doing lookups for each and every operation then you lose all the advantage of maps.
Concerning the writer thread, all your block of code (for the writer) can be efficiently replaced by just this line:
msg_map[client_id].push(msg);
Note that operator[] for both std::map and std::unordered_map is defined as:
Inserts a new element to the container using key as the key and a default constructed mapped value and returns a reference to the newly constructed mapped value. If an element with key key already exists, no insertion is performed and a reference to its mapped value is returned.
Concerning your reader threads, you can't directly use operator[] because it would create a new entry if none currently exists for a specific client_id so instead, you need to cache the iterator returned by find in order to reuse it and thus avoid useless lookups:
auto iter = msg_map.find(client_id);
// iter will be either std::map<int, std::queue<int>>::iterator
// or std::unordered_map<int, std::queue<int>>::iterator
if (iter != msg_map.end()) {
std::queue<int>& q = iter->second;
if (!q.empty()) {
int msg = q.front();
q.pop();
// process msg
}
}
The reason why I pop the message immediately, before processing it, is because it will improve concurrency when we add mutexes (we can unlock the mutex sooner, which is always good).
Making the code thread-safe
#hmjd's idea about multiple locks (one for the map, and one per queue) is interesting, but based on the code you showed us I disagree: any benefit you'll get from the additional concurrency will quite probably be negated by the additional time it takes to lock the queue mutexes (indeed, locking mutexes is a very expensive operation), not to mention the additional code complexity you'll have to handle. I'll bet my money on a single mutex (protecting the map and all the queues at once) being more efficient.
Incidentally, a single mutex solves the iterator invalidation problem if you want to use the more efficient std::unordered_map (std::map doesn't suffer from that problem though).
Assuming C++11, just declare a std::mutex along with your map:
std::mutex msg_map_mutex;
std::map<int, std::queue<int>> msg_map; // or std::unordered_map
Protecting the writer thread is quite straightforward, just lock the mutex before accessing the map:
std::lock_guard<std::mutex> lock(msg_map_mutex);
// the lock is held while the lock_guard object stays in scope
msg_map[client_id].push(msg);
Protecting the reader threads is barely any harder, the only trick is that you'll probably want to unlock the mutex ASAP in order to improve concurrency so you'll have to use std::unique_lock (which can be unlocked early) instead of std::lock_guard (which can only unlock when it goes out of scope):
std::unique_lock<std::mutex> lock(msg_map_mutex);
auto iter = msg_map.find(client_id);
if (iter != msg_map.end()) {
std::queue<int>& q = iter->second;
if (!q.empty()) {
int msg = q.front();
q.pop();
// assuming you don't need to access the map from now on, let's unlock
lock.unlock();
// process msg, other threads can access the map concurrently
}
}
If you can't use C++11, you'll have to replace std::mutex et al. with whatever your platform provides (pthreads, Win32, ...) or with the boost equivalent (which has the advantage of being as portable and as easy to use as the new C++11 classes, unlike the platform-specific primitives).

Answer 2

Read and write access to both the map and the queue need synchronized as both structures are being modified, including the map:
map<int, queue<int>> msg_map;
if (msg_map.find(client_id) != msg_map.end())
{
queue<int> dummy_queue;
dummy_queue.push(msg); //msg is an int
msg_map.insert(make_pair(client_id, dummy_queue);
}
else
{
msg_map[client_id].push(msg); // Modified here.
}
Two options are a mutex that locks both the map and queue or have a mutex for the map and a mutex per queue. The second approach is preferable as it reduces the length of time a single lock is held and means multiple threads can be updating several queues concurrently.