I have a std::map that is used by multiple threads to store data. The map is declared like this:
std::map<int, Call> calls;
From each thread, I have to acquire a mutex lock, obtain a pointer or reference to the object belonging to that thread, then release the mutex lock. I can modify the object after that because each object is used only by one thread. As soon as the thread dies, the corresponding pair in the map would also get deleted.
I would like to know the best way to implement this. I was thinking of two ways:
1) I know this one could look outrageously crazy, but still
std::map<int, Call> calls;
...
{
mutex.lock();
Call* callptr = &calls[id];
mutex.unlock();
// use callptr
}
or 2) I think this one looks more sensible
std::map<int, std::auto_ptr<Call> > calls;
...
{
mutex.lock();
std::auto_ptr<Call> callptr = map[id];
mutex.unlock();
// use callptr
mutex.lock();
map[id] = callptr;
mutex.unlock();
}
The threads actually are created in a different dll and I don't have the code for that. This dll that I'm writing now gets imported by that dll and used. So it has to be implemented with std::map only, but could anyone tell me if one of these methods is ok or if there are ways to make it more stable.
Thanks
You should use iterators:
mutex.lock();
std::map<int, Call>::iterator callptr = calls.find(id);
callptr->second.foo();
...
mutex.unlock();
Your first solution with pointers is problematic, because the lifetime of the object in the map is uncertain - it may be moved when the tree is rebalanced when elements are inserted or deleted.
Your second solution won't work at all, because std::auto_ptr does not fulfil the requirements for mapped_type of std::map - mostly because its copy constructor and operator= don't actually copy. You likely won't get a compiler error, but you'll get very weird behavior at run-time.
According to me Thread Local storage is the best option for you.
If you need thread specific data, you can make use of Thread Local Storage and completely eliminate the need for map and mutex locking.
Don't use auto_ptr's in STL containers. DON'T. The implementors are actually required to try and make its use there a compilation error. Standard containers tend to copy things around. That's a very wrong thing with auto_ptr.
Related
I'm using boost to start a thread and the thread function is a member function of my class just like this:
class MyClass {
public:
void ThreadFunc();
void StartThread() {
worker_thread_ = boost::shared_ptr<boost::thread>(
new boost::thread(boost::bind(&MyClass::ThreadFunc, this)));
}
};
I will access some member variables in ThreadFunc:
while (!stop) {
Sleep(1000); // here will be some block operations
auto it = this->list.begin();
if (it != this->list.end())
...
}
I can not wait forever for thread return, so I set timeout:
stop = true;
worker_thread_->interrupt();
worker_thread_->timed_join(boost::posix_time::milliseconds(timeout_ms));
After timeout, I will delete this MyClass pointer. Here will be a problem, the ThreadFunc hasn't return, it will have chances to access this and its member variables. In my case, the iterator will be invalid and it != this->list.end() will be true, so my program will crash if using invalid iterator.
My question is how to avoid it ? Or how to check whether this is valid or member variables is valid ? Or could I set some flags to tell ThreadFunc the destructor has been called ?
There are a lot of possible solutions. One is to use a shared_ptr to the class and let the thread hold its own shared_ptr to the class. That way, the object will automatically get destroyed only when both threads are done with it.
How about you create a stopProcessing flag (make it atomic) as a member of MyClass and in your ThreadFunc method check at each cycle if this flag is set?
[EDIT: making clearer the answer]
There a two orthogonal problems:
stopping the processing (I lost my patience, stop now please). This can be arranged by setting a flag into MyClass and make ThreadFunc checking it as often as reasonable possible
deallocation of resources. This is best by using RAII - one example being the use of shared_ptr
Better keep them as separate concerns.
Combining them in a single operation may be possible, but risky.
E.g. if using shared_ptr, the once the joining thread decided "I had enough", it simply goes out of the block which keeps its "copy" of shared_ptr, thus the shared_ptr::use_count gets decremented. The thread function may notice this and decide to interpret it as an "caller had enough" and cut short the processing.However, this implies that, in the future releases, nobody else (but the two threads) may acquire a shared_ptr, otherwise the "contract" of 'decremented use_count means abort' is broken.
(a use_count==1 condition may still be usable - interpretation "Only me, the processing thread, seems to be interested in the results; no consumer for them, better abort the work").
I wonder, is it safe to implement like this? :
typedef shared_ptr<Foo> FooPtr;
FooPtr *gPtrToFooPtr // global variable
// init (before any thread has been created)
void init()
{
gPtrToFooPtr = new FooPtr(new Foo);
}
// thread A, B, C, ..., K
// Once thread Z execute read_and_drop(),
// no more call to read() from any thread.
// But it is possible even after read_and_drop() has returned,
// some thread is still in read() function.
void read()
{
FooPtr a = *gPtrToFooPtr;
// do useful things (read only)
}
// thread Z (executed once)
void read_and_drop()
{
FooPtr b = *gPtrToFooPtr;
// do useful things with a (read only)
b.reset();
}
We do not know which thread would do the actual realease.
Does boost's shared_ptr do the release safely under circumstance like this?
According to boost's document, thread safety of shared_ptr is:
A shared_ptr instance can be "read" (accessed using only const
operations) simultaneously by multiple threads. Different shared_ptr
instances can be "written to" (accessed using mutable operations such
as operator= or reset) simultaneosly by multiple threads.
As far as I am concerned, the code above does not violate any of thread safety criteria I mentioned above. And I believe the code should run fine. Does anyone tell me if I am right or wrong?
Thanks in advance.
Editted 2012-06-20 01:00 UTC+9
The pseudo code above works fine. The shared_ptr implementation guarantees to work correctly under circumstances where multiple thread is accessing instances of it (each thread MUST access its own instance of shared_ptr instantiated by using copy constructor).
Note that in the pseudo code above, you must delete gPtrToFooPtr to have the shared_ptr implementation finally release (drop the reference count by one) the object it owns(not proper expression since it is not an auto_ptr, but who cares ;) ). And in this case, you must be aware of the fact that it may cause SIGSEGV in multithreaded application.
How do you define 'safe' here? If you define it as 'I want to make sure that the object is destroyed exactly once', then YES, the release is safe. However, the problem is that the two threads share one smart pointer in your example. This is not safe at all. The reset() performed by one thread might not be visible to the other thread.
As stated by the documentation, smart pointers offer the same guarantees as built in types (i.e., pointers). Therefore, it is problematic to perform an unguarded write while an other thread might still be reading. It is undefined when that other reading thread will see writes of the other one. Therefore, while one thread calls reset() the pointer might NOT be reset in the other thread, since the shared_ptr instance itself is shared.
If you want some sort of thread safety, you have to use two shared pointer instances. Then, of course, resetting one of them WILL NOT release the object, since the other thread still has a reference to it. Usually this behaviour is intended.
However, I think the bigger problem is that you are misusing shared_ptrs. It is quite uncommon to use pointers of shared_ptrs and to allocate the shared_ptr on the heap (using new). If you do that, you have the problem you wanted to avoid using smart pointers again (you have to manage the lifetime of the shared_ptr now). Maybe check out some example code about smart pointers and their usage first.
For your own good, I will be honest.
Your code is doing many things and almost all are simply useless and absurd.
typedef shared_ptr<Foo> FooPtr;
FooPtr *gPtrToFooPtr // global variable
A raw pointer to a smart pointer, cancels the advantage of automatic resource management and does not solve any problem.
void read()
{
FooPtr a = *gPtrToFooPtr;
// do useful things (read only)
}
a is not used in any meaningful way.
{
FooPtr b = ...
b.reset();
}
b.reset() is useless here, b is about to be destroyed anyway. b has no purpose in this function.
I am afraid you have no idea what you are doing, what smart pointers are for, how to use shared_ptr, and how to do MT programming; so, you end up with this absurd pile of useless features to not solve the problem.
What about doing simple things simply:
Foo f;
// called before others functions
void init() {
// prepare f
}
// called in many threads {R1, R2, ... Rn} in parallel
void read()
{
// use f (read-only)
}
// called after all threads {R1, R2, ... Rn} have terminated
void read_and_drop()
{
// reset f
}
read_and_drop() must not be called before it can be guaranteed that other threads are not reading f.
To your edit:
Why not call reset() first on the global shared_ptr?
If you were the last one to access the object, fine it is deleted, then you delete the shared_ptr on the heap.
If some other thread still uses it, you reduce the ref count by one, and "disconnect" the global ptr from the (still existing) object that is pointed-to. You can then safely delete the shared_ptr on the heap without affecting any thread that might still use it.
I am working on a set that is frequently read but rarely written.
class A {
boost::shared_ptr<std::set<int> > _mySet;
public:
void add(int v) {
boost::shared_ptr<std::set<int> > tmpSet(new std::set<int>(*_mySet));
tmpSet->insert(v); // insert to tmpSet
_mySet = tmpSet; // swap _mySet
}
void check(int v) {
boost::shared_ptr<std::set<int> > theSet = _mySet;
if (theSet->find(v) != theSet->end()) {
// do something irrelevant
}
}
};
In the class, add() is only called by one thread and check() is called by many threads. check() does not care whether _mySet is the latest or not. Is the class thread-safe? Is it possible that the thread executing check() would observe swap _mySet happening before insert to tmpSet?
This is an interesting use of shared_ptr to implement thread safety.
Whether it is OK depends on the thread-safety guarantees of
boost::shared_ptr. In particular, does it establish some sort of
fence or membar, so that you are guaranteed that all of the writes in
the constructor and insert functions of set occur before any
modification of the pointer value becomes visible.
I can find no thread safety guarantees whatsoever in the Boost
documentation of smart pointers. This surprizes me, as I was sure that
there was some. But a quick look at the sources for 1.47.0 show none,
and that any use of boost::shared_ptr in a threaded environment will
fail. (Could someone please point me to what I'm missing. I can't
believe that boost::shared_ptr has ignored threading.)
Anyway, there are three possibilities: you can't use the shared pointer
in a threaded environment (which seems to be the case), the shared
pointer ensures its own internal consistency in a threaded environment,
but doesn't establish ordering with regards to other objects, or the
shared pointer establishes full ordering. Only in the last case will
your code be safe as is. In the first case, you'll need some form of
lock around everything, and in the second, you'll need some sort of
fences or membar to ensure that the necessary writes are actually done
before publishing the new version, and that they will be seen before
trying to read it.
You do need synchronization, it is not thread safe. Generally it doesn't matter, even something as simple as shared += value; is not thread safe.
look here for example with regards to thread safety of shared_ptr: Is boost shared_ptr <XXX> thread safe?
I would also question your allocation/swapping in add() and use of shared_ptr in check()
update:
I went back and re-rad dox for shared_ptr ... It is most likely thread-safe in your particular since the reference counting for shared_ptr is thread-safe. However you are doing (IMHO) unnecessary complexity by not using read/write lock.
Eventually this code should be thread safe:
atomic_store(&_my_set,tmpSet);
and
theSet = atomic_load(&_mySet);
(instead of simple assignments)
But I don't know the current status of atomicity support for shared_ptr.
Note, that adding atomicity to shared_ptr in lock-free manner is really dificult thing; so even atomicity is implemented it may relay on mutexes or usermode spinlocks and, therefore, may sometimes suffer from performance issues
Edit: Perhaps, volatile qualifier for _my_set member variable should also be added.. but I'm not sure that it is strictly required by semantics of atomic operations
I'm aware, that I need to use mutex, when I perform operations on single STL container inside multiple threads. However I want to know if there are any exceptions from this rule. Please consider simplified scenario I'm trying to implement.
I have multiple threads adding elements to container and operation is surrounded with mutex lock/unlock. Then threads notify somehow (e.g. using eventfd on linux) single thread dedicated to dispatch elements in this container. What I want to do is to access first element in container without using mutex. Sample code based on deque but note that I ca use any container with queue capability:
std::mutex locker;
std:deque<int> int_queue;
int fd; // eventfd
eventfd_t buffer;
bool some_condition;
Thread 1, 2, 3, etc.
locker.lock ();
int_queue.push_back (1);
locker.unlock ();
eventfd_write (fd, 1);
Thread dedicated to dispatch elements:
while (true)
{
bool some_condition (true);
locker.lock ();
if (int_quque.empty () == false)
{
locker.unlock ();
}
else
{
locker.unlock ();
eventfd_read (fd, &buffer);
}
while (some_condition)
{
int& data (int_queue.front ());
some_condition = some_operation (data); // [1]
}
locker.lock ();
int_queue.pop ();
locker.unlock ();
}
[1] I will do some_operation() on signle element many times, that's why I want to avoid mutex locking here. It's to expensive.
I want to know if this code can lead to any synchronisation problems or something.
What you need is reference stability. I.e. you can use containers this way if the reference to the first element is not invalidated when the container is push_back'd. And even then, you'll want to obtain the reference to the front element under the lock.
I'm more familiar with std::condition_variable for the event notification, so I'll use that:
#include <mutex>
#include <condition_variable>
#include <deque>
std::mutex locker;
std::deque<int> int_queue;
std::condition_variable cv;
void thread_1_2_3()
{
// use lock_guard instead of explicit lock/unlock
// for exception safety
std::lock_guard<std::mutex> lk(locker);
int_queue_.push_back(1);
cv.notify_one();
}
void dispatch()
{
while (true)
{
bool some_condition = true;
std::unique_lock<std::mutex> lk(locker);
while (int_queue.empty())
cv.wait(lk);
// get reference to front under lock
int& data = int_queue.front();
lk.unlock();
// now use the reference without worry
while (some_condition)
some_condition = some_operation(data);
lk.lock();
int_queue.pop_front();
}
}
23.3.3.4 [deque.modifiers] says this about push_back:
An insertion at either end of the deque invalidates all the iterators
to the deque, but has no effect on the validity of references to
elements of the deque.
That is the key to allowing you to hang onto that reference outside of the lock. If thread_1_2_3 starts inserting or erasing in the middle, then you can no longer hang on to this reference.
You can't use a vector this way. But you could use a list this way. Check each container you want to use this way for reference stability.
I can't really see through your question or your code, but in general, the containers in the standard C++ library offer you a loose guarantee that concurrent access at different elements is thread-safe. Be sure to understand the implications and limitations of that, though: If you have a random-access container, or iterators to elements, and you only use those to read or change an element value, then as long as you're doing that at different elements, the result should be well-defined. What isn't OK is changing the container itself, so any erase or insert operations have to be serialized (e.g. by locking access to the entire container), and be sure to understand your container's iterator and reference invalidation rules when you do that.
For individual containers you might be able to say a bit more - for example, insert/erase in a tree-based container, and insert/erase in the middle of a random-access container almost certainly requires a global lock. In a vector/deque you'll need to reacquire iterators. In a list, you might get away with performing insertions concurrently at distinct locations.
Any global operations like size() and empty() need to be serialized as well.
For this particular example this is not safe
int& data (int_queue.front ());
You take a reference to the first element, it could be moved by another thread adding element adding to the queue forcing it to re allocate (deques are typically implemented as "wrap-around" arrays). If you copy the value as opposed to taking a reference, depending on the implementation you might get away with it. If you want to be able to do this, a std::deque doesn't come with any standard "exceptions" to this rule. It's certainly possible to write a data structure similar to a deque where this would be safe, but a deque is not guaranteed to be written like (and is unlikley to be written like) that.
Why do you want to do this? Why does the consumer thread not extract the object within the lock, and then process it out of band?
Assuming that what you want to avoid is having to copy the object outside of the container, a simpler easier to maintain approach could be dynamically allocating the objects, using a container of (smart) pointers and extracting it within the lock (minimal cost). Then you no longer need to consider thread safety issues.
Note that even if you might be able to pull this off in this particular scenario, you cannot use more than one consumer thread. I would recommend against the approach and just find a different approach where you can meet your requirements without walking over the bleeding edge. Multithreading is hard to do right, and very hard to debug or even detect that there is an issue. By adhering to common patterns you make your code easier to reason about and to maintain.
If you do want to a lock free queue, I also recommend you look at http://drdobbs.com/cpp/210604448?pgno=2
I have an unordered map which stores a pointer of objects. I am not sure whether I am doing the correct thing to maintain the thread safety.
typedef std::unordered_map<string, classA*>MAP1;
MAP1 map1;
pthread_mutex_lock(&mutexA)
if(map1.find(id) != map1.end())
{
pthread_mutex_unlock(&mutexA); //already exist, not adding items
}
else
{
classA* obj1 = new classA;
map1[id] = obj1;
obj1->obtainMutex(); //Should I create a mutex for each object so that I could obtain mutex when I am going to update fields for obj1?
pthread_mutex_unlock(&mutexA); //release mutex for unordered_map so that other threads could access other object
obj1->field1 = 1;
performOperation(obj1); //takes some time
obj1->releaseMutex(); //release mutex after updating obj1
}
Several thoughts.
If you do have one mutex per stored object, then you should try to create that mutex in the constructor for the stored object. In other words, to maintain encapsulation, you should avoid having external code manipulate that mutex. I would convert "field1" into a setter "SetField1" that handles the mutex internally.
Next, I agree with the comment that you could move pthread_mutex_unlock(&mutexA); to occur before obj1->obtainMutex();
Finally, I don't think you need obtainMutex at all. Your code looks as if only one thread will ever be allowed to create an object, and therefore only one thread will manipulate the contents during object creation. So if I consider only what little code you've shown here, it does not seem that mutex-per-object is needed at all.
One problem I see with the code is that it will lead to problems especially when exceptions occur.
obj1->obtainMutex(); //Should I create a mutex for each object so that I could obtain mutex when I am going to update fields for obj1?
pthread_mutex_unlock(&mutexA); //release mutex for unordered_map so that other threads could access other object
obj1->field1 = 1;
performOperation(obj1);
If performOperation throws an exception then obj1->releaseMutex(); will never get called thus leaving the object locked and potentially leading to deadlocks sometime in the future.
And even if you do not use exceptions yourself some library code you use in performOperation might. Or you might mistakenly sometime in the future insert a return and forget to unlock all owned locks before and so on...
The same goes for the pthread_mutex_lock and pthread_mutex_unlock calls.
I would recommend using RAII for locking / unlocking.
I.e. the code could look like this:
typedef std::unordered_map<string, classA*>MAP1;
MAP1 map1;
Lock mapLock(&mutexA); //automatci variable. The destructor of the Lock class
//automatically calls pthread_mutex_unlock in its destructor if it "owns" the
//mutex
if(map1.find(id) == map1.end())
{
classA* obj1 = new classA;
map1[id] = obj1;
Lock objLock(obj);
mapLock.release(); //we explicitly release mapLock here
obj1->field1 = 1;
performOperation(obj1); //takes some time
}
I.e. for a reference for some minimalistic RAAI threading support please refer to "Modern C++ design: generic programming and design patterns applied" by Andrei Alexandrescu (see here). Other resources also exist (here)
I will try to describe in the end one other problem I see with the code. More exactly, the problem I see with having the obtainMutex and releaseMutex as methods and calling them explicitly. Let's imagine thread 1 locks the map, creates an object calls obtainMutex and unlocks the map. Another thread (lets call it Thread 2) gets scheduled for execution locks the map obtains an iterator to the map1[id] of the object and calls releaseMutex() on the pObject (i.e. let's say due to a bug the code does not attempt to call obtainMutex first). Now Thread 1 gets scheduled and calls at some point releaseMutex() also. So the object got locked once but released twice. What I am trying to say is that it's going to be hard work making sure the calls are always correctly paired in the face of exceptions, potential early returns that do not unlock and incorrect usage of the object locking interface. Also Thread 2 might just delete the pObject it obtained from the map and erase it from the map. thread 1 will then go on an work with an already deleted object.
When used judiciously RAII would make the code simpler to understand (even shorter if you compare our versions) and also help a lot with some of the problems I enumerated above.
Thought of combining my comments into an answer:
1) When you are adding an entry, and therefore are modifying the container, you should not allow read access from other threads, as the container may be in a transition between legal states. Complementary, you should not modify the container when other threads are reading it. This calls for the use of read-write lock. The pseudo-code is something like:
set read lock
search container
if found
release read lock
operate on the found object
else
set write lock
release read lock
add entry
release write lock
endif
(it's been some time since I've done multi-threaded programming, so I may be rusty on details)
2) When I worked on MSVC some years ago we used the multi-threaded (i.e. thread-safe) version of the standard libraries. It could save you all this trouble. Didn't bother (yet) to check if thread-safe std exists also on gcc/Linux.