I am reading Effective C++, in Rule 14: Think carefully about copying behavior in resource-managing classes, there is an example:
class Lock {
public:
explicit Lock(Mutex* pm) : mutexPtr(pm) {
lock(mutexPtr);
}
~Lock() {
unlock(mutexPtr);
}
private:
Mutex *mutexPtr;
};
It points out that if we construct the Lock as above, there will be a problem if we run the code below:
Mutex m;
Lock ml1(&m);
Lock ml2(ml1);
I think it may because the code may runs like below:
// ml1 constructes
lock(m)
// copy ml2, but ml1.mutexPtr and ml2.mutexPtr both point to m
ml2.mutexPtr = ml1.mutexPtr
// ml1 destructs
unlock(m)
// ml2 destructs
unlock(m)
So the m will be unlock for twice. So is it the real reason that cause the problem below? thx!
Yes, that is why the author is saying to be careful. If you use a recursive_mutex instead of plain mutex you can simply lock in the copy ctor and copy-assign operator, but if it's a non-recursive mutex it would likely be better to make the lock type non-copyable.
In Scott Meyers' "Effective Modern C++", he writes:
It's worth nothing that because std::mutex is a move-only type (i.e., a type that can be
moved, but not copied), a side effect of adding m to Polynomial is that Polynomial loses the ability to be copied. It can still be moved, however.
Here, he adds a std::mutex m as a member variable to a class Polynomial in the context of explaining why one should protect access to const member variable of a class(while multi-threading). I understood the concept he explained. but one thing which I need to have some more explanation is "why Adding a move only type to a class makes that class as move only type not copy-able"?, why types like std::mutex and std::atomic is move only type by default? how can we get over with it if we want to perform copy operation even when adding move only type of variable to our class?
"why Adding a move only type to a class makes that class as move only type not copy-able"
This is not entierly true. adding non copyable member variable prevents you from writing something like Type (const Type& t) = default; , but you can still implement you own copy constructor which ignores the non copyable member
class Type{
int integer;
std::mutex m;
public:
Type() = default;
Type(const Type& rhs): integer (rhs.integer){}
}
Other than this, you can find numerous examples of objects that contain move-only members, but they are themselves non movable, simply because the developer deleted the move constructor.
why types like std::mutex and std::atomic is move only type by default?
Neither of them implement move constructor. this is a wrong statement.
std::mutex is not copyable only because the standard says so. They could probably have written a std::mutex that was copyable, but they decided not to.
As a guess, a non-copyable std::mutex might be more efficient, safer, or easier to understand.
As an example, here is why it might be safer/easier to understand: What does it mean to copy a std::mutex that is locked? I, personally, have no idea what the right answer is; I suspect that the right answer is "that is nonsense". There is no "answer of least surprise".
Instead of having a surprising answer in concurrency code, by blocking copy we avoid the question. People storing a mutex who want to copy have to decide themselves what they want it to do.
C++ will under some circumstances automatically generate a copy constructor. Under more circumstances it will let you do MyClass(MyClass const&)=default and write one when you ask for it.
If your class contains a non-copyable class, it won't generate one, and you cannot ask for it. This is because the generated copy constructor basically copies its members; if you cannot copy a member, it cannot generate a copy constructor.
So mutex cannot be copied because the standard says so.
If a struct or class contains a non-copyable member, the default copy constructor cannot be used. You have to write one yourself explicitly.
If your type needs to store a mutex, one approach to prevent the headache of having to maintain a copy ctor that manually copies all of its other members is to stick your non-mutex state into a sub-struct, and then use its default copy. Then you decide what to do with the mutex during the copy, and your copy ctor (and assignment copy operator) remain simple.
struct bob_with_mutex {
sturct bob_simple_data {
int x, y, z;
std::vector<char> buff;
};
bob_simple_data data;
std::mutex m;
bob_with_mutex( bob_with_mutex const& o ):
data(o.data)
{}
bob_with_mutex& operator=( bob_with_mutex const& o )
{
data = o.data;
return *this;
}
};
the type bob has a mixture of a mutex and some data. The data is stored in a sub-structure. The copy ctor of bob_with_mutex cannot be defaulted, but it simply says "copy the data, and nothing else".
The copy of the bob has its own mutex.
One possibility is to change the mutex type to a pointer to a mutex. You can copy the pointer, but there will still only be one copy of the mutex. You may want to use a smart pointer.
I'm trying to make a class thread safe by using a mutex.
class Container
{
private:
vector<Foo> v;
boost::mutex m;
public:
void add(Foo item)
{
m.lock();
v.push_back(item);
m.unlock();
}
};
The problem is that boost::mutex is non-copyable, so this makes Container noncopyable. Of course if I copy Container the new instance presumably doesn't need to keep the same mutex as the old one - it can have a new mutex of its own. I could write custom copy constructors for Container that do this but in reality it's a complex class and I don't want to. So how about this:
class CopyableMutex
{
private:
boost::mutex m;
public:
CopyableMutex() {}
CopyableMutex(CopyableMutex&) {} //don't copy, just create a new one
CopyableMutex& operator=(CopyableMutex&) {return *this;} //don't assign, keep it the same
void lock() {m.lock();}
void unlock() {m.unlock();}
};
...and then replacing boost::mutex in Container with CopyableMutex.
Is this a hideous thing to do? If not then am I reinventing the wheel - is there a library class that does this already?
Yes it's hideous.
The correct solution to the problem is a custom copy constructor and assignment operator for your container.
If the class is "too complex" to write a custom copy constructor for, then separate the thread-safety from the container and have base class container which doesn't contain a mutex and, perhaps, a derived class "thread safe container" which contains a mutex and has a custom copy constructor and assignment op that just call through to the automatically generated base class ones.
I believe that one of the things that doing it this way invalidates is the correct way to use mutexes these days:
void func()
{
std::lock_guard<std::mutex> lock(mtx);
// do things
}
As you can not return a mutex and so there would be no way to up-scope it to where you want to use it. The reason for the above style of usage is to prevent issues that arise when you use a lock, and then some form of unhanded exception happens prior to you unlock (ie it ensures a more reliable unlock).
I agree with other answer though, that a better way would be to just encapsulate your thread safety part and isolate it from your complex code (if possible), and then just make explicit copy constructors for that smaller class.
I have a ConcurrentQueue class that is based around a user provided container with a constructor like this...
ConcurrentQueue(const ConcurrentQueue& other) : m_Queue(other.m_Queue) {}
But, I need to lock other's mutex while it's being copied.
Option 1:
So I could not use the copy constructor at all, and do...
ConcurrentQueue(const ConcurrentQueue& other) : m_Queue(other.m_Queue)
{
std::lock_guard<std::mutex> lock(other.m_Mutex);
m_Queue = other.m_Queue;
}
But I can't guarantee that copy assignment and copy construction are equivalent functionality.
Option 2:
I could have a private method...
std::queue<T, Container> GetQueue() const
{
std::lock_guard<std::mutex> lock(other.m_Mutex);
return m_Queue;
}
And then in the constructor do this...
ConcurrentQueue(const ConcurrentQueue& other) : m_Queue(other.GetQueue()) {}
But this potentially (depending on optimizations) uses m_Queue's copy constructor once and it's move constructor once. And I also can't guarantee that a copy and a move is equivalent to just a copy. Additionally the user provided container could be bizarre and be copyable but unmoveable, which would also cause this approach to have problems.
So, what do I do?
ConcurrrentQueue::ConcurrrentQueue(
ConcurrrentQueue const& other )
: m_Queue( (std::lock_guard<std::mutex>( other.m_Mutex ),
other.m_Queue ) )
{
}
should work.
Lock, create a copy of the content and then swap it with the member. At least that's the easiest and IMHO cleanest way. Another less clean way is to use the comma operator: (a, b) yields b, but if a is a scoped lock, the temporary will live until the next sequence point, i.e. until you have used b to initialize your local copy.
That said, there are two things to consider:
Maybe copying is not such a smart idea anyway and your design works as well if you just disable copying.
If you have access to the queue and you can read it for copying, doesn't that imply that the mutex must already be locked? If not, how are you sure that you really want to copy the queue? I don't question that there are answers that justify the design, but it's unusual.
I'm looking for the best-practice of dealing with non-copyable objects.
I have a mutex class, that obviously should not be copyable.
I added a private copy constructor to enforce that.
That broke the code - some places simply needed to be fixed, but I have a generic problem
where a class, using the mutex either as a data member, or by inheritance, is being inserted into a container.
This is usually happening during the container initialization, so the mutex is not initialized yet, and is therefore ok, but without a copy constructor it does not work. Changing the containers to contain pointers is not acceptable.
Any advise?
Three solutions here:
1. Use Pointers - The quick fix is to make it a container of pointers - e.g. a shared_ptr.
That would be the "good" solution if your objects are truly noncopyable, and using other containers is not possible.
2. Other containers - Alternatively, you could use non-copying containers (that use in-place-construction), however they aren't very common and largely incompatible with STL. (I've tried for a while, but it's simply no good)
That would be the "god" solution if your objects are truly noncopyable, and using pointers is not possible.
[edit] With C++13, std::vector allows inplace construction (emplace_back), and can be used for noncopyable objects that do implement move semantics.
[/edit]
3. Fix your copyability - If your class is copyable as such, and the mutex is not, you "simply" need to fix the copy constructor and assignment operator.
Writing them is a pain, since you usually have to copy & assign all members except the mutex, but that can often be simplified by:
template <typename TNonCopyable>
struct NeverCopy : public T
{
NeverCopy() {}
NeverCopy(T const & rhs) {}
NeverCopy<T> & operator=(T const & rhs) { return *this; }
}
And changing you mutex member to
NeverCopy<Mutex> m_mutex;
Unfortunately, using that template you lose special constructors of Mutex.
[edit] Warning: "Fixing" the Copy CTor/asignment often requires you to lock the right hand side on copy construct, and lock both sides on assignment. Unfortunately, there is no way to override the copy ctor/assignment and call the default implementation, so the NeverCopy trick might not work for you without external locking. (There are some other workarounds with their own limitations.)
If they are non-copyable, the container has to store (smart) pointers to those objects, or reference wrappers, etc, although with C++0x, noncopyable objects can still be movable (like boost threads), so that they can be stored in containers as-is.
to give examples: reference wrapper (aka boost::ref, a pointer under the hood)
#include <vector>
#include <tr1/functional>
struct Noncopy {
private:
Noncopy(const Noncopy&) {}
public:
Noncopy() {}
};
int main()
{
std::vector<std::tr1::reference_wrapper<Noncopy> > v;
Noncopy m;
v.push_back(std::tr1::reference_wrapper<Noncopy>(m));
}
C++0x, tested with gcc:
#include <vector>
struct Movable {
private:
Movable(const Movable&) = delete;
public:
Movable() {}
Movable(Movable&&) {}
};
int main()
{
std::vector<Movable> v;
Movable m;
v.emplace_back(std::move(m));
}
EDIT: Nevermind, C++0x FCD says, under 30.4.1/3,
A Mutex type shall not be copyable nor movable.
So you're better off with pointers to them. Smart or otherwise wrapped as necessary.
If an object is non-copyable then there is usually a good reason. And if there's a good reason then you shouldnt subvert that by putting it into a container that attempts to copy it.
There is no real answer to the question given how you've framed it. There is no way to do what you want. The actual answer is for the container to contain pointers, and you've said that isn't OK for some unspecified reason.
Some have talked about things being movable and using C++0x in which containers often require their elements to be movable, but do not require them to be copyable. I think this is a poor solution as well because I suspect that mutexes should not be moved while they are held, and this makes it effectively impossible to move them.
So, the only real remaining answer is to point at the mutexes. Use ::std::tr1::shared_ptr (in #include <tr1/memory>) or ::boost::shared_ptr to point at the mutexes. This requires changing the definitions of the classes that have the mutexes inside them, but it sounds like you're doing that anyway.
STL containers rely heavily on their contents being copyable, so either make them copyable or do not put them into container.
The best option is to use pointers or some type of wrapper class that uses pointers under the hood. That would allow these to be sanely copied, and actually do what a copy would be expected to do (share the mutex).
But, since you said no pointers, there is one other option. It sounds like Mutexes are "sometimes copyable" perhaps you should write a copy constructor and an assignment operator and have those throw an exception if a mutex is ever copied after it has been initialized. The down side is there's no way to know you're doing it wrong until runtime.
Use smart pointers like boost::shared_ptr or use another containers, like boost::intrusive. Both will require to modify your code, through.
Using a mutex in a class does not necessarily mean that the class has to be non-copyable. You can (almost) always implement it like this:
C::C (C const & c)
// No ctor-initializer here.
{
MutexLock guard (c.mutex);
// Do the copy-construction here.
x = c.x;
}
While this makes it somewhat possible to copy classes with mutexes, you probably should not do it. Chances are that your design will be better without per-instance mutex.
Using c++11 on Ubuntu 14.04 (which includes emplace_back), I've gotten this to work.
I found that emplace_back worked fine, but erase (and probably insert) didn't work because, when the vector was shuffling the elements along to fill in the gap, it mas using:
*previous = *current;
I found the trick was allowing move assignment in my resource class:
Watch& operator=(Watch &&other);
This is my inotify_watch class, which can live in a std::vector:
class Watch {
private:
int inotify_handle = 0;
int handle = -1;
// Erases all knowledge of our resources
void erase() {
inotify_handle = 0;
handle = -1;
}
public:
Watch(int inotify_handle, const char *path, uint32_t mask)
: inotify_handle(inotify_handle),
handle(inotify_add_watch(inotify_handle, path, mask)) {
if (handle == -1)
throw std::system_error(errno, std::system_category());
}
Watch(const Watch& other) = delete; // Can't copy it, it's a real resource
// Move is fine
Watch(Watch &&other)
: inotify_handle(other.inotify_handle), handle(other.handle) {
other.erase(); // Make the other one forget about our resources, so that
// when the destructor is called, it won't try to free them,
// as we own them now
}
// Move assignment is fine
Watch &operator=(Watch &&other) {
inotify_handle = other.inotify_handle;
handle = other.handle;
other.erase(); // Make the other one forget about our resources, so that
// when the destructor is called, it won't try to free them,
// as we own them now
return *this;
}
bool operator ==(const Watch& other) {
return (inotify_handle == other.inotify_handle) && (handle == other.handle);
}
~Watch() {
if (handle != -1) {
int result = inotify_rm_watch(inotify_handle, handle);
if (result == -1)
throw std::system_error(errno, std::system_category());
}
}
};
std::vector can not store non-copyable objects (due to resize) thus you can't store objects of type Foo:
struct Foo {
std::mutex mutex;
...
};
One way around this is to use std::unique_ptr:
struct Foo {
std::unique_ptr<std::mutex> pmutex;
Foo() : pmutex{std::make_unique<std::mutex>()} {}
...
};
Another option is to use a std::deque which can hold non-copyable objects (like instances of the first version of Foo above. Typically use emplace_back method to construct objects "in place" to add elements -- no copies happen. For example,
objects of type Foo here do not need to be copiable:
struct FooPool {
std::deque<Foo> objects;
ObjectPool(std::initializer_list<T> argList) {
for (auto&& arg : argList)
objects.emplace_back(arg);
...
};