I'm trying to implement what I think is a fairly simple design. I have a bunch of objects, each containing a std::map and there will be multiple processes accessing them. I want to make sure that there is only one insert/erase to each of these maps at a time.
So I've been reading about boost::thread and class member mutexes and using bind to pass to class member which are all new things to me. I started with a simple example from a Dr. Dobbs article and tried modifying that. I was getting all kinds of compiler errors due to my Threaded object having to be noncopyable. After reading up on that, I decided I can avoid the hassle by keeping a pointer to a mutex instead. So now I have code that compiles but results in the following error:
/usr/include/boost/shared_ptr.hpp:419:
T* boost::shared_ptr< <template-parameter-1-1> >::operator->() const
[with T = boost::mutex]: Assertion `px != 0' failed. Abort
Now I'm really stuck and would really appreciate help with the code as well as comments on where I'm going wrong conceptually. I realize there are some answered questions around these issues here already but I guess I'm still missing something.
#include <boost/thread/thread.hpp>
#include <boost/thread/mutex.hpp>
#include <boost/bind.hpp>
#include <boost/shared_ptr.hpp>
#include <iostream>
#include <map>
using namespace std;
class Threaded {
public:
std::map<int,int> _tsMap;
void count(int id) {
for (int i = 0; i < 100; ++i) {
_mx->lock();
//std::cout << id << ": " << i << std::endl;
_tsMap[i] ++;
_mx->unlock();
}
}
private:
boost::shared_ptr<boost::mutex> _mx;
};
int main(int argc, char* argv[]) {
Threaded th;
int i = 1;
boost::thread thrd1(boost::bind(&Threaded::count, &th, 1));
//boost::thread thrd2(boost::bind(&th.count, 2));
thrd1.join();
//thrd2.join();
return 0;
}
It looks like you're missing a constructor in your Threaded class that creates the mutex that _mx is intended to point at. In its current state (assuming you ran this code just as it is), the default constructor for Threaded calls the default constructor for shared_ptr, resulting in a null pointer (which is then dereferenced in your count() function.
You should add a constructor along the following lines:
Threaded::Threaded(int id)
: _mx(new boost::mutex())
, _mID(id)
{
}
Then you could remove the argument from your count function as well.
A mutex is non-copyable for good reasons. Trying to outsmart the compiler by using a pointer to a mutex is a really bad idea. If you succeed, the compiler will fail to notice the problems, but they will still be there and will turn round and bite you at runtime.
There are two solutions
store the mutex in your class as a static
store the mutex outside your class.
There are advantages for both - I prefer the second.
For some more discussion of this, see my answer here mutexes with objects
Conceptually, I think you do have a problem. Copying a std::shared_ptr will just increase its reference count, and the different objects will all use the same underlying mutex - meaning that whenever one of your objects is used, none of the rest of them can be used.
You, on the other hand, need each object to get its own mutex guard which is unrelated to other objects mutex guards.
What you need is to keep the mutex defined in the class private section as it is - but ensure that your copy constructor and copy assignment operator are overloaded to create a new one from scratch - one bearing no relation to the mutex in the object being copied/assigned from.
Related
Languages like Swift, Vala and C++ (through shared_ptr) manage memory by reference counting. As far as i know updates to the reference count in these systems are performed atomically and thus thread-safe.
However, each time a reference/pointer is reassigned, the former referenced object needs a reference count decrement, the newly referenced object a reference increment, and finally the reference itself must be reassigned. So if the same reference is accessible from multiple threads (i.e. through a global variable) and is reassigned by multiple threads at the same time, reference counts might become garbled.
So do C++ shared pointers, Vala references, Swift references take steps to avoid such problems? If not what steps are necessary in each of the three languages to make such access safe?
Any insights are appreciated. Thanks!
see the last paragraph of http://en.cppreference.com/w/cpp/memory/shared_ptr
All member functions (including copy constructor and copy assignment) can be called by multiple threads on different instances of shared_ptr without additional synchronization even if these instances are copies and share ownership of the same object. If multiple threads of execution access the same shared_ptr without synchronization and any of those accesses uses a non-const member function of shared_ptr then a data race will occur; the shared_ptr overloads of atomic functions can be used to prevent the data race.
A shared_ptr variable is not thread safe and shouldn't be accessed from multiple threads if one or more threads modify the variable. Multiple variables managing the same pointer are atomic and each thread is free to modify its own copy of the shared_ptr.
For example this is not safe:
#include <iostream>
#include <string>
#include <memory>
#include <vector>
#include <thread>
int main()
{
std::shared_ptr< std::string > str( new std::string() );
std::vector< std::thread > threads;
for ( int i = 0; i < 10; i++ )
{
threads.emplace_back([&]
{
if ( str->empty() )
{
str.reset( new std::string( "thread string" ) );
}
else
{
str.reset();
}
});
}
for ( auto& thread : threads )
{
thread.join();
}
}
but this is as the threads don't modify the str variable but do increase its reference count:
#include <iostream>
#include <string>
#include <memory>
#include <vector>
#include <thread>
int main()
{
std::shared_ptr< std::string > str( new std::string() );
std::vector< std::thread > threads;
for ( int i = 0; i < 10; i++ )
{
threads.emplace_back([&]
{
std::shared_ptr< std::string > str2 = str;
if ( str2->empty() )
{
str2.reset( new std::string( "thread string" ) );
}
else
{
str2.reset();
}
});
}
for ( auto& thread : threads )
{
thread.join();
}
}
C++20 adds std::atomic_shared_ptr which is completely thread safe. Before that you can use the atomic non member functions.
Reference counting is threadsafe in Swift, because the underlying NSObject is threadsafe. In that case the reference count is an inherent property of the the object itself so your question is moot. It looks like the same is true of Vala.
Which leaves C++, always late to the ball.
std::shared_ptr's implementation of reference counting is threadsafe, as the quote in Alan's post makes clear, but telling one to start looking after a different object apparently is not.
It is not common to do this. That rather undermines it's purpose, certainly if you're trying it make your code threadsafe. More details at cppreference - some of those overloads of operator= are threadsafe and some are not.
I have a class called MatrixAlt and i'm trying to multi thread a function to do some work on that matrix.
My general method worked when I just implemented it in a couple of functions. But when I try to bring it into the class methods, I get an error.
The problematic line (or where it highlights anyway) is 4 lines from the end and the error message is in the comments just above it.
#include <vector>
#include <future>
#include <thread>
class MatrixAlt
{
public:
MatrixAlt();
// initilaise the matrix to constant value for each entry
void function01(size_t maxThreads);
void function02(size_t threadIndex);
};
MatrixAlt::MatrixAlt()
{
}
void MatrixAlt::function02(size_t threadIndex)
{
// do some stuff
return;
}
void MatrixAlt::function01(size_t maxThreads)
{
// To control async threads and their results
std::vector<std::future<bool>> threadsIssued;
// now loop through all the threads and orchestrate the work to be done
for (size_t threadIndex = 0; threadIndex < maxThreads; ++threadIndex)
{
// line 42 gives error:
// 'MatrixAlt::function02': non-standard syntax; use '&' to create a pointer to member
// 'std::async': no matching overloaded function found
threadsIssued.push_back(std::async(function02, threadIndex));
}
return;
}
Your first problem is solved like this
threadsIssued.push_back(std::async(&MatrixAlt::function02, this, threadIndex));
You need to specify the exact class::function and take its address and which instance of the class your doing it for, and then the parameters.
The second problem which you haven't see yet is this line
std::vector<std::future<bool>> threadsIssued;
All those futures will be lost in scope exit, like tears in rain. Time to destroy.
Freely after Blade runner.
All those moments will be lost in time, like tears in rain. Time to
die.
Whenever you have a member function in C++, that function takes the object itself as an implicit first argument. So you need to pass the object as well, but even then, it can't be called with the same syntax as a normal function that takes the object.
The simplest way to setup an asynchronous job in C++ is typically just to use lambdas. They've very clear and explicit. So, for example, you could change your call to:
threadsIssued.push_back(std::async([this] (size_t t) { this->function02(t);}, threadIndex));
This lambda is explicitly capturing the this pointer, which tells us that all of the function02 calls will be called on the same object that the calling function01 is called on.
In addition to being correct, and explicit, this also helps highlight an important point: all of the function02 objects will be running with mutable access to the same MatrixAlt object. This is very dangerous, so you need to make sure that function02 is thread safe, one way or another (usually easy if its conceptually const, otherwise perhaps need a mutex, or something else).
I'm working with some multithreaded code (working with concurrent data structures), and part of it requires that I lock a set of mutexes. For my implementation I am utilizing a vector of lock_guards, since I don't necessarily know how many mutexes I'm going to need to lock, and I may run into exception conditions which will force me to unlock all of my mutexes and restart. Hence the reason for the vector.
The code I'm trying to use boils down to this:
#include <mutex>
#include <vector>
using namespace std;
int main( int argc, char** argv ) {
vector<recursive_mutex> vec(10);
vector<lock_guard<recursive_mutex>> lgv;
for( auto it = vec.begin(); it != vec.end(); ++it ) {
lgv.emplace_back( *it );
}
return 0;
}
When I try to compile this (G++ 5.3.1 using --std=c++11), I get the following error (somewhat distilled):
In file included from foo.cpp:1:0:
/usr/include/c++/5.3.1/mutex:385:7: note: declared here
lock_guard(const lock_guard&) = delete;
Based upon my understanding of emplace_back, the library should NOT be attempting to use the copy constructor for lock_guard -- it should be performing a construct-in-place. Am I understanding what SHOULD be happening properly, or is my understanding flawed?
FYI, I have attempted to use a unique_lock and this will compile just fine. However, I'm curious about this (apparent) discrepancy.
I assume, the question reflects an attemtpt to use unique_lock, which OP is saying works. The same example with lock_guard would not work, since std::vector::emplace_back requires the type to be both MoveInsertable and EmplaceConstructible, and std::lock_guard does not suit.
I am trying to pass my class instance into threads and the return the processed objects from threads. I've googled about C++ multithreading, and found that std::promising can be helpful.
However, I am stuck at the very beginning. Here is my code:
void callerFunc()
{
//...
std::promise<DataWareHouse> data_chunks;
// DataWareHouse is my customized class
//data_chunks has a vector<vector<double>> member variable
std::thread(&run_thread,data_chunks);
// ............
}
void run_thread(std::promise<DataWareHouse> data_chunks)
{
// ...
vector<vector<double>> results;
// ...
data_chunks.set_value(results);
}
The above code generates an error:
`error C2248: 'std::promise<_Ty>::promise' : cannot access private member declared in class 'std::promise<_Ty>'`
May I know what am I wrong and how to fix it?
Many thanks. :-)
Your first problem is that you are using std::thread -- std::thread is a low level class which you should build higher abstractions up on. Threading is newly standardized in C++ in C++11, and all of the rough parts are not filed off yet.
There are three different patterns for using threading in C++11 that might be useful to you.
First, std::async. Second, std::thread mixed with std::packaged_task. And third, dealing with std::thread and std::promise in the raw.
I'll illustrate the third, which is the lowest level and most dangerous, because that is what you asked for. I would advise looking at the first two options.
#include <future>
#include <vector>
#include <iostream>
typedef std::vector<double> DataWareHouse;
void run_thread(std::promise<DataWareHouse> data_chunks)
{
DataWareHouse results;
results.push_back( 3.14159 );
data_chunks.set_value(results);
}
std::future<DataWareHouse> do_async_work()
{
std::promise<DataWareHouse> data_chunks;
std::future<DataWareHouse> retval = data_chunks.get_future();
// DataWareHouse is my customized class
//data_chunks has a vector<vector<double>> member variable
std::thread t = std::thread(&run_thread,std::move(data_chunks));
t.detach(); // do this or seg fault
return retval;
}
int main() {
std::future<DataWareHouse> result = do_async_work();
DataWareHouse vec = result.get(); // block and get the data
for (double d: vec) {
std::cout << d << "\n";
}
}
Live example
With std::async, you'd have a function returning DataWareHouse, and it would return a std::future<DataWareHouse> directly.
With std::packaged_task<>, it would take your run_thread and turn it into a packaged_task that can be executed, and a std::future extracted from it.
std::promise<> is not copyable, and in calling run_thread() you are implicitly trying to invoke the copy constructor. The error message is telling you that you cannot use the copy constructor since it is marked private.
You need to pass a promise by reference (std::promise<DataWareHouse> &). This is safe if callerFunc() is guaranteed not to return until run_thread() is finished with the object (otherwise you will be using a reference to a destroyed stack-allocated object, and I don't have to explain why that's bad).
You're trying to pass the promise to the thread by value; but you need to pass by reference to get the results back to the caller's promise. std::promise is uncopyable, to prevent this mistake.
std::thread(&run_thread,std::ref(data_chunks));
^^^^^^^^
void run_thread(std::promise<DataWareHouse> & data_chunks)
^
The error is telling you you cannot copy an std::promise, which you do here:
void run_thread(std::promise<DataWareHouse> data_chunks)
and here:
std::thread(&run_thread,data_chunks); // makes copy of data_chunks
You should pass a reference:
void run_thread(std::promise<DataWareHouse>& data_chunks);
// ^
And then pass an std::reference_wrapper to the thread, otherwise it too will attempt to copy the promise. This is easily done with std::ref:
std::thread(&run_thread, std::ref(data_chunks));
// ^^^^^^^^
Obviously data_chunks must be alive until the thread finished running, so you will have to join the thread in callerFunc().
I have an object for which I'd like to track the number of threads that reference it. In general, when any method on the object is called I can check a thread local boolean value to determine whether the count has been updated for the current thread. But this doesn't help me if the user say, uses boost::bind to bind my object to a boost::function and uses that to start a boost::thread. The new thread will have a reference to my object, and may hold on to it for an indefinite period of time before calling any of its methods, thus leading to a stale count. I could write my own wrapper around boost::thread to handle this, but that doesn't help if the user boost::bind's an object that contains my object (I can't specialize based on the presence of a member type -- at least I don't know of any way to do that) and uses that to start a boost::thread.
Is there any way to do this? The only means I can think of requires too much work from users -- I provide a wrapper around boost::thread that calls a special hook method on the object being passed in provided it exists, and users add the special hook method to any class that contains my object.
Edit: For the sake of this question we can assume I control the means to make new threads. So I can wrap boost::thread for example and expect that users will use my wrapped version, and not have to worry about users simultaneously using pthreads, etc.
Edit2: One can also assume that I have some means of thread local storage available, through __thread or boost::thread_specific_ptr. It's not in the current standard, but hopefully will be soon.
In general, this is hard. The question of "who has a reference to me?" is not generally solvable in C++. It may be worth looking at the bigger picture of the specific problem(s) you are trying to solve, and seeing if there is a better way.
There are a few things I can come up with that can get you partway there, but none of them are quite what you want.
You can establish the concept of "the owning thread" for an object, and REJECT operations from any other thread, a la Qt GUI elements. (Note that trying to do things thread-safely from threads other than the owner won't actually give you thread-safety, since if the owner isn't checked it can collide with other threads.) This at least gives your users fail-fast behavior.
You can encourage reference counting by having the user-visible objects being lightweight references to the implementation object itself [and by documenting this!]. But determined users can work around this.
And you can combine these two-- i.e. you can have the notion of thread ownership for each reference, and then have the object become aware of who owns the references. This could be very powerful, but not really idiot-proof.
You can start restricting what users can and cannot do with the object, but I don't think covering more than the obvious sources of unintentional error is worthwhile. Should you be declaring operator& private, so people can't take pointers to your objects? Should you be preventing people from dynamically allocating your object? It depends on your users to some degree, but keep in mind you can't prevent references to objects, so eventually playing whack-a-mole will drive you insane.
So, back to my original suggestion: re-analyze the big picture if possible.
Short of a pimpl style implementation that does a threadid check before every dereference I don't see how you could do this:
class MyClass;
class MyClassImpl {
friend class MyClass;
threadid_t owning_thread;
public:
void doSomethingThreadSafe();
void doSomethingNoSafetyCheck();
};
class MyClass {
MyClassImpl* impl;
public:
void doSomethine() {
if (__threadid() != impl->owning_thread) {
impl->doSomethingThreadSafe();
} else {
impl->doSomethingNoSafetyCheck();
}
}
};
Note: I know the OP wants to list threads with active pointers, I don't think that's feasible. The above implementation at least lets the object know when there might be contention. When to change the owning_thread depends heavily on what doSomething does.
Usually you cannot do this programmatically.
Unfortuately, the way to go is to design your program in such a way that you can prove (i.e. convince yourself) that certain objects are shared, and others are thread private.
The current C++ standard does not even have the notion of a thread, so there is no standard portable notion of thread local storage, in particular.
If I understood your problem correctly I believe this could be done in Windows using Win32 function GetCurrentThreadId().
Below is a quick and dirty example of how it could be used. Thread synchronisation should rather be done with a lock object.
If you create an object of CMyThreadTracker at the top of every member function of your object to be tracked for threads, the _handle_vector should contain the thread ids that use your object.
#include <process.h>
#include <windows.h>
#include <vector>
#include <algorithm>
#include <functional>
using namespace std;
class CMyThreadTracker
{
vector<DWORD> & _handle_vector;
DWORD _h;
CRITICAL_SECTION &_CriticalSection;
public:
CMyThreadTracker(vector<DWORD> & handle_vector,CRITICAL_SECTION &crit):_handle_vector(handle_vector),_CriticalSection(crit)
{
EnterCriticalSection(&_CriticalSection);
_h = GetCurrentThreadId();
_handle_vector.push_back(_h);
printf("thread id %08x\n",_h);
LeaveCriticalSection(&_CriticalSection);
}
~CMyThreadTracker()
{
EnterCriticalSection(&_CriticalSection);
vector<DWORD>::iterator ee = remove_if(_handle_vector.begin(),_handle_vector.end(),bind2nd(equal_to<DWORD>(), _h));
_handle_vector.erase(ee,_handle_vector.end());
LeaveCriticalSection(&_CriticalSection);
}
};
class CMyObject
{
vector<DWORD> _handle_vector;
public:
void method1(CRITICAL_SECTION & CriticalSection)
{
CMyThreadTracker tt(_handle_vector,CriticalSection);
printf("method 1\n");
EnterCriticalSection(&CriticalSection);
for(int i=0;i<_handle_vector.size();++i)
{
printf(" this object is currently used by thread %08x\n",_handle_vector[i]);
}
LeaveCriticalSection(&CriticalSection);
}
};
CMyObject mo;
CRITICAL_SECTION CriticalSection;
unsigned __stdcall ThreadFunc( void* arg )
{
unsigned int sleep_time = *(unsigned int*)arg;
while ( true)
{
Sleep(sleep_time);
mo.method1(CriticalSection);
}
_endthreadex( 0 );
return 0;
}
int _tmain(int argc, _TCHAR* argv[])
{
HANDLE hThread;
unsigned int threadID;
if (!InitializeCriticalSectionAndSpinCount(&CriticalSection, 0x80000400) )
return -1;
for(int i=0;i<5;++i)
{
unsigned int sleep_time = 1000 *(i+1);
hThread = (HANDLE)_beginthreadex( NULL, 0, &ThreadFunc, &sleep_time, 0, &threadID );
printf("creating thread %08x\n",threadID);
}
WaitForSingleObject( hThread, INFINITE );
return 0;
}
EDIT1:
As mentioned in the comment, reference dispensing could be implemented as below. A vector could hold the unique thread ids referring to your object. You may also need to implement a custom assignment operator to deal with the object references being copied by a different thread.
class MyClass
{
public:
static MyClass & Create()
{
static MyClass * p = new MyClass();
return *p;
}
static void Destroy(MyClass * p)
{
delete p;
}
private:
MyClass(){}
~MyClass(){};
};
class MyCreatorClass
{
MyClass & _my_obj;
public:
MyCreatorClass():_my_obj(MyClass::Create())
{
}
MyClass & GetObject()
{
//TODO:
// use GetCurrentThreadId to get thread id
// check if the id is already in the vector
// add this to a vector
return _my_obj;
}
~MyCreatorClass()
{
MyClass::Destroy(&_my_obj);
}
};
int _tmain(int argc, _TCHAR* argv[])
{
MyCreatorClass mcc;
MyClass &o1 = mcc.GetObject();
MyClass &o2 = mcc.GetObject();
return 0;
}
The solution I'm familiar with is to state "if you don't use the correct API to interact with this object, then all bets are off."
You may be able to turn your requirements around and make it possible for any threads that reference the object subscribe to signals from the object. This won't help with race conditions, but allows threads to know when the object has unloaded itself (for instance).
To solve the problem "I have an object and want to know how many threads access it" and you also can enumerate your threads, you can solve this problem with thread local storage.
Allocate a TLS index for your object. Make a private method called "registerThread" which simply sets the thread TLS to point to your object.
The key extension to the poster's original idea is that during every method call, call this registerThread(). Then you don't need to detect when or who created the thread, it's just set (often redundantly) during every actual access.
To see which threads have accessed the object, just examine their TLS values.
Upside: simple and pretty efficient.
Downside: solves the posted question but doesn't extend smoothly to multiple objects or dynamic threads that aren't enumerable.