How to implement own thread-safe shared pointer in C++? - c++

I need to make own simple thread-safe shared pointer class for embedded devices.
I made counting master pointer and handle as described in Jeff Alger's book (C++ for real programmers). This is my sources:
template <class T>
class counting_ptr {
public:
counting_ptr() : m_pointee(new T), m_counter(0) {}
counting_ptr(const counting_ptr<T>& sptr) :m_pointee(new T(*(sptr.m_pointee))), m_counter(0) {}
~counting_ptr() {delete m_pointee;}
counting_ptr<T>& operator=(const counting_ptr<T>& sptr)
{
if (this == &sptr) return *this;
delete m_pointee;
m_pointee = new T(*(sptr.m_pointee));
return *this;
}
void grab() {m_counter++;}
void release()
{
if (m_counter > 0) m_counter--;
if (m_counter <= 0)
delete this;
}
T* operator->() const {return m_pointee;}
private:
T* m_pointee;
int m_counter;
};
template <class T>
class shared_ptr {
private:
counting_ptr<T>* m_pointee;
public:
shared_ptr() : m_pointee(new counting_ptr<T>()) { m_pointee->grab(); }
shared_ptr(counting_ptr<T>* a_pointee) : m_pointee(a_ptr) { m_pointee->grab(); }
shared_ptr(const shared_ptr<T>& a_src) : m_pointee(a_src.m_pointee) {m_pointee->grab(); }
~shared_ptr() { m_pointee->release(); }
shared_ptr<T>& operator=(const shared_ptr<T>& a_src)
{
if (this == &a_src) return *this;
if (m_pointee == a_src.m_pointee) return *this;
m_pointee->release();
m_pointee = a_src.m_pointee;
m_pointee->grab();
return *this;
}
counting_ptr<T>* operator->() const {return m_pointee;}
};
This works fine if it used in one thread. Suppose I have two threads:
//thread 1
shared_ptr<T> p = some_global_shared_ptr;
//thread 2
some_global_shared_ptr = another_shared_ptr;
This case I can get undefined behaviour if one of threads will be interrupted between memory allocating/deallocating and counter changing. Of course I can enclose shared_ptr::release() into critical section so deletion of the pointer can be made safety. But what can I do with copy constructor? It is possible that constructor will be interrupted during m_pointee construction by another thread which will delete this m_pointee.
The only way I see to make shared_ptr assignement thread-safe is to enclose the assignment (or creation) into critical section. But this must be done in "user code". In other words user of shared_ptr class must take care about safety.
Is it possible to change this realization somehow to make the shared_ptr class thread safe?
=== EDIT ===
After some investigations (thanks to Jonathan) I realized that my shared_ptr has three unsafe places:
Unatomic counter changing
Unatomic assignment operator (source object can be deleted during copying)
shared_ptr copy constructor (very similar to previous case)
First two cases could be easily fixed by adding crtical sections. But I can't realize how to add critical section into copy constructor? Copy of a_src.m_pointee created before any other code in the constructor executed and can be deleted before calling grab. As Jonathan said in his comment it is very difficult to fix this problem.
I made such test:
typedef shared_ptr<....> Ptr;
Ptr p1, p2;
//thread 1
while (true)
{
Ptr p;
p2 = p;
}
//thread 2
while (!stop)
{
p1 = p2;
Ptr P(p2);
}
Of course, it crashed. But I have tried to use std::shared_ptr in VS 2013 C++. And it works!
So it is possible to make thread-safe copy constructor for shared_ptr. But stl sources too difficult for me and I don't understand how they did the trick. Please anyone explain me how it works in STL?
=== EDIT 2 ===
I am sorry, but the test for std::shared_ptr was made wrong. It doesn't pass too exactly as boost::shared_ptr does. Sometimes copy constructor fails to make a copy because source was deleted during copying. In this case empty pointer will be created.

This is hard to get right, I would seriously consider whether you actually need to support concurrent reads and writes of a single object (boost::shared_ptr and std::shared_ptr do not support that unless all accesses are done through the atomic_xxx() functions that are overloaded for shared_ptr and which typically acquire a lock).
For a start you would need to change shared_ptr<T>::m_pointee to atomic<counting_ptr<T>*> so that you can store a new value in it atomically. counting_ptr<T>::m_counter would need to be atomic<int> so the ref-count updates can be done atomically.
Your assignment operator is a big problem, you would need to at least re-order the operations so you increase the ref-count first, and avoid time of check to time of use bugs, something like this (not even compiled, let alone tested):
shared_ptr<T>& operator=(const shared_ptr<T>& a_src)
{
counter_ptr<T>* new_ptr = a_src.m_pointee.load();
new_ptr->grab();
counter_ptr<T>* old_ptr = m_pointee.exchange(new_ptr);
old_ptr->release();
return *this;
}
This form is safe against self-assignment (it just increases the ref-count then decreases it again if the two objects share the same pointee). It's still not safe against a_src changing while you try to copy it. Consider the case where a_src.m_pointee->m_counter == 1 initially. The current thread could call load() to get the other object's pointer, then a second thread could call release() on that pointer, which would delete it, making the grab() call undefined behaviour because it accesses an object that has been destroyed and memory that has been deallocated. Fixing that requires a pretty major redesign and maybe atomic operations that can operate on two words at once.
Getting this right is possible but is hard and you should really reconsider whether it's necessary, or if the code using it can just avoid modifying objects while other threads are reading them, except while the user has locked a mutex or other form of manual synchronisation.

After some investigations I can conclude that it is impossible to make thread-safe shared_ptr class where thread-safety understood as follow:
//thread 1
shared_ptr<T> p = some_global_shared_ptr;
//thread 2
some_global_shared_ptr = another_shared_ptr;
This example doesn't guarantees that p in first thread will point to old or new value of some_global_shared_ptr. In general this example leads to undefined behavior. The only way to make the example safety is to wrap both operators into critical sections or mutial exclusions.
The main problem caused by copy constructor of shared_ptr class. Other problems could be solved using critical sections inside shared_ptr methods.

Just inherit your class from CmyLock and you can make everything thread safe.
I use this class already many years in all my code, usually combined with class CmyThread, which creates a thread that has a very safe mutex. Maybe my answer is a little late, but above answers are not good practice.
/** Constructor */
CmyLock::CmyLock()
{
(void) pthread_mutexattr_init( &m_attr);
pthread_mutexattr_settype( &m_attr, PTHREAD_MUTEX_RECURSIVE);
pthread_mutex_init( &m_mutex, &m_attr);
}
/** Lock the thread for other threads. */
void CmyLock::lock()
{
pthread_mutex_lock( &m_mutex);
}
/** Unlock the thread for other threads. */
void CmyLock::unlock()
{
pthread_mutex_unlock( &m_mutex);
}
Here also the thread class. Try Please copy CmyLock and CmyThread classes to your project and tell when it's working! Although it's made for Linux, also Windows and Mac should be able to run this.
For the include file:
// #brief Class to create a single thread.
class CmyThread : public CmyLock
{
friend void *mythread_interrupt(void *ptr);
public:
CmyThread();
virtual ~CmyThread();
virtual void startWorking() {}
virtual void stopWorking() {}
virtual void work();
virtual void start();
virtual void stop();
bool isStopping() { return m_stopThread; }
bool isRunning() { return m_running && !m_stopThread; }
private:
virtual void run();
private:
bool m_running; ///< Thread is now running.
pthread_t m_thread; ///< Pointer to thread.
bool m_stopThread; ///< Indicate to stop thread.
};
The C++ file:
/** #brief Interrupt handler.
* #param ptr [in] SELF pointer for the instance.
*/
void *mythread_interrupt(void *ptr)
{
CmyThread *irq =
static_cast<CmyThread*> (ptr);
if (irq != NULL)
{
irq->run();
}
return NULL;
}
/** Constructor new thread. */
CmyThread::CmyThread()
: m_running( false)
, m_thread( 0)
, m_stopThread( false)
{
}
/** Start thread. */
void CmyThread::start()
{
m_running =true;
m_stopThread =false;
pthread_attr_t attr;
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
int stack_size =8192*1024;
pthread_attr_setstacksize(&attr, stack_size);
pthread_create(&m_thread, &attr, mythread_interrupt, (void*) this);
}
/** Thread function running. */
void CmyThread::run()
{
startWorking();
while (m_running && m_stopThread==false)
{
work();
}
m_running =false;
stopWorking();
pthread_exit(0);
}
/** Function to override for a thread. */
virtual void CmyThread::work()
{
delay(5000);
}
For example, here a simplistic example to store and retrieve 1000 data:
class a : public CmyLock
{
set_safe(int *data)
{
lock();
fileContent =std::make_shared<string>(data);
unlock();
}
get_safe(char *data)
{
lock();
strcpy( data, fileContent->c_str());
unlock();
}
std::shared_ptr<string> fileContent;
};

Related

std::thread : how to declare a thread in class body as a normal member?

I want to have a thread for each instance of Page object. At a time only one of them can execute (simply checks if pointer to current running thread is joinable or not..)
class Page : public std::vector<Step>
{
// ....
void play();
void start(); // check if no other thread is running. if there is a running thread, return. else join starter
std::thread starter; // std::thread running this->play()
static std::thread* current; // pointer to current running thread
// ...
};
I want to be able to fire-up starter threads of Page objects. for example like this:
Page x , y , z;
// do some stuff for initialize pages..
x.start();
// do some other work
y.start(); // if x is finished, start y otherwise do nothing
// do some other lengthy work
z.start(); // if x and y are not running, start z
I can't manage to declare started as a member of Page. I found that it's because of the fact std::threads can only initialized at declaration time. (or something like that, cause it's not possible to copy a thread)
void x()
{
}
//...
std::thread t(x); // this is ok
std::thread r; // this is wrong, but I need this !
r = std::thread(this->y); // no hope
r = std::thread(y); // this is wrong too
You can initialize the thread to the function to run by using a member initializer list. For example, consider this constructor for Page:
class Page {
public:
Page(); // For example
private:
std::thread toRun;
};
Page::Page() : toRun(/* function to run */) {
/* ... */
}
Notice how we use the initialization list inside the Page constructor to initialize toRun to the function that ought to be run. This way, toRun is initialized as if you had declared it as a local variable
std::string toRun(/* function to run */);
That said, there are two major problems I think that you must address in your code. First, you should not inherit from std::vector or any of the standard collections classes. Those classes don't have their destructors marked virtual, which means that you can easily invoke undefined behavior if you try to treat your Page as a std::vector. Instead, consider making Page hold a std::vector as a direct subobject. Also, you should not expose the std::thread member of the class. Data members should, as a general rule, be private to increase encapsulation, make it easier to modify the class in the future, and prevent people from breaking all of your class's invariants.
Hope this helps!
Never publicly inherit from a std container, unless the code is meant to be throw away code. An honestly it's terrifying how often throw away code becomes production code when push comes to shove.
I understand you don't want to reproduce the whole std::vector interface. That is tedious write, a pain to maintain, and honestly could create bugs.
Try this instead
class Page: private std::vector
{
public:
using std::vector::push_back;
using std::vector::size;
// ...
};
Ignoring the std::vector issue this should work for the concurrency part of the problem.
class Page
{
~Page( void )
{
m_thread.join();
}
void start( void );
private:
// note this is private, it must be to maintain the s_running invariant
void play( void )
{
assert( s_current == this );
// Only one Page at a time will execute this code.
std::lock_guard<std::mutex> _{ s_mutex };
s_running = nullptr;
}
std::thread m_thread;
static Page* s_running;
static std::mutex s_mutex;
};
Page* Page::s_running = nullptr;
std::mutex Page::s_mutex;
std::condition Page::s_condition;
void Page::start( void )
{
std::lock_guard<std::mutex> _{ s_mutex };
if( s_running == nullptr )
{
s_running = this;
m_thread = std::thread{ [this](){ this->play(); } };
}
}
This solution is may have initialization order issues if Page is instantiate before main()

Boost.thread possible memory leak

Since I've started making a little project aiming to have a crossplatform support, I chose boost 1.47 to interact with the underlying OS. My project needed some multithreading, so I made a little wrapper over boost threads to fulfill my needs.
Little I knew, boost apparently leaves the thread on memory after destructing its object(?), or then it may have some sort of memory leak possibility.
The implementation of my wrapper has a scoped_ptr of type thread, and the scoped ptr will get initialized when one calls the start() function in the wrapper class. The running thread will be stopped from main thread using thread->interrupt(), and the destructor will be called from the wrapper function. (Destructor of the thread's procedure structure, which has operator()() in it.
Here's the implementation of the wrapper class:
(note: i_exception and couple of other functions are parts of other project components)
#define TIMED_JOIN boost::posix_time::milliseconds(1)
namespace utils
{
struct thread_threadable
{
template<typename T> friend class ut_thread;
private:
boost::shared_ptr<thread_threadable> instance;
public:
virtual ~thread_threadable() {}
virtual void operator()() = 0;
};
template<typename T = thread_threadable>
class ut_thread
{
public:
typedef T proc_t;
private:
boost::scoped_ptr<boost::thread> thr;
boost::shared_ptr<proc_t> proc;
public:
explicit ut_thread(const boost::shared_ptr<proc_t> &procedure) : proc(procedure) {}
~ut_thread();
void start();
void stop();
bool running() const {return this->thr.get() != NULL;}
proc_t &procedure() const
{
BOOST_ASSERT(this->proc.get() != NULL);
return *this->proc;
}
};
}
typedef utils::thread_threadable threadable;
template<typename T>
utils::ut_thread<T>::~ut_thread()
{
if(this->thr.get() != NULL)
{
BOOST_ASSERT(this->proc.get() != NULL);
this->stop();
}
}
template<typename T>
void utils::ut_thread<T>::start()
{
if(this->thr.get() != NULL)
i_exception::throw_this("another thread of this procedure is already running");
if(this->proc.get() == NULL)
i_exception::throw_this("procedure object not initialized");
this->proc->instance = this->proc;
this->thr.reset(new boost::thread(boost::ref(*this->proc)));
this->thr->timed_join(TIMED_JOIN);
}
template<typename T>
void utils::ut_thread<T>::stop()
{
if(this->thr.get() == NULL)
i_exception::throw_this("no thread was running");
this->thr->interrupt();
this->proc->~T();
this->thr.reset(NULL);
}
And then by checking the functionality of this wrapper class, I made test to main.cpp:
struct my_thr : public utils::thread_threadable
{
void operator()()
{
while(true);
}
};
int main()
{
while(true)
{
utils::ut_thread<> thr(boost::shared_ptr<threadable>(new my_thr));
utils::ut_thread<> thr1(boost::shared_ptr<threadable>(new my_thr));
thr.start();
thr1.start();
boost::this_thread::sleep(boost::posix_time::seconds(1));
}
return 0;
}
At which point I noticed that these threads do not destruct, they will stay in memory until program gets terminated. They also keep executing the 'while(true)' statement.
So I'm asking, what would cause this kind of behaviour? Is it something defined, or just a bug or something else?
First of all interrupt will only stop the thread at certain ìnterruption points (taken from boost::threads documentation, slightly reformated):
Predefined Interruption Points
The following functions are interruption points, which will throw
boost::thread_interrupted if interruption is enabled for the current
thread, and interruption is requested for the current thread:
boost::thread::join()
boost::thread::timed_join()
boost::condition_variable::wait()
boost::condition_variable::timed_wait()
boost::condition_variable_any::wait()
boost::condition_variable_any::timed_wait()
boost::thread::sleep()
boost::this_thread::sleep()
boost::this_thread::interruption_point()
Since you don't have any of those in your thread execution calling interrupt()on it should have no effect.
Now for destroying the thread:
~thread();
Effects: If *this has an associated thread of execution, calls detach(). Destroys *this.
Throws: Nothing.
The timed_join() you called on the thread should fail, since the thread won't have finished it's execution that fast. Therefore you didn't join (or detach, but that wouldn't change the ultimate outcome) your threads, meaning they do have an associated thread of execution when they are destroyed. Therefore they are detached, meaning that they will run till they are finished even through they are no longer controllable through the boost::thread object. Since they are executing and infinite loop, finishing their execution might take some time so to say.
As a Sidenote: if you choose to change to C++11 std::threads later, you should note that destroying those without manually calling join() or detach() is not valid code.

Read-write thread-safe smart pointer in C++, x86-64

I develop some lock free data structure and following problem arises.
I have writer thread that creates objects on heap and wraps them in smart pointer with reference counter. I also have a lot of reader threads, that work with these objects. Code can look like this:
SmartPtr ptr;
class Reader : public Thread {
virtual void Run {
for (;;) {
SmartPtr local(ptr);
// do smth
}
}
};
class Writer : public Thread {
virtual void Run {
for (;;) {
SmartPtr newPtr(new Object);
ptr = newPtr;
}
}
};
int main() {
Pool* pool = SystemThreadPool();
pool->Run(new Reader());
pool->Run(new Writer());
for (;;) // wait for crash :(
}
When I create thread-local copy of ptr it means at least
Read an address.
Increment reference counter.
I can't do these two operations atomically and thus sometimes my readers work with deleted object.
The question is - what kind of smart pointer should I use to make read-write access from several threads with correct memory management possible? Solution should exist, since Java programmers don't even care about such a problem, simply relying on that all objects are references and are deleted only when nobody uses them.
For PowerPC I found http://drdobbs.com/184401888, looks nice, but uses Load-Linked and Store-Conditional instructions, that we don't have in x86.
As far I as I understand, boost pointers provide such functionality only using locks. I need lock free solution.
boost::shared_ptr have atomic_store which uses a "lock-free" spinlock which should be fast enough for 99% of possible cases.
boost::shared_ptr<Object> ptr;
class Reader : public Thread {
virtual void Run {
for (;;) {
boost::shared_ptr<Object> local(boost::atomic_load(&ptr));
// do smth
}
}
};
class Writer : public Thread {
virtual void Run {
for (;;) {
boost::shared_ptr<Object> newPtr(new Object);
boost::atomic_store(&ptr, newPtr);
}
}
};
int main() {
Pool* pool = SystemThreadPool();
pool->Run(new Reader());
pool->Run(new Writer());
for (;;)
}
EDIT:
In response to comment below, the implementation is in "boost/shared_ptr.hpp"...
template<class T> void atomic_store( shared_ptr<T> * p, shared_ptr<T> r )
{
boost::detail::spinlock_pool<2>::scoped_lock lock( p );
p->swap( r );
}
template<class T> shared_ptr<T> atomic_exchange( shared_ptr<T> * p, shared_ptr<T> r )
{
boost::detail::spinlock & sp = boost::detail::spinlock_pool<2>::spinlock_for( p );
sp.lock();
p->swap( r );
sp.unlock();
return r; // return std::move( r )
}
With some jiggery-pokery you should be able to accomplish this using InterlockedCompareExchange128. Store the reference count and pointer in a 2 element __int64 array. If reference count is in array[0] and pointer in array[1] the atomic update would look like this:
while(true)
{
__int64 comparand[2];
comparand[0] = refCount;
comparand[1] = pointer;
if(1 == InterlockedCompareExchange128(
array,
pointer,
refCount + 1,
comparand))
{
// Pointer is ready for use. Exit the while loop.
}
}
If an InterlockedCompareExchange128 intrinsic function isn't available for your compiler then you may use the underlying CMPXCHG16B instruction instead, if you don't mind mucking around in assembly language.
The solution proposed by RobH doesn't work. It has the same problem as the original question: when accessing the reference count object, it might already have been deleted.
The only way I see of solving the problem without a global lock (as in boost::atomic_store) or conditional read/write instructions is to somehow delay the destruction of the object (or the shared reference count object if such thing is used). So zennehoy has a good idea but his method is too unsafe.
The way I might do it is by keeping copies of all the pointers in the writer thread so that the writer can control the destruction of the objects:
class Writer : public Thread {
virtual void Run() {
list<SmartPtr> ptrs; //list that holds all the old ptr values
for (;;) {
SmartPtr newPtr(new Object);
if(ptr)
ptrs.push_back(ptr); //push previous pointer into the list
ptr = newPtr;
//Periodically go through the list and destroy objects that are not
//referenced by other threads
for(auto it=ptrs.begin(); it!=ptrs.end(); )
if(it->refCount()==1)
it = ptrs.erase(it);
else
++it;
}
}
};
However there are still requirements for the smart pointer class. This doesn't work with shared_ptr as the reads and writes are not atomic. It almost works with boost::intrusive_ptr. The assignment on intrusive_ptr is implemented like this (pseudocode):
//create temporary from rhs
tmp.ptr = rhs.ptr;
if(tmp.ptr)
intrusive_ptr_add_ref(tmp.ptr);
//swap(tmp,lhs)
T* x = lhs.ptr;
lhs.ptr = tmp.ptr;
tmp.ptr = x;
//destroy temporary
if(tmp.ptr)
intrusive_ptr_release(tmp.ptr);
As far as I understand the only thing missing here is a compiler level memory fence before lhs.ptr = tmp.ptr;. With that added, both reading rhs and writing lhs would be thread-safe under strict conditions: 1) x86 or x64 architecture 2) atomic reference counting 3) rhs refcount must not go to zero during the assignment (guaranteed by the Writer code above) 4) only one thread writing to lhs (using CAS you could have several writers).
Anyway, you could create your own smart pointer class based on intrusive_ptr with necessary changes. Definitely easier than re-implementing shared_ptr. And besides, if you want performance, intrusive is the way to go.
The reason this works much more easily in java is garbage collection. In C++, you have to manually ensure that a value is not just starting to be used by a different thread when you want to delete it.
A solution I've used in a similar situation is to simply delay the deletion of the value. I create a separate thread that iterates through a list of things to be deleted. When I want to delete something, I add it to this list with a timestamp. The deleting thread waits until some fixed time after this timestamp before actually deleting the value. You just have to make sure that the delay is large enough to guarantee that any temporary use of the value has completed.
100 milliseconds would have been enough in my case, I chose a few seconds to be safe.

C++ Critical Section not working

My critical section code does not work!!!
Backgrounder.run IS able to modify MESSAGE_QUEUE g_msgQueue and LockSections destructor hadn't been called yet !!!
Extra code :
typedef std::vector<int> MESSAGE_LIST; // SHARED OBJECT .. MUST LOCK!
class MESSAGE_QUEUE : MESSAGE_LIST{
public:
MESSAGE_LIST * m_pList;
MESSAGE_QUEUE(MESSAGE_LIST* pList){ m_pList = pList; }
~MESSAGE_QUEUE(){ }
/* This class will be shared between threads that means any
* attempt to access it MUST be inside a critical section.
*/
void Add( int messageCode ){ if(m_pList) m_pList->push_back(messageCode); }
int getLast()
{
if(m_pList){
if(m_pList->size() == 1){
Add(0x0);
}
m_pList->pop_back();
return m_pList->back();
}
}
void removeLast()
{
if(m_pList){
m_pList->erase(m_pList->end()-1,m_pList->end());
}
}
};
class Backgrounder{
public:
MESSAGE_QUEUE* m_pMsgQueue;
static void __cdecl Run( void* args){
MESSAGE_QUEUE* s_pMsgQueue = (MESSAGE_QUEUE*)args;
if(s_pMsgQueue->getLast() == 0x45)printf("It's a success!");
else printf("It's a trap!");
}
Backgrounder(MESSAGE_QUEUE* pMsgQueue)
{
m_pMsgQueue = pMsgQueue;
_beginthread(Run,0,(void*)m_pMsgQueue);
}
~Backgrounder(){ }
};
int main(){
MESSAGE_LIST g_List;
CriticalSection crt;
ErrorHandler err;
LockSection lc(&crt,&err); // Does not work , see question #2
MESSAGE_QUEUE g_msgQueue(&g_List);
g_msgQueue.Add(0x45);
printf("%d",g_msgQueue.getLast());
Backgrounder back_thread(&g_msgQueue);
while(!kbhit());
return 0;
}
#ifndef CRITICALSECTION_H
#define CRITICALSECTION_H
#include <windows.h>
#include "ErrorHandler.h"
class CriticalSection{
long m_nLockCount;
long m_nThreadId;
typedef CRITICAL_SECTION cs;
cs m_tCS;
public:
CriticalSection(){
::InitializeCriticalSection(&m_tCS);
m_nLockCount = 0;
m_nThreadId = 0;
}
~CriticalSection(){ ::DeleteCriticalSection(&m_tCS); }
void Enter(){ ::EnterCriticalSection(&m_tCS); }
void Leave(){ ::LeaveCriticalSection(&m_tCS); }
void Try();
};
class LockSection{
CriticalSection* m_pCS;
ErrorHandler * m_pErrorHandler;
bool m_bIsClosed;
public:
LockSection(CriticalSection* pCS,ErrorHandler* pErrorHandler){
m_bIsClosed = false;
m_pCS = pCS;
m_pErrorHandler = pErrorHandler;
// 0x1AE is code prefix for critical section header
if(!m_pCS)m_pErrorHandler->Add(0x1AE1);
if(m_pCS)m_pCS->Enter();
}
~LockSection(){
if(!m_pCS)m_pErrorHandler->Add(0x1AE2);
if(m_pCS && m_bIsClosed == false)m_pCS->Leave();
}
void ForceCSectionClose(){
if(!m_pCS)m_pErrorHandler->Add(0x1AE3);
if(m_pCS){m_pCS->Leave();m_bIsClosed = true;}
}
};
/*
Safe class basic structure;
class SafeObj
{
CriticalSection m_cs;
public:
void SafeMethod()
{
LockSection myLock(&m_cs);
//add code to implement the method ...
}
};
*/
#endif
Two questions in one. I don't know about the first, but the critical section part is easy to explain. The background thread isn't trying to claim the lock and so, of course, is not blocked. You need to make the critical section object crt visible to the thread so that it can lock it.
The way to use this lock class is that each section of code that you want serialised must create a LockSection object and hold on to it until the end of the serialised block:
Thread 1:
{
LockSection lc(&crt,&err);
//operate on shared object from thread 1
}
Thread 2:
{
LockSection lc(&crt,&err);
//operate on shared object from thread 2
}
Note that it has to be the same critical section instance crt that is used in each block of code that is to be serialised.
This code has a number of problems.
First of all, deriving from the standard containers is almost always a poor idea. In this case you're using private inheritance, which reduces the problems, but doesn't eliminate them entirely. In any case, you don't seem to put the inheritance to much (any?) use anyway. Even though you've derived your MESSAGE_QUEUE from MESSAGE_LIST (which is actually std::vector<int>), you embed a pointer to an instance of a MESSAGE_LIST into MESSAGE_QUEUE anyway.
Second, if you're going to use a queue to communicate between threads (which I think is generally a good idea) you should make the locking inherent in the queue operations rather than requiring each thread to manage the locking correctly on its own.
Third, a vector isn't a particularly suitable data structure for representing a queue anyway, unless you're going to make it fixed size, and use it roughly like a ring buffer. That's not a bad idea either, but it's quite a bit different from what you've done. If you're going to make the size dynamic, you'd probably be better off starting with a deque instead.
Fourth, the magic numbers in your error handling (0x1AE1, 0x1AE2, etc.) is quite opaque. At the very least, you need to give these meaningful names. The one comment you have does not make the use anywhere close to clear.
Finally, if you're going to go to all the trouble of writing code for a thread-safe queue, you might as well make it generic so it can hold essentially any kind of data you want, instead of dedicating it to one specific type.
Ultimately, your code doesn't seem to save the client much work or trouble over using the Windows functions directly. For the most part, you've just provided the same capabilities under slightly different names.
IMO, a thread-safe queue should handle almost all the work internally, so that client code can use it about like it would any other queue.
// Warning: untested code.
// Assumes: `T::T(T const &) throw()`
//
template <class T>
class queue {
std::deque<T> data;
CRITICAL_SECTION cs;
HANDLE semaphore;
public:
queue() {
InitializeCriticalSection(&cs);
semaphore = CreateSemaphore(NULL, 0, 2048, NULL);
}
~queue() {
DeleteCriticalSection(&cs);
CloseHandle(semaphore);
}
void push(T const &item) {
EnterCriticalSection(&cs);
data.push_back(item);
LeaveCriticalSection(&cs);
ReleaseSemaphore(semaphore, 1, NULL);
}
T pop() {
WaitForSingleObject(semaphore, INFINITE);
EnterCriticalSection(&cs);
T item = data.front();
data.pop_front();
LeaveCriticalSection(&cs);
return item;
}
};
HANDLE done;
typedef queue<int> msgQ;
enum commands { quit, print };
void backgrounder(void *qq) {
// I haven't quite puzzled out what your background thread
// was supposed to do, so I've kept it really simple, executing only
// the two commands listed above.
msgQ *q = (msgQ *)qq;
int command;
while (quit != (command = q->pop()))
printf("Print\n");
SetEvent(done);
}
int main() {
msgQ q;
done = CreateEvent(NULL, false, false, NULL);
_beginthread(backgrounder, 0, (void*)&q);
for (int i=0; i<20; i++)
q.push(print);
q.push(quit);
WaitForSingleObject(done, INFINITE);
return 0;
}
Your background thread needs access to the same CriticalSection object and it needs to create LockSection objects to lock it -- the locking is collaborative.
You are trying to return the last element after popping it.

Need some advice to make the code multithreaded

I received a code that is not for multi-threaded app, now I have to modify the code to support for multi-threaded.
I have a Singleton class(MyCenterSigltonClass) that based on instruction in:
http://en.wikipedia.org/wiki/Singleton_pattern
I made it thread-safe
Now I see inside the class that contains 10-12 members, some with getter/setter methods.
Some members are declared as static and are class pointer like:
static Class_A* f_static_member_a;
static Class_B* f_static_member_b;
for these members, I defined a mutex(like mutex_a) INSIDE the class(Class_A) , I didn't add the mutex directly in my MyCenterSigltonClass, the reason is they are one to one association with my MyCenterSigltonClass, I think I have option to define mutex in the class(MyCenterSigltonClass) or (Class_A) for f_static_member_a.
1) Am I right?
Also, my Singleton class(MyCenterSigltonClass) contains some other members like
Class_C f_classC;
for these kind of member variables, should I define a mutex for each of them in MyCenterSigltonClass to make them thread-safe? what would be a good way to handle these cases?
Appreciate for any suggestion.
-Nima
Whether the members are static or not doesn't really matter. How you protect the member variables really depends on how they are accessed from public methods.
You should think about a mutex as a lock that protects some resource from concurrent read/write access. You don't need to think about protecting the internal class objects necessarily, but the resources within them. You also need to consider the scope of the locks you'll be using, especially if the code wasn't originally designed to be multithreaded. Let me give a few simple examples.
class A
{
private:
int mValuesCount;
int* mValues;
public:
A(int count, int* values)
{
mValuesCount = count;
mValues = (count > 0) ? new int[count] : NULL;
if (mValues)
{
memcpy(mValues, values, count * sizeof(int));
}
}
int getValues(int count, int* values) const
{
if (mValues && values)
{
memcpy(values, mValues, (count < mValuesCount) ? count : mValuesCount);
}
return mValuesCount;
}
};
class B
{
private:
A* mA;
public:
B()
{
int values[5] = { 1, 2, 3, 4, 5 };
mA = new A(5, values);
}
const A* getA() const { return mA; }
};
In this code, there's no need to protect mA because there's no chance of conflicting access across multiple threads. None of the threads can modify the state of mA, so all concurrent access just reads from mA. However, if we modify class A:
class A
{
private:
int mValuesCount;
int* mValues;
public:
A(int count, int* values)
{
mValuesCount = 0;
mValues = NULL;
setValues(count, values);
}
int getValues(int count, int* values) const
{
if (mValues && values)
{
memcpy(values, mValues, (count < mValuesCount) ? count : mValuesCount);
}
return mValuesCount;
}
void setValues(int count, int* values)
{
delete [] mValues;
mValuesCount = count;
mValues = (count > 0) ? new int[count] : NULL;
if (mValues)
{
memcpy(mValues, values, count * sizeof(int));
}
}
};
We can now have multiple threads calling B::getA() and one thread can read from mA while another thread writes to mA. Consider the following thread interaction:
Thread A: a->getValues(maxCount, values);
Thread B: a->setValues(newCount, newValues);
It's possible that Thread B will delete mValues while Thread A is in the middle of copying it. In this case, you would need a mutex within class A to protect access to mValues and mValuesCount:
int getValues(int count, int* values) const
{
// TODO: Lock mutex.
if (mValues && values)
{
memcpy(values, mValues, (count < mValuesCount) ? count : mValuesCount);
}
int returnCount = mValuesCount;
// TODO: Unlock mutex.
return returnCount;
}
void setValues(int count, int* values)
{
// TODO: Lock mutex.
delete [] mValues;
mValuesCount = count;
mValues = (count > 0) ? new int[count] : NULL;
if (mValues)
{
memcpy(mValues, values, count * sizeof(int));
}
// TODO: Unlock mutex.
}
This will prevent concurrent read/write on mValues and mValuesCount. Depending on the locking mechanisms available in your environment, you may be able to use a read-only locking mechanism in getValues() to prevent multiple threads from blocking on concurrent read access.
However, you'll also need to understand the scope of the locking you need to implement if class A is more complex:
class A
{
private:
int mValuesCount;
int* mValues;
public:
A(int count, int* values)
{
mValuesCount = 0;
mValues = NULL;
setValues(count, values);
}
int getValueCount() const { return mValuesCount; }
int getValues(int count, int* values) const
{
if (mValues && values)
{
memcpy(values, mValues, (count < mValuesCount) ? count : mValuesCount);
}
return mValuesCount;
}
void setValues(int count, int* values)
{
delete [] mValues;
mValuesCount = count;
mValues = (count > 0) ? new int[count] : NULL;
if (mValues)
{
memcpy(mValues, values, count * sizeof(int));
}
}
};
In this case, you could have the following thread interaction:
Thread A: int maxCount = a->getValueCount();
Thread A: // allocate memory for "maxCount" int values
Thread B: a->setValues(newCount, newValues);
Thread A: a->getValues(maxCount, values);
Thread A has been written as though calls to getValueCount() and getValues() will be an uninterrupted operation, but Thread B has potentially changed the count in the middle of Thread A's operations. Depending on whether the new count is larger or smaller than the original count, it may take a while before you discover this problem. In this case, class A would need to be redesigned or it would need to provide some kind of transaction support so the thread using class A could block/unblock other threads:
Thread A: a->lockValues();
Thread A: int maxCount = a->getValueCount();
Thread A: // allocate memory for "maxCount" int values
Thread B: a->setValues(newCount, newValues); // Blocks until Thread A calls unlockValues()
Thread A: a->getValues(maxCount, values);
Thread A: a->unlockValues();
Thread B: // Completes call to setValues()
Since the code wasn't initially designed to be multithreaded, it's very likely you'll run into these kinds of issues where one method call uses information from an earlier call, but there was never a concern for the state of the object changing between those calls.
Now, begin to imagine what could happen if there are complex state dependencies among the objects within your singleton and multiple threads can modify the state of those internal objects. It can all become very, very messy with a large number of threads and debugging can become very difficult.
So as you try to make your singleton thread-safe, you need to look at several layers of object interactions. Some good questions to ask:
Do any of the methods on the singleton reveal internal state that may change between method calls (as in the last example I mention)?
Are any of the internal objects revealed to clients of the singleton?
If so, do any of the methods on those internal objects reveal internal state that may change between method calls?
If internal objects are revealed, do they share any resources or state dependencies?
You may not need any locking if you're just reading state from internal objects (first example). You may need to provide simple locking to prevent concurrent read/write access (second example). You may need to redesign the classes or provide clients with the ability to lock object state (third example). Or you may need to implement more complex locking where internal objects share state information across threads (e.g. a lock on a resource in class Foo requires a lock on a resource in class Bar, but locking that resource in class Bar doesn't necessarily require a lock on a resource in class Foo).
Implementing thread-safe code can become a complex task depending on how all your objects interact. It can be much more complicated than the examples I've given here. Just be sure you clearly understand how your classes are used and how they interact (and be prepared to spend some time tracking down difficult to reproduce bugs).
If this is the first time you're doing threading, consider not accessing the singleton from the background thread. You can get it right, but you probably won't get it right the first time.
Realize that if your singleton exposes pointers to other objects, these should be made thread safe as well.
You don't have to define a mutex for each member. For example, you could instead use a single mutex to synchronize access each to member, e.g.:
class foo
{
public:
...
void some_op()
{
// acquire "lock_" and release using RAII ...
Lock(lock_);
a++;
}
void set_b(bar * b)
{
// acquire "lock_" and release using RAII ...
Lock(lock_);
b_ = b;
}
private:
int a_;
bar * b_;
mutex lock_;
}
Of course a "one lock" solution may be not suitable in your case. That's up to you to decide. Regardless, simply introducing locks doesn't make the code thread-safe. You have to use them in the right place in the right way to avoid race conditions, deadlocks, etc. There are lots of concurrency issues you could run in to.
Furthermore you don't always need mutexes, or other threading mechanisms like TSS, to make code thread-safe. For example, the following function "func" is thread-safe:
class Foo;
void func (Foo & f)
{
f.some_op(); // Foo::some_op() of course needs to be thread-safe.
}
// Thread 1
Foo a;
func(&a);
// Thread 2
Foo b;
func(&b);
While the func function above is thread-safe the operations it invokes may not be thread-safe. The point is you don't always need to pepper your code with mutexes and other threading mechanisms to make the code thread safe. Sometimes restructuring the code is sufficient.
There's a lot of literature on multithreaded programming. It's definitely not easy to get right so take your time in understanding the nuances, and take advantage of existing frameworks like Boost.Thread to mitigate some of the inherent and accidental complexities that exist in the lower-level multithreading APIs.
I'd really recommend the Interlocked.... Methods to increment, decrement and CompareAndSwap values when using code that needs to be multi-thread-aware. I don't have 1st-hand C++ experience but a quick search for http://www.bing.com/search?q=c%2B%2B+interlocked reveals lots of confirming advice. If you need perf, these will likely be faster than locking.
As stated by #Void a mutex alone is not always the solution to a concurrency problem:
Regardless, simply introducing locks doesn't make the code
thread-safe. You have to use them in the right place in the right way
to avoid race conditions, deadlocks, etc. There are lots of
concurrency issues you could run in to.
I want to add another example:
class MyClass
{
mutex m_mutex;
AnotherClass m_anotherClass;
void setObject(AnotherClass& anotherClass)
{
m_mutex.lock();
m_anotherClass = anotherClass;
m_mutex.unlock();
}
AnotherClass getObject()
{
AnotherClass anotherClass;
m_mutex.lock();
anotherClass = m_anotherClass;
m_mutex.unlock();
return anotherClass;
}
}
In this case the getObject() method is always safe because is protected with mutex and you have a copy of the object which is returned to the caller which may be a different class and thread. This means you are working on a copy which might be old (in the meantime another thread might have changed the m_anotherClass by calling setObject() ).Now what if you turn m_anotherClass to a pointer instead of an object-variable ?
class MyClass
{
mutex m_mutex;
AnotherClass *m_anotherClass;
void setObject(AnotherClass *anotherClass)
{
m_mutex.lock();
m_anotherClass = anotherClass;
m_mutex.unlock();
}
AnotherClass * getObject()
{
AnotherClass *anotherClass;
m_mutex.lock();
anotherClass = m_anotherClass;
m_mutex.unlock();
return anotherClass;
}
}
This is an example where a mutex is not enough to solve all the problems.
With pointers you can have a copy only of the pointer but the pointed object is the same in the both the caller and the method. So even if the pointer was valid at the time that the getObject() was called you don't have any guarantee that the pointed value will exists during the operation you are performing with it. This is simply because you don't have control on the object lifetime. That's why you should use object-variables as much as possible and avoid pointers (if you can).