Basically I'm maintaining a set of states for a bunch of objects:
#define SUBSCRIPTION_TYPE int
std::map< SUBSCRIPTION_TYPE , bool > is_object_valid;
And I need to protect each element in is_object_valid with their respective mutex_t(Rather than a big lock). As valid values of SUBSCRIPTION_TYPE are sparse (say, dozens of values ranging from 0 to 10000000 ), a std::map is prefered over std::vector, C-style array, etc.
I'm trying to achieve something like:
std::map< SUBSCRIPTION_TYPE , pthread_mutex_t > mutex_array;
But it doesn't seem to work. (Well, data race may occur when std::map is being updated).
So what's the best way to achieve this? Must I write a thread-safe subscription allocator that maps SUBSCRIPTION_TYPE into consecutive integers so that I can store the mutex_ts in an array?
If any thread is modifying the map itself (inserting, etc.), you
need to protect all accesses to the map. After that: if the
member is just a bool, how much processing can you be doing on
it that adding this time to the time the map level mutex is held
would change anything.
Otherwise: if you need a mutex per object, the simple solution
would be to put them into same object as the one on the map.
But it mutex_t copyable? pthread_mutex_t and std::mutex
aren't. This could make the insertion code overly complex,
since you can't initialize the pthread_mutex_t, or construct
the std::mutex, before the object is inserted. (In C++11, you
could use emplace to solve this problem; contents of a map
don't have to be copyable if you use emplace.) In C++03,
however, you'll have to separate allocation from initialization;
the struct which contains your mapped value and the mutex will
in fact have to be declared with raw memory for the mutex, and
then placement new used to initialize it using the iterator you
get back from insert.
If you have multiple threads reading and writing to the mutex_array, you will need another mutex to guard it.
Are you sure you will have multiple threads writing to the mutex_array?
The other thing is, instead of having two maps, you can have a map<subscription_type, object_struct>
struct object_struct {
bool valid;
pthread_mutex_t mutex;
};
And then have a single overarcing mutex to guard that map.
Related
Does it risk if use c++ std::unordered_map like this?
std::unordered_map<std::string, std::unordered_set<std::shared_ptr<SomeType>>> _some_map;
...
void init() {
auto item = std::make_shared<SomeType>();
_some_map["circle"].insert(item);
}
_some_map is a member variable. init() function can be called only once, so insert is thread-safe. After initialization, the map will be read only in multi-thread. I think it is thread-safe too.
I'm not sure if I can use insert like this. Because it will make a unordered_set and a pair when there is no val of key "circle". The code works normally. Just want to make sure it is without potential risk.
(I am not good at English writing. )
Thank you.
I'm not sure if I can use insert like this.
Yes, you can, because operator[]
Returns a reference to the value that is mapped to a key equivalent to key, performing an insertion if such key does not already exist.
Your value_type is std::unordered_set which is default constructible, so there is no problem.
After initialization, the map will be read only in multi-thread
Here also you are safe, because according to the containers documentation
All const member functions can be called concurrently by different threads on the same container.
so if you are only reading the values without modifying them, it is OK. You could even perform write operations if you could guarantee that different threads are accessing different elements in your container.
I have a
std::map<int, std::bitset<256 > > m;
After construction no new keys will be inserted and no keys will be removed. Can I safely assign the bitset in one thread while reading it in other threads without using a mutex?
// thread 1:
std::bitset<256> a = getBitset();
m[1] = a;
// thread 2:
std::bitset<256> b = m[1];
// or alternatively
std::bitset<256> c = m.at(1);
I think that the program will not crash but a data race could occur in the bitset. A data race would be acceptable if the read would deliver a combination of the old and the new bitset.
No, it is not safe.
operator= of the bitset is a modifying operation, and as such is not guaranteed to be free of a data race if the bitset is simultaneously accessed in another thread. And in practice, it will almost surely cause a data race, since it needs to write to the object. This is not specific to std::bitset, but generally true for pretty much all non-empty non-atomic standard library types (and most other non-empty types, as well).
Any data race will result in undefined behavior. There are no partial updates. In that sense, it should never be acceptable.
If you want to be able to do this, either wrap the bitset in a struct with a mutex to protect access to the bitset, or use something like an atomic shared_ptr that allows atomically exchanging the old bitset with a new one and delaying destruction of the old one until all references are gone.
Although I think it is not guaranteed, std::bitset may also be trivially copyable. You could check that with a static_assert on std::is_trivially_copyable_v and if true, you could also use a std::atomic or std::atomic_ref which will allow accessing the bitset atomically (i.e. free of data race) and will probably use a lock only if the underlying architecture doesn't support atomic access to an object of the corresponding size.
Also note that std::bitset b = m[1]; and m[1] also cause undefined behavior, because operator[] of std::map is also a modifying operation, and is not specified to be free of a data race, even if it doesn't insert a new element into the map. You would need to use e.g. the find() member function instead.
I need a FIFO structure that supports indexing. Each element is an array of data that is saved off a device I'm reading from. The FIFO has a constant size, and at start-up each element is zeroed out.
Here's some pseudo code to help understand the issue:
Thread A (Device Reader):
1. Lock the structure.
2. Pop oldest element off of FIFO (don't need it).
3. Read next array of data (note this is a fixed size array) from the device.
4. Push new data array onto the FIFO.
5. Unlock.
Thread B (Data Request From Caller):
1. Lock the structure.
2. Determine request type.
3. if (request = one array) memcpy over the latest array saved (LIFO).
4. else memcpy over the whole FIFO to the user as a giant array (caller uses arrays).
5. Unlock.
Note that the FIFO shouldn't be changed in Thread B, the caller should just get a copy, so data structures where pop is destructive wouldn't necessarily work without an intermediate copy.
My code also has a boost dependency already and I am using a lockfree spsc_queue elsewhere. With that said, I don't see how this queue would work for me here given the need to work as a LIFO in some cases and also the need to memcpy over the entire FIFO at times.
I also considered a plain std::vector, but I'm worried about performance when I'm constantly pushing and popping.
One point not clear in the question is the compiler target, whether or not the solution is restricted to partial C++11 support (like VS2012), or full support (like VS2015). You mentioned boost dependency, which lends similar features to older compilers, so I'll rely on that and speak generally about options on the assumption that boost may provide what a pre-C++11 compiler may not, or you may elect C++11 features like the now standardized mutex, lock, threads and shared_ptr.
There's no doubt in my mind that the primary tool for the FIFO (which, as you stated, may occasionally need LIFO operation) is the std::deque. Even though the deque supports reasonably efficient dynamic expansion and shrinking of storage, contrary to your primary requirement of a static size, it's main feature is the ability to function as both FIFO and LIFO with good performance in ways vectors can't as easily manage. Internally most implementations provide what may be analogized as a collection of smaller vectors which are marshalled by the deque to function as if a single vector container (for subscripting) while allowing for double ended pushing and popping with efficient memory management. It can be tempting to use a vector, employing a circular buffer technique for fixed sizes, but any performance improvement is minimal, and deque is known to be reliable.
Your point regarding destructive pops isn't entirely clear to me. That could mean several things. std::deque offers back and front as a peek to what's at the ends of the deque, without destruction. In fact, they're required to look because deque's pop_front and pop_back only remove elements, they don't provide access to the element being popped. Taking an element and popping it is a two step process on std::deque. An alternate meaning, however, is that a read only requester needs to pop strictly as a means of navigation, not destruction, which is not really a pop, but a traversal. As long as the structure is under lock, that is easily managed with iterators or indexes. Or, it could also mean you need a independent copy of the queue.
Assuming some structure representing device data:
struct DevDat { .... };
I'm immediately faced with that curious question, should this not be a generic solution? It doesn't matter for the sake of discussion, but it seems the intent is an odd combination of application specific operation and a generalized thread-safe stack "machine", so I'll suggest a generic solution which is easily translated otherwise (that is, I suggest template classes, but you could easily choose non-templates if preferred). These psuedo code examples are sparse, just illustrating container layout ideas and proposed concepts.
class SafeStackBase
{ protected: std::mutex sync;
};
template <typename Element>
class SafeStack : public SafeStackBase
{ public:
typedef std::deque< Element > DeQue;
private:
DeQue que;
};
SafeStack could handle any kind of data in the stack, so that detail is left for Element declaration, which I illustrate with typedefs:
typedef std::vector< DevDat > DevArray;
typedef std::shared_ptr< DevArray > DevArrayPtr;
typedef SafeStack< DevArrayPtr > DeviceQue;
Note I'm proposing vector instead of array because I don't like the idea of having to choose a fixed size, but std::array is an option, obviously.
The SafeStackBase is intended for code and data that isn't aware of the users data type, which is why the mutex is stored there. It could easily part of the template class, but the practice of placing non-type aware data and code in a non-template base helps reduce code bloat when possible (functions which don't use Element, for example, need not be expanded in template instantiations). I suggest the DevArrayPtr so that the arrays can be "plucked out" of the queue without copying the arrays, then shared and distributed outside the structure under shared_ptr's shared ownership. This is a matter of illustration, and does not adequately deal with questions regarding content of those arrays. That could be managed by DevDat, which could marshal reading of the array data, while limiting writing of the array data to an authorized friend (a write accessor strategy), such that Thread B (a reader only) is not carelessly able to modify the content. In this way it's possible to provide these arrays without copying data..just return a copy of the DevArrayPtr for communal access to the entire array. This also supports returning a container of DevArrayPtr's supporting ThreadB point 4 (copy the whole FIFO to the user), as in:
typedef std::vector< DevArrayPtr > QueArrayVec;
typedef std::deque< DevArrayPtr > QueArrayDeque;
typedef std::array< DevArrayPtr, 12 > QueArrays;
The point is that you can return any container you like, which is merely an array of pointers to the internal std::array< DevDat >, letting DevDat control read/write authorization by requiring some authorization object for writing, and if this copy should be operable as a FIFO without potential interference with Thread A's write ownership, QueArrayDeque provides the full feature set as an independent FIFO/LIFO structure.
This brings up an observation about Thread A. There you state lock is step 1, while unlock is step 5, but I submit that only steps 2 and 4 are really required under lock. Step 3 can take time, and even if you assume that is a short time, it's not as short as a pop followed by a push. The point is that the lock is really about controlling the FIFO/LIFO queue structure, and not about reading data from the device. As such, that data can be fashioned into DevArray, which is THEN provided to SafeStack to be pop/pushed under lock.
Assume code inside SafeStack:
typedef std::lock_guard< std::mutex > Lock; // I use typedefs a lot
void StuffIt( const Element & e )
{ Lock l( sync );
que.pop_front();
que.push_back( e );
}
StuffIt does that simple, generic job of popping the front, pushing the back, under lock. Since it takes an const Element &, step 3 of Thread A is already done. Since Element, as I suggest, is a DevArrayPtr, this is used with:
DeviceQue dq;
auto p = std::make_shared<DevArray>();
dq.StuffIt( p );
How the DevArray is populated is up to it's constructor or some function, the point is that a shared_ptr is used to transport it.
This brings up a more generic point about SafeStack. Obviously there is some potential for standard access functions, which could mimic std::deque, but the primary job for SafeStack is to lock/unlock for access control, and do something while under lock. To that end, I submit a generic functor is sufficient to generalize the notion. The preferred mechanics, especially with respect to boost, is up to you, but something like (code inside SafeStack):
bool LockedFunc( std::function< bool(DevQue &)> f )
{
Lock l( sync );
f( que );
}
Or whatever mechanics you like for calling a functor taking a DevQue as a parameter. This means you could fashion callbacks with complete access to the deque (and it's interface) while under lock, or provide functors or lambdas which perform specific tasks under lock.
The design point is to make SafeStack small, focused on that minimal task of doing a few things under lock, taking most any kind of data in the queue. Then, using that last point, provide the array under shared_ptr to provide the service of Thread B steps 3 and 4.
To be clear about that, keep in mind that whatever is done to the shared_ptr to copy it is similar to what can be done to simple POD types, like ints, with respect to containers. That is, one could loop through the elements of the DevQue fashioning a copy of those elements into another container in the same code which would do that for a container of integers (remember, it's a member function of a template - that type is generic). The resulting work is only copying pointers, which is less effort than copying entire arrays of data.
Now, step 4 isn't QUITE clear to me. It appears to say that you need to return a DevArray which is the accumulated content of all entries in the queue. That's trivial to arrange, but it might work a little better with a vector (as that's dynamically expandable), but as long as the std::array has sufficient room, it's certainly possible.
However, the only real difference between such an array and the queue's native "array of arrays" is how it is traversed (and counted). Returning one Element (step 3) is quick, but since step 4 is indicated under lock, that's a bit more than most locked functions should really do if they don't have to.
I'd suggest SafeStack should be able to provide a copy of que (a DeQue typedef), which is quick. Then, outside of the lock, Thread B has a copy of the DeQue ( a std::deque< DevArrayPtr > ) to fashion into it's own "giant array".
Now, more about that array. To this point I've not adequately dealt with marshalling it. I've just suggested that DevDat does that, but this may not be adequate. Certainly the content of the std::array or std::vector conveying a collection of DevDats could be written. Perhaps that deserves it's own outer structure. I'll leave that to you, because the point I've made is that SafeStack is now focused on it's small task (lock/access/unlock) and can take anything which can be owned by a share_ptr (or POD's and copyable objects). In the same way SafeStack is an outer shell marshalling a std::deque with a mutex, some similar outer shell could marshal read only access to the std::vector or std::array of DevDats, with a kind of write accessor used by Thread A. That could be a simple as something that only allows construction of the std::array to create it's content, after which read only access could be all that's provided.
I would suggest you to use boost::circular_buffer which is a fixed size container that supports random access iteration, constant time insert and erase at the beginning and end. You can use it as a FIFO with push_back(), read back() for the latest data saved and iterate over the whole container via begin(), end() or using operator[].
But at start-up the elements are not zeroed out. It has in my opinion an even more convenient interface. The container is empty at first and insertion will increase size until it reaches max size.
I have two threads, each of which has a function that manipulates the same std:map and std:vector variables.
What is the best way to keep these variables.
Thanks
It depends on the kind of manipulations. Do you only overwrite the stored values, or do you also insert / remove elements? In the former case you could lock only a specific element of the container (e.g. by embedding a std::mutex inside each element), whereas in the latter case you need to lock the entire container during each manipulation.
There is no universal best way. You need to sanitize all read/write calls to your synchronized structure through one functions that locks/unlocks mutex accordingly. You might have multiple functions but they should all operate on the same common mutex.
Its better to have a storage class and keep the map and vector as private member variables. and write forwarding functions in that class that locks/unlocks the mutex and forwards the read/write call to actual map or vector. then you have limited number of doors to access actual structures. and it will be easier to manage.
You may use boost::mutex as member variable of that class.
I've the following code pattern:
class A {
double a, b, c;
...
};
class B {
map<int, A> table; // Can have maximum of MAX_ROWS elements.
...
};
class C {
B entries;
queue<int> d;
queue<int> e;
...
};
Now I want to store an object of type C in a shared memory, so that different processes can append, update and read it. How can I do this? (Note: I know how to store a simple C array that has a fixed size in shared memory. Also, remember that B.table may have arbitrary entries.
Use boost::interprocess, this library exposes this functionality.
EDIT: Here are some changes you'll need to do:
The example already defines an allocator that will allocate from the shared memory block, you need to pass this to the map and the queue. This means you'll have to change your definitions:
class B
{
map<int, A, less<int>, MapShmemAllocator> table;
// Constructor of the map needs the instance of the allocator
B(MapShmemAllocator& alloc) : table(less<int>(), alloc)
{ }
}
For queue, this is slightly complicated, because of the fact that it's really just an adapter, so you need to pass in the real implementation class as a template parameter:
typedef queue<int, deque<int, QueueShmemAllocator> > QueueType;
Now your class C changes slightly:
class C
{
B entries;
QueueType d, e;
C(MapShmemAllocator& allocM, QueueShmemAllocator& allocQ) : entries(allocM), d(allocQ), e(allocQ)
{ }
}
Now from the segment manager, construct an instance of C with the allocator.
C *pC = segment.construct<C>("CInst")(allocM_inst, allocQ_inst);
I think that should do the trick. NOTE: You will need to provide two allocators (one for queue and one for map), not sure if you can construct two allocators from the same segment manager, but I don't see why not.
Building and using STL objects in shared memory is not tricky yet (especially using boost::interprocess wrappers). For sure you should also use syncing mechanisms (also not a problem with boost's named_mutex).
The real challenge is to keep consistency of STL objects in a shared memory. Basically, if one of the processes crashes in a bad point in time, it leaves other processes with a two big problems:
A locked mutex (can be resolved using tricky PID-to-mutex mappings, robust mutexes (wherever available), timed mutexes etc.
An STL object in the inconsistent state (e.g. semi-updated map structure during erase() procedure). In general, this is not recoverable yet, you need to destroy and re-construct object in a shared memory region from the scratch (probably killing all other processes as well). You may try to intercept all possible external signals in your app and crossing fingers hope everything will go well and process never fail in a bad moment.
Just keep this in mind when deciding to use shared memory in your system.
UPD: check shmaps (https://github.com/rayrapetyan/shmaps) project to get an idea of how things should work.
This can be tricky. For starters, you'll need a custom allocator: Boost
Interprocess has one, and I'd start with it. In your exact example,
this may be sufficient, but more generally, you'll need to ensure that
all subtypes also use the shared memory. Thus, if you want to map from
a string, that string will also need a custom allocator, which means
that it has a different type than std::string, and you can't copy or
assign to it from an std::string (but you can use the two iterator
constructor, e.g.:
typedef std::basic_string<char, std::char_traits<char>, ShmemAllocator> ShmemString;
std::map<ShmemString, X, std::less<ShmemString>, ShmemAllocator> shmemMap;
with accesses like:
shmemMap[ShmemString(key.begin(), key.end())] ...
And of course, any types you define which go into the map must also use
shared memory for any allocations: Boost Interprocess has an
offset_ptr which may help here.