I am using a library named "JSON for Modern C++" (https://github.com/nlohmann/json) which is pretty slick, letting me use JSON configuration files by a C++ program that are shared with a Javascript server side application. This library essentially creates another datatype that is accessed and manipulated in way that is very close to the same as a Javascript JSON objects.
My question is, do I need to be concerned about thread safety on JSON variable accesses and manipulations or can I trust the library is thread safe. I've looked in the documentation and I don't see it say it is thread safe but I also don't see anywhere that says it isn't thread safe.
Is anyone else using this library in a multithreaded environment? Did you need to protect it yourself or did the library protect itself. Maybe I'm really lucky and the repository author nlohmann will answer directly!
Any help is greatly appreciated!
nlohmann library is NOT thread safe. Take a look at the header file. It's a single one. There's no mutexes, locks or atomics or anything related to threads.
https://github.com/nlohmann/json/blob/develop/src/json.hpp
You are responsible for protecting against concurrency of multiple threads accessing this data.
Per the author in About thread safety #2366:
No, the container is like a map or a vector: you have to ensure thread safety yourself.
Related
Problem (in short):
I'm using POSIX Shared Memory and currently just used POSIX semaphores and i need to control multiple readers, multiple writers. I need help with what variables/methods i can use to control access within the limitations described below.
I've found an approach that I want to implement but i'm unsure of what methodology i can use to implement it when using POSIX Shared memory.
What I've Found
https://stackoverflow.com/a/28140784
This link has the algorithm i'd like to use but i'm unsure how to implement it with shared memory. Do i store the class in shared memory somehow? This is where I need help please.
The reason I'm unsure is a lot of my research, points towards keeping shared memory to primitives only to avoid addressing problems and STL objects can't be used.
NOTE:
For all my multi-threading i'm using C++11 features. This shared memory will be completely seperate program executables using C++11 std::threads from which any thread of any process/executable will want access. I have avoided the Linux pthread for any of my multi-threading and will continue to do so (except if its just control variable not actual pThreads).
Solution Parameters aimed for
Must be shareable between 2+ processes which will be running multiple C++11 std::thread that may wish access. I.e. Multiple Writers (exclusive one at a time) while allowing multiple simultaneous readers when no writer wants access.
Not using BOOST libraries. Ideally native C++11 or built in linux libraries, something that will work without the need to install abstract libraries.
Not using pThread actual threads but could use some object from there that will work with C++11 std::thread.
Ideally can handle a process crash while in operation. E.g. Using POSIX semaphore if a process crashes while it has the semaphore, everyone is screwed. I have seen people using file locks?
Thanks in advance
keeping shared memory to primitives only to avoid addressing problems
You can use pointers in and to shared memory objects across programs, so long as the memory is mmaped to the same address. This is actually a straightforward proposition, especially on 64 bit. See this open source C library I wrote for implementation details: rszshm - resizable pointer-safe shared memory.
Using POSIX semaphore if a process crashes while it has the semaphore, everyone is screwed.
If you want to use OS mediated semaphores, the SysV semaphores have SEM_UNDO, which recovers in this case. OTOH pthread offers robust mutexes that can be embedded and shared in shared memory. This can be used to build more sophisticated mechanisms.
The SysV scheme of providing multiple semaphores in a semaphore set, where a group of actions must all succeed, or the call blocks, permits building sophisticated mechanism too. A read/write lock can be made with a set of three semaphores.
I'm programming in C++ on linux. My program uses two threads when one reads and writes to a shared data-structure. The data-structure is from type - Boost::bimaps::unordered_set_of .
So my question is whether I need to worry regarding any synchronizing issues. I.E, do I need to protect the read and write fro the data-structure with locks (or something like that)? Or maybe I will work fine without any use in mutexes?
Thanks.
You should work with the mutex provided by boost
http://www.boost.org/doc/libs/1_41_0/doc/html/thread/synchronization.html#thread.synchronization.mutex_concepts
In common with the standard containers, Boost.Bimap does not provide thread synchronisation. You will have to provide that yourself.
While reading up on POSIX threading, I came across an example of thread-specific-data. I did have one area of confusion in my mind...
The thread-specific-data interface looks a little clunky, especially once you mix in having to use pthread_once, the various initializers, etc.
Is there any reason I can't just use a static std::map where the key is the pthread_self() id and the data value is held in the second part of the std::pair?
I can't think of a reason that this wouldn't work as long as it was wrapped in a mutex, but I see no suggestion of it or anything similar which confuses me given it sounds much easier than the provided API. I know threading can have alot of catch-22's so I thought I'd ask and see if I was about to step in... something unpleasant? :)
I can't think of a reason that this wouldn't work as long as it was wrapped in a mutex
That in itself is a very good reason; implemented properly, you can access your thread-specific data without preventing other threads from simultaneously creating or accessing theirs.
There's also general efficiency (constant time access, versus logarithmic time if you use std::map), no guarantee that pthread_t has a suitable ordering defined, and automatic cleanup along with all the other thread resources.
You could use C++11's thread_local keyword, or boost::thread_specific_ptr, if you don't like the Posix API.
pthread thread-specific-data existed before the standard library containers
thread-specific-data avoids the need of locking and makes sure no other thread messes with the data
The data is cleaned up automatically when the thread disappears
Having said that, nothing stops you from using your own solution. If you can be sure that the container is completely constructed before any threads are running (static threading model), you don't even need the mutex.
I found contradictory information on the web:
http://www.sgi.com/tech/stl/thread_safety.html
The SGI implementation of STL is thread-safe only in the sense that
simultaneous accesses to distinct containers are safe, and
simultaneous read accesses to to shared containers are safe. If
multiple threads access a single container, and at least one thread
may potentially write, then the user is responsible for ensuring
mutual exclusion between the threads during the container accesses.
http://gcc.gnu.org/onlinedocs/libstdc++/manual/using_concurrency.html
The user-code must guard against concurrent method calls which may
access any particular library object's state. Typically, the
application programmer may infer what object locks must be held based
on the objects referenced in a method call. Without getting into great
detail, here is an example which requires user-level locks:
All library objects are safe to use in a multithreaded program as long
as each thread carefully locks out access by any other thread while it
uses any object visible to another thread, i.e., treat library objects
like any other shared resource. In general, this requirement includes
both read and write access to objects; unless otherwise documented as
safe, do not assume that two threads may access a shared standard
library object at the same time.
I bolded the imporant part - maybe I dont understand what they mean by that,when I read object state I think of STL containers
How I understand this:
both documents say the same in different manner. MS STL implementation (actually Dinkumware one) says almost the same as your quoted SGI doc. They mean that they did nothing to make STL objects (e.g. containers) thread-safe, most probably because this would add an overhead unnecessary in many single-threaded applications. Any object is thread-safe in their terms, you can read it from multiple threads.
Also docs guarantee that STL objects are not modified under the hood in some background threads.
FWIW I updated the libstdc++ docs a while ago, it now says (emphasis mine):
The user code must guard against concurrent function calls which access any particular library object's state when one or more of those accesses modifies the state.
The information you cite is not contradictory. STL libraries should be safe to be used in a multi-threaded environment (actually, I've worked with one implementation where it was not the case) but it is users' burden to synchronize access to library objects. For instance, if you create a set of ints in one thread and another set of ints in another thread and you don't share either of them among threads, you should be able to use them; if you share an instance of a set among threads, it's up to you to synch the access to the set.
STL is no more. It is superseded by the C++ Standard Library. If you use the ISO C++ and the Standard Library, you should read (a) the Standard and (b) documentation that comes with your implementation of C++.
SGI STL documentation is mostly of historical interest, unless you for some reason actually use SGI STL.
I have implemented a Observer Pattern, in C++ project.
My Subject is a XML File reader which reads tags and publishes its value.
I have some "processing objects" which are my observers. They check the tag that has currently been read, if they have subsribed to the tag they will process it else ignore it.
I have banks of memory into which the tags and their values are dumped into.
My problem no is, how do I synchronise the memory operations?
When my XML reader wants to publish some tag/value, its should get a unused block of memory and "lock" it so that on its unavailable for editing. Once all the "Processing Objects" are done with the memory, they should be able to "unlock" for further use.
How can I achieve this? please help.
have you checked out boost shared memory? It has various synchronization mechanisms and examples...
The synchronization mechanisms outlined in the interprocess library are specifically useful if you want to put the mutexes in the shared memory block itself.
I assume your main task is not to learn/develop the synchronization mechanism.
You should reuse existing components available, and there are many. This is very good http://www.rabbitmq.com/getstarted.html
It will support multiple (including distributed/networking) models. Although there might initial learning period, but once integrated you can continue using it for your feature extensions, and focus on the meat of problem instead.