I intend to perform opening for reading a single file from many threads using std::ifstream. My concern is if std::ifstream is thread-safe & lock-free?
More details:
I use g++ 4.4 on Ubuntu & Windows XP, 4.0 on Leopard.
Each thread creates its own instance of std::ifstream
Thanks in advance!
That is implementation defined. Standard C++ says absolutely nothing about threading, and therefore any assumptions about threads inherently invoke unspecified or implementation defined behavior.
We need the platform you are using to be more specific, but it's probably unreasonable to assume ifstream is either thread safe or lock free. If nothing else, there are probably locks involved in the OS level calls that actually do the reading from the file, in which case no true lock-free implementation is possible. Even without that, each read from an ifstream needs to check several format flags, and needs to update the flags bits depending on what occurs during the read. (i.e. istream::good() and istream::operator bool) Since there is no way all of that can be done atomicly, it's unreasonable to assume much about istream's thread safety characteristics.
See http://gcc.gnu.org/onlinedocs/libstdc++/manual/using_concurrency.html.
As of the writing of that manual page, GCC's standard library defers to the operating system's C stdio file buffering. They avoid keeping state outside the C FILE structure and achieve some level of safety through it.
Since the C stdio library implements a buffer of a single range within the file around the last I/O operation, I don't see how a lock-free implementation is possible. The operations on a file must be processed serially. Perhaps unbuffered mode could help; that's a little more research than I'd like to do right now.
All std libraries are thread safe but not "async" safe. So you can call the same functions from different threads but not on the same objects.
Related
What would happen if you call read (or write, or both) in two different thread, on the same file descriptor (lets says we are interested about a local file, and a it's a socket file descriptor), without using explicitly a synchronization mechanism?
Read and Write are syscall, so, on a single core CPU, it's probably unlucky that two read would be executed "at the same time". But with multiple cores...
What the linux kernel will do?
And let's be a bit more general : is the behavior always the same for other kernels (like BSDs) ?
Edit : According to the close documentation, we should be sure that the file descriptor isn't used by a syscall in an other thread. So it seams that explicit synchronization would be required before closing a file descriptor (and so, also around read/write if thread that may call it are still running).
Any system level (syscall) file descriptor access is thread safe in all mainstream UNIX-like OSes.
Though depending on the age they are not necessarily signal safe.
If you call read, write, accept or similar on a file descriptor from two different tasks then the kernel's internal locking mechanism will resolve contention.
For reads each byte may be only read once though and writes will go in any undefined order.
The stdio library functions fread, fwrite and co. also have by default internal locking on the control structures, though by using flags it is possible to disable that.
The comment about close is because it doesn't make a lot of sense to close a file descriptor in any situation in which some other thread might be trying to use it. So while it is 'safe' as far as the kernel is concerned, it can lead to odd, hard to diagnose corner cases.
If a thread closes a file descriptor while a second thread is trying to read from it, the second thread may get an unexpected EBADF error. Worse, if a third thread is simultaneously opening a new file, that might reallocate the same fd, and the second thread might accidentally read from the new file rather than the one it was expecting...
Have a care for those who follow in your footsteps
It's perfectly normal to protect the file descriptor with a mutex semaphore. It removes any dependence on kernel behaviour so your message boundaries are now certain. You then don't have to cite the last paragraph at the bottom of a 15,489 line manpage which explains why the mutex isn't necessary (I exaggerated, but you get my meaning)
It also makes it clear to anyone reading your code that the file descriptor is being used by more than one thread.
Fringe Benefit
There is a fringe benefit to using a mutex that way. Suppose you've got different messages coming from the different threads and some of those messages are more important than others. All you need to do is set the thread priorities to reflect their messages' importance. That way the OS will ensure that your messages will be sent in order of importance for minimal effort on your part.
The result would depend on how the threads are scheduled to run at that particular instant in time.
One way to potentially avoid undefined behavior with multi-threading is to assume that you are doing memory operations. E.g. updating a linked list or changing a variable, etc.
If you use mutex/semaphores/lock or some other synchronization mechanism, it should work as intended.
Visual Studio's fread "locks out other threads." There is an alternate version _fread_nolock, which reads "without locking other threads", which should only be used "in thread-safe contexts such as single-threaded applications or where the calling scope already handles thread isolation."
Even after reading other somewhat relevant discussions on the two, I'm confused if the locking fread implements is on a specific FILE struct, a specific actual file, or on all fread calls on totally different files.
If you use the nolock versions, what level of locking do you need to provide? Can multiple threads in parallel be reading separate files without any locking? Can multiple threads in parallel be writing separate files without any locking? Or are there global or static variables involved that would be corrupted?
So, by using the nolock versions, are you able to potentially achieve better I/O throughput (if you aren't needlessly moving heads, like reading off separate drives, or a SSD drive), or is the potential gain just reducing redundant locks to a single lock (which should be negligible.)
Does VS' ifstream.read function work just like the regular fread? (I don't see a nolock version of it.)
The MS standard library implementation fully supports multi-threading. The C++ standard explain this requirement:
27.2.3: Concurrent access to a stream object, stream buffer object, or C Library stream by multiple threads may result in a data
race unless otherwise specified.
If one thread makes a library call a that writes a value to a stream
and, as a result, another thread reads this value from the stream
through a library call b such that this does not result in a data
race, then a’s write synchronizes with b’s read.
This means that if you write on a stream, a locking (not file locking, but concurrent access locking to the in-memory stream data structure) is done, to be sure that concurrency is well manageged for all the other threads using the same stream.
This locking overhead is always there, even if not needed. This could have a performance aspect, according to Microsoft:
the performance of the multithreaded libraries has been improved and
is close to the performance of the now-eliminated single-threaded
libraries. For those situations when even higher performance is
required, there are several new features.
This is why _nolock functions are provided. They access the stream directly without thread locking. It must be used with extreme care, for example:
if your application is single threaded (another process using the same stream has its own data structure, and OS manageds concurrency here)
if you're sure that no two threads use the same stream (for example if you have only one reader thread and writing is done outside your porgramme).
if you have other synchronisation mechasnism that protect a critical section of your code. For example, if you use a mutex lock, or an thread safe non blocking algorithm that makes use of atomics.
In such cases, the additional lock for stream access is not needed/redundant. For file intensive functions, it could be worth using the no_lock then.
Note: as you've pointed out: it's only worth using the nolock for intensive file accesses where you make millions of accesses.
fread_no_lock() appears to be used once you make sure that the file is locked with an external mechanism (some form of mutex, probably), and then you use it to reduce overhead: related: What's the intended use of _fread_nolock, _fseek_nolock?
This may also answer any further questions you might have: it may or may not be possible for your hard-drive to actually perform more than I/O operation at the same time depending on what type of hard drive you have: https://superuser.com/questions/252959/which-is-faster-copying-everything-at-once-or-one-thing-at-a-time
We are writing a multi threaded application that does a bunch of bit twiddling and writes the binary data to disk. Is it possible to have each thread std::fopen the same file for writing at the same time? The reasoning would be each thread could do its work and have its own access to the writable file.
std::fstream has functionality defined in terms of the C stdio library. I would be surprised if it were actually specified, but the most likely behavior from opening the same file twice is multiple internal buffers bound to the same file descriptor.
The usual way to simultaneously write to multiple points in the same file is POSIX pwrite or writev. This functionality is not wrapped by C stdio, and by extension not by C++ iostreams either. But, having multiple descriptors to the same filesystem file might work too.
Edit: POSIX open called twice on the same file in Mac OS X produces different file descriptors. So, it might work on your platform, but it's probably not portable.
A definitive answer would require connecting these dots:
Where the C++ standard specifies that fstream works like a C (stdio) stream.
Where the C standard defines when a stream is created (fopen is only defined to associate a stream with a newly-opened file).
Where the POSIX standard defines its requirements for C streams.
Where POSIX defines the case of opening the same file twice.
This is a bit more research than I'm up for at the moment, but I'm sure someone out there has done the legwork.
I've written some high speed multi-threaded data capture utilities, but the output went to separate files on separate hard drives, and then were post-processed.
I seem to recall that you can have fopen not lock the file so in theory that would allow different threads to all write to the same file with independent handles. In practice you're going to run into other issues, namely concurrency. Your threads are almost certainly going to step all over each other and scramble the results unless you implement some synchronization. And if you have to do that, why not just use one handle across all the threads?
I/O access is not a parallelizable task (it can't be, you simply can't send two or more data addresses over the device bus at the same time) so you'd better implement a queue in which many threads posts their chunks of data and one single consumer actually writes them to disk.
Ive looked online and have not been able to satisfy myself with an answer.
Is memcpy threadsafe? (in Windows)
What I mean is if I write to an area of memory shared between processes (using boost::shared_memory_object) using a single memcpy and then try read that area from another
process using a single memcpy then will one process be blocked automatically
while that write is happening? Where can I read about this?
memcpy is typically coded for raw speed. It will not be thread safe. If you require this, you need to perform the memcpy call inside of a critical section or use some other semaphor mechanism.
take_mutex(&mutex);
memcpy(dst, src, count);
yield_mutex(&mutex);
memcpy is not thread/process safe
Routines like memcpy() (or memmove()) are part of standard C library, are included through standard <string.h> header and know nothing about any locking mechanics. Locking should be provided by some external way like inter-process mutexes, semaphores or things like this.
You are confusing "atomic" and "thread safe". If you read and write data (with or without memcpy) concurrently in a shared region, that is not safe. But of course copying data itself is thread safe.
memcpy itself is also thread safe, at least on POSIX systems see this one, and therefore I guess it is also on Windows. Anything else would make it quite useless.
If it would be "automatically blocking", it would have to be atomic (or at least manage it's own locks) which would slow down your system. So in your case you should use your own locks.
Reading the question Why doesn’t C++ STL support atoi(const string& ) like functions?, I encountered a comment which warned that GCC (at least) has a bug that can slow down multi-threaded applications which use ostringstream frequently. This is apparently due to a mutex 'needed' by the C++ locale machinery.
Given my recent interest in globalization practices, I wonder if anyone can explain to me why a locale object would need a mutex? What is it that can be changed in a locale that needs a mutex? Shouldn't it be a read-only facility?
It's really an implementation issue, but std::locale has a static function that retrieves and set the 'global' locale. The global locale is defined to be used in several areas of the standard library which implies that there must be a global locale somewhere.
In implementations that support threads it is very likely that this global locale needs to be protected via some sort of locking mechanism to prevent simultaneous access between threads from causing undesired behaviour.
As the current standard does not explicitly deal with threading at all, it's a set implementation choices as to how (or if) this locking mechanism is implemented and whether other shared data and locks are required.
The answer is probably lazy initialization. There's a lot of data behind the locale system, and it's pretty easy to make the mistake of coding the sequence:
take lock
check for initialization
read data if needed
release lock
and there you are.
Some of us don't trust the entire iostream mechanism as far as we can throw it from a threading performance standpoint. Since, oh, 1987, it has been full of unwanted locks with no way to declare that a single stream will be only used in a single thread.