I am wondering if you can : lock only a line or a single character in a file in linux and the rest of the file should remain accessible for other processes?
I received a task regarding simulating transaction on a file with c/c++ under linux .
Please give me an answer and if this answer is yes ,give me some links from where i could take a peek to make this task.
Thanks,
Madicemickael
fcntl() is the one API to choose, since it is the least broken and is POSIX. It is the only one that works across NFS. That said it is a complete disaster, too, since locks are bound to processes, not file descriptors. That means that if you lock a file and then some other thread or some library function locks/unlocks it, your lock will be broken too. Also, you cannot use file system locks to protect two threads of the same process to interfere with each other. Also, you should not use file locks on files that are accessible to more than one user, because that effectively enables users to freeze each others processes.
In summary: file locking on Unix creates more problems than it solves. Before you use it you need to be really sure you fully understand the semantics.
Yes, this is possible.
The Unix way to do this is via fcntl or lockf.
Whatever you choose, make sure to use only it and not mix the two. Have a look at this question (with answer) about it: fcntl, lockf, which is better to use for file locking?.
If you can, have a look at section 14.3 in Advanced Programming in the UNIX Environment.
lockf(3) can apply a lock to a section of a file.
Related
Basically, I want to compress a file in a pthread thread using gzip.
The first solution that pops up in mind and on Google is to call system().
What does the stackoverflow community suggest?
Shall I use system() in a pthread?
Or shall I myself just fork and exec in pthread? But since pthread is a thread, is it advisable to do a fork() and exec() in pthread thread?
Or what is the better approach than the above?
You shouldn't use system for this, but not because it's expensive. (For any file that is worth bothering to compress, the overhead of any technique for invoking a background gzip compression is negligible relative to the cost of doing the compression itself.) The reason you shouldn't use system is, system invokes a shell, and that means you have to worry about quoting the arguments. If you use fork and execvp instead, you don't have to worry about quoting anything.
The problems associated with mixing fork and wait with threads are real, but they are tractable. If your OS has posix_spawn, it will take care of some of those problems for you. I don't normally recommend posix_spawn because code that uses it is, in general, harder to maintain than code that uses fork, but for this application it should be OK. I cannot fit a comprehensive guide to mixing fork and wait with threads into this answer box.
An alternative you should consider is compressing the data in the thread that would be waiting for the gzip process, using zlib. This avoids the problems of mixing fork and wait with threads, but it adds a library dependency to your program, which may not be as convenient as relying on an external gzip executable.
Start with system call in another thread and only add complexity when needed.
The extra complexity of doing fork/exec or using a zip library is only worth the effort if system is not sufficient for some reason (i.e. you want to redirect both stdin and stdout of the child process into your parent process, or your want to compress a file in memory for sending it over the network without writing new files).
Suppose I have two processes. One always resides in memory and periodically reads some settings from a file on a disk. If it detects that settings was changed then it applies them.
The other process runs under command line by demand and modifies the settings. Thus the first process only read the file and never write to it while the second can only write to the file.
Should I synchronize the access to the file to ensure that the first process will always get consistent settings i.e. before or after modifications not some intermediate contents? If yes, what is the simplest way to do this in C++.
I'm interested mainly in cross-platform ways. But also curious about Windows- and/or Linux-specific ones.
Use a named semaphore and require either process to hold the semaphore before editing the file on disk. Named semaphores can be connected to by any running application.
Look at man 7 sem_overview for more information on named semaphores on linux machines.
The closest equivalent for windows I can find is http://msdn.microsoft.com/en-us/library/windows/desktop/ms682438(v=vs.85).aspx
You are using C++ so your first option should be to check through the usual cross-platform libs - POCO, Boost, ACE, and so forth to see if there is anything that already does what you require.
You really have two separate issues: (1) file synchronization and (2) notification.
On Linux to avoid having your daemon constantly polling to see if a file has changed you can use inotify calls and set up events that will tell you when the file has been changed by the command line program. It might be simplest to look for IN_CLOSE_WRITE events since a CL prog will presumably be opening, changing, and closing the file.
For synchronization, since you are in control of both programs, you can just use file or record locking e.g. lockf, flock or fcntl.
The most obvious solution is to open the file in exclusive mode. If the file can not be opened, wait some time and try to open the file again. This will prevent possible access/modification conflicts.
The benefit of this approach is that it's simple and doesn't have significant drawbacks.
Of course you could use some synchronization primitives (Mutex, Semaphore depending on the OS) but this would be an overkill in your scenario, when speedy response is not required (waiting 200 msec between open attempts is fine, and writing of config file won't take more).
C++:
Is there a way to check if a file has been opened for writing by another process/ class/ device ?
I am trying to read files from a folder that may be accessed by other processes for writing. If I read a file that is simultaneously being written on, both the read and the write process give me errors (the writing is incomplete, I might only get a header).
So I must check for some type of condition before I decide whether to open that specific file.
I have been using boost::filesystem to get my file list. I want compatibility with both Unix and Windows.
You must use a file advisory lock. In Unix, this is flock, in Windows it is LockFile.
However, the fact that your reading process is erroring probably indicates that you have not opened the file in read-only mode in that process. You must specify the correct flags for read-only access or from the OS' perspective you have two writers.
Both operating systems support reader-writer locks, where unlimited readers are allowed, but only in the absence of writers, and only at most one writer at a time will have access.
Since you say your system is multi-process (ie, not multi thread), you can't use a condition variable (unless it's in interprocess shared memory). You also can't use a single writer as a coordinator unless you're willing to shuttle your data there via sockets or shared memory.
From what I understand about boost::filesystem, you're not going to get the granularity you need from that feature-set in order to perform the tasks you're requesting. In general, there are two different approaches you can take:
Use a synchronization mechanism such as a named semaphore visible at the file-system level
Use file-locks (i.e., fcntl or flock on POSIX systems)
Unfortunately both approaches are going to be platform-specific, or at least specific to POSIX vs. Win32.
A very nice solution can be found here using Sutter's active object https://sites.google.com/site/kjellhedstrom2/active-object-with-cpp0x
This is quite advanced but really scaled well on many cores.
I have found a few references to implementation but only one clear description in C++ (Joseph Newcomers article http://www.flounder.com/nomultiples.htm#CreateMutex), but it was (c)1999 so I was a little reluctant to use it without first checking if there were "newer/better" ways today.
Thanks
Any named object will do, can be a file, mutex, event, mailslot, TCP port, etc. ERROR_ALREADY_EXISTS tells you whether an instance already existed.
For objects in the Win32 kernel namespace, there is one change since 1999 -- because of terminal services, you now can use a prefix of Global\ or Local\ to specify whether it's one instance on the entire computer vs one per user logon session.
If you want something more portable, then binding a TCP port, or creating a file and exclusively locking it, tend to work well across a variety of OSes.
If you are programming on Windows, the standard way to do this is indeed to create a mutex at the start of your program. The mutex should live as long as the lifetime of your program and during this time, attempts to create a mutex with the same name will fail.
There hasn't been any significant improvement since then. For that matter, there hasn't been any significant improvement since a Usenet post I wrote a few years before that.
What will happen if files are accessed concurrently from different processes/threads?
I understand there is no standard way of locking a file, only os specific functions.
In my case files will be read often and written seldom.
Now if A open a file for reading (ifstream) and starts reading chunks. And B opens the same file for writing (ofstream) and starts writing. What will happen? Is there a defined behavior?
edit
My goal is concurrent read, write access to many files. But write access will not occur very often. I would be content if the fstreams guarantee that file content doesn't get mixed up.
E.g.:
Process 1 and 2 write to file A. If they write concurrently I dont't care if the version of 1 or 2 is written to disc, as long as it is a consistent version of the file.
If a process reads a file and another writes to it at the same time, I want the reading process to get the "old" version of the file.
If fstreams don't handle this I will use a database.
There is certainly no portable way to do efficient file sharing (with simultaneous access) using C++.
You can share files using a "lock" file. Before opening "foo.dat", try to create file "foo.lock". Keep looping until you succeed. After access, delete foo.lock. That allows serial access, but not concurrent access.
You can use byte-level locking in platform-specific ways. Windows has LockFileEx(). POSIX has fcntl and flock. If you need multi-platforms you will need separate implementations. You can encapsulate them in a class and use #if to handle the platform-specific bits.
This is the most efficient (fastest) by a lot, but it involves very complex programming and is prone to bugs.
You can use a DBMS.
A DBMS will be simplest by a lot, but it does tie you to an external product which may or may not be a problem. Byte-wise locking is much faster than anything else, but will add a lot to devel and maintenance costs.
What is your goal? Are you trying to prevent concurrent read/write operations to files or do you want to implement some form of IPC via files?
Either way, look at boost interprocess, it provides you the opportunity to use file locks (and other cool stuff for IPC) And it has the added advantage of being portable!