I have to develop an application which parses a log file and sends specific data to a server. It has to run on both Linux and Windows.
The problem appears when I want to test the log rolling system (which appends .1 to the name of the creates a new one with the same name). On Windows (haven't tested yet on Linux) I can't rename a file that I have opened with std::ifstream() (exclusive access?) even if I open it in "input mode" (ios::in).
Is there a cross-platform way to open file in a non-exclusive way?
Is there a way to open file in a non-exclusive way,
Yes, using Win32, passing the various FILE_SHARE_Xxxx flags to CreateFile.
is it cross platform?
No, it requires platform-specific code.
Due to annoying backwards compatibility concerns (DOS applications, being single-tasking, assume that nothing can delete a file out from under them, i.e. that they can fclose() and then fopen() without anything going amiss; Win16 preserved this assumption to make porting DOS applications easier, Win32 preserved this assumption to make porting Win16 applications easier, and it's awful), Windows defaults to opening files exclusively.
The underlying OS infrastructure supports deleting/renaming open files (although I believe it does have the restriction that memory-mapped files cannot be deleted, which I think isn't a restriction found on *nix), but the default opening semantics do not.
C++ has no notion of any of this; the C++ operating environment is much the same as the DOS operating environment--no other applications running concurrently, so no need to control file sharing.
It's not the reading operation that's requiring the exclusive mode, it's the rename, because this is essentially the same as moving the file to a new location.
I'm not sure but I don't think this can be done. Try copying the file instead, and later delete/replace the old file when it is no longer read.
Win32 filesystem semantics require that a file you rename not be open (in any mode) at the time you do the rename. You will need to close the file, rename it, and then create the new log file.
Unix filesystem semantics allow you to rename a file that's open because the filename is just a pointer to the inode.
If you are only reading from the file I know it can be done with windows api CreateFile. Just specify FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE as the input to dwShareMode.
Unfortunally this is not crossplatform. But there might be something similar for Linux.
See msdn for more info on CreateFile.
EDIT: Just a quick note about Greg Hewgill comment. I've just tested with the FILE_SHARE* stuff (too be 100% sure). And it is possible to both delete and rename files in windows if you open read only and specify the FILE_SHARE* parameters.
I'd make sure you don't keep files open. This leads to weird stuff if your app crashes for example.
What I'd do:
Abstract (reading / writing / rolling over to a new file) into one class, and arrange closing of the file when you want to roll over to a new one in that class. (this is the neatest way, and since you already have the roll-over code you're already halfway there.)
If you must have multiple read/write access points, need all features of fstreams and don't want to write that complete a wrapper then the only cross platform solution I can think of is to always close the file when you don't need it, and have the roll-over code try to acquire exclusive access to the file a few times when it needs to roll-over before giving up.
Related
Is it safe enough to store a file in the %TEMP% directory via GetTempPath, GetTempPath and CreateFile for more than two hours? Is there any guarantees that this file won't be deleted earlier?
Thanks in advance.
A file you create in the TEMP directory must be created with the CreateFile's FILE_FLAG_DELETE_ON_CLOSE option. This ensures that the file will always be cleaned-up and you cannot spray garbage files, even if your program crashes before it has a chance to delete the file again.
This option then also inevitably forces you to do the Right Thing, keeping the file opened while you are using it. Which in turn prevents anybody from the deleting the file, even if they use a sledge-hammer.
Lots of programs don't follow this advice and a user's TEMP directory tends to be a big olde mess, forcing the user to clean it up manually once in a while. A built-in feature of Windows, he'll use the "Disk Cleanup" applet. The kind of scenario where you will lose the file if you don't follow this advice. Best to use %AppData% instead.
There are no guarantees. This folder is usually not cleared except the user starts any cleanup.
But everyone can delete files here. And it is wise to do that on a regular base
To prevent the file from being deleted, you can keep a handle open (assuming the application is running the whole time) and do not specify FILE_SHARE_DELETE (and, if applicable, neither FILE_SHARE_WRITE).
Alternative:
Use a path in %APPDATA% or %PROGRAMDATA% that you clear yourself regulary, or let the user specify a path.
In addition, you could register a scheduled task to clean the folder regulary.
If you do not want that another process can delete your files, just keep them open with a share mode of FILE_SHARE_READ | FILE_SHARE_WRITE. That way any attempt to delete them will fail, but any other process will be able to read or write them.
BTW : this is not related with the files living in %TEMP% folder.
If you cannot have a process to keep them open all the time, you must rely on other processes (and other users) on your system not doing anything ...
It is easier to explain with example.
When 2 text editors edit the same text file in the same time, when one editor saves the file, the other one understands that it was modified and asks to do smth.
How is it possible to get a signal that a file was modified outside the program?
I am working with c++ (though I think it isn't important) and on linux. (solution for windows would be good too)
ISO-C++ does not offer this functionality, so you have to stick with what the operating system provides.
On Linux that would be inotify, on Windows you would use directory change notifications.
① Check the timestamp of the file as close as possible before writing. If it is not what it was when you last opened this file for reading, then beware!
② You can build a checksum of the file and compare this to one you built earlier.
③ Register to a system service which informs you about file activities. This depends on the goodwill of the OS you are using; if this notification service isn't working properly, your stuff will fail. On Linux have a look at Inotify.
There is a static library I use in my program which can only take filenames as its input, not actual file contents. There is nothing I can do about the library's source code. So I want to: create a brand-new file, store data to being processed into it, flush it onto the disk(?), pass its name to the library, then delete it.
But I also want this process to be rather secure:
1) the file must be created anew, without any bogus data (maybe it's not critical, but whatever);
2) anyone but my process must not be able read or write from/to this file (I want the library to process my actual data, not bogus data some wiseguy managed to plug in);
3) after I'm done with this file, it must be deleted (okay, if someone TerminateProcess() me, I guess there is nothing much can be done, but still).
The library seems to use non-Unicode fopen() to open the given file though, so I am not quite sure how to handle all this, since the program is intended to run on Windows. Any suggestions?
You have a lot of suggestions already, but another option that I don't think has been mentioned is using named pipes. It will depend on the library in question as to whether it works or not, but it might be worth a try. You can create a named pipe in your application using the CreateNamedPipe function, and pass the name of the pipe to the library to operate on (the filename you would pass would be \\.\pipe\PipeName). Whether the library accepts a filename like that or not is something you would have to try, but if it works the advantage is your file never has to actually be written to disk.
This can be achieved using the CreateFile and GetTempFileName functions (if you don't know if you can write to the current working directory, you may also want to use , GetTempPath).
Determine a directory to store your temporary file in; the current directory (".") or the result of GetTempPath would be good candidates.
Use GetTempFileName to create a temporary file name.
Finally, call CreateFile to create the temporary file.
For the last step, there are a few things to consider:
The dwFlagsAndAttributes parameter of CreateFile should probably include FILE_ATTRIBUTE_TEMPORARY.
The dwFlagsAndAttributes parameter should probably also include FILE_FLAG_DELETE_ON_CLOSE to make sure that the file gets deleted no matter what (this probably also works if your process crashes, in which case the system closes all handles for you).
The dwShareMode parameter of CreateFile should probably be FILE_SHARE_READ so that other attempts to open the file will succeed, but only for reading. This means that your library code will be able to read the file, but nobody will be able to write to it.
This article should give you some good guidelines on the issue.
The gist of the matter is this:
The POSIX mkstemp() function is the secure and preferred solution where available. Unfortunately, it is not available in Windows, so you would need to find a wrapper that properly implements this functionality using Windows API calls.
On Windows, the tmpfile_s() function is the only one that actually opens the temporary file atomically (instead of simply generating a filename), protecting you from a race condition. Unfortunately, this function does not allow you to specify which directory the file will be created in, which is a potential security issue.
Primarily, you can create file in user's temporary folder (eg. C:\Users\\AppData\Local\Temp) - it is a perfect place for such files. Secondly, when creating a file, you can specify, what kind of access sharing do you provide.
Fragment of CreateFile help page on MSDN:
dwShareMode
0 Prevents other processes from opening a file or device
if they request delete, read, or write access.
FILE_SHARE_DELETE Enables subsequent open operations on a file or device to
request delete access. Otherwise, other processes cannot open the file or device if they
request delete access. If this flag is not specified, but the file or device has been opened for delete access, the function fails. Note: Delete access allows both delete and rename operations.
FILE_SHARE_READ Enables subsequent open operations on a
file or device to request read access. Otherwise, other processes cannot open the file or device if they request read access. If this flag is not specified, but the file or device has been opened for read access, the function fails.
FILE_SHARE_WRITE Enables subsequent open operations on a file or device to request
write access.
Otherwise, other processes cannot open the file or device if they
request write access.
If this flag is not specified, but the file or device has been opened
for write access or has a file mapping with write access, the function
fails.
Whilst suggestions given are good, such as using FILE_SHARE_READ, FILE_DELETE_ON_CLOSE, etc, I don't think there is a completely safe way to do thist.
I have used Process Explorer to close files that are meant to prevent a second process starting - I did this because the first process got stuck and was "not killable and not dead, but not responding", so I had a valid reason to do this - and I didn't want to reboot the machine at that particular point due to other processes running on the system.
If someone uses a debugger of some sort [including something non-commercial, written specifically for this purpose], attaches to your running process, sets a breakpoint and stops the code, then closes the file you have open, it can write to the file you just created.
You can make it harder, but you can't stop someone with sufficient privileges/skills/capabilities from intercepting your program and manipulating the data.
Note that file/folder protection only works if you reliably know that users don't have privileged accounts on the machine - typical Windows users are either admins right away, or have another account for admin purposes - and I have access to sudo/root on nearly all of the Linux boxes I use at work - there are some fileservers that I don't [and shouldn't] have root access. But all the boxes I use myself or can borrow of testing purposes, I can get to a root environment. This is not very unusual.
A solution I can think of is to find a different library that uses a different interface [or get the sources of the library and modify it so that it]. Not that this prevents a "stop, modify and go" attack using the debugger approach described above.
Create your file in your executable's folder using CreateFile API, You can give the file name some UUID, each time its created, so that no other process can guess the file name to open it. and set its attribute to hidden. After using it, just delete the file .Is it enough?
It is possible to move a .sqlite file somewhere while not corrupting it in C or C++?
Somewhere could be another folder or something.
If so could you give me some tips/pointers.
Yes, of course. It is a regular file that can be moved around just like any other file.
The sqlite engine itself will make sure that it does not corrupt. Either it is in a clean state, or locked when being written to (with journaling).
One thing to remember that you should not use the database file on filesystems that do not have reliable locking, e.g. on network disks.
Well, it depends a little. It is just a regular file that (assuming nothing is connected to it, writing to it at the time) can be copied without problem. Welcome to the world of open source systems where things really are that simple (ie you don't need to update the registry, a config file or two, reapply installer settings or anything like that).
However, things are tricky if something is using the file. When an app opens a sqlite file it can tell the OS to lock it - the default is open with exclusive access for the opening app (using a certain flag). In such a case, you will not be able to move the file as the OS will prevent it until the app that has it locked is stopped.
Under Windows is there a way to modify a file/executable opened by another process using c++?
Is there a way to modify an open executable in windows?
No.
Is there a way to modify an open file in windows using c++?
Yes. If it has been opened with the proper share permissions. See http://msdn.microsoft.com/en-us/library/aa363858(VS.85).aspx FILE_SHARE_WRITE
It may be possible but perhaps not easy to achieve. You need inject thread in destination process and know PE format for correctly edit opened file and modify it.
All information is on web.
Good Luck.
I find this freeware tool, it proposes to unlock files and folders.
The OS holds the executable file open for read-only sharing as long as it's running, so there's no way to modify it directly. You can, however, open it for reading (if you specify read-sharing in your CreateFile call), and make a modified copy of it, while it's running.
I don't know if that's what you had in mind, but if it's your own program you're doing this to, you can start the new copy and have it pick up where the previous one left off... not straightforward, but not all that difficult either.