Solved:
Workable solution: sbi's answer
Explanation for what really happens: Hans's answer
Explanation for why OpenFile doesn't pass through "DELETE PENDING": Benjamin's answer
The Problem:
Our software is in large part an interpreter engine for a proprietary scripting language. That scripting language has the ability to create a file, process it, and then delete the file. These are all separate operations, and no file handles are kept open in between these operations.
(I.e. during the file creation, a handle is created, used for writing, then closed. During the file processing portion, a separate file handle opens the file, reads from it, and is closed at EOF. And finally, delete uses ::DeleteFile which only has use of a filename, not a file handle at all).
Recently we've come to realize that a particular macro (script) fails sometimes to be able to create the file at some random subsequent time (i.e. it succeeds during the first hundred iterations of "create, process, delete", but when it comes back to creating it a hundred and first time, Windows replies "Access Denied").
Looking deeper into the issue, I have written a very simple program that loops over something like this:
while (true) {
HANDLE hFile = CreateFileA(pszFilename, FILE_ALL_ACCESS, FILE_SHARE_READ,
NULL, CREATE_NEW, FILE_ATTRIBUTE_NORMAL, NULL);
if (hFile == INVALID_HANDLE_VALUE)
return OpenFailed;
const DWORD dwWrite = strlen(pszFilename);
DWORD dwWritten;
if (!WriteFile(hFile, pszFilename, dwWrite, &dwWritten, NULL) || dwWritten != dwWrite)
return WriteFailed;
if (!CloseHandle(hFile))
return CloseFailed;
if (!DeleteFileA(pszFilename))
return DeleteFailed;
}
As you can see, this is direct to the Win32 API and is pretty darn simple. I create a file, write to it, close the handle, delete it, rinse, repeat...
But somewhere along the line, I'll get an Access Denied (5) error during the CreateFile() call. Looking at sysinternal's ProcessMonitor, I can see that the underlying issue is that there is a pending delete on the file while I'm trying to create it again.
Questions:
Is there a way to wait for the delete to complete?
Is there a way to detect that a file is pending deletion?
We have tried the first option, by simply WaitForSingleObject() on the HFILE. But the HFILE is always closed before the WaitForSingleObject executes, and so WaitForSingleObject always returns WAIT_FAILED. Clearly, trying to wait for the closed handle doesn't work.
I could wait on a change notification for the folder that the file exists in. However, that seems like an extremely overhead-intensive kludge to what is a problem only occasionally (to wit: in my tests on my Windows 7 x64 E6600 PC it typically fails on iteration 12000+ -- on other machines, it can happen as soon as iteration 7 or 15 or 56 or never).
I have been unable to discern any CreateFile() arguments that would explicitly allow for this ether. No matter what arguments CreateFile has, it really is not okay with opening a file for any access when the file is pending deletion.
And since I can see this behavior on both an Windows XP box and on an x64 Windows 7 box, I am quite certain that this is core NTFS behavior "as intended" by Microsoft. So I need a solution that allows the OS to complete the delete before I attempt to proceed, preferably without tying up CPU cycles needlessly, and without the extreme overhead of watching the folder that this file is in (if possible).
1 Yes, this loop returns on a failure to write or a failure to close which leaks, but since this is a simple console test application, the application itself exits, and Windows guarantees that all handles are closed by the OS when a process completes. So no leaks exist here.
bool DeleteFileNowA(const char * pszFilename)
{
// Determine the path in which to store the temp filename
char szPath[MAX_PATH];
strcpy(szPath, pszFilename);
PathRemoveFileSpecA(szPath);
// Generate a guaranteed to be unique temporary filename to house the pending delete
char szTempName[MAX_PATH];
if (!GetTempFileNameA(szPath, ".xX", 0, szTempName))
return false;
// Move the real file to the dummy filename
if (!MoveFileExA(pszFilename, szTempName, MOVEFILE_REPLACE_EXISTING))
return false;
// Queue the deletion (the OS will delete it when all handles (ours or other processes) close)
if (!DeleteFileA(szTempName))
return false;
return true;
}
There are other processes in Windows that want a piece of that file. The search indexer is an obvious candidate. Or a virus scanner. They'll open the file for full sharing, including FILE_SHARE_DELETE, so that other processes aren't heavily affected by them opening the file.
That usually works out well, unless you create/write/delete at a high rate. The delete will succeed but the file cannot disappear from the file system until the last handle to it got closed. The handle held by, say, the search indexer. Any program that tries to open that pending-delete file will be slapped by error 5.
This is otherwise a generic problem on a multitasking operating system, you cannot know what other process might want to mess with your files. Your usage pattern seems unusual, review that first. A workaround would be to catch the error, sleep and try again. Or moving the file into the recycle bin with SHFileOperation().
First rename the file to be deleted, and then delete it.
Use GetTempFileName() to obtain a unique name, and then use MoveFile() to rename the file. Then delete the renamed file. If the actual deletion is indeed asynchronous and might conflict with the creation of the same file (as your tests seems to indicate), this should solve the problem.
Of course, if your analysis is right and file operations happen somewhat asynchronous, this might introduce the problem that you attempt to delete the file before the renaming is done. But then you could always keep trying to delete in a background thread.
If Hans is right (and I'm inclined to believe his analysis), then moving might not really help, because you might not be able to actually rename a file that's open by another process. (But then you might, I don't know this.) If that's indeed the case, the only other way I can come up with is "keep trying". You would have to wait for a few milliseconds and retry. Keep a timeout to give up when this doesn't help.
Silly suggestion - since it fails so infrequently, simply wait some milliseconds on failure and try again.
Or, if latency is important, switch to another file name, leaving the old file to be deleted later.
This is maybe not your particular issue, but it's possible so I suggest you get out Process Monitor (Sysinternals) and see.
I had exactly the same problem and discovered that Comodo Internet Security (cmdagent.exe) was contributing to the problem. Previously I had a dual-core machine, but when I upgraded to an Intel i7 suddenly my working software (jam.exe by Perfore software) no longer worked because it had the same pattern (a delete then create, but no check). After debugging the problem I found GetLastError() was returning access denied, but Process Monitor reveals a 'delete pending'. Here is the trace:
10:39:10.1738151 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
10:39:10.1738581 AM jam.exe 5032 QueryAttributeTagFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS Attributes: ANCI, ReparseTag: 0x0
10:39:10.1738830 AM jam.exe 5032 SetDispositionInformationFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS Delete: True
10:39:10.1739216 AM jam.exe 5032 CloseFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1739438 AM jam.exe 5032 IRP_MJ_CLOSE C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1744837 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat DELETE PENDING Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0
10:39:10.1788811 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat DELETE PENDING Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0
10:39:10.1838276 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat DELETE PENDING Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0
10:39:10.1888407 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat DELETE PENDING Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0
10:39:10.1936323 AM System 4 FASTIO_ACQUIRE_FOR_SECTION_SYNCHRONIZATION C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS SyncType: SyncTypeOther
10:39:10.1936531 AM System 4 FASTIO_RELEASE_FOR_SECTION_SYNCHRONIZATION C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1936647 AM System 4 IRP_MJ_CLOSE C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1939064 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat DELETE PENDING Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0
10:39:10.1945733 AM cmdagent.exe 1188 CloseFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1946532 AM cmdagent.exe 1188 IRP_MJ_CLOSE C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1947020 AM cmdagent.exe 1188 IRP_MJ_CLOSE C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1948945 AM cfp.exe 1832 QueryOpen C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat FAST IO DISALLOWED
10:39:10.1949781 AM cfp.exe 1832 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat NAME NOT FOUND Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a
10:39:10.1989720 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0, OpenResult: Created
As you can see, there is a request to delete followed by several attempts to open the file again by jam.exe (it's an fopen in a loop). You can see cmdagent.exe presumably had the file open as it closes its handle and then suddenly jam.exe is able to now open the file.
Of course, the suggested solution to wait and try again, and it works just fine.
Is there a way to detect that a file is pending deletion?
Use the GetFileInformationByHandleEx function with the FILE_STANDARD_INFO structure.
But the function can't solve your problem. sbi's solution neither.
Since you're creating a new file, processing it, then deleting it, it sounds like you don't really care about what the file name is. If that's truly the case, you should consider always creating a temporary file. That way, each time through the process, you don't have to care that the file didn't yet get deleted.
I actually had the same issue while using the LoadLibrary(path). I couldn't delete the file in path.
The solution was to "close the handle" or use the FreeLibrary(path) method.
NOTE: Please read the "Remarks" on MSDN regarding the FreeLibrary().
On Windows Vista/Windows 7 there is DeleteFileTransacted which deletes a file using transactions which ensures they are deleted (flushes file buffers, etc.). For Windows XP compatibility this is not an option though.
Another idea how this might be done is use OpenFile() with the flag OF_CREATE which sets the length to zero if the file exists or creates it if it doesn't and then to call FlushFileBuffers on the file handle to wait for this operation (making the file zero length) to complete. On completion, the file is of size 0 and then simply call DeleteFile.
You can later test if the file exists or if it has zero-length to treat it the same way.
According to [1], you could use NtDeleteFile to avoid the asynchronous nature of DeleteFile. Also [1] gives some details on how DeleteFile works.
Unfortunately the official documentation on NtDeleteFile [2] doesn't mention any particular details on this issue.
[1] Undocumented functions of NTDLL
[2] ZwDeleteFile function
The best answer was given by sbi, but in the interest of completeness, some people might also want to know about a new way now available from Windows 10 RS1/1603.
It involves calling the SetFileInformationByHandle API with class FileDispositionInfoEx, and setting flags FILE_DISPOSITION_DELETE | FILE_DISPOSITION_POSIX_SEMANTICS. See the full answer by RbMm.
If CreateFile returns INVALID_HANDLE_VALUE then you should determine what GetLastError returns in your particular situation (pending delete) and loop back to CreateFile based on that error code only.
The FILE_FLAG_DELETE_ON_CLOSE flag might buy you something.
I just had this exact issue and took TWO steps to address it; I stopped using C/C++ stdlib apis and ::DeleteFile(..), and switched to:
::MoveFileEx(src,dest,MOVEFILE_WRITE_THROUGH);. See: MOVEFILE_WRITE_THROUGH
h = ::CreateFile(DELETE | SYNCHRONIZE,OPEN_EXISTING,FILE_FLAG_DELETE_ON_CLOSE | FILE_FLAG_OPEN_REPARSE_POINT); ::CloseHandle(h);
The above are pseudo calls showing the relevant flags, specifically note that there is NO sharing on the CreateFile call used to achieve delete.
Together they gave me better precision on the rename and delete semantics. They are working in my code now and have improved the precision and control from other threads/processes (watching the file-system for changes) interjecting actions on the file due to latencies [or sharing] in the other rename and/or delete APIs. Without that control, a file set to delete when its last kernel-handle was closed might actually languish open until the system was rebooted, and you might not know.
Hopefully those feedback snippets might prove useful to others.
Addendum: I happen to use hardlinks for a portion of the work I do. It turns out that although you can create hardlinks on a file that is OPEN, you cannot delete ANY of them until all handles to ANY of the underlying data-stream(s) to that NTFS file are closed. That is weird since:
the OS tracks by them using a single ID in what is effectively an INode (ntfs uid).
https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-openfilebyid
https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getfinalpathnamebyhandlew
https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-findfirstfilenamew
https://learn.microsoft.com/en-us/windows/win32/api/winbase/ns-winbase-file_id_info
you can create hardlinks while it is open, just not delete them
hardlinks are actually alias-names in the attributes (XATTRs) in NTFS
you can rename them while it is open
so changing the name doesn't matter
Which would lead you to think that only the last hardlink should be non-deletable while the kernel has one or more open-file handles referring to the hardlinked NTFS File's MFT-Entry/ATTRs. anyway, just another thing to know.
I think this is just by poor design in the file system. I have seen the same problem when I worked with communication ports, opening/closing them.
Unfortunately I think the simplest solution would be to just retry to create the file a number of times if you get an INVALID_HANDLE_VALUE. GetLastError() might also give you a better way of detecting this particular INVALID_HANDLE_VALUE.
I would have preferred overlapped I/O, but their CloseHandle() and DeleteFile() don't handle overlapped operations :(
Related
I had the following code which got executed every two
minutes all day long:
int sucessfully_deleted = DeleteFile(dest_filename);
if (!sucessfully_deleted)
{
// this never happens
}
rename(source_filename,dest_filename);
Once every several hours the rename() would fail with errno=13 (EACCES). The files involved were all sitting on a DropBox directory and I had a hunch that DropBox could be the cause. I figured that it might just be possible that the DeleteFile() function may return with a non-zero successfully_deleted but actually DropBox could still be busy doing some stuff in relation to the deletion that prevented rename() from succeeding. What I did next was to change rename() to my_rename() which would attempt a rename() and upon any failure would Sleep() for one second and try a second time. Sure enough that has worked perfectly ever since. What's more, I get a diagnostic message displaying first-attempt-failures every several hours. It has never failed on the second attempt.
So you could say that the problem is entirely solved... but I would like to understand what might be going on so as to better defend myself against any related DropBox issues in the future...
Really I would like to have a new super_delete() function which does not return until the file is properly deleted and finished with in all respects.
under windows request to delete file really never delete file just. it mark it FCB (File Control Block) with special flag (FCB_STATE_DELETE_ON_CLOSE). real deletion will be only when the last file handle will be closed.
The DeleteFile function marks a file for deletion on close. Therefore,
the file deletion does not occur until the last handle to the file is
closed. Subsequent calls to CreateFile to open the file fail with
ERROR_ACCESS_DENIED.
also if exist section ( memory-mapped file ) open on file - file even can not be marked for delete. api call fail with STATUS_CANNOT_DELETE. so in general impossible always delete file.
in case exist another open handles for file (but not section !) begin from windows 10 rs1 exist new functional for delete - FileDispositionInformationEx with FILE_DISPOSITION_POSIX_SEMANTICS. in this case:
Normally a file marked for deletion is not actually deleted until all
open handles for the file have been closed and the link count for the
file is zero. When marking a file for deletion using
FILE_DISPOSITION_POSIX_SEMANTICS, the link gets removed from the visible namespace as soon as the POSIX delete handle has been closed,
but the file’s data streams remain accessible by other existing
handles until the last handle has been closed.
ULONG DeletePosix(PCWSTR lpFileName)
{
HANDLE hFile = CreateFileW(lpFileName, DELETE, FILE_SHARE_VALID_FLAGS, 0, OPEN_EXISTING,
FILE_FLAG_BACKUP_SEMANTICS|FILE_FLAG_OPEN_REPARSE_POINT, 0);
if (hFile == INVALID_HANDLE_VALUE)
{
return GetLastError();
}
static FILE_DISPOSITION_INFO_EX fdi = { FILE_DISPOSITION_DELETE| FILE_DISPOSITION_POSIX_SEMANTICS };
ULONG dwError = SetFileInformationByHandle(hFile, FileDispositionInfoEx, &fdi, sizeof(fdi))
? NOERROR : GetLastError();
// win10 rs1: file removed from parent folder here
CloseHandle(hFile);
return dwError;
}
Update
Sorry i didn't get the question correctly the first time. I thought DeleteFile returned error 13.
Now I understand that DeleteFile succeeds but rename fails immediatlely after.
It could be just a sync issue with the filesystem. After calling DeleteFile the file will be deleted when the OS commits the changes to the filesystem. That may not appen immediately.
If you need to perform multiple operations to the same path, you should have a look at transactions https://learn.microsoft.com/it-it/windows/desktop/api/winbase/nf-winbase-deletefiletransacteda.
-- OLD ANSWER --
That is correct. If the another application handles to that file, DeleteFile will fail.
Citing MSDN docs https://learn.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-deletefile :
The DeleteFile function fails if an application attempts to delete a file that has other handles open for normal I/O or as a memory-mapped file (FILE_SHARE_DELETE must have been specified when other handles were opened).
This applies to dropbox, the antivirus, or in general, any other application that may open those files.
Dropbox may open the file to compute its hash (to look for changes) at any moment. Same goes with the antivirus.
Using Windows API's WriteFileGather, I am writing a file to the disk.
I want to append new buffers to the existing file.
What is the way to prevent WriteFileGather from overwriting the existing file?
WriteFileGather will never overwrite the file unless you ask it to - theres no implied overwrite/append option, theres ONLY a 'please write data at file position X option'.
You should open the file handle normally (making sure you've got GENERIC_WRITE access and specifying flags at least flags FILE_FLAG_OVERLAPPED and FILE_FLAG_NO_BUFFERING by using CreateFile
Then you set the position the file writes at by using the Offset and OffsetHigh members of the OVERLAPPED you pass in as the 5th parameter.
This is similar to the way WriteFile works when its running in asynchronous mode - you must specify the position to write at. Its probably easier to learn how to do positional asyncronous writes using WriteFile first then move onto WriteFileGather if you need its additional power.
See here for docs.
EDIT: To answer the comment from Harry, to get the end of file you can either remember how much you've written before (assuming this is a new file you created) or get the current file size from a HANDLE using SetFilePointerEx with distance 0 and method FILE_END which will return to you the end of the file. There are other ways of getting a file size but beware you may get a cached answer (e.g. if iterating over a directory) and so the above is recommended.
I shouldnt be able to delete a file with an open handle, correct? So i create a file, then i straight away try to delete it, expecting this to fail. Or am i wrong and the handle doesnt have to be closed before deleting the file?
HANDLE hFile = CreateFile (TEXT(file),
GENERIC_WRITE,
0,
NULL,
OPEN_ALWAYS,
FILE_ATTRIBUTE_NORMAL,
NULL);
if (hFile == INVALID_HANDLE_VALUE)
{
//FAIL
}
if(DeleteFile(file))
{
//Should it ever get here cos i dont close the handle?
}
It depends on how the file has been opened. If the share mode has FILE_SHARE_DELETE specified, then it may be deleted by others.
Even if you memory map the file, and it has been opened with this flag (and read/write sharing), then it can still be deleted by the shell (at least I've tried this and it happens, but perhaps the file has simply been renamed and moved to the recycle bin). In such cases, subsequently accessing the memory will result in an 'InPageError' C-style exception.
Yes, it would fail.
The DeleteFile function fails if an application attempts to delete a
file that is open for normal I/O or as a memory-mapped file.
Have you tried this? MS documentation states that:
The DeleteFile function fails if an application attempts to delete a file that is open for normal I/O or as a memory-mapped file.
So if you're not getting that behaviour I'd suggest it's down to the way you've opened the file. Are you sure that your check on whether the file is open is completely comprehensive?Have you tried writing to the file first? Can you see the file outside of your own code? (i.e. from Explorer) Look here for more details.
Imagine you have the following in C++:
ofstream myfile;
myfile.open (argv[1]);
if (myfile.is_open()){
for(int n=0;n<=10;n++){
myfile << "index="<<n<<endl;
sleep(1);
}
}else{
cerr << "Unable to open file";
}
myfile.close();
And while writing, the disk or medium you are writing to becomes unavailable but comes back on for the close() so that you have missing data in between. Or imagine you write to a USB flash drive and the device is withdrawn and re-inserted during the writing process.
How can you detect that ? I tried checking putting the write in try {} catch, flags(), rdstate(), you name it, but none thus far seem to work.
I don't think that is something you can detect at the stdio level. Typically when a hard drive temporarily stops responding, the operating system will automatically retry the commands either until they succeed or a timeout is reached, at which point your system call may receive an error. (OTOH it may not, because your call may have returned already, after the data was written into the in-memory filesystem cache but before any commands were sent to the actual disk)
If you really want to detect flakey hard drive, you'll probably need to code to a much lower level, e.g. write your own hardware driver.
IMHO you can try to:
Use ios:exceptions
Use low-level OS interactions
Verify that IO was successful (if 1 and 2 doesn't work)
I'm not sure if this will cover your scenario (removing a USB drive mid-write), but you can try enabling exceptions on the stream:
myfile.exceptions(ios::failbit | ios::badbit);
In my experience, iostreams do a "great" job of making it hard to detect errors and the type of error.
for(int n=0;n<=10;n++){
if (!(myfile << "index="<<n<<endl))
throw std::runtime_error("WRITE FAILED")
sleep(1);
}
If the std::ostream fails for any reason, it sets it's state bit, which is checked then the std::stream is in a boolean context. This is the same way you check if an std::istream read in data to a variable correctly.
However, this is the same as rdstate(), which you say you tried. If that's the case, the write has gotten to a buffer. endl, which flushes the programs buffer, shows that it's in the Operating System's buffer. From there, you'll have to use OS-specific calls to force it to flush the buffer.
[Edit] According to http://msdn.microsoft.com/en-us/library/17618685(v=VS.100).aspx, you can force a flush with _commit if you have a file descriptor. I can't find such a guarantee for std::ostreams.
In Windows, is there an easy way to tell if a folder has a subfile that has changed?
I verified, and the last modified date on the folder does not get updated when a subfile changes.
Is there a registry entry I can set that will modify this behavior?
If it matters, I am using an NTFS volume.
I would ultimately like to have this ability from a C++ program.
Scanning an entire directory recursively will not work for me because the folder is much too large.
Update: I really need a way to do this without a process running while the change occurs. So installing a file system watcher is not optimal for me.
Update2: The archive bit will also not work because it has the same problem as the last modification date. The file's archive bit will be set, but the folders will not.
This article should help. Basically, you create one or more notification object such as:
HANDLE dwChangeHandles[2];
dwChangeHandles[0] = FindFirstChangeNotification(
lpDir, // directory to watch
FALSE, // do not watch subtree
FILE_NOTIFY_CHANGE_FILE_NAME); // watch file name changes
if (dwChangeHandles[0] == INVALID_HANDLE_VALUE)
{
printf("\n ERROR: FindFirstChangeNotification function failed.\n");
ExitProcess(GetLastError());
}
// Watch the subtree for directory creation and deletion.
dwChangeHandles[1] = FindFirstChangeNotification(
lpDrive, // directory to watch
TRUE, // watch the subtree
FILE_NOTIFY_CHANGE_DIR_NAME); // watch dir name changes
if (dwChangeHandles[1] == INVALID_HANDLE_VALUE)
{
printf("\n ERROR: FindFirstChangeNotification function failed.\n");
ExitProcess(GetLastError());
}
and then you wait for a notification:
while (TRUE)
{
// Wait for notification.
printf("\nWaiting for notification...\n");
DWORD dwWaitStatus = WaitForMultipleObjects(2, dwChangeHandles,
FALSE, INFINITE);
switch (dwWaitStatus)
{
case WAIT_OBJECT_0:
// A file was created, renamed, or deleted in the directory.
// Restart the notification.
if ( FindNextChangeNotification(dwChangeHandles[0]) == FALSE )
{
printf("\n ERROR: FindNextChangeNotification function failed.\n");
ExitProcess(GetLastError());
}
break;
case WAIT_OBJECT_0 + 1:
// Restart the notification.
if (FindNextChangeNotification(dwChangeHandles[1]) == FALSE )
{
printf("\n ERROR: FindNextChangeNotification function failed.\n");
ExitProcess(GetLastError());
}
break;
case WAIT_TIMEOUT:
// A time-out occurred. This would happen if some value other
// than INFINITE is used in the Wait call and no changes occur.
// In a single-threaded environment, you might not want an
// INFINITE wait.
printf("\nNo changes in the time-out period.\n");
break;
default:
printf("\n ERROR: Unhandled dwWaitStatus.\n");
ExitProcess(GetLastError());
break;
}
}
}
This is perhaps overkill, but the IFS kit from MS or the FDDK from OSR might be an alternative. Create your own filesystem filter driver with simple monitoring of all changes to the filesystem.
ReadDirectoryChangesW
Some excellent sample code in this CodeProject article
If you can't run a process when the change occurs, then there's not much you can do except scan the filesystem, and check the modification date/time. This requires you to store each file's last date/time, though, and compare.
You can speed this up by using the archive bit (though it may mess up your backup software, so proceed carefully).
An archive bit is a file attribute
present in many computer file systems,
notably FAT, FAT32, and NTFS. The
purpose of an archive bit is to track
incremental changes to files for the
purpose of backup, also called
archiving.
As the archive bit is a binary bit, it
is either 1 or 0, or in this case more
frequently called set (1) and clear
(0). The operating system sets the
archive bit any time a file is
created, moved, renamed, or otherwise
modified in any way. The archive bit
therefore represents one of two
states: "changed" and "not changed"
since the last backup.
Archive bits are not affected by
simply reading a file. When a file is
copied, the original file's archive
bit is unaffected, however the copy's
archive bit will be set at the time
the copy is made.
So the process would be:
Clear the archive bit on all the files
Let the file system change over time
Scan all the files - any with the archive bit set have changed
This will eliminate the need for your program to keep state, and since you're only going over the directory entries (where the bit is stored) and they are clustered, it should be very, very fast.
If you can run a process during the changes, however, then you'll want to look at the FileSystemWatcher class. Here's an example of how you might use it.
It also exists in .NET (for future searchers of this type of problem)
Perhaps you can leave a process running on the machine watching for changes and creating a file for you to read later.
-Adam
Perhaps you can use the NTFS 5 Change Journal with DeviceIoControl as explained here
If you are not opposed to using .NET the FileSystemWatcher class will handle this for you fairly easily.
From the double post someone mentioned: WMI Event Sink
Still looking for a better answer though.
Nothing easy - if you have a running app you can use the Win32 file change notification apis (FindFirstChangeNotification) as suggested with the other answers. warning: circa 2000 trend micro real-time virus scanner would group the changes together making it necessary to use really large buffers when requesting the file system change lists.
If you don't have a running app, you can turn on ntfs journaling and scan the journal for changes http://msdn.microsoft.com/en-us/library/aa363798(VS.85).aspx but this can be slower than scanning the whole directory when the # of changes is larger than the # of files.