Is it possible to implement transactional file saving in standard C++ without using system specific API calls like CopyFile, ReplaceFile, CreateFile etc.
Currently I am doing it as follows (using Win32 API specific code):
void TransactionalFileSave( const std::wstring& szFileName )
{
std::wofstream fileTemp( L"temp" );
for( auto data : vecData )
{
fileTemp << data;
}
fileTemp.close();
CloseHandle( CreateFile( szFileName.c_str(), (GENERIC_READ | GENERIC_WRITE), (FILE_SHARE_READ | FILE_SHARE_WRITE), NULL, CREATE_NEW, FILE_ATTRIBUTE_NORMAL, NULL ) );
CloseHandle( CreateFile( L"temp_backup", GENERIC_ALL, (FILE_SHARE_READ | FILE_SHARE_WRITE), NULL, CREATE_NEW, FILE_ATTRIBUTE_HIDDEN | FILE_ATTRIBUTE_TEMPORARY, NULL ) );
ReplaceFile( szFileName.c_str(), L"temp", L"temp_backup", REPLACEFILE_WRITE_THROUGH, NULL, NULL );
}
Is there a way I could achieve a similar thing without using Win32 API specific functions (i.e. just the standard library or boost)?
Short answer is: no.
Long answer: The standard file-functions support only a fairly basic set of functions. If you need to specific functionality like this, that is only supported by some operating systems, then you will need to use system-specific functionality - or find another way to achieve the same result (for example, using rename instead of ReplaceFile)
Related
I'm using:
DWORD d = GetLogicalDrives();
for (int i = 0; i < 26; i++)
{
if ((1 << i) & d) // drive letter 'A' + i present on computer
{
wstring s = std::wstring(L"\\\\.\\") + wchar_t('A' + i) + L":";
PARTITION_INFORMATION diskInfo;
DWORD dwResult;
HANDLE dev = CreateFile(LPWSTR(s.c_str()), GENERIC_READ, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_READONLY, NULL);
DeviceIoControl(dev, IOCTL_DISK_GET_PARTITION_INFO, NULL, 0, &diskInfo, sizeof(diskInfo), &dwResult, NULL);
CloseHandle(dev);
if (diskInfo.PartitionType == PARTITION_IFS)
{
...
}
}
}
to enumerate all NTFS partitions of a computer.
It works on my Windows 7, on a Windows 8.1 I tried it on, and on a Windows 10 computer.
But it fails on another Windows 10 computer: on this one, the volume C: has a diskInfo.PartitionType value equal to 0x00, instead of 0x07 (PARTITION_IFS).
This value is (see the doc here):
PARTITION_ENTRY_UNUSED : 0x00 : An unused entry partition.
This is strange, since, I can confirm, the partition is really NTFS.
Questions:
Is it well-known that IOCTL_DISK_GET_PARTITION_INFO is not 100% reliable to get the partition type?
What would be a more reliable way to enumerate all NTFS volumes?
Note: I also looked at using IOCTL_DISK_GET_PARTITION_INFO_EX instead of IOCTL_DISK_GET_PARTITION_INFO but then the structure PARTITION_INFORMATION_EX does not seem to give informations about PartitionType, whereas the structure PARTITION_INFORMATION does give access to PartitionType.
As #RemyLebeau says, you are not checking the return value for each call.
PARTITION_ENTRY_UNUSED often means the DeviceIoControl() call failed. It depends on the permissions of your user. You should check your user's access rights to see if it has the FILE_READ_DATA permission (included in GENERIC_READ) on volume C:. In my test environment, if you have no access to open volume C: with GENERIC_READ, CreateFile() returns INVALID_HANDLE_VALUE, and then DeviceIoControl() fails as well.
EDIT:
I suggest using GetVolumeInformation(), for example:
wchar_t fs[MAX_PATH + 1] = { 0 };
GetVolumeInformationW(L"C:\\", NULL, 0, NULL, NULL, NULL, fs, MAX_PATH + 1);
And you will see the Type info in the fs buffer.
I did further investigation thanks to #RemyLebeau's comments with:
HANDLE dev = CreateFile(..., GENERIC_READ, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_READONLY, NULL);
if (dev == INVALID_HANDLE_VALUE)
{
DWORD err = GetLastError(); // then MessageBox
}
else
{
BOOL ret = DeviceIoControl(dev, IOCTL_DISK_GET_PARTITION_INFO, NULL, 0, &diskInfo, sizeof(diskInfo), &dwResult, NULL);
if (ret == FALSE)
{
DWORD err = GetLastError(); // then MessageBox
}
CloseHandle(dev);
}
on the computer where it failed (computer with Windows 10). I found that CreateFile succeeded but then DeviceIoControl failed with GetLastError being 1 i.e. ERROR_INVALID_FUNCTION (see System Error Codes (0-499)).
Conclusion (I quote Remy's comment):
That means IOCTL_DISK_GET_PARTITION_INFO is not supported by the device you passed to DeviceIoControl().
I then tried with IOCTL_DISK_GET_PARTITION_INFO_EX:
PARTITION_INFORMATION_EX diskInfo;
BOOL ret = DeviceIoControl(dev, IOCTL_DISK_GET_PARTITION_INFO_EX, NULL, 0, &diskInfo, sizeof(diskInfo), &lpBytesReturned, NULL);
and then it worked. I could see that diskInfo.PartitionStyle was PARTITION_STYLE_GPT (=1), and this was the reason why IOCTL_DISK_GET_PARTITION_INFO failed. I quote Remy's comment again:
IOCTL_DISK_GET_PARTITION_INFO is not supported on GPT partitioned drives.
So here's the conclusion:
use IOCTL_DISK_GET_PARTITION_INFO_EX instead of IOCTL_DISK_GET_PARTITION_INFO
if diskInfo.PartitionStyle is 0 (PARTITION_STYLE_MBR) then diskInfo.Mbr.PartitionType can be tested. If it's 0x07, it's NTFS.
if diskInfo.PartitionStyle is 1 (PARTITION_STYLE_GPT) then diskInfo.Gpt.PartitionType can be tested, see here: https://learn.microsoft.com/en-us/windows/desktop/api/winioctl/ns-winioctl-_partition_information_gpt. Even if the NTFS Wikipedia page mentions the GUID EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 for NTFS in the GPT case, this GUID is in fact unrelated to file system (see comment below about this).
it's probably easier to use GetVolumeInformation() instead and just compare if the result is the "NTFS" string, as in the other answer
in my particular case, I initially wanted to test if a volume is NTFS or not before attempting an indexing with DeviceIoControl(hVol, FSCTL_ENUM_USN_DATA, ...) because I thought such MFT querying would be limited to NTFS volumes. In fact, an easier solution would be to NOT TEST if it's NTFS or not, and just do the FSCTL_ENUM_USN_DATA. The worst that can happen is that FSCTL_ENUM_USN_DATA fails with ERROR_INVALID_FUNCTION error, per the documentation:
"ERROR_INVALID_FUNCTION The file system on the specified volume does not support this control code."
I've been trying to call a sample driver. I have written DriverEntry method, where I initialize both the driver name and symbolic ling pointing to the driver.
// UNICODE_STRING DriverName, SymbolName; // Driver registry paths
...
// Driver Entrypoint
NTSTATUS
DriverEntry(PDRIVER_OBJECT pDriverObject, PUNICODE_STRING pRegistryPath) {
Q_UNUSED(pRegistryPath);
DbgPrintEx(0, 0, "Driver Loaded\n");
// The PsSetLoadImageNotifyRoutine routine registers a driver-supplied
// callback that is subsequently notified whenever
// an image is loaded (or mapped into memory).
PsSetLoadImageNotifyRoutine(ImageLoadCallback);
// initialize driver name
RtlInitUnicodeString(&DriverName, L"\\Device\\Explorer");
// initialize symbolic link
RtlInitUnicodeString(&SymbolName, L"\\DosDevices\\Explorer");
IoCreateDevice(pDriverObject, 0, &SymbolName, FILE_DEVICE_UNKNOWN,
FILE_DEVICE_SECURE_OPEN, FALSE, &pDeviceObject);
IoCreateSymbolicLink(&DriverName, &SymbolName);
pDriverObject->MajorFunction[IRP_MJ_CREATE] = CreateCall;
pDriverObject->MajorFunction[IRP_MJ_CLOSE] = CloseCall;
pDriverObject->MajorFunction[IRP_MJ_DEVICE_CONTROL] = IoControl;
pDriverObject->DriverUnload = UnloadDriver;
pDeviceObject->Flags |= DO_DIRECT_IO;
pDeviceObject->Flags &= ~DO_DEVICE_INITIALIZING;
return STATUS_SUCCESS;
}
When I load the driver up (using OSR Driver Loader, could be done using cmd also, by registering the driver as a new service), I get expected output in DebugView (sysinternals tool allowing to see kernel debug logs)
Now I needed to make sure that both the device and symlink are present in Windows Object Directories. To do that, I use WinObj (another tool from sysinternals), here is the output
What confuses me here, is that the symbolic link is in Device folder, instead of GLOBAL??.
Symbolic link in Device
Device in GLOBAL??
Now, finally, calling the driver itself. I use c++ for that purpose and this is my code,
class Test
{
public:
HANDLE hDriver; // Handle to driver
// Initializer
Test::Test(LPCSTR RegistryPath)
{
LPCSTR path = "\\\\.\\Explorer";
hDriver = CreateFileA(path, GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_READ | FILE_SHARE_WRITE, 0, OPEN_EXISTING, 0, 0);
if (hDriver == INVALID_HANDLE_VALUE)
{
// Handle the error.
char result = GetLastError();
bool zadek = false;
}
}
The problem is that I can't get a valid handle for the driver. The value of hDriver is always either 0x00000000000000a0 or 0xffffffff, no matter the path I use. I'm using createFileA because I want to access system memory.
Is there some blatant mistake I made?
I should say it is over 8-9 year since last time I written a device driver, but what comes off the top of my head are:
You say you get 0xa0 for hDriver which is a valid handle value.
Right now, you can only use device IO control, because you only have callback for IRP_MJ_DEVICE_CONTROL.
Try L"\\??\\Explorer" or L"\\GLOBAL??\\Explorer" for symbolic link.
You need to use DriverName for IoCreateDevice.
You are passing incorrect arguments to IoCreateSymbolicLink.
So your code should become like this:
...
// initialize driver name
RtlInitUnicodeString(&DriverName, L"\\Device\\Explorer");
// initialize symbolic link
RtlInitUnicodeString(&SymbolName, L"\\??\\Explorer");
IoCreateDevice(pDriverObject, 0, &DriverName, FILE_DEVICE_UNKNOWN,
FILE_DEVICE_SECURE_OPEN, FALSE, &pDeviceObject);
IoCreateSymbolicLink(&SymbolName, &DriverName);
...
I am working in Windows,using vc++2010 and MFC.
Following is my code:
CFile File;
TCHAR lpCause[1024];
CFileException eException;
CString strErrorMessage;
// a very long file path name means a file name over 255 characters
if (!File.Open(_T("a very long file path name"), CFile::modeCreate, &eException))
{
eException.GetErrorMessage(lpCause, 1024);
strErrorMessage = lpCause;
}
else
File.Close();
When I run the code, I got error message:"a very long file path name contains an incorrect path".
My questions are:
How to modify my code to make it work?
I learn that CreateFile() function can add "\\\\?\" in the beginning of file path, then it will extend this limit to 32767 wide characters.How can I do the same thing in MFC?
Cause
In the source of CFile::Open(), there is an explicit check if the path length exceeds _MAX_PATH:
if (lpszFileName != NULL && SUCCEEDED(StringCchLength(lpszFileName, _MAX_PATH, NULL)) )
If _MAX_PATH is exceeded, the function sets pException->m_cause = CFileException::badPath and returns FALSE.
This is true even for the MFC version that comes with VS2017.
So the standard technique to circumvent the _MAX_PATH limit, that is prefixing the path with \\?\ won't work.
Possible Solutions
Call CreateFileW() directly to pass it a path with \\?\ prefix. Use the CFile constructor that accepts a HANDLE to manage the file through a CFile object. The CFile object will take ownership of the handle so you must not call CloseHandle() on the handle.
HANDLE hFile = CreateFileW( L"\\\\?\\a very long file path name", GENERIC_WRITE, 0, nullptr, CREATE_ALWAYS, 0, NULL );
if( hFile != INVALID_HANDLE_VALUE )
{
// Manage the handle using CFile
CFile file( hFile );
// Use the file...
// The CFile destructor closes the handle here.
}
else
{
DWORD err = GetLastError();
// TODO: do your error handling...
}
Another possibility is to derive a class from CFile that overrides CFile::Open() (which is virtual). For the implementation copy/paste the MFC source, but leave out the _MAX_PATH check. For a big project, this class could be a drop-in replacement for CFile to enable long paths. You could even go so far to prepend the \\?\ prefix if it isn't already there (but that is more involved as the prefix also disables the regular transformations from a Win32 path to a NT-style path, like converting / to \, resolving dots and so on).
I have a Visual Studio 2008 C++ application for Windows 7 where I would like to watch a file for changes.
The file may be changed like this:
std::ofstream myfile_;
void LogData( const char* data )
{
myfile_ << data << std::endl;
// note that the file output buffer is flushed by std::endl, but the file is not closed.
}
I have tried watching the file's directory using both ReadDirectoryChangesW and FindFirstChangeNotification with FILE_NOTIFY_CHANGE_SIZE | FILE_NOTIFY_CHANGE_LAST_WRITE | FILE_NOTIFY_CHANGE_LAST_ACCESS | FILE_NOTIFY_CHANGE_SECURITY | FILE_NOTIFY_CHANGE_CREATION | FILE_NOTIFY_CHANGE_FILE_NAME flags. But, neither of those APIs will detect file changes until the file handle is actually closed.
Is there any way to detect a change when the file is actually written, but before the file handle is closed?
Thanks,
PaulH
Update
On #Edwin's suggestion, I'm attempting to use the Journal feature. But, I'm having a couple issues.
FSCTL_READ_USN_JOURNAL returns instantly. It does not block. (though, this may be related to issue 2)
Regardless of where my handle points to (I have tried opening a handle to the directory "C:\Foo\Bar" and to the file "C:\Foo\Bar\MyFile.txt") I seem to get any changes made to the the C: volume. Is there a way to limit what FSCTL_READ_USN_JOURNAL gives me?
Error checking omitted for brevity.
boost::shared_ptr< void > directory(
::CreateFileW( L"C:\\Foo\\Bar\\Myfile.txt",
GENERIC_READ,
FILE_SHARE_READ,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL ),
::CloseHandle );
USN_JOURNAL_DATA journal = { 0 };
DWORD returned = 0;
::DeviceIoControl( directory.get(), FSCTL_QUERY_USN_JOURNAL, NULL, 0, &journal, sizeof( journal ), &returned, NULL );
BYTE buffer[ 4096 ] = { 0 };
READ_USN_JOURNAL_DATA read = { 0, USN_REASON_DATA_EXTEND | USN_REASON_DATA_TRUNCATION, FALSE, 0, 0, journal.UsnJournalID };
::DeviceIoControl( directory.get(), FSCTL_READ_USN_JOURNAL, &read, sizeof( read ), &buffer, sizeof( buffer ), &returned, NULL );
for( USN_RECORD* record = ( USN_RECORD* )( buffer + sizeof( USN ) );
( ( BYTE* )record - buffer ) < returned;
record = ( USN_RECORD* )( ( BYTE* )record + record->RecordLength ) )
{
ATLTRACE( L"%s\r\n", record->FileName );
}
Example output (none of these are in the C:\Foo\Bar directory):
AeXProcessList.txt`
AeXProcessList.txt`
AeXAMInventory.txt`
AeXAMInventory.txt`
AeXProcessList.txt`
AeXProcessList.txtP
access.log`
mysqlgeneral.log
E804.tmp
apache_error.log
E804.tmp
CHROME.EXE-5FE9909D.pfh
CHROME.EXE-5FE9909D.pfp
SyncData.sqlite3-journal
CHROME.EXE-5FE9909D.pfh
CHROME.EXE-5FE9909D.pfP
1211.tmp
SyncData.sqlite3-journal
AeXAMInventory.txt
You can use
Change Journal Operations
(see MSDN docs)
That's the only 100% garanteed way to detect any change in the filesystem.
But it's pretty complicated.
To read data for a specific file or directory, I believe you want to use FSCTL_READ_FILE_USN_DATA instead of FSCTL_READ_USN_JOURNAL. I believe the latter always retrieves data for an entire volume. That does not, however, fill in the TimeStamp, Reason, or SourceInfo fields of the USN record you get. If you need those, I believe you can read them with FSCTL_READ_USN_JOURNAL, specifying the exact USN you want to read.
No, because until you close the file handle there is no guarantee a single byte ever gets written by the OS.
The exception would probably be by calling flush on your file handle and then call the Windows API function FlushFileBuffers, but unless the program writing into the file does this no bytes probably get written.
This can be done with a filter driver that monitors the FASTIO_WRITE and IRP_MJ_WRITE operations. Here is a pretty good how-to article.
How do you set MOTW (Mark of the Web) on an executable that is downloaded from the internet?
This data is stored in an NTFS alternative file stream alongside an executable. The stream is called Zone.Identifier:
Windows® Internet Explorer® uses the stream name Zone.Identifier for storage of URL security zones.
The fully qualified form is sample.txt:Zone.Identifier:$DATA
The stream is a simple text stream of the form:
[ZoneTransfer]
ZoneId=3
MSDN-SECZONES gives an explanation of security zones.
(N.B. The original has a space between the colon and "Zone" but I think this is erroneous.)
You can find the ZoneIds in UrlMon.h in the SDK; there's an enum which equates to
enum URLZONE {
URLZONE_LOCAL_MACHINE = 0,
URLZONE_INTRANET = 1,
URLZONE_TRUSTED = 2,
URLZONE_INTERNET = 3,
URLZONE_RESTRICTED = 4
};
(The original uses previous value + 1 rather than absolute values.)
As Hans says in the comments, these can be written with the standard Win32 file APIs CreateFile and WriteFile.
Firefox always writes Internet Zone, zone 3 - Firefox code here (MPL/LGPL/GPL tri-license):
bool SetInternetZoneIdentifier(const FilePath& full_path) {
const DWORD kShare = FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE;
std::wstring path = full_path.value() + L":Zone.Identifier";
HANDLE file = CreateFile(path.c_str(), GENERIC_WRITE, kShare, NULL,
OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
if (INVALID_HANDLE_VALUE == file)
return false;
const char kIdentifier[] = "[ZoneTransfer]\nZoneId=3";
DWORD written = 0;
BOOL result = WriteFile(file, kIdentifier, arraysize(kIdentifier), &written,
NULL);
CloseHandle(file);
if (!result || written != arraysize(kIdentifier)) {
DCHECK(FALSE);
return false;
}
return true;
}
Alternatively there's an IE COM API CLSID_PersistentZoneIdentifier you can use to abstract this all for you.
It is not explicitly stated in RFC 3514, but today, due to increased security requirements, implementations should really retain the information of the presence or absence of the RFC3514 bit in a network transmission, when they write files out to disk, and vice-versa for reading from disk.