CreateFile Win32 API Call with OPEN_ALWAYS failed in an Odd Way - c++

We had a line of code
if( !CreateFile( m_hFile, szFile, GENERIC_READ|GENERIC_WRITE, 0, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL ) )
{
DWORD dwErr = GetLastError();
CString czInfo;
czInfo.Format ("CMemoryMapFile::OpenAppend SetFilePointer call failed - GetLastError returned %d", dwErr);
LOG(czInfo);
return false;
}
This code worked great for many years. A few weeks ago, we had a customer with a problem. Turns out, the problem could be traced to this line of code, where the function would return a INVALID_HANDLE_VALUE handle and GetLastError() returned ERROR_FILE_NOT_FOUND(2).
Now, this is very confusing to us. OPEN_ALWAYS should direct the file to be created if it does not exist. So, why are we getting a ERROR_FILE_NOT_FOUND?
More confusion: For this customer, this only happened on one network share point (we were using a UNC path). Other UNC paths to other machines for this customer worked. Local paths worked. All our other customers (10000+ installs) have no problem at all.
The customer was using XP as the client OS, and the servers were running what appeared to be standard Windows Server 2003 (I think the Small Business Server version). We could not replicate their errors in our test lab using the same OS's. They could repeat the problem with several XP clients, but the problem was only on one server (other Server 2003 servers did not exhibit the problem).
We fixed the problem by nesting two CreateFile calls, the first with OPEN_EXISTING, and the second with CREATE_ALWAYS if OPEN_EXISTING failed. So, we have no immediate need for a fix.
My question: Does anyone have any idea why this API call would fail in this particular way? We are puzzled.
Addendum:
The CreateFile function above is a wrapper on the Windows API function. Here's the code:
bool CMemoryMapFile::CreateFile( HANDLE & hFile, LPCSTR szFile, DWORD dwDesiredAccess, DWORD dwShareMode, DWORD dwCreationDisposition, DWORD dwFlagsAndAttributes )
{
hFile = ::CreateFile (szFile, dwDesiredAccess, dwShareMode, NULL,
dwCreationDisposition, dwFlagsAndAttributes, NULL);
return (hFile != INVALID_HANDLE_VALUE)
}

First, you should always check for success of the CreateFile() API like this:
if (CreateFile(...) == INVALID_HANDLE_VALUE)
{
// handle the error
}
because CreateFile() doesn't return !=0 in case of success, but anything other than INVALID_HANDLE_VALUE (which is -1).
Then, the CreateFile() can fail in the situation you described if either the directory doesn't exist where you want to create/open the file in, or it can fail if the user has the rights to open and write files in that directory, but no rights to create new files.

Maybe the directory you wanted to create the file in did not exist?
Are you sure you fixed it by using 2 CreateFile calls? Or did you just not reproduce it?

We have also seen this problem when a file is accessed by a UNC share. None of the ideas and suggestions applied in our case - the directory always exists, the parameters to CreateFile are correct, we are checking the return correctly.
We believe this to be a shares issue. We solved the problem with a simple retry loop:
If a CreateFile with OPEN_ALWAYS fails with ERROR_FILE_NOT_FOUND, we simply sleep for a few ms and try again. It always works second time (if it failed first time).

If CreateFile fails, it will return INVALID_HANDLE_VALUE which is non-zero (I think it's negative 1). So CreateFile might be succeeding and returning zero as the handle.
It's only safe to check GetLastError() after a function fails but it looks like you might be checking the last error when CreateFile has succeeded (returned zero).
According to this article, some functions set the "last error" if they succeed:

My guesses would be something server related like the server not implementing support correctly for that filesystem operation. Maybe a linux server compared to a windows server?
Your arguments to create file are incorrect, so I assume that this is some sort of CreateFile helper function.
A call to CreateFile should look like:
m_hFile = CreateFile( szFile, GENERIC_READ|GENERIC_WRITE, 0, 0, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0 );
if(m_hFile == INVALID_HANDLE_VALUE)

Related

CoImpersonateClient causes CreateFileW to fail

I am working in a client/server scenario. I have a COM-based server which is running on the same machine as the client (the server is running as admin).
Anyways, there is a specific method in the server application which opens a file with CreateFileW. This is some pseudocode that makes up the call.
HRESULT __fastcall CxChmCallback::ParseQueryFile(CxChmCallback* this, wchar_t const* lpFileName, ...)
{
if (FAILED(CoImpersonateClient())
return E_FAIL;
HANDLE hFile = CreateFileW(lpFileName, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL, NULL);
CoRevertToSelf();
if (hFile == INVALID_HANDLE_VALUE)
return (HRESULT)-1;
...
}
Now, I know CreateFileW works because if I skip the call to CoImpersonateClient(), the function succeeds. Otherwise, the function fails and GetLastError() returns ERROR_BAD_IMPERSONATION_LEVEL (1364).
The client is running as a standard user with a medium integrity level, under a user account in the Administrators group. I have tried testing this with the client running as admin, but I get the same problem. Is there anything special about CreateFileW which causes it to fail?

Windows 8: CreateFile() returns INVALID_HANDLE_VALUE, immediate GetLastError shows 0

I am opening a file with CreateFile on Windows 8 from within a printer filter environment. The code is effectively straight C, even though the filter is built in C++. CreateFile returns INVALID_HANDLE_VALUE but an immediate call to GetLastError returns 0. I have seen this before, back in the old NT4 days (and through to Windows 7) if a directory of the same name as the file existed, a file open attempt would fail with error 0; but I have checked and the file name is different from any subdirectories in the destination directory.
Code:
io_buf[fh].fh = CreateFile(name, GENERIC_READ|GENERIC_WRITE, 0, p_sa,
CREATE_ALWAYS, 0, (HANDLE)0);
io_buf[fh].read_mode = FALSE;
io_buf[fh].file_start = 0L;
io_buf[fh].folder = 0;
if (io_buf[fh].fh == INVALID_HANDLE_VALUE)
{
LogMsg("BufCreate: CreateFile failed (%ld); retrying with SharedWrite\r\n", GetLastError());
io_buf[fh].fh = CreateFile(name, GENERIC_READ|GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_WRITE, p_sa,
CREATE_ALWAYS, 0, (HANDLE) 0);
}
if (io_buf[fh].fh == INVALID_HANDLE_VALUE)
{
LogMsg("BufCreate: CreateFile failed (%ld)\r\n", GetLastError());
return (Vers_FAILURE);
}
LogMsg output:
Render 15:41:46.715: BufCreate: CreateFile failed (0); retrying with SharedWrite
Render 15:41:46.715: BufCreate: CreateFile failed (0)
Because this occurs relatively randomly, sending it to MS would not work; they'd fire it back to me with a "no rep" and I'd get dinged for the SRX. Has anyone got any clue on how I would proceed?
This is vexing. Turns out that I was not writing the file where I thought I was, and that's why it failed. But it should have returned error 5 (Access denied), and I was led up a stump because of this bedamned 0 in GetLastError.
For what it's worth: I had created a path to a temp directory. Because my security context at the time was LocalSystem (I'm in the PrinterPipeline service), that file was in c:\Windows\System32\Services\LocalService\AppData\Local\Temp. No ordinary user can reach that space, and the attempt will quite possibly leave your OS hooped. I had changed to user security context to write to C:\Temp, and was getting a failure because I had forgotten to change my filename to C:\Temp. The error 0 made it look like possibly I was faced with file system redirection mapping c:\temp into LocalService space.
Correcting that so that I was actually writing where I thought I was got rid of the failure... but I still don't understand why Windows was handing me back an error 0.

INVALID_HANDLE_VALUE when calling CreateFileA several times

I'm using CreateFileA and the first time I call it, it works as expected. But when i call it the second time, it returns handle INVALID_HANDLE_VALUE. What could be the problem? Just for information, I'm calling it every time I need to check if my USB device is connected..
int port = 500;
char port_name [MAX_CAR] = {0};
sprintf_s (port_name, MAX_CAR, "\\\\.\\COM%d", port);
com->id = CreateFileA (port_name,
GENERIC_READ | GENERIC_WRITE,
0,
NULL,
OPEN_EXISTING,
FILE_FLAG_OVERLAPPED,
NULL);
EDIT: I did try to use CloseHandle like CloseHandle(com->id); but it doesn't help.
From the documentation:
When an application is finished using the object handle returned by
CreateFile, use the CloseHandle function to close the handle. This not
only frees up system resources, but can have wider influence on things
like sharing the file or device and committing data to disk. Specifics
are noted within this topic as appropriate.
Use GetLastError to get the error code, and use the FormatMessage to get a human readable error description, or just simply Google the error code.
There are many reasons can cause the same error (CreateFile returns INVALID_HANDLE_VALUE), without the GetLastError, you will very hard to find out what is the real reason.

ShellExecuteW not working well on Windows 8.1?

I call standard ShellExecuteW call on Windows8.1 to open PPS (powerpoint slide) file.
This works just fine on Windows 7. On Windows 8.1. it reports "There is no program associated to open the file". Of course, the file association is set and if file is saved and run from Explorer (double clicked) it opens just fine. I also tried to change association and to associate another program and then associate back to PPS viewer, no improvement. It just doesn't work for W8.1 but the same call works on earlier Windows.
Can anyone give me a clue what might be wrong here?
The code used to open file is very simple and I see no errors with it:
HINSTANCE hinst = ShellExecuteW(NULL, L"open", L"C:\\path\\to\\file.pps", NULL, NULL, SW_SHOWNORMAL);
// Check if result is error
if ((int)hinst <= 32)
{
wchar_t buf[512] = { 0 };
FormatMessageW(FORMAT_MESSAGE_FROM_SYSTEM, NULL, GetLastError(), MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), buf, 512, NULL);
MSGBOX(buf);
}
I use free PPS viewer as found here:
http://www.microsoft.com/en-us/download/details.aspx?id=13
I found something similar which points to that this could be a bug in Win8.1. Can anyone confirm this? Or reveal a workaround?
I found the solution myself.
The problem on W8.1 was that the verb open was not registered to the application so it used different default verb. So if the ShellExecute call is replaced with:
HINSTANCE hinst = ShellExecuteW(NULL, NULL, L"C:\\path\\to\\file.pps", NULL, NULL, SW_SHOWNORMAL);
Then the system looks for a default verb which may or may not be open (usually is), so by not using this verb explicitly it leaves this decision to the system.

CreateFile() Failed With GetLastError() = 5

I have written a sample application to read the file from the other file. When I run this application form virtual machine I am getting Access denied. Below is the code.
int _tmain(int argc, _TCHAR* argv[])
{
WCHAR *wcsPath = L"\\\\150.160.130.22\\share\\123.XML";
HANDLE hFile = CreateFileW(wcsPath,
GENERIC_READ,
FILE_SHARE_READ,
NULL,
OPEN_EXISTING,
0,
0);
if (NULL == hFile)
{
printf("failed - %d", GetLastError());
}
return 0;
}
Please let me know any changes.
Error code 5 stands for "Access is Denied". You should check your user's access rights.
I believe the documentation for CreateFile holds the answer.
It may be that your dwShareMode is causing the problem. Using FILE_SHARE_READ there says, "allow other openers to open the file for READ access". If you do not specify FILE_SHARE_WRITE` , then other openers will not be able to open the file for writing - your call would prevent that.
But, CreateFile, I believe, also fails when the sharemode would be violated by prior openers. If this is true, then if another application already has the file open for write access, then your call to CreateFile will fail, if you specify dwShareMode = FILE_SHARE_READ. Do you see? You may need to specify FILE_SHARE_WRITE | FILE_SHARE_READ for that dwShareMode parameter.
Try it.
The error output of CreateFileW() is INVALID_HANDLE_VALUE, not NULL. Now, NULL definitely sounds like a wrong value for a file handle too, but still.
Is the pasted code snippet exactly the content of your program, or a retelling?
EDIT: I see there's a VM involved. Can you open the file in Notepad from the virtual machine where the program is running and erroring out?