ERROR_IO_PENDING on FTP Asynchronous Operation [Wininet C++] - c++

Question

I've recently been working on an application that needs to establish an FTP connection with a server and download/upload files from it. For performance reasons, I would like to download multiple files at a time. For that reason I've tried to implement asynchronous operation on the Wininet API using the InternetOpen function along with the INTERNET_FLAG_ASYNC flag, as well as the InternetSetStatusCallback function. Here is a sample of my code, where I want to list all the files in the main directory of the remote server recursively:
/*Global variables*/
HANDLE MayContinue=0;
DWORD LatestResult=1;
/*Prototypes*/
void CALLBACK CallbackFunction(HINTERNET,DWORD_PTR,DWORD,LPVOID,DWORD);
//Iteration function called by main()
void FTPIterate()
{
WIN32_FIND_DATAA data;
HINTERNET Internet;
INTERNET_STATUS_CALLBACK call;
HINTERNET h_file;
MayContinue = ::CreateEvent (NULL, FALSE, FALSE, NULL);
iconnect=InternetOpen(NULL,INTERNET_OPEN_TYPE_PROXY,proxy_url,NULL,INTERNET_FLAG_ASYNC);
call=InternetSetStatusCallback(iconnect,(INTERNET_STATUS_CALLBACK)CallbackFunction);
while(f[FLAG_FTP_ITERATE])
{
MayContinue = ::CreateEvent(NULL,FALSE,FALSE,NULL);
InternetConnect(iconnect,ftp_url,INTERNET_DEFAULT_FTP_PORT,ftp_user,ftp_pass,INTERNET_SERVICE_FTP,NULL,LatestResult);
WaitForSingleObject (MayContinue, INFINITE);
server=(HINTERNET)LatestResult;
printf("Server handle: %i\n",(int)server);
printf("Server Error: %i\n",GetLastError());
SetLastError(0);
MayContinue = ::CreateEvent(NULL,FALSE,FALSE,NULL);
FtpFindFirstFile(server,ftp_base,&data,INTERNET_FLAG_NO_CACHE_WRITE,LatestResult);
WaitForSingleObject(MayContinue,INFINITE);
h_file=(HINTERNET)LatestResult;
//do stuff
printf("FindFirstFile handle: %i\n",(int)h_File);
while((MayContinue = ::CreateEvent(NULL,FALSE,FALSE,NULL)) && InternetFindNextFileA(h_file,&data))
{
WaitForSingleObject(MayContinue,INFINITE);
//do stuff
}
printf("FindNextFile Error: %i\n",GetLastError()); //loop is never executed
InternetCloseHandle(server);
}
}
void CALLBACK CallbackFunction(HINTERNET hInternet,DWORD_PTR dwContext,DWORD dwInternetStatus,LPVOID lpvStatusInformation,DWORD dwStatusInformationLength)
{
if (dwInternetStatus == INTERNET_STATUS_REQUEST_COMPLETE)
{
LatestResult = ((LPINTERNET_ASYNC_RESULT)lpvStatusInformation)->dwResult;
SetEvent (MayContinue);
}
}
My code is based on this post from Stack Overflow. When I run it, I first of all get an error after the call to InternetConnect, which is ERROR_IO_PENDING. According to the WinAPI reference this means that there's still some operation being performed. Shouldn't the call to WaitForSingleObject prevent this from happening? (actually, the HINTERNET handle returned by InternetConnect seems to be valid).
When I call the FtpFindFirstFile function it correctly retrieves the first file, but when I use the HINTERNET handle (which, again, seems to be valid) returned by it in the InternetFindNextFile function it fails with error INVALID_HANDLE_VALUE.
EDIT: I egt those errors when using Remy's code:
Connect Handle 00CC0004
Waiting for server handle
Unable to find first file. OS Error: 6 //aren't those calls to FindFirstFile weird if InternetConnect hasn't returned yet?
Waiting for server handle
Unable to find first file. OS Error: 6
Waiting for server handle
Unable to find first file. OS Error: 6
Waiting for server handle
Unable to find first file. OS Error: 6
Waiting for server handle
Unable to find first file. OS Error: 6
Waiting for server handle
Unable to find first file. OS Error: 6
Waiting for server handle.
Unable to connect to Server. Inet Error: 12015 Waiting for server handle
Can someone help me find the mistake?
Thank you in advance.

Answer 1

The ERROR_IO_PENDING error is coming from InternetOpen() itself. Since WaitForSingleObject() is succeeding, it is not overwriting GetLastError() (it only does so on error, as most APIs do), so the error is being carried over from the result of InternetOpen(). This is not the correct way to use GetLastError(). Assume that all APIs overwrite GetLastError() (if documented to use GetLastError() at all), and make sure you call it immediately only if an API fails (unless documented as being used during success conditions).
What your code is doing is NOT asynchronous! You are issuing asynchronous API calls, but you are waiting for their results, which defeats the purpose. Your code is acting synchronously, the same as if you were to omit the INTERNAL_FLAG_ASYNC flag and WaitForSingleObject() calls (not to mention you are leaking event resource by calling CreateEvent() unnecessarily), eg:
void LogError(const char *Op)
{
DWORD err = GetLastError();
if (err == ERROR_INTERNET_EXTENDED_ERROR)
{
LPSTR szBuffer;
DWORD dwLength = 0;
InternetGetLastResponseInfoA(&err, NULL, &dwLength);
if (GetLastError() != INSUFFICIENT_BUFFER)
{
printf("%s. Unknown Inet Error. OS Error: %u", Op, GetLastError());
return;
}
szBuffer = new char[dwLength+1];
InternetGetLastResponseInfoA(&err, szBuffer, &dwLength);
szBuffer[dwLength] = 0;
printf("%s. Inet Error: %u %s", Op, err, szBuffer);
delete[] szBuffer;
}
else
{
LPSTR lpBuffer = NULL;
DWORD dwLen = FormatMessageA(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_ARGUMENT_ARRAY | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, NULL, err, 0, (LPSTR)&lpBuffer, 0, NULL);
if (lpBuffer)
{
printf("%s. OS Error: %u %s", Op, err, lpBuffer);
LocalFree(lpBuffer);
}
else
printf("%s. OS Error: %u", Op, err);
}
printf("\n");
}
void FTPIterate()
{
WIN32_FIND_DATAA data;
HINTERNET hConnect;
HINTERNET hServer;
HINTERNET hFile;
hConnect = InternetOpen(NULL, INTERNET_OPEN_TYPE_PROXY, proxy_url, NULL, 0);
if (hConnect == NULL)
{
LogError("Unable to Open Internet");
return;
}
printf("Connect handle: %p\n", hConnect);
while (f[FLAG_FTP_ITERATE])
{
printf("Connecting to Server\n");
hServer = InternetConnect(hConnect, ftp_url, INTERNET_DEFAULT_FTP_PORT, ftp_user, ftp_pass, INTERNET_SERVICE_FTP, NULL, 0);
if (hServer == NULL)
{
LogError("Unable to connect to Server");
continue;
}
printf("Connected to Server. Server handle: %p\n", hServer);
printf("Finding first file\n");
hFile = FtpFindFirstFileA(hServer, ftp_base, &data, INTERNET_FLAG_NO_CACHE_WRITE, 0);
if (hFile == NULL)
{
if (GetLastError() == ERROR_NO_MORE_FILES)
printf("No files were found\n");
else
LogError("Unable to find first file");
}
else
{
printf("Find handle: %p\n", hFile);
do
{
//do stuff
printf("Finding next file\n");
if (!InternetFindNextFileA(hFile, &data))
{
if (GetLastError() == ERROR_NO_MORE_FILES)
printf("No more files were found\n");
else
LogError("Unable to find next file")
break;
}
}
while (true);
InternetCloseHandle(hFile);
}
InternetCloseHandle(hServer);
}
InternetCloseHandle(hConnect);
}
To make this code run asynchronously, get rid of all off the waits and implement a state machine that your callback advances as needed, eg:
enum FTPState {ftpConnect, ftpWaitingForConnect, ftpFindFirstFile, ftpWaitingForFirstFind, ftpFindNextFile, ftpWaitingForNextFind, ftpProcessFile, ftpDisconnect};
struct REQ_CONTEXT
{
FTPState State;
WIN32_FIND_DATAA data;
HINTERNET hConnect;
HINTERNET hServer;
HINTERNET hFile;
HANDLE hDoneEvent;
};
void LogError(const char *Op, DWORD err)
{
if (err == ERROR_INTERNET_EXTENDED_ERROR)
{
LPSTR szBuffer;
DWORD dwLength = 0;
InternetGetLastResponseInfoA(&err, NULL, &dwLength);
if (GetLastError() != ERROR_INSUFFICIENT_BUFFER)
{
printf("%s. Unknown Inet Error. OS Error: %u", Op, GetLastError());
return;
}
szBuffer = new char[dwLength+1];
InternetGetLastResponseInfoA(&err, szBuffer, &dwLength);
szBuffer[dwLength] = 0;
printf("%s. Inet Error: %u %s", Op, err, szBuffer);
delete[] szBuffer;
}
else
{
LPSTR lpBuffer = NULL;
DWORD dwLen = FormatMessageA(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_ARGUMENT_ARRAY | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, NULL, err, 0, (LPSTR)&lpBuffer, 0, NULL);
if (lpBuffer)
{
printf("%s. OS Error: %u %s", Op, err, lpBuffer);
LocalFree(lpBuffer);
}
else
printf("%s. OS Error: %u", Op, err);
}
printf("\n");
}
void LogError(const char *Op)
{
LogError(Op, GetLastError());
}
void DoNextStep(REQ_CONTEXT *ctx)
{
do
{
if ((ctx->State == ftpConnect) && (!f[FLAG_FTP_ITERATE]))
{
printf("Done!\n");
SetEvent(ctx->hDoneEvent);
return;
}
switch (ctx->State)
{
case ftpConnect:
{
printf("Connecting to Server\n");
HINTERNET hServer = InternetConnect(ctx->hConnect, ftp_url, INTERNET_DEFAULT_FTP_PORT, ftp_user, ftp_pass, INTERNET_SERVICE_FTP, NULL, (DWORD_PTR)ctx);
if (hServer != NULL)
{
if (ctx->hServer == NULL)
{
ctx->hServer = hServer;
printf("Server handle: %p\n", ctx->hServer);
}
printf("Connected to Server\n");
ctx->State = ftpFindFirstFile;
}
else if (GetLastError() == ERROR_IO_PENDING)
{
if (ctx->hServer == NULL)
printf("Waiting for Server handle\n");
printf("Waiting for Server connect to complete\n");
ctx->State = ftpWaitingForConnect;
}
else
LogError("Unable to connect to Server");
break;
}
case ftpWaitingForConnect:
return;
case ftpFindFirstFile:
{
printf("Finding first file\n");
HINTERNET hFile = FtpFindFirstFileA(ctx->hServer, ftp_base, &ctx->data, INTERNET_FLAG_NO_CACHE_WRITE, (DWORD_PTR)ctx);
if (hFile != NULL)
{
if (ctx->hFile == NULL)
{
ctx->hFile = hFile;
printf("Find handle: %p\n", ctx->hFile);
}
ctx->State = ftpProcessFile;
}
else if (GetLastError() == ERROR_IO_PENDING)
{
if (ctx->hFile == NULL)
printf("Waiting for Find handle\n");
printf("Waiting for Find to complete\n");
ctx->State = ftpWaitingForFirstFind;
}
else
{
if (GetLastError() == ERROR_NO_MORE_FILES)
printf("No files were found\n");
else
LogError("Unable to find first file");
ctx->State = ftpDisconnect;
}
break;
}
case ftpWaitingForFirstFind:
case ftpWaitingForNextFind:
return;
case ftpProcessFile:
{
//do stuff
printf("Finding next file\n");
if (!InternetFindNextFileA(ctx->hFile, &ctx->data))
{
if (GetLastError() == ERROR_IO_PENDING)
{
printf("Waiting for next file to complete\n");
ctx->State = ftpWaitingForNextFind;
}
else
{
if (GetLastError() == ERROR_NO_MORE_FILES)
printf("No more files were found\n");
else
LogError("Unable to find next file");
ctx->State = ftpDisconnect;
}
}
break;
}
case ftpDisconnect:
{
printf("Disconnecting\n");
if (ctx->hFile != NULL)
{
InternetCloseHandle(ctx->hFile);
ctx->hFile = NULL;
}
if (ctx->hServer != NULL)
{
InternetCloseHandle(ctx->hServer);
ctx->hServer = NULL;
}
ctx->State = ftpConnect;
break;
}
}
}
while (true);
}
void CALLBACK CallbackFunction(HINTERNET hInternet, DWORD_PTR dwContext, DWORD dwInternetStatus, LPVOID lpvStatusInformation, DWORD dwStatusInformationLength)
{
REQ_CONTEXT *ctx = (REQ_CONTEXT*) dwContext;
switch (dwInternetStatus)
{
case INTERNET_STATUS_HANDLE_CREATED:
{
LPINTERNET_ASYNC_RESULT Result = (LPINTERNET_ASYNC_RESULT) lpvStatusInformation;
switch (ctx->State)
{
case ftpConnect:
case ftpWaitingForConnect:
ctx->hServer = (HINTERNET) Result->dwResult;
printf("Server handle: %p\n", ctx->hServer);
break;
case ftpFindFirstFile:
case ftpWaitingForFirstFind:
ctx->hFile = (HINTERNET) Result->dwResult;
printf("Find handle: %p\n", ctx->hFile);
break;
}
break;
}
case INTERNET_STATUS_REQUEST_COMPLETE:
{
LPINTERNET_ASYNC_RESULT Result = (LPINTERNET_ASYNC_RESULT) lpvStatusInformation;
switch (ctx->State)
{
case ftpWaitingForConnect:
{
if (!Result->dwResult)
{
LogError("Unable to connect to Server", Result->dwError);
ctx->State = ftpDisconnect;
}
else
{
printf("Connected to Server\n");
ctx->State = ftpFindFirstFile;
}
break;
}
case ftpWaitingForFirstFind:
case ftpWaitingForNextFind:
{
if (!Result->dwResult)
{
if (Result->dwError == ERROR_NO_MORE_FILES)
printf("No %sfiles were found\n", (ctx->State == ftpWaitingForNextFind) ? "more " : "");
else if (ctx->State == ftpWaitingForFirstFind)
LogError("Unable to find first file", Result->dwError);
else
LogError("Unable to find next file", Result->dwError);
ctx->State = ftpDisconnect;
}
else
ctx->State = ftpProcessFile;
break;
}
}
DoNextStep(ctx);
break;
}
}
}
void FTPIterate()
{
REQ_CONTEXT ctx = {0};
ctx.State = ftpConnect;
ctx.hDoneEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
if (ctx.hDoneEvent == NULL)
{
LogError("Unable to Create Done Event");
return;
}
ctx.hConnect = InternetOpen(NULL, INTERNET_OPEN_TYPE_PROXY, proxy_url, NULL, INTERNET_FLAG_ASYNC);
if (ctx.hConnect == NULL)
{
LogError("Unable to Open Internet");
CloseHandle(ctx.hDoneEvent);
return;
}
printf("Connect handle: %p\n", ctx.hConnect);
InternetSetStatusCallback(ctx.hConnect, &CallbackFunction);
DoNextStep(&ctx);
WaitForSingleObject(ctx.hDoneEvent, INFINITE);
InternetCloseHandle(ctx.hConnect);
CloseHandle(ctx.hDoneEvent);
}