I have a part of the project that is intended to re-read some portion of a file in a loop (prepended with the pointer movement to the beginning of the file). By start, the code opens the file and writes there the 'minimal data' correctly, but the further read (on the same file handle(!)) fails with error code 2 ('File not found').
Here the part of code related to the process:
virtual-mem-buffer.h:
/**
* Allocates and manages page-aligned virtual memory of the given amount
*/
class VirtualMemBuffer {
public:
explicit VirtualMemBuffer(size_t size) { /* skipped */ };
/* skipped */
protected:
void * data;
public:
const void * buff() const { return this->data; };
};
header-file.h:
static const size_t cFileSize = 4096;
typedef std::map<std::wstring, HANDLE> handlers_conrainer_type;
typedef std::pair<std::wstring, bool> item_type;
class Config {
public:
typedef std::vector<item_type > container_type;
container_type files;
/* skipped */
};
code-file.cpp (inside some function):
VirtualMemBuffer buffer(cFileSize);
Config config(...);
config->files.push_back(item_type(L"C:\\lock-file.lock", true));
/* skipped */
for (Config::container_type::const_iterator it = config->files.begin();
it != config->files.end();
++it)
{
HANDLE hFile = CreateFile(
(LPCWSTR)(it->first.c_str()),
GENERIC_READ | GENERIC_WRITE,
0,
NULL,
OPEN_ALWAYS,
FILE_ATTRIBUTE_NORMAL,
NULL);
if (hFile == INVALID_HANDLE_VALUE) {
DWORD error = GetLastError();
// Could not open the file, ignore it
continue;
} else {
DWORD bytes_written = 0;
BOOL write_ok = WriteFile(
hFile,
buffer.buff(),
cFileSize,
&bytes_written,
NULL);
if (!write_ok || (bytes_written != (DWORD)(cFileSize))) {
// Could not initialize the file, skip the file
CloseHandle(hFile);
continue;
};
handlers_container.insert(
std::pair<std::wstring, HANDLE>(it->first, hFile)
);
};
};
/* skipped */
for (handlers_conrainer_type::const_iterator it = handlers_container.begin();
it != handlers_container.end();
++it)
{
DWORD bytes_read = 0;
LARGE_INTEGER li;
li.HighPart = 0;
li.LowPart = 0;
BOOL move_ok = SetFilePointerEx(it->second, li, NULL, FILE_BEGIN);
BOOL read_ok = ReadFile(
it->second,
buffer.buff(),
cFileSize,
&bytes_read,
NULL);
if (!read_ok || (bytes_read != cFileSize)) {
DWORD error = GetLastError(); // error == 2 :-(
/* skipped */
};
};
As you can see, SetFilePointerEx() and ReadFile() both operate on the same file handle. The first one (and CreateFile(), WriteFile() ) never failed, but ReadFile() never succeded.
Did anybody observe such a behavior or at least have any clue on this? What is wrong and how can it be fixed (or avoided)?
The code compiled on Windows XP SP3 using MS Visual C++ 2008 Express Edition
Thank you for your time and advices!
I have found the root of the problem - the error code for the ReadFile() fail was tested in MS VC++ debugger where (and when) the variable was actually out of the (block) scope and thus flooded with garbage. The value 2 was just a compiler preference for such condition.
I have just noticed that and added some further error checking and code around that, and found that the real error code is 998 ('Invalid access to memory location') which itself came from the VirtualMemBuffer class where VirtualAlloc() was called with PAGE_READONLY flag :-(.
So, it was my fault, sorry.
Thanks to everyone who spent time trying to help me with this problem.
Try FlushFileBuffers(handle) to commit the writes to disk before you try and read them
Related
I have several processes that try to read and write the same file. I want each of them to lock the file so that only one of them accesses it at a time.
I tried this (edit: this is a complete test code this time):
#include "stdafx.h"
#include "Windows.h"
bool test()
{
const char* path = "test.txt";
HANDLE hFile = CreateFileA(path,
GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_READ | FILE_SHARE_WRITE,
NULL,
OPEN_ALWAYS,
FILE_ATTRIBUTE_NORMAL,
NULL);
if (hFile == INVALID_HANDLE_VALUE)
{
printf("ERROR: Cannot open file %s\n", path);
return false;
}
// Lock the file
{
OVERLAPPED overlapped = {0};
BOOL res = LockFileEx(hFile, LOCKFILE_EXCLUSIVE_LOCK, 0, ~0, ~0, &overlapped);
if (!res)
{
printf("ERROR: Cannot lock file %s\n", path);
return false;
}
}
DWORD fileSize = GetFileSize(hFile, NULL);
if (fileSize > 0)
{
char* content = new char[fileSize+1];
// Read the file
BOOL res = ReadFile(hFile, content, fileSize, NULL, NULL);
if (!res)
{
printf("ERROR: Cannot read file %s\n", path);
}
delete[] content;
}
const char* newContent = "bla";
int newContentSize = 3;
// Write the file
BOOL res = WriteFile(hFile, newContent, newContentSize, NULL, NULL);
if (!res)
{
//int err = GetLastError();
printf("ERROR: Cannot write to file\n");
}
// Unlock the file
{
OVERLAPPED overlapped = {0};
UnlockFileEx(hFile, 0, ~0, ~0, &overlapped);
}
CloseHandle(hFile);
return true;
}
int _tmain(int argc, _TCHAR* argv[])
{
bool res = test();
return 0;
}
This works fine on my computer, which has Windows 8. But on my colleague's computer, which has Windows 7, it crashes. Specifically, the calls to ReadFile and WriteFile crash, always.
Note that it never enters the code paths with the error printfs. This code triggers no error except for a write at location 0x00000000 in ReadFile (when run on Windows 7).
We tried to also pass the overlapped struct to the ReadFile and WriteFile calls. It prevents the crash but the lock doesn't work anymore, the file is all scrambled (not with this test code, with the real code).
What am I doing wrong?
Looks like your problem is:
lpNumberOfBytesRead [out, optional] argument is null in your call.
This parameter can be NULL only when the lpOverlapped parameter is not NULL.
http://msdn.microsoft.com/en-us/library/windows/desktop/aa365467%28v=vs.85%29.aspx
Heres your problem :
You are missing a necessary struct-member and:
0 and ~0 and {0} are all bad code, constant expressions like these will always produce unepected results -- WINAPI doesnt work like libc, parameters are not always compared against constants, instead they are tested against/via macros and other preprocessor-definitions themselves so passing constant values or initializing WINAPI structs with constants will often lead to errors like these.
After years of experimenting i have found that there is only one surefire way of avoiding them, i will express it in corrected code :
OVERLAPPED overlapped;
overlapped.hEvent = CreateEvent( ........... ); // put valid parameters here!
UnlockFileEx(hFile, 0 /*"reserved"*/, ULONG_MAX, ULONG_MAX, &overlapped);
please read this carefully : http://msdn.microsoft.com/en-us/library/windows/desktop/aa365716%28v=vs.85%29.aspx
I'm experiencing a curious problem (very strange, let me say hehe). During a FTP download of an EXE file (24 MB), if the connection is ever interrupted, it appears that the function FtpGetFile of the WinINEt library has a bug and it never returns. This causes that future file transfers fail (the connection is already opened).
Apparently, I found a workaround by increasing the timeout of the server transfers but I do not like it. I didn't found a similar problem by googling (maybe I introduced the wrong keywords).
I read some forums on the internet and it seems that everyone does not recommend using the FtpGetFile because it is buggy.
This appears in a network scenario that has a big lag (and not always) but in good conditions it disappears (downloads take place correctly and FtpGetFile returns always).
Here is how I use the function:
if( FtpGetFile(m_hFtpSession, strSourcePath.c_str(), strTargetPath.c_str(), 0, 0, FTP_TRANSFER_TYPE_BINARY, 0)==TRUE)
Can anyone confirm that? Should I refactor my code and look for an update?
Thank you
I found a way to download files without using FtpGetFile. I hope this code can help someone:
bool RetrieveFile(const string& strSource, const string& strTarget) {
/* The handle for the transfer */
HINTERNET hTransfer = NULL;
/*
* Set default error
*/
DWORD error = ERROR_SUCCESS;
if( !isConnected ) {
debug("%s(): ERROR not connected\n", __FUNCTION__);
return false;
}
/* Initiate access to a remote FTP connection */
hTransfer = FtpOpenFile(hFtpSession, strSource.c_str(), GENERIC_READ,
FTP_TRANSFER_TYPE_BINARY, 0);
if(hTransfer) {
std::ofstream myostream(strTarget.c_str(), std::ios::binary);
if ( myostream.is_open() ) {
static const DWORD SIZE = 1024;
BYTE data[SIZE];
DWORD size = 0;
do {
BOOL result = InternetReadFile(hTransfer, data, SIZE, &size);
if ( result == FALSE ) {
error = GetLastError();
Debug("InternetReadFile(): %lu\n", error);
}
myostream.write((const char*)data, size);
}
while ((error == ERROR_SUCCESS) && (size > 0));
// Close the stream
myostream.close();
}
else {
Debug("Could not open '%s'.\n", strTarget.c_str());
error = ERROR_FILE_NOT_FOUND; // Not necessarily not found, but it is to describe a file error which is different from ERROR_SUCCESS
}
// Close
const BOOL result = InternetCloseHandle(hTransfer);
if ( result == FALSE ) {
const DWORD error = GetLastError();
debug("InternetClose(): %lu\n", error);
}
/* Check error status of the process */
return (error == ERROR_SUCCESS);
}
DWORD dwInetError;
DWORD dwExtLength = 1000;
TCHAR *szExtErrMsg = NULL;
TCHAR errmsg[1000];
szExtErrMsg = errmsg;
int returned = InternetGetLastResponseInfo( &dwInetError, szExtErrMsg, &dwExtLength );
debug("dwInetError: %d Returned: %d\n", dwInetError, returned);
debug("Buffer: %s\n", szExtErrMsg);
debug("%s() : ERROR to get '%s' file (errorCode=%d)\n", __FUNCTION__, strSource.c_str(), GetLastError());
return false;
}
I am coding a programming to watch folder. I use FileWatch.h library. This is my FileWatch.h
#ifndef FILEWATCH_H
#define FILEWATCH_H
class FileChangeObserver
{
public:
virtual ~FileChangeObserver()
{
}
virtual void OnFileChanged() = 0;
};
// information concerning a directory being watched
class FileWatcher
{
public:
// Watching file modifications using a loop
void Init(LPCTSTR filefullpath);
bool CheckForChanges(DWORD waittime=0);
// Watching file modification via a thread
void StartWatchThread();
bool IsThreadRunning();
void SynchronousAbort();
FileWatcher(FileChangeObserver *observer) : hDir(NULL), curBuffer(0),
filePath(NULL), hWatchingThread(NULL), observer(observer)
{
ZeroMemory(&this->overl, sizeof(this->overl));
// create the event used to abort the "watching" thread
hEvtStopWatching = CreateEvent(NULL, TRUE, FALSE, NULL);
}
FileWatcher()
{
}
~FileWatcher()
{
SynchronousAbort();
delete observer;
free(filePath);
CloseHandle(hEvtStopWatching);
}
public:
HANDLE hDir; // handle of the directory to watch
FileChangeObserver *observer; // function called when a file change is detected
TCHAR * filePath; // path to the file watched
FILE_NOTIFY_INFORMATION buffer[2][512];
// a double buffer where the Windows API ReadDirectory will store the list
// of files that have been modified.
int curBuffer; // current buffer used (alternate between 0 and 1)
bool NotifyChange();
public:
// fields for use by the WathingThread
OVERLAPPED overl; // object used for asynchronous API calls
HANDLE hWatchingThread; // handle of the watching thread
HANDLE hEvtStopWatching; // this event is fired when the watching thread needs to be aborted
};
static DWORD WINAPI WatchingThread(void *param);
#endif
This is my FileWatch.cpp
#include "stdafx.h"
#include "FileWatch.h"
#include "assert.h"
#if _MSC_VER > 1600
extern "C" {
WINBASEAPI BOOL WINAPI
GetOverlappedResult(_In_ HANDLE hFile, _In_ LPOVERLAPPED lpOverlapped, _Out_ LPDWORD lpNumberOfBytesTransferred, _In_ BOOL bWait);
}
#endif
bool FileWatcher::IsThreadRunning()
{
return hWatchingThread && (WaitForSingleObject(hWatchingThread, 0) == WAIT_TIMEOUT);
}
// Ask for the thread to stop and waith until it ends
void FileWatcher::SynchronousAbort()
{
SetEvent(hEvtStopWatching);
if (hWatchingThread)
{
WaitForSingleObject(hWatchingThread, INFINITE);
CloseHandle(hWatchingThread);
Sleep(500);
hWatchingThread = NULL;
}
CloseHandle(overl.hEvent);
overl.hEvent = NULL;
CloseHandle(hDir);
hDir = NULL;
}
// Start watching a file for changes
void FileWatcher::StartWatchThread()
{
// if the thread already exists then stop it
if (IsThreadRunning())
SynchronousAbort();
assert(hDir);
if (!hDir)
{
return;
}
// reset the hEvtStopWatching event so that it can be set if
// some thread requires the watching thread to stop
ResetEvent(hEvtStopWatching);
DWORD watchingthreadID;
hWatchingThread = CreateThread(NULL, 0, WatchingThread, this, 0, &watchingthreadID);
}
void FileWatcher::Init(const TCHAR* fileFullPath)
{
// if the thread already exists then stop it
if (IsThreadRunning())
SynchronousAbort();
// str::ReplacePtr(&filePath, fileFullPath);
//TCHAR *dirPath = path::GetDir(filePath);
hDir = CreateFile(
L"C:\\", // pointer to the directory containing the tex files
FILE_LIST_DIRECTORY, // access (read-write) mode
FILE_SHARE_READ|FILE_SHARE_DELETE|FILE_SHARE_WRITE, // share mode
NULL, // security descriptor
OPEN_EXISTING, // how to create
FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OVERLAPPED , // file attributes
NULL); // file with attributes to copy
// free(dirPath);
ZeroMemory(&overl, sizeof(overl));
ZeroMemory(buffer, sizeof(buffer));
overl.hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
// watch the directory
ReadDirectoryChangesW(
hDir, /* handle to directory */
&buffer[curBuffer], /* read results buffer */
sizeof(buffer[curBuffer]), /* length of buffer */
FALSE, /* monitoring option */
//FILE_NOTIFY_CHANGE_CREATION|
FILE_NOTIFY_CHANGE_LAST_WRITE, /* filter conditions */
NULL, /* bytes returned */
&overl, /* overlapped buffer */
NULL); /* completion routine */
}
// Thread responsible of watching the directory containg the file to be watched for modifications
DWORD WINAPI WatchingThread(void *param)
{
//qDebug()<<"in WatchingThread";
FileWatcher *fw = (FileWatcher *)param;
HANDLE hp[2] = { fw->hEvtStopWatching, fw->overl.hEvent };
for (;;)
{
DWORD dwObj = WaitForMultipleObjects((sizeof(hp)/(sizeof(hp[0])))
, hp, FALSE, INFINITE);
if (dwObj == WAIT_OBJECT_0) // the user asked to quit the program
{
//qDebug()<<"in WatchingThread the user asked to quit the program";
//exit(-1);
break;
}
if (dwObj != WAIT_OBJECT_0 + 1)
{
// BUG!
//assert(0);
// qDebug()<<"dwObj "<<dwObj<<" last error "<<GetLastError();
break;
}
//qDebug()<<"WatchingThread fw->NotifyChange() ";
//if (fw->wakeup)
fw->NotifyChange();
}
return 0;
}
// Call ReadDirectoryChangesW to check if the file has changed since the last call.
bool FileWatcher::CheckForChanges(DWORD waittime)
{
if (!overl.hEvent)
{
return false;
}
DWORD dwObj = WaitForSingleObject(overl.hEvent, waittime);
if (dwObj != WAIT_OBJECT_0)
{
return false;
}
return NotifyChange();
}
// Call the ReadDirectory API and determine if the file being watched has been modified since the last call.
// Returns true if it is the case.
bool FileWatcher::NotifyChange()
{
//qDebug()<<"in NotifyChange";
DWORD dwNumberbytes;
GetOverlappedResult(hDir, &overl, &dwNumberbytes, FALSE);
FILE_NOTIFY_INFORMATION *pFileNotify = (FILE_NOTIFY_INFORMATION *)buffer[curBuffer];
// Switch the 2 buffers
curBuffer = (curBuffer + 1) % (sizeof(buffer)/(sizeof(buffer[0])));
SecureZeroMemory(buffer[curBuffer], sizeof(buffer[curBuffer]));
// start a new asynchronous call to ReadDirectory in the alternate buffer
ReadDirectoryChangesW(
hDir, /* handle to directory */
&buffer[curBuffer], /* read results buffer */
sizeof(buffer[curBuffer]), /* length of buffer */
TRUE, /* monitoring option */
FILE_NOTIFY_CHANGE_FILE_NAME |
FILE_NOTIFY_CHANGE_DIR_NAME |
FILE_NOTIFY_CHANGE_ATTRIBUTES |
FILE_NOTIFY_CHANGE_SIZE |
FILE_NOTIFY_CHANGE_LAST_WRITE |
FILE_NOTIFY_CHANGE_LAST_ACCESS |
FILE_NOTIFY_CHANGE_CREATION |
FILE_NOTIFY_CHANGE_SECURITY,
//FILE_NOTIFY_CHANGE_LAST_WRITE, /* filter conditions */
NULL, /* bytes returned */
&overl, /* overlapped buffer */
NULL); /* completion routine */
// Note: the ReadDirectoryChangesW API fills the buffer with WCHAR strings.
for (;;)
{
if (pFileNotify->Action == FILE_ACTION_ADDED)
{
//qDebug()<<"in NotifyChange if ";
char szAction[42];
char szFilename[MAX_PATH] ;
memset(szFilename,'\0',sizeof( szFilename));
strcpy(szAction,"added");
wcstombs( szFilename, pFileNotify->FileName, MAX_PATH);
if(observer)
observer->OnFileChanged();
return true;
//OnFileChanged(szFilename,szAction);
// qDebug()<<"in NotifyChange after OnFileChanged ";
}
// step to the next entry if there is one
if (!pFileNotify->NextEntryOffset)
{
return false;
}
pFileNotify = (FILE_NOTIFY_INFORMATION *)((PBYTE)pFileNotify + pFileNotify->NextEntryOffset);
}
pFileNotify=NULL;
return true;
}
In main program, I have:
case IDM_ABOUT:
//DialogBox(hInst, MAKEINTRESOURCE(IDD_ABOUTBOX), hWnd, About);
{
FileWatcher *cWatcher = new FileWatcher();
cWatcher->Init(L"AB");
cWatcher->NotifyChange();
break;
}
I have a error messeage that: Access violation reading location 0xba2f1498.
What's solution?
You have a default constructor that leaves all your variables uninitialized.
FileWatcher()
{
}
Which you use here.
FileWatcher *cWatcher = new FileWatcher();
cWatcher->Init(L"AB");
Init also leaves several variables uninitialized, such as curBuffer in this line.
&buffer[curBuffer], /* read results buffer */
That is likely why you are getting Access violation reading location 0xba2f1498
A good practice would be to make sure that your object is always completely valid before the constructor finishes.
In your destructor you have:
delete observer;
free(filePath);
but you do not check to make sure either one is allocated beforehand which is clearly a problem, especially since your default constructor does not initialize any of these variables. This is beyond the fact that you are mixing C style allocation with C++ style allocation.
I am currently trying to write a C++ program to automate retrieving information about the partitions of a sample hard-drive image, the information in question being the number of partitions on the disk and for each partition its start sector, size and and file system type.
I'm pretty sure at this point the best way to achieve this is through MSDN functions, microsofts inbuilt commands. I am trying to use the "IOCTL_DISK_GET_DRIVE_LAYOUT_EX" function, but according to my get error call my function is incorrect. When I debug the program is appears that the bool value is also unchanged after the "IOCTL_DISK_GET_DRIVE_LAYOUT_EX" call, meaning it is not returning the bResult value.
I am using Microsoft Visual C++ Express Edition. If people could take a look at my code and tell me what they think I did wrong it would be much appreciated.
#define UNICODE 1
#define _UNICODE 1
#include <windows.h>
#include <winioctl.h>
#include <stdio.h>
#define wszDrive L"\\\\.\\PhysicalDrive6"
BOOL GetDriveParition(LPWSTR wszPath, DRIVE_LAYOUT_INFORMATION_EX *pdg)
{
HANDLE hDevice = INVALID_HANDLE_VALUE; // handle to the drive to be examined
BOOL bResult = FALSE; // results flag
DWORD junk = 0; // discard results
hDevice = CreateFileW(wszPath, // drive to open
0, // no access to the drive
FILE_SHARE_READ | // share mode
FILE_SHARE_WRITE,
NULL, // default security attributes
OPEN_EXISTING, // disposition
0, // file attributes
NULL); // do not copy file attributes
if (hDevice == INVALID_HANDLE_VALUE) // cannot open the drive
{
return (FALSE);
}
bResult = DeviceIoControl(
hDevice, // handle to device
IOCTL_DISK_GET_DRIVE_LAYOUT_EX, // dwIoControlCode
NULL, // lpInBuffer
0, // nInBufferSize
pdg, // lpOutBuffer
sizeof(*pdg), // nOutBufferSize
&junk, // lpBytesReturned
NULL); // lpOverlapped
CloseHandle(hDevice);
return (bResult);
}
int wmain(int argc, wchar_t *argv[])
{
DRIVE_LAYOUT_INFORMATION_EX pdg; // disk drive partition structure
BOOL bResult = FALSE; // generic results flag
bResult = GetDriveParition (wszDrive, &pdg);
if (bResult)
{
wprintf(L"Drive path = %ws\n", wszDrive);
wprintf(L"Partition Style = %I64d\n", pdg.PartitionStyle);
wprintf(L"Partition Count = %ld\n", pdg.PartitionCount);
system("Pause");
}
else
{
wprintf (L"GetDrivePartition failed. Error %ld.\n", GetLastError ());
system("Pause");
}
return ((int)bResult);
}
DRIVE_LAYOUT_INFORMATION_EX is a weird structure. It's defined as
struct {
DWORD PartitionStyle;
DWORD PartitionCount;
union {
DRIVE_LAYOUT_INFORMATION_MBR Mbr;
DRIVE_LAYOUT_INFORMATION_GPT Gpt;
};
PARTITION_INFORMATION_EX PartitionEntry[ 1 ];
}
but usually PartitionEntry is treated as a much larger array, with PartitionCount entries. This is similar to the C99 VLA mechanism. Since you'va allocated just sizeof(*pdg) bytes, there's no room for even a second PartitionEntry.
C++ hack:
struct ExtraEntries : DRIVE_LAYOUT_INFORMATION_EX
{
PARTITION_INFORMATION_EX PartitionEntry[ 9 ]; // Or some other reasonable value
};
Even if this post is a bit old, I found another way to get a fully populated PartitionEntry without creating a tricky struct. This is how I did it:
Inspired of an answer from this post: How-to-call-deviceiocontrol-to-retrieve-the-amount-of-memory-it-needs
DRIVE_LAYOUT_INFORMATION_EX dli;
DWORD bytesReturned = 0;
if (!DeviceIoControl(hDevice, IOCTL_DISK_GET_DRIVE_LAYOUT_EX, NULL, 0, (void*)&dli, sizeof(dli), &bytesReturned, NULL))
{
// Check last error if not ERROR_INSUFFICIENT_BUFFER then return
int nError = GetLastError();
if (nError != ERROR_INSUFFICIENT_BUFFER)
{
// std::cout << "DeviceIoControl() Failed: " << nError << std::endl;
CloseHandle(hDevice);
return false;
}
// Allocate enough buffer space based of the value of Partition Count:
size_t size = offsetof(DRIVE_LAYOUT_INFORMATION_EX, PartitionEntry[dli.PartitionCount]);
std::vector<BYTE> buffer(size);
if (!DeviceIoControl(hDevice, IOCTL_DISK_GET_DRIVE_LAYOUT_EX, NULL, 0, (void*)buffer.data(), size, &bytesReturned, NULL))
{
nError = GetLastError();
// std::cout << "DeviceIoControl() Failed: " << nError << std::endl;
CloseHandle(hDevice);
return false;
}
const DRIVE_LAYOUT_INFORMATION_EX& result = *reinterpret_cast<const DRIVE_LAYOUT_INFORMATION_EX*>(buffer.data());
// Here all parition entry are populated ...
// TO DO... Do your stuff with result
}
else
{
// Call succeeded; dli is populated with a signle partition entry
// TO DO... Do your stuff with dli
}
I don't have much experience with Serial I/O, but have recently been tasked with fixing some highly flawed serial code, because the original programmer has left the company.
The application is a Windows program that talks to a scientific instrument serially via a virtual COMM port running on USB. Virtual COMM port USB drivers are provided by FTDI, since they manufacture the USB chip we use on the instrument.
The serial code is in an unmanaged C++ DLL, which is shared by both our old C++ software, and our new C# / .Net (WinForms) software.
There are two main problems:
Fails on many XP systems
When the first command is sent to the instrument, there's no response. When you issue the next command, you get the response from the first one.
Here's a typical usage scenario (full source for methods called is included below):
char szBuf [256];
CloseConnection ();
if (OpenConnection ())
{
ClearBuffer ();
// try to get a firmware version number
WriteChar ((char) 'V');
BOOL versionReadStatus1 = ReadString (szBuf, 100);
...
}
On a failing system, the ReadString call will never receive any serial data, and times out. But if we issue another, different command, and call ReadString again, it will return the response from the first command, not the new one!
But this only happens on a large subset of Windows XP systems - and never on Windows 7. As luck would have it, our XP dev machines worked OK, so we did not see the problem until we started beta testing. But I can also reproduce the problem by running an XP VM (VirtualBox) on my XP dev machine. Also, the problem only occurs when using the DLL with the new C# version - works fine with the old C++ app.
This seemed to be resolved when I added a Sleep(21) to the low level BytesInQue method before calling ClearCommError, but this exacerbated the other problem - CPU usage. Sleeping for less than 21 ms would make the failure mode reappear.
High CPU usage
When doing serial I/O CPU use is excessive - often above 90%. This happens with both the new C# app and the old C++ app, but is much worse in the new app. Often makes the UI very non-responsive, but not always.
Here's the code for our Port.cpp class, in all it's terrible glory. Sorry for the length, but this is what I'm working with. Most important methods are probably OpenConnection, ReadString, ReadChar, and BytesInQue.
//
// Port.cpp: Implements the CPort class, which is
// the class that controls the serial port.
//
// Copyright (C) 1997-1998 Microsoft Corporation
// All rights reserved.
//
// This source code is only intended as a supplement to the
// Broadcast Architecture Programmer's Reference.
// For detailed information regarding Broadcast
// Architecture, see the reference.
//
#include <windows.h>
#include <stdio.h>
#include <assert.h>
#include "port.h"
// Construction code to initialize the port handle to null.
CPort::CPort()
{
m_hDevice = (HANDLE)0;
// default parameters
m_uPort = 1;
m_uBaud = 9600;
m_uDataBits = 8;
m_uParity = 0;
m_uStopBits = 0; // = 1 stop bit
m_chTerminator = '\n';
m_bCommportOpen = FALSE;
m_nTimeOut = 50;
m_nBlockSizeMax = 2048;
}
// Destruction code to close the connection if the port
// handle was valid.
CPort::~CPort()
{
if (m_hDevice)
CloseConnection();
}
// Open a serial communication port for writing short
// one-byte commands, that is, overlapped data transfer
// is not necessary.
BOOL CPort::OpenConnection()
{
char szPort[64];
m_bCommportOpen = FALSE;
// Build the COM port string as "COMx" where x is the port.
if (m_uPort > 9)
wsprintf(szPort, "\\\\.\\COM%d", m_uPort);
else
wsprintf(szPort, "COM%d", m_uPort);
// Open the serial port device.
m_hDevice = CreateFile(szPort,
GENERIC_WRITE | GENERIC_READ,
0,
NULL, // No security attributes
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL);
if (m_hDevice == INVALID_HANDLE_VALUE)
{
SaveLastError ();
m_hDevice = (HANDLE)0;
return FALSE;
}
return SetupConnection(); // After the port is open, set it up.
} // end of OpenConnection()
// Configure the serial port with the given settings.
// The given settings enable the port to communicate
// with the remote control.
BOOL CPort::SetupConnection(void)
{
DCB dcb; // The DCB structure differs betwwen Win16 and Win32.
dcb.DCBlength = sizeof(DCB);
// Retrieve the DCB of the serial port.
BOOL bStatus = GetCommState(m_hDevice, (LPDCB)&dcb);
if (bStatus == 0)
{
SaveLastError ();
return FALSE;
}
// Assign the values that enable the port to communicate.
dcb.BaudRate = m_uBaud; // Baud rate
dcb.ByteSize = m_uDataBits; // Data bits per byte, 4-8
dcb.Parity = m_uParity; // Parity: 0-4 = no, odd, even, mark, space
dcb.StopBits = m_uStopBits; // 0,1,2 = 1, 1.5, 2
dcb.fBinary = TRUE; // Binary mode, no EOF check : Must use binary mode in NT
dcb.fParity = dcb.Parity == 0 ? FALSE : TRUE; // Enable parity checking
dcb.fOutX = FALSE; // XON/XOFF flow control used
dcb.fInX = FALSE; // XON/XOFF flow control used
dcb.fNull = FALSE; // Disable null stripping - want nulls
dcb.fOutxCtsFlow = FALSE;
dcb.fOutxDsrFlow = FALSE;
dcb.fDsrSensitivity = FALSE;
dcb.fDtrControl = DTR_CONTROL_ENABLE;
dcb.fRtsControl = RTS_CONTROL_DISABLE ;
// Configure the serial port with the assigned settings.
// Return TRUE if the SetCommState call was not equal to zero.
bStatus = SetCommState(m_hDevice, &dcb);
if (bStatus == 0)
{
SaveLastError ();
return FALSE;
}
DWORD dwSize;
COMMPROP *commprop;
DWORD dwError;
dwSize = sizeof(COMMPROP) + sizeof(MODEMDEVCAPS) ;
commprop = (COMMPROP *)malloc(dwSize);
memset(commprop, 0, dwSize);
if (!GetCommProperties(m_hDevice, commprop))
{
dwError = GetLastError();
}
m_bCommportOpen = TRUE;
return TRUE;
}
void CPort::SaveLastError ()
{
DWORD dwLastError = GetLastError ();
LPVOID lpMsgBuf;
FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER |
FORMAT_MESSAGE_FROM_SYSTEM |
FORMAT_MESSAGE_IGNORE_INSERTS,
NULL,
dwLastError,
MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), // Default language
(LPTSTR) &lpMsgBuf,
0,
NULL);
strcpy (m_szLastError,(LPTSTR)lpMsgBuf);
// Free the buffer.
LocalFree( lpMsgBuf );
}
void CPort::SetTimeOut (int nTimeOut)
{
m_nTimeOut = nTimeOut;
}
// Close the opened serial communication port.
void CPort::CloseConnection(void)
{
if (m_hDevice != NULL &&
m_hDevice != INVALID_HANDLE_VALUE)
{
FlushFileBuffers(m_hDevice);
CloseHandle(m_hDevice); ///that the port has been closed.
}
m_hDevice = (HANDLE)0;
// Set the device handle to NULL to confirm
m_bCommportOpen = FALSE;
}
int CPort::WriteChars(char * psz)
{
int nCharWritten = 0;
while (*psz)
{
nCharWritten +=WriteChar(*psz);
psz++;
}
return nCharWritten;
}
// Write a one-byte value (char) to the serial port.
int CPort::WriteChar(char c)
{
DWORD dwBytesInOutQue = BytesInOutQue ();
if (dwBytesInOutQue > m_dwLargestBytesInOutQue)
m_dwLargestBytesInOutQue = dwBytesInOutQue;
static char szBuf[2];
szBuf[0] = c;
szBuf[1] = '\0';
DWORD dwBytesWritten;
DWORD dwTimeOut = m_nTimeOut; // 500 milli seconds
DWORD start, now;
start = GetTickCount();
do
{
now = GetTickCount();
if ((now - start) > dwTimeOut )
{
strcpy (m_szLastError, "Timed Out");
return 0;
}
WriteFile(m_hDevice, szBuf, 1, &dwBytesWritten, NULL);
}
while (dwBytesWritten == 0);
OutputDebugString(TEXT(strcat(szBuf, "\r\n")));
return dwBytesWritten;
}
int CPort::WriteChars(char * psz, int n)
{
DWORD dwBytesWritten;
WriteFile(m_hDevice, psz, n, &dwBytesWritten, NULL);
return dwBytesWritten;
}
// Return number of bytes in RX queue
DWORD CPort::BytesInQue ()
{
COMSTAT ComStat ;
DWORD dwErrorFlags;
DWORD dwLength;
// check number of bytes in queue
ClearCommError(m_hDevice, &dwErrorFlags, &ComStat ) ;
dwLength = ComStat.cbInQue;
return dwLength;
}
DWORD CPort::BytesInOutQue ()
{
COMSTAT ComStat ;
DWORD dwErrorFlags;
DWORD dwLength;
// check number of bytes in queue
ClearCommError(m_hDevice, &dwErrorFlags, &ComStat );
dwLength = ComStat.cbOutQue ;
return dwLength;
}
int CPort::ReadChars (char* szBuf, int nMaxChars)
{
if (BytesInQue () == 0)
return 0;
DWORD dwBytesRead;
ReadFile(m_hDevice, szBuf, nMaxChars, &dwBytesRead, NULL);
return (dwBytesRead);
}
// Read a one-byte value (char) from the serial port.
int CPort::ReadChar (char& c)
{
static char szBuf[2];
szBuf[0] = '\0';
szBuf[1] = '\0';
if (BytesInQue () == 0)
return 0;
DWORD dwBytesRead;
ReadFile(m_hDevice, szBuf, 1, &dwBytesRead, NULL);
c = *szBuf;
if (dwBytesRead == 0)
return 0;
return dwBytesRead;
}
BOOL CPort::ReadString (char *szStrBuf , int nMaxLength)
{
char str [256];
char str2 [256];
DWORD dwTimeOut = m_nTimeOut;
DWORD start, now;
int nBytesRead;
int nTotalBytesRead = 0;
char c = ' ';
static char szCharBuf [2];
szCharBuf [0]= '\0';
szCharBuf [1]= '\0';
szStrBuf [0] = '\0';
start = GetTickCount();
while (c != m_chTerminator)
{
nBytesRead = ReadChar (c);
nTotalBytesRead += nBytesRead;
if (nBytesRead == 1 && c != '\r' && c != '\n')
{
*szCharBuf = c;
strncat (szStrBuf,szCharBuf,1);
if (strlen (szStrBuf) == nMaxLength)
return TRUE;
// restart timer for next char
start = GetTickCount();
}
// check for time out
now = GetTickCount();
if ((now - start) > dwTimeOut )
{
strcpy (m_szLastError, "Timed Out");
return FALSE;
}
}
return TRUE;
}
int CPort::WaitForQueToFill (int nBytesToWaitFor)
{
DWORD start = GetTickCount();
do
{
if (BytesInQue () >= nBytesToWaitFor)
break;
if (GetTickCount() - start > m_nTimeOut)
return 0;
} while (1);
return BytesInQue ();
}
int CPort::BlockRead (char * pcInputBuffer, int nBytesToRead)
{
int nBytesRead = 0;
int charactersRead;
while (nBytesToRead >= m_nBlockSizeMax)
{
if (WaitForQueToFill (m_nBlockSizeMax) < m_nBlockSizeMax)
return nBytesRead;
charactersRead = ReadChars (pcInputBuffer, m_nBlockSizeMax);
pcInputBuffer += charactersRead;
nBytesRead += charactersRead;
nBytesToRead -= charactersRead;
}
if (nBytesToRead > 0)
{
if (WaitForQueToFill (nBytesToRead) < nBytesToRead)
return nBytesRead;
charactersRead = ReadChars (pcInputBuffer, nBytesToRead);
nBytesRead += charactersRead;
nBytesToRead -= charactersRead;
}
return nBytesRead;
}
Based on my testing and reading, I see several suspicious things in this code:
COMMTIMEOUTS is never set. MS docs say "Unpredictable results can occur if you fail to set the time-out values". But I tried setting this, and it didn't help.
Many methods (e.g. ReadString) will go into a tight loop and hammer the port with repeated reads if they don't get data immediately . This seems to explain the high CPU usage.
Many methods have their own timeout handling, using GetTickCount(). Isn't that what COMMTIMEOUTS is for?
In the new C# (WinForms) program, all these serial routines are called directly from the main thread, from a MultiMediaTimer event. Maybe should be run in a different thread?
BytesInQue method seems to be a bottleneck. If I break to debugger when CPU usage is high, that's usually where the program stops. Also, adding a Sleep(21) to this method before calling ClearCommError seems to resolve the XP problem, but exacerbates the CPU usage problem.
Code just seems unnecessarily complicated.
My Questions
Can anyone explain why this only works with a C# program on a small number of XP systems?
Any suggestions on how to rewrite this? Pointers to good sample code would be most welcome.
There are some serious problems with that class and it makes things even worse that there is a Microsoft copyright on it.
There is nothing special about this class. And it makes me wonder why it even exists except as an Adapter over Create/Read/WriteFile. You wouldnt even need this class if you used the SerialPort class in the .NET Framework.
Your CPU usage is because the code goes into an infinite loop while waiting for the device to have enough available data. The code might as well say while(1); If you must stick with Win32 and C++ you can look into Completion Ports and setting the OVERLAPPED flag when invoking CreateFile. This way you can wait for data in a separate worker thread.
You need to be careful when communicating to multiple COM ports. It has been a long time since I've done C++ but I believe the static buffer szBuff in the Read and Write methods is static for ALL instances of that class. It means if you invoke Read against two different COM ports "at the same time" you will have unexpected results.
As for the problems on some of the XP machines, you will most certainly figure out the problem if you check GetLastError after each Read/Write and log the results. It should be checking GetLastError anyways as it sometimes isn't always an "error" but a request from the subsystem to do something else in order to get the result you want.
You can get rid of the the whole while loop for blocking if you set COMMTIMEOUTS correctly. If there is a specific timeout for a Read operation use SetCommTimeouts before you perform the read.
I set ReadIntervalTimeout to the max timeout to ensure that the Read won't return quicker than m_nTimeOut. This value will cause Read to return if the time elapses between any two bytes. If it was set to 2 milliseconds and the first byte came in at t, and the second came in at t+1, the third at t+4, ReadFile would of only returned the first two bytes since the interval between the bytes was surpassed. ReadTotalTimeoutConstant ensures that you will never wait longer than m_nTimeOut no matter what.
maxWait = BytesToRead * ReadTotalTimeoutMultiplier + ReadTotalTimeoutConstant. Thus (BytesToRead * 0) + m_nTimeout = m_nTimeout
BOOL CPort::SetupConnection(void)
{
// Snip...
COMMTIMEOUTS comTimeOut;
comTimeOut.ReadIntervalTimeout = m_nTimeOut; // Ensure's we wait the max timeout
comTimeOut.ReadTotalTimeoutMultiplier = 0;
comTimeOut.ReadTotalTimeoutConstant = m_nTimeOut;
comTimeOut.WriteTotalTimeoutMultiplier = 0;
comTimeOut.WriteTotalTimeoutConstant = m_nTimeOut;
SetCommTimeouts(m_hDevice,&comTimeOut);
}
// If return value != nBytesToRead check check GetLastError()
// Most likely Read timed out.
int CPort::BlockRead (char * pcInputBuffer, int nBytesToRead)
{
DWORD dwBytesRead;
if (FALSE == ReadFile(
m_hDevice,
pcInputBuffer,
nBytesToRead,
&dwBytesRead,
NULL))
{
// Check GetLastError
return dwBytesRead;
}
return dwBytesRead;
}
I have no idea if this is completely correct but it should give you an idea. Remove the ReadChar and ReadString methods and use this if your program relies on things being synchronous. Be careful about setting high time outs also. Communications are fast, in the milliseconds.
Here's a terminal program I wrote years ago (probably at least 15 years ago, now that I think about it). I just did a quick check, and under Windows 7 x64, it still seems to work reasonably well -- connects to my GPS, read, and displays the data coming from it.
If you look at the code, you can see that I didn't spend much time selecting the comm timeout values. I set them all to 1, intending to experiment with longer timeouts until the CPU usage was tolerable. To make a long story short, it uses so little CPU time I've never bothered. For example, on the Task Manager's CPU usage graph, I can't see any difference between it running and not. I've left it running collecting data from the GPS for a few hours at a time, and the Task Manager still says its total CPU usage is 0:00:00.
Bottom line: I'm pretty sure it could be more efficient -- but sometimes good enough is good enough. Given how heavily I don't use it any more, and the chances of ever adding anything like file transfer protocols, making it more efficient probably won't ever get to the top of the pile of things to do.
#include <stdio.h>
#include <conio.h>
#include <string.h>
#define STRICT
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
void system_error(char *name) {
// Retrieve, format, and print out a message from the last error. The
// `name' that's passed should be in the form of a present tense noun
// (phrase) such as "opening file".
//
char *ptr = NULL;
FormatMessage(
FORMAT_MESSAGE_ALLOCATE_BUFFER |
FORMAT_MESSAGE_FROM_SYSTEM,
0,
GetLastError(),
0,
(char *)&ptr,
1024,
NULL);
fprintf(stderr, "\nError %s: %s\n", name, ptr);
LocalFree(ptr);
}
int main(int argc, char **argv) {
int ch;
char buffer[64];
HANDLE file;
COMMTIMEOUTS timeouts;
DWORD read, written;
DCB port;
HANDLE keyboard = GetStdHandle(STD_INPUT_HANDLE);
HANDLE screen = GetStdHandle(STD_OUTPUT_HANDLE);
DWORD mode;
char port_name[128] = "\\\\.\\COM3";
char init[] = "";
if ( argc > 2 )
sprintf(port_name, "\\\\.\\COM%s", argv[1]);
// open the comm port.
file = CreateFile(port_name,
GENERIC_READ | GENERIC_WRITE,
0,
NULL,
OPEN_EXISTING,
0,
NULL);
if ( INVALID_HANDLE_VALUE == file) {
system_error("opening file");
return 1;
}
// get the current DCB, and adjust a few bits to our liking.
memset(&port, 0, sizeof(port));
port.DCBlength = sizeof(port);
if (!GetCommState(file, &port))
system_error("getting comm state");
if (!BuildCommDCB("baud=19200 parity=n data=8 stop=1", &port))
system_error("building comm DCB");
if (!SetCommState(file, &port))
system_error("adjusting port settings");
// set short timeouts on the comm port.
timeouts.ReadIntervalTimeout = 1;
timeouts.ReadTotalTimeoutMultiplier = 1;
timeouts.ReadTotalTimeoutConstant = 1;
timeouts.WriteTotalTimeoutMultiplier = 1;
timeouts.WriteTotalTimeoutConstant = 1;
if (!SetCommTimeouts(file, &timeouts))
system_error("setting port time-outs.");
// set keyboard to raw reading.
if (!GetConsoleMode(keyboard, &mode))
system_error("getting keyboard mode");
mode &= ~ ENABLE_PROCESSED_INPUT;
if (!SetConsoleMode(keyboard, mode))
system_error("setting keyboard mode");
if (!EscapeCommFunction(file, CLRDTR))
system_error("clearing DTR");
Sleep(200);
if (!EscapeCommFunction(file, SETDTR))
system_error("setting DTR");
if (!WriteFile(file, init, sizeof(init), &written, NULL))
system_error("writing data to port");
if (written != sizeof(init))
system_error("not all data written to port");
// basic terminal loop:
do {
// check for data on port and display it on screen.
ReadFile(file, buffer, sizeof(buffer), &read, NULL);
if (read)
WriteFile(screen, buffer, read, &written, NULL);
// check for keypress, and write any out the port.
if ( kbhit() ) {
ch = getch();
WriteFile(file, &ch, 1, &written, NULL);
}
// until user hits ctrl-backspace.
} while ( ch != 127);
// close up and go home.
CloseHandle(keyboard);
CloseHandle(file);
return 0;
}
I would add
Sleep(2);
to the while loop in CPort::WaitForQueToFill()
This will give the OS a chance to actually place some bytes in the queue.