I am writing an application responsible for uploading files to server, in C++ Winsock, i wrote it successfully using the multithreading technique and it works fine with no errors. But when i want to make use of Overlapped I/O techniques, i faced a problem that not all the file is received by the server.
I will post code sections that are related to sending and receiving files.
int iRecv = WSARecv(AcceptSocket, &DataBuf, 1, &RecvBytes, &Flags, &AcceptOverlapped, NULL);
int iBytesReceived = 0;
while(true)
{
printf("error =%d\r",WSAGetLastError());
// Step 7:
// Determine the status of the overlapped
// request
WSAGetOverlappedResult(AcceptSocket, &AcceptOverlapped, &BytesTransferred, FALSE, &Flags);
// Step 5:
// Wait for the overlapped I/O call to complete
Index = WSAWaitForMultipleEvents(EventTotal, EventArray, FALSE, WSA_INFINITE, FALSE);
// Index should be 0 because we
// have only one event handle in EventArray
// Step 6:
// Reset the signaled event
WSAResetEvent(EventArray[Index - WSA_WAIT_EVENT_0]);
if( iRecv>=0)
{
iBytesReceived += RecvBytes;
fwrite(DataBuf.buf,sizeof(char), RecvBytes,flUploadedFile);
}
if (lReceivedBytes>=iFilesize) break;
Flags = 0;
ZeroMemory(&AcceptOverlapped, sizeof(WSAOVERLAPPED));
AcceptOverlapped.hEvent = EventArray[Index - WSA_WAIT_EVENT_0];
DataBuf.len = DATA_BUFSIZE;//1024
DataBuf.buf = buffer;
iRecv=WSARecv(AcceptSocket, &DataBuf, 1, &RecvBytes, &Flags, &AcceptOverlapped, NULL);
}
I have to mention that server is receiving the filename and file size correctly.
Client-side
void UploadFile(....)
{
.
.
.
hFile = CreateFile(fp,GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,FILE_FLAG_SEQUENTIAL_SCAN,NULL);
bTrans= TransmitFile(connectedSocket,hFile,0,0,NULL,NULL,0) ;
.
.
}
Does their exist any error in my code? Any suggestion or help? i am stuck.
In seems that the first WSARecv() returns immediate with the first portion of data (file name and size). Then the code blocks because you want to get the overlapped result before the OS signals that the result is available (by setting the event handle).
Perhaps you should use a completion routine when using overlapped I/O, this will make it more easy.
Related
I've been working on a program which dialog with an external device via a serial RS422 bus. The goal is to send commands to the device, which send an answer back.
So far, the code for sending a message look like this:
OVERLAPPED osWrite = {0};
void init()
{
// Create this write operation's OVERLAPPED structure's hEvent.
osWrite.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
if (osWrite.hEvent == NULL)
// error creating overlapped event handle
std::cout << "Error osWrite.hEvent" << std::endl; // Error in communications; report it.
*hPort = CreateFile("\\\\.\\COM18", (GENERIC_READ | GENERIC_WRITE), FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, FILE_FLAG_OVERLAPPED, NULL);
if (*hPort == INVALID_HANDLE_VALUE) {
std::cout << "Invalid port com handle" << GetLastError() << std::endl;
return;
}
COMMTIMEOUTS commTimeout;
if (GetCommTimeouts(*hPort, &commTimeout)) {
commTimeout.ReadIntervalTimeout = 10;
commTimeout.ReadTotalTimeoutConstant = 10;
commTimeout.ReadTotalTimeoutMultiplier = 10;
commTimeout.WriteTotalTimeoutConstant = 10;
commTimeout.WriteTotalTimeoutMultiplier = 10;
} else
return;
if (!SetCommTimeouts(*hPort, &commTimeout)) {
std::cout << "Error comm timeout" << std::endl;
}
DCB dcb;
if (!GetCommState(*hPort, &dcb)) {
std::cout << "Invalid port com settings" << std::endl;
return;
}
dcb.BaudRate = CBR_115200;
dcb.ByteSize = 8;
dcb.Parity = NOPARITY;
dcb.StopBits = ONESTOPBIT;
SetCommMask(*hPort, EV_RXCHAR);
SetCommState(*hPort, &dcb);
return;
}
DWORD serial_send(HANDLE *hPort, char *msg, int length) {
DWORD dwWritten;
DWORD dwRes;
BOOL fRes;
PurgeComm(*hPort, PURGE_TXCLEAR);
ResetEvent(osWrite.hEvent);
// Issue write.
if (!WriteFile(*hPort, msg, length, &dwWritten, &osWrite)) {
if (GetLastError() != ERROR_IO_PENDING) {
// WriteFile failed, but isn't delayed. Report error and abort.
fRes = FALSE;
} else {
fRes = FALSE;
while (!fRes) {
// Write is pending.
dwRes = WaitForSingleObject(osWrite.hEvent, INFINITE);
switch (dwRes) {
// OVERLAPPED structure's event has been signaled.
case WAIT_OBJECT_0:
if (!GetOverlappedResult(*hPort, &osWrite, &dwWritten, FALSE))
fRes = FALSE;
else
// Write operation completed successfully.
fRes = TRUE;
break;
default:
// An error has occurred in WaitForSingleObject.
// This usually indicates a problem with the
// OVERLAPPED structure's event handle.
fRes = FALSE;
break;
}
}
}
} else {
// WriteFile completed immediately.
fRes = TRUE;
}
return dwWritten;
}
The last function can't return until the write operation is successful. The init() function load without error.
I've used a lot of code from here : https://learn.microsoft.com/en-us/previous-versions/ff802693(v=msdn.10)
Each message is 210 bytes long, and the serial port is running at 115200 bit/s, meaning that I should send a message every ~18.2ms. (210 bytes * 10 bits / 115200)
However, when I measure the time elapsed between 2 messages, I sometimes get a duration much inferior than the expected 18ms (it can go down to 11ms).
Is this a normal behavior for an asynchronous WriteFile + WaitForSingleObject?
What happens if I send another message just after 11ms, will it corrupt the previous message, or does it gets buffered?
I used std::chrono::high_resolution_clock::now() and std::chrono::duration<double, std::milli>(end - start).count() to get the duration of the frame, is it really accurate?
Since Windows is not a real-time OS and is a multi-process & multi-thread OS, time accuracy should not be guaranteed.
If the system is lightly loaded, most of it will work as intended, but not always.
Conversely, the completion of WriteFile() may be notified earlier than it actually is, depending on the hardware and device driver stack configuration.
For example, it may be considered that the process is completed at the time when the data is completely stored in the buffer of the device driver or when the last data is written to the FIFO buffer of the interface chip.
It is better to think that WriteFile() may be completed even if not all the data actually reaches the other party.
It is considered the same as writing file data to the hard disk. Completion of writing to a file on disk is done in the system buffer, and should be written to the actual media at a different time.
If the next serial_send() function is called before all the WriteFile data of the previous time has reached the other party due to bad conditions, there is a possibility that some of the previous transmission data will be discarded.
Because PurgeComm(*hPort, PURGE_TXCLEAR); is called at the beginning of the serial_send() function.
It's not as critical as specifying PURGE_TXABORT, but there is still the possibility of data being discarded with PURGE_TXCLEAR.
PurgeComm function
PURGE_TXABORT 0x0001 Terminates all outstanding overlapped write operations and returns immediately, even if the write operations have not been completed.
PURGE_TXCLEAR 0x0004 Clears the output buffer (if the device driver has one).
If a thread uses PurgeComm to flush an output buffer, the deleted characters are not transmitted. To empty the output buffer while ensuring that the contents are transmitted, call the FlushFileBuffers function (a synchronous operation).
The workaround is to simply not call PurgeComm().
If it is a serial port API, it may be possible to wait for the completion of transmission by specifying/detecting EV_TXEMPTY with SetCommMask()/WaitCommEvent(), but this will only be complicated.
SetCommMask function / WaitCommEvent function
EV_TXEMPTY 0x0004 The last character in the output buffer was sent.
Then your usage of WriteFile() + WaitForSingleObject() + GetOverlappedResult() will eventually work similar to calling WriteFile() synchronously.
Asynchronous operation is not always necessary, but it is better to consider in detail based on what kind of behavior your system requires.
I am using non overlapped WaitCommEvent to read the file data.
I would like to provide a piece of code as follows...
SetCommMask (io_ptr->comPortHandles->hComPort, EV_RXCHAR|EV_TXEMPTY);
WaitCommEvent (io_ptr->comPortHandles->hComPort, &dwMask, 0);
if (dwMask &= EV_RXCHAR) {
// Loop getting data.
// Need to loop because our buffer is only 1024 bytes
while (TRUE)
{
ClearCommError( io_ptr->comPortHandles->hComPort, &dwError, &comstat);
if (!comstat.cbInQue) continue;
else
{
if(comstat.cbInQue > 0)
ReceiveInterrupt(io_ptr, comstat);
}
// Loop around and check for more data
// In case additional byte has arrived while reading.
}
}
WaitCommEvent is blocking if the file handle was not open with overlapped flag.In you case it will wait until either a char received or last char has been sent.
MSDN about WaitCommEvent:
If hFile was not opened with FILE_FLAG_OVERLAPPED, WaitCommEvent does not return until one of the specified events or an error occurs.
I come to you because I have some problems when I trying to write a driver supporting the QL355P from TTI. I already wrote the driver in VB6, but now, I am trying to write it in C++. I use the ReadFile/WriteFile function to communicate with the interface. The problem is when I send the identification command (*IDN?) and the endchar (.), I have no response (I have Statut_timeout).
My code is:
DCB l_dcb;
BOOL l_Success;
COMMTIMEOUTS l_TimeOuts;
const char *command = "*IDN?.";
// open an access to serial port
m_ComPort = CreateFile(L"COM2",
GENERIC_READ | GENERIC_WRITE,
0, // comm devices must be opened w/exclusive-access
NULL, // no security attrs
OPEN_EXISTING, // comm devices must use OPEN_EXISTING
0, // not overlapped I/O
NULL // hTemplate must be NULL for comm devices
);
// get the current configuration
l_Success = GetCommState(m_ComPort, &l_dcb);
// fill in the DCB with configuation:
// baud=19200, 8 data bits, no parity, 1 stop bit
l_dcb.BaudRate = 19200;
l_dcb.ByteSize = 8;
l_dcb.Parity = NOPARITY;
l_dcb.StopBits = ONESTOPBIT;
// configure the port
l_Success = SetCommState(m_ComPort, &l_dcb);
// get the current time-outs configuration
l_Success = GetCommTimeouts(m_ComPort, &l_TimeOuts);
// change the time-outs
l_TimeOuts.ReadIntervalTimeout=50;
l_TimeOuts.ReadTotalTimeoutMultiplier=50;
l_TimeOuts.ReadTotalTimeoutConstant=1000;
// sets the time-outs configuration
l_Success = SetCommTimeouts(m_ComPort, &l_TimeOuts);
// first purge the input buffer
PurgeComm(m_ComPort,PURGE_RXCLEAR);
// send the command
DWORD l_NumBytesTransfered=0;
DWORD l_NumBytesToTransfer=strlen(command);
l_Success = WriteFile(m_ComPort,command,l_NumBytesToTransfer,&l_NumBytesTransfered,NULL);
//Try to read
char l_Char='*';
int i=0;
char l_Pointer[100];
while(l_Char !=0x0a && i<100)
{
l_Success = ReadFile(m_ComPort,&l_Char,1,&l_NumBytesTransfered,NULL);
l_Pointer[i] = l_Char;
i++;
}
CloseHandle(m_ComPort); //close the handle
ReadFile can return 0 in case of a timeout, or various possible errors. Your code snippet ignores that possibility and puts garbage into the input array even when ReadFile fails.
I'm communicating between two processes on different machines via a pipe, using IO completion routines.
Occasionally, when the completion routine for WriteFileEx gets called, the completion routine parameter dwErrorCode is 0 (i.e. no error), GetOverlappedResult returns true (i.e. no error), but dwNumberOfBytesTransfered does not match nNumberOfBytesToWrite in the call to WriteFileEx. I only see this on the client end of the pipe however.
If the number of bytes transferred does not match the number of bytes that was requested to transfer, how can this be deemed a success?
This is how the client's handle to the pipe is created:
mHPipe = CreateFile(pipeName, // pipe name
GENERIC_READ | // read and write access
GENERIC_WRITE,
0, // no sharing
NULL, // default security attributes
OPEN_EXISTING, // opens existing pipe
FILE_FLAG_OVERLAPPED | // overlapped
FILE_FLAG_WRITE_THROUGH, // write through mode
NULL); // no template file
// do some checking...
// The pipe connected; change to message-read mode.
DWORD dwMode = PIPE_READMODE_MESSAGE;
BOOL fSuccess = SetNamedPipeHandleState(mHPipe, // pipe handle
&dwMode, // new pipe mode
NULL, // don't set maximum bytes
NULL); // don't set maximum time
Can anyone see why this would happen?
Thanks
EDIT:
The relevant WriteFileEx code is as follows:
void WINAPI CompletedWriteRoutine(DWORD dwErrorCode, DWORD dwNumberOfBytesTransfered, LPOVERLAPPED lpOverLap)
{
BOOL fWrite = FALSE;
LPPIPEINST lpPipeInst = (LPPIPEINST)lpOverLap;
//
// ! 99.9% of the time, dwNumberOfBytesTransfered == lpPipeInst->cbDataSize
// but 0.1% of the time, they do not match
//
// Some stuff
// Copy next message to send
memcpy_s(lpPipeInst->chData, sizeof(lpPipeInst->chData), pMsg->msg, pMsg->size);
lpPipeInst->cbDataSize = pMsg->size;
// Some other stuff
fWrite = WriteFileEx(lpPipeInst->hPipeInst,
lpPipeInst->chData,
lpPipeInst->cbDataSize,
(LPOVERLAPPED) lpPipeInst,
(LPOVERLAPPED_COMPLETION_ROUTINE)CompletedWriteRoutine);
// Some other, other stuff
}
Where LPPIPEINST is declared as:
typedef struct
{
OVERLAPPED oOverlap; // must remain first item
HANDLE hPipeInst;
TCHAR chData[BUFSIZE];
DWORD cbDataSize;
} PIPEINST, *LPPIPEINST;
And the initial call to CompletedWriteRoutine is given the lpOverlap parameter declared thusly:
PIPEINST pipeInstWrite = {0};
pipeInstWrite.hPipeInst = client.getPipeHandle();
pipeInstWrite.oOverlap.hEvent = hEvent[eventWriteComplete];
EDIT:
After trying re-initializing the overlapped structure as Harry suggested, I noticed something peculiar.
I memset the OVERLAPPED structure to zero before each WriteFileEx, and roughly 1/5000 completion routine callbacks, the cbWritten parameter and the OVERLAPPED structure's InternalHigh member was now set to the size of the previous message, instead of the most recent message. I added some logging to file on both the client and server ends of the pipe inside the completion routines, and the data sent and received at both ends was an exact match (and the correct, expected data). This then unveiled that in the time taken to write the data to a file, the InternalHigh member in the OVERLAPPED structure had changed to now reflect the size of the message I was expecting (cbWritten remains the old message size). I removed the file logging, and am now able to reproduce the issue like clockwork with this code:
void WINAPI CompletedWriteRoutine(DWORD dwErr, DWORD cbWritten, LPOVERLAPPED lpOverLap)
{
LPPIPEINST lpPipeInst = (LPPIPEINST)lpOverLap;
// Completion routine says it wrote the amount of data from the previous callback
if (cbWritten != lpPipeInst->cbDataSize)
{
// Roughly 1 in 5000 callbacks ends up in here
OVERLAPPED ovl1 = lpPipeInst->oOverlap; // Contains size of previous message, i.e. cbWritten
Sleep(100);
OVERLAPPED ovl2 = lpPipeInst->oOverlap; // Contains size of most recent message, i.e lpPipeInst->cbDataSize
}
...
}
It seems that sometimes, the completion routine is being called before the OVERLAPPED structure and the completion routine input parameter is updated. I'm using MsgWaitForMultipleObjectsEx(eventLast, hEvent, INFINITE, QS_POSTMESSAGE, MWMO_ALERTABLE); for the completion routines to be called on Windows 7 64 bit.
This MSDN page says:
"The system does not use the OVERLAPPED structure after the completion routine is called, so the completion routine can deallocate the memory used by the overlapped structure."
...so apparently, what this code can reproduce should never happen?
Is this a WINAPI bug?
Added FILE_FLAG_NO_BUFFERING to the CreateFile call - haven't seen the problem since. Thanks everyone who commented for your time.
I want to check if the WriteFile function is done writing to UART so that i can call ReadFile on the same ComDev without causing an Exception.
It seems the WriteFile function can return before writing is done.
BOOL WriteCommBlock(HANDLE * pComDev, char *pBuffer , int BytesToWrite)
{
while(fComPortInUse){}
fComPortInUse = 1;
BOOL bWriteStat = 0;
DWORD BytesWritten = 0;
COMSTAT ComStat = {0};
OVERLAPPED osWrite = {0,0,0};
if(WriteFile(*pComDev,pBuffer,BytesToWrite,&BytesWritten,&osWrite) == FALSE)
{
short Errorcode = GetLastError();
if( Errorcode != ERROR_IO_PENDING )
short breakpoint = 5; // Error
Sleep(1000); // complete write operation TBD
fComPortInUse = 0;
return (FALSE);
}
fComPortInUse = 0;
return (TRUE);
}
I used Sleep(1000) as an workaround, but how can i wait for an appropriate time?
You can create a Event, store it in your overlapped structure and wait for it to be signalled. Like this (untested):
BOOL WriteCommBlock(HANDLE * pComDev, char *pBuffer , int BytesToWrite)
{
while(fComPortInUse){}
fComPortInUse = 1;
BOOL bWriteStat = 0;
DWORD BytesWritten = 0;
COMSTAT ComStat = {0};
OVERLAPPED osWrite = {0,0,0};
HANDLE hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
if (hEvent != NULL)
{
osWrite.hEvent = hEvent;
if(WriteFile(*pComDev,pBuffer,BytesToWrite,&BytesWritten,&osWrite) == FALSE)
{
short Errorcode = GetLastError();
if( Errorcode != ERROR_IO_PENDING )
short breakpoint = 5; // Error
WaitForSingleObject(hEvent, INFINITE);
fComPortInUse = 0;
return (FALSE);
}
CloseHandle(hEvent);
}
fComPortInUse = 0;
return (TRUE);
}
Note that depending on what else you are trying to do simply calling WaitForSingleObject() might not be the best idea. And neither might an INFINITE timeout.
Your problem is the incorrect use of the overlapped I/O, regardless to the UART or whatever underlying device.
The easiest (though not necessarily the most optimal) way to fix your code is to use an event to handle the I/O completion.
// ...
OVERLAPPED osWrite = {0,0,0};
osWrite.hEvent = CreateEvent(FALSE, NULL, NULL, FALSE);
if(WriteFile(*pComDev,pBuffer,BytesToWrite,&BytesWritten,&osWrite) == FALSE)
{
DWORD Errorcode = GetLastError();
// ensure it's ERROR_IO_PENDING
WaitForSingleObject(osWrite.hEvent, INFINITE);
}
CloseHandle(osWrite.hEvent);
Note however that the whole I/O is synchronous. It's handles by the OS in an asynchronous way, however your code doesn't go on until it's finished. If so, why do you use the overlapped I/O anyway?
One should use it to enable simultaneous processing of several I/Os (and other tasks) within the same thread. To do this correctly - you should allocate the OVERLAPPED structure on heap and use one of the available completion mechanisms: event, APC, completion port or etc. Your program flow logic should also be changed.
Since you didn't say that you need asynchronous I/O, you should try synchronous. It's easier. I think if you just pass a null pointer for the OVERLAPPED arg you get synchronous, blocking, I/O. Please see the example code I wrote in the "Windows C" section of this document:
http://www.pololu.com/docs/0J40/
Your Sleep(1000); is of no use, it will only execute after the writefile completes its operation.You have to wait till WriteFile is over.
if(WriteFile(*pComDev,pBuffer,BytesToWrite,&BytesWritten,&osWrite) == FALSE)
{}
You must be knowing that anything inside conditionals will only execute if the result is true.
And here the result is sent to the program after completion(whether complete or with error) of WriteFile routine.
OK, I missed the overlapped I/O OVL parameter in the read/write code, so It's just as well I only replied yesterday as a comment else I would be hammered with downvotes:(
The classic way of handling overlapped I/O is to have an _OVL struct as a data member of the buffer class that is issued in the overlapped read/write call. This makes it easy to have read and write calls loaded in at the same time, (or indeed, multiple read/write calls with separate buffer instances).
For COM posrts, I usually use an APC completion routine whose address is passed in the readFileEx/writeFileEx APIs. This leaves the hEvent field of the _OVL free to use to hold the instance pointer of the buffer so it's easy to cast it back inside the completion routine, (this means that each buffer class instance contains an _OVL memebr that contains an hEvent field that points to the buffer class instance - sounds a but weird, but works fine).