Losing data with GetOverlappedResult? - c++

I have a thread that constantly looks for new data and if the data is not already in the serial buffer, ReadFile and GetOverlappedResult seem to tell me there's data, and that it read it, but not transfer it into my buffer...
func read()
{
if(state == 0)
{
memset(bytes, '\0', sizeof(amount_to_read));
readreturn = ReadFile(h, bytes, amount_to_read,NULL, osReader);
if(readreturn <= 0)
{
errorcode = GetLastError();
if(errorcode != ERROR_IO_PENDING)
{
SetEAIError(ERROR_INTERNALERROR);
return -1;
}
}
}
if (GetOverlappedResult(h, osReader, &dwRead, FALSE) == false)
{
errorcode = GetLastError();
if (errorcode == ERROR_IO_INCOMPLETE || errorcode == 0)
{
if(dwRead > 0)
{
return 1;
}
//timeout
SetEAIError(ERROR_EAITIMEOUT);
return -1;
}
else
{
//other error
SetEAIError(ERROR_WIN_ERROR);
return -1;
}
}
else
{
//read succeded, check if we read the amount required
if(dwRead != amount_to_read)
{
if(dwRead == 0)
{
//nothing read, treat as timeout
SetEAIError(ERROR_EAINOREAD);
return -1;
}
else
{
//memcpy_s(bytes, sizeof(bytes), readbuf, dwRead);
SetEAIError(ERROR_PARTIALREAD);
*_bytesRead = dwRead;
return -1;
}
}
else
{
if(strlen((char*)bytes) == 0)
{
//nothing read, treat as timeout
SetEAIError(ERROR_EAINOREAD);
return -1;
}
//memcpy_s(bytes, sizeof(bytes), readbuf, dwRead);
*_bytesRead = dwRead;
return 0;
}
}
}
This is what the error codes mean:
ERROR_TIMEOUT - switches the state to 1 so that it does not read again, which calls GetOverlappedResult again
INTERNALERROR,ERROR_EAINOREAD - it resets state to 0
ERROR_PARTIALREAD - starts a new read with new amount of bytes to read
If I swtich GetOverlappedResult to blocking (pass TRUE) it works every time.
If I switch my thread to only read when I know there is data there it works every time.
But if there is not data there, when there is data there it seems to "lose" the data, it my read amount parameter dwRead shows the correct number of bytes read (can see it read with a port monitor) but the bytes are not stored in my char*.
I constantly get ERROR_EAINOREAD
What am I doing wrong?
I do not want to use flags, I want to just use ReadFile and GetOverlappedResult, I should be able to accomplish this with the code I have....... I assume

The problem was exactly what was stated the data was getting lost... the REASON it was getting lost is because the bytes parameter passed into the readfile is a local variable in the parents thread. being local it gets re initialized each cycle so after I come into the read again, skip the readfile and go to the overlappedresults, I am now potentially working with a different area of memory

Related

Get loaded module from another process

I want to get all modules from another process. But it returns absurd values. Program stay in do-while loop for one time. After that exits from do-while loop.
I can't find where is the mistake - how can I fix this? I know that program must be in do-while loop for several time but it don't.
NTSTATUS Status;
PROCESS_BASIC_INFORMATION pbi;
ULONG ReturnLength;
Status = NtQueryInformationProcess(
INJECTOR_INFO.process.processHandle,
ProcessBasicInformation,
&pbi,
sizeof(PROCESS_BASIC_INFORMATION),
&ReturnLength);
if (!NT_SUCCESS(Status)) {
printf("NtQueryInformationProcess failed.(pbi)\n");
return;
}
else {
PLIST_ENTRY HeadEntry = pbi.PebBaseAddress->LoaderData->InMemoryOrderModuleList.Flink;
PLIST_ENTRY nextEntry = pbi.PebBaseAddress->LoaderData->InMemoryOrderModuleList.Blink;
DWORD dwBytesRead = 0;
PLDR_MODULE pLdrModule = nullptr;
LDR_MODULE LdrModule;
do
{
LDR_DATA_TABLE_ENTRY LdrEntry;
PLDR_DATA_TABLE_ENTRY Base = CONTAINING_RECORD(HeadEntry, LDR_DATA_TABLE_ENTRY, InMemoryOrderLinks);
if (NT_SUCCESS(Status = NtReadVirtualMemory(INJECTOR_INFO.process.processHandle, Base, &LdrEntry, sizeof(LdrEntry), &dwBytesRead)))
{
if (dwBytesRead != sizeof(LdrEntry)) {
printf("length doesn't match");
return;
}
char* pLdrModuleOffset = reinterpret_cast<char*>(HeadEntry) - sizeof(LIST_ENTRY);
if (!NT_SUCCESS(Status = NtReadVirtualMemory(INJECTOR_INFO.process.processHandle, pLdrModuleOffset, &pLdrModule, sizeof(pLdrModule), &dwBytesRead))) {
printf("pLdrModuleOffset doesn't read"); return;
}else if (dwBytesRead != sizeof(pLdrModule)) { printf("pLdrModule length doesn't match"); return; }
if (!NT_SUCCESS(Status = NtReadVirtualMemory(INJECTOR_INFO.process.processHandle, pLdrModule, &LdrModule, sizeof(LdrModule), &dwBytesRead))) {
printf("pLdrModule doesn't read"); return;
}else if (dwBytesRead != sizeof(LdrModule)) { printf("LdrModule length doesn't match"); return; }
if (LdrEntry.DllBase)
{
printf("BaseAddress: %p\n", LdrModule.BaseAddress);
printf("Reference Count: %d\n", LdrModule.LoadCount);
}
HeadEntry = LdrEntry.InMemoryOrderLinks.Flink;
}
else { printf("LDR_DATA_TABLE_ENTRY doesn't read"); return; }
} while (HeadEntry != nextEntry);
}
I put breakpoint on !NT_SUCCESS(Status) after NtQueryInformationProcess for values of variables:
Values after NtQueryInformationProcess
Another breakpoint for values of variables in do while for the end of the first cycle:
Values for the end of the first cycle
You are querying a remote process for it's PROCESS_BASIC_INFORMATION, but then you proceed to follow the pointers in your own process:
PLIST_ENTRY HeadEntry = pbi.PebBaseAddress->LoaderData->InMemoryOrderModuleList.Flink;
For this to work, read the PEB from the remote process (from pbi.PebBaseAddress),
then read the LoaderData from the remote process (PEB.LoaderData).
Then, follow the InMemoryOrderModuleList (again, read the data from the remote process).
At this point you can iterate over the entire list, by reading each entry from the remote process.

multithread list shared performance

I am developing an application that reads data from a named pipe on Windows 7 at around 800 Mbps. I have to develop it with several threads since the FIFO at the other side of the pipe overflows if I am not able to read at the given speed. The performance though is really pitifull and I cannot understand why. I already read several things I tried to split the memory to avoid bad memory sharing.
At the beginning I has thinking I could be a problem with contiguous memory possitions, but the memory sections are queued in a list the main thread is not using them any more after queue it. The amount of memory are huge so I don't thing they lay on same pages or so.
This is the threaded function:
void splitMessage(){
char* bufferMSEO;
char* bufferMDO;
std::list<struct msgBufferStr*> localBufferList;
while(1)
{
long bytesProcessed = 0;
{
std::unique_lock<std::mutex> lk(bufferMutex);
while(bufferList.empty())
{
// Wait until the map has data
listReady.wait(lk);
}
//Extract the data from the list and copy to the local list
localBufferList.splice(localBufferList.end(),bufferList);
//Unlock the mutex and notify
// Manual unlocking is done before notifying, to avoid waking up
// the waiting thread only to block again (see notify_one for details)
lk.unlock();
//listReady.notify_one();
}
for(auto nextBuffer = localBufferList.begin(); nextBuffer != localBufferList.end(); nextBuffer++)
{
//nextBuffer = it->second();
bufferMDO = (*nextBuffer)->MDO;
bufferMSEO = (*nextBuffer)->MSEO;
bytesProcessed += (*nextBuffer)->size;
//Process the data Stream
for(int k=0; k<(*nextBuffer)->size; k++)
{
}
//localBufferList.remove(*nextBuffer);
free(bufferMDO);
free(bufferMSEO);
free(*nextBuffer);
}
localBufferList.clear();
}
}
And here the thread that reads the data and queue them:
DWORD WINAPI InstanceThread(LPVOID lpvParam)
// This routine is a thread processing function to read from and reply to a client
// via the open pipe connection passed from the main loop. Note this allows
// the main loop to continue executing, potentially creating more threads of
// of this procedure to run concurrently, depending on the number of incoming
// client connections.
{
HANDLE hHeap = GetProcessHeap();
TCHAR* pchRequest = (TCHAR*)HeapAlloc(hHeap, 0, BUFSIZE*sizeof(TCHAR));
DWORD cbBytesRead = 0, cbReplyBytes = 0, cbWritten = 0;
BOOL fSuccess = FALSE;
HANDLE hPipe = NULL;
double totalRxData = 0;
char* bufferPnt;
char* bufferMDO;
char* bufferMSEO;
char* destPnt;
// Do some extra error checking since the app will keep running even if this
// thread fails.
if (lpvParam == NULL)
{
printf( "\nERROR - Pipe Server Failure:\n");
printf( " InstanceThread got an unexpected NULL value in lpvParam.\n");
printf( " InstanceThread exitting.\n");
if (pchRequest != NULL) HeapFree(hHeap, 0, pchRequest);
return (DWORD)-1;
}
if (pchRequest == NULL)
{
printf( "\nERROR - Pipe Server Failure:\n");
printf( " InstanceThread got an unexpected NULL heap allocation.\n");
printf( " InstanceThread exitting.\n");
return (DWORD)-1;
}
// Print verbose messages. In production code, this should be for debugging only.
printf("InstanceThread created, receiving and processing messages.\n");
// The thread's parameter is a handle to a pipe object instance.
hPipe = (HANDLE) lpvParam;
try
{
msgSplitter = std::thread(&splitMessage);
//msgSplitter.detach();
}
catch(...)
{
_tprintf(TEXT("CreateThread failed, GLE=%d.\n"), GetLastError());
return -1;
}
while (1)
{
struct msgBufferStr *newBuffer = (struct msgBufferStr* )malloc(sizeof(struct msgBufferStr));
// Read client requests from the pipe. This simplistic code only allows messages
// up to BUFSIZE characters in length.
fSuccess = ReadFile(
hPipe, // handle to pipe
pchRequest, // buffer to receive data
BUFSIZE*sizeof(TCHAR), // size of buffer
&cbBytesRead, // number of bytes read
NULL); // not overlapped I/O
if (!fSuccess || cbBytesRead == 0)
{
if (GetLastError() == ERROR_BROKEN_PIPE)
{
_tprintf(TEXT("InstanceThread: client disconnected.\n"), GetLastError());
break;
}
else if (GetLastError() == ERROR_MORE_DATA)
{
}
else
{
_tprintf(TEXT("InstanceThread ReadFile failed, GLE=%d.\n"), GetLastError());
}
}
//timeStart = omp_get_wtime();
bufferPnt = (char*)pchRequest;
totalRxData += ((double)cbBytesRead)/1000000;
bufferMDO = (char*) malloc(cbBytesRead);
bufferMSEO = (char*) malloc(cbBytesRead/3);
destPnt = bufferMDO;
//#pragma omp parallel for
for(int i = 0; i < cbBytesRead/12; i++)
{
msgCounter++;
if(*(bufferPnt + (i * 12)) == 0) continue;
if(*(bufferPnt + (i * 12)) == 8)
{
errorCounter++;
continue;
}
//Use 64 bits variables in order to make less operations
unsigned long long *sourceAddrLong = (unsigned long long*) (bufferPnt + (i * 12));
unsigned long long *destPntLong = (unsigned long long*) (destPnt + (i * 8));
//Copy the data bytes from source to destination
*destPntLong = *sourceAddrLong;
//Copy and prepare the MSEO lines for the data processing
bufferMSEO[i*4]=(bufferPnt[(i * 12) + 8] & 0x03);
bufferMSEO[i*4 + 1]=(bufferPnt[(i * 12) + 8] & 0x0C) >> 2;
bufferMSEO[i*4 + 2]=(bufferPnt[(i * 12) + 8] & 0x30) >> 4;
bufferMSEO[i*4 + 3]=(bufferPnt[(i * 12) + 8] & 0xC0) >> 6;
}
newBuffer->size = cbBytesRead/3;
newBuffer->MDO = bufferMDO;
newBuffer->MSEO = bufferMSEO;
{
//lock the mutex
std::lock_guard<std::mutex> lk(bufferMutex);
//add data to the list
bufferList.push_back(newBuffer);
} // bufferMutex is automatically released when lk goes out of scope
//Notify
listReady.notify_one();
}
// Flush the pipe to allow the client to read the pipe's contents
// before disconnecting. Then disconnect the pipe, and close the
// handle to this pipe instance.
FlushFileBuffers(hPipe);
DisconnectNamedPipe(hPipe);
CloseHandle(hPipe);
HeapFree(hHeap, 0, pchRequest);
//Show memory leak isues
_CrtDumpMemoryLeaks();
//TODO: Join thread
printf("InstanceThread exitting.\n");
return 1;
}
The think that really blows my mind is that I a let it like this the splitMessage thread takes minutes to read the data even though the first thread finished reading the data long ago. I mean the read thread reads like 1,5Gb or information in seconds and waits for more data from the pipe. This data are processed by the split thread (the only one really "doing" something in almost one minute or more). The CPU is moreover only to less than 20% percent used. (It is a i7 labtop with 16 Gb RAM and 8 cores!)
On the other hand, if I just comment the for loop in the process thread:
for(int k=0; k<(*nextBuffer)->size; k++)
Then the data are read slowly and the FIFO on the other side of the pipe overflows. With 8 processors and at more than 2 GHz should be fast enought to go throw the buffers without many problems, isn't it? I think it has to be a memory access issue or that the scheduler is sending the thread somehow to sleep but I cannot figure out why!!. Other possibility is that the iteration throw the linked list with the iterator is not optimal.
Any help would be geat because I am trying to understand it since a couple of days, I made several changes in the code and tried to simplified at the maximum and I am getting crazy :).
best regards,
Manuel

SSL_connect() not working with select()

I have a non blocking Socket and I'm using openssl for creating a ssl connection. My problem is the way SSL_connect works. It Returns SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE in these cases SSL_connect has to be called again. Now this is actually working. The issue I have is, that the select() which should make the program wait for the socket to be ready is not working. This results in over 2000 calls to SSL_connect which is not nice.
Here is my code for this Part of the Program
int test = 0;
while (test != 1) {
test = SSL_connect(sslHandle);
if (test != 1) { //test again or we would quit if we made a connection
if (SSL_get_error(sslHandle, test) != SSL_ERROR_WANT_READ && SSL_get_error(sslHandle, test) != SSL_ERROR_WANT_WRITE) {
//error happend
return -1;
}
FD_ZERO(&socketSet); //reset socketSet
FD_SET(socket, &socketSet); //init socketSet with our socket
if (test == SSL_ERROR_WANT_READ) {
result = select(socket + 1, &socketSet, NULL, NULL, &timeoutCopy); //wait for the socket to be readable
if (result == 0) {
//timeout
return -1;
} else if (result == -1) {
//error
return -1;
}
} else {
result = select(socket + 1, NULL, &socketSet, NULL, &timeoutCopy); //wait for the socket to be writable
if (result == 0) {
//timeout
return -1;
} else if (result == -1) {
//error
return -1;
}
}
}
}
I Removed all the error handling stuff and logs here so it is better readable. timeoutCopy is defined elsewhere. I'm using the same kind of selects for other parts of the program there it is working just fine.
As I said the program is working, but just not waiting at the select. I could just sleep, but that is not a nice solution. Thanks in advance!
Ok I'm a dummy.
Issue is in line
if (test == SSL_ERROR_WANT_READ)
This has to be
if (SSL_get_error(sslHandle, sslConnectResult) == SSL_ERROR_WANT_READ)
Than the code works just fine. It iterates 4 to 5 times and than has the connection.

IOCP and overwritten buffer

Well i make a IOCP for handling client connections with the following details:
- Threads = (CPU cores * 2)
- Assigning an completion port to each socket
- Accessing the socket context by Client Index or overlapped struct (either way is the same)
So i am trying to debug the incoming packets, its works like a charm, except for a little but nasty detail... I set a break point on WorkersThread function (where i recv the packet) i am watching the buffer with the packet i recv, when suddenly the buffer gets overwritten with a new packet that i got from client.
Why is that? according to what i read, IOCP should wait till i process the packet, send a response to client before recv any other packet. So i set a flag on my socket context called "Processing" and still got the overwritten buffer with an incoming packet. So it doesn't let me debug at all and its driving me crazy
Is ollydbg (debugger) fault that let the other threads running while i set a break point? Or is some error in my IOCP implementation?
Here is how my WorkerThread is coded:
DWORD WINAPI WorkerThread(void* argument)
{
int BytesTransfer;
int BytesRecv;
int ClientID;
int result;
OVERLAPPED* overlapped = 0;
ClientInfo* clientinfo = 0;
WSABUF wsabuf;
int flags;
//Exit only when shutdown signal is recv
while (WaitForSingleObject(IOCPBase::internaldata->sockcontext.ShutDownSignal, NULL) != WAIT_OBJECT_0)
{
flags = 0; BytesTransfer = 0; BytesRecv = 0; ClientID = 0;
//Get from queued list
if (GetQueuedCompletionStatus(IOCPBase::internaldata->sockcontext.CompletionPort, (LPDWORD)&BytesTransfer, (PULONG_PTR)&ClientID, &overlapped, INFINITE) == TRUE)
{
if (overlapped == 0)
{
//Fatal error
break;
}
clientinfo = (ClientInfo*)overlapped;
if (BytesTransfer != 0)
{
//Assign the buffer pointer and buffer len to WSABUF local
clientinfo->RecvContext.RecvBytes = BytesTransfer;
wsabuf.buf = (char*)clientinfo->RecvContext.Buffer;
wsabuf.len = clientinfo->RecvContext.Len;
//Switch for OperationCode
//switch (IOCPBase::internaldata->ClientContext[ClientID].OperationCode)
switch (clientinfo->OperationCode)
{
case FD_READ:
// Check if we have send all data to the client from a previous send
if (clientinfo->SendContext.SendBytes < clientinfo->SendContext.TotalBytes)
{
clientinfo->OperationCode = FD_READ; //We set FD_READ caused on the next send, there could still be bytes left to send
wsabuf.buf += clientinfo->SendContext.SendBytes; //The buffer position is + sended bytes
wsabuf.len = clientinfo->SendContext.TotalBytes - clientinfo->SendContext.SendBytes; //the buffer len is total - sended bytes
//Send the remain bytes
result = WSASend(clientinfo->sock, &wsabuf, 1, (LPDWORD)&BytesRecv, flags, &clientinfo->overlapped, NULL);
if (result == SOCKET_ERROR && (WSAGetLastError() != WSA_IO_PENDING))
{
CloseClient(ClientID);
}
clientinfo->SendContext.SendBytes += BytesRecv;
}
else
{
if (clientinfo->Processing == 0)
{
clientinfo->OperationCode = FD_WRITE; //If no more bytes left to send now we can set the operation code to write (in fact is read)
memset(clientinfo->RecvContext.Buffer, NULL, MAX_DATA_BUFFER_SIZE); //Clean the buffer for recv new data
//Recv data from our client
clientinfo->RecvContext.RecvBytes = WSARecv(clientinfo->sock, &wsabuf, 1, (LPDWORD)&BytesRecv, (LPDWORD)&flags, &clientinfo->overlapped, NULL);
if (clientinfo->RecvContext.RecvBytes == SOCKET_ERROR && WSAGetLastError() != WSA_IO_PENDING)
{
CloseClient(ClientID);
break;
}
}
}
break;
case FD_WRITE:
//Send data to the RecvProtocol
clientinfo->Processing = 1;
IOCPBase::internaldata->callback.RecvProtocol(clientinfo->RecvContext.Buffer, clientinfo->RecvContext.Len, ClientID);
clientinfo->Processing = 0;
default:
break;
}
}
}
}
return false;
}
The problem appears when looking at clientinfo->RecvContext.Buffer. I am watching the packet, past a few seconds and boom the buffer is overwritten with a new packet.
Thanks !
Never mind i fix the debug problem by copy the packet to the stack frame of the function i use to analyze the packet, this way i have no overwritten problem.

What might cause an infinite loop error

I am working on a network programming and I have this code
void WorkHandler::workLoop(){
.
.
.
while(1){
if(remainLength >= MAX_LENGTH)
currentSentLength = send(client->getFd(), sBuffer, MAX_LENGTH, MSG_NOSIGNAL);
else
currentSentLength = send(client->getFd(), sBuffer, remainLength,MSG_NOSIGNAL);
if(currentSentLength == -1){
log("WorkHandler::workLoop, connection has been lost \n");
break;
}
sBuffer += currentSentLength;
remainLength -= currentSentLength;
if(remainLength == 0)
break;
}
}
Also, I am creating a child thread like this
bool WorkHandler::initThreads(){
for(int i=0; i < m_maxThreads; i++){
pthread_t *thread(new pthread_t);
m_workThreadList.push_back(thread);
if(pthread_create(thread, NULL, runWorkThread, reinterpret_cast<void *>(this))!=0){
log("WorkHandler::initThreads, pthread_create error \n");
return false;
}
pthread_detach(*thread);
}
return true;
}
void* WorkHandler::runWorkThread(void *delegate){
printf("WorkHandler::runWorkThread, called\n");
WorkHandler *ptr = reinterpret_cast<WorkHandler*>(delegate);
ptr->workLoop();
return NULL;
}
I am running this code on gdb and it doesn't blow up but it gets stuck at the second send function in the if then else loop. I put log statements every single line and it prints a log right above the second send function and stopped.
currentSentLength = send(client->getFd(), sBuffer, remainLength, MSG_NOSIGNAL);
What might cause this problem and how do I fix this issue?
Thanks in advance..
With blocking IO send will block if the kernel buffer is full and will block untill the clients have read the data. Do you send large chunks? If so, check your client.
If you don't trust clients (they can abuse this to do denial of service attacks) there are a couple of ways to do this properly: poll (with timeout) on the sockets for writeability, send with timeout, use nonblocking I/O, ...
I guess you're calling send() with a negative size...
Your test to exit the while should be
remainLength <= 0
and not
remainLength == 0