Properly use WinInet in parallel thread - c++

I am writing a user-interfaced program that parses Google results page for finding stuff from shopping sites, in particularly, from Amazon.
Because it has user interface, it, obviously should make all internet actions in parallel thread, or the program will be hanged.
I am not actually good in multithreading, and obviously do it not correctly, in example:
HINTERNET internet;
int GetRequest(wchar_t* url, char *buffer, int bufersize)
{
int read = 0;
if (internet)
{
DWORD dwBytesRead;
HINTERNET hu = InternetOpenUrl(internet, url, L"Content-Type: application/x-www-form-urlencoded\r\n", wcslen(L"Content-Type: application/x-www-form-urlencoded\r\n"), INTERNET_FLAG_KEEP_CONNECTION, 0);
if (hu)
{
int err = GetLastError();
for (;;)
{
char b[1024] = {};
BOOL bRead = InternetReadFile(hu, buffer + read, sizeof(b), &dwBytesRead);
read += dwBytesRead;
if (dwBytesRead == 0)
break;
}
}
else
read = -1;
InternetCloseHandle(hu);
}
else
read = -2;
return read;
}
Usage
int status;
void run();
int main()
{
internet = InternetOpen(L"My user agent", INTERNET_OPEN_TYPE_PRECONFIG, 0, 0, 0);
std::thread t(run);
t.detach();
while(status==0){}
if(status==1)
{
char *response = new char[1024*1024*2];
MessageBoxA(NULL, response,"Aga",MB_OK);
}
else
MessageBoxA(NULL, "Error","Aga",MB_OK);
delete[] response;
status = 0;
return 0;
}
void run()
{
char *buffer = new char[1024*1024*2]; // 2mb buffer
int read = GetRequest(L"https://www.google.com/search?q=lolkek", buffer, 1024*1024*2); // or www.search.yahoo.com
strcat_s(response,1024*1024*2,buffer);
delete[] buffer;
if(read>=0)
status = 1;
else
status = -1;
}
Actually it works perfectly. The program does what I want.
But I faced an issue: when the program is running, I open Chrome, and some buttons in Amazon’s site doesn’t work. In particularly, sell button in product page - by clicking on it, nothing is happens, closing my program and now that button opens login page. Also, seems it breaks data in some google sheets, even, when program works only with Yahoo.
Program does not access amazon.com or any shopping site directly. It has been connecting to google.com, but now even only to search.yahoo.com.
Has anyone faced something like that? Is the problem in not correct multithreading? Maybe I should change internet flags(not exactly to asynchronous, but, maybe add INTERNET_FLAG_NO_CACHE_WRITE or something like that?
I have one assumption: it could happened because three instance of program was run at one time. Also the tabs with Google Sheets that was corrupted was opened in Chrome, when the program was running.

Related

What is the preferred way to get a device path for CreateFile() in a UWP C++ App?

I am converting a project to a UWP App, and thus have been following guidelines outlined in the MSDN post here. The existing project heavily relies on CreateFile() to communicate with connected devices.
There are many posts in SO that show us how to get a CreateFile()-accepted device path using SetupAPI's SetupDiGetDeviceInterfaceDetail() Is there an alternative way to do this using the PnP Configuration Manager API? Or an alternative, user-mode way at all?
I had some hope when I saw this example in Windows Driver Samples github, but quickly became dismayed when I saw that the function they used in the sample is ironically not intended for developer use, as noted in this MSDN page.
function GetDevicePath in general correct and can be used as is. about difference between CM_*(..) and CM_*_Ex(.., HMACHINE hMachine) - the CM_*(..) simply call CM_*_Ex(.., NULL) - so for local computer versions with and without _Ex suffix the same.
about concrete GetDevicePath code - call CM_Get_Device_Interface_List_Size and than CM_Get_Device_Interface_List only once not 100% correct - because between this two calls new device with this interface can be arrived to system and buffer size returned by CM_Get_Device_Interface_List_Size can be already not enough for CM_Get_Device_Interface_List. of course possibility of this very low, and you can ignore this. but i prefer make code maximum theoretical correct and call this in loop, until we not receive error other than CR_BUFFER_SMALL. also need understand that CM_Get_Device_Interface_List return multiple, NULL-terminated Unicode strings - so we need iterate here. in [example] always used only first returned symbolic link name of an interface instance. but it can be more than 1 or at all - 0 (empty). so better name function - GetDevicePaths - note s at the end. i be use code like this:
ULONG GetDevicePaths(LPGUID InterfaceClassGuid, PWSTR* pbuf)
{
CONFIGRET err;
ULONG len = 1024;//first try with some reasonable buffer size, without call *_List_SizeW
for(PWSTR buf;;)
{
if (!(buf = (PWSTR)LocalAlloc(0, len * sizeof(WCHAR))))
{
return ERROR_NO_SYSTEM_RESOURCES;
}
switch (err = CM_Get_Device_Interface_ListW(InterfaceClassGuid, 0, buf, len, CM_GET_DEVICE_INTERFACE_LIST_PRESENT))
{
case CR_BUFFER_SMALL:
err = CM_Get_Device_Interface_List_SizeW(&len, InterfaceClassGuid, 0, CM_GET_DEVICE_INTERFACE_LIST_PRESENT);
default:
LocalFree(buf);
if (err)
{
return CM_MapCrToWin32Err(err, ERROR_UNIDENTIFIED_ERROR);
}
continue;
case CR_SUCCESS:
*pbuf = buf;
return NOERROR;
}
}
}
and usage example:
void example()
{
PWSTR buf, sz;
if (NOERROR == GetDevicePaths((GUID*)&GUID_DEVINTERFACE_VOLUME, &buf))
{
sz = buf;
while (*sz)
{
DbgPrint("%S\n", sz);
HANDLE hFile = CreateFile(sz, FILE_GENERIC_READ, FILE_SHARE_VALID_FLAGS, 0, OPEN_EXISTING, 0, 0);
if (hFile != INVALID_HANDLE_VALUE)
{
// do something
CloseHandle(hFile);
}
sz += 1 + wcslen(sz);
}
LocalFree(buf);
}
}
so we must not simply use in returned DevicePathS (sz) only first string, but iterate it
while (*sz)
{
// use sz
sz += 1 + wcslen(sz);
}
I got a valid Device Path to a USB Hub Device, and used it successfully to get various device descriptors by sending some IOCTLs, by using the function I posted in my own answer to another question
I'm reporting the same function below:
This function returns a list of NULL-terminated Device Paths (that's what we get from CM_Get_Device_Interface_List())
You need to pass it the DEVINST, and the wanted interface GUID.
Since both the DEVINST and interface GUID are specified, it is highly likely that CM_Get_Device_Interface_List() will return a single Device Path for that interface, but technically you should be prepared to get more than one result.
It is responsibility of the caller to delete[] the returned list if the function returns successfully (return code 0)
int GetDevInstInterfaces(DEVINST dev, LPGUID interfaceGUID, wchar_t**outIfaces, ULONG* outIfacesLen)
{
CONFIGRET cres;
if (!outIfaces)
return -1;
if (!outIfacesLen)
return -2;
// Get System Device ID
WCHAR sysDeviceID[256];
cres = CM_Get_Device_ID(dev, sysDeviceID, sizeof(sysDeviceID) / sizeof(sysDeviceID[0]), 0);
if (cres != CR_SUCCESS)
return -11;
// Get list size
ULONG ifaceListSize = 0;
cres = CM_Get_Device_Interface_List_Size(&ifaceListSize, interfaceGUID, sysDeviceID, CM_GET_DEVICE_INTERFACE_LIST_PRESENT);
if (cres != CR_SUCCESS)
return -12;
// Allocate memory for the list
wchar_t* ifaceList = new wchar_t[ifaceListSize];
// Populate the list
cres = CM_Get_Device_Interface_List(interfaceGUID, sysDeviceID, ifaceList, ifaceListSize, CM_GET_DEVICE_INTERFACE_LIST_PRESENT);
if (cres != CR_SUCCESS) {
delete[] ifaceList;
return -13;
}
// Return list
*outIfaces = ifaceList;
*outIfacesLen = ifaceListSize;
return 0;
}
Please note that, as RbMm already said in his answer, you may get a CR_BUFFER_SMALL error from the last CM_Get_Device_Interface_List() call, since the device list may have been changed in the time between the CM_Get_Device_Interface_List_Size() and CM_Get_Device_Interface_List() calls.

Reading on serial port returns what i just wrote

I just started a project where i'm struggling since days now about serial ports. I wrote a static library that can handle all the serial routine and give an interface with "readLine()" and "writeLine()" functions.
Everything works flawlessly on the write and read (which are threaded by the way) except if the slave does not anwser after he gets the data, then, the data is sent back to me, and i read it.
I open my fd with O_NDELAY and configure my read system call as Non blocking with fcntl.
here are the two threaded loops that work perfectly beside that.
void *Serial_Port::readLoop(void *param)
{
Serial_Port *sp = static_cast<Serial_Port*>(param);
std::string *line = NULL;
char buffer[128];
while (1)
{
line = new std::string();
while ((line->find("\r\n")) == std::string::npos)
{
usleep(100);
bzero(buffer, 128);
pthread_mutex_lock(sp->getRLock());
if (read(sp->getDescriptor(), buffer, 127) > 0)
*line += buffer;
pthread_mutex_unlock(sp->getRLock());
}
pthread_mutex_lock(sp->getRLock());
sp->getRStack()->push(line->substr(0, line->find("\r\n")));
pthread_mutex_unlock(sp->getRLock());
delete (line);
}
return (param);
}
void *Serial_Port::writeLoop(void *param)
{
Serial_Port *sp = static_cast<Serial_Port*>(param);
std::string *line;
while (1)
{
line = NULL;
pthread_mutex_lock(sp->getWLock());
if (!sp->getWStack()->empty())
{
line = new std::string(sp->getWStack()->front());
sp->getWStack()->pop();
}
pthread_mutex_unlock(sp->getWLock());
if (line != NULL)
{
pthread_mutex_lock(sp->getWLock());
write(sp->getDescriptor(), line->c_str(), line->length());
// fsync(sp->getDescriptor());
pthread_mutex_unlock(sp->getWLock());
}
usleep(100);
}
return (param);
}
I tried to flush the file descriptor, but i can't manage to receive any data after doing that. How can I get rid of that duplicate, needless data?
Thanks.
After multiple tests and behavior analysis, I discovered it was the "Pulsar3" (the device i was using on serial) that kept giving me back what i sent as "Acknowledge". Nice to know!

czmq asynchronous send/receive (PUB/SUB)

I'm using czmq for interprocess communication.
There are 2 processes :
The server, receiving requests and sending replies but also sending events.
The client, sending requests and receiving replies but also listening to the events.
I have already successfuly implemented the "request/reply" pattern with REQ/REP (details below)
Now I want to implement the notification mechanism.
I want my server to send its events without caring whether anyone receives them or not and without being blocked in anyway.
The client listens to those events but should it crash, it mustn't have any impact on the server.
I believe PUB/SUB is the most appropriate pattern, but if not do not hesitate to enlighten me.
Here's my implementation (cleaned from checks and logs) :
The server publishes the events
Server::eventIpcPublisher = zsock_new_pub("#ipc:///tmp/events.ipc");
void Server::OnEvent(uint8_t8* eventData, size_t dataSize) {
if (Server::eventIpcPublisher != nullptr) {
int retCode = zsock_send(Server::eventIpcPublisher, "b", eventData, dataSize);
}
The client listens to them in a dedicated thread
void Client::RegisterToEvents(const std::function<void(uint8_t*, size_t)>& callback) {
zsock_t* eventIpcSubscriber = zsock_new_sub(">ipc:///tmp/events.ipc", "");
listening = true;
while (listening) {
byte* receptionBuffer;
size_t receptionBufferSize;
int retCode = zsock_recv(eventIpcSubscriber, "b", &receptionBuffer, &receptionBufferSize);
--> NEVER REACHED <--
if (retCode == 0) {
callback(static_cast<uint8_t*>(receptionBuffer), receptionBufferSize);
}
}
zsock_destroy(&eventIpcSubscriber);
}
It doesn't work:
The server sends with return code 0, as if everything is ok,
The client doesn't receive anything (blocked on receive).
Help would be much appreciated, thanks in advance!
Chris.
PS: here is the REQ/REP that I have already implemented with success (no help needed here, just for comprehension)
The client sends a request and then waits for the answer.
uint8_t* MulticamApi::GetDatabase(size_t& sizeOfData) {
zsock_t* requestSocket = zsock_new_req(">ipc:///tmp/requests.ipc");
if (requestSocket == nullptr)
return nullptr;
byte* receptionBuffer;
size_t receptionBufferSize;
int retCode = zsock_send(requestSocket, "i", static_cast<int>(IpcComm_GetClipDbRequest));
if (retCode != 0) {
sizeOfData = 0;
return nullptr;
}
retCode = zsock_recv(requestSocket, "b", &receptionBuffer, &receptionBufferSize);
databaseData.reset(new MallocGuard(static_cast<void*>(receptionBuffer)));
sizeOfData = receptionBufferSize;
return static_cast<uint8_t*>(databaseData->Data());
}
A dedicated thread in the server listens to requests, processes them and replies. (don't worry, delete is handled somewhere else)
U32 Server::V_OnProcessing(U32 waitCode) {
protocolIpcWriter = zsock_new_rep("#ipc:///tmp/requests.ipc");
while (running) {
int receptionInt = 0;
int retCode = zsock_recv(protocolIpcWriter, "i", &receptionInt);
if ((retCode == 0) && (receptionInt == static_cast<int>(IpcComm_GetClipDbRequest))) {
GetDatabase();
}
sleep(1);
}
zsock_destroy(&protocolIpcWriter);
return 0;
}
void Server::GetDatabase() {
uint32_t dataSize = 10820 * 340;
uint8_t* data = new uint8_t[dataSize];
uint32_t nbBytesWritten = DbHelper::SaveDbToBuffer(data, dataSize);
int retCode = zsock_send(protocolIpcWriter, "b", data, nbBytesWritten);
}
I know my question's old but for the record, I switched from czmq to base zmq api and everything went smooth. A colleague of mine also had issues with the czmq layer and switched to zmq to fix them so that's definitely what I recommend.

why shutdown on udp socket blocks?

I'm writing a UDP server application for windows desktop/server.
My code uses the WSA API suggested by windows the following way (This is my simplified receivePacket method):
struct Packet
{
unsigned int size;
char buffer[MAX_SIZE(1024)];
}
bool receivePacket(Packet packet)
{
WSABUFFER wsa_buffer[2];
wsa_buffer[0].buf = &packet.size;
wsa_buffer[0].len = sizeof(packet.size);
wsa_buffer[1].buf = packet.buffer;
wsa_buffer[1].len = MAX_SIZE;
bool retval = false;
int flags = 0;
int recv_bytes = 0;
inet_addr client_addr;
int client_addr_len = sizeof(client_addr);
if(WSARecvFrom(_socket, wsa_buffer, sizeof(wsa_buffer)/sizeof(wsa_buffer[0]), &bytes_recv, &flags, (sockaddr *)&client_addr, &client_addr_len, NULL, NULL) == 0)
{
//Packet received successfully
}
else
{
//Report
}
}
Now, when I'm trying to close my application gracefully, not network-wise, but rather application-wise (going through all the d'tors and stuff), i'm trying to unblock this call.
To do this, I call the shutdown(_socket, SD_BOTH) method. Unfortunately, the call to shutdown itself BLOCKS!
After reading every possible page in the MSDN, I didn't find any reference to why this happens, other ways of attacking the problem or any way out.
Another thing I checked was using the SO_RCVTIMEO. Surprisingly, this sockopt didn't work as expected as well.
Is there any problem with my code/approach?
Did you run shutdown on duplicated handle? Shutdown on the same handle will wait any active operation on this handle to complete.

Multiple threads writing to same socket causing issues

I have written a client/server application where the server spawns multiple threads depending upon the request from client.
These threads are expected to send some data to the client(string).
The problem is, data gets overwritten on the client side. How do I tackle this issue ?
I have already read some other threads on similar issue but unable to find the exact solution.
Here is my client code to receive data.
while(1)
{
char buff[MAX_BUFF];
int bytes_read = read(sd,buff,MAX_BUFF);
if(bytes_read == 0)
{
break;
}
else if(bytes_read > 0)
{
if(buff[bytes_read-1]=='$')
{
buff[bytes_read-1]='\0';
cout<<buff;
}
else
{
cout<<buff;
}
}
}
Server Thread code :
void send_data(int sd,char *data)
{
write(sd,data,strlen(data));
cout<<data;
}
void *calcWordCount(void *arg)
{
tdata *tmp = (tdata *)arg;
string line = tmp->line;
string s = tmp->arg;
int sd = tmp->sd_c;
int line_no = tmp->line_no;
int startpos = 0;
int finds = 0;
while ((startpos = line.find(s, startpos)) != std::string::npos)
{
++finds;
startpos+=1;
pthread_mutex_lock(&myMux);
tcount++;
pthread_mutex_unlock(&myMux);
}
pthread_mutex_lock(&mapMux);
int t=wcount[s];
wcount[s]=t+finds;
pthread_mutex_unlock(&mapMux);
char buff[MAX_BUFF];
sprintf(buff,"%s",s.c_str());
sprintf(buff+strlen(buff),"%s"," occured ");
sprintf(buff+strlen(buff),"%d",finds);
sprintf(buff+strlen(buff),"%s"," times on line ");
sprintf(buff+strlen(buff),"%d",line_no);
sprintf(buff+strlen(buff),"\n",strlen("\n"));
send_data(sd,buff);
delete (tdata*)arg;
}
On the server side make sure the shared resource (the socket, along with its associated internal buffer) is protected against the concurrent access.
Define and implement an application level protocol used by the server to make it possible for the client to distinguish what the different threads sent.
As an additional note: One cannot rely on read()/write() reading/writing as much bytes as those two functions were told to read/write. It is an essential necessity to check their return value to learn how much bytes those functions actually read/wrote and loop around them until all data that was intended to be read/written had been read/written.
You should put some mutex to your socket.
When a thread use the socket it should block the socket.
Some mutex example.
I can't help you more without the server code. Because the problem is probably in the server.