Here is the method where the while loop resides. I am just connecting to a server , sending an HTTP Request and reading the response. When I debug, I still cannot see why this while loop is not passed.
void HttpSocket::Get(string address)
{
std::string response, host, httpRequest;
uint32_t ipAddress;
ParseRequest(address, host, httpRequest);
ResolveHostAddress(host, ipAddress);
HttpSocket::Connect(ipAddress);
strcpy(bufferToSend, httpRequest.c_str());
n = write(sockfd,bufferToSend,strlen(bufferToSend));
if (n < 0) { throw IO_Exception("Cannot send request."); }
memset(bufferToSend, 0, 500);
memset(bufferToReceive, 0, 200);
n = read(sockfd,bufferToReceive,200);
if (n <= 0){
throw IO_Exception("Cannot read response.");
}
else
{
response += bufferToReceive;
while(n != 0)
{
n = 0;
memset(bufferToReceive, 0, 200);
n = read(sockfd,bufferToReceive,200);
response += bufferToReceive;
cout << "still in the loop" << n << endl;
}
cout << "Response: " << response << endl;
}
}
By the way n is volatile int thus, I don't think compiler optimization causes it. FYI, Everything is fine and working until the last while loop.
::read() is a synchronous function. When there is nothing left on your socket to be read, the call to n = read(sockfd,bufferToReceive,200); just hangs blocks waiting new information.
To solve your problem, you should set the socket as non-blocking and read it with ::recv() which would return -E_WOULDBLOCK if there is no data available:
#include <fcntl.h>
flags = ::fcntl(fd, F_GETFL, 0);
flags |= O_NONBLOCK;
::fcntl(fd, F_SETFL, flags);
An alternative would be to check for available data before reading the file descriptor:
#include <sys/ioctl.h>
int count;
::ioctl(fd, FIONREAD, &count);
Related
I'm trying to build an http server using c++. and so among the conditions based in which i decide how to extract the body entity, is if there's a content length present? , here's a minimal code on how i extract body using Content-Length :
req_t *Webserver::_recv(int client_fd, bool *closed)
{
string req;
static string rest;
// string extracted_req;
char buff[1024];
// while (true) {
// std::cout << "client_fd: " << client_fd << std::endl;
int n = recv(client_fd, buff, 1024, 0);
// std::cout << "n: " << n << std::endl;
if (n == -1)
{
_set_error_code("500", "Internal Server Error");
return NULL;
}
if (n == 0)
{
*closed = true;
return NULL;
}
buff[n] = '\0';
req += buff;
req_t *extracted_req = _extract_req(client_fd, req, rest, closed);
return extracted_req;
}
...
else if (headers.find("Content-Length") != string::npos) {
string body = extract_body_len(client_fd, rest_of_req, content_length);
}
req_t is a simple struct that contains three strings status_line, headers, body.
req_t *Webserver::_extract_req(int client_fd, const string &req, string &rest, bool *closed)
{
req_t *ret;
try
{
ret = new req_t;
}
catch (std::bad_alloc &e)
{
std::cerr << "\033[1;31mError:\033[0m " << e.what() << std::endl;
exit(1);
}
string status_line = req.substr(0, req.find("\r\n"));
string headers = req.substr(req.find("\r\n") + 2, req.find("\r\n\r\n") - req.find("\r\n") - 2);
rest = req.substr(req.find("\r\n\r\n") + 4, req.size() - req.find("\r\n\r\n") - 4);
ret->status_line = status_line;
ret->headers = headers;
// if method is get request body is empty
// if the header contains a content-length, extract number of buytes for body;
if (headers.find("Content-Length") != string::npos)
{
long long content_length = _get_content_len(headers);
if (content_length == -1)
{
_set_error_code("400", "Bad Request");
return NULL;
}
// substracting the length of the body from the length of the request
ret->body = _extract_body_len(client_fd, rest, content_length, closed);
// if body is not complete, return an error
...
string extract_body_len(int client_fd, string& rest, unsigned long long len) {
string body;
unsigned long long total = 0;
body = rest;
// starting total with first bytes of body
total += rest.size();
// if we have it all that's it
if (total >= len) {
body = rest.substr(0, len);
rest = rest.substr(len);
return body;
}
else
{
while (total < len)
{
char buf[1024];
int ret = recv(client_fd, buf, 1024, 0);
// after a lot of debugging , i've noticed that recv starts to read less than 1024 only when total is closer to len, so i added this condition naively.
if (ret != 1024)
{
if ((total + ret) >= len)
{
body += string(buf).substr(0, len - total);
rest = string(buf).substr(len - total);
break;
}
}
if (ret == 0)
{
if (total == len)
{
rest = "";
break;
}
// client closed connection and it's still incomplete: 400
else
{
res->status_code = "400";
res->status_message = "Bad Request";
return NULL;
}
}
else if (ret == -1)
{
res->status_code = "500";
res->status_message = "Internal Server Error";
return body;
}
total += ret;
body += string(buf, ret);
}
}
return body;
}
Now, The problem is i've tested requests with varying sized body entities(8MB, 1.9MB, 31 MB) and all the time i never receive the whole body (as per content-length), the pattern is like the following:
recv keeps reading all 1024 bytes until total gets closer to len then it starts reading smaller numbers. until the difference between total and len is around 400...600 bytes then recv blocks at some point (there's nothing more to read) before total == len.
That really confused me, i tried with different api clients (postman, insonomia) but the same results, i doubted maybe Content-Length isn't that accurate but it obviously should be, what do you think is the problem , why am i receiving or reading less than Content-Length ?
int n = recv(client_fd, buff, 1024, 0);
The above code appears to assume that this recv call returns only the header portion of the HTTP request. Not one byte more, not one byte less.
Unfortunately, you will not find anything in your textbook on network programming that gives you any such guarantee, like that, whatsoever.
Your only guarantee (presuming that there is no socket-level error), is that recv() will return a value between 1 and 1024, representing however many bytes were already received on the socket, or arrived in the first packet that it blocked and waited for.
Using an example of a completely made up HTTP request that looks something like this:
POST /cgi-bin/upload.cgi HTTP/1.0<CR><LF>
Host: www.example.com<CR><LF>
Content-Type: application/octet-stream<CR><LF>
Content-Length: 4000<CR><LF>
<CR><LF>
[4000 octets follow]
When your web browser, or a simulated browser, sends this request this recv call can return any value between 1 and 1024 (excluding the case of network errors).
This means that this recv call can cough up anything between:
a return value of 1, and placing just the letter "P" into buff.
a return value of 1024, and placing the entire HTTP header, plus as much of the initial part of the HTTP content portion of the request into the buffer that's needed to produce 1024 bytes total.
The shown logic is completely incapable of correctly handling all of these possibilities, and that's why it fails. It will need to be reimplemented, pretty much from scratch, using the correct logic.
So I am writing a Windows chat and for testing purposes my client program sends a "hello" message to the server every 300 ms.
First couple messages come good but then like for no reason they start to become junk-
Obviously I want to fix it and I seek for your help :) Here is my code:
Send function:
bool Target::Send(char *message)
{
int length = strlen(message);
int result = send(this->ccSock, (char*)&length, sizeof(int), 0);
if (result <= 0)
return false;
Sleep(10);
result = send(this->ccSock, message, length, 0);
return ((result > 0) ? true : false);
}
Receive function:
Message Server::Receive(SOCKET socket)
{
int length = 0;
int result = recv(socket, (char*)&length, sizeof(int), 0);
Sleep(10);
char *rcvData = new char[length];
result = recv(socket, rcvData, length, 0);
return { rcvData, result };
}
Message struct:
struct Message {
char *msg;
int size;
};
Main send code:
while (true)
{
if (!target->Send("hello"))
{
cout << "Connection broken\n";
target->Clean();
break;
}
Sleep(300);
}
Main receive code:
while (target.sock)
{
Message message = server->Receive(target.sock);
if (message.size > 0)
cout << message.msg << " (" << message.size << ")\n";
else
{
cout << "Target disconnected\n";
server->Clean();
break;
}
Sleep(1);
}
I would really appreciate your help as well as explanation why this is happening!
Your buffer is not null terminated. So when you are trying to print it using std::cout buffer overrun occurs. Correct version of receive code should be:
char *rcvData = new char[length+1];
result = recv(socket, rcvData, length, 0);
rcvData[length] = '\0';
Also you never free allocated memory buffer, so your code leaks it on each Receive call.
I'm having problems with getting winsock RIO working.
It seems that every time I post a RIOReceive it returns immediately with 0 bytes transferred, and my peer can't get a message through.
After posting a RIOReceive, I wait on the RIODequeCompletion, which deques immediately with numResults = 1, but when I inspect the bytesTransferred of the RIORESULT struct, it's 0. This tells me that I'm not setting this thing up properly, but I can't find docs or examples that tell me what else I should be doing.
The internet seems to have very little on RIO. I've looked through MSDN, Len Holgate with TheServerFramework, this site, and two GitHub RIO servers.
RIOEchoServer and RIOServer_sm9 are on GitHub, but I can't post more than two links (this is my first question on this site).
This code is just to get things proven. It's currently not set to use the sendCQ, doesn't handle errors well, etc...
Here's the prep work:
void serverRequestThread() {
//init buffers
//register big buffers
recvBufferMain = rio.RIORegisterBuffer(pRecvBufferMain, bufferSize);
sendBufferMain = rio.RIORegisterBuffer(pSendBufferMain, bufferSize);
if (recvBufferMain == RIO_INVALID_BUFFERID) {
cout << "RIO_INVALID_BUFFERID" << endl;
}
if (sendBufferMain == RIO_INVALID_BUFFERID) {
cout << "RIO_INVALID_BUFFERID" << endl;
}
//create recv buffer slice
recvBuffer1.BufferId = recvBufferMain;
recvBuffer1.Offset = 0;
recvBuffer1.Length = 10000;
//create send buffer slice
sendBuffer1.BufferId = sendBufferMain;
sendBuffer1.Offset = 0;
sendBuffer1.Length = 10000;
//completion queue
recvCQ = rio.RIOCreateCompletionQueue(CQsize, NULL);
sendCQ = rio.RIOCreateCompletionQueue(CQsize, NULL);
if (recvCQ == RIO_INVALID_CQ) {
cout << "RIO_INVALID_CQ" << endl;
}
if (sendCQ == RIO_INVALID_CQ) {
cout << "RIO_INVALID_CQ" << endl;
}
//start a loop for newly accept'd socket
while (recvCQ != RIO_INVALID_CQ && sendCQ != RIO_INVALID_CQ) {
//get accept'd socket
struct sockaddr_in saClient;
int iClientSize = sizeof(saClient);
acceptSocket = accept(listenSocket, (SOCKADDR*)&saClient, &iClientSize);
if (acceptSocket == INVALID_SOCKET) {
cout << "Invalid socket" << endl;
printError();
}
//register request queue
requestQueue = rio.RIOCreateRequestQueue(
acceptSocket, //socket
10, //max RECVs on queue
1, //max recv buffers, set to 1
10, //max outstanding sends
1, //max send buffers, set to 1
recvCQ, //recv queue
recvCQ, //send queue
pOperationContext //socket context
);
if (requestQueue == RIO_INVALID_RQ) {
cout << "RIO_INVALID_RQ" << endl;
printError();
}
I now post a RIOReceive:
//start a loop to repin recv buffer for socket
while (acceptSocket != INVALID_SOCKET) {
//pin a recv buffer to wait on data
recvSuccess = rio.RIOReceive(
requestQueue, //socketQueue
&recvBuffer1, //buffer slice
1, //set to 1
RIO_MSG_WAITALL, //flags
0); //requestContext
if (recvSuccess == false) {
cout << "RECV ERROR!!!!!!!!\n";
printError();
}
//wait for recv to post in queue
//std::this_thread::sleep_for(std::chrono::milliseconds(3000));
As soon as I call RIODequeCompletion, it returns 1:
numResults = 0;
while (numResults == 0) numResults = rio.RIODequeueCompletion(recvCQ, recvArray, 10);
if (numResults == RIO_CORRUPT_CQ) {
cout << "RIO_CORRUPT_CQ" << endl;
} else if (numResults == 0) {
cout << "no messages on queue\n";
} else if (numResults > 0) {
but when I inspect the bytesTransferred of the RIORESULT, it's always 0:
if (recvArray[0].BytesTransferred > 0) {
//process results
if (pRecvBufferMain[0] == 'G') {
//set respnose html
strcpy(pSendBufferMain, responseHTTP);
sendSuccess = rio.RIOSend(
requestQueue, //socketQueue
&sendBuffer1, //buffer slice
1, //set to 1
0, //flags
0); //requestContext
} else if (pRecvBufferMain[0] == 'P') {
//process post
} else {
//recv'd a bad message
}
} //end bytesTransferred if statement
//reset everything and post another recv
}//end response if statement
std::this_thread::sleep_for(std::chrono::milliseconds(100));
}//end while loop for recv'ing
std::this_thread::sleep_for(std::chrono::milliseconds(100));
}//end while loop for accept'ing
}// end function
Like I said, I'm probably not using RIOReceive correctly, and/or I'm not setting the correct socket options that I need to (none right now).
I appreciate any help with this.
Try removing RIO_MSG_WAITALL. There may be a bug whereby you're only getting the close notification (bytes == 0) rather than getting a completion with the data in it. Either way it would be interesting to see if the code works without the flag.
Do my example servers and tests work on your hardware?
I encountered a similar issue of having zero bytesReceived in my dequeued completion result while using RioReceive with RioNotify and RioDequeueCompletion. I would also see the 'Status' value of WSAEINVAL (Invalid Parameter = 10022) in my dequeued completion result, this seems to indicate the WSA error code for the Receive call.
The particular reason I had the error is because I had allocated memory for a receiveBuffer and I was trying to pass that buffer pointer as my buffer handle in the RIO_BUFFER_SEGMENT given to RioReceive instead of passing the IntPtr returned by RioRegisterBuffer.
I fully blame myself for using too many untyped IntPtrs and losing type checking. :)
I'm developing a server-client application using Winsock in c++ and have a problem.
For getting the message from the client by the server I use the code below.
int result;
char buffer[200];
while (true)
{
result = recv(client, buffer, 200, NULL);
if (result > 0)
cout << "\n\tMessage from client: \n\n\t" << message << ";";
}
I send the message "Hello" from the client to the server. However the buffer is actually this:
HelloÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌ
What am I missing?
Since recv might not receive as many bytes as you told it, you typically use a function
like this to receive specified number of bytes. Modified from here
int receiveall(int s, char *buf, int *len)
{
int total = 0; // how many bytes we've received
int bytesleft = *len; // how many we have left to receive
int n = -1;
while(total < *len) {
n = recv(s, buf+total, bytesleft, 0);
if (n <= 0) { break; }
total += n;
bytesleft -= n;
}
*len = total; // return number actually received here
return (n<=0)?-1:0; // return -1 on failure, 0 on success
}
It's up to you to null terminate the string if you receive string which is not null terminated.
The result tells you how many bytes were received. recv doesn't add a terminator since, in general, network data is binary data which might not be usable as a C-style string.
You can add a terminator yourself, if you know the message won't contain the termination character:
buffer[result] = 0; // make sure the buffer is large enough
or make a string (or vector, or whatever) from it:
std::string message_str(message, result);
Note that what you receive might not be a single "message", especially if you're uses a stream protocol like TCP. It might contain more than one message, or just the start of one.
memset(&receive[0], 0, sizeof(receive));
To clear the buffer
You didn't initialize your buffer
char buffer[200] = {0};
while (true)
{
result = recv(client, buffer, 200, NULL);
if (result > 0)
cout << "\n\tMessage from client: \n\n\t" << message << ";";
memset(buffer, 0, 200);
}
My send() and recv() looks like this:
int Send(const char* buffer, int size)
{
cout << "SIZE: " << size << endl;
int offset;
while(offset < size)
{
int n = ::send(getSocket(), buffer + offset, size - offset, 0);
if(n == SOCKET_ERROR)
{
break;
}
offset += n;
if(offset != size)
{
Sleep(1);
}
}
return offset;
}
int Recv(char* buffer, int size)
{
int n = ::recv(getSocket(), buffer, size, 0);
if(n == SOCKET_ERROR)
{
cout << "Error receiving data" << endl;
}
if(n == 0)
{
cout << "Remote host closed connection" << endl;
}
return n;
}
But my output show kind of many bytes sent that seems strange to me:
Received from client: 669
Sent to web server: 3990336
So it should supose to sent 669 bytes, so from where did it get 3990336 ? It is some kind of error or ?
Thanks.
Did you notice that int offset; is not initialize ?
You have to initialize offset with zero. Otherwise it could be any random value.
You do not need Sleep as send call is blocking.
Buffer that you are sending could be split. So if you send, for example, 2K buffer, you could get it in two parts - 1.5K and 0.5K, so you have to perform multiple reads on a client side. MTU is usually set to 1500 bytes.
Maybe it's just your (stripped down?) example code, but you never actually initialize offset. It might have any value, e.g. -5000 and will cause the loop to send 5669 bytes.