My send() and recv() looks like this:
int Send(const char* buffer, int size)
{
cout << "SIZE: " << size << endl;
int offset;
while(offset < size)
{
int n = ::send(getSocket(), buffer + offset, size - offset, 0);
if(n == SOCKET_ERROR)
{
break;
}
offset += n;
if(offset != size)
{
Sleep(1);
}
}
return offset;
}
int Recv(char* buffer, int size)
{
int n = ::recv(getSocket(), buffer, size, 0);
if(n == SOCKET_ERROR)
{
cout << "Error receiving data" << endl;
}
if(n == 0)
{
cout << "Remote host closed connection" << endl;
}
return n;
}
But my output show kind of many bytes sent that seems strange to me:
Received from client: 669
Sent to web server: 3990336
So it should supose to sent 669 bytes, so from where did it get 3990336 ? It is some kind of error or ?
Thanks.
Did you notice that int offset; is not initialize ?
You have to initialize offset with zero. Otherwise it could be any random value.
You do not need Sleep as send call is blocking.
Buffer that you are sending could be split. So if you send, for example, 2K buffer, you could get it in two parts - 1.5K and 0.5K, so you have to perform multiple reads on a client side. MTU is usually set to 1500 bytes.
Maybe it's just your (stripped down?) example code, but you never actually initialize offset. It might have any value, e.g. -5000 and will cause the loop to send 5669 bytes.
Related
I'm trying to build an http server using c++. and so among the conditions based in which i decide how to extract the body entity, is if there's a content length present? , here's a minimal code on how i extract body using Content-Length :
req_t *Webserver::_recv(int client_fd, bool *closed)
{
string req;
static string rest;
// string extracted_req;
char buff[1024];
// while (true) {
// std::cout << "client_fd: " << client_fd << std::endl;
int n = recv(client_fd, buff, 1024, 0);
// std::cout << "n: " << n << std::endl;
if (n == -1)
{
_set_error_code("500", "Internal Server Error");
return NULL;
}
if (n == 0)
{
*closed = true;
return NULL;
}
buff[n] = '\0';
req += buff;
req_t *extracted_req = _extract_req(client_fd, req, rest, closed);
return extracted_req;
}
...
else if (headers.find("Content-Length") != string::npos) {
string body = extract_body_len(client_fd, rest_of_req, content_length);
}
req_t is a simple struct that contains three strings status_line, headers, body.
req_t *Webserver::_extract_req(int client_fd, const string &req, string &rest, bool *closed)
{
req_t *ret;
try
{
ret = new req_t;
}
catch (std::bad_alloc &e)
{
std::cerr << "\033[1;31mError:\033[0m " << e.what() << std::endl;
exit(1);
}
string status_line = req.substr(0, req.find("\r\n"));
string headers = req.substr(req.find("\r\n") + 2, req.find("\r\n\r\n") - req.find("\r\n") - 2);
rest = req.substr(req.find("\r\n\r\n") + 4, req.size() - req.find("\r\n\r\n") - 4);
ret->status_line = status_line;
ret->headers = headers;
// if method is get request body is empty
// if the header contains a content-length, extract number of buytes for body;
if (headers.find("Content-Length") != string::npos)
{
long long content_length = _get_content_len(headers);
if (content_length == -1)
{
_set_error_code("400", "Bad Request");
return NULL;
}
// substracting the length of the body from the length of the request
ret->body = _extract_body_len(client_fd, rest, content_length, closed);
// if body is not complete, return an error
...
string extract_body_len(int client_fd, string& rest, unsigned long long len) {
string body;
unsigned long long total = 0;
body = rest;
// starting total with first bytes of body
total += rest.size();
// if we have it all that's it
if (total >= len) {
body = rest.substr(0, len);
rest = rest.substr(len);
return body;
}
else
{
while (total < len)
{
char buf[1024];
int ret = recv(client_fd, buf, 1024, 0);
// after a lot of debugging , i've noticed that recv starts to read less than 1024 only when total is closer to len, so i added this condition naively.
if (ret != 1024)
{
if ((total + ret) >= len)
{
body += string(buf).substr(0, len - total);
rest = string(buf).substr(len - total);
break;
}
}
if (ret == 0)
{
if (total == len)
{
rest = "";
break;
}
// client closed connection and it's still incomplete: 400
else
{
res->status_code = "400";
res->status_message = "Bad Request";
return NULL;
}
}
else if (ret == -1)
{
res->status_code = "500";
res->status_message = "Internal Server Error";
return body;
}
total += ret;
body += string(buf, ret);
}
}
return body;
}
Now, The problem is i've tested requests with varying sized body entities(8MB, 1.9MB, 31 MB) and all the time i never receive the whole body (as per content-length), the pattern is like the following:
recv keeps reading all 1024 bytes until total gets closer to len then it starts reading smaller numbers. until the difference between total and len is around 400...600 bytes then recv blocks at some point (there's nothing more to read) before total == len.
That really confused me, i tried with different api clients (postman, insonomia) but the same results, i doubted maybe Content-Length isn't that accurate but it obviously should be, what do you think is the problem , why am i receiving or reading less than Content-Length ?
int n = recv(client_fd, buff, 1024, 0);
The above code appears to assume that this recv call returns only the header portion of the HTTP request. Not one byte more, not one byte less.
Unfortunately, you will not find anything in your textbook on network programming that gives you any such guarantee, like that, whatsoever.
Your only guarantee (presuming that there is no socket-level error), is that recv() will return a value between 1 and 1024, representing however many bytes were already received on the socket, or arrived in the first packet that it blocked and waited for.
Using an example of a completely made up HTTP request that looks something like this:
POST /cgi-bin/upload.cgi HTTP/1.0<CR><LF>
Host: www.example.com<CR><LF>
Content-Type: application/octet-stream<CR><LF>
Content-Length: 4000<CR><LF>
<CR><LF>
[4000 octets follow]
When your web browser, or a simulated browser, sends this request this recv call can return any value between 1 and 1024 (excluding the case of network errors).
This means that this recv call can cough up anything between:
a return value of 1, and placing just the letter "P" into buff.
a return value of 1024, and placing the entire HTTP header, plus as much of the initial part of the HTTP content portion of the request into the buffer that's needed to produce 1024 bytes total.
The shown logic is completely incapable of correctly handling all of these possibilities, and that's why it fails. It will need to be reimplemented, pretty much from scratch, using the correct logic.
I am learning socket programming in C++. I have initialised by buffer value at 10. I have used select() function to monitor the socket. When the client sends data which is greater than my buffer can accommodate, it will read buffer size worth of data and will again go through the all my clients and read the remaining data. How can I read all the data at once without parsing through the loop again to read the remaining data ?
Thank you for your help.
for(i = 0;i < max_clients; i++){
sd = clients_list[i];
if(FD_ISSET(sd,&temp)){
char buf[10];
int n = recv(sd, buf, sizeof(buf),0);
if(n == -1){
perror("recv");
}else if(n == 0){
cout << "Client is GONE " << endl;
close(sd);
clients_list[i] = 0;
}
else{
buf[n] = '\0';
cout << "From the node: " << buf << endl;
}
}
}
Just to complete my comment - this code snippet may help to understand
not tested code
for (i = 0;i < max_clients; i++) {
sd = clients_list[i];
if (FD_ISSET(sd,&temp)) {
char buf[10];
int n;
int flag;
do {
n = recv(sd, buf, sizeof(buf),0);
if (n > 0) {
buf[n] = '\0';
cout << "From the node: " << buf << endl;
ioctl(sd, FIONREAD, &flag);
} else {
flag = 0;
}
} while (flag > 0);
if (n < 0) {
perror("recv");
// in case of an error you may actively close the socket
// and end transmission by server
close(sd);
clients_list[i] = 0;
} else {
cout << "Client has currently no further transmission " << endl;
// don't close socket maybe later new transmission
// active close socket by server process only if
// e.g. a timeout has reached
}
}
}
By my knowledge you can't determine the buffer size that was sent by the other side. I think your way is the best way to deal with it, but you always can use some tricks.
1. Determine the buffer size that will be sent as a constant and thus you may get the wanted amount every time.
2. A way which I think is the best: Use a self made protocol to determine the length of the message that was sent. You may for example read the 4 first bytes to determine the size of your message and then read the data off the buffer by the given size.
You can use the next transformation:
char* c = new char[4];
recv(sd, c, sizeof(c), 0);
int len = (*(int*)c);
delete[] c;
char* but = new char[len];
recv(sd, but, len, 0);
Firstly, the network is unreliable and unpredictable, so we can not know how many bytes we will receive, that is the start point of designing a server.
Secondly, I think if you want a more elegent server, you may want to try epoll or IOCP, they still have loops, but the I/O multiplexing performance are much better.
Thirdly, if you want to accept all the data, you may try to construct a buffer. In real network, we always buffer the data using kafka or some other apps.
How can I read all the data at once without parsing through the loop
again to read the remaining data ?
Generally speaking, you can't, since all of the data might not be available yet. If you want, you can tell recv() to block until sizeof(buf) bytes are available, by passing in MSG_WAITALL instead of 0 to its final ("flags") argument... but note that even then it is possible for recv() to return fewer than sizeof(buf) bytes in some cases (e.g. if a signal is caught, or the connection is closed before sizeof(buf) bytes are received, and so on). So even if you use MSG_WAITALL you will still want to implement short-read-handling logic in order to get 100% correct behavior.
Also note that the sender is under no obligation to send all of the bytes you are expecting in a timely manner, nor is the network under any obligation to deliver all of the bytes in a timely manner. So it's perfectly possible that a sender might send sizeof(buf)-1 bytes to you, then wait 15 minutes before sending the last byte, and if you are blocked in a recv() call waiting for that last byte, then your server will be unresponsive to all of its other clients during that long period.
Therefore when implementing a multiplexed/single-threaded server like this, it's usually best to set the sockets to non-blocking mode, and keep a separate received-data buffer for each client's socket. That way you can loop through your sockets list, recv() as much data as you can from each socket (without blocking), and append that data to that socket's associated buffer, then check the buffer to see if you have enough data in it yet to handle the next chunk, and if you do, handle the chunk and then remove the data from the buffer, and continue. That way client B never has to wait for client A to finish sending a message to the server.
I have used ioctl() function with FIONREAD to check whether there is more data to read it or not and it has been working.
Here is the code:
for(i = 0;i < max_clients; i++){
sd = clients_list[i];
if(FD_ISSET(sd,&temp)){
receive_more:
char buf[10];
int arg = 0;
int n = recv(sd, buf, sizeof(buf),0);
if(n == -1){
perror("recv");
}else if(n == 0){
cout << "Client is GONE " << endl;
close(sd);
clients_list[i] = 0;
}
else{
buf[n] = '\0';
cout << "From the node: " << buf << endl;
ioctl(sd,FIONREAD,&arg);
if(arg > 0){
goto receive_more;
}
}
}
}
So I am writing a Windows chat and for testing purposes my client program sends a "hello" message to the server every 300 ms.
First couple messages come good but then like for no reason they start to become junk-
Obviously I want to fix it and I seek for your help :) Here is my code:
Send function:
bool Target::Send(char *message)
{
int length = strlen(message);
int result = send(this->ccSock, (char*)&length, sizeof(int), 0);
if (result <= 0)
return false;
Sleep(10);
result = send(this->ccSock, message, length, 0);
return ((result > 0) ? true : false);
}
Receive function:
Message Server::Receive(SOCKET socket)
{
int length = 0;
int result = recv(socket, (char*)&length, sizeof(int), 0);
Sleep(10);
char *rcvData = new char[length];
result = recv(socket, rcvData, length, 0);
return { rcvData, result };
}
Message struct:
struct Message {
char *msg;
int size;
};
Main send code:
while (true)
{
if (!target->Send("hello"))
{
cout << "Connection broken\n";
target->Clean();
break;
}
Sleep(300);
}
Main receive code:
while (target.sock)
{
Message message = server->Receive(target.sock);
if (message.size > 0)
cout << message.msg << " (" << message.size << ")\n";
else
{
cout << "Target disconnected\n";
server->Clean();
break;
}
Sleep(1);
}
I would really appreciate your help as well as explanation why this is happening!
Your buffer is not null terminated. So when you are trying to print it using std::cout buffer overrun occurs. Correct version of receive code should be:
char *rcvData = new char[length+1];
result = recv(socket, rcvData, length, 0);
rcvData[length] = '\0';
Also you never free allocated memory buffer, so your code leaks it on each Receive call.
I'm developing a server-client application using Winsock in c++ and have a problem.
For getting the message from the client by the server I use the code below.
int result;
char buffer[200];
while (true)
{
result = recv(client, buffer, 200, NULL);
if (result > 0)
cout << "\n\tMessage from client: \n\n\t" << message << ";";
}
I send the message "Hello" from the client to the server. However the buffer is actually this:
HelloÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌÌ
What am I missing?
Since recv might not receive as many bytes as you told it, you typically use a function
like this to receive specified number of bytes. Modified from here
int receiveall(int s, char *buf, int *len)
{
int total = 0; // how many bytes we've received
int bytesleft = *len; // how many we have left to receive
int n = -1;
while(total < *len) {
n = recv(s, buf+total, bytesleft, 0);
if (n <= 0) { break; }
total += n;
bytesleft -= n;
}
*len = total; // return number actually received here
return (n<=0)?-1:0; // return -1 on failure, 0 on success
}
It's up to you to null terminate the string if you receive string which is not null terminated.
The result tells you how many bytes were received. recv doesn't add a terminator since, in general, network data is binary data which might not be usable as a C-style string.
You can add a terminator yourself, if you know the message won't contain the termination character:
buffer[result] = 0; // make sure the buffer is large enough
or make a string (or vector, or whatever) from it:
std::string message_str(message, result);
Note that what you receive might not be a single "message", especially if you're uses a stream protocol like TCP. It might contain more than one message, or just the start of one.
memset(&receive[0], 0, sizeof(receive));
To clear the buffer
You didn't initialize your buffer
char buffer[200] = {0};
while (true)
{
result = recv(client, buffer, 200, NULL);
if (result > 0)
cout << "\n\tMessage from client: \n\n\t" << message << ";";
memset(buffer, 0, 200);
}
I have two simple programs set up that share data through a unix domain socket. One program reads data out of a Queue and sends it to the other application. Before it is sent each piece of data is front-appended by four bytes with the length, if it is less then four bytes the left over bytes are the '^' symbol.
The client application then reads the first four bytes, sets a buffer to the appropriate size and then reads the rest. The problem that I'm having is that the first time through the message will be sent perfectly. Every other time after that there is extra data being sent so a message like "what a nice day out" would come out like "what a nice day out??X??". So I feel like a buffer is not being cleared correctly but I can't seem to find it.
Client code:
listen(sock, 5);
for (;;)
{
msgsock = accept(sock, 0, 0);
if (msgsock == -1)
perror("accept");
else do
{
char buf[4];
bzero(buf, sizeof(buf));
if ((rval = read(msgsock, buf, 4)) < 0)
perror("reading stream message");
printf("--!%s\n", buf);
string temp = buf;
int pos = temp.find("^");
if(pos != string::npos)
{
temp = temp.substr(0, pos);
}
int sizeOfString = atoi(temp.c_str());
cout << "TEMP STRING: " << temp << endl;
cout << "LENGTH " << sizeOfString << endl;
char feedWord[sizeOfString];
bzero(feedWord, sizeof(feedWord));
if ((rval = read(msgsock, feedWord, sizeOfString)) < 0)
perror("reading stream message");
else if (rval == 0)
printf("Ending connection\n");
else
printf("-->%s\n", feedWord);
bzero(feedWord, sizeof(feedWord));
sizeOfString = 0;
temp.clear();
}
while (rval > 0);
close(msgsock);
}
close(sock);
unlink(NAME);
Server Code
pthread_mutex_lock(&mylock);
string s;
s.clear();
s = dataQueue.front();
dataQueue.pop();
pthread_mutex_unlock(&mylock);
int sizeOfString = strlen(s.c_str());
char sizeofStringBuffer[10];
sprintf(sizeofStringBuffer, "%i", sizeOfString);
string actualString = sizeofStringBuffer;
int tempSize = strlen(sizeofStringBuffer);
int remainder = 4 - tempSize;
int x;
for(x =0; x < remainder; x++)
{
actualString = actualString + "^";
}
cout << "LENGTH OF ACTUAL STRING: " << sizeOfString << endl;
actualString = actualString + s;
cout << "************************" << actualString << endl;
int length = strlen(actualString.c_str());
char finalString[length];
bzero(finalString, sizeof(finalString));
strcpy(finalString, actualString.c_str());
if (write(sock, finalString, length) < 0)
perror("writing on stream socket");
Rather than padding your packet length with '^', you'd be far better off just doing:
snprintf(sizeofStringBuffer, 5, "%04d", sizeOfString);
so that the value is 0 padded - then you don't need to parse out the '^' characters in the receiver code.
Please also edit out your debug code - there's only one write() in the current code, and it doesn't match your description of the protocol.
Ideally - split your sending routine into a function of its own. You can also take advantage of writev() to handle coalescing the string holding the "length" field with the buffer holding the actual data and then sending them as a single atomic write().
Untested code follows:
int write_message(int s, std::string msg)
{
struct iovec iov[2];
char hdr[5];
char *cmsg = msg.c_str();
int len = msg.length();
snprintf(hdr, 5, "%04d", len); // nb: assumes len <= 9999;
iov[0].iov_base = hdr;
iov[0].iov_len = 4;
iov[1].iov_base = cmsg;
iov[1].iov_len = len;
return writev(s, iov, 2);
}
You have to check return values of both write and read not only for -1 but for short (less then requested) writes/reads. You also seem to just continue after printing an error with perror - do an exit(2) or something there.
Two things:
First - on the Server side you are writing off the end of your array.
char finalString[length];
bzero(finalString, sizeof(finalString));
strcpy(finalString, actualString.c_str());
The strcpy() will copy length+1 characters into finalString (character pull the null terminator).
Second (and most likely to be the problem) - on the client side you are not null terminating the string you read in, therefore the printf() will print your string, and then whatever is on the stack up to the point it hits a null.
Increase both buffers by one, and you should be in better shape.