I have a client/server application in C++. The server sends a rather large file(27KB) to the client. The client reads from the socket fixed length of 1024 bytes, which I then concatenate to a string. However, when I use the += operator, it doesn't seem to allocate more than 4048 bytes and I end up with a 4KB file on the clients side.
Clients code:
#define BUFFER_SIZE 1024
string outStr="";
char buf[BUFFER_SIZE];
while(1){
int numread;
if ((numread = read(clientSocket, buf, sizeof(buf) -1)) == -1){
fprintf(stderr,"Error: reading from socket");
exit(1);
}
fprintf(stderr,"received answer with numread: %d\n",numread);
if (numread == 0){
break;
}
buf[numread] = '\0';
outStr+=buf;
}
fprintf(stderr,"Transmission is over with total length: %d\n",outStr.length());
The output i get is:
26 times:
received answer with numread: 1023
and after that:
received answer with numread: 246
received answer with numread: 0
transmission is over with total length: 4048
The outputs confirms that the whole file is transferred, but the concatenation does not let me append over the (system limit?) of 4048. However, the c++ string should automatically reallocate its memory when the content needs to be larger. So why is this happening?
Thank you for answers.
You could use str::append (overload no. 4) and explicitly provide the number of bytes to append. This will then properly also append null bytes. So, instead of:
buf[numread] = '\0';
outStr+=buf;
do
outStr.append(numread, buf);
Strings end at '\0' so if your byte array from the socket has something like that, when you concatenate at the end of your response string it will only concatenate until that point. So I think you should use a std::vector to store the whole response.
Related
I know C++ for quite long, but started using it for my purposes some what a year and a half ago.
I started learning network programming on C++ and the first networking project is "File Transfering between hosts over TCP/IP" which sounds kind easy but I am stuck with sending data.
I am trying to send small buffer less than 4KB, so buffer[4096] works fine for me, but I am planning to expand this. WSAStartup(), socket(), bind(), listen(), accept() functions work fine and values for them are initialised for both Server and Client, but I am dealing with other problems, maybe recv(), send() etc.
I still couldn't find the source of the problem.
Also it would be a ton helpful if somebody give me an example of transfering files over TCP/IP, but not in one packet, I want the file to be chunked and sent in parts or as it's called "ring model", but I couldn't find a working model;
P.S. This is first time I am asking here, pls give feedback about how well all of this is written, so that I could write more informative for community help, thanks)
Server
char* buffer = new char[4096];
ZeroMemory(buffer, sizeof(buffer));
ofstream file("a.txt", ios::binary);
int err = recv(conn, buffer, sizeof(buffer), 0);
file << buffer;
file.close();
if (err == 0)
{
printf("Client diconnected...\n");
}
printf("Quitting...\n");
delete[] buffer;
Client
ifstream file("a.txt", ios::binary);
file.seekg(0, ios::end);
int size = file.tellg();
file.seekg(0, ios::beg);
char* buffer = new char[size];
file.read(buffer, size);
file.close();
int err = send(client, buffer, size, 0);
if (err == 0)
{
printf("Disconnecting...\n");
}
printf("Quitting...\n");
delete[] buffer;
"a.txt" file on Client side is 45 bytes in here are 45 * 'a'
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
And this is what I get on Server side, file size is 14 bytes
aaaaaaaa ’pÈ/
In C++, sizeof(buffer) is the size of the pointer type.
You may want to read up on more modern (as in after 1998) C++. We have std::vector nowadays, and that has a convenient size method. It would return 4096 for your buffer. Also, vector handles new[] and delete[] for you.
The fact that you get 8 "a"'s suggests that you built for x64. The remaining bytes are garbage; you should check how many bytes recv actually wrote to buffer. You cannot assume that you got all the bytes you asked for (whether that's 8 or 4096).
I believe that sizeof(buffer) in this line -->
int err = recv(conn, buffer, sizeof(buffer), 0);
will return sizeof(char*) which is 4 bytes in a 32 bit program or 8 bytes in a 64 bits program instead of 4096 because it is not a static array as in you did not declare it as char buffer[4096]. So, either declare it as char buffer[4096] or convert the above code to
int err = recv(conn, buffer, 4096, 0);
Two additional points:
TCP is a streaming protocol (not "message based"), so there's no guarantee that a single recv() will get everything sent in a single send().
The server line file << buffer; assumes buffer is zero terminated.
MSDN state that :
If no error occurs, recv returns the number of bytes received and the
buffer pointed to by the buf parameter will contain this data
received. If the connection has been gracefully closed, the return
value is zero.
Otherwise, a value of SOCKET_ERROR is returned, and a specific error
code can be retrieved by calling WSAGetLastError.
https://learn.microsoft.com/en-us/windows/win32/api/winsock/nf-winsock-recv
test if you actually read 45 bytes and check if there's an error (WSAGetLastError function)
I am trying to send a Base64 encoded image from TCP client using GO and TCP server in C++.
Here is the code snippet for C++ Receiver
std::string recieve(int bufferSize=1024,const char *eom_flag = "<EOF>"){
char buffer[bufferSize];
std::string output;
int iResult;
char *eom;
do{
iResult = recv(client, buffer, sizeof(buffer), 0);
//If End OF MESSAGE flag is found.
eom = strstr(buffer,eom_flag);
//If socket is waiting , do dot append the json, keep on waiting.
if(iResult == 0){
continue;
}
output+=buffer;
//Erase null character, if exist.
output.erase(std::find(output.begin(), output.end(), '\0'), output.end());
//is socket connection is broken or end of message is reached.
}while(iResult > -1 and eom == NULL);
//Trim <EOF>
std::size_t eom_pos = output.rfind(eom_flag);
return output.substr(0,eom_pos);}
Idea is to receive the message until End of Message is found, thereafter continue to listen for another message on the same TCP connection.
Golang TCP client code snippet.
//Making connection
connection, _ := net.Dial("tcp", "localhost"+":"+PortNumber)
if _, err := fmt.Fprintf(connection, B64img+"<EOF>"); err != nil {
log.Println(err)
panic(err)
}
Tried approaches:
Increasing the buffer size in the C++ receiver.
Removing the null character from the end of the string in the C++ receiver.
Observations:
Length of string sent by the client is fixed, while the length of the string after receive function is larger and
random. Example: Go client string length is 25243. For the same string, length after receive when i
run send and receive in the loop is 25243, 26743, 53092, 41389, 42849.
On Saving the received string in a file, I see <0x7f> <0x02> character in the string.
I am using winsock2.h for c++ socket.
You are treating the received data as a C string - a sequence of bytes ending with a 0 byte - which is not correct.
recv receives some bytes and puts them in buffer. Let's say it received 200 bytes.
You then do strstr(buffer,eom_flag);. strstr doesn't know that 200 bytes were received. strstr starts from the beginning of the buffer, and keeps looking until it finds either , or a 0 byte. There is a chance that it might find a in the other 824 bytes of the buffer, even though you didn't receive one.
Then you do output += buffer;. This also treats the buffer as if it ends with a 0 byte. This will look through the whole buffer (not just the first 200 bytes) to find a 0 byte. It will then add everything up to that point into output. Again, it might find a 0 byte in the last 824 bytes of the buffer, and add too much data. It might not find a 0 byte in the buffer at all, and then it will keep on adding extra data from other variables that are stored next to buffer in memory. Or it might find a 0 byte in the first 200 bytes, and stop there (but only if you sent a 0 byte).
What you should do is pay attention to the number of bytes received (which is iResult) and add that many bytes to the output. You could use:
output.insert(output.end(), buffer, buffer+iResult);
Also (as Phillipe Thomassigny has pointed out in a comment), the "" might not be received all at once. You might receive "" separately. You should check whether output has an "" instead of checking whether buffer has an "". (The performance implications of this are left as an exercise to the reader)
By the way, this line doesn't do anything at the moment:
output.erase(std::find(output.begin(), output.end(), '\0'), output.end());
because '\0' never gets added to output, because with output += buffer;, a '\0' tells it where to stop adding.
First question: I am confused between Buffers in TCP. I am trying to explain my proble, i read this documentation TCP Buffer, author said a lot about TCP Buffer, thats fine and a really good explanation for a beginner. What i need to know is this TCP Buffer is same buffer with the one we use in our basic client server program (Char *buffer[Some_Size]) or its some different buffer hold by TCP internally ?
My second question is that i am sending a string data with prefix length (This is data From me) from client over socket to server, when i print my data at console along with my string it prints some garbage value also like this "This is data From me zzzzzz 1/2 1/2....." ?. However i fixed it by right shifting char *recvbuf = new char[nlength>>3]; nlength to 3 bits but why i need to do it in this way ?
My third question is in relevance with first question if there is nothing like TCP Buffer and its only about the Char *buffer[some_size] then whats the difference my program will notice using such static memory allocation buffer and by using dynamic memory allocation buffer using char *recvbuf = new char[nlength];. In short which is best and why ?
Client Code
int bytesSent;
int bytesRecv = SOCKET_ERROR;
char sendbuf[200] = "This is data From me";
int nBytes = 200, nLeft, idx;
nLeft = nBytes;
idx = 0;
uint32_t varSize = strlen (sendbuf);
bytesSent = send(ConnectSocket,(char*)&varSize, 4, 0);
assert (bytesSent == sizeof (uint32_t));
std::cout<<"length information is in:"<<bytesSent<<"bytes"<<std::endl;
// code to make sure all data has been sent
while (nLeft > 0)
{
bytesSent = send(ConnectSocket, &sendbuf[idx], nLeft, 0);
if (bytesSent == SOCKET_ERROR)
{
std::cerr<<"send() error: " << WSAGetLastError() <<std::endl;
break;
}
nLeft -= bytesSent;
idx += bytesSent;
}
std::cout<<"Client: Bytes sent:"<< bytesSent;
Server code:
int bytesSent;
char sendbuf[200] = "This string is a test data from server";
int bytesRecv;
int idx = 0;
uint32_t nlength;
int length_received = recv(m_socket,(char*)&nlength, 4, 0);//Data length info
char *recvbuf = new char[nlength];//dynamic memory allocation based on data length info
//code to make sure all data has been received
while (nlength > 0)
{
bytesRecv = recv(m_socket, &recvbuf[idx], nlength, 0);
if (bytesRecv == SOCKET_ERROR)
{
std::cerr<<"recv() error: " << WSAGetLastError() <<std::endl;
break;
}
idx += bytesRecv;
nlength -= bytesRecv;
}
cout<<"Server: Received complete data is:"<< recvbuf<<std::endl;
cout<<"Server: Received bytes are"<<bytesRecv<<std::endl;
WSACleanup();
system("pause");
delete[] recvbuf;
return 0;
}
You send 200 bytes from the client, unconditionally, but in the server you only receive the actual length of the string, and that length does not include the string terminator.
So first of all you don't receive all data that was sent (which means you will fill up the system buffers), and then you don't terminate the string properly (which leads to "garbage" output when trying to print the string).
To fix this, in the client only send the actual length of the string (the value of varSize), and in the receiving server allocate one more character for the terminator, which you of course needs to add.
First question: I am confused between Buffers in TCP. I am trying to
explain my proble, i read this documentation TCP Buffer, author said a
lot about TCP Buffer, thats fine and a really good explanation for a
beginner. What i need to know is this TCP Buffer is same buffer with
the one we use in our basic client server program (Char
*buffer[Some_Size]) or its some different buffer hold by TCP internally ?
When you call send(), the TCP stack will copy some of the bytes out of your char array into an in-kernel buffer, and send() will return the number of bytes that it copied. The TCP stack will then handle the transmission of those in-kernel bytes to its destination across the network as quickly as it can. It's important to note that send()'s return value is not guaranteed to be the same as the number of bytes you specified in the length argument you passed to it; it could be less. It's also important to note that sends()'s return value does not imply that that many bytes have arrived at the receiving program; rather it only indicates the number of bytes that the kernel has accepted from you and will try to deliver.
Likewise, recv() merely copies some bytes from an in-kernel buffer to the array you specify, and then drops them from the in-kernel buffer. Again, the number of bytes copied may be less than the number you asked for, and generally will be different from the number of bytes passed by the sender on any particular call of send(). (E.g if the sender called send() and his send() returned 1000, that might result in you calling recv() twice and having recv() return 500 each time, or recv() might return 250 four times, or (1, 990, 9), or any other combination you can think of that eventually adds up to 1000)
My second question is that i am sending a string data with prefix
length (This is data From me) from client over socket to server, when
i print my data at console along with my string it prints some garbage
value also like this "This is data From me zzzzzz 1/2 1/2....." ?.
However i fixed it by right shifting char *recvbuf = new
char[nlength>>3]; nlength to 3 bits but why i need to it in this way ?
Like Joachim said, this happens because C strings depend on the presence of a NUL-terminator byte (i.e. a zero byte) to indicate their end. You are receiving strlen(sendbuf) bytes, and the value returned by strlen() does not include the NUL byte. When the receiver's string-printing routine tries to print the string, it keeps printing until if finds a NUL byte (by chance) somewhere later on in memory; in the meantime, you get to see all the random bytes that are in memory before that point. To fix the problem, either increase your sent-bytes counter to (strlen(sendbuf)+1), so that the NUL terminator byte gets received as well, or alternatively have your receiver manually place the NUL byte at the end of the string after it has received all of the bytes of the string. Either way is acceptable (the latter way might be slightly preferable as that way the receiver isn't depending on the sender to do the right thing).
Note that if your sender is going to always send 200 bytes rather than just the number of bytes in the string, then your receiver will need to always receive 200 bytes if it wants to receive more than one block; otherwise when it tries to receive the next block it will first get all the extra bytes (after the string) before it gets the next block's send-length field.
My third question is in relevance with first question if there is
nothing like TCP Buffer and its only about the Char *buffer[some_size]
then whats the difference my program will notice using such static
memory allocation buffer and by using dynamic memory allocation buffer
using char *recvbuf = new char[nlength];. In short which is best and
why ?
In terms of performance, it makes no difference at all. send() and receive() don't care a bit whether the pointers you pass to them point at the heap or the stack.
In terms of design, there are some tradeoffs: if you use new, there is a chance that you can leak memory if you don't always call delete[] when you're done with the buffer. (This can particularly happen when exceptions are thrown, or when error paths are taken). Placing the buffer on the stack, on the other hand, is guaranteed not to leak memory, but the amount of space available on the stack is finite so a really huge array could cause your program to run out of stack space and crash. In this case, a single 200-byte array on the stack is no problem, so that's what I would use.
My goal is create an app client server, written in C++.
When the server read an input from the client, should process the string and give an output.
Basically, I have a simply echo server that send the same message.
But if the user types a special string (like "quit"), the program have to do something else.
My problem is that this one dont happend, because the comparison between strings is not working... I dunno why!
Here a simple code:
while(1) {
int num = recv(client,buffer,BUFSIZE,0);
if (num < 1) break;
send(client, ">> ", 3, 0);
send(client, buffer, num, 0);
char hello[6] ="hello";
if(strcmp(hello,buffer)==0) {
send(client, "hello dude! ", 12, 0);
}
buffer[num] = '\0';
if (buffer[num-1] == '\n')
buffer[num-1] = '\0';
std::cout << buffer;
strcpy(buffer, "");
}
Why the comparison is not working?
I have tried many solutions...but all failed :(
Your data in buf may not be NULL-terminated, because buf contains random data if not initialized. You only know the content of the first num bytes. Therefore you also have to check how much data you've received before comparing the strings:
const char hello[6] ="hello";
size_t hello_sz = sizeof hello - 1;
if(num == hello_sz && memcmp(hello, buffer, hello_sz) == 0) { ...
As a side note, this protocol will be fragile unless you delimit your messages, so in the event of fragmented reads (receive "hel" on first read, "lo" on the second) you can tell where one message starts and another one ends.
strcmp requires null terminated strings. The buffer you read to might have non-null characters after the received message.
Either right before the read do:
ZeroMemory(buffer, BUFSIZE); //or your compiler defined equivalent
Or right after the read
buffer[num] = '\0';
This will ensure that there is a terminating null at the end of the received message and the comparison should work.
A string is defined to be an array of chars upto and including the terminating \0 byte. Initially your buffer contains arbitrary bytes, and is not even guaranteed to contain a string. You have to set buffer[num] = '\0' to make it a string.
That of course means that recv should not read sizeof buffer bytes but one byte less.
I have:
char buf[320];
read(soc, buf, sizeof(buf));
//print buf;
However, sometimes the reply is much bigger then 320 characters, so I'm trying to run the read in a loop to avoid taking up too much memory space. I tried read(soc, buf, sizeof(buf)) but that only prints the same first x characters over again. How would I print the leftover characters that did not fit into the first 320 characters in a loop?
Thanks
Change your loop to something like:
int numread;
while(1) {
if ((numread = read(soc, buf, sizeof(buf) - 1)) == -1) {
perror("read");
exit(1);
}
if (numread == 0)
break;
buf[numread] = '\0';
printf("Reply: %s\n", buf);
}
for the reasons Nikola states.
Every time you call read( s, buf, buf_size ) the kernel copies min( buf_size, bytes_available ) into the buf, where bytes_available is the number of bytes already received and waiting in socket receive buffer. The read(2) system call returns the number of bytes placed into application buffer, or -1 on error, or 0 to signal EOF, i.e. a close(2) of the socket on the sending end. Thus when you reuse the buffer, only part of it might be overwritten with new data. Also note that -1 evaluates to true in C and C++. This is probably the case you are hitting.
printf(3) expects zero-terminated string for the %s format specifier. The bytes read from the socket might not contain the '\0' byte, thus letting printf(3) print till it finds zero further down somewhere. This might lead to buffer overrun.
The points here are:
Always check the value returned from read(2)
If you print strings read from a socket - always zero-terminate them manually.
Hope this helps.