C Socket file transfer corrupted data - c++

According to this solution to send out an image through TCP. Since the code is very elegant compared to other ways and both image and file are data, I believe that we can use almost the same code to send out a file.
So if I want to send a file from a client to a server.
On the client side
get file size
send file size
// Above steps will always work, so I will only show code after here
read file content into a buffer
char buf[size];
read(fs,buf,size);
send the buffer
int bytes = 0;
for (uint i = 0;i<size;i+=bytes)
{
if ((bytes = send(sock,buf+i,size-i,0))<0)
{
fprintf(stderr,"Can not send file\n");
close(fd);
return false;
}
fprintf(stderr,"bytes write = %d\n",bytes);
}
And on the server side
recv file size
recv stuff into a buffer with size from step 1
char buf[size];
int bytes=0;
for (uint i = 0;i<size;i+=bytes)
{
if ((bytes = recv(sock,buf+i,size-i,0))<0)
{
fprintf(stderr,"Can not receive file\n");
return false;
}
fprintf(stderr,"bytes read = %d\n",bytes);
}
write buffer to a file
fwrite(buf,sizeof(char),size,fs);
This code will compile and run.
When I send out an cpp binary file(24k) from client to server, since both client and server are on the same machine (OS X), this binary file will be received and can be executed.
But if the server forward the file back to the client, and client forward this file back to the server multiple times, this binary file will be corrupted. But the number of bytes sent and number of bytes received are the same, and the file size is still 24k.
I am wondering what is going wrong here.
Is this an OS bug?
Thanks,

Neither send(), nor recv(), guarantees that the number of bytes requested will actually be sent or received. In that case, the return value will still be positive, but less than the number of bytes that was requested in the system call.
This is extensively documented in the manual page documentation for send() and recv(). Please reread your operating system's documentation for these system call.
It is the application's responsibility to try again, to send or receive the remaining bytes.
This code assumes that the number of bytes that was sent is the number of bytes it requested to be sent. It does appear to handle recv()'s return status properly, but not send()'s. After a fewer number of bytes was sent, this code still assumes that the entire contents were sent or received, and the fwrite() system call will end up writing junk instead of the latter part of the file.

If both client and server are in the same folder, then in this case it is just like copying and pasting a file.
So when client send out a file, it will
open file
get file name/size + send name/size + send data
close file
On the server side,
get file name/size
open the same file again
get file content
close file
So the problem will occur on step 2 by causing a race condition.

Related

How send big string in winsock using c++ from client to server

I am writing a server-client application using Winsock in c++ for sending a file line by line and I have a problem in sending huge string. The line size is very huge.
For getting the message from the client by the server I use the code below.
int result;
char message[200];
while (true)
{
recv(newSd, (char*)&message, sizeof(message), 0);
cout << "The Message from client: " << message << ";";
}
The above code working fine if I send small length of the message. But, what I wanted is to send an unknown size of lines in a file.
How to send a big unknown string instead of char message[200];
TCP is a byte stream, it knows nothing about messages or lines or anything like that. When you send data over TCP, all it knows about is raw bytes, not what the bytes represent. It is your responsibility to implement a messaging protocol on top of TCP to delimit the data in some meaningful way so the receiver can know when the data is complete. There are two ways to do that:
send the data length before sending the actual data. The receiver reads the length first, then reads however many bytes the length says.
send a unique terminator after sending the data. Make sure the terminator never appears in the data. The receiver can then read until the terminator is received.
You are not handling either of those in your recv() code, so I suspect you are not handling either of them in your send() code, too (which you did not show).
Since you are sending a text file, you can either:
send the file size, such as in a uint32_t or uint64_t (depending on how large the file is), then send the raw file bytes.
send each text line individually as-is, terminated by a CRLF or bare-LF line break after each line, and then send a final terminator after the last line.
You are also ignoring the return value of recv(), which tells you how many bytes were actually received. It can, and usually does, return fewer bytes than requested, so you must be prepared to call recv() multiple times, usually in a loop, to receive data completely. Same with send().

QTcpSocket data transfer stops when read buffer is full and does not resumes when it frees up

I have server-client Qt application, where client sends data packets to server and server reads them at a set time intervals. It happens that client sends data faster than server can read thus filling all the memory on the server side. I am using QAbstractSocket::setReadBufferSize(size) to set max read buffer size on the server side and when it fills up, socket data transferring stops, and data is buffered on client side, which is what i want, but the problem is when server's QTcpSocket's internal read buffer frees up (is not full anymore), data transfer between client and server does not resume.
I've tried to use QAbstractSocket::resume() which seems to work, but Qt5.10 documentation says:
Continues data transfer on the socket. This method should only be used
after the socket has been set to pause upon notifications and a
notification has been received. The only notification currently
supported is QSslSocket::sslErrors(). Calling this method if the
socket is not paused results in undefined behavior.
I feel like I should not use that function in this situation, but is there any other solution? How do i know if socket is paused? Why data transfer does not continue automaticaly when QTcpSocket's internal read buffer is not full anymore?
EDIT 1 :
I have downloaded Qt(5.10.0) sources and pdb's to debug this situation and I can see that QAbstractSocket::readData() internal function have line "d->socketEngine->setReadNotificationEnabled(true)" which re-enables data transfering, but QAbstractSocket::readData() gets called only when QTcpSocket internal read buffer is empty (qiodevice.cpp; QIODevicePrivate::read(); line 1176) and in My situation it is never empty, because I read it only when it has enough data for complete packet.
Shouldn't QAbstractSocket::readData() be called when read buffer is not full anymore and not when it's completely empty? Or maybe i do something wrong?
Found a Workaround!
In Qt5.10 sources i can clearly see that QTcpSpcket internal read notifications is disabled (qabstractsocket.cpp; bool QAbstractSocketPrivate::canReadNotification(); line 697) when read buffer is full and to enable read notifications you need to read all buffer to make it empty OR use QAbstractSocket::setReadBufferSize(newSize) which internally enables read notifications WHEN newSize is not 0 (unlimited) and not equal to oldSize (qabstractsocket.cpp; void QAbstractSocket::setReadBufferSize(qint64 size); line 2824).
Here's a short function for that:
QTcpSocket socket;
qint64 readBufferSize; // Current max read buffer size.
bool flag = false; // flag for changing max read buffer size.
bool isReadBufferLimitReached = false;
void App::CheckReadBufferLimitReached()
{
if (readBufferSize <= socket.bytesAvailable())
isReadBufferLimitReached = true;
else if (isReadBufferLimitReached)
{
if (flag)
{
readBufferSize++;
flag = !flag;
}
else
{
readBufferSize--;
flag = !flag;
}
socket.setReadBufferSize(readBufferSize);
isReadBufferLimitReached = false;
}
}
In the function which reads data from QTcpSocket at the set intervals, BEFORE reading data, I call this function, which checks if read buffer is full and sets isReadBufferLimitReached if true. Then I read needed amount of data from QTcpSocket and AT THE END I call that function again, which, if buffer were full before, calls QAbstractSocket::setReadBufferSize(size) to set new buffer size and enable internal read notifications. Changing read buffer size by +/-1 should be safe, because you read at least 1 byte from socket.

Concatenating buffer when the sending is too fast in a socket

I have a socket server receiving XML files each 500ms and sometimes it goes wrong concatenating more than 1 file as only one file.
do
{
char* buf = (char*)MALLOCZ(IP_BUF_SZ);
chrs_read = recv(sockfd, buf, IP_BUF_SZ, 0);
if (chrs_read > 0)
sBuffer.append(buf, chrs_read);
FREE(buf);
buf = NULL;
}
while (chrs_read > 0);
So, sometime chrs_read doesn't return me -1 to stop the receiving and save the file to start a new receiving.
Do I forget some configuration in the socket - it's async and non-blocking by default - and I supposed to keep using this way?
Thank you in advance
The problem is that all files are sent through the same connection, without having a delimiter between them. When the files are sent often, and there is some latency in the network, you can't know where a file ends and a new one begins.
Solutions:
Insert a delimiter between the files, such that you can close the file when you receive the delimiter, and open a new one. Note that the delimiter may be received anywhere inside of buf, or it could even, if the delimiter is longer than one byte, be received partially in one recv call, and the rest follows in the next recv call.
On the sending end, close the connection after sending the file and open a new one for the new file.

How to stop receiving data from socket C/C++

the key is that I send 4096 bytes but only 119 bytes aprox. carry useful information.
The 100 bytes ends with \r\n\r\n so in the client, when I read \r\n\r\n I want to stop receiving information from that string, and start over.
I don't know if I have to flush, or close the socket, or whatever...
They are sockets TCP.
In the client I do:
buf details[4096];
strcpy(details,"1");
strcat(details,"10/04/12");
strcat(details,"Kevin Fire");
strcat(detils,"abcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcde\r\n\r\n");
nbytes_sent = send(sock,(char *)details,sizeof(details),0);
On the other hand, the server...
char buf[20];
memset(buf,'\0',20);
while(!end){
nbytes_read=recv(sclient,(char *)ress,sizeof(ress),0);
if(strcmp(ress,"1")==0){
printf("Details: %s (%i)\n",buf,nbytes_read);
while(strcmp(buf,"\r\n\r\n") != 0){
nbytes_read=recv(sclient,(char *)buf,sizeof(buf),0);
cout.flush();
printf("Details: %s (%i)\n",buf,nbytes_read);
} }
if(strcmp(buf,"\r\n\r\n")==0) printf("The End\n");
cout.flush();
}
}
I just want to read a new "ress" and not being retrieving the rest of bytes that are not useful.
Thanks in advance.
If you mean you want to discard rest of data and read a new block you can't do it with TCP because it is stream oriented and do not have a concept of message and have no idea about the rest of message that you want to ignore. if you mean something else please describe it more.
but beside that why you use nbytes_sent = send(sock,(char *)details,sizeof(details),0); when only data until \r\n' is important. you can usenbytes_sent = send(sock,(char *)details,strlen(details),0);` that only send valid data and reduce garbage that you send over network and you don't need to start over in the server??
I'm not sure if I'm following your question entirely, but it appears that you can just set end=true whenever you detect the end of the message you're receiving:
char buf[20];
memset(buf,'\0',20);
while(!end)
{
nbytes_read=recv(sclient,(char *)ress,sizeof(ress),0);
if(strcmp(ress,"1")==0)
{
printf("Details: %s (%i)\n",buf,nbytes_read);
while(strcmp(buf,"\r\n\r\n") != 0)
{
nbytes_read=recv(sclient,(char *)buf,sizeof(buf),0);
cout.flush();
printf("Details: %s (%i)\n",buf,nbytes_read);
}
}
if(strcmp(buf,"\r\n\r\n")==0)
{
end = true; // <--- This should do it for you, right?
printf("The End\n");
}
cout.flush();
}
However, if the client is still connected and writing the next message to the socket, then you just need to start reading the next message. So what happens with the client once the message is written? Does it start writing the next message or does it close the socket connection?
In addition: you need to take what's in your buffer and create a message from it. When the current message is done, then consider creating a new message with the contents of the buffer from the next message.
If you design your protocol like HTTP 1.0, where each request opens a new socket, then you close the socket after you've read enough.
Otherwise, you need to keep on reading until you skipped the entire 4096 bytes. The easiest thing to do is to keep on reading till you get 4096 bytes in the first place (you'll need to call recv in a loop), and then parse the contents of the buffer. Then again, you might be better off redesigning your protocol.
My thought would be to just to peek at the first x chars.
The 4 chars could be the size of the buffer expected.
So for example if your message is:
abcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcde\r\n\r\n
It's (to use your schematic) 100 bytes, plus the \r\n\r\n. So it's 100 + 4, so 104.
I would send char(104) at the beginning of your string, as a sentinal value
then the string with it right after so it'd appear similar to
char(104)abcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcde\r\n\r\n
Then use recv's peek_MSG ability to get the first char, make your string size, read only that value and whatever's left get's discarded by a socket flush call you make.

Socket send data but the recv program dont work well

i created 2 programs, one to send and the other to recv the data.
So,
The portion to recv data is:
while ((recvMsgSize = sock->recv(echoBuffer, RCVBUFSIZE))>0) {
write(fileno(stdout), echoBuffer, recvMsgSize);
}
If i use it to recv a large file data, it works well, with small amount of data it dont work.
I know the problem is with the recv portion because if i use netcat to recv data it works well, it recv the entire data.
Is there any other way to receive the data?
Thanks
I would guess your socket is blocking and recv is waiting for RCVBUFSIZE bytes to be sent. You should send the size of the file that is going to be sent first and then count how much data you've received and only request the remaining portion when what you're missing is less than RCVBUFSIZE bytes.