the key is that I send 4096 bytes but only 119 bytes aprox. carry useful information.
The 100 bytes ends with \r\n\r\n so in the client, when I read \r\n\r\n I want to stop receiving information from that string, and start over.
I don't know if I have to flush, or close the socket, or whatever...
They are sockets TCP.
In the client I do:
buf details[4096];
strcpy(details,"1");
strcat(details,"10/04/12");
strcat(details,"Kevin Fire");
strcat(detils,"abcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcde\r\n\r\n");
nbytes_sent = send(sock,(char *)details,sizeof(details),0);
On the other hand, the server...
char buf[20];
memset(buf,'\0',20);
while(!end){
nbytes_read=recv(sclient,(char *)ress,sizeof(ress),0);
if(strcmp(ress,"1")==0){
printf("Details: %s (%i)\n",buf,nbytes_read);
while(strcmp(buf,"\r\n\r\n") != 0){
nbytes_read=recv(sclient,(char *)buf,sizeof(buf),0);
cout.flush();
printf("Details: %s (%i)\n",buf,nbytes_read);
} }
if(strcmp(buf,"\r\n\r\n")==0) printf("The End\n");
cout.flush();
}
}
I just want to read a new "ress" and not being retrieving the rest of bytes that are not useful.
Thanks in advance.
If you mean you want to discard rest of data and read a new block you can't do it with TCP because it is stream oriented and do not have a concept of message and have no idea about the rest of message that you want to ignore. if you mean something else please describe it more.
but beside that why you use nbytes_sent = send(sock,(char *)details,sizeof(details),0); when only data until \r\n' is important. you can usenbytes_sent = send(sock,(char *)details,strlen(details),0);` that only send valid data and reduce garbage that you send over network and you don't need to start over in the server??
I'm not sure if I'm following your question entirely, but it appears that you can just set end=true whenever you detect the end of the message you're receiving:
char buf[20];
memset(buf,'\0',20);
while(!end)
{
nbytes_read=recv(sclient,(char *)ress,sizeof(ress),0);
if(strcmp(ress,"1")==0)
{
printf("Details: %s (%i)\n",buf,nbytes_read);
while(strcmp(buf,"\r\n\r\n") != 0)
{
nbytes_read=recv(sclient,(char *)buf,sizeof(buf),0);
cout.flush();
printf("Details: %s (%i)\n",buf,nbytes_read);
}
}
if(strcmp(buf,"\r\n\r\n")==0)
{
end = true; // <--- This should do it for you, right?
printf("The End\n");
}
cout.flush();
}
However, if the client is still connected and writing the next message to the socket, then you just need to start reading the next message. So what happens with the client once the message is written? Does it start writing the next message or does it close the socket connection?
In addition: you need to take what's in your buffer and create a message from it. When the current message is done, then consider creating a new message with the contents of the buffer from the next message.
If you design your protocol like HTTP 1.0, where each request opens a new socket, then you close the socket after you've read enough.
Otherwise, you need to keep on reading until you skipped the entire 4096 bytes. The easiest thing to do is to keep on reading till you get 4096 bytes in the first place (you'll need to call recv in a loop), and then parse the contents of the buffer. Then again, you might be better off redesigning your protocol.
My thought would be to just to peek at the first x chars.
The 4 chars could be the size of the buffer expected.
So for example if your message is:
abcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcde\r\n\r\n
It's (to use your schematic) 100 bytes, plus the \r\n\r\n. So it's 100 + 4, so 104.
I would send char(104) at the beginning of your string, as a sentinal value
then the string with it right after so it'd appear similar to
char(104)abcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcde\r\n\r\n
Then use recv's peek_MSG ability to get the first char, make your string size, read only that value and whatever's left get's discarded by a socket flush call you make.
Related
I am writing a server-client application using Winsock in c++ for sending a file line by line and I have a problem in sending huge string. The line size is very huge.
For getting the message from the client by the server I use the code below.
int result;
char message[200];
while (true)
{
recv(newSd, (char*)&message, sizeof(message), 0);
cout << "The Message from client: " << message << ";";
}
The above code working fine if I send small length of the message. But, what I wanted is to send an unknown size of lines in a file.
How to send a big unknown string instead of char message[200];
TCP is a byte stream, it knows nothing about messages or lines or anything like that. When you send data over TCP, all it knows about is raw bytes, not what the bytes represent. It is your responsibility to implement a messaging protocol on top of TCP to delimit the data in some meaningful way so the receiver can know when the data is complete. There are two ways to do that:
send the data length before sending the actual data. The receiver reads the length first, then reads however many bytes the length says.
send a unique terminator after sending the data. Make sure the terminator never appears in the data. The receiver can then read until the terminator is received.
You are not handling either of those in your recv() code, so I suspect you are not handling either of them in your send() code, too (which you did not show).
Since you are sending a text file, you can either:
send the file size, such as in a uint32_t or uint64_t (depending on how large the file is), then send the raw file bytes.
send each text line individually as-is, terminated by a CRLF or bare-LF line break after each line, and then send a final terminator after the last line.
You are also ignoring the return value of recv(), which tells you how many bytes were actually received. It can, and usually does, return fewer bytes than requested, so you must be prepared to call recv() multiple times, usually in a loop, to receive data completely. Same with send().
i am trying to send data via tcp socket to a server. The idea behind that is a really simple chat programm.
The string I am trying to send looks like the following:
1:2:e9e633097ab9ceb3e48ec3f70ee2beba41d05d5420efee5da85f97d97005727587fda33ef4ff2322088f4c79e8133cc9cd9f3512f4d3a303cbdb5bc585415a00:2:xc_[z kxc_[z kxc_[z kxc_[==
As you can see there a few unprintable characters which I don't think are a problem here.
To send this data I am using the following code snippet.
bool tcp_client::send_data(string data)
{
if( send(sock , data.c_str(), strlen(data.c_str()) , 0) < 0)
{
perror("Send failed : ");
return false;
}
return true;
}
After a few minutes of trying things out I came up, that data.c_str() cuts my string of.
The result is:
1:2:e9e633097ab9ceb3e48ec3f70ee2beba41d05d5420efee5da85f97d97005727587fda33ef4ff2322088f4c79e8133cc9cd9f3512f4d3a303cbdb5bc585415a00:2:xc_[z
I think that there is some kind of null sequence inside my string which is a problem for the c_str() function.
Is there a way to send the whole string as I mentioned aboved without cutting it off?
Thanks.
Is there a way to send the whole string as I mentioned aboved without cutting it off?
What about:
send(sock , data.c_str(), data.size() , 0);
There are only two sane ways to send arbitrary data (such as a array of characters) over stream sockets:
On the server: close the socket after data was sent (like in ftp, http 0.9, etc). On the client - read until socket is closed in a loop.
On the server: prefix the data with fixed-length size (nowadays people usualy use 64 bit integers for size, watch out for endiannes). On the client - read the size first (in a loop!), than read the data until size bytes are read (in a loop).
Everything else is going to backfire sooner or later.
I've been reading through Beej's Guide to Network Programming to get a handle on TCP connections. In one of the samples the client code for a simple TCP stream client looks like:
if ((numbytes = recv(sockfd, buf, MAXDATASIZE-1, 0)) == -1) {
perror("recv");
exit(1);
}
buf[numbytes] = '\0';
printf("Client: received '%s'\n", buf);
close(sockfd);
I've set the buffer to be smaller than the total number of bytes that I'm sending. I'm not quite sure how I can get the other bytes. Do I have to loop over recv() until I receive '\0'?
*Note on the server side I'm also implementing his sendall() function, so it should actually be sending everything to the client.
See also 6.1. A Simple Stream Server in the guide.
Yes, you will need multiple recv() calls, until you have all data.
To know when that is, using the return status from recv() is no good - it only tells you how many bytes you have received, not how many bytes are available, as some may still be in transit.
It is better if the data you receive somehow encodes the length of the total data. Read as many data until you know what the length is, then read until you have received length data. To do that, various approaches are possible; the common one is to make a buffer large enough to hold all data once you know what the length is.
Another approach is to use fixed-size buffers, and always try to receive min(missing, bufsize), decreasing missing after each recv().
The first thing you need to learn when doing TCP/IP programming: 1 write/send call might take
several recv calls to receive, and several write/send calls might need just 1 recv call to receive. And anything in-between.
You'll need to loop until you have all data. The return value of recv() tells you how much data you received. If you simply want to receive all data on the TCP connection, you can loop until recv() returns 0 - provided that the other end closes the TCP connection when it is done sending.
If you're sending records/lines/packets/commands or something similar, you need to make your own protocol over TCP, which might be as simple as "commands are delimited with \n".
The simple way to read/parse such a command would be to read 1 byte at a time, building up a buffer with the received bytes and check for a \n byte every time. Reading 1 byte is extremely inefficient, so you should read larger chunks at a time.
Since TCP is stream oriented and does not provide record/message boundaries it becomes a bit more tricky - you'd
have to recv a piece of bytes, check in the received buffer for a \n byte, if it's there - append the bytes to previously received bytes and output that message. Then check the remainder of the buffer after the \n - which might contain another whole message or just the start of another message.
Yes, you have to loop over recv() until you receive '\0' or an
error happen (negative value from recv) or 0 from recv().
For the first option: only if this zero is part of your
protocol (the server sends it). However from your code it seems that
the zero is just to be able to use the buffer content as a
C-string (on the client side).
The check for a return value of 0 from recv:
this means that the connection was closed (it could be part
of your protocol that this happens.)
I wrote a simple client-server program. Network.h is a header file which uses Winsock2.h (TCP/IP mode) to create socket, accept/connect in blocking mode, send/recv in non-blocking mode. I made it so that the function string TNetwork::Recv(int size) will return the string "Nothing" if it gets WSAWOULDBLOCK error (no data is received yet)
Here is my main function:
int main(){
string Ans;
TNetwork::StartUp(); //WSA start up, etc
cin >> Ans;
if (Ans == "0"){ // 0 --> server
TNetwork::SetupAsServer(); //accept connection (in blocking mode!)
while (true){
TNetwork::Send("\nAss" + '\0'); //without null terminator, the client may read extra bytes, causing undefined behavior (?)
TNetwork::Send("embly" + '\0');
cin >> Ans;
}
}
else{ // others --> regard Ans as IP address. e.g. I can type "127.0.0.1"
TNetwork::SetupAsClient(Ans);
string Rec;
while (true){
Rec = TNetwork::Recv(1000);
if (Rec != "Nothing"){
cout << Rec;
}
}
}
system("PAUSE");
}
Supposedly, the client would print "Assembly" when connected, and when the server enters anything to its console window. Sometimes, though, the client would only print out "\nAss" in the console without the "embly.
To my understanding, TCP/IP ensures all data to be sent and in the correct order, so I guess what happens is that both packets arrive at the same time, which happen quite often over the unstable internet. And due to this null terminator, the client would ignore the "embly", since the Recv() function stopped reading when it hits a null terminator.
So, how can I ensure that the client will always read all data packets correctly?
Yes, the network stack will send the data in the correct order and doesn't care what termination type you use. This has to do with how you're receiving and processing the data stream (note: not packets, stream). If you receive all 11 bytes and print it to the screen, the print function will stop when it reaches the zero, but the rest of the data is still there.
Note: since it's a stream, what happens if you received only 10 bytes of data from the stream? You need to scan what you receive for the zero to know if you've received a full "zero-terminated string" if that's how you want to communicate your data.
EDIT: Also, I don't think "\nAss" + '\0' is doing what you think it is. Instead of adding a 0 character to the end of the string (which already has one, by the way), it's adding 0 to your string pointer.
As #mark points out, TCP is all about streams, not packets. TCP takes care of ensuring that data is reliably transmitted from A to B and that the data is delivered to the consumer in the order in which it was transmitted. Yes, the data is packetized on the wire, but the TCP stack on the system takes those packets and builds the stream which it makes available to you through the recv() function. The TCP stack handles out-of-order data, missing data, and duplicated data such that by the time your application sees it, the stream is a mirror-copy of when the sender sent.
To properly receive TCP data, you will typically need some kind of loop that reads data from the socket when it becomes available. The way I normally do this is to have a thread that is dedicated to servicing the socket. In the thread function is a loop that reads data from the socket when it becomes available and is idle otherwise. This loop reads data into a buffer of, say, 1 KB. Once the data is received from the socket into this buffer, the buffer is copied to another thread for processing. In the thread function for the processing thread is a loop that receives the 1 KB buffers from the socket thread and adds them to the back end of a master buffer of, say, 1 MB. The processing thread then processes the messages out of this master buffer and makes them available to the application.
For a simple demo application, two threads may be overkill. The two threads I've described could be certainly be combined into one, but for my application, it is more efficient to have two threads and take advantage of the multiple cores on my system. The point is, if you're going to have a front-end UI, there's not going to be a way around using at least one thread and still have the UI be responsive.
One other thing. There are two commonly-used mechanisms for protocol design. You're using one, namely, a marker (e.g., a null terminator, etc.) to signal the begin/end of a message. I don't prefer this mechanism mainly because the marker may actually need to be part of the message at some point. The other mechanism is to have a header on each message that tells, at a minimum, how long the message is. I prefer this mechanism and include in my headers a sync word and the message type as well. For example,
struct Header
{
__int16 _sync; // a hex pattern, e.g., 0xABCD
__int16 _type;
__int32 _length;
}
That's a total of 8 bytes. So when processing from the master buffer, I read the first 8 bytes, verify the sync word, and get the length. I determine if there are 'length' bytes available in the master buffer. If not, I have to wait until the socket thread provides me more data before checking again. If so, I extract 'length' bytes from the master buffer and pass that to an object created according to the specified type, which knows how to interpret that particular message. Then repeat.
As I mentioned, I use a master buffer of 1 MB or so. As messages are processed, it is important to remove them from the master buffer so there is additional space available for new data on the back end. This involves simply copying the unprocessed data, if any, to the beginning of the buffer. In cases where data comes in faster than you can process it, the master buffer may need the ability to resize itself to accommodate the additional data.
I hope that's not overwhelming. Start simple and add as you go.
I've been reading through Beej's Guide to Network Programming to get a handle on TCP connections. In one of the samples the client code for a simple TCP stream client looks like:
if ((numbytes = recv(sockfd, buf, MAXDATASIZE-1, 0)) == -1) {
perror("recv");
exit(1);
}
buf[numbytes] = '\0';
printf("Client: received '%s'\n", buf);
close(sockfd);
I've set the buffer to be smaller than the total number of bytes that I'm sending. I'm not quite sure how I can get the other bytes. Do I have to loop over recv() until I receive '\0'?
*Note on the server side I'm also implementing his sendall() function, so it should actually be sending everything to the client.
See also 6.1. A Simple Stream Server in the guide.
Yes, you will need multiple recv() calls, until you have all data.
To know when that is, using the return status from recv() is no good - it only tells you how many bytes you have received, not how many bytes are available, as some may still be in transit.
It is better if the data you receive somehow encodes the length of the total data. Read as many data until you know what the length is, then read until you have received length data. To do that, various approaches are possible; the common one is to make a buffer large enough to hold all data once you know what the length is.
Another approach is to use fixed-size buffers, and always try to receive min(missing, bufsize), decreasing missing after each recv().
The first thing you need to learn when doing TCP/IP programming: 1 write/send call might take
several recv calls to receive, and several write/send calls might need just 1 recv call to receive. And anything in-between.
You'll need to loop until you have all data. The return value of recv() tells you how much data you received. If you simply want to receive all data on the TCP connection, you can loop until recv() returns 0 - provided that the other end closes the TCP connection when it is done sending.
If you're sending records/lines/packets/commands or something similar, you need to make your own protocol over TCP, which might be as simple as "commands are delimited with \n".
The simple way to read/parse such a command would be to read 1 byte at a time, building up a buffer with the received bytes and check for a \n byte every time. Reading 1 byte is extremely inefficient, so you should read larger chunks at a time.
Since TCP is stream oriented and does not provide record/message boundaries it becomes a bit more tricky - you'd
have to recv a piece of bytes, check in the received buffer for a \n byte, if it's there - append the bytes to previously received bytes and output that message. Then check the remainder of the buffer after the \n - which might contain another whole message or just the start of another message.
Yes, you have to loop over recv() until you receive '\0' or an
error happen (negative value from recv) or 0 from recv().
For the first option: only if this zero is part of your
protocol (the server sends it). However from your code it seems that
the zero is just to be able to use the buffer content as a
C-string (on the client side).
The check for a return value of 0 from recv:
this means that the connection was closed (it could be part
of your protocol that this happens.)