The title probably explains itself best.
Anyway, I have a data buffer received from an another source, and I want to send it in a single UDP packet that contains a sequence number (as the first byte) -> I want to add the sequence number to the given buffer!
Instead of allocating a new buffer, settings it's size to size+4, setting the sequence number as the first byte and copying the data into the buffer, I would like to just use the scatter gather mechanism of WSA.
Sadly though, no WSA document specifies explicitly that WSASend guarantees that all buffer will be sent a single packet (The packet size will be held as < 1500 bytes).
Can I be certain that it will work that way? Or should I re-build the packet?
Best,
Daniel
It is documented in a round-about way:
For message-oriented sockets, do not exceed the maximum message size of the underlying provider, which can be obtained by getting the value of socket option SO_MAX_MSG_SIZE. If the data is too long to pass atomically through the underlying protocol the error WSAEMSGSIZE is returned, and no data is transmitted.
So clearly it combines the data from the buffers to make a single UDP packet. If it didn't then there would no point in returning the WSAEMSGSIZE error.
Related
I'm trying to implement a simple client-server application where a client or the server can send a dynamic number of bytes in a single write() call.
For example, let's assume that the client sends a byte stream of 1500 bytes. And server reads every 1000 bytes.
int BUFFER_SIZE = 1000;
...
read( iSockFD, cBuffer, BUFFER_SIZE );
I can use a loop and call read until its return value is 0. But the client may have multiple write() calls in a loop (i.e. sending multiple messages).
My question is, will it affect the read() on the server side? Meaning, will two consecutive write() of 1000 bytes in the client side, be read by a single read() with 2000 bytes buffer size at the server side?
If that's the case, what are the recommended ways of implementing such a scenario? Should I use a separator for messages (using an encoding algorithm)?
I understand this more related to sockets itself rather than C++. But, your help and guidance are highly appreciated.
UPDATE:
The intention is to implement a simple middleware system to send different types of messages, where the messages will be encoded in binary before sending.
Nobody can guarantee you that a write(x) will trigger a read(x) at the receiver side. If x is larger than your socket receive buffer, or if you call read() before the entire message has been received in the socket receive buffer, then read() will only return a fraction of the data and require you to issue a subsequent read() to get the rest.
The recommended way of doing this would be to define a message buffer of sufficient size. Every call to read() will return 1 or more bytes, which you keep enqueuing into the buffer. Now, once the buffer is larger than 4 bytes + the be32toh(integer) stored in the first 4 bytes of the buffer, you have to consume the integer plus the following x bytes from the beginning of the buffer (and process them further). This will allow you to nicely handle cases where read() contains the end of a previously unfinished packet and at the same time contains the beginning of the next (incomplete) packet.
Just make sure that every payload you transmit is prepended by a htobe32(length).
I have a UDP socket in blocking mode, I have bursts of packets and some are getting lost.
How can I find out current used size in receive buffer in winsock?
How can I understand whether system is discarding packets?
WSAIoctl passed FIONREAD is documented this way:
If the socket passed in the s parameter is message oriented (for
example, type SOCK_DGRAM), FIONREAD returns the reports the total
number of bytes available to read, not the size of the first datagram
(message) queued on the socket.
I think this answers your first question. As for the second, I see no way to programmatically figure that out. You should use sequence numbers in your application to detect gaps, and then look at the receive buffer size and guess that if it's close to full, the losses are due to running out of buffer space.
The receive buffer size for any socket is given by calling getsockopt() with the SO_RCVBUF option
I don't see how you can distinguish at the receiver between a packet lost in the network and a packet discarded at the local host. All you can tell is that it didn't arrive, and you need a higher-level protocol than UDP to tell you so, a sequence-numbered protocol with ACKs or NACKs.
I have 3 components client-proxy-server, at times when the proxy gets heavily loaded the socket buffers configure to say 1 MB gets filled. Is there a way to read Entire buffer 1 MB in one shot and then process?
FYI:
all the data grams never goes beyond MTU size are in per-defined structural format, where in length of each packet is also added.
Proxy routes data in between client & server, so tried having Producer & consumer thread but problem is NOT solved
Short answer: no.
Long answer:
The Berkeley style socket implementation allows to receive or send only one packet per call. Therefore it is not possible to read a complete network stream and replay it at the other side.
One reason is that your UDP socket can receive data from several sources. The interface should be able to pass the meta information like sender socket address, and at least the packet size to the caller. This is bunch of data should be parsed and you would pick the packets that meet a criteria. Finally you could build the bunch of packets to send.
Since you have to have the possibility to check each packet, if the packet is really expected you need a function to read a packet from the bunch. This is the function recvfrom.
I cannot find the answer for this one: what will happen if I read from socket 4bytes (I set the limit for 4 bytes) but there are actually 256bytes awaiting to be read? Will they be lost or will they wait until the next call of read function?
If it's a TCP socket, then no data will get lost; it'll get queued up.
Bear in mind that you have to be prepared to deal with partial reads, i.e. where you get fewer bytes than requested and have to call read() again to get more.
It depends what kind of socket you use. If it is stream socket (created with SOCK_STREAM), then it supports a stream of data, and you can read it even by 1 byte (though it will be not efficient), on another side you may request 1024 bytes but get only 1. And that almost irrelevant by what portions sender put them into stream (there is dependency, but you should not rely on that). So with stream you need to define end of data by higher level protocol. You may send strings with \n at the end, or use zero terminated string, or send some bytes of size of coming data before that data.
On another side if you use datagram protocol (created with SOCK_DGRAM) you will get data by packets - whatever size sender sent them. If you provide smaller buffer than data available, it will be truncated and remaining data is discarded.
I have to implement a (client) socket which requires high throughput (> 800Mbps) and low latency running on Windows 7 server. Overlapped IO seems the way to go for high performance.
Read some documentation on the subject, as far as I can see the advantage of overlapped I/O is that you pass some structures with buffer to the OS and you are notified when they are filled.
No I am wondering what the common ways are to combine this with a packet based protocol (length delimited packets, header contains size of datablock)
Of course I can just read arbitrary chunks of data and copy the required number of bytes into a message struture. This means an additional copy action.
Second option might be passing the message structure as a buffer with the header size, after getting it back pass the same structure again to read the requested number of databytes. In this case the first chunk read is small, but the data is saved into the message structure, and while the data block read is pending the read of the next header block can be initialized.
Any experiences or ideas how to handle length delimited packets the most efficient ?
Thanks,
Check out scatter/gather I/O if you know the packet sizes.