Could anyone please help me on improving the performance of a udp receiver. I am only able to get 1Mb/s but need to enhance the performance to almost 5Mb/s. There are also missing of Logs because the receiver is not able to receive all the messages due to this less performance. Is there any tips on how we could we increase the performance. I am using socket calls to get the data packets.
#define MAX_PACKET_SIZE 65535
#define UPD_DATAGRAM_BUFFER_SIZE 1536
m_nSocket = socket(AF_INET, SOCK_DGRAM, 0);
/* Set socket buffer size */
int buffer_size = m_nBufferSize;
ret = setsockopt(m_nSocket, SOL_SOCKET, SO_RCVBUF, (char*) &buffer_size, sizeof(buffer_size));
ret = setsockopt(m_nSocket6, SOL_SOCKET, SO_RCVBUF, (char*) &buffer_size, sizeof(buffer_size));
/* Set socket timeout */
#if defined (WIN32) || defined (WIN64)
int timeout = m_nTimeout;
ret = setsockopt(m_nSocket, SOL_SOCKET, SO_RCVTIMEO, (char*) &timeout, sizeof(timeout));
ret = setsockopt(m_nSocket6, SOL_SOCKET, SO_RCVTIMEO, (char*) &timeout, sizeof(timeout));
#else
struct timeval timeout;
timeout.tv_sec = 0;
timeout.tv_usec = m_nTimeout * 1000; //must be in microseconds
ret = setsockopt(m_nSocket, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof(timeout));
ret = setsockopt(m_nSocket6, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof(timeout));
#endif
//bind
m_address.sin_family = AF_INET;
m_address.sin_addr.s_addr = htonl(INADDR_ANY);
m_address.sin_port = htons(m_nPort);
ret = bind(m_nSocket, (struct sockaddr*) &m_address, sizeof(m_address));
//receive data
recvfrom(m_nSocket, m_sBuffer, UPD_DATAGRAM_BUFFER_SIZE, 0, (struct sockaddr*) &m_address, &server_length);
Does increase in buffer size increase udp performance? What else could we do to increase udp performance?
Increasing the RCVBUF size will not make it faster but more reliable. If the RCVBUF is full, next incoming packets are dropped.
Details:
A recvfrom() call receives exactly one UDP packet which has - for IPv4 - a maximum size of 65535 bytes. UDP packets may be split into fragments, but this is hidden from the user.
If your sendto() call sends more bytes than a single recvfrom() receive buffer accepts, the remaining data will be dropped.
The SO_RCVBUF is accepting packets while no recvfrom() call is running. If you call recvfrom() it checks for a packet in the RCVBUF and only blocks and waits for a new packet if the receive buffer is empty.
If you have a sender sending huge amounts of data for example within a for loop, then it is likely you loose some data if your RCVBUF is not large enough and your recvfrom() calls are not fast enough (i.e. when processing packets between revfrom() calls).
UDP is not made for burst transfers. It even does not guarantee the delivery of a packet and the packet receive order may differ from the send order.
Maybe you should use TCP/IP ?
If you try to implement your own UDP based stream communication you could do the following:
1) Send UDP packets with a maximum size of about 1400 Bytes.
2) Add a 32 or 64-Bit Header to your UDP packets containing the stream offset where this packet belongs to (i.e. the first 1400 byte packet has a stream offset of 0 and the second one has an offset of 1400, the third one 2800 and so on)
3) The client allocates a buffer huge enough to store the whole transmission. Every packet is copied into this buffer at the location specified within the first 32 or 64 bit of the packet. (This sorts your packet)
4) The server sends only loads of - for example - 10 MiB and the client requests more data while it reads from the RCVBUF (using recvfrom()). So the server does not fill the RCVBUF and no packets are being dropped by the receiving machine. For maximum performance the client should request the next load while still receiving data from the previous load. (This makes sure the receive buffer does not overflow)
5) The client request the retransmission of any missing packets (this can be combined with the request in step 4) (This makes sure the transmission is complete and no packets are lost)
Why only 1400 bytes? Because you don't want to fragment your packets on high speed networks. (On fast networks the 16-bit packet-id can overflow within the reassembly timeframe and - if the checksum matches or is not set - fragments of different packets could be reassembled. Took me hours to find out why)
Related
I wasn't running into this problem on my local system (of course), but now that I am setting up a virtual server, I am having some issues with a part of my code.
In order to receive all data from a nonblocking TCP recv(), I have this function
ssize_t Server::recvAll(int sockfd, const void *buf, size_t len, int flags) {
// just showing here that they are non-blocking sockets
u_long iMode=1;
ioctlsocket(sockfd,FIONBIO,&iMode);
ssize_t result;
char *pbuf = (char *)buf;
while ( len > 0 ) {
result = recv(sockfd,pbuf,len,flags);
printf("\tRES: %d", result);
if ( result <= 0 ) break;
pbuf += result;
len -= result;
}
return result;
}
I noticed that recvAll will usually print RES: 1024 (1024 being the amount of bytes I'm sending) and it works great. But less frequently, there is data loss and it prints only RES: 400 (where 400 is some number greater than 0 and less than 1024) and my code does not work, as it expects all 1024 bytes.
I tried also printing WSAGetLastError() and also running in debug, but it looks like the program runs slow enough due to the print/debug that I don't come across this issue.
I assume this function works great for blocking sockets, but not non-blocking sockets.
Any suggestions on measurements I can take to make sure that I do receive all 1024 bytes without data loss on non-blocking sockets?
If you use non-blocking mode then you read all data that has already arrived to the system. Once you read out all data recv returns error and reason is depending on system:
EWOULDBLOCK (in posix system)
WSAEWOULDBLOCK in windows sockets system
Once you receive this error you need to wait arrival of another data. You can do it in several ways:
Wait with special function like select/poll/epoll
Sleep some time and try to recv again (user-space polling)
If you need to reduce delay select/poll/epoll is preferable. Sleep is much more simple to implement.
Also you need consider that TCP is stream protocol and does NOT keep framing. This means that you can send, for example, 256 bytes then another 256 bytes but receive 512 bytes at once. This also true in opposite way: you may send 512 bytes at once and receive 256 bytes with first read and another 256 bytes in next read.
How do I set the MTU value pragmatically?
Apparently there are three candidates: IP_MTU, TCP_MSS, and TCP_MAXSEG
I tried:
uint16_t tcp_mss = 2048 * 8;
setsockopt(sock_descriptor, IPPROTO_TCP, TCP_MSS, &tcp_mss, sizeof(tcp_mss));
My intention is to increase the amount of data the TCP buffer should have before sending the packet. TCP_CORK is in use.
Also helps if you know how to change the time from the apparent default of 200ms.
Based on my understanding, each socket is associated with two buffers, a send buffer and a receive buffer, so when I call the send() function, what happens is that the data to send will be placed into the send buffer, and it is the responsibility of Windows now to send the content of this send buffer to the other end.
In a blocking socket, the send() function does not return until the entire data supplied to it has been placed into the send buffer.
So what is the size of the send buffer?
I performed the following test (sending 1 GB worth of data):
#include <stdio.h>
#include <WinSock2.h>
#pragma comment(lib, "ws2_32.lib")
#include <Windows.h>
int main()
{
// Initialize Winsock
WSADATA wsa;
WSAStartup(MAKEWORD(2, 2), &wsa);
// Create socket
SOCKET s = socket(AF_INET, SOCK_STREAM, 0);
//----------------------
// Connect to 192.168.1.7:12345
sockaddr_in address;
address.sin_family = AF_INET;
address.sin_addr.s_addr = inet_addr("192.168.1.7");
address.sin_port = htons(12345);
connect(s, (sockaddr*)&address, sizeof(address));
//----------------------
// Create 1 GB buffer ("AAAAAA...A")
char *buffer = new char[1073741824];
memset(buffer, 0x41, 1073741824);
// Send buffer
int i = send(s, buffer, 1073741824, 0);
printf("send() has returned\nReturn value: %d\nWSAGetLastError(): %d\n", i, WSAGetLastError());
//----------------------
getchar();
return 0;
}
Output:
send() has returned
Return value: 1073741824
WSAGetLastError(): 0
send() has returned immediately, does this means that the send buffer has a size of at least 1 GB?
This is some information about the test:
I am using a TCP blocking socket.
I have connected to a LAN machine.
Client Windows version: Windows 7 Ultimate 64-bit.
Server Windows version: Windows XP SP2 32-bit (installed on Virtual Box).
Edit: I have also attempted to connect to Google (173.194.116.18:80) and I got the same results.
Edit 2: I have discovered something strange, setting the send buffer to a value between 64 KB and 130 KB will make send() work as expected!
int send_buffer = 64 * 1024; // 64 KB
int send_buffer_sizeof = sizeof(int);
setsockopt(s, SOL_SOCKET, SO_SNDBUF, (char*)send_buffer, send_buffer_sizeof);
Edit 3: It turned out (thanks to Harry Johnston) that I have used setsockopt() in an incorrect way, this is how it is used:
setsockopt(s, SOL_SOCKET, SO_SNDBUF, (char*)&send_buffer, send_buffer_sizeof);
Setting the send buffer to a value between 64 KB and 130 KB does not make send() work as expected, but rather setting the send buffer to 0 makes it block (this is what I noticed anyway, I don't have any documentation for this behavior).
So my question now is: where can I find a documentation on how send() (and maybe other socket operations) work under Windows?
After investigating on this subject. This is what I believe to be the correct answer:
When calling send(), there are two things that could happen:
If there are pending data which are below SO_SNDBUF, then send() would return immediately (and it does not matter whether you are sending 5 KB or you are sending 500 MB).
If there are pending data which are above or equal SO_SNDBUF, then send() would block until enough data has been sent to restore the pending data to below SO_SNDBUF.
Note that this behavior is only applicable to Windows sockets, and not to POSIX sockets. I think that POSIX sockets only use one fixed sized send buffer (correct me if I'm wrong).
Now back to your main question "What is the size of a socket send buffer in Windows?". I guess if you have enough memory it could grow beyond 1 GB if necessary (not sure what is the maximum limit though).
I can reproduce this behaviour, and using Resource Monitor it is easy to see that Windows does indeed allocate 1GB of buffer space when the send() occurs.
An interesting feature is that if you do a second send immediately after the first one, that call does not return until both sends have completed. The buffer space from the first send is released once that send has completed, but the second send() continues to block until all the data has been transferred.
I suspect the difference in behaviour is because the second call to send() was already blocking when the first send completed. The third call to send() returns immediately (and 1GB of buffer space is allocated) just as the first one did, and so on, alternating.
So I conclude that the answer to the question ("how large are the send buffers?") is "as large as Windows sees fit". The upshot is that, in order to avoid exhausting the system memory, you should probably restrict blocking sends to no more than a few hundred megabytes.
Your call to setsockopt() is incorrect; the fourth argument is supposed to be a pointer to an integer, not an integer converted to a pointer. Once this is corrected, it turns out that setting the buffer size to zero causes send() to always block.
To summarize, the observed behaviour is that send() will return immediately provided:
there is enough memory to buffer all the provided data
there is not a send already in progress
the buffer size is not set to zero
Otherwise, it will return once the data has been sent.
KB214397 describes some of this - thanks Hans! In particular it describes that setting the buffer size to zero disables Winsock buffering, and comments that "If necessary, Winsock can buffer significantly more than the SO_SNDBUF buffer size."
(The completion notification described does not quite match up to the observed behaviour, depending I guess on how you interpret "previously buffered send". But it's close.)
Note that apart from the risk of inadvertently exhausting the system memory, none of this should matter. If you really need to know whether the code at the other end has received all your data yet, the only reliable way to do that is to get it to tell you.
In a blocking socket, the send() function does not return until the entire data supplied to it has been placed into the send buffer.
That is not guaranteed. If there is available buffer space, but not enough space for the entire data, the socket can (and usually will) accept whatever data it can and ignore the rest. The return value of send() tells you how many bytes were actually accepted. You have to call send() again to send the remaining data.
So what is the size of the send buffer?
Use getsockopt() with the SO_SNDBUF option to find out.
Use setsockopt() with the SO_SNDBUF option to specify your own buffer size. However, the socket may impose a max cap on the value you specify. Use getsockopt() to find out what size was actually assigned.
I'wm writing an app, which transmits video and obviously uses UDP protocol fot this purpose.
So I am wondering how can I increase a size of send/recieve buffer, cause currently the maximal size of data, which I can send is 65000 bytes.
I already tried to do it in following way:
int option = 262144;
if(setsockopt(m_SocketHandle,SOL_SOCKET,SO_RCVBUF ,(char*)&option,sizeof(option)) < 0)
{
printf("setsockopt failed\n");
}
But it did not work. So how can I do it?
How can I do it?
You can't. The maximum size of an IPv4 UDP datagram is 65535-20-8=65507 bytes. Increasing the buffer size cannot change that. Datagrams larger than the path MTU (< 1500 bytes) will be fragmented, and fragmented datagrams are more likely to be lost, statistically, so using datagram sizes up around 64k is contra-indicated anyway.
In socket programming, some data is sent to the server, and as soon as server receives it sends the acknowledgement response message. it is more than 1 byte, so i check for more than one byte check while receiving, here i am losing around 120-200ms. Which is a very big issue. As client need to send ack back for this acknowledgement. I have sniffed to see data is arrived to my IP at the same time when server has sent. but recv or ioctlsocket(to check more than 1 byte is ready to be read) takes time to read more than one byte. How can i resolve this. The code is as follows.
DWORD RecvCount = 0;
char szBuff1[2048];
bool stop = false;
while(!stop)
{
ioctlsocket(*socket, FIONREAD, &RecvCount);
if(RecvCount > 1)
stop = true;
}
int Res = recv(*socket, szBuff1, RecvCount,0);
You should disable the Nagle algorithm on windows as otherwise the socket will sit on your data until the buffer is full (or at least wait a couple of hundred milliseconds before sending it anyway).
You do this by setting the TCP_NODELAY socket option:
int flag = 1;
int result = setsockopt(m_Socket,IPPROTO_TCP,TCP_NODELAY,(char *) &flag,sizeof(int));