I don't see these sort of question asked. Which is odd because of good benefits from single threaded server applications. But how would I be able to implement a timeout system in my code when the server is in nonblock state?
Currently I'm using this method.
while(true)
{
FD_ZERO(&readfds);
FD_SET(server_socket, &readfds);
for (std::size_t i = 0; i < cur_sockets.size(); i++)
{
uint32_t sd = cur_sockets.at(i).socket;
if(sd > 0)
FD_SET(sd, &readfds);
if(sd > max_sd){
max_sd = sd;
}
}
int activity = select( max_sd + 1 , &readfds, NULL , NULL, NULL);
if(activity < 0)
{
continue;
}
if (FD_ISSET(server_socket, &readfds))
{
struct sockaddr_in cli_addr;
uint32_t newsockfd = (uint_fast32_t)accept((int)server_socket,
(struct sockaddr *) &cli_addr,
&clientlength);
if(newsockfd < 1) {
continue;
}
//Ensure we can even accept the client...
if (num_clients >= op_max_clients) {
close(newsockfd);
continue;
}
fcntl(newsockfd, F_SETFL, O_NONBLOCK);
/* DISABLE TIMEOUT EXCEPTION FROM SIGPIPE */
#ifdef __APPLE__
int set = 1;
setsockopt(newsockfd, SOL_SOCKET, SO_NOSIGPIPE, (void *) &set, sizeof(int));
#elif __LINUX__
signal(SIGPIPE, SIG_IGN);
#endif
/* ONCE WE ACCEPTED THE CONNECTION ADD CLIENT TO */
num_clients++;
client_con newCon;
newCon.socket = newsockfd;
time_t ltime;
time(<ime);
newCon.last_message = (uint64_t) ltime;
cur_sockets.push_back(newCon);
}
handle_clients();
}
As you can tell, I've added a unix timestap to the client when they successfully connected. I was thinking of maybe adding another thread that sleeps every 1 second, and simply checks if any clients haven't made any sends for the max duration, but I'm afraid I'll run into bottlenecking because of the second thread locking up constantly when dealing with large amounts of connections.
Thank you,
Ajm.
The last argument for select is the timeout for the select call and the return code of select tells you, if it returned because a socket was ready or because of a timeout.
In order to implement your own timeout handling for all sockets you could have a time stamp for each socket and update it on any socket operation. Then before calling select compute the timeout for each socket and use the minimal value for the timeout of the select call. This is just the basic idea and one can implement it more efficient so that you don't need to recompute all timeouts before calling select. But I consider a separate thread overkill.
Related
I have a program that sends arp packets to all hosts in a network to figure out which ones are up, which works fine so far.
I just have some difficulties setting a proper timeout for the reply, so far I got this:
struct timeval tv;
tv.tv_sec = 0;
tv.tv_usec = 10000; //0.01sec
if (setsockopt(sock_fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv)) < 0) {
...
}
...
if ((bytes = sendto(sock_fd, ether_frame, packet_length, 0, (struct sockaddr *) &device, sizeof (device))) <= 0) {
//error handle
} else {
if ((bytes = recv(sock_fd, packet, sizeof(packet), 0)) <= 0) {
...
} else {
...
When running the program with the above set value of 0.01sec, I sometimes get all replies and sometimes some replies are missing. When I increase the value to e.g. 1sec I get all values. But since there are a lot of requests to send, with 1sec it takes a huge amount of time till it finishes.
Is there a better way to calculate the timeout for the replies? Probably dynamically?
I am trying to set a 100ms timeout on a UDP Socket. I am using C. I have posted relavent pieces of my code below. I am not sure why this is not timing out, but just hangs when it doesn't receive a segment. Does this only work on sockets that are not bound using the bind() method?
#define TIMEOUT_MS 100 /* Seconds between retransmits */
if ((rcv_sock = socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP)) < 0)
DieWithError("socket() failed");
if ((rcv_sock = socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP)) < 0)
DieWithError("socket() failed");
//set timer for recv_socket
static int timeout = TIMEOUT_MS;
setsockopt(rcv_sock, SOL_SOCKET, SO_RCVTIMEO,(char*)&timeout,sizeof(timeout));
if(recvfrom(rcv_sock, ackBuffer,sizeof(ackBuffer), 0,
(struct sockaddr *) &servAddr2, &fromSize) < 0){
//timeout reached
printf("Timout reached. Resending segment %d\n", seq_num);
num_timeouts++;
}
The SO_RCVTIMEO option expects a struct timeval defined in sys/time.h, not an integer like you're passing to it. The timeval struct has as field for seconds and a field for microseconds. To set the timeout to 100ms, the following should do the trick:
struct timeval tv;
tv.tv_sec = 0;
tv.tv_usec = 100000;
if (setsockopt(rcv_sock, SOL_SOCKET, SO_RCVTIMEO,&tv,sizeof(tv)) < 0) {
perror("Error");
}
I have the same problem. I tried to adopt the solution you suggested, using the timeval struct. But it did not seem to work.
I have read on the Microsoft documentation and the time should be a DWORD with the number of milliseconds, but there is also another thing to do, If the socket is created using the WSASocket function, then the dwFlags parameter must have the WSA_FLAG_OVERLAPPED attribute set for the timeout to function properly.
Otherwise the timeout never takes effect.
I have written simple C/S applications to test the characteristics of non-blocking sockets, here is some brief information about the server and client:
//On linux The server thread will send
//a file to the client using non-blocking socket
void *SendFileThread(void *param){
CFile* theFile = (CFile*) param;
int sockfd = theFile->GetSocket();
set_non_blocking(sockfd);
set_sock_sndbuf(sockfd, 1024 * 64); //set the send buffer to 64K
//get the total packets count of target file
int PacketCOunt = theFile->GetFilePacketsCount();
int CurrPacket = 0;
while (CurrPacket < PacketCount){
char buffer[512];
int len = 0;
//get packet data by packet no.
GetPacketData(currPacket, buffer, len);
//send_non_blocking_sock_data will loop and send
//data into buffer of sockfd until there is error
int ret = send_non_blocking_sock_data(sockfd, buffer, len);
if (ret < 0 && errno == EAGAIN){
continue；
} else if (ret < 0 || ret == 0 ){
break;
} else {
currPacket++;
}
......
}
}
//On windows, the client thread will do something like below
//to receive the file data sent by the server via block socket
void *RecvFileThread(void *param){
int sockfd = (int) param; //blocking socket
set_sock_rcvbuf(sockfd, 1024 * 256); //set the send buffer to 256
while (1){
struct timeval timeout;
timeout.tv_sec = 1;
timeout.tv_usec = 0;
fd_set rds;
FD_ZERO(&rds);
FD_SET(sockfd, &rds)'
//actually, the first parameter of select() is
//ignored on windows, though on linux this parameter
//should be (maximum socket value + 1)
int ret = select(sockfd + 1, &rds, NULL, NULL, &timeout );
if (ret == 0){
// log that timer expires
CLogger::log("RecvFileThread---Calling select() timeouts\n");
} else if (ret) {
//log the number of data it received
int ret = 0;
char buffer[1024 * 256];
int len = recv(sockfd, buffer, sizeof(buffer), 0);
// handle error
process_tcp_data(buffer, len);
} else {
//handle and break;
break;
}
}
}
What surprised me is that the server thread fails frequently because of socket buffer full, e.g. to send a file of 14M size it reports 50000 failures with errno = EAGAIN. However, via logging I observed there are tens of timeouts during the transfer, the flow is like below:
on the Nth loop, select() succeeds and read 256K's data successfully.
on the (N+1)th loop, select() failed with timeout.
on the (N+2)th loop, select() succeeds and read 256K's data successfully.
Why there would be timeouts interleaved during the receving? Can anyone explain this phenomenon?
[UPDATE]
1. Uploading a file of 14M to the server only takes 8 seconds
2. Using the same file with 1), the server takes nearly 30 seconds to send all data to the client.
3. All sockets used by the client are blocking. All sockets used by the server are non-blocking.
Regarding #2, I think timeouts are the reason why #2 takes much more time then #1, and I wonder why there would be so many timeouts when the client is busy in receiving data.
[UPDATE2]
Thanks for comments from #Duck, #ebrob, #EJP, #ja_mesa , I will do more investigation today
then update this post.
Regarding why I send 512 bytes per loop in the server thread, it is because I found the server thread sends data much faster than the client thread receiving them. I am very confused that why timeout happened to the client thread.
Consider this more of a long comment than an answer but as several people have noted the network is orders of magnitude slower than your processor. The point of non-blocking i/o is that the difference is so great that you can actually use it to do real work rather than blocking. Here you are just pounding on the elevator button hoping that makes a difference.
I'm not sure how much of your code is real and how much is chopped up for posting but in the server you don't account for (ret == 0) i.e. normal shutdown by the peer.
The select in the client is wrong. Again, not sure if that was sloppy editing or not but if not then the number of parameters are wrong but, more concerning, the first parameter - i.e. should be the highest file descriptor for select to look at plus one - is zero. Depending on the implementation of select I wonder if that is in fact just turning select into a fancy sleep statement.
You should be calling recv() first and then call select() only if recv() tells you to do so. Don't call select() first, that is a waste of processing. recv() knows if data is immediately available or if it has to wait for data to arrive:
void *RecvFileThread(void *param){
int sockfd = (int) param; //blocking socket
set_sock_rcvbuf(sockfd, 1024 * 256); //set the send buffer to 256
char buffer[1024 * 256];
while (1){
int ret = 0;
int len = recv(sockfd, buffer, sizeof(buffer), 0);
if (len == -1) {
if (WSAGetLastError() != WSAEWOULDBLOCK) {
//handle error
break;
}
struct timeval timeout;
timeout.tv_sec = 1;
timeout.tv_usec = 0;
fd_set rds;
FD_ZERO(&rds);
FD_SET(sockfd, &rds)'
//actually, the first parameter of select() is
//ignored on windows, though on linux this parameter
//should be (maximum socket value + 1)
int ret = select(sockfd + 1, &rds, NULL, &timeout );
if (ret == -1) {
// handle error
break;
}
if (ret == 0) {
// log that timer expires
break;
}
// socket is readable so try read again
continue;
}
if (len == 0) {
// handle graceful disconnect
break;
}
//log the number of data it received
process_tcp_data(buffer, len);
}
}
Do something similar on the sending side as well. Call send() first, and then call select() waiting for writability only if send() tells you to do so.
I'm doing a c++ project that requires a server to create a new thread to handle connections each time accept() returns a new socket descriptor. I am using select to decide when a connection attempt has taken place as well as when a client has sent data over the newly created client socket (the one that accept creates). So two functions and two selects - one for polling the socket dedicated to listening for connections, one for polling the socket created when a new connection is successful.
The behavior of the first case is what I expect - FD_ISSET returns true for the id of my listening socket only when a connection is requested, and is false until the next connection attempt. The second case does not work, even though the code is exactly the same with different fd_set and socket objects. I'm wondering if this stems from the TCP socket? Do these sockets always return true when polled by a select due to their streamy nature?
//working snippet
struct timeval tv;
tv.tv_sec = 0;
tv.tv_usec = 500000;
fd_set readfds;
FD_ZERO(&readfds);
FD_SET(sid,&readfds);
//start server loop
for(;;){
//check if listening socket has any client requrests, timeout at 500 ms
int numsockets = select(sid+1,&readfds,NULL,NULL,&tv);
if(numsockets == -1){
if(errno == 4){
printf("SIGINT recieved in select\n");
FD_ZERO(&readfds);
myhandler(SIGINT);
}else{
perror("server select");
exit(1);
}
}
//check if listening socket is ready to be read after select returns
if(FD_ISSET(sid, &readfds)){
int newsocketfd = accept(sid, (struct sockaddr*)&client_addr, &addrsize);
if(newsocketfd == -1){
if(errno == 4){
printf("SIGINT recieved in accept\n");
myhandler(SIGINT);
}else{
perror("server accept");
exit(1);
}
}else{
s->forkThreadForClient(newsocketfd);
}
}
//non working snippet
//setup clients socket with select functionality
struct timeval ctv;
ctv.tv_sec = 0;
ctv.tv_usec = 500000;
fd_set creadfds;
FD_ZERO(&creadfds);
FD_SET(csid,&creadfds);
for(;;){
//check if listening socket has any client requrests, timeout at 500 ms
int numsockets = select(csid+1,&creadfds,NULL,NULL,&ctv);
if(numsockets == -1){
if(errno == 4){
printf("SIGINT recieved in client select\n");
FD_ZERO(&creadfds);
myhandler(SIGINT);
}else{
perror("server select");
exit(1);
}
}else{
printf("Select returned %i\n",numsockets);
}
if(FD_ISSET(csid,&creadfds)){
//read header
unsigned char header[11];
for(int i=0;i<11;i++){
if(recv(csid, rubyte, 1, 0) != 0){
printf("Received %X from client\n",*rubyte);
header[i] = *rubyte;
}
}
Any help would be appreciated.
Thanks for the responses, but I don't believe it has much todo with the timeout value being inside the loop. I tested it and even with tv being reset and the fd_set being zeroed every time the server loops, select still returns 1 immediately. I feel like there's a problem with how select is treating my TCP socket. Any time I set selects highest socket id to encompass my TCP socket, it returns immediately with that socket set. Also, client does not send anything, just connects.
One thing you must do is reset the value of tv to your desired timeout every time before you call select(). The select() function changes the values in tv to indicate how much time is left in the timeout, after returning from the function. If you fail to do this, your select() calls will end up using a timeout of zero, which is not efficient.
Some other operating systems implement select() differently, in such a way that they don't change the value of tv. Linux does change it, so you must reset it.
Move
FD_ZERO(&creadfds);
FD_SET(csid,&creadfds);
into the loop. The function select() reports the result in this structure. You already retrieve the result with
FD_ISSET(csid,&creadfds);
One of my projects on Linux uses blocking sockets. Things happen very serially so non-blocking would just make things more complicated. Anyway, I am finding that often a recv() call is returning -1 with errno set to EAGAIN.
The man page only really mentions this happening for non-blocking sockets, which makes sense. With non-blocking, the socket may or may not be available so you might need to try again.
What would cause it to happen for a blocking socket? Can I do anything to avoid it?
At the moment, my code to deal with it looks something like this (I have it throw an exception on error, but beyond that it is a very simple wrapper around recv()):
int ret;
do {
ret = ::recv(socket, buf, len, flags | MSG_NOSIGNAL);
} while(ret == -1 && errno == EAGAIN);
if(ret == -1) {
throw socket_error(strerror(errno));
}
return ret;
Is this even correct? The EAGAIN condition gets hit pretty often.
EDIT: some things which I've noticed which may be relevant.
I do set a read timeout on the socket using setsockopts(), but it is set to 30 seconds. the EAGAIN's happen way more often than once every 30 secs. CORRECTION my debugging was flawed, EAGAIN's don't happen as often as I thought they did. Perhaps it is the timeout triggering.
For connecting, I want to be able to have connect timeout, so I temporarily set the socket to non-blocking. That code looks like this:
int error = 0;
fd_set rset;
fd_set wset;
int n;
const SOCKET sock = m_Socket;
// set the socket as nonblocking IO
const int flags = fcntl (sock, F_GETFL, 0);
fcntl(sock, F_SETFL, flags | O_NONBLOCK);
errno = 0;
// we connect, but it will return soon
n = ::connect(sock, addr, size_addr);
if(n < 0) {
if (errno != EINPROGRESS) {
return -1;
}
} else if (n == 0) {
goto done;
}
FD_ZERO(&rset);
FD_ZERO(&wset);
FD_SET(sock, &rset);
FD_SET(sock, &wset);
struct timeval tval;
tval.tv_sec = timeout;
tval.tv_usec = 0;
// We "select()" until connect() returns its result or timeout
n = select(sock + 1, &rset, &wset, 0, timeout ? &tval : 0);
if(n == 0) {
errno = ETIMEDOUT;
return -1;
}
if (FD_ISSET(sock, &rset) || FD_ISSET(sock, &wset)) {
socklen_t len = sizeof(error);
if (getsockopt(SOL_SOCKET, SO_ERROR, &error, &len) < 0) {
return -1;
}
} else {
return -1;
}
done:
// We change the socket options back to blocking IO
if (fcntl(sock, F_SETFL, flags) == -1) {
return -1;
}
return 0;
The idea is that I set it to non-blocking, attempt a connect and select on the socket so I can enforce a timeout. Both the set and restore fcntl() calls return successfully, so the socket should end up in blocking mode again when this function completes.
It's possible that you have a nonzero receive timeout set on the socket (via setsockopt(sock, SOL_SOCKET, SO_RCVTIMEO,...)) as that would also cause recv to return EAGAIN
Is it possible that you're using MSG_DONTWAIT is being specified as part of your flags? The man page says EAGAIN will occur if no data is available and this flag is specified.
If you really want to force a block until the recv() is somewhat successful, you may wish to use the MSG_WAITALL flag.
I don't suggest this as a first-attempt fix, but if you're all out of options, you can always select() on the socket with a reasonably long timeout to force it to wait for data.
EAGAIN is generated by the OS almost like an "Oops! I'm sorry for disturbing you.". In case of this error, you may try reading again, This is not a serious or fatal error. I have seen these interrupts occur in Linux and LynxOS anywhere from one a day to 100 times a day.