I want to avoid the TIME_WAIT state when closing a TCP socket (I am aware of the pros and cons of circumventing TIME_WAIT).
I am using Windows and WinSock2/.Net sockets and am having great difficulty getting the SO_LINGER socket option to work as described in the documentation.
My test code with most of the error checking removed for brevity is:
#include <winsock2.h>
#include <ws2tcpip.h>
#include <iostream>
int main()
{
std::cout << "starting..." << std::endl;
WSADATA w = { 0 };
int error = WSAStartup(0x0202, &w);
if (error || w.wVersion != 0x0202) {
std::cerr << "Could not initialise Winsock2." << std::endl;
return -1;
}
auto clientSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
// Set socket options
linger lingerOpt = { 1, 0 };
setsockopt(clientSocket, SOL_SOCKET, SO_LINGER, (char*)&lingerOpt, sizeof(lingerOpt));
linger checkLingerOpt{ 0 };
int optLen = sizeof(checkLingerOpt);
int getOptResult = getsockopt(clientSocket, SOL_SOCKET, SO_LINGER, (char*)&checkLingerOpt, &optLen);
if (getOptResult < 0) {
wprintf(L"Failed to get SO_LINGER socket option on client socket, error: %ld\n", WSAGetLastError());
}
else {
std::cout << "Linger option set to onoff " << checkLingerOpt.l_onoff << ", linger seconds " << checkLingerOpt.l_linger << "." << std::endl;
}
// Bind local client socket.
sockaddr_in clientBindAddr;
clientBindAddr.sin_family = AF_INET;
clientBindAddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
clientBindAddr.sin_port = htons(15064);
bind(clientSocket, (SOCKADDR*)&clientBindAddr, sizeof (clientBindAddr));
sockaddr_in serverSockAddr;
serverSockAddr.sin_family = AF_INET;
serverSockAddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
serverSockAddr.sin_port = htons(5060);
// Connect to server.
connect(clientSocket, (SOCKADDR*)&serverSockAddr, sizeof (serverSockAddr));
std::cout << "connected." << std::endl;
Sleep(1000);
//shutdown(clientSocket, SD_BOTH);
closesocket(clientSocket);
std::cout << "finished." << std::endl;
}
Result:
starting...
Linger option set to onoff 1, linger seconds 0.
connected.
finished.
The sample above does avoid the TIME_WAIT state but does so because the client socket sends a RST packet.
If the Linger option is changed to:
linger lingerOpt = { 1, 5 };
Result
starting...
Linger option set to onoff 1, linger seconds 5.
connected.
finished.
Then closing the socket does result in a TIME_WAIT but of 30s which is the same result as not setting the SO_LINGER option.
Another observation is that if the socket is shutdown (which is the recommended way to cleanly close) with shutdown(clientSocket, SD_BOTH); then the Linger option of {1,0} will have no affect.
In summary:
Set Linger option as {1,0} & close with closesocket => RST.
Set Linger option as {1,5} & close with closesocket => FIN-ACK & TIME_WAIT of 30s.
Set Linger option as {1,0} & close with shutdown, closesocket => FIN-ACK & TIME_WAIT of 30s.
Set Linger option as {1,5} & close with shutdown, closesocket => FIN-ACK & TIME_WAIT of 30s.
What I'd like is:
Set Linger option as {1,0} & close with shutdown, closesocket => FIN-ACK & TIME_WAIT of 0s.
Update: As pointed out in the closesocket reference by Remy Lebeau a Linger option of {nonzero,0} is hard coded to generate a RST.
A short TIME_WAIT of a few seconds would be just as good, i.e. a linger option of {1,1} caused closesocket to exit gracefully with a 1s TIME_WAIT period, and which according to the closesocket documentation should be possible.
Update 2: As again pointed out by Remy Lebeau the Linger option and TIME_WAIT period are NOT linked. If you're reading this you probably made the same mistake I did and were trying to shorten the TIME_WAIT period via setsockopt and SO_LINGER.
By all accounts that can't be done and in cases where careful consideration judges TIME_WAIT needs to be avoided (such as in my case where the application layer protocol can deal with stray or orphaned TCP data packets) the ideal option looks to be a Linger setting of {1,0} to force a hard RST socket close which will allow the connection to be immediately re-attempted without the OS blocking the attempt.
You can't really avoid TIME_WAIT when your app is the one closing the TCP connection first (TIME_WAIT does not happen when the peer closes the connection first). No amount of SO_LINGER settings will change that fact, other than performing an abortive socket closure (ie sending a RST packet). It is simply part of how TCP works (look at the TCP state diagram). SO_LINGER simply controls how long closesocket() waits before actually closing an active connection.
The only way to prevent the socket from entering the TIME_WAIT state is to set the l_linger duration to 0, and don't call shutdown(SD_SEND) or shutdown(SD_BOTH) at all (calling shutdown(SD_RECEIVE) is OK). This is documented behavior:
The closesocket call will only block until all data has been delivered to the peer or the timeout expires. If the connection is reset because the timeout expires, then the socket will not go into TIME_WAIT state. If all data is sent within the timeout period, then the socket can go into TIME_WAIT state.
If the l_onoff member of the linger structure is nonzero and the l_linger member is a zero timeout interval on a blocking socket, then a call to closesocket will reset the connection. The socket will not go to the TIME_WAIT state.
The real problem with your code (aside from the lack of error handling) is that your client is bind()'ing a client socket before connect()'ing it to a server. Typically, you should not bind() a client socket at all, you should let the OS choose an appropriate binding for you. However, if you must bind() a client socket, you will likely need to enable the SO_REUSEADDR option on that socket to avoid being blocked when a previous connection boudn to the same local IP/Port is still in TIME_WAIT state and you are trying to connect() in a short amount of time after the previous closesocket().
See How to avoid TIME_WAIT state after closesocket() ? for more details. Also, the document you linked to in your question also explains ways to avoid TIME_WAIT without resorting to messing with SO_LINGER.
Related
I am working on robot control project where I should primarily (1)send some data about speed and fixed positions to the robot then (2)start a realtime control using my PC and repeat.
I chose to use TCP/IP for the first step and for realtime UDP/IP.
I tried to code this, but I believe something is missing.
I have some problems:
1- When there is no incoming (clients) connections accept function didn't return valid socket nor Message[10] is printed!!
2- Is it ok to Sendto using the same server IP and port?
3- Is this the right way to switch between UDP and TCP back and forth or there are some considerations should be undertaken?
while(Comm_Loop)
{
while (TCP_Loop)
{
TCP_Client = accept(TCP_Server, (struct sockaddr*) &Client_Address, &Client_Address_Length) ;
if ( TCP_Client != INVALID_SOCKET)
{
if (Data_Trans)
{
State = send(TCP_Client, Msg, sizeof(Msg), 0) ;
if (State != SOCKET_ERROR)
TCP_Loop = 0 ;
else
cout << Message[8] ;
}
}
else cout << Message[10] ;
}
Check_Send = sendto(UDP_Socket,
UDP_Send_Packet,
sizeof(UDP_Send_Packet),
0,
(LPSOCKADDR) &Socket_Address,
sizeof(Socket_Address));
fd_set SockSet ;
timeval Time_Out ;
FD_ZERO(&SockSet) ;
FD_SET(UDP_Socket, &SockSet);
Time_Out.tv_sec = 5;
Time_Out.tv_usec = 0 ;
State = select(0, &SockSet, (fd_set *)NULL, (fd_set *)NULL, &Time_Out) ;
if((State > 0) && (FD_ISSET(UDP_Socket, &SockSet) != 0))
{
Check_Recieve = recvfrom(UDP_Socket,
UDP_Recieve_Packet,
Buffer_Length,
0,
NULL,
NULL);
TCP_Loop = 1 ;
}
else
{
// Reception timeout
Comm_Loop = 0;
}
}
closesocket(TCP_Client) ;
closesocket(TCP_Server) ;
closesocket(UDP_Socket) ;
WSACleanup() ;
return 0;
}
1- When there is no incoming (clients) connections accept function
didn't return valid socket nor Message[10] is printed!!
Normal behavior by accept() is to block (i.e. not return) until either an incoming TCP connection is received, or an error occurs. If you don't want accept() to block, you can set the TCP_Server socket to be non-blocking, so that instead of blocking it returns -1 and sets errno to EWOULDBLOCK instead.
As far as Message[10] not printing, keep in mind that the stdout/cout stream is buffered, which means any text you send to it won't actually show up in the Terminal window until either you've printed a newline character ('\n' or std::endl) or manually flushed the stream. So if you aren't seeing the output you expected in a timely manner, you might want to verify that your Message[10] string contains a newline character, or alternatively append << std::endl; to the end of your cout << blah calls.
2- Is it ok to Sendto using the same server IP and port?
That question is ambiguous -- same server IP and port as what?
If you are asking if you can send your UDP sockets to the same IP address and port that were set in the accept() call (i.e. Client_Address in your code), you can do that, but it will only work if your robot has set up a UDP socket to bind to that port number in the UDP port-space. (Note that a UDP socket and a TCP socket can both bind to the same port without interfering with each other since they have different port-spaces, but by the same token you won't ever see TCP-data appear on your UDP socket or vice versa)
3- Is this the right way to switch between UDP and TCP back and forth
or there are some considerations should be undertaken?
It looks like it could more or less work, although a more robust design would be one that uses non-blocking sockets and always passes all of the sockets (i.e. the original TCP socket that you pass in to accept(), the per-TCP-connection TCP socket(s) that were returned by accept(), and your UDP socket) into every select call's read-fd_set, and then always checks all of those sockets (via FD_ISSET()) after select() returns and reacts appropriately when any of them is ready-for-read. That way your program could always handle incoming TCP data while simultaneously handling incoming UDP data, rather than switching from a mode where it handles only TCP to a mode where it handles only UDP (and ignoring any data that doesn't correspond to its current mode; because usually ignoring incoming data makes for a bad experience for the client/robot whose requests you are ignoring). This would also allow you to accept() incoming TCP connections at any time (since an incoming TCP connection would cause your TCP_Server socket to select as ready-for-read, at which point you could then call accept() on it again).
Right now, I am trying to specify options with setsockopt() using the following code:
// bind socket
// Use setsockopt() function to make sure the port is not in use
int yes = 1;
setsockopt(TCPSocket, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(int));
setsockopt(TCPSocket, SOL_SOCKET, SO_RCVTIMEO, (char *)&tv,sizeof(struct timeval));
status = bind(TCPSocket, host_info_list->ai_addr, host_info_list->ai_addrlen);
if (status == -1) std::cout << "bind error" << std::endl ;
// listen for connections
status = listen(TCPSocket, 5);
if (status == -1) std::cout << "listen error" << std::endl ;
int new_sd;
struct sockaddr_storage their_addr;
socklen_t addr_size = sizeof(their_addr);
new_sd = accept(TCPSocket, (struct sockaddr *)&their_addr, &addr_size);
if (new_sd == -1) std::cout << "listen error" << std::endl ;
Note tv is an already-specified timeval.
When I make only the first setsockopt() call, everything works fine. However, with the addition of the second (which does not return any errors), I encounter the second "listen error" specified in the code. I'm not sure why setting the timeout value affect this, can someone explain?
I do not take credit for the code specified; it is modified from the code presented in the tutorial here: http://codebase.eu/tutorial/linux-socket-programming-c/
If you see a TCP state diagram like this one you see there's a state called TIME_WAIT when actively closing a socket. This state can take some time before it ends, up to four minutes according to RFC793.
While the socket is in the TIME_WAIT you can not bind to an interface using the same address-port pair as the socket that is in the wait state. Setting the SO_REUSEADDR flag om a socket enables other sockets to bind to the address when the current socket (with the flag set) is in the TIME_WAIT state.
The SO_REUSEADDR option is most useful for server (passive, listening) sockets.
As for your problem, after each call to setsockopt check what it returns, and if it's -1 then you check errno to see what went wrong. You can use perror or strerror to print or get a printable string for the error, like
if (setsockopt(TCPSocket, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(int)) < 0)
{
std::cerr << "Error setting the SO_REUSEADDR: " << strerror(errno) << '\n';
// Do something appropriate
}
Joachim's solution did a great job of answering my intial question and explaining setsockopt(). To answer my own question after realizing the issue was further down in the code, the timeout affects the server being able to listen to a port. Say the timeout is only 10ms, the server must be started, then the client, and a connection must be established in that time. This wasn't happening in my case, thus the resulting error.
I'm using winsocks and I am coding an IDS/Honeypot, this is just a small section of it, because at the moment I want the server to listen on multiple sockets (7) and accept the connections, but I've tried to dynamically create the sockets with an array (and the listener etc) but I am still having trouble - I've tried it multiple ways but so far, all I've managed to do is get it working successfully on ONE socket, and LISTEN to all sockets, but not accept them.
So, this was my last attempt but not sure, maybe I need to use threads or declare the sockets differently?
So far, in this small test code, I want:
Initialize server
listen on all 7 ports (1111,2222 ...etc)
Accept an incoming connection on ANY of them
display both messages on client/server
drop the connection
and continue
It's a little sloppy I know, but here is the code so far and I think you can see where I am going with it:
#include <iostream>
#include <winsock2.h>
#include <string>
#pragma comment(lib, "ws2_32.lib")
int main()
{
std::cout<<"Honeypot server [test #1] by Dreamwalker"<<std::endl;
WSADATA wsa;
SOCKET s[7] , new_socket[7];
struct sockaddr_in server , client;
int c, port[7] = {1111,2222,3333,4444,5555,6666,7777};
char *message;
std::cout<<"\nInitialising Winsock and other components...";
if (WSAStartup(MAKEWORD(2,2),&wsa) != 0)
{
std::cout<<"Failed. Error Code :"<<WSAGetLastError()<<std::endl;
return 1;
}
//!IMPORTANT: create multiple new sockets on different ports
int i = 0;
for( i = 0; i < 7; i++)
{
//Create socket
if((s[i] = socket(AF_INET , SOCK_STREAM , 0 )) == INVALID_SOCKET)
{
std::cout<<"Could not create socket : "<< WSAGetLastError()<<std::endl;
}
//Prepare the sockaddr_in structure
server.sin_family = AF_INET;
server.sin_addr.s_addr = INADDR_ANY;
server.sin_port = htons( port[i] );
//Bind
if( bind(s[i] ,(struct sockaddr *)&server , sizeof(server)) == SOCKET_ERROR)
{
std::cout<<"Bind failed with error code : "<< WSAGetLastError()<<std::endl;
}
/*!ALL CREATION CHECKING DONE, now create multiple sockets on the server
and listen for connections*/
c = sizeof(struct sockaddr_in);
listen(s[i] , SOMAXCONN);
}
///ALL INITIALIZED
std::cout<<"DONE!"<<std::endl;
//Listen/accept incoming connections
std::cout<<"Now listening for connections"<<std::endl;
new_socket[i] = accept(s[i] , (struct sockaddr *)&client, &c);
if (new_socket[i] == INVALID_SOCKET)
{
std::cout<<"accept failed with error code : "<< WSAGetLastError()<<std::endl;
}
//Accepted connection
else{
std::cout<<"Someone has connected to this machine!"<<std::endl;
message = "Hello Client , I have received your connection.\n";
send(new_socket[i] , message , strlen(message) , 0);
closesocket(s[i]);
}
std::cout<<"FINISHED"<<std::endl;
WSACleanup();
getchar();
return 0;
}
And now it's throwing a runtime error as well:
WSAENOTSOCK
10038
Socket operation on nonsocket.
An operation was attempted on something that is not a socket. Either the socket handle parameter did not reference a valid socket,
or for select, a member of an fd_set was not valid.
Which (including debugging) indicates that the socket isn't declared properly when creating on an array, advice?
You code to create/bind/listen is all good. Then:
new_socket[i] = accept(s[i] , (struct sockaddr *)&client, &c);
Firstly, by the time this runs you're outside the loop, and i is 7 which is past the end of the array of sockets, which is why you get the not-a-socket error.
Secondly, accept() is a blocking call, so you can't just call accept() on all the sockets from the same thread the way you did for listen. You need to either have a separate thread block in accept() for each of the ports, or find out which one has a client connection attempt in progress using e.g. select (or epoll - does Windows have that?), then accept() a client on that specific socket (but then you've still got to either create a thread to handle the client read/recvs and write/sends or use select/epoll to find out when there's input ready to read, or more space in output buffers for transmission). There's also a race condition to be wary of if you use select/epoll - a listening socket might signal readiness for accepting a client connection, but by the time you call accept() that connection attempt's failed and forgotten, then if the listening socket hasn't been set to non-blocking mode it'll hang there waiting for another client to connect to that specific socket. IMHO, this is a case where threading is actually easier.
I think it's more "Windowsy" to use IO Completion Ports (you might want to Google), but AFAIK they're totally unportable. Winsock's not an exact match for BSD sockets, but the porting or dual-support effort's small.
In my C++ application, I am using ::bind() for a UDP socket, but on rare occasions, after reconnection due to lost connection, I get errno EADDRINUSE, even after many retries. The other side of the UDP connection which will receive the data reconnected fine and is waiting for select() to indicate there is something to read.
I presume this means the local port is in use. If true, how might I be leaking the local port such that the other side connects to it fine? The real issue here is that other side connected fine and is waiting but this side is stuck on EADDRINUSE.
--Edit--
Here is a code snippet showing that I am already doing SO_REUSEADDR on my TCP socket, not on this UDP socket for which I am having issue:
// According to "Linux Socket Programming by Example" p. 319, we must call
// setsockopt w/ SO_REUSEADDR option BEFORE calling bind.
// Make the address is reuseable so we don't get the nasty message.
int so_reuseaddr = 1; // Enabled.
int reuseAddrResult
= ::setsockopt(getTCPSocket(), SOL_SOCKET, SO_REUSEADDR, &so_reuseaddr,
sizeof(so_reuseaddr));
Here is my code to close the UDP socket when done:
void
disconnectUDP()
{
if (::shutdown(getUDPSocket(), 2) < 0) {
clog << "Warning: error during shutdown of data socket("
<< getUDPSocket() << "): " << strerror(errno) << '\n';
}
if (::close(getUDPSocket()) < 0 && !seenWarn) {
clog << "Warning: error while closing data socket("
<< getUDPSocket() << "): " << strerror(errno) << '\n';
}
}
Yes, that's normal. You need to set the socket SO_REUSEADDR before you bind, eg on *nix:
int sock = socket(...);
int yes = 1;
setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));
If you have separate code that reconnects by creating a new socket, set it on that one too. This is just to do with the default behaviour of the OS -- the port on a broken socket is kept defunct for a while.
[EDIT] This shouldn't apply to UDP connections. Maybe you should post the code you use to set up the socket.
In UDP there's no such thing as lost connection, because there's no connection. You can lose sent packets, that's all.
Don't reconnect, simply reuse the existing fd.
I have created a socket using the following lines of code and i get a valid socket and connection is established between the client and server machines. Is there a possibility that the socket becomes invalid due to network disturbances or any other reason.
If so how do we check whether the socket is valid or not.
SOCKET SocServer;
//To Set up the sockaddr structure
ServerSock.sin_family = AF_INET;
ServerSock.sin_addr.s_addr = INADDR_ANY
ServerSock.sin_port = htons(PortNumber);//port number of 5005
// To Create a socket for listening on PortNumber
if(( SocServer = socket( AF_INET, SOCK_STREAM, 0 )) == INVALID_SOCKET )
{
return FALSE;
}
//To bind the socket with wPortNumber
if(bind(SocServer,(sockaddr*)&ServerSock,sizeof(ServerSock))!=0)
{
return FALSE;
}
// To Listen for the connection on wPortNumber
if(listen(SocServer,SOMAXCONN)!=0)
{
return FALSE;
}
I know i can check for INVALID_SOCKET which in other words means the socket is 0. Is there any other way out because my SocServer will have a value say 2500, i want to check if this socket is valid.
Pass your socket to any one of the windows socket functions (eg. getsockopt()), if the socket is invalid, it will return SOCKET_ERROR while WSAGetLastError() will return WSAENOTSOCK.
It is important to note that INVALID_SOCKET does not equal 0(the actual value, which you should not use specifically is ((SOCKET)(~0))
The socket can become "invalid" when either side (expected or unexpected) disconnects, but not out of the blue due to network interferences (unless it disconnects, read above).
You can detect this by checking the return values of send() and recv() for -1 (SOCKET_ERROR)
The socket will not go invalid. But after you listen() you must accept() a connection. That connection may be lost at some point. It could be detected by a failure to write to the socket or recv() returning with error. Then you just recycle back to listen().
First, assuming that INVALID_SOCKET (which you don't show) is actually defined as 0 this is incorrect. socket returns -1 on failure like the other system calls (0 is a valid fd that could be returned).
Then once you accept a connection, your send/write or recv/read call will return -1. If you're trying to write, errno will be set to EPIPE to indicate the socket was closed (by your client). There's no way to just ask if a socket is closed.