In my C++ application, I am using ::bind() for a UDP socket, but on rare occasions, after reconnection due to lost connection, I get errno EADDRINUSE, even after many retries. The other side of the UDP connection which will receive the data reconnected fine and is waiting for select() to indicate there is something to read.
I presume this means the local port is in use. If true, how might I be leaking the local port such that the other side connects to it fine? The real issue here is that other side connected fine and is waiting but this side is stuck on EADDRINUSE.
--Edit--
Here is a code snippet showing that I am already doing SO_REUSEADDR on my TCP socket, not on this UDP socket for which I am having issue:
// According to "Linux Socket Programming by Example" p. 319, we must call
// setsockopt w/ SO_REUSEADDR option BEFORE calling bind.
// Make the address is reuseable so we don't get the nasty message.
int so_reuseaddr = 1; // Enabled.
int reuseAddrResult
= ::setsockopt(getTCPSocket(), SOL_SOCKET, SO_REUSEADDR, &so_reuseaddr,
sizeof(so_reuseaddr));
Here is my code to close the UDP socket when done:
void
disconnectUDP()
{
if (::shutdown(getUDPSocket(), 2) < 0) {
clog << "Warning: error during shutdown of data socket("
<< getUDPSocket() << "): " << strerror(errno) << '\n';
}
if (::close(getUDPSocket()) < 0 && !seenWarn) {
clog << "Warning: error while closing data socket("
<< getUDPSocket() << "): " << strerror(errno) << '\n';
}
}
Yes, that's normal. You need to set the socket SO_REUSEADDR before you bind, eg on *nix:
int sock = socket(...);
int yes = 1;
setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));
If you have separate code that reconnects by creating a new socket, set it on that one too. This is just to do with the default behaviour of the OS -- the port on a broken socket is kept defunct for a while.
[EDIT] This shouldn't apply to UDP connections. Maybe you should post the code you use to set up the socket.
In UDP there's no such thing as lost connection, because there's no connection. You can lose sent packets, that's all.
Don't reconnect, simply reuse the existing fd.
Related
I want to avoid the TIME_WAIT state when closing a TCP socket (I am aware of the pros and cons of circumventing TIME_WAIT).
I am using Windows and WinSock2/.Net sockets and am having great difficulty getting the SO_LINGER socket option to work as described in the documentation.
My test code with most of the error checking removed for brevity is:
#include <winsock2.h>
#include <ws2tcpip.h>
#include <iostream>
int main()
{
std::cout << "starting..." << std::endl;
WSADATA w = { 0 };
int error = WSAStartup(0x0202, &w);
if (error || w.wVersion != 0x0202) {
std::cerr << "Could not initialise Winsock2." << std::endl;
return -1;
}
auto clientSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
// Set socket options
linger lingerOpt = { 1, 0 };
setsockopt(clientSocket, SOL_SOCKET, SO_LINGER, (char*)&lingerOpt, sizeof(lingerOpt));
linger checkLingerOpt{ 0 };
int optLen = sizeof(checkLingerOpt);
int getOptResult = getsockopt(clientSocket, SOL_SOCKET, SO_LINGER, (char*)&checkLingerOpt, &optLen);
if (getOptResult < 0) {
wprintf(L"Failed to get SO_LINGER socket option on client socket, error: %ld\n", WSAGetLastError());
}
else {
std::cout << "Linger option set to onoff " << checkLingerOpt.l_onoff << ", linger seconds " << checkLingerOpt.l_linger << "." << std::endl;
}
// Bind local client socket.
sockaddr_in clientBindAddr;
clientBindAddr.sin_family = AF_INET;
clientBindAddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
clientBindAddr.sin_port = htons(15064);
bind(clientSocket, (SOCKADDR*)&clientBindAddr, sizeof (clientBindAddr));
sockaddr_in serverSockAddr;
serverSockAddr.sin_family = AF_INET;
serverSockAddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
serverSockAddr.sin_port = htons(5060);
// Connect to server.
connect(clientSocket, (SOCKADDR*)&serverSockAddr, sizeof (serverSockAddr));
std::cout << "connected." << std::endl;
Sleep(1000);
//shutdown(clientSocket, SD_BOTH);
closesocket(clientSocket);
std::cout << "finished." << std::endl;
}
Result:
starting...
Linger option set to onoff 1, linger seconds 0.
connected.
finished.
The sample above does avoid the TIME_WAIT state but does so because the client socket sends a RST packet.
If the Linger option is changed to:
linger lingerOpt = { 1, 5 };
Result
starting...
Linger option set to onoff 1, linger seconds 5.
connected.
finished.
Then closing the socket does result in a TIME_WAIT but of 30s which is the same result as not setting the SO_LINGER option.
Another observation is that if the socket is shutdown (which is the recommended way to cleanly close) with shutdown(clientSocket, SD_BOTH); then the Linger option of {1,0} will have no affect.
In summary:
Set Linger option as {1,0} & close with closesocket => RST.
Set Linger option as {1,5} & close with closesocket => FIN-ACK & TIME_WAIT of 30s.
Set Linger option as {1,0} & close with shutdown, closesocket => FIN-ACK & TIME_WAIT of 30s.
Set Linger option as {1,5} & close with shutdown, closesocket => FIN-ACK & TIME_WAIT of 30s.
What I'd like is:
Set Linger option as {1,0} & close with shutdown, closesocket => FIN-ACK & TIME_WAIT of 0s.
Update: As pointed out in the closesocket reference by Remy Lebeau a Linger option of {nonzero,0} is hard coded to generate a RST.
A short TIME_WAIT of a few seconds would be just as good, i.e. a linger option of {1,1} caused closesocket to exit gracefully with a 1s TIME_WAIT period, and which according to the closesocket documentation should be possible.
Update 2: As again pointed out by Remy Lebeau the Linger option and TIME_WAIT period are NOT linked. If you're reading this you probably made the same mistake I did and were trying to shorten the TIME_WAIT period via setsockopt and SO_LINGER.
By all accounts that can't be done and in cases where careful consideration judges TIME_WAIT needs to be avoided (such as in my case where the application layer protocol can deal with stray or orphaned TCP data packets) the ideal option looks to be a Linger setting of {1,0} to force a hard RST socket close which will allow the connection to be immediately re-attempted without the OS blocking the attempt.
You can't really avoid TIME_WAIT when your app is the one closing the TCP connection first (TIME_WAIT does not happen when the peer closes the connection first). No amount of SO_LINGER settings will change that fact, other than performing an abortive socket closure (ie sending a RST packet). It is simply part of how TCP works (look at the TCP state diagram). SO_LINGER simply controls how long closesocket() waits before actually closing an active connection.
The only way to prevent the socket from entering the TIME_WAIT state is to set the l_linger duration to 0, and don't call shutdown(SD_SEND) or shutdown(SD_BOTH) at all (calling shutdown(SD_RECEIVE) is OK). This is documented behavior:
The closesocket call will only block until all data has been delivered to the peer or the timeout expires. If the connection is reset because the timeout expires, then the socket will not go into TIME_WAIT state. If all data is sent within the timeout period, then the socket can go into TIME_WAIT state.
If the l_onoff member of the linger structure is nonzero and the l_linger member is a zero timeout interval on a blocking socket, then a call to closesocket will reset the connection. The socket will not go to the TIME_WAIT state.
The real problem with your code (aside from the lack of error handling) is that your client is bind()'ing a client socket before connect()'ing it to a server. Typically, you should not bind() a client socket at all, you should let the OS choose an appropriate binding for you. However, if you must bind() a client socket, you will likely need to enable the SO_REUSEADDR option on that socket to avoid being blocked when a previous connection boudn to the same local IP/Port is still in TIME_WAIT state and you are trying to connect() in a short amount of time after the previous closesocket().
See How to avoid TIME_WAIT state after closesocket() ? for more details. Also, the document you linked to in your question also explains ways to avoid TIME_WAIT without resorting to messing with SO_LINGER.
I'm working on a vision-application, which have two modes:
1) parameter setting
2) automatic
The problem is in 2), when my app waits for a signal via TCP/IP. The program is freezing while accept()-methode is called. I want to provide the possibility on a GUI to change the mode. So if the mode is changing, it's provided by another signal (message_queue). So I want to interrupt the accept state.
Is there a simple possibility to interrupt the accept?
std::cout << "TCPIP " << std::endl;
client = accept(slisten, (struct sockaddr*)&clientinfo, &clientinfolen);
if (client != SOCKET_ERROR)
cout << "client accepted: " << inet_ntoa(clientinfo.sin_addr) << ":"
<< ntohs(clientinfo.sin_port) << endl;
//receive the message from client
//recv returns the number of bytes received!!
//buf contains the data received
int rec = recv(client, buf, sizeof(buf), 0);
cout << "Message: " << rec << " bytes and the message " << buf << endl;
I read about select() but I have no clue how to use it. Could anybody give me a hint how to implement for example select() in my code?
Thanks.
Best regards,
T
The solution is to call accept() only when there is an incoming connection request. You do that by polling on the listen socket, where you can also add other file descriptors, use a timeout etc.
You did not mention your platform. On Linux, see epoll(), UNIX see poll()/select(), Windows I don't know.
A general way would be to use a local TCP connection by which the UI thread could interrupt the select call. The general architecture would use:
a dedicated thread waiting with select on both slisten and the local TCP connection
a TCP connection (Unix domain socket on a Unix or Unix-like system, or 127.0.0.1 on Windows) between the UI thread and the waiting one
various synchronizations/messages between both threads as required
Just declare that select should read slisten and the local socket. It will return as soon as one is ready, and you will be able to know which one is ready.
As you haven't specified your platform, and networking, especially async, is platform-specific, I suppose you need a cross-platform solution. Boost.Asio fits perfectly here: http://www.boost.org/doc/libs/1_39_0/doc/html/boost_asio/reference/basic_socket_acceptor/async_accept/overload1.html
Example from the link:
void accept_handler(const boost::system::error_code& error)
{
if (!error)
{
// Accept succeeded.
}
}
...
boost::asio::ip::tcp::acceptor acceptor(io_service);
...
boost::asio::ip::tcp::socket socket(io_service);
acceptor.async_accept(socket, accept_handler);
If Boost is a problem, Asio can be a header-only lib and used w/o Boost: http://think-async.com/Asio/AsioAndBoostAsio.
One way would be to run select in a loop with a timeout.
Put slisten into nonblocking mode (this isn't strictly necessary but sometimes accept blocks even when select says otherwise) and then:
fd_set read_fds;
FD_ZERO(&read_fds);
FD_SET(slisten, &read_fds);
struct timeval timeout;
timeout.tv_sec = 1; // 1s timeout
timeout.tv_usec = 0;
int select_status;
while (true) {
select_status = select(slisten+1, &read_fds, NULL, NULL, &timeout);
if (select_status == -1) {
// ERROR: do something
} else if (select_status > 0) {
break; // we have data, we can accept now
}
// otherwise (i.e. select_status==0) timeout, continue
}
client = accept(slisten, ...);
This will allow you to catch signals once per second. More info here:
http://man7.org/linux/man-pages/man2/select.2.html
and Windows version (pretty much the same):
https://msdn.microsoft.com/pl-pl/library/windows/desktop/ms740141(v=vs.85).aspx
Right now, I am trying to specify options with setsockopt() using the following code:
// bind socket
// Use setsockopt() function to make sure the port is not in use
int yes = 1;
setsockopt(TCPSocket, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(int));
setsockopt(TCPSocket, SOL_SOCKET, SO_RCVTIMEO, (char *)&tv,sizeof(struct timeval));
status = bind(TCPSocket, host_info_list->ai_addr, host_info_list->ai_addrlen);
if (status == -1) std::cout << "bind error" << std::endl ;
// listen for connections
status = listen(TCPSocket, 5);
if (status == -1) std::cout << "listen error" << std::endl ;
int new_sd;
struct sockaddr_storage their_addr;
socklen_t addr_size = sizeof(their_addr);
new_sd = accept(TCPSocket, (struct sockaddr *)&their_addr, &addr_size);
if (new_sd == -1) std::cout << "listen error" << std::endl ;
Note tv is an already-specified timeval.
When I make only the first setsockopt() call, everything works fine. However, with the addition of the second (which does not return any errors), I encounter the second "listen error" specified in the code. I'm not sure why setting the timeout value affect this, can someone explain?
I do not take credit for the code specified; it is modified from the code presented in the tutorial here: http://codebase.eu/tutorial/linux-socket-programming-c/
If you see a TCP state diagram like this one you see there's a state called TIME_WAIT when actively closing a socket. This state can take some time before it ends, up to four minutes according to RFC793.
While the socket is in the TIME_WAIT you can not bind to an interface using the same address-port pair as the socket that is in the wait state. Setting the SO_REUSEADDR flag om a socket enables other sockets to bind to the address when the current socket (with the flag set) is in the TIME_WAIT state.
The SO_REUSEADDR option is most useful for server (passive, listening) sockets.
As for your problem, after each call to setsockopt check what it returns, and if it's -1 then you check errno to see what went wrong. You can use perror or strerror to print or get a printable string for the error, like
if (setsockopt(TCPSocket, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(int)) < 0)
{
std::cerr << "Error setting the SO_REUSEADDR: " << strerror(errno) << '\n';
// Do something appropriate
}
Joachim's solution did a great job of answering my intial question and explaining setsockopt(). To answer my own question after realizing the issue was further down in the code, the timeout affects the server being able to listen to a port. Say the timeout is only 10ms, the server must be started, then the client, and a connection must be established in that time. This wasn't happening in my case, thus the resulting error.
I have written C++ client server application and the server is crashing.
The scenario
Start server
1 hour later (not before) Client connect
Then the Server which is waiting in accept returns -1 with errno "Too many open files".
Nothing else special is running on the machine which led me to believe that accept is opening many file descriptors while waiting.
Is that true?
How can I fix this so the client could connect anytime?
the relevant server code:
int sockClient;
while (true) {
sockaddr_in* clientSockAddr = new sockaddr_in();
socklen_t clientSockAddrLen = sizeof(sockaddr_in);
sockClient = accept(sockServer, (sockaddr *) clientSockAddr,
&clientSockAddrLen);
if(sockClient == -1 ){
std::ostringstream s;
s << "TCP Server: accept connection error." << std::strerror(errno);
throw runtime_error(s.str());
}
connection->communicate(sockClient, clientSockAddr, clientSockAddrLen);
}
You have a file descriptor leak somewhere. Possibly you aren't closing accepted sockets when you've finished with them, or else it's on a file somewhere.
So I made a port scanner in C++ this morning and it seems to work alright, just having one rather annoying issue with it- whenever I use it to scan an IP over the network, it takes a good 10-20 seconds PER port.
It seems like the connect() method is what's taking it so long.
Now aside from multi-threading, which I'm sure will speed up the process but not by much, how could I make this faster? Here is the section of code that does the scanning:
for (i = 0; i < a_size(port_no); i++)
{
sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
target.sin_family = AF_INET;
target.sin_port = htons(port_no[i]);
target.sin_addr.s_addr = inet_addr(argv[1]);
if (connect(sock, (SOCKADDR *)&target, sizeof(target)) != SOCKET_ERROR)
cout << "Port: " << port_no[i] << " - open" << endl;
else
cout << "Port: " << port_no[i] << " - closed" << endl;
closesocket(sock);
}
If you need more let me know.
Oh also, I am using the winsock2.h file. Is it because of this that its so slow?
When you call connect(2), the OS initiates the three-way handshake by sending a SYN packet to the other peer. If no response is received, it waits a little bit and sends a few more SYN packets. If no response is still received after a given timeout, then the operation fails, and connect(2) returns with the error code ETIMEODOUT.
Ordinarily, if a peer is up but not accepting TCP connections on a given port, it will reply to any SYN packets with a RST packet. This will cause connect(2) to fail much more quickly (one network round-trip time) with the error ECONNREFUSED. However, if the peer has a firewall set up, it'll just ignore your SYN packets and won't send those RST packets, which will cause connect(2) to take a long time to fail.
So, if you want to avoid waiting for that timeout for every port, you need to do multiple connections in parallel. You can do this multithreading (one synchronous connect(2) call per thread), but this doesn't scale well since threads take up a fair amount of resources.
The better method would be to use non-blocking sockets. To make a socket non-blocking, call fcntl(2) with the F_SETFL option and the O_NONBLOCK option. Then, connect(2) will return immediately with either EWOULDBLOCK or EAGAIN, at which point you can use either select(2) or poll(2) and friends to monitor a large number of sockets at once.
Try creating an array of non-blocking sockets to queue up a bunch of connection attempts at once.
Read about it here
I figured out a solution that works on windows. First I added:
u_long on = 1;
timeval tv = {0, 1000}; //timeout value in microseconds
fd_set fds;
FD_ZERO(&fds);
then i changed this code to look like this:
for (i = 0; i < a_size(port_no); i++)
{
sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
FD_SET(sock, &fds);
ioctlsocket(sock, FIONBIO, &on);
target.sin_family = AF_INET;
target.sin_port = htons(port_no[i]);
target.sin_addr.s_addr = inet_addr(argv[1]);
connect(sock, (SOCKADDR *)&target, sizeof(target));
err = select(sock, &fds, &fds, &fds, &tv);
if (err != SOCKET_ERROR && err != 0)
cout << "Port: " << port_no[i] << " - open" << endl;
else
cout << "Port: " << port_no[i] << " - closed" << endl;
closesocket(sock);
}
and it seems to function much faster now! I will do some work to optimize it & clean it up a bit, but thank you for all your input everyone who responded! :)