Simple socket program fails sporadically - c++

Question

I've written a small socket program that connects 500 clients to a server all on localhost in very quick succession. I'm running macOS 10.12 with Xcode 8.0 and I'm noticing that this program fails about 50% of the time. The output shows the correct number of client connection attempt calls, but after that the connection never gets made. Like I said, sometimes it works just fine.
My expected output is that the output should contain 500 lines each of CLIENT TRYCONNECT, CLIENT CONNECT, and SERVER ACCEPT. I always get the correct number of TRYCONNECT messages, but often the other two fall significantly short of the expected 500 -- the program just locks waiting for the connections from the client that never arrive.
Any idea what could be going on here?
Here's the source (as simple as I could make it):
#include <arpa/inet.h>
#include <iostream>
#include <mutex>
#include <netdb.h>
#include <netinet/in.h>
#include <sys/fcntl.h>
#include <sys/poll.h>
#include <sys/select.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <thread>
const int PORT = 1111;
const int CLIENTS = 500;
using namespace std;
mutex mtx;
#define thr throw runtime_error("error");
void server()
{
int err;
addrinfo hints = {}, *res;
hints.ai_family = AF_INET6;
hints.ai_flags = AI_PASSIVE;
hints.ai_socktype = SOCK_STREAM;
err = getaddrinfo(nullptr, to_string(PORT).c_str(), &hints, &res);
if (err != 0) thr;
int fd = socket(res->ai_family, res->ai_socktype, res->ai_protocol);
if (fd == -1) thr;
err = fcntl(fd, F_SETFL, O_NONBLOCK);
if (err == -1) thr;
socklen_t val;
val = 0;
err = setsockopt(fd, IPPROTO_IPV6, IPV6_V6ONLY, &val, sizeof val);
val = 1;
err = setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &val, sizeof val);
val = 1;
err = setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, &val, sizeof val);
err = ::bind(fd, res->ai_addr, res->ai_addrlen);
if (err == -1) thr;
err = listen(fd, 0);
if (err == -1) thr;
freeaddrinfo(res);
int connected = 0;
for (;;) {
sockaddr_storage client;
socklen_t sz = sizeof client;
int cfd = accept(fd, (sockaddr*)&client, &sz);
if (cfd < 0) {
if (errno == EAGAIN || errno == EWOULDBLOCK) {
continue;
} else {
thr;
}
} else {
lock_guard<mutex> lk(mtx);
cout << "SERVER ACCEPT ON " << cfd << endl;
if (++connected == CLIENTS) return;
}
}
}
void client()
{
addrinfo hints = {}, *res;
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
int err;
err = getaddrinfo("localhost", to_string(PORT).c_str(), &hints, &res);
if (err == -1) thr;
int fd = socket(res->ai_family, res->ai_socktype, res->ai_protocol);
if (fd == -1) thr;
err = fcntl(fd, F_SETFL, O_NONBLOCK);
if (err == -1) thr;
socklen_t val;
val = 1;
err = setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, &val, sizeof val);
err = connect(fd, res->ai_addr, res->ai_addrlen);
if (err == -1 && errno != EINPROGRESS) thr;
freeaddrinfo(res);
{
lock_guard<mutex> lk(mtx);
cout << "CLIENT TRYCONNECT ON " << fd << endl;
}
for (;;) {
this_thread::sleep_for(chrono::milliseconds(100));
pollfd ufds = { 0 };
ufds.fd = fd;
ufds.events = POLLOUT;
poll(&ufds, 1, 0);
if (!(ufds.revents & POLLOUT)) {
continue;
}
int val;
socklen_t val_sz = sizeof val;
getsockopt(fd, SOL_SOCKET, SO_ERROR, &val, &val_sz);
if (val < 0) {
if (errno == EINPROGRESS) {
continue;
}
}
lock_guard<mutex> lk(mtx);
cout << "CLIENT CONNECT ON " << fd << endl;
return;
}
}
int main(int argc, char* argv[])
{
thread t1(server);
thread clients[CLIENTS];
for (int i = 0; i < CLIENTS; ++i)
clients[i] = thread(client);
t1.join();
for (int i = 0; i < CLIENTS; ++i)
clients[i].join();
}
EDIT: Properly freeing address info in client
UPDATE: By running lsof on the process, I've found that several of the "missing" connections are marked as CLOSED instead of ESTABLISHED, so somehow they're getting shut down. I'd not yet sure what could trigger that, but I'm looking into it.
UPDATE: By adding a short (1 millisecond) sleep between starting up client connections the program runs flawlessly. It looks like perhaps a fault in the operating system implementation of sockets that prevents macOS from starting up too many connections simultaneously?

Answer 1

Your client has memory leaks.
Calls to addrinfo are dynamically allocated fromgetaddrinfo.
You need to call freeaddrinfo after their usage.
Failing to clean up addrinfo might be causing problems when trying to re-use the port number.