I'm working on a distributed system with multiple workers.
Each worker is assigned a specific address - host and port.
As a worker, I'd like to know when another worker comes online and starts listening to their assigned address.
Currently I create a socket and keep trying to connect until I reach a timeout.
An external function calls connectionOk in a loop.
bool MyClass::connectionOk(struct sockaddr_storage other_worker) {
//connect and disconnect to verify peer is available
int fd = socket(other_worker.ss_family, SOCK_STREAM, 0);
int res = connect(fd, (const sockaddr*)&other_worker, sizeof(other_worker));
close(fd);
return res == 0;
}
However that doesn't seem to work.
A worker gets created, starts listening, but sometimes connectionOk returns false and timeout is reached.
Is there a better way to accomplish this?
Related
I have a system that can start multiple instances.
Every instance has a client and a server.
They are connected over socket/TCP
Every instance is started by starting a client.
The client starts (checks if IP is available, if not increase the IP by 1, checks again ...) -
The client starts the server with the free IP and connects to it. (for legacy reasons has to be like this)
Instance numbers 2, 3, 4, 5 work without issues.
...
Instance number 6. -> Fails on checking if the first IP in the range is available.
To check if IP is already in use, I do not close the socket on the server side so that it can accept the additional connection.
On the client-side, I check if I can connect to the server-side with the following code:
bool CheckIPInUse(char *ip)
{
bool ret = false;
int port = 12345;
int sock;
struct sockaddr_in serv_addr;
serv_addr.sin_family = AF_INET;
serv_addr.sin_port = htons(port);
// **non blocking** because I want the check to be fast.
sock = socket(AF_INET, SOCK_STREAM | SOCK_NONBLOCK, 0);
inet_pton(AF_INET, ip, &serv_addr.sin_addr);
int ret_conn = connect(sock, (struct sockaddr *)&serv_addr, sizeof(serv_addr));
if (ret_conn == 0){
fprintf(stdout, "connected");
ret = true;
}
else if (ret_conn < 0 && (errno != EINPROGRESS)){
fprintf(stdout, "failed to connect");
}
else
{
int check_if_connected = 10;
while (check_if_connected--)
{
socklen_t len = sizeof(serv_addr);
int ret_getpeer = getpeername(sock, (struct sockaddr *)&serv_addr, &len);
if (ret_getpeer == 0)
{
fprintf(stdout, "connected");
ret = true;
break;
}
usleep(100000);
}
}
close(sock);
return ret;
}
This works for the first 5 instances.
6th instance fails to connect to the first IP in range and tries to start the server with IP which is already in use. (always the 6th).
Is there any better way to check programmatically if IP/Port is already busy?
Any ideas on what to check. for failure in the instance number 6?
The only way to check if an ip/port on a server is available is to bind() to it. If it worked, it was available (but not any more).
Any approach that involves a test connect()ion first, to see if it fails, or anything along the lines of poking somewhere in /proc to see which IPs and ports are in use -- nothing along these lines will ever be 100% foolproof. That's because even if you reach the conclusion that the port is available, it may no longer be by the time you get around to try to bind() to it.
Now, you can take, as a starting position, that a particular IP and/or port range is reserved for your application's use, and you only wish to arbitrate IP/port allocation between different instances of your application. In that case you can do that pretty much whatever you want, you're not limited to attempting to actually start instances of your application, and hope for the best. One simplistic approach is to use lock files in /var/tmp to represent all possible IP/port combination, and have your application try, in turn, to acquire a lock on the corresponding lock file, first, and once it's official, and the lock file is acquired, then the corresponding IP/port then can be established at your leisure, but the lock file must remain locked until the IP/port is no longer in use.
But in terms of attempting to check if a socket port is available, or not, the only way to do it is to bind() it, because that, by definition, is what it does. You could attempt to implement a multi-layered approach, like trying to connect() first, and then attempt to bind() it, and if the bind() fails, then keep looking for a free port. But that's creating extra complexity, without much of a benefit.
Did you check that the server did not meet its maximum backlog length ?
You may be getting "connection refused" if the server you are trying to connect to
has more pending connections then the defined backlog.
So if multiple clients are testing at the same time, one of them may encounter this.
The most probable cause of your problem is that your client is getting a connect from the server due to the listen queue. The best way to avoid this problem is to close the socket on which you call accept(2) once all the instances are in use, and reopen it again when any of the server instances are finished.
The listen queue makes the kernel to accept (send the SYN/ACK segment) connections on the otherwise not yet open socket waiting, and this will make the connection establishment quicker for the next server instances if many such connections are entering in the system. All those connections are handled in the accept(2) socket, so the best way to accept five such connections is to close the accept socket as soon as the last connection has been established (this will not avoid the problem if a connection happens to enter the server in the time between one accept(2) and the next, but the connection so established will be closed as soon as the accept socket is still open)
In my opinion, you should have a master server process that forks new processes to handle the different connection and closes the accept socket as soon as it reaches the full capacity. Once one of the servers attending the connections closes one of them, it should reopen the accept socket and accept a new connection.
IMHO, also the most robust way of implementing such a system is to allow the extra connections to get in, but not attend them, so the connection remains open in case a new client happens to enter, and it can close it if the server doesn't attend it in a timeout interval. Having a sixth client already connected, but waiting for the server to say hello, will leave you in a state in which you can start talking to the server as soon as the last service ends.
I am learning Winsock and trying to create some easy programs to get to know it. I managed to create server which can handle multiple connections also manage them and client according to all tutorials, it is working how it was supposed to but :
I tried to make loop where I check if any of clients has disconnected and if it has, I wanted to close it.
I managed to write something which would check if socket is disconnected but it does not connect 2 or more sockets at one time
Anyone can give me reply how to make working loop checking through every client if it has disconnected and close socket ? It is all to make something like max clients connected to server at one time. Thanks in advance.
while (true)
{
ConnectingSocket = accept (ListeningSocket, (SOCKADDR*)&addr, &addrlen);
if (ConnectingSocket!=INVALID_SOCKET)
{
Connections[ConnectionsCounter] = ConnectingSocket;
char *Name = new char[64];
ZeroMemory (Name,64);
sprintf (Name, "%i",ConnectionsCounter);
send (Connections[ConnectionsCounter],Name,64,0);
cout<<"New connection !\n";
ConnectionsCounter++;
char data;
if (ConnectionsCounter>0)
{
for (int i=0;i<ConnectionsCounter;i++)
{
if (recv(Connections[i],&data,1, MSG_PEEK))
{
closesocket(Connections[i]);
cout<<"Connection closed.\n";
ConnectionsCounter=ConnectionsCounter-1;
}
}
}
}
Sleep(50);
}
it seems that you want to manage multiple connections using a single thread. right?
Briefly socket communication has two mode, block and non-block. The default one is block mode. let's focus your code:
for (int i=0;i<ConnectionsCounter;i++)
{
if (recv(Connections[i],&data,1, MSG_PEEK))
{
closesocket(Connections[i]);
cout<<"Connection closed.\n";
ConnectionsCounter=ConnectionsCounter-1;
}
}
In the above code, you called the recv function. and it will block until peer has sent msg to you, or peer closed the link. So, if you have two connection now namely Connections[0] and Connections[1]. If you were recv Connections[0], at the same time, the Connections[1] has disconnected, you were not know it. because you were blocking at recv(Connections[0]). when the Connections[0] sent msg to you or it closed the socket, then loop continue, finally you checked it disconnect, even through it disconnected 10 minutes ago.
To solve it, I think you need a book Network Programming for Microsoft Windows . There are some method, such as one thread one socket pattern, asynchronous communication mode, non-block mode, and so on.
Forgot to point out the bug, pay attention here:
closesocket(Connectons[i]);
cout<<"Connection closed.\n";
ConnectionsCounter=ConnectionsCounter-1;
Let me give an example to illustrate it. now we have two Connections with index 0 and 1, and then ConnectionsCount should be 2, right? When the Connections[0] is disconnected, the ConnectionsCounter is changed from 2 to 1. and loop exit, a new client connected, you save the new client socket as Connections[ConnectionsCounter(=1)] = ConnectingSocket; oops, gotting an bug. because the disconnected socket's index is 0, and index 1 was used by another link. you are reusing the index 1.
why not try to use vector to save the socket.
hope it helps~
I'm writing program in UNIX on C. I have to write client-server(TCP) program on sockets. Client sends some information and server answer. No matter what client sends or receives because I successfully wrote code for it. But last part of task is very hard for me.
1)One connection - one child process.
2)For new connections using pre-running processes from a pool.
3)Pool size is dinamic.If the number of free processes(which is not servicing client) became less than N - should create new processes, if it became more than K - "extra" processes must be terminated.
This is my code. Every connection make new child process using fork().Each connection runs in new process. But how to make dynamic pool that I said above?
Please, help, it's very important! This is the last what I should do.
Server Code:
int main(int argc, char * argv[])
{
int cfd;
int listener = socket(AF_INET, SOCK_STREAM, 0); //create listiner socket
if(listener < 0){
perror("socket error");
return 1;
}
struct sockaddr_in addr;
addr.sin_family = AF_INET;
addr.sin_port = htons(PORT);
addr.sin_addr.s_addr = htonl(INADDR_ANY);
int binding = bind(listener, (struct sockaddr *)&addr, sizeof(addr));
if(binding < 0){
perror("binding error");
return 1;
}
listen(listener, 1); //listen for new clients
signal(SIGCHLD,handler);
int pid;
for(;;) // infinity loop on server
{
cfd = accept(listener, NULL, NULL); //client socket descriptor
pid = fork(); //make child proc
if(pid == 0) //in child proc...
{
close(listener); //close listener socket descriptor
... //some server actions that I do.(receive or send)
close(cfd); // close client fd
return 0;
}
close(cfd);
}
return 0;
}
This is a design or architecture question, too broad for a definitive answer with code.
So, you know you want to service each new connection in its own process. Your other two constraints pose (at least) two questions:
First, how is a new connection routed to one of N already running workers?
This is comparatively easy. The most common designs here are:
Each worker inherits the listening socket and performs its own accept()
The workers might mutex the listening socket, so that only one is ready to call accept() at any given time, or they might each simply call accept() — but in that case beware the thundering herd.
Workers receive already accept()ed connections from some other process via UNIX file descriptor passing.
See, for instance, this SO question.
Second, how do we ensure that N – K idle worker processes are available?
This is a larger question, one that you'll have to answer based on your comfort and any other constraints.
You need to know, not merely how many workers are alive, but which workers are idle ("free"). Having a parent process keep track of its children is an obvious start, but that won't distinguish idle workers from busy workers by itself. Could you use a shared, mutexed status table, either a file or in shared memory? Or perhaps each child communicates its status over a socketpair() — the same used for file descriptor passing — to the parent?
Then, how do you safely kill an idle worker if for some reason you exceed K? Signals? A command given over that same socketpair()? Can a newly-idle worker check the status table terminate itself if to go idle would exceed K? How do you recover from accidental worker termination (e.g., SEGV)? Etc. Etc.
Apache's MPM prefork module implements one possible design in this problem space. You may want to consult it for ideas.
Your code doesn't seem to satisfy condition number 2. Your process pool is not pre-running. The processes are created when you accept the connection. One interpretation would have you do a bunch of forks and then have the forked processes wait for the accept. One would get it and then it would do the processing.
The parent process needs to keep track of how many children are out there. You could do this in a thread which does a wait. This will wait for a child to die. (see man 2 wait for various flavors.) When the number of processes becomes too big, you could send a signal which the children could catch in order to terminate properly. However, I am assuming that the parent process would NOT fork more children until some of them died and would not over-subscribe the 'K' limit.
When doing socket programming, with multi-threading,
if a thread is blocked on Accept Function,
and main thread is trying to shut down the process,
how to break the accept function in order to pthread_join safely?
I have vague memory of how to do this by connection itself to its own port in order to break the accept function.
Any solution will be thankful.
Cheers
Some choices:
a) Use non-blocking
b) Use AcceptEx() to wait on an extra signal, (Windows)
c) Close the listening socket from another thread to make Accept() return with an error/exception.
d) Open a temporary local connection from another thread to make Accept() return with the temp connection
The typical approach to this is not to use accept() unless there is something to accept! The way to do this is to poll() the corresponding socket with a suitable time-out in a loop. The loop checks if it is meant to exit because a suitably synchronized flag was set.
An alternative is to send the blocked thread a signal, e.g., using pthread_kill(). This gets out of the blocked accept() with a suitable error indication. Again, the next step is to check some flag to see if the thread is meant to exit. My preference is the first approach, though.
Depending on your system, if it is available, I would use a select function to wait for the server socket to have a read, indicating a socket is trying to connect. The amount of time to time to wait for a connection can be set/adjusted to to what every time you want to wait for a client to connect(infinity, to seconds, to 0 which will just check and return). The return status needs to be checked to see if the time limit was reached (no socket is trying to connect), or if there is something waiting to be serviced (your server socket indicating there is a client which would like to connect). You can then execute the accept knowing there is a socket to connect based on the returned status.
If available I would use a select function with a timeout in a loop to achieve this functionality.
as Glenn suggested
The select function with a timeout value will wait for a socket to connect for a set period of time. If a socket attempts to connect it can be accepted during that period. By looping this select with a timeout it is possible to check for new connections until the break condition is met.
Here is an example:
std::atomic<bool> stopThread;
void theThread ( std::atomic<bool> & quit )
{
struct timeval tv;
int activity;
...
while(!quit)
{
// reset the time value for select timeout
tv.tv_sec = 0;
tv.tv_usec = 1000000;
...
//wait for an activity on one of the sockets
activity = select( max_sd + 1 , &readfds , NULL , NULL , &tv);
if ((activity < 0) && (errno!=EINTR))
{
printf("select error");
}
if (FD_ISSET(master_socket, &readfds))
{
if ((new_socket = accept(master_socket, (struct sockaddr *)&address, (socklen_t*)&addrlen))<0)
{
perror("accept");
exit(EXIT_FAILURE);
}
...
}
}
int main(int argc, char** argv)
{
...
stopThread = false;
std::thread foo(theThread, std::ref(stopThread));
...
stopThread = true;
foo.join();
return 0;
}
A more complete example of 'Select' http://www.binarytides.com
I am pretty new to C++ so I am sure my code and answer can be improved.
Sounds like what you are looking for is this: You set a special flag variable known to the listening/accepting socket, and then let the main thread open a connection to the listening/accepting socket. The listening/accepting socket/thread has to check the flag every time it accepts a connection in order to know when to shut down.
Typically if you want to do multi-threaded networking, you would spawn a thread once a connection is made (or ready to be made). If you want to lower the overhead, a thread pool isn't too hard to implement.
I'm writing a program using the Winsock API because a friend wanted a simple program to check and see if a Minecraft server was running or not. It works fine if it is running, however if it is not running, the program freezes until, I'm assuming, the connection times out. Another issue is, if I have something like this (pseudo-code):
void connectButtonClicked()
{
setLabel1Text("Connecting");
attemptConnection();
setLabel1Text("Done Connecting!");
}
it seems to skip right to attemptConnection(), completely ignoring whats above it. I notice this because the program will freeze, but it wont change the label to "Connecting".
Here is my actual connection code:
bool CConnectionManager::ConnectToIp(String^ ipaddr)
{
if(!m_bValid)
return false;
const char* ip = StringToPConstChar(ipaddr);
m_socket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
if(isalpha(ip[0]))
{
ip = getIPFromAddress(ipaddr);
}
sockaddr_in service;
service.sin_family = AF_INET;
service.sin_addr.s_addr = inet_addr(ip);
service.sin_port = htons(MINECRAFT_PORT);
if(m_socket == NULL)
{
return false;
}
if (connect(m_socket, (SOCKADDR*)&service, sizeof(service)) == SOCKET_ERROR)
{
closesocket(m_socket);
return false;
}
else
{
closesocket(m_socket);
return true;
}
return true;
}
There is also code in the CConnectionManager's contructor to start up Winsock API and such.
So, how do I avoid this freeze, and allow me to update something like a progress bar during connection? Do I have to make the connection in a separate thread? I have only worked with threads in Java, so I have no idea how to do that :/
Also: I am using a CLR Windows Form Application
I am using Microsoft Visual C++ 2008 Express Edition
Your code does not skip the label update. The update simply involves issuing window messages that have not been processed yet, that is why you do not see the new text appear before connecting the socket. You will have to pump the message queue for new messages before connecting the socket.
As for the socket itself, there is no connect timeout in the WinSock API, unfortunately. You have two choices to implement a manual timeout:
1) Assuming you are using a blocking socket (sockets are blocking by default), perform the connect in a separate worker thread.
2) If you don't want to use a thread then switch the socket to non-blocking mode. Connecting the socket will always exit immediately, so your main code will not be blocked, then you will receive a notification later on if the connection was successful or not. There are several ways to detect that, depending on which API you use - WSAAsyncSelect(), WSAAsyncEvent(), or select().
Either way, while the connect is in progress, run a timer in your main thread. If the connect succeeds, stop the timer. If the timer elapses, disconnect the socket, which will cause the connect to abort with an error.
Maybe you want to read here:
To assure that all data is sent and received on a connected socket before it is closed, an application should use shutdown to close connection before calling closesocket. http://msdn.microsoft.com/en-us/library/ms740481%28v=VS.85%29.aspx
Since you are in the blocking mode there still might be some data...