I am trying to understand how are file descriptors related to sockets. As per my understanding, you listen on a particular file descriptor, once a connection comes in , you accept it , which returns you another file descriptor ( 2 in all ) and you use this 2nd descriptor to send/recv data.
The strange behaviour i am observing is that after accept , i have 3 file descriptors instead of two.... and i am not sure why is this the case....
I am either using lsof or /proc/pid to observe the increase in number of fd's.
ps : these are af_unix sockets.
EDIT : CODE
Here is the code to create the scoket.
int s, s2, len;
socklen_t t;
struct sockaddr_un local, remote;
if ((s = socket(AF_UNIX, SOCK_STREAM, 0)) == -1)
{
syslog(LOG_ERR,"Failed to create a socket");
exit(1);
}
int flags = fcntl(s, F_GETFD);
if (flags == -1)
{
syslog(LOG_ERR,"Failed to get socket flags");
exit(1);
}
flags |= FD_CLOEXEC;
if (fcntl(s, F_SETFD, flags) == -1)
{
syslog(LOG_ERR,"Failed to set socket flags");
exit(1);
}
local.sun_family = AF_UNIX;
strcpy(local.sun_path, SOCK_PATH.c_str());
unlink(local.sun_path);
len = strlen(local.sun_path) + sizeof(local.sun_family);
if (bind(s, (struct sockaddr *)&local, len) == -1)
{
syslog(LOG_ERR,"Failed to bind socket");
exit(1);
}
if (listen(s, 5) == -1)
{
syslog(LOG_ERR,"Failed to listen at socket");
exit(1);
}
Code where connection is accepted
while (1)
{
stat =0;
execReturn=0;
t = len;
read_fds = master;
if (select(fdmax+1, &read_fds, NULL, NULL, &tv) != -1)
{
if(FD_ISSET(s,&read_fds))
{
//Accept new connection
//fork child -> fork grand child
//child will return value back
if ((s2 = accept(s, (struct sockaddr*)&remote, &t)) == -1)
{
syslog(LOG_ERR,"Failed to acceptconnection at socket");
exit(1);
}
I am stepping through gdb and exactly after accept , the fd's become 3. The OS is fedora core 13.
The reason i need to validate this is i do not want my process to hold on to FD's ; since being a daemon over time it may walk the system into a corner...
This did seem odd behaviour. After closing the accepted connection i am still left with two fd's .
i.e. one for listen and one ghost fd... Whats even more strange is that even if 10 connections are made , only one ghost fd remains at the end of all of them closing....
It does sound like OS specific implementation..
Cheers!
Your extra file descriptor is most likely related to syslog. Syslog has to open a socket to the syslogd to report messages. Unless you explicitly call openlog this socket is opened upon the first call to syslog, and since you aren't calling syslog until you have an error you are most likely observing syslog's side effects.
The easiest way to debug this sort of issues is to run your app under the strace(1) utility. Check what system calls are made, what the parameters and return values are, and correlate that to file descriptors used.
More code please.
But I am guessing that you are looking at the OS implementation of the socket.
It probably uses one file descriptor for reading and the other for writing (but that is a guess).
What does it matter to you what the OS is doing in /proc/pid the stuff in there is not really there for your direct usage.
You are right in that it's two. You must be confusing the third with something else.
Without more information it's hard to help.
Related
I'm familiarising myself with Unix Sockets for Inter-Process Communication, and following a guide here. Each time I run the program that acts as a server (referred to as echos.c in the guide), I am getting an error that says bind: Address already in use, except for the first time I run it after deleting all build files.
The error is coming from this section of code:
#define SOCK_PATH "echo_socket"
int main(void)
{
int s, s2, len;
socklen_t t;
struct sockaddr_un local, remote;
char str[100];
if ((s = socket(AF_UNIX, SOCK_STREAM, 0)) == -1) {
perror("socket");
exit(1);
}
local.sun_family = AF_UNIX;
strcpy(local.sun_path, SOCK_PATH);
unlink(local.sun_path);
len = strlen(local.sun_path) + sizeof(local.sun_family);
if (bind(s, (struct sockaddr *)&local, len) == -1) {
perror("bind");
exit(1);
}
// ...
In the initial run, the program creates a socket file called echo_socke (I have checked this with ls -l, and it is strange that the name is missing the 't' to make it echo_socket), and then goes on to successfully bind and listen for the client process.
However, in subsequent runs where I do not delete the build files, I get the error bind: Address already in use, coming from the lines
if ((s = socket(AF_UNIX, SOCK_STREAM, 0)) == -1) {
perror("socket");
exit(1);
}
Doing additional testing I found that unlink(local.sun_path) is returning a No such file or directory error, so I figure the socket file that the bind is attempting to access is not being found. Could this be due to the file name missing a character as mentioned earlier? (But it works on the first time around, so it can't be?). Any help figuring out what is going on would be much appreciated.
I have a C++ program, using mpi, that follows a typical client server model. Each mpi instance of the client connects to a corresponding mpi instance of the server. This has worked relatively well until I have had to do some testing with added latency (1 second of added latency to be precise).
Problem:
Sometimes one of the server processes do not think the client has connected but the client thinks it has connected. i.e. After using gdb, the server is waiting at accept() but the client has continued on past connect(). Thus, it appears the client thinks it has connected when the server does not think it has connected.
My best guess is that I need to set an sock-option somewhere, however talking to fellow programmers and googling has not yielded any helpful results.
EDIT:
There are two sets of MPI processes (so two different calls to mpirun), the accept() and connect() calls are for the sockets, which are between the two sets of MPI processes. It is openmpi.
The code (from someone else's code, actually) [reduced]:
Client (connect code): (m_socket is the actual socket)
if (-1 == m_socket)
{
perror("cannot create socket");
exit(EXIT_FAILURE);
}
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_port = htons(port);
res = inet_pton(AF_INET, host_ip, &addr.sin_addr);
if (0 > res)
{
perror("error: first parameter is not a valid address family");
close(m_socket);
exit(EXIT_FAILURE);
}
else if (0 == res)
{
perror("error: second parameter does not contain valid IP address");
close(m_socket);
exit(EXIT_FAILURE);
}
//backoff
for (int sec = 1; sec < 20000; sec++ )
{
int ret;
if (0 == (ret = connect(m_socket, (struct sockaddr *)&addr, sizeof(addr))))
{
return;
}
sleep(1);
close(m_socket);
m_socket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
}
perror("connect failed");
close(m_socket);
exit(EXIT_FAILURE);
Server: (m_socket is the actual socket)
int socket = ::accept(m_socket, NULL, NULL);
if(socket < 0)
{
fprintf(stderr, "accept() failed: %s\n", strerror(errno));
close(m_socket);
exit(EXIT_FAILURE);
}
It looks like you're trying to do your connect/accept manually rather than with MPI. You might take a look at the example on Deino (http://mpi.deino.net/mpi_functions/MPI_Comm_accept.html) if you're trying to use MPI for your connections.
Alternatively, you might just need to look at a more general tutorial (some available here: http://www.mcs.anl.gov/research/projects/mpi/tutorial/) of MPI to get a feel for how communication works. Most of the time and application doesn't use Connect/Accept to communicate, but uses MPI Communicators to set up communication mechanisms between processes. It's a different model (SPMD as opposed to MPMD).
I am trying to use non-blocking TCP sockets. The problem is that they are still blocking. The code is below -
server code -
struct sockaddr name;
char buf[80];
void set_nonblock(int socket) {
int flags;
flags = fcntl(socket,F_GETFL,0);
assert(flags != -1);
fcntl(socket, F_SETFL, flags | O_NONBLOCK);
}
int main(int agrc, char** argv) {
int sock, new_sd, adrlen; //sock is this socket, new_sd is connection socket
name.sa_family = AF_UNIX;
strcpy(name.sa_data, "127.0.0.1");
adrlen = strlen(name.sa_data) + sizeof(name.sa_family);
//make socket
sock = socket(AF_UNIX, SOCK_STREAM, 0);
if (sock < 0) {
printf("\nBind error %m", errno);
exit(1);
}
//unlink and bind
unlink("127.0.0.1");
if(bind (sock, &name, adrlen) < 0)
printf("\nBind error %m", errno);
//listen
if(listen(sock, 5) < 0)
printf("\nListen error %m", errno);
//accept
new_sd = accept(sock, &name, (socklen_t*)&adrlen);
if( new_sd < 0) {
cout<<"\nserver accept failure "<<errno;
exit(1);
}
//set nonblock
set_nonblock(new_sd);
char* in = new char[80];
std::string out = "Got it";
int numSent;
int numRead;
while( !(in[0] == 'q' && in[1] == 'u' && in[2] == 'i' && in[3] == 't') ) {
//clear in buffer
for(int i=0;i<80;i++)
in[i] = ' ';
cin>>out;
cin.get();
//if we typed something, send it
if(strlen(out.c_str()) > 0) {
numSent = send(new_sd, out.c_str(), strlen(out.c_str()), 0);
cout<<"\n"<<numSent<<" bytes sent";
}
numRead = recv(new_sd, in, 80, 0);
if(numRead > 0)
cout<<"\nData read from client - "<<in;
} //end while
cout<<"\nExiting normally\n";
return 0;
}
client code -
struct sockaddr name;
void set_nonblock(int socket) {
int flags;
flags = fcntl(socket,F_GETFL,0);
assert(flags != -1);
fcntl(socket, F_SETFL, flags | O_NONBLOCK);
}
int main(int agrc, char** argv) {
int sock, new_sd, adrlen;
sock = socket(AF_UNIX, SOCK_STREAM, 0);
if (sock < 0) {
printf("\nserver socket failure %m", errno);
exit(1);
}
//stuff for server socket
name.sa_family = AF_UNIX;
strcpy(name.sa_data, "127.0.0.1");
adrlen = strlen(name.sa_data) + sizeof(name.sa_family);
if(connect(sock, &name, adrlen) < 0) {
printf("\nclient connection failure %m", errno);
exit(1);
}
cout<<"\nSuccessful connection\n";
//set nonblock
set_nonblock(sock);
std::string out;
char* in = new char[80];
int numRead;
int numSent;
while(out.compare("quit")) {
//clear in
for(int i=0;i<80;i++)
in[i] = '\0';
numRead = recv(sock, in, 80, 0);
if(numRead > 0)
cout<<"\nData read from server - "<<in;
cout<<"\n";
out.clear();
cin>>out;
cin.get();
//if we typed something, send it
if(strlen(out.c_str())) {
numSent = send(sock, out.c_str(), strlen(out.c_str()), 0);
cout<<"\n"<<numSent<<" bytes sent";
}
} //end while
cout<<"\nExiting normally\n";
return 0;
}
Whenever I run it, the server still waits for me to send something before it will read and output what the client has sent. I want either the server or client to be able to send the message as soon as I type it, and have the other read and output the message at that time. I thought non-blocking sockets was the answer, but maybe I am just doing something wrong?
Also, I was using a file instead of my 127.0.0.1 address as the sockaddr's data. If that is not how it should be properly used, feel free to say so (it worked how it worked previously with a file so I just kept it like that).
Any help is appreciated.
General approach for a TCP server where you want to handle many connections at the same time:
make listening socket non-blocking
add it to select(2) or poll(2) read event set
enter select(2)/poll(2) loop
on wakeup check if it's the listening socket, then
accept(2)
check for failure (the client might've dropped the connection attempt by now)
make newly created client socket non-blocking, add it to the polling event set
else, if it's one of the client sockets
consume input, process it
watch out for EAGAIN error code - it's not really an error, but indication that there's no input now
if read zero bytes - client closed connection, close(2) client socket, remove it from event set
re-init event set (omitting this is a common error with select(2))
repeat the loop
Client side is a little simpler since you only have one socket. Advanced applications like web browsers that handle many connections often do non-blocking connect(2) though.
Whenever I run it, the server still waits for me to send something before it will read and output what the client has sent.
Well, that is how you wrote it. You block on IO from stdin, and then and only then do you send/receive.
cin>>out;
cin.get();
Also, you are using a local socket (AF_UNIX) which creates a special file in your filesystem for interprocess communication - this is a different mechanism than IP, and is definitely not TCP as you indicate in your question. I suppose you could name the file 127.0.0.1, but that really doesn't make sense and implies confusion on your part, because that is an IP loopback address. You'll want to use AF_INET for IP.
For an excellent starter guide on unix networking, I'd recommend http://beej.us/guide/bgnet/
If you want the display of messages received to be independant of your cin statements, either fork() off a seperate process to handle your network IO, or use a separate thread.
You might be interested in select(). In my opinion non-blocking sockets are usually a hack, and proper usage of select() or poll() is generally much better design and more flexible (and more portable). try
man select_tut
for more information.
I think you have to set non-block sooner (ie get the socket then set it non block)
also check that the fcntl to set it actually worked
If you want non-blocking i/o, you want to use select. You can set it with stdin as one of the sockets it is listening on, along with the client sockets (just add file descriptor 1, which is stdin, to the fd_set).
http://beej.us/guide/bgnet/output/html/multipage/advanced.html
I would recommend reading through what beej has to say about select. It looks a little intimidating but is really useful and simple to use if you take a little time to wrap your head around it.
I am working on a small networking project using Winsock2. I am using a TCP connection and actually am working with IRC as an example since IRC is fairly simple. What I am doing is connecting to the server and sending an initial buffer so the server recognizes a connection. This works fine.
What concerns me is that I cannot write to the socket again. It seems my program hangs if I do not use shutdown() (on SD_SEND) after I send the initial buffer.
So the next data (based on RFC 1459) I want to send is the USER and NICK information, however, I feel like using shutdown() is what is causing my current issue. Is there a way to reinitialize the write socket?
Thanks!
ADDED CODE
Note that these are located within a class so it still may be slightly obscured. I am writing it into a simpler example using the elements I have. Everything is properly defined, so if I forget to define things, I apologize, but many of my recurring variables are defined for the scope of the class.
int main(int argc,char *argv[])
{
int iResult;
SOCKET Connection;
iResult = WSAStartup(MAKEWORD(2,2), &wsaData);
if(iResult != 0)
throw "Startup failed!";
// Prep stuff
ZeroMemory(&hints,sizeof(hints)); // This struct is defined addrinfo
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
hints.ai_protocol = IPPROTO_TCP;
// Now resolve server addr
iResult = getaddrinfo(argv[1],argv[2],&hints,&result);
if(iResult != 0)
throw "getaddrinfo() failed!";
// Now try to connect
for(ptr=result;ptr != NULL;ptr = ptr->ai_next)
{
Connection = socket(ptr->ai_family, ptr->ai_socktype, ptr->ai_protocol); // defined in that "hints" struct. argument number 2
if(Connection == INVALID_SOCKET)
{
freeaddrinfo(result);
WSACleanup();
throw "Error at socket();";
}
// Connect to server
iResult = connect(Connection, ptr->ai_addr, (int)ptr->ai_addrlen);
if(iResult != 0)
{
closesocket(Connection);
Connection = INVALID_SOCKET;
continue;
}
break;
}
freeaddrinfo(result);
// Send initial buffer so server know you're there :)
iResult = send(Connection, "", 1, 0);
if(iResult == SOCKET_ERROR)
{
close();
throw "Could not send initial buffer!";
}
// Close this connection for the inital buffer
iResult = shutdown(Connection, SD_SEND);
if(iResult == SOCKET_ERROR)
{
close();
throw "Could not close initial buffer socket!";
}
bool connected = true;
// This is taken from my read function within the class
// BEGIN READ FUNCTION
iResult = 0; // Reset
std::string data = ""; // Capture the output and send it all at once!
// This only works if we're connected sweet cakes <3
if(connected)
{
do
{
iResult = recv(socket, recvbuf, BUFLEN, 0);
if(iResult > 0)
{
// Working properly
// Save all data even if there is more than BUFLEN sent
continue;
}
else if(iResult == 0)
// Connection closed properly
break;
else
printf("ERROR!");
} while(iResult > 0);
}
data += recvbuf;
ZeroMemory(&recvbuf,sizeof(recvbuf));
// Function returns std::string but essentially this is what happens
printf("%s",data.c_str());
// END READ FUNCTION
// BEGIN WRITE FUNCTION
iResult = 0; // Reset
SOCKET socket = Connection; // Write function arg 1
char *data; // Write function arg 2
iResult = send(socket,data,(int)strlen(data),0);
if(iResult == SOCKET_ERROR)
{
close();
printf("Could not write data: %ld",WSAGetLastError());
return 1;
}
// Data sent, let's close the write socket
iResult = shutdown(socket, SD_SEND);
if(iResult != 0)
{
close();
printf("Could not close write socket!");
return 1;
}
//return iResult;
// END WRITE FUNCTION
// Now that will produce "Could not write data: 0" for any value of data
// So realistically I want to send the USER and NICK data, then read
// and probably process a PING string from the server and send my PONG response
return 0;
}
I hope that clarifies things!
EDIT
I think I have figured out what is going wrong. I made the corrections listed below to my code; thanks guys. However, it's my read loop which is messing with things. Even after it has all the information it seems that it is waiting for the connection to be closed before it sends the output. Any ideas? My output currently looks like this (the bytes written/total is something I added to make sure everything was going down the wire correctly)
Bytes Written: 41
Bytes Total: 41
Data: ERROR :Closing Link: raged123[127.0.0.1] 6667 (Ping timeout)
...
:irc.foonet.com NOTICE AUTH :*** Found your hostname (cached)
PING :2ED39CE5
[A bunch of funny characters]WinSock 2.0
So it appears to have timed out because the PING did not receive PONG in time, however, I cannot send the PONG without first processing the PING request which means I would need to be able to read the output before the connection is closed. Any ideas?
May I suggest a fun document on the subject? Chapter's 6 and 7 of Beej's Guide to Network Programming
It has several examples.
There shouldn't be any need to send an "initial buffer" like you've done. The server will receive notification when a client connects, it doesn't depend on the client actually sending anything. (And in particular, the IRC protocol says that the server will start sending you stuff as soon as you connect.)
The call to shutdown() is highly suspicious. Why did you expect to need to do this? Shutting down a socket is something you do when you're done with the connection, not when you're just starting. You should remove this completely.
I'm not sure what type recvbuf is, but it looks like you're using it incorrectly. Something that can be appended to a std::string probably can't also have ZeroMemory() called on it, without one or the other of those being wrong. You also aren't using iResult which is the actual number of bytes received from the server.
Your write function also contains a call to shutdown(), which you should remove.
According to man send(2)
On success, these calls return the
number of characters sent. On error,
-1 is returned, and errno is set appropriately.
What happens is probably that send does not send the full buffer at once, you must use a loop around it.
This might not be your actual problem however since you,re sending an empty string...
I'd highly recommend using Wireshark so you can check what goes down to the wire
data += recvbuf;
This can't work. There's no way string::operator+= to know how many bytes have been received. This function expects a C-style string, not an arbitrary chunk of bytes.
But you also have a very fundamental design problem. You're expecting your program to speak the IRC protocol, but it contains no implementation of that protocol whatsoever. For example, the IRC protocol specifies a particular way that messages are delimited, and you have no code whatsoever to parse those messages.
As a result, your transition from reading to writing occurs at essentially a random time determined by the vagaries of TCP timing and how the server chooses to segment its output. Since the server is permitted to segment its output however it pleases (the protocol is clear that the client cannot rely on segmentation to parse the protocol but instead must rely on the line-oriented nature), your program's behavior is unpredictable.
I'm having a strange problem while attempting to transform a blocking socket server into a nonblocking one. Though the message was only received once when being sent with blocking sockets, using nonblocking sockets the message seems to be received an infinite number of times.
Here is the code that was changed:
return ::write(client, message, size);
to
// Nonblocking socket code
int total_sent = 0, result = -1;
while( total_sent < size ) {
// Create a temporary set of flags for use with the select function
fd_set working_set;
memcpy(&working_set, &master_set, sizeof(master_set));
// Check if data is available for the socket - wait 1 second for timeout
timeout.tv_sec = 1;
timeout.tv_usec = 0;
result = select(client + 1, NULL, &working_set, NULL, &timeout);
// We are able to write - do so
result = ::write(client, &message[total_sent], (size - total_sent));
if (result == -1) {
std::cerr << "An error has occured while writing to the server."
<< std::endl;
return result;
}
total_sent += result;
}
return 0;
EDIT: The initialization of the master set looks like this:
// Private member variables in header file
fd_set master_set;
int sock;
...
// Creation of socket in class constructor
sock = ::socket(PF_INET, socket_type, 0);
// Makes the socket nonblocking
fcntl(sock,F_GETFL,0);
FD_ZERO(&master_set);
FD_SET(sock, &master_set);
...
// And then when accept is called on the socket
result = ::accept(sock, NULL, NULL);
if (result > 0) {
// A connection was made with a client - change the master file
// descriptor to note that
FD_SET(result, &master_set);
}
I have confirmed that in both cases, the code is only being called once for the offending message. Also, the client side code hasn't changed at all - does anyone have any recommendations?
fcntl(sock,F_GETFL,0);
How does that make the socket non-blocking?
fcntl(sock, F_SETFL, O_NONBLOCK);
Also, you are not checking if you can actually write to the socket non-blocking style with
FD_ISSET(client, &working_set);
I do not believe that this code is really called only once in the "non blocking" version (quotes because it is not really non-blocking yet as Maister pointed out, look here), check again. If the blocking and non blocking versions are consistent, the non blocking version should return total_sent (or size). With return 0 instead caller is likely to believe nothing was sent. Which would cause infinite sending... is it not what's happening ?
Also your "non blocking" code is quite strange. You seem to use select to make it blocking anyway... Ok, with a timeout of 1s, but why don't you make it really non blocking ? ie: remove all the select stuff and test for error case in write() with errno being EWOULDBLOCK. select or poll are for multiplexing.
Also you should check errors for select and use FD_ISSET to check if socket is really ready. What if the 1 s timeout really happen ? Or if select is stopped by some interruption ? And if an error occurs in write, you should also write which error, that is much more useful than your generic message. But I guess this part of code is still far from finished.
As far as I understand your code it should probably look somewhat like that (if the code is running in an unique thread or threaded, or forking when accepting a connection would change details):
// Creation of socket in class constructor
sock = ::socket(PF_INET, socket_type, 0);
fcntl(sock, F_SETFL, O_NONBLOCK);
// And then when accept is called on the socket
result = ::accept(sock, NULL, NULL);
if (result > 0) {
// A connection was made with a client
client = result;
fcntl(client, F_SETFL, O_NONBLOCK);
}
// Nonblocking socket code
result = ::write(client, &message[total_sent], (size - total_sent));
if (result == -1) {
if (errno == EWOULDBLOCK){
return 0;
}
std::cerr << "An error has occured while writing to the server."
<< std::endl;
return result;
}
return size;