So I am trying to setup a multithreaded server with ACE. I am using non-blocking client sockets to prevent recv()/send() from blocking. The problem is when I use recv() and the client disconnects ungraceful, the result of recv() does not give me a hind that the client disconnected. Is there any other methode to check the connectivity.
Here is a short snippet
char buffer[4096];
ssize_t bytesReceived = peer.recv(buffer, sizeof(buffer));
if (bytesReceived < 1 && errno != EWOULDBLOCK)
{
printf("Disconnected:\n");
}
else if (bytesReceived > 0)
{
buffer[bytesReceived] = '\0';
printf(buffer);
}
So if the client disconnects, recv returns -1 but errno is still EWOULDBLOCK.
I also tried to use a short timeout in recv, but it leads to the same result as without just with errno = ETIME(TIME-Out).
Related
Can anyone help me, how to change the below code to non-blocking
struct sockaddr_un server_address;
int server_len, err;
int ret = 1;
int ipc_sockfd = socket(AF_UNIX, SOCK_STREAM, 0);
if (ipc_sockfd < 0) {
printf("%s\n","SHM_IPC: socket creation failed");
return 0;
}
server_address.sun_family = AF_UNIX;
strcpy(server_address.sun_path, SHM_IPC_SOCKET_NAME);
server_len = sizeof(server_address);
err = connect(ipc_sockfd, (struct sockaddr*)&server_address,
server_len);
if (err < 0) {
printf("%s %d\n", "IPC socket server not ready for"
". Try after few moments, Errno:", errno);
close(ipc_sockfd);
return 0;
}
err = write(ipc_sockfd, (void *)msg, sizeof(shm_ipc_msg));
if (err <=0) {
printf("%s %d\n", "SHM_IPC: socket write failed:",errno);
ret = 0;
}
close(ipc_sockfd);
return ret;
This is my client side socket, i need my client side write() and connect() to be non-blocking (I dont care about server side), do I need to change the server socket also to non-blocking to take effect?
Really appreciate your help!
This thread might help you: Does connect() block for TCP socket?
You can make your connect non-blocking but for what reason? if the client is not connected to the server you cannot expect socket.write to work. You need to wait until the connection it's done. The write operation though it is not a blocking operation. If the connection is established the write function should return immediately after sending your data.
I'm currently trying to get a client/daemon communication via an AF_UNIX socket up and running.
At the moment the client successfully sends a message, the daemon receives and processes it and then should send the message back.
Well, that's where the problem is. As soon as the daemon tries to send the message back nothing happens, the client hangs, trying to read a message, and if I kill the client the daemon dies with it.
Following is the daemon code:
//successful call to accept, I have a file descriptor now...
int c = 0;
while((c = recv(fd, (char*)&buf[0], bufferSize, 0)))
{
if(c == -1 || c == 0)
break;
tmp.append(buf.begin(), buf.begin()+c);
}
writeLog(tmp);
tmp = evaluateMsg(tmp);
writeLog(tmp);
//I assume this send call is hanging
if(send(fd, tmp.c_str(), tmp.size(), 0) < 0)
writeLog("Could not write message back!");
close(fd);
And this is the client code:
//connect(); is successful
//send(); as well - the recv(); call is hanging forever
while((c = recv(sockfd, (char*)&buf[0], 1024, 0)))
{
if(c == -1)
{
cout<<"Error";
break;
}
else if(c == 0)
break;
tmp.append(buf.begin(), buf.begin()+c);
}
Please note that the code is heavily cut down for the sake of simplicity and readability (especially the code to daemonize and create the actual AF_UNIX socket (which are both successful)).
UPDATE:
I could verify that the client-side recv() call is never returning, which means that the daemon-side send() call is hanging. Why?
I don't see any reason that the daemon side recv() loop will end. Why would recv() return 0 or -1 if the socket is still open?
You should understand when the client finished sending data on the application level, the content should make it clear, and then finish the recv() loop and continue to the send() part of the server.
Alright, the solution was pretty simple.
#selalerer was right about the return value of recv() which leads to this working code snippet:
while((c = recv(fd, (char*)&buf[0], bufferSize, 0)))
{
if(c == -1)
/* handle error */
tmp.append(buf.begin(), buf.begin()+c);
if(c < bufferSize)
//no more to read, therefore stop reading
break;
}
I have two threads running in my program. One is sending data to the other. The socket is connecting fine on both threads, and the receiving thread is accepting the connection. However, once the recv is called, it just sits there blocking as if its getting nothing in, despite the fact that the other thread is sending.
Sending thread
send(orig_sock, buf, BUFSIZ,0);
printf("Client sending chunk\n");
The printf manages to get displayed.
Receiving thread
printf("START ACCEPTING\n");
if ((new_sock = accept( orig_sock, (struct sockaddr *) &clnt_adr, &clnt_len)) < 0) {
perror("accept error");
close(orig_sock);
return NULL;
}
printf("PASS ACCEPT\n");
if ( fork( ) == 0 ) { // Generate a CHILD
printf("FORK\n");
len=recv(new_sock, buf, BUFLEN,0 );
printf("message received");
receiveBuffer.push(*p);
//write(new_sock, buf, len);
//if ( buf[0] == '.' ) break;
printf("Did not receive message\n");
close(new_sock);
return NULL;
} else
close(new_sock);
All the messages up untill "FORK" are being diplayed, and the thrad hangs on the recv call. buf is defined as static char buf[BUFSIZ];
Any reason why the recv call would not be seeing any data?
send(orig_sock, buf, BUFSIZ,0);
and
if ((new_sock = accept( orig_sock, (struct sockaddr *) &clnt_adr, &clnt_len)) < 0) {
don't make sense together. Either orig_sock is connected or it is listening. Not both. Your lack of error checking on the send() is masking this error.
You can't send to a listening socket.
I am trying to use non-blocking TCP sockets. The problem is that they are still blocking. The code is below -
server code -
struct sockaddr name;
char buf[80];
void set_nonblock(int socket) {
int flags;
flags = fcntl(socket,F_GETFL,0);
assert(flags != -1);
fcntl(socket, F_SETFL, flags | O_NONBLOCK);
}
int main(int agrc, char** argv) {
int sock, new_sd, adrlen; //sock is this socket, new_sd is connection socket
name.sa_family = AF_UNIX;
strcpy(name.sa_data, "127.0.0.1");
adrlen = strlen(name.sa_data) + sizeof(name.sa_family);
//make socket
sock = socket(AF_UNIX, SOCK_STREAM, 0);
if (sock < 0) {
printf("\nBind error %m", errno);
exit(1);
}
//unlink and bind
unlink("127.0.0.1");
if(bind (sock, &name, adrlen) < 0)
printf("\nBind error %m", errno);
//listen
if(listen(sock, 5) < 0)
printf("\nListen error %m", errno);
//accept
new_sd = accept(sock, &name, (socklen_t*)&adrlen);
if( new_sd < 0) {
cout<<"\nserver accept failure "<<errno;
exit(1);
}
//set nonblock
set_nonblock(new_sd);
char* in = new char[80];
std::string out = "Got it";
int numSent;
int numRead;
while( !(in[0] == 'q' && in[1] == 'u' && in[2] == 'i' && in[3] == 't') ) {
//clear in buffer
for(int i=0;i<80;i++)
in[i] = ' ';
cin>>out;
cin.get();
//if we typed something, send it
if(strlen(out.c_str()) > 0) {
numSent = send(new_sd, out.c_str(), strlen(out.c_str()), 0);
cout<<"\n"<<numSent<<" bytes sent";
}
numRead = recv(new_sd, in, 80, 0);
if(numRead > 0)
cout<<"\nData read from client - "<<in;
} //end while
cout<<"\nExiting normally\n";
return 0;
}
client code -
struct sockaddr name;
void set_nonblock(int socket) {
int flags;
flags = fcntl(socket,F_GETFL,0);
assert(flags != -1);
fcntl(socket, F_SETFL, flags | O_NONBLOCK);
}
int main(int agrc, char** argv) {
int sock, new_sd, adrlen;
sock = socket(AF_UNIX, SOCK_STREAM, 0);
if (sock < 0) {
printf("\nserver socket failure %m", errno);
exit(1);
}
//stuff for server socket
name.sa_family = AF_UNIX;
strcpy(name.sa_data, "127.0.0.1");
adrlen = strlen(name.sa_data) + sizeof(name.sa_family);
if(connect(sock, &name, adrlen) < 0) {
printf("\nclient connection failure %m", errno);
exit(1);
}
cout<<"\nSuccessful connection\n";
//set nonblock
set_nonblock(sock);
std::string out;
char* in = new char[80];
int numRead;
int numSent;
while(out.compare("quit")) {
//clear in
for(int i=0;i<80;i++)
in[i] = '\0';
numRead = recv(sock, in, 80, 0);
if(numRead > 0)
cout<<"\nData read from server - "<<in;
cout<<"\n";
out.clear();
cin>>out;
cin.get();
//if we typed something, send it
if(strlen(out.c_str())) {
numSent = send(sock, out.c_str(), strlen(out.c_str()), 0);
cout<<"\n"<<numSent<<" bytes sent";
}
} //end while
cout<<"\nExiting normally\n";
return 0;
}
Whenever I run it, the server still waits for me to send something before it will read and output what the client has sent. I want either the server or client to be able to send the message as soon as I type it, and have the other read and output the message at that time. I thought non-blocking sockets was the answer, but maybe I am just doing something wrong?
Also, I was using a file instead of my 127.0.0.1 address as the sockaddr's data. If that is not how it should be properly used, feel free to say so (it worked how it worked previously with a file so I just kept it like that).
Any help is appreciated.
General approach for a TCP server where you want to handle many connections at the same time:
make listening socket non-blocking
add it to select(2) or poll(2) read event set
enter select(2)/poll(2) loop
on wakeup check if it's the listening socket, then
accept(2)
check for failure (the client might've dropped the connection attempt by now)
make newly created client socket non-blocking, add it to the polling event set
else, if it's one of the client sockets
consume input, process it
watch out for EAGAIN error code - it's not really an error, but indication that there's no input now
if read zero bytes - client closed connection, close(2) client socket, remove it from event set
re-init event set (omitting this is a common error with select(2))
repeat the loop
Client side is a little simpler since you only have one socket. Advanced applications like web browsers that handle many connections often do non-blocking connect(2) though.
Whenever I run it, the server still waits for me to send something before it will read and output what the client has sent.
Well, that is how you wrote it. You block on IO from stdin, and then and only then do you send/receive.
cin>>out;
cin.get();
Also, you are using a local socket (AF_UNIX) which creates a special file in your filesystem for interprocess communication - this is a different mechanism than IP, and is definitely not TCP as you indicate in your question. I suppose you could name the file 127.0.0.1, but that really doesn't make sense and implies confusion on your part, because that is an IP loopback address. You'll want to use AF_INET for IP.
For an excellent starter guide on unix networking, I'd recommend http://beej.us/guide/bgnet/
If you want the display of messages received to be independant of your cin statements, either fork() off a seperate process to handle your network IO, or use a separate thread.
You might be interested in select(). In my opinion non-blocking sockets are usually a hack, and proper usage of select() or poll() is generally much better design and more flexible (and more portable). try
man select_tut
for more information.
I think you have to set non-block sooner (ie get the socket then set it non block)
also check that the fcntl to set it actually worked
If you want non-blocking i/o, you want to use select. You can set it with stdin as one of the sockets it is listening on, along with the client sockets (just add file descriptor 1, which is stdin, to the fd_set).
http://beej.us/guide/bgnet/output/html/multipage/advanced.html
I would recommend reading through what beej has to say about select. It looks a little intimidating but is really useful and simple to use if you take a little time to wrap your head around it.
One of my projects on Linux uses blocking sockets. Things happen very serially so non-blocking would just make things more complicated. Anyway, I am finding that often a recv() call is returning -1 with errno set to EAGAIN.
The man page only really mentions this happening for non-blocking sockets, which makes sense. With non-blocking, the socket may or may not be available so you might need to try again.
What would cause it to happen for a blocking socket? Can I do anything to avoid it?
At the moment, my code to deal with it looks something like this (I have it throw an exception on error, but beyond that it is a very simple wrapper around recv()):
int ret;
do {
ret = ::recv(socket, buf, len, flags | MSG_NOSIGNAL);
} while(ret == -1 && errno == EAGAIN);
if(ret == -1) {
throw socket_error(strerror(errno));
}
return ret;
Is this even correct? The EAGAIN condition gets hit pretty often.
EDIT: some things which I've noticed which may be relevant.
I do set a read timeout on the socket using setsockopts(), but it is set to 30 seconds. the EAGAIN's happen way more often than once every 30 secs. CORRECTION my debugging was flawed, EAGAIN's don't happen as often as I thought they did. Perhaps it is the timeout triggering.
For connecting, I want to be able to have connect timeout, so I temporarily set the socket to non-blocking. That code looks like this:
int error = 0;
fd_set rset;
fd_set wset;
int n;
const SOCKET sock = m_Socket;
// set the socket as nonblocking IO
const int flags = fcntl (sock, F_GETFL, 0);
fcntl(sock, F_SETFL, flags | O_NONBLOCK);
errno = 0;
// we connect, but it will return soon
n = ::connect(sock, addr, size_addr);
if(n < 0) {
if (errno != EINPROGRESS) {
return -1;
}
} else if (n == 0) {
goto done;
}
FD_ZERO(&rset);
FD_ZERO(&wset);
FD_SET(sock, &rset);
FD_SET(sock, &wset);
struct timeval tval;
tval.tv_sec = timeout;
tval.tv_usec = 0;
// We "select()" until connect() returns its result or timeout
n = select(sock + 1, &rset, &wset, 0, timeout ? &tval : 0);
if(n == 0) {
errno = ETIMEDOUT;
return -1;
}
if (FD_ISSET(sock, &rset) || FD_ISSET(sock, &wset)) {
socklen_t len = sizeof(error);
if (getsockopt(SOL_SOCKET, SO_ERROR, &error, &len) < 0) {
return -1;
}
} else {
return -1;
}
done:
// We change the socket options back to blocking IO
if (fcntl(sock, F_SETFL, flags) == -1) {
return -1;
}
return 0;
The idea is that I set it to non-blocking, attempt a connect and select on the socket so I can enforce a timeout. Both the set and restore fcntl() calls return successfully, so the socket should end up in blocking mode again when this function completes.
It's possible that you have a nonzero receive timeout set on the socket (via setsockopt(sock, SOL_SOCKET, SO_RCVTIMEO,...)) as that would also cause recv to return EAGAIN
Is it possible that you're using MSG_DONTWAIT is being specified as part of your flags? The man page says EAGAIN will occur if no data is available and this flag is specified.
If you really want to force a block until the recv() is somewhat successful, you may wish to use the MSG_WAITALL flag.
I don't suggest this as a first-attempt fix, but if you're all out of options, you can always select() on the socket with a reasonably long timeout to force it to wait for data.
EAGAIN is generated by the OS almost like an "Oops! I'm sorry for disturbing you.". In case of this error, you may try reading again, This is not a serious or fatal error. I have seen these interrupts occur in Linux and LynxOS anywhere from one a day to 100 times a day.