I have two threads running in my program. One is sending data to the other. The socket is connecting fine on both threads, and the receiving thread is accepting the connection. However, once the recv is called, it just sits there blocking as if its getting nothing in, despite the fact that the other thread is sending.
Sending thread
send(orig_sock, buf, BUFSIZ,0);
printf("Client sending chunk\n");
The printf manages to get displayed.
Receiving thread
printf("START ACCEPTING\n");
if ((new_sock = accept( orig_sock, (struct sockaddr *) &clnt_adr, &clnt_len)) < 0) {
perror("accept error");
close(orig_sock);
return NULL;
}
printf("PASS ACCEPT\n");
if ( fork( ) == 0 ) { // Generate a CHILD
printf("FORK\n");
len=recv(new_sock, buf, BUFLEN,0 );
printf("message received");
receiveBuffer.push(*p);
//write(new_sock, buf, len);
//if ( buf[0] == '.' ) break;
printf("Did not receive message\n");
close(new_sock);
return NULL;
} else
close(new_sock);
All the messages up untill "FORK" are being diplayed, and the thrad hangs on the recv call. buf is defined as static char buf[BUFSIZ];
Any reason why the recv call would not be seeing any data?
send(orig_sock, buf, BUFSIZ,0);
and
if ((new_sock = accept( orig_sock, (struct sockaddr *) &clnt_adr, &clnt_len)) < 0) {
don't make sense together. Either orig_sock is connected or it is listening. Not both. Your lack of error checking on the send() is masking this error.
You can't send to a listening socket.
Related
I have been struggling with this for a few days now figuring out how they work. I have read the documentation and looked at some examples but I am still in need of guidance.
Specifically, when the client calls connect() and successfully connects to the server host, should I issue the SSL_connect() right after it to Initiate the handshake ? The client then tries to write some bytes to the socket using SSL_write().
On the other hand, the server uses pselect() to monitor any read fds ready for read and issues the accept() call successfully for the incoming connection. Should I issue the SSL_accept() call right after the accept() returns to complete the handshake ?
I have noticed that the SSL_connect() returns SSL_ERROR_WANT_READ (this is when the SSL_connect() is issued after a select() call to monitor the write fd set and returns and as per the Openssl documentation).
What is the right procedure here on issuing the calls and in what order ?
Edit to add snippet of code -
Client side :
err = connect(fd, addr, addrlen);
if ( err == -1 && errno == EINPROGRESS )
{
// check if this is a true error,
// or wait until connect times out
fd_set fdset;
FD_ZERO(&fdset);
FD_SET(sfd, &fdset);
timeval tv = {F_sockwaitconnect, 0}; TEMP_FAILURE_RETRY(err = select(fd + 1,\
NULL,\
&fdset,\
NULL,\
&tv));
// what happened?
if ( err == 1 )
{
connect was successful
}
else
return 0;
const SSL_METHOD *method;
SSL_CTX *cctx;
SSL *cssl;
FILE *fp;
fp = stdout;
ERR_clear_error();
method = TLSv1_client_method();
cctx = SSL_CTX_new(method);
if ( cctx == NULL )
{
ERR_print_errors_fp(stdout);
return 0;
}
SSL_CTX_set_verify(cctx, SSL_VERIFY_PEER, NULL);
SSL_CTX_set_verify_depth(cctx, 4);
if (SSL_CTX_load_verify_locations(cctx, "mycert.pem", NULL) == 0)
return 0;
SSL_CTX_set_options(cctx, SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_TLSv1_1 | SSL_OP_NO_COMPRESSION);
ERR_clear_error();
cssl = SSL_new(cctx); /* create new SSL connection state */
SSL_set_fd(cssl, fd); * attach the socket descriptor */
ERR_clear_error();
int rconnect = SSL_ERROR_WANT_READ;
while ( rconnect == SSL_ERROR_WANT_READ || rconnect == SSL_ERROR_WANT_WRITE )
{
char *buf = (char *) malloc(124);
ERR_error_string(SSL_get_error(cssl, rconnect), buf);
ERR_clear_error();
if ( rconnect == SSL_ERROR_WANT_READ ) {
int err = 0;
fd_set fdset;
FD_ZERO(&fdset);
FD_SET(fd, &fdset);
timeval tv = {F_sockwaitconnect, 0};
TEMP_FAILURE_RETRY(err1 = select(fd + 1,\
&fdset,\
NULL,\
NULL,\
&tv));
// what happened?
if ( err == 1 )
{
rconnect = SSL_connect(cssl);
}
}
}
X509 *cert;
cert = SSL_get_peer_certificate(cssl);
char line[2000+1];
if ( cert != NULL )
{
X509_NAME_oneline(X509_get_subject_name(cert), line, MAX_SIZE);
X509_NAME_oneline(X509_get_issuer_name(cert), line1, MAX_SIZE);
X509_free(cert);
}
ERR_clear_error();
r = SSL_write(cssl, buffer, len);
< check error >
Server side :
int res = pselect(max_fd + 1, // host socket file descriptor
&fd_setw, // set of ds wait 4 incoming data
NULL, // no write operations
NULL, // no exception operations
&tm, // how much time to wait
&sig_set); // block all signals
if ( event on listening socket )
{
client = accept(sfd, &peer, &peerl);
}
else // incoming data to receive on existing connection
{
SSL *ssl;
FILE *fp = stdout;
if ( !ctx )
{
return 0;
}
ERR_clear_error();
ssl = SSL_new(ctx);
SSL_set_fd(ssl, soc);
int ret = SSL_accept(ssl);
while (ret <= 0) {
ERR_print_errors_fp(fp);
char *buf = (char *) malloc(124);
ERR_error_string(SSL_get_error(ssl, ret), buf);
ERR_clear_error();
ret = SSL_accept(ssl);
}
X509 *cert;
cert = SSL_get_peer_certificate(ssl);
char line[2000+1];
if ( cert != NULL )
{
X509_NAME_oneline(X509_get_subject_name(cert), line, MAX_SIZE);
X509_NAME_oneline(X509_get_issuer_name(cert), line1, MAX_SIZE);
X509_free(cert);
}
// get data and analyze result
int rc = 0;
bool recv_called = false;
rc = SSL_read(ssl, buffer, len);
< check error >
}
Before all the above, the server opens, binds and listens on a non-blocking socket for any new incoming client connections.
When I run the above, the client does the connect() and the server does the accept().
The server is now waiting at pselect() for any fd's to be ready to receive data.
The client on the other hand is at the SSL_connect() and keeps getting the SSL_ERROR_WANT_READ error. The select() returns the socket is ready to read.
My guess is the client is waiting for the SSL_accept() part of the handshake ? I do not know why the server is waiting at pselect(). The code around SSL_accept() is wrong i.e it loops and does not look for the WANT_READ and WANT_WRITE errors but I do not get to that point in the code.
SSL_connect can be called whenever the connect is finished. Since both connect and SSL_connect need to exchange data with the peer they might not succeed immediately when using non-blocking sockets. If SSL_connect returns with an error of SSL_WANT_READ or SSL_WANT_WRITE you have to call it again after new data got available on the socket (SSL_WANT_READ) or the socket is writable (SSL_WANT_WRITE). You can check or wait for this with select, pselect, poll, epoll, kqueue or whatever API your OS provides for this.
SSL_accept and accept are similar, i.e. SSL_accept can be called directly after a successful accept, might not succeed immediately since data exchange is needed with the SSL client and thus you have to call it again if it returns an error of SSL_WANT_READ or SSL_WANT_WRITE.
Note that SSL_write and SSL_read might also result in such errors. i.e. you need to deal with SSL_WANT_READ and SSL_WANT_WRITE also for these functions and also the same way as with SSL_connect and SSL_accept. It might even be that a SSL_read results in a SSL_WANT_WRITE since a SSL renegotiation might happen even if the SSL session was already established.
I'm sending and receiving info with a unix socket, but I do not completely understand how it works. Basically, I send a message like this:
int wr_bytes = write(sock, msg.c_str(), msg.length());
And receive message like this:
int rd_bytes = read(msgsock, buf, SOCKET_BUFFER_SIZE);
This code works perfectly with thousands of bytes, what I don't understand is, how does the read function knows when the other part is done sending the message? I tried to read the read documentation and, on my understanding read will return once it reaches EOF or the SOCKET_BUFFER_SIZE, is that correct?
So I'm guessing that when I give my string to the write function, it adds an EOF at the end of my content so the read function knows when to stop.
I'm asking this question because, I did not add any code that checks whether the other part finished sending the message, however, I'm receiving big messages (thousands of bytes) without any problem, why is that happening, why am I not getting only parts of the message?
Here is the full function I'm using to send a message to a unix socket server:
string sendSocketMessage(string msg) {
int sock;
struct sockaddr_un server;
char buf[1024];
sock = socket(AF_UNIX, SOCK_STREAM, 0);
if (sock < 0) {
throw runtime_error("opening stream socket");
}
server.sun_family = AF_UNIX;
strcpy(server.sun_path, "socket");
if (connect(sock, (struct sockaddr *) &server, sizeof(struct sockaddr_un)) < 0) {
close(sock);
throw runtime_error("connecting stream socket");
}
if (write(sock, msg.c_str(), msg.length()) < 0){
throw runtime_error("writing on stream socket");
close(sock);
}
bzero(buf, sizeof(buf));
int rval = read(sock, buf, 1024);
return string( reinterpret_cast< char const* >(buf), rval );
}
And here is my server function (a little bit more complicated, the type vSocketHandler represents a function that I call to handle requests):
void UnixSocketServer::listenRequests(vSocketHandler requestHandler){
int sock, msgsock, rval;
struct sockaddr_un server;
char buf[SOCKET_BUFFER_SIZE];
sock = socket(AF_UNIX, SOCK_STREAM, 0);
if (sock < 0) {
throw runtime_error("opening stream socket");
}
server.sun_family = AF_UNIX;
strcpy(server.sun_path, SOCKET_FILE_PATH);
if (bind(sock, (struct sockaddr *) &server, sizeof(struct sockaddr_un))) {
throw runtime_error("binding stream socket");
}
listen(sock, SOCKET_MAX_CONNECTIONS);
while(true) {
msgsock = accept(sock, 0, 0);
if (msgsock == -1){
throw runtime_error("accept socket");
} else {
bzero(buf, sizeof(buf));
if((rval = read(msgsock, buf, SOCKET_BUFFER_SIZE)) < 0)
throw runtime_error("reading stream message");
else if (rval == 0){
//do nothing, client closed socket
break;
} else {
string msg = requestHandler(string( reinterpret_cast< char const* >(buf), rval ));
if(write(msgsock, msg.c_str(), msg.length()) < 0)
throw runtime_error("sending stream message");
}
close(msgsock);
}
}
close(sock);
unlink(SOCKET_FILE_PATH);
}
what I don't understand is, how does the read function knows when the other part is done sending the message?
For a stream-type socket, such as you're using, it doesn't. For a datagram-type socket, communication is broken into distinct chunks, but if a message spans multiple datagrams then the answer is again "it doesn't". This is indeed one of the key things to understand about the read() and write() (and send() and recv()) functions in general, and about sockets more specifically.
For the rest of this answer I'll focus on stream oriented sockets, since that's what you're using. I'll also suppose that the socket is not in non-blocking mode. If you intend for your data transmitted over such a socket to be broken into distinct messages, then it is up to you to implement an application-level protocol by which the other end can recognize message boundaries.
I tried to read the read documentation and, on my understanding read will return once it reaches EOF or the SOCKET_BUFFER_SIZE, is that correct?
Not exactly. read() will return if it reaches the end of the file, which happens when the peer closes its socket (or at least shuts down the write side of it) so that it is certain that no more data will be sent. read() will also return in the event of any of a variety of error conditions. And read() may return under other unspecified circumstances, provided that it has transferred at least one byte. In practice, this last case is generally invoked if the socket buffer fills, but it may also be invoked under other circumstances, such as when the buffer empties.
So I'm guessing that when I give my string to the write function, it adds an EOF at the end of my content so the read function knows when to stop.
No, it does no such thing. On success, the write() function sends some or all of the bytes you asked it to send, and nothing else. Note that it is not guaranteed even to send all the requested bytes; its return value tells you how many of them it actually did send. If that's fewer than "all", then ordinarily you should simply perform another write() to transfer the rest. You may need to do this multiple times to send the whole message. In any event, only the bytes you specify are sent.
I'm asking this question because, I did not add any code that checks whether the other part finished sending the message, however, I'm receiving big messages (thousands of bytes) without any problem, why is that happening, why am I not getting only parts of the message?
More or less because you're getting lucky, but the fact that you're using UNIX-domain sockets (as opposed to network sockets) helps. Your data are transferred very efficiently from sending process to receiving process through the kernel, and it is not particularly surprising that large writes() are received by single read()s. You cannot safely rely on that always to happen, however.
i want to write a simple TCP proxy in C++ for the University. The proxy works with two threads one reads from source port and writes to the destination port and the other thread does the same in the other direction. The aim is to read and manipulate the packets in the future. If i use the mutex to lock the port for read and write on the same port i got package loss. Can you help me to locate the problem because i tried it a long time now?
thread1 = 0;
thread2 = 0;
//Client
struct sockaddr_in address;
int size;
if ((create_socket=socket (AF_INET, SOCK_STREAM, 0)) > 0)
printf ("Socket wurde angelegt\n");
address.sin_family = AF_INET;
address.sin_port = htons (PORT);
inet_aton (IP, &address.sin_addr);
if (connect ( create_socket, (struct sockaddr *) &address, sizeof (address)) == 0)
printf ("Verbindung mit dem Server (%s) hergestellt\n", inet_ntoa (address.sin_addr));
//Server
socklen_t addrlen;
struct sockaddr_in address2;
const int y = 1;
if ((create_socket2=socket (AF_INET, SOCK_STREAM, 0)) > 0)
printf ("Socket wurde angelegt\n");
setsockopt( create_socket2, SOL_SOCKET, SO_REUSEADDR, &y, sizeof(int));
address2.sin_family = AF_INET;
address2.sin_addr.s_addr = INADDR_ANY;
address2.sin_port = htons (PORT2);
if (bind ( create_socket2, (struct sockaddr *) &address2, sizeof (address2)) != 0) {
printf( "Der Port ist nicht frei – belegt!\n");
}
listen (create_socket2, 5);
addrlen = sizeof (struct sockaddr_in);
new_socket2 = accept ( create_socket2, (struct sockaddr *) &address2, &addrlen );
if (new_socket2 > 0)
printf ("Ein Client (%s) ist verbunden ...\n", inet_ntoa (address2.sin_addr));
thread apm(apm_gcs);
thread gcs(gcs_apm);
apm.join();
gcs.join();
}
inline void apm_gcs()
{
while (STOP==FALSE)
{
{
lock_guard<mutex> lock(tcp60Mutex);
res = read(create_socket, buffer2, sizeof(buffer2)); // returns after 5 chars have been input
}
{
lock_guard<mutex> lock(tcp65Mutex);
write(new_socket2, buffer2, res);
}
}
}
inline void gcs_apm()
{
while (STOP==FALSE)
{
{
lock_guard<mutex> lock(tcp65Mutex);
res2 = read(new_socket2, buffer, sizeof(buffer)); // returns after 5 chars have been input
}
{
lock_guard<mutex> lock(tcp60Mutex);
write(create_socket, buffer, res2);
}
}
}
Thank you for your help.
Greets
Tobi
There are several things to improve.
First of all: It's not clear what exactly you want to protect. I would understand if you would use one mutex to protect one buffer, and the other mutex for the other buffer, so each buffer will always be accessed by only one thread. However, that does not happen - both threads can read+write the same buffer at the same time. Instead, each mutex protects a socket against read+write at the same time, which is pointless because sockets can handle that perfectly. You can read+write on the same socket at the same time. sockets are used to do that for more than 30 years now.
Once that is changed and your mutexes protect buffers, you will run into blocking again, though less often. You will experience that a thread tries to read or write data while none is available, or the socket connection is full (which happens if you try to quickly write large amounts of data) and it takes time to transfer the data.
This can be solved then by select() or maybe by poll(). Thus the way to go is:
Each thread uses select() or poll() to find out if it can read or write data. Only if it can, it locks the mutex for the buffer, then read or write data (which won't block after select() or poll() made that sure) and then releases the mutex.
So I am trying to setup a multithreaded server with ACE. I am using non-blocking client sockets to prevent recv()/send() from blocking. The problem is when I use recv() and the client disconnects ungraceful, the result of recv() does not give me a hind that the client disconnected. Is there any other methode to check the connectivity.
Here is a short snippet
char buffer[4096];
ssize_t bytesReceived = peer.recv(buffer, sizeof(buffer));
if (bytesReceived < 1 && errno != EWOULDBLOCK)
{
printf("Disconnected:\n");
}
else if (bytesReceived > 0)
{
buffer[bytesReceived] = '\0';
printf(buffer);
}
So if the client disconnects, recv returns -1 but errno is still EWOULDBLOCK.
I also tried to use a short timeout in recv, but it leads to the same result as without just with errno = ETIME(TIME-Out).
I am trying to use non-blocking TCP sockets. The problem is that they are still blocking. The code is below -
server code -
struct sockaddr name;
char buf[80];
void set_nonblock(int socket) {
int flags;
flags = fcntl(socket,F_GETFL,0);
assert(flags != -1);
fcntl(socket, F_SETFL, flags | O_NONBLOCK);
}
int main(int agrc, char** argv) {
int sock, new_sd, adrlen; //sock is this socket, new_sd is connection socket
name.sa_family = AF_UNIX;
strcpy(name.sa_data, "127.0.0.1");
adrlen = strlen(name.sa_data) + sizeof(name.sa_family);
//make socket
sock = socket(AF_UNIX, SOCK_STREAM, 0);
if (sock < 0) {
printf("\nBind error %m", errno);
exit(1);
}
//unlink and bind
unlink("127.0.0.1");
if(bind (sock, &name, adrlen) < 0)
printf("\nBind error %m", errno);
//listen
if(listen(sock, 5) < 0)
printf("\nListen error %m", errno);
//accept
new_sd = accept(sock, &name, (socklen_t*)&adrlen);
if( new_sd < 0) {
cout<<"\nserver accept failure "<<errno;
exit(1);
}
//set nonblock
set_nonblock(new_sd);
char* in = new char[80];
std::string out = "Got it";
int numSent;
int numRead;
while( !(in[0] == 'q' && in[1] == 'u' && in[2] == 'i' && in[3] == 't') ) {
//clear in buffer
for(int i=0;i<80;i++)
in[i] = ' ';
cin>>out;
cin.get();
//if we typed something, send it
if(strlen(out.c_str()) > 0) {
numSent = send(new_sd, out.c_str(), strlen(out.c_str()), 0);
cout<<"\n"<<numSent<<" bytes sent";
}
numRead = recv(new_sd, in, 80, 0);
if(numRead > 0)
cout<<"\nData read from client - "<<in;
} //end while
cout<<"\nExiting normally\n";
return 0;
}
client code -
struct sockaddr name;
void set_nonblock(int socket) {
int flags;
flags = fcntl(socket,F_GETFL,0);
assert(flags != -1);
fcntl(socket, F_SETFL, flags | O_NONBLOCK);
}
int main(int agrc, char** argv) {
int sock, new_sd, adrlen;
sock = socket(AF_UNIX, SOCK_STREAM, 0);
if (sock < 0) {
printf("\nserver socket failure %m", errno);
exit(1);
}
//stuff for server socket
name.sa_family = AF_UNIX;
strcpy(name.sa_data, "127.0.0.1");
adrlen = strlen(name.sa_data) + sizeof(name.sa_family);
if(connect(sock, &name, adrlen) < 0) {
printf("\nclient connection failure %m", errno);
exit(1);
}
cout<<"\nSuccessful connection\n";
//set nonblock
set_nonblock(sock);
std::string out;
char* in = new char[80];
int numRead;
int numSent;
while(out.compare("quit")) {
//clear in
for(int i=0;i<80;i++)
in[i] = '\0';
numRead = recv(sock, in, 80, 0);
if(numRead > 0)
cout<<"\nData read from server - "<<in;
cout<<"\n";
out.clear();
cin>>out;
cin.get();
//if we typed something, send it
if(strlen(out.c_str())) {
numSent = send(sock, out.c_str(), strlen(out.c_str()), 0);
cout<<"\n"<<numSent<<" bytes sent";
}
} //end while
cout<<"\nExiting normally\n";
return 0;
}
Whenever I run it, the server still waits for me to send something before it will read and output what the client has sent. I want either the server or client to be able to send the message as soon as I type it, and have the other read and output the message at that time. I thought non-blocking sockets was the answer, but maybe I am just doing something wrong?
Also, I was using a file instead of my 127.0.0.1 address as the sockaddr's data. If that is not how it should be properly used, feel free to say so (it worked how it worked previously with a file so I just kept it like that).
Any help is appreciated.
General approach for a TCP server where you want to handle many connections at the same time:
make listening socket non-blocking
add it to select(2) or poll(2) read event set
enter select(2)/poll(2) loop
on wakeup check if it's the listening socket, then
accept(2)
check for failure (the client might've dropped the connection attempt by now)
make newly created client socket non-blocking, add it to the polling event set
else, if it's one of the client sockets
consume input, process it
watch out for EAGAIN error code - it's not really an error, but indication that there's no input now
if read zero bytes - client closed connection, close(2) client socket, remove it from event set
re-init event set (omitting this is a common error with select(2))
repeat the loop
Client side is a little simpler since you only have one socket. Advanced applications like web browsers that handle many connections often do non-blocking connect(2) though.
Whenever I run it, the server still waits for me to send something before it will read and output what the client has sent.
Well, that is how you wrote it. You block on IO from stdin, and then and only then do you send/receive.
cin>>out;
cin.get();
Also, you are using a local socket (AF_UNIX) which creates a special file in your filesystem for interprocess communication - this is a different mechanism than IP, and is definitely not TCP as you indicate in your question. I suppose you could name the file 127.0.0.1, but that really doesn't make sense and implies confusion on your part, because that is an IP loopback address. You'll want to use AF_INET for IP.
For an excellent starter guide on unix networking, I'd recommend http://beej.us/guide/bgnet/
If you want the display of messages received to be independant of your cin statements, either fork() off a seperate process to handle your network IO, or use a separate thread.
You might be interested in select(). In my opinion non-blocking sockets are usually a hack, and proper usage of select() or poll() is generally much better design and more flexible (and more portable). try
man select_tut
for more information.
I think you have to set non-block sooner (ie get the socket then set it non block)
also check that the fcntl to set it actually worked
If you want non-blocking i/o, you want to use select. You can set it with stdin as one of the sockets it is listening on, along with the client sockets (just add file descriptor 1, which is stdin, to the fd_set).
http://beej.us/guide/bgnet/output/html/multipage/advanced.html
I would recommend reading through what beej has to say about select. It looks a little intimidating but is really useful and simple to use if you take a little time to wrap your head around it.