SSL_connect() and SSL_accept() calls - c++

I have been struggling with this for a few days now figuring out how they work. I have read the documentation and looked at some examples but I am still in need of guidance.
Specifically, when the client calls connect() and successfully connects to the server host, should I issue the SSL_connect() right after it to Initiate the handshake ? The client then tries to write some bytes to the socket using SSL_write().
On the other hand, the server uses pselect() to monitor any read fds ready for read and issues the accept() call successfully for the incoming connection. Should I issue the SSL_accept() call right after the accept() returns to complete the handshake ?
I have noticed that the SSL_connect() returns SSL_ERROR_WANT_READ (this is when the SSL_connect() is issued after a select() call to monitor the write fd set and returns and as per the Openssl documentation).
What is the right procedure here on issuing the calls and in what order ?
Edit to add snippet of code -
Client side :
err = connect(fd, addr, addrlen);
if ( err == -1 && errno == EINPROGRESS )
{
// check if this is a true error,
// or wait until connect times out
fd_set fdset;
FD_ZERO(&fdset);
FD_SET(sfd, &fdset);
timeval tv = {F_sockwaitconnect, 0}; TEMP_FAILURE_RETRY(err = select(fd + 1,\
NULL,\
&fdset,\
NULL,\
&tv));
// what happened?
if ( err == 1 )
{
connect was successful
}
else
return 0;
const SSL_METHOD *method;
SSL_CTX *cctx;
SSL *cssl;
FILE *fp;
fp = stdout;
ERR_clear_error();
method = TLSv1_client_method();
cctx = SSL_CTX_new(method);
if ( cctx == NULL )
{
ERR_print_errors_fp(stdout);
return 0;
}
SSL_CTX_set_verify(cctx, SSL_VERIFY_PEER, NULL);
SSL_CTX_set_verify_depth(cctx, 4);
if (SSL_CTX_load_verify_locations(cctx, "mycert.pem", NULL) == 0)
return 0;
SSL_CTX_set_options(cctx, SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_TLSv1_1 | SSL_OP_NO_COMPRESSION);
ERR_clear_error();
cssl = SSL_new(cctx); /* create new SSL connection state */
SSL_set_fd(cssl, fd); * attach the socket descriptor */
ERR_clear_error();
int rconnect = SSL_ERROR_WANT_READ;
while ( rconnect == SSL_ERROR_WANT_READ || rconnect == SSL_ERROR_WANT_WRITE )
{
char *buf = (char *) malloc(124);
ERR_error_string(SSL_get_error(cssl, rconnect), buf);
ERR_clear_error();
if ( rconnect == SSL_ERROR_WANT_READ ) {
int err = 0;
fd_set fdset;
FD_ZERO(&fdset);
FD_SET(fd, &fdset);
timeval tv = {F_sockwaitconnect, 0};
TEMP_FAILURE_RETRY(err1 = select(fd + 1,\
&fdset,\
NULL,\
NULL,\
&tv));
// what happened?
if ( err == 1 )
{
rconnect = SSL_connect(cssl);
}
}
}
X509 *cert;
cert = SSL_get_peer_certificate(cssl);
char line[2000+1];
if ( cert != NULL )
{
X509_NAME_oneline(X509_get_subject_name(cert), line, MAX_SIZE);
X509_NAME_oneline(X509_get_issuer_name(cert), line1, MAX_SIZE);
X509_free(cert);
}
ERR_clear_error();
r = SSL_write(cssl, buffer, len);
< check error >
Server side :
int res = pselect(max_fd + 1, // host socket file descriptor
&fd_setw, // set of ds wait 4 incoming data
NULL, // no write operations
NULL, // no exception operations
&tm, // how much time to wait
&sig_set); // block all signals
if ( event on listening socket )
{
client = accept(sfd, &peer, &peerl);
}
else // incoming data to receive on existing connection
{
SSL *ssl;
FILE *fp = stdout;
if ( !ctx )
{
return 0;
}
ERR_clear_error();
ssl = SSL_new(ctx);
SSL_set_fd(ssl, soc);
int ret = SSL_accept(ssl);
while (ret <= 0) {
ERR_print_errors_fp(fp);
char *buf = (char *) malloc(124);
ERR_error_string(SSL_get_error(ssl, ret), buf);
ERR_clear_error();
ret = SSL_accept(ssl);
}
X509 *cert;
cert = SSL_get_peer_certificate(ssl);
char line[2000+1];
if ( cert != NULL )
{
X509_NAME_oneline(X509_get_subject_name(cert), line, MAX_SIZE);
X509_NAME_oneline(X509_get_issuer_name(cert), line1, MAX_SIZE);
X509_free(cert);
}
// get data and analyze result
int rc = 0;
bool recv_called = false;
rc = SSL_read(ssl, buffer, len);
< check error >
}
Before all the above, the server opens, binds and listens on a non-blocking socket for any new incoming client connections.
When I run the above, the client does the connect() and the server does the accept().
The server is now waiting at pselect() for any fd's to be ready to receive data.
The client on the other hand is at the SSL_connect() and keeps getting the SSL_ERROR_WANT_READ error. The select() returns the socket is ready to read.
My guess is the client is waiting for the SSL_accept() part of the handshake ? I do not know why the server is waiting at pselect(). The code around SSL_accept() is wrong i.e it loops and does not look for the WANT_READ and WANT_WRITE errors but I do not get to that point in the code.

SSL_connect can be called whenever the connect is finished. Since both connect and SSL_connect need to exchange data with the peer they might not succeed immediately when using non-blocking sockets. If SSL_connect returns with an error of SSL_WANT_READ or SSL_WANT_WRITE you have to call it again after new data got available on the socket (SSL_WANT_READ) or the socket is writable (SSL_WANT_WRITE). You can check or wait for this with select, pselect, poll, epoll, kqueue or whatever API your OS provides for this.
SSL_accept and accept are similar, i.e. SSL_accept can be called directly after a successful accept, might not succeed immediately since data exchange is needed with the SSL client and thus you have to call it again if it returns an error of SSL_WANT_READ or SSL_WANT_WRITE.
Note that SSL_write and SSL_read might also result in such errors. i.e. you need to deal with SSL_WANT_READ and SSL_WANT_WRITE also for these functions and also the same way as with SSL_connect and SSL_accept. It might even be that a SSL_read results in a SSL_WANT_WRITE since a SSL renegotiation might happen even if the SSL session was already established.

Related

FD_ISSET always true even if there is no new data?

I am trying to check if a client has send some new data. This actually tells me that i always have new data:
bool ClientHandle::hasData()
{
fd_set temp;
FD_ZERO(&temp);
FD_SET(m_sock, &temp);
//setup the timeout to 1000ms
struct timeval tv;
tv.tv_sec = 0;
tv.tv_usec = 1000;
//temp.fd_count possible?
if (select(m_sock+1, &temp, nullptr, nullptr, &tv) == -1)
{
return false;
}
if (FD_ISSET(m_sock, &temp))
return true;
return false;
}
I am connecting with a java client and send a "connection" message which i read inside of the ctor:
ClientHandle::ClientHandle(SOCKET s) : m_sock(s)
{
while (!hasData())
{
}
char buffer[5];
recv(m_sock, buffer, 4, NULL);
auto i = atoi(buffer);
LOG_INFO << "Byte to receive: " << i;
auto dataBuffer = new char[i + 1]{'\0'};
recv(m_sock, dataBuffer, i, NULL);
LOG_INFO << dataBuffer;
//clean up
delete[] dataBuffer;
}
This seems to work right. After that i keep checking if there is new data which always is true even if the java client does not send any new data.
Here is the java client. Don't judge me it's just for checking the connections. It wont stay like this to send the size information as char[].
public static void main(String[] args) throws UnknownHostException,
IOException {
Socket soc = null;
soc = new Socket("localhost", 6060);
PrintWriter out = new PrintWriter(soc.getOutputStream(), true);
BufferedReader in = new BufferedReader(new InputStreamReader(
soc.getInputStream()));
if (soc != null)
System.out.println("Connected");
out.write("10\0");
out.flush();
out.write("newCon\0");
out.flush();
out.close();
in.close();
soc.close();
}
So what is wrong with the hasData FD_ISSET method?
So what is wrong with the hasData FD_ISSET method?
Actually no. There is a problem with your use of recv().
recv() will return 0 if the client is disconnected and will return this until you close the socket (server-side). You can find this information in the manual.
Even if recv() returns 0, it will "trigger" select().
Knowing that, it's easy to find out the problem: you never check the return value of recv() and so you're unable to say if the client is still connected or not. However, you still add it with FD_SET!
#include <sys/types.h> // for ssize_t
#include <stdio.h> // for perror()
ClientHandle::ClientHandle(SOCKET s) : m_sock(s)
{
while (!hasData())
{
}
char buffer[5];
ssize_t ret = recv(m_sock, buffer, 4, NULL);
if (ret == -1) // error
{
perror("recv");
return ;
}
else if (ret == 0) // m_sock disconnects
{
close(m_sock);
// DO NOT FD_SET m_sock since the socket is now closed
}
else
{
auto i = atoi(buffer);
LOG_INFO << "Byte to receive: " << i;
auto dataBuffer = new char[i + 1]{'\0'};
recv(m_sock, dataBuffer, i, NULL);
LOG_INFO << dataBuffer;
//clean up
delete[] dataBuffer;
}
}
From Steven's book UNIX Networking Programming:
A socket is ready for reading if any of the following four conditions is true:
The number of bytes of data in the socket receive buffer is greater than or equal to the current size of the low-water mark for the socket receive buffer. A read operation on the socket will not block and will return a value greater than 0 (i.e., the data that is ready to be read). We can set this low-water mark using the SO_RCVLOWAT socket option. It defaults to 1 for TCP and UDP sockets.
The read half of the connection is closed (i.e., a TCP connection that has received a FIN). A read operation on the socket will not block and will return 0 (i.e., EOF).
The socket is a listening socket and the number of completed connections is nonzero. An accept on the listening socket will normally not block, although we will describe a timing condition in Section 16.6 under which the accept can block.
A socket error is pending. A read operation on the socket will not block and will return an error (–1) with errno set to the specific error condition. These pending errors can also be fetched and cleared by calling getsockopt and specifying the SO_ERROR socket option.
ISSET is going to return true in all the cases above. After your Java client closes the connection, the socket will be ready for reading in the server.
In ClientHandle::ClientHandle you are not checking the return value of recv and if any data is returned.
Is it blocking in the second call to recv?
You don't check the return value of recv and you don't handle receiving fewer bytes than you asked for. So what do you expect to happen when the connection is closed?

multiclent server connected but not receiving messages properly

I have to create a basic p2p connection with c++ sockets, which means each user has a server for listening onto connections and and a client for connecting, right?
For now I'm trying to create a master client which has a dedicated server and is a client too.
This means creating the server and client in the same program and I have used fork() which creates a child process of the server and the parent is the client. Now, fork works fine and I'm using select() to check sockets for reading data and i have modeled the server on this http://beej.us/guide/bgnet/output/html/multipage/advanced.html#select
Now when I run the program, the master client is able to connect to its own dedicated server, but the messages don't always get received by the server. Sometimes, it receives it, sometimes it doesn't. Any idea why?
Also, when a second client gets connected to the master client, and it doesn't have it's own server for now, the server shows that it gets a new connection, but when I write the message and send it, it doesn't receive any message from the second client, but it receives a message from the master client sometimes and not always.
EDIT: Added cout.flush
EDIT: I think forking the process causes some delay when a client and server run on the same program.
UPDATE: Added the new server code which causes a delay by one message (in response to the comments)
Here's the code.
SERVER CODE
while (1) {
unsigned int s;
readsocks = socks;
if (select(maxsock + 1, &readsocks, NULL, NULL, NULL) == -1) {
perror("select");
return ;
}
for (s = 0; s <= maxsock; s++) {
if (FD_ISSET(s, &readsocks)) {
//printf("socket %d was ready\n", s);
if (s == sock) {
/* New connection */
cout<<"\n New Connection";
cout.flush();
int newsock;
struct sockaddr_in their_addr;
socklen_t size = sizeof(their_addr);
newsock = accept(sock, (struct sockaddr*)&their_addr, &size);
if (newsock == -1) {
perror("accept");
}
else {
printf("Got a connection from %s on port %d\n",
inet_ntoa(their_addr.sin_addr), htons(their_addr.sin_port));
FD_SET(newsock, &socks);
if (newsock > maxsock) {
maxsock = newsock;
}
}
}
else {
/* Handle read or disconnection */
handle(s, &socks);
}
}
}
}
void handle(int newsock, fd_set *set)
{
char buf[256];
bzero(buf, 256);
/* send(), recv(), close() */
if(read(newsock, buf, 256)<=0){
cout<<"\n No data";
FD_CLR(newsock, set);
cout.flush();
}
else {
string temp(buf);
cout<<"\n Server: "<<temp;
cout.flush();
}
/* Call FD_CLR(newsock, set) on disconnection */
}

Non-blocking ACE_SOCK_Stream and client disconnect

So I am trying to setup a multithreaded server with ACE. I am using non-blocking client sockets to prevent recv()/send() from blocking. The problem is when I use recv() and the client disconnects ungraceful, the result of recv() does not give me a hind that the client disconnected. Is there any other methode to check the connectivity.
Here is a short snippet
char buffer[4096];
ssize_t bytesReceived = peer.recv(buffer, sizeof(buffer));
if (bytesReceived < 1 && errno != EWOULDBLOCK)
{
printf("Disconnected:\n");
}
else if (bytesReceived > 0)
{
buffer[bytesReceived] = '\0';
printf(buffer);
}
So if the client disconnects, recv returns -1 but errno is still EWOULDBLOCK.
I also tried to use a short timeout in recv, but it leads to the same result as without just with errno = ETIME(TIME-Out).

C++: Recv blocking forever despite data being sent

I have two threads running in my program. One is sending data to the other. The socket is connecting fine on both threads, and the receiving thread is accepting the connection. However, once the recv is called, it just sits there blocking as if its getting nothing in, despite the fact that the other thread is sending.
Sending thread
send(orig_sock, buf, BUFSIZ,0);
printf("Client sending chunk\n");
The printf manages to get displayed.
Receiving thread
printf("START ACCEPTING\n");
if ((new_sock = accept( orig_sock, (struct sockaddr *) &clnt_adr, &clnt_len)) < 0) {
perror("accept error");
close(orig_sock);
return NULL;
}
printf("PASS ACCEPT\n");
if ( fork( ) == 0 ) { // Generate a CHILD
printf("FORK\n");
len=recv(new_sock, buf, BUFLEN,0 );
printf("message received");
receiveBuffer.push(*p);
//write(new_sock, buf, len);
//if ( buf[0] == '.' ) break;
printf("Did not receive message\n");
close(new_sock);
return NULL;
} else
close(new_sock);
All the messages up untill "FORK" are being diplayed, and the thrad hangs on the recv call. buf is defined as static char buf[BUFSIZ];
Any reason why the recv call would not be seeing any data?
send(orig_sock, buf, BUFSIZ,0);
and
if ((new_sock = accept( orig_sock, (struct sockaddr *) &clnt_adr, &clnt_len)) < 0) {
don't make sense together. Either orig_sock is connected or it is listening. Not both. Your lack of error checking on the send() is masking this error.
You can't send to a listening socket.

BSD Sockets - How to use non-blocking sockets?

I am trying to use non-blocking TCP sockets. The problem is that they are still blocking. The code is below -
server code -
struct sockaddr name;
char buf[80];
void set_nonblock(int socket) {
int flags;
flags = fcntl(socket,F_GETFL,0);
assert(flags != -1);
fcntl(socket, F_SETFL, flags | O_NONBLOCK);
}
int main(int agrc, char** argv) {
int sock, new_sd, adrlen; //sock is this socket, new_sd is connection socket
name.sa_family = AF_UNIX;
strcpy(name.sa_data, "127.0.0.1");
adrlen = strlen(name.sa_data) + sizeof(name.sa_family);
//make socket
sock = socket(AF_UNIX, SOCK_STREAM, 0);
if (sock < 0) {
printf("\nBind error %m", errno);
exit(1);
}
//unlink and bind
unlink("127.0.0.1");
if(bind (sock, &name, adrlen) < 0)
printf("\nBind error %m", errno);
//listen
if(listen(sock, 5) < 0)
printf("\nListen error %m", errno);
//accept
new_sd = accept(sock, &name, (socklen_t*)&adrlen);
if( new_sd < 0) {
cout<<"\nserver accept failure "<<errno;
exit(1);
}
//set nonblock
set_nonblock(new_sd);
char* in = new char[80];
std::string out = "Got it";
int numSent;
int numRead;
while( !(in[0] == 'q' && in[1] == 'u' && in[2] == 'i' && in[3] == 't') ) {
//clear in buffer
for(int i=0;i<80;i++)
in[i] = ' ';
cin>>out;
cin.get();
//if we typed something, send it
if(strlen(out.c_str()) > 0) {
numSent = send(new_sd, out.c_str(), strlen(out.c_str()), 0);
cout<<"\n"<<numSent<<" bytes sent";
}
numRead = recv(new_sd, in, 80, 0);
if(numRead > 0)
cout<<"\nData read from client - "<<in;
} //end while
cout<<"\nExiting normally\n";
return 0;
}
client code -
struct sockaddr name;
void set_nonblock(int socket) {
int flags;
flags = fcntl(socket,F_GETFL,0);
assert(flags != -1);
fcntl(socket, F_SETFL, flags | O_NONBLOCK);
}
int main(int agrc, char** argv) {
int sock, new_sd, adrlen;
sock = socket(AF_UNIX, SOCK_STREAM, 0);
if (sock < 0) {
printf("\nserver socket failure %m", errno);
exit(1);
}
//stuff for server socket
name.sa_family = AF_UNIX;
strcpy(name.sa_data, "127.0.0.1");
adrlen = strlen(name.sa_data) + sizeof(name.sa_family);
if(connect(sock, &name, adrlen) < 0) {
printf("\nclient connection failure %m", errno);
exit(1);
}
cout<<"\nSuccessful connection\n";
//set nonblock
set_nonblock(sock);
std::string out;
char* in = new char[80];
int numRead;
int numSent;
while(out.compare("quit")) {
//clear in
for(int i=0;i<80;i++)
in[i] = '\0';
numRead = recv(sock, in, 80, 0);
if(numRead > 0)
cout<<"\nData read from server - "<<in;
cout<<"\n";
out.clear();
cin>>out;
cin.get();
//if we typed something, send it
if(strlen(out.c_str())) {
numSent = send(sock, out.c_str(), strlen(out.c_str()), 0);
cout<<"\n"<<numSent<<" bytes sent";
}
} //end while
cout<<"\nExiting normally\n";
return 0;
}
Whenever I run it, the server still waits for me to send something before it will read and output what the client has sent. I want either the server or client to be able to send the message as soon as I type it, and have the other read and output the message at that time. I thought non-blocking sockets was the answer, but maybe I am just doing something wrong?
Also, I was using a file instead of my 127.0.0.1 address as the sockaddr's data. If that is not how it should be properly used, feel free to say so (it worked how it worked previously with a file so I just kept it like that).
Any help is appreciated.
General approach for a TCP server where you want to handle many connections at the same time:
make listening socket non-blocking
add it to select(2) or poll(2) read event set
enter select(2)/poll(2) loop
on wakeup check if it's the listening socket, then
accept(2)
check for failure (the client might've dropped the connection attempt by now)
make newly created client socket non-blocking, add it to the polling event set
else, if it's one of the client sockets
consume input, process it
watch out for EAGAIN error code - it's not really an error, but indication that there's no input now
if read zero bytes - client closed connection, close(2) client socket, remove it from event set
re-init event set (omitting this is a common error with select(2))
repeat the loop
Client side is a little simpler since you only have one socket. Advanced applications like web browsers that handle many connections often do non-blocking connect(2) though.
Whenever I run it, the server still waits for me to send something before it will read and output what the client has sent.
Well, that is how you wrote it. You block on IO from stdin, and then and only then do you send/receive.
cin>>out;
cin.get();
Also, you are using a local socket (AF_UNIX) which creates a special file in your filesystem for interprocess communication - this is a different mechanism than IP, and is definitely not TCP as you indicate in your question. I suppose you could name the file 127.0.0.1, but that really doesn't make sense and implies confusion on your part, because that is an IP loopback address. You'll want to use AF_INET for IP.
For an excellent starter guide on unix networking, I'd recommend http://beej.us/guide/bgnet/
If you want the display of messages received to be independant of your cin statements, either fork() off a seperate process to handle your network IO, or use a separate thread.
You might be interested in select(). In my opinion non-blocking sockets are usually a hack, and proper usage of select() or poll() is generally much better design and more flexible (and more portable). try
man select_tut
for more information.
I think you have to set non-block sooner (ie get the socket then set it non block)
also check that the fcntl to set it actually worked
If you want non-blocking i/o, you want to use select. You can set it with stdin as one of the sockets it is listening on, along with the client sockets (just add file descriptor 1, which is stdin, to the fd_set).
http://beej.us/guide/bgnet/output/html/multipage/advanced.html
I would recommend reading through what beej has to say about select. It looks a little intimidating but is really useful and simple to use if you take a little time to wrap your head around it.