I am having a hard time figuring out a bug in my TCP client-server app. The problem I am facing: in my recv function do-while loop, if the condition is bytes > 0, the function hangs forever. Replacing that with bytes == NMAX, everything works fine, UNLESS NMAX is equal to 1. A few side notes: doing a single send-recv works fine, but trying to do a send-recv and then recv-send hangs forever. NMAX is a constant set to 4096 by default. Server is ran first, then the client.
This is my send function:
ssize_t sendData(const std::string data, int fd)
{
ssize_t total = data.length(), bytes, sent = 0;
do
{
ssize_t chunk = total > NMAX ? NMAX : total;
bytes = send(fd, data.c_str() + sent, chunk, 0);
if (bytes == -1)
{
throw std::system_error(errno, std::generic_category(), "Error sending data");
}
total -= bytes;
sent += bytes;
} while (total > 0);
return sent;
}
This is my recv function:
std::string recvData(int fd)
{
ssize_t bytes;
std::string buffer;
do
{
std::vector<char> data(NMAX, 0);
bytes = recv(fd, &data[0], NMAX, 0);
if (bytes == -1)
{
throw std::system_error(errno, std::generic_category(), "Error receiving data");
}
buffer.append(data.cbegin(), data.cend());
} while (bytes > 0); // Replacing with bytes == NMAX partially fixes the issue, why?
return buffer;
}
This is the client's main function:
std::cout << "Sent " << sendData(data) << " bytes\n";
std::cout << "Received: " << recvData() << "\n";
And this is the server's main function:
std::cout << "Received: " << recvData(client) << "\n";
std::cout << "Sent " << sendData("Hello from the server side!", client) << " bytes\n";
The problem with your program is that the receiving side does not know how many bytes to receive in total. Therefore it will just endlessly try to read more bytes.
The reason why it "hangs" is that you perform a blocking system call (recv) which will only unblock if at least 1 more byte had been received. However since the peer does not send more data this will never happen.
To fix the issue you need to have a proper wire-format for your data which indicates how big the transmitted data is, or where it starts and ends. A common way to do this is to prefix data with it's length in binary form (e.g. a 32bit unsigned int in big endian format). Another way is to have indicators inside the data that indicate it's end (e.g. the \r\n\r\n line breaks in HTTP).
Btw: Your send function is not ideal for cases where data.length() == 0. In this case you perform a send system call with 0 bytes - which is rather unnecessary.
Related
I am currently writing a socket wrapper in C++ for Linux. It is basically a collection of classes that handle the creation, connection, sending, reading, and closing of a TCP socket.
In my socket class, all functions work well except for the send and receive function. They do not return an error; instead, it only sends the first four bytes of data.
My send function:
int Socket::sends(char* buffer){
int bytes; // for number of bytes sent
/* First, send the size of buffer */
int datalen = strlen(buffer); // get sizeof buffer
int len = htonl(datalen); // reformat
// send the size of the buffer
bytes = send(socketfd, (char*)&len, sizeof(len), 0); // send the size
if (bytes < 0){
cerr << "Error sending size of buffer to socket" << endl;
return 1;
}
/* Now acutally send the data */
bytes = send(socketfd, buffer, datalen, 0);
if (bytes < 0){
cerr << "Error writing buffer to socket" << endl;
return 1;
}
cout << bytes << " written" << endl;
return 0;
}
The ideas behind it is that It sends the buffer (char* buffer) by first sending the size of the buffer, and then sending the actual buffer. If an error is encountered (returning -1) the function terminates by returning 1.
Now, here is the read method:
int Socket::reads(char* buffer){
int bytes, buflen; // for bytes written and size of buffer
/* Read the incoming size */
bytes = recv(socketfd, (char*)&buflen, sizeof(buflen), 0);
if (bytes < 0){
cerr << "Error reading size of data" << endl;
return 1;
}
buflen = ntohl(buflen);
/* Read the data */
bytes = recv(socketfd, buffer, buflen, 0);
if (bytes < 0){
cerr << "Error reading data" << endl;
return 1;
}
return 0;
}
Here, the idea is to read the size of the data first, and then set the buffer to that size and read into it. The function returns 1 on error (recv returns -1).
Using the methods would look something like this:
socket.sends("Hello World"); // socket object sends the message
char* buffer;
socket.reads(buffer); // reads into the buffer
However, when ever I use these functions, I only receive the first 4 bytes of data, followed by strange, non-ASCII characters. I have no idea why this happens. No error is not encountered in the send and recv functions, and the functions say that only 4 bytes were written. Is there a better way that I should send or receive data? I am overlooking a very simple error?
Thanks for your help!
you're passing an uninitialized pointer (buffer) to your reads method, which probably explains that it works partially (undefined behaviour).
And you shouldn't pass buffer as a parameter since it won't be modified (and you don't know the size yet anyway)
Plus, you have to null-terminate your message when you recieve it.
I would do like this:
char *Socket::reads(){
char* buffer;
int bytes, buflen; // for bytes written and size of buffer
/* Read the incoming size */
bytes = recv(socketfd, (char*)&buflen, sizeof(buflen), 0);
if (bytes < 0){
cerr << "Error reading size of data" << endl;
return 1;
}
buflen = ntohl(buflen);
buffer = new char[buflen+1]; // +1 for the NUL-terminator
/* Read the data */
bytes = recv(socketfd, buffer, buflen, 0);
if (bytes < 0){
cerr << "Error reading data" << endl;
return 1;
}
buffer[buflen] = '\0'; // NUL-terminate the string
return buffer;
}
the main:
socket.sends("Hello World"); // socket object sends the message
char* buffer = socket.reads(); // reads into the buffer
don't forget to delete [] the buffer in the end.
Could also be done with std::string or std::vector<char> to avoid new and delete
Here are the simple echo server I'm working on, the server will accept the request from client and return what client sends to it. The program works fine with socat, but will freeze when using my own client.
The problem that my old code has is that I use read instead of read_some. read will block the pipe until it reads certain number of bytes or get a broken pipe exception, whereas read_some will read a chunk at a time. The updated version uses read_some to read input stream and check if the last character the program read is \0, if it is \0, that means it reaches the end of command, so it will echo back. This works because I only pass string literals and there is no binary data in the pipe.
The code of the server is
using namespace std;
const char* epStr = "/tmp/socketDemo";
int main() {
namespace local = boost::asio::local;
boost::asio::io_service io_service;
::unlink(epStr);
local::stream_protocol::endpoint ep(epStr);
local::stream_protocol::acceptor acceptor(io_service, ep);
while(1) {
local::stream_protocol::socket *socket = new local::stream_protocol::socket(io_service);
acceptor.accept(*socket);
char buf[2048] = {0};
boost::system::error_code error;
size_t len = 0;
while(1) {
len += socket->read_some(boost::asio::buffer(buf + len, 2048 - len));
cout << "read " << len << endl;
if (buf[len] == '\0') {
break;
}
}
cout << "read " << len << " bytes" << endl;
cout << buf << endl;
boost::asio::write(*socket, boost::asio::buffer(buf, len), boost::asio::transfer_all());
}
}
When testing the server with socat command, for example
echo "12345" | socat - UNIX-CONNECT:/tmp/socketDemo
it will return the desired result.
My client code is
const char* epStr = "/tmp/socketDemo";
int main(int argc, const char* argv[]) {
boost::asio::io_service io_service;
boost::asio::local::stream_protocol::endpoint ep(epStr);
boost::asio::local::stream_protocol::socket socket(io_service);
socket.connect(ep);
boost::asio::write(socket, boost::asio::buffer(argv[1], strlen(argv[1])), boost::asio::transfer_all());
char buf[1024] = {0};
size_t len = 0;
while(1) {
len += socket.read_some(boost::asio::buffer(buf + len, 2048 - len));
std::cout << "read " << len << std::endl;
if (buf[len] == '\0') {
break;
}
}
std::cout << "read " << len << " bytes\n";
std::cout << buf << std::endl;
socket.close();
When execute the client, at first both have no output, after I killed the client, the server will output that it reads n bytes and get a broken pipe exception.
Can this be caused by the read function in the server? If so is there a way to let it know how much data it should read without sending the size of data chunk at the beginning of each message? I am also wondering why socat can work with this server without any problem? Thanks!
I am also wondering why socat can work with this server without any
problem?
Probably because socat closes the socket and your client doesn't.
If so is there a way to let it know how much data it should read
without sending the size of data chunk at the beginning of each
message?
For instance, reading one byte at a time until you read an end-of-message character, assuming that you're defining / using a protocol that includes EOM.
I am trying to accomplish, that my ssl server does not break down, when a client does not collect all data. (fixed with one minor bug)
when the data is too long.
Basically what I'm trying to do is write in a non-blocking way. For that I found two different approaches:
First approach
using this code
int flags = fcntl(ret.fdsock, F_GETFL, 0);
fcntl(ret.fdsock, F_SETFL, flags | O_NONBLOCK);
and creating the ssl connection with it
Second approach:
Doing this directly after creating the SSL Object using SSL_new(ctx)
BIO *sock = BIO_new_socket(ret.fdsock, BIO_NOCLOSE);
BIO_set_nbio(sock, 1);
SSL_set_bio(client, sock, sock);
Both of which have their downsides, but neither of which helps solving the problem.
The first approach seems to read in a unblocking way just fine, but when I write more data, than the client reads, my server crashes.
The second approach does not seem to do anything, so my guess is, that I did something wrong or did not understand what a BIO actually does.
For more Information here is how the server writes to the client:
int SSLConnection::send(char* msg, const int size){
int rest_size = size;
int bytes_sent = 0;
char* begin = msg;
std::cout << "expected bytes to send: " << size << std::endl;
while(rest_size > 0) {
int tmp_bytes_sent = SSL_write(connection, begin, rest_size);
std::cout << "any error : " << ERR_get_error()<< std::endl;
std::cout << "tmp_bytes_sent: " << tmp_bytes_sent << std::endl;
if (tmp_bytes_sent < 0){
std::cout << tmp_bytes_sent << std::endl;
std::cout << "ssl error : " << SSL_get_error(this->connection, tmp_bytes_sent)<< std::endl;
} else {
bytes_sent += tmp_bytes_sent;
rest_size -= tmp_bytes_sent;
begin = msg+bytes_sent;
}
}
return bytes_sent;
}
Output:
expected bytes to send: 78888890
Betätigen Sie die <RETURN> Taste, um das Fenster zu schließen...
(means: hit <return> to close window)
EDIT: After people said, that I need to cache errors appropriate, here is my new code:
Setup:
connection = SSL_new(ctx);
if (connection){
BIO * sbio = BIO_new_socket(ret.fdsock, BIO_NOCLOSE);
if (sbio) {
BIO_set_nbio(sbio, false);
SSL_set_bio(connection, sbio, sbio);
SSL_set_accept_state(connection);
} else {
std::cout << "Bio is null" << std::endl;
}
} else {
std::cout << "client is null" << std::endl;
}
Sending:
int SSLConnection::send(char* msg, const int size){
if(connection == NULL) {
std::cout << "ERR: Connection is NULL" << std::endl;
return -1;
}
int rest_size = size;
int bytes_sent = 0;
char* begin = msg;
std::cout << "expected bytes to send: " << size << std::endl;
while(rest_size > 0) {
int tmp_bytes_sent = SSL_write(connection, begin, rest_size);
std::cout << "any error : " << ERR_get_error()<< std::endl;
std::cout << "tmp_bytes_sent: " << tmp_bytes_sent << std::endl;
if (tmp_bytes_sent < 0){
std::cout << tmp_bytes_sent << std::endl;
std::cout << "ssl error : " << SSL_get_error(this->connection, tmp_bytes_sent)<< std::endl;
break;
} else if (tmp_bytes_sent == 0){
std::cout << "tmp_bytes are 0" << std::endl;
break;
} else {
bytes_sent += tmp_bytes_sent;
rest_size -= tmp_bytes_sent;
begin = msg+bytes_sent;
}
}
return bytes_sent;
}
Using a client, that fetches 60 bytes, here is the output:
Output writing 1,000,000 Bytes:
expected bytes to send: 1000000
any error : 0
tmp_bytes_sent: 16384
any error : 0
tmp_bytes_sent: 16384
Betätigen Sie die <RETURN> Taste, um das Fenster zu schließen...
(translates to: hit <RETURN> to close window)
Output writing 1,000 bytes:
expected bytes to send: 1000
any error : 0
tmp_bytes_sent: 1000
connection closed <- expected output
First, a warning: non-blocking I/O over SSL is a rather baroque API, and it's difficult to use correctly. In particular, the SSL layer sometimes needs to read internal data before it can write user data (or vice versa), and the caller's code is expected to be able to handle that based on the error-codes feedback it gets from the SSL calls it makes. It can be made to work correctly, but it's not easy or obvious -- you are de facto required to implement a state machine in your code that echoes the state machine inside the SSL library.
Below is a simplified version of the logic that is required (it's extracted from the Write() method in this file which is part of this library, in case you want to see a complete, working implementation)
enum {
SSL_STATE_READ_WANTS_READABLE_SOCKET = 0x01,
SSL_STATE_READ_WANTS_WRITEABLE_SOCKET = 0x02,
SSL_STATE_WRITE_WANTS_READABLE_SOCKET = 0x04,
SSL_STATE_WRITE_WANTS_WRITEABLE_SOCKET = 0x08
};
// a bit-chord of SSL_STATE_* bits to keep track of what
// the SSL layer needs us to do next before it can make more progress
uint32_t _sslState = 0;
// Note that this method returns the number of bytes sent, or -1
// if there was a fatal error. So if this method returns 0 that just
// means that this function was not able to send any bytes at this time.
int32_t SSLSocketDataIO :: Write(const void *buffer, uint32 size)
{
int32_t bytes = SSL_write(_ssl, buffer, size);
if (bytes > 0)
{
// SSL was able to send some bytes, so clear the relevant SSL-state-flags
_sslState &= ~(SSL_STATE_WRITE_WANTS_READABLE_SOCKET | SSL_STATE_WRITE_WANTS_WRITEABLE_SOCKET);
}
else if (bytes == 0)
{
return -1; // the SSL connection was closed, so return failure
}
else
{
// The SSL layer's internal needs aren't being met, so we now have to
// ask it what its problem is, then give it what it wants. :P
int err = SSL_get_error(_ssl, bytes);
if (err == SSL_ERROR_WANT_READ)
{
// SSL can't write anything more until the socket becomes readable,
// so we need to go back to our event loop, wait until the
// socket select()'s as readable, and then call SSL_Write() again.
_sslState |= SSL_STATE_WRITE_WANTS_READABLE_SOCKET;
_sslState &= ~SSL_STATE_WRITE_WANTS_WRITEABLE_SOCKET;
bytes = 0; // Tell the caller we weren't able to send anything yet
}
else if (err == SSL_ERROR_WANT_WRITE)
{
// SSL can't write anything more until the socket becomes writable,
// so we need to go back to our event loop, wait until the
// socket select()'s as writeable, and then call SSL_Write() again.
_sslState &= ~SSL_STATE_WRITE_WANTS_READABLE_SOCKET;
_sslState |= SSL_STATE_WRITE_WANTS_WRITEABLE_SOCKET;
bytes = 0; // Tell the caller we weren't able to send anything yet
}
else
{
// SSL had some other problem I don't know how to deal with,
// so just print some debug output and then return failure.
fprintf(stderr,"SSL_write() ERROR!");
ERR_print_errors_fp(stderr);
}
}
return bytes; // Returns the number of bytes we actually sent
}
I think your problem is
rest_size -= bytes_sent;
You should do rest_size -= tmp_bytes_sent;
Also
if (tmp_bytes_sent < 0){
std::cout << tmp_bytes_sent << std::endl;
//its an error condition
return bytes_sent;
}
I dont know whether this will fix the issue, but the code you pasted has the above mentioned issues
When I write more data, than the client reads, my server crashes.
No it doesn't, unless you've violently miscoded something else that you haven't posted here. It either loops forever or it gets an error: probably ECONNRESET, which means the client has behaved as you described, and you've detected it, so you should close the connection and forget about him. Instead of which, you are just looping forever, trying to send the data to a broken connection, which can never happen.
And when you get an error, there's not much use in just printing a -1. You should print the error, with perror() or errno or strerror().
Speaking of looping forever, don't loop like this. SSL_write() can return 0, which you aren't handling at all: this will cause an infinite loop. See also David Schwartz's comments below.
NB you should definitely use the second approach. OpenSSL needs to know that the socket is in non-blocking mode.
Both of which have their downsides
Such as?
And as noted in the other answer,
rest_size -= bytes_sent;
should be
rest_size -= tmp_bytes_sent;
After opening a connection between client and server, I need to handle any write command sent to the server using the command read(); (i.e. when the client write(); the server should read(); right away).
It sounds to be a trivial problem. Firstly, I sent 58 bytes from the client. But, I am always receiving huge amount of data on the server side. Here you could find just the relevant part of code:
int sockfd, newsockfd;//, n0,n1,n2;
socklen_t clilen;
struct sockaddr_in serv_addr, cli_addr;
int reuse=1;
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd < 0)
cerr << "ERROR opening socket"<< endl;
if (setsockopt(sockfd, SOL_SOCKET,SO_REUSEADDR,&reuse, sizeof(int)) == -1)
cerr << "ERROR on reusing port"<< endl;
bzero((char *) &serv_addr, sizeof(serv_addr));
serv_addr.sin_family = AF_INET;
serv_addr.sin_port = htons(iport);
serv_addr.sin_addr.s_addr = INADDR_ANY;
if (bind(sockfd, (struct sockaddr *) &serv_addr,sizeof(serv_addr)) < 0)
cerr << "ERROR on binding"<< endl;
cout << "Listening on port: "<< iport<< endl;
listen(sockfd,1);
clilen = sizeof(cli_addr);
newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen);
if (newsockfd < 0)
cerr << "ERROR on accept" << endl;
while (1) {
size_t msgSize=0;
int n = read(newsockfd,&msgSize,sizeof(size_t));
cout << "Breakpoint " << msgSize<< endl;
// Reading bytes size from socket until 10MB
if ( n> 0 && msgSize< 10485760) {
byte bytes [msgSize];
if (read(newsockfd, bytes, msgSize) > 0) {
char ip [16];
memset (bytes + msgSize, '\0', MSGMAXSIZE - msgSize - 1);
if (read(newsockfd,ip,15) > 0) {
string cIP = (string)ip;
//cout << "Sender Ip: " << cIP << endl;
process p = currentView.getProcess(cIP);
message m(bytes,p);
cout << "*************************" << endl
<< "Message received:" << endl
<< "*****************" << endl;
m.print();
}
}
}
}
This is the result i got:
Listening on port: 4444
Connected to: 127.0.0.1:6666
Breakpoint 58
*************************
Message received:
*****************
Message text: I am trying to send a message
Message size: 58
Message sender: 127.0.0.1
Message stability: 0
**************************************************
Breakpoint 825634866
Breakpoint 808600630
Breakpoint 842478647
Breakpoint 959854903
Breakpoint 926303542
Breakpoint 876032050
Breakpoint 808601142
Breakpoint 892744503
Breakpoint 875971894
Breakpoint 825634866
Breakpoint 1144401970
Breakpoint 859256118
Breakpoint 825635639
Breakpoint 892745526
Breakpoint 775369265
Breakpoint 774909488
Breakpoint 14897
Segmentation fault
And here you could find the relevant part of code from the client side:
while (1)
{
if (!bufferMsg(m)) break;
}
bool bufferMsg(message m) // Sends a message (m) to a process (p)
{
mtx.lock();
if(fifoBuffer.size() < 5)
{
fifoBuffer.push_back(m);
size_t sizeMsg = m.getHeader().sizeMsg;
byte * bytes = m.getBytes();
if (!write(sendsockfd,&sizeMsg,sizeof(size_t)) || !write(sendsockfd,bytes,sizeMsg) || !write(sendsockfd,(char*)m.getHeader().sender.getIp().c_str(),strlen(m.getHeader().sender.getIp().c_str())))
cerr << "ERROR writing to socket"<< endl;
mtx.unlock();
return true;
}
else{
mtx.unlock();
return false;
}
}
Here you could find the header of the message:
typedef struct HeaderType {
size_t sizeMsg;
process sender; // The header.sender process
//view currentView; // the Current view
//iClock C; // reserved for later use
bool stability; // reserved for later use
}HeaderT;
PS: The terms message and process are some classes which I already created but are out of our concern.
Please feel free should you need more clarification or information.
I have the impression you think that client side write should be blocking and waits until the data is eaten up by the server. The OS is free to deliver as many bytes as it likes on a TCP stream.
You have a lot of if if(read(newsockfd, bytes, msgSize) > 0) in your code where you seem to silently assume that the read either fails completely or delivers exactly the amount of data you're waiting for. That doesn't need to be the case.
This:
if ( n> 0 && msgSize< 10485760) {
byte bytes [msgSize];
is dangerous since the byte array (which I assume is a typedef) gets allocated on the stack and I assume no OS on the planet accepts a 10MB local variable. But I might be wrong or even modern compilers start to silently allocate it on the heap. It's the top candidate for your segfault the first time msgSize <10MB. Better do something like:
std::auto_ptr<byte> bytes(new byte[msgSize]);
For your read in of msgSize better do something like:
int n = 0;
int nn = 0;
while((nn=read(newsockfd,((char *)&msgSize)+n,sizeof(size_t)-n)>0
&& n<sizeof(size_t)) {
n+=nn;
}
On the client site you do something like:
write(sendsockfd,(char*)m.getHeader().sender.getIp().c_str(),strlen(m.getHeader().sender.getIp().c_str())
To transfer something like an IP (I assume a string like 88.1.2.250) But on the server side you read it like:
read(newsockfd,ip,15)
which doesn't need to fit each other. That would lead to a frame shift in your read and the next msgSize is bogus. May I assume the the first msgSize you ever read is correct ? Under the assumption that the first read actually delivers sizeof(size-t).
size_t msgSize=0;
int n = 0;
do{
int t=read(newsockfd,((char*)&msgSize) + n, sizeof(size_t) - n);
if(t<0)
continue; //if no data is available (in nonblocking mode, or on timeout)
if(t==0)
break; //connection closed
n+=t; //increase counter n by the amount actually read
} while(n<sizeof(size_t));
cout << "Breakpoint " << msgSize<< endl;
// Reading msgSize bytes from socket until 10MB
if ( n> 0 && msgSize< 10485760) {
byte bytes [msgSize];
n=0;
int t;
while((t=read(newsockfd, bytes + n, msgSize - n)) > 0 //if something was read
&& (n+=t)<msgSize //and the total is below msgSize, we continue reading
|| t<0) //or when there is no data available, we will give it another attempt
{
}
if(t>0){
cout << "successful: " << n << endl;
} else {
cout << "only " << n << " of " << msgSize << "read" << endl;
}
}
Tricky parts explained:
((char*)&msgSize) + n
This casts the pointer to size_t to a pointer to char and + n increments the pointer by n-times the size of the type it points to.
(t=read(newsockfd, bytes + n, msgSize - n)) > 0
An assignment returns the assigned value. It has to be inside brackets, as without brackets the boolean result of the > comparison would be assigned to t.
Sidenote:
You should not send the raw binary representation of an integer value to another computer. The sender might uses a MSB byte order while the recipient could be using LSB. You should use the methods provided to convert from host byte order to network byte order. They are called htonl and ntohl (h:host, to:to, n:network l:long [4 bytes]).
I am developing an application with C++ and having some difficulty with boost sockets. The server sends an image but not all the bytes are received by the client; the client always receives about 500 bytes less than the server sent. Provided below is the pertinent code and screenshots of the program running.
Server code:
int sent = boost::asio::write(*socket, response, boost::asio::transfer_all(), error);
std::cout << "Sent: " << sent << std ::endl;
Client code (I know that read_some will block if the total bytes sent by the server is divisible by 10000; this code is just for testing):
int len = 0;
int count = 0;
do {
len = socket->read_some( boost::asio::buffer( imageData, 10000 ) );
count += len;
std::cout << "len: " << len << std::endl;
std::cout << "count: " << count << std::endl;
} while(len == 10000);
std::cout << "Image Received of size: " << count << std::endl;
Screenshot of server:
Screenshot of client:
Thanks for your time; any help or suggestions would be greatly appreciated.
There're no guarantee you'll receive complete buffers of 10000 bytes.
I'd recommend following approach:
To send some binary data w/o any explicit terminator, first send its size and only then data itself. In this case client will know how many data in this chunk it should receive.