Excessive message size with boost asio and SSL - c++

I am currently working on a server that uses SSL to secure the transmissions. After accepting a conenction, the server makes the hadshake and if it succeeds, it sends a packet to the client.
My code is working perfectly fine on Windows, but when I run it on Debian 7, I get an "Excessive message size" error when I try to write that packet.
I'm pretty sure it does not come from my certificates, since I tried running the boost asio SSL example with the same certificates and it works perfectly fine.
Here is the code:
Initialization of the server:
Server::Server(configuration::Configuration const & conf, utils::Logger & log): _cmds(conf, log), _conf(conf), _logger(log), _ctx(boost::asio::ssl::context::sslv23)
{
tcp::endpoint endpoint(tcp::v4(), conf.getPort());
configuration::SSL_conf ssl = conf.getSLLInfos();
try
{
this->_ctx.set_options(boost::asio::ssl::context::default_workarounds
| boost::asio::ssl::context::no_sslv2
| boost::asio::ssl::context::single_dh_use);
this->_ctx.set_password_callback(boost::bind(&Server::_getPass, this, _1, _2));
this->_ctx.use_certificate_file(ssl.certificate.string(), boost::asio::ssl::context_base::pem);
this->_ctx.use_private_key_file(ssl.key.string(), boost::asio::ssl::context_base::pem);
this->_ctx.use_tmp_dh_file(ssl.certificate.string());
this->_acceptor.reset(new tcp::acceptor(this->_service, endpoint));
this->_logger.logMsg(3, "Starting listen on port " + std::to_string(endpoint.port()));
for (int i = 0; i < 256; ++i)
this->_c.push_back(std::shared_ptr<Client>(new Client(this->_service, this->_cmds, conf, log, i, this->_ctx)));
this->_acceptor->async_accept(this->_c.front()->getSocket(),
boost::bind(&Server::_handleAccept, this,
this->_c.front(),
boost::asio::placeholders::error));
}
catch (boost::system::system_error & err)
{
this->_logger.logErr(err.what());
}
}
The initialization of the session:
void Client::init()
{
this->_logger.logMsg(3, "Client [" + std::to_string(this->_id) + "] initialized");
this->_isAlive = true;
this->_socket.async_handshake(boost::asio::ssl::stream_base::server,
boost::bind(&Client::handleHandshake, this,
boost::asio::placeholders::error));
}
The handhsake callback:
void Client::handleHandshake(boost::system::error_code const & err)
{
if (!err)
{
memset(&this->_header, 0, sizeof(protocol::header));
this->_keepAlive = this->_conf.getKeepAlive();
this->_header.keepAlive = 1;
this->_logger.logStruct<protocol::header>("Client [" + std::to_string(this->_id) + "] Sending handhsake: ", this->_header);
boost::asio::async_write(this->_socket, boost::asio::buffer(&this->_header, sizeof(protocol::header)),
boost::bind(&Client::handleWrite, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
else
this->kill();
}
And finally, the write callback where I receive the error:
void Client::handleWrite(boost::system::error_code const & err, std::size_t)
{
if (!err && this->_keepAlive)
{
clearBuff(this->_outBuff);
boost::asio::async_read(this->_socket, boost::asio::buffer(&this->_header, sizeof(protocol::header)),
boost::bind(&Client::handleHeaderRead, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
else
{
this->_logger.logErr("Client [" + std::to_string(this->_id) + "] write failed : " + err.message());
this->kill();
}
}
As I said the handleWrite callback receives an error with the value "Excessive message size". I really don't understand why since my use of boost asio is pretty standard and my certificates, that are self signed work with other applications.
Thank you for helping.

Try to configure a specific SSL protocol version (e.g. only SSLv3). This reduces the list of items to be sent.
Also, consider sending a minimized certificate bundle (in the certificate file on the server) because the client won't be interested in most root certificates in the chain, unless it already knows them anyways

From your description, the problem is not with the SSL handshake itself, but with the buffer in the call to async_write in Client::handleHandshake.
I'd recommend logging both sizeof(_header) and sizeof(protocol::header) and comparing them in both implementations.
BTW where do you allocate the memory for _header? Is it done in the Constructor? Because it's not done in Client::init()

Alright, so as said in a comment, the problem was that for some reason, letting the client and server decide which version to choose from seemed to screw things up. By forcing the use of SSLV3, it solve it. Thank you all for your help though.

Related

BOOST::ASIO - UDP - endpoint gets overwritten

I am trying to implement some keep-alive service in UDP using BOOST::ASIO, these are the general steps:
Sending keep-alives to 2 processes on the same machine, they are listening on the same ip with a different port.
Loop to send async_send_to to both, and the callback is a function that calls async_receive_from with a callback F().
Both refer to the same endpoint and data buffers.
while loop with io_service.run_one() inside.
The processes reply immediately.
The issue is that sporadically I either get the 2 differing ports when I check the endpoints' ports (the wanted case) F() runs, or, I get twice the same port.
It seems as the endpoint buffer (and probably the data) is getting overwritten by the later packet.
I was thinking the since I'm using run_one() the packets should be processed one by one and there will be no overwriting.
Initial send -
void GetInstancesHeartbeat(udp::endpoint &sender_endpoint)
{
int instanceIndex = 0;
for (; instanceIndex <= amountOfInstances ; instanceIndex++)
{
udp::endpoint endpoint = udp::endpoint(IP, Port+ instanceIndex);
m_instancesSocket->async_send_to(
boost::asio::buffer((char*)&(message),
sizeof(message)),endpoint,
boost::bind(&ClusterManager::handle_send_to_instance,
this, boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred,
sender_endpoint));
}
}
Then the handler -
void handle_send_to_instance(const boost::system::error_code& error, size_t
bytes_recvd, udp::endpoint &sender_endpoint)
{
m_instancesSocket->async_receive_from(
boost::asio::buffer(m_dataBuffer, m_maxLength), m_endpoint,
boost::bind(&ClusterManager::handle_receive_from_instance, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred,
sender_endpoint));
}
While loop -
while(true){
io_service.run_one();
}
And the handle receive where the port results twice the same -
void handle_receive_from_instance(const boost::system::error_code& error, size_t
bytes_recvd, udp::endpoint&sender_endpoint)
{
if (!error && bytes_recvd > 0)
{
int instancePort = m_endpoint.port();
} else {
//PRINT ERROR
}
}
The actual operations are asynchronous, so there's no telling when the endpoint reference gets written to. That's the nature of asynchronous calls.
So, what you need to have is an endpoint receiving variable per asynchronous call (you might store it per instance index).
There are a number of other really suspicious bits:
what's the type of message? For most types you'd write just boost::asio::buffer(message) (which deals with T [], std::vector<T>, array<T> etc). This works when T is char or any POD type.
If message is actually a struct of some type, consider using a single-element array to avoid having to dangerous casting:
Live On Coliru
POD message[1] = {pod};
s.async_send_to(boost::asio::buffer(message), udp::endpoint{{}, 6767}, [](boost::system::error_code ec, size_t transferred) {
std::cout << "Transferred: " << transferred << " (" << ec.message() << ")\n";
});
(Sends 12 bytes on a typical system).
Whatever you do, don't write the unsafe C-style cast (Why use static_cast<int>(x) instead of (int)x?).
You have while(true) { io.run_one(); } which is an infinite loop. A better way to write it would be: while(io.run_one()) {}
However, that would basically be the same as io.run();, but less correctly and less efficiently (see https://www.boost.org/doc/libs/1_68_0/boost/asio/detail/impl/scheduler.ipp line 138), so why not use it?

Boost sockets. Working code no longer working after format / reinstall of ubuntu. permission problems?

i am running ubuntu 16.04 and using ROS kinetic (irrelevant for this question). unrelated to this project, i broke my system and decided to reinstall ubuntu. everything went fine and i thought that i had put my system back together. so now i have this code to send a packet to a LiDAR and decode the information i get back. you can see i put a couple cout's for debugging. the program freezes after sending the packet and waiting for the data back. this code has been working correctly for several months now and has not changed after the reinstall of ubuntu.
things i have tried
sudo adduser my_username dialout
sudo adduser my_username root
sudo adduser my_username sudo
I have also edited /etc/sudoers.d to add my username in there with all permissions.
the reason i suspect that the hangup is due to permissions is because i have another piece of code that uses UART communication through /dev/ttyUSB1 and i must use sudo chmod a+rw /dev/ttyUSB1 before it will work. this was not necessary before.
The relevant code is below although i strongly suspect this is not the issue. the code compiles and runs as expected except for the hangup. i would appreciate any feedback and suggestions and i will do my best to keep on top of replies.
#include <iostream>
#include <boost/array.hpp>
#include <boost/asio.hpp>
#include <sensor_msgs/LaserScan.h>
using boost::asio::ip::udp;
class KeyenceLaser
{
public:
KeyenceLaser(boost::asio::io_service& io_service, const std::string& host, const std::string& port):io_service_(io_service), socket_(io_service, udp::endpoint(udp::v4(), 0))
{
udp::resolver resolver(io_service_);
udp::resolver::query query(udp::v4(), host, port);
udp::resolver::iterator iter = resolver.resolve(query);
endpoint_ = *iter;
if (socket_.is_open()){std::cout<<"socket open"<<std::endl;;}else{std::cout<<"socket not open"<<std::endl;}
}
~KeyenceLaser()
{
socket_.close();
}
void send(const std::string& msg)
{
socket_.send_to(boost::asio::buffer(msg, msg.size()), endpoint_);
}
void poll(sensor_msgs::LaserScan::Ptr scan)
{
char request[1920];
char srequest[]={0x90,0,1,0,0,0,0,0,0,0};// sends the request to get all beam data back
size_t request_length = strlen(srequest);
unsigned char reply[1920];
scan->ranges.resize(953);
try
{
std::cout<<"1 sending request"<<std::endl;
boost::system::error_code errcode;
boost::asio::socket_base::message_flags flags=NULL;
int bytesSent =-1;
bytesSent= socket_.send_to(boost::asio::buffer(srequest, 10), endpoint_ );
std::cout<<"sent " << bytesSent << " bytes\n2 waiting for data back"<<std::endl;
size_t reply_length = socket_.receive_from(boost::asio::buffer(reply, 1920), endpoint_);
std::cout<<"3 got data back"<<std::endl;
scan->angle_min = -.090757;
scan->angle_max = 3.23234;
scan->angle_increment = 3.323106 / 953.0;
scan->range_min = 0.00;
scan->range_max = 5.0;
scan->ranges.reserve(953);
scan->ranges.resize(953);
scan->ranges.clear();
for (int i=14;i < 1921; i++)
{
//std::cout << i/2-7<<":"<<(((reply[i+1]<<8) + reply[i])/1000.0) << std::endl;
scan->ranges.push_back(((reply[i+1]<<8) + reply[i])/1000.0);
i++;
}
}
catch (std::exception& e)
{
std::cout << "Exception: " << e.what() << "\n";
}
}
private:
boost::asio::io_service& io_service_;
udp::socket socket_;
udp::endpoint endpoint_;
};
EDIT 1:
i have modified the code to the following...
std::cout<<"1 sending request"<<std::endl;
boost::system::error_code errcode;
boost::asio::socket_base::message_flags flags;
int bytesSent =-1;
bytesSent= socket_.send_to(boost::asio::buffer(srequest, 10), endpoint_ ,flags, errcode);
std::cout<<"sent " << bytesSent << " bytes\ngot error code: " << errcode << "\n2 waiting for data back"<<std::endl;
size_t reply_length = socket_.receive_from(boost::asio::buffer(reply, 1920), endpoint_);
std::cout<<"3 got data back"<<std::endl;
and i get error 95 - operation not supported , but i think thats cause i didnt instantiate my flags correctly (and google isnt helping me do that)
EDIT 2:
i just wanted to make it clear that i think i've installed all the required software. i'm open to apt-get some more stuff if you think that i'm just missing some library.
P.S. i did modify the code again slightly and the socket is being shown as open with 10 bytes sent. the LiDAR "link" light does not blink after i send the packet though (it did before), so I don't think the packet is actually getting through the cable
So it turns out that this issue was unrelated to Boost Sockets. The error came from the connection to the LiDAR itself. It showed as connected via my network connections but it was not actually connected to the LiDAR. After I switched that conecttion IPv4 method to Link-Local, it worked immediately.thank you to everyone who read and considered my issue

boost::asio::ip::tcp::resolver::iterator Check if values are null

I've got a block of code that throws an Access Violation when executed. It is the async_connect handler for boost::asio.
//------------------------------------------------------------------------------
void MyClient::OnConnect(const boost::system::error_code & errorCode, boost::asio::ip::tcp::resolver::iterator endpoint)
{
if (errorCode || endpoint == boost::asio::ip::tcp::resolver::iterator())
{
// Error - An error occured while attempting to connect
// Most likely these error codes correspond to https://msdn.microsoft.com/en-us/library/windows/desktop/ms740668(v=vs.85).aspx
std::ostringstream msg;
msg << "An error occured while attempting to connect to " << endpoint->host_name()
<< ". Error code: " << errorCode
<< ". Error Message: " << ErrorCodeToString(errorCode);
LogError(msg.str().c_str(), __FUNCSIG__, __FILE__, __LINE__);
return;
}
// We connected to an endpoint
m_connectionState |= CONNECTED;
In the debugger it looks like the problem is inside endpoint->host_name(), because it tried to get values_[0] while values_ are null.
This is a common Connection Refused scenario. I thought the handler got the endpoint so that it knew who it was trying to connect to! Is there some manner of check I can do on the iterator before I try to call a method on it?
It seems to pass and still throw access violation on
if( endpoint != boost::asio::ip::tcp::resolver::iterator() )
{
std::string blah = endpoint->host_name();
}
Probably you've figured this out by now, but I'll post a few lines in case someone else happens upon it.
This problem looks similar to one I had.
This will not work, and will give similar results to what you're describing.
void establish_connection() {
tcp::resolver resolver(get_asio_io_service()); // this is a problem
tcp::resolver::query q{server_name, "https"};
resolver.async_resolve(q,
[&](asio::error_code ec,
tcp::resolver::iterator it) {
this->handle_resolve(ec, it);
});
}
where establish_connection() is a method in the object that deals with communication to the server.
The resolver object disappears after the establish_connection() exits. You have to find a way to make it stick around. The various demos on the web have it as an attribute of the client object.
Please share your experience with this problem.

async_receive_from stops receiving after a few packets under Linux

I have a setup with multiple peers broadcasting udp packets (containing images) every 200ms (5fps).
While receiving both the local stream as external streams works fine under Windows, the same code (except for the socket->cancel(); in Windows XP, see comment in code) produces rather strange behavior under Linux:
The first few (5~7) packets sent by another machine (when this machine starts streaming) are received as expected;
After this, the packets from the other machine are received after irregular, long intervals (12s, 5s, 17s, ...) or get a time out (defined after 20 seconds). At certain moments, there is again a burst of (3~4) packets received as expected.
The packets sent by the machine itself are still being received as expected.
Using Wireshark, I see both local as external packets arriving as they should, with correct time intervals between consecutive packages. The behavior also presents itself when the local machine is only listening to a single other stream, with the local stream disabled.
This is some code from the receiver (with some updates as suggested below, thanks!):
Receiver::Receiver(port p)
{
this->port = p;
this->stop = false;
}
int Receiver::run()
{
io_service io_service;
boost::asio::ip::udp::socket socket(
io_service,
boost::asio::ip::udp::endpoint(boost::asio::ip::udp::v4(),
this->port));
while(!stop)
{
const int bufflength = 65000;
int timeout = 20000;
char sockdata[bufflength];
boost::asio::ip::udp::endpoint remote_endpoint;
int rcvd;
bool read_success = this->receive_with_timeout(
sockdata, bufflength, &rcvd, &socket, remote_endpoint, timeout);
if(read_success)
{
std::cout << "read succes " << remote_endpoint.address().to_string() << std::endl;
}
else
{
std::cout << "read fail" << std::endl;
}
}
return 0;
}
void handle_receive_from(
bool* toset, boost::system::error_code error, size_t length, int* outsize)
{
if(!error || error == boost::asio::error::message_size)
{
*toset = length>0?true:false;
*outsize = length;
}
else
{
std::cout << error.message() << std::endl;
}
}
// Update: error check
void handle_timeout( bool* toset, boost::system::error_code error)
{
if(!error)
{
*toset = true;
}
else
{
std::cout << error.message() << std::endl;
}
}
bool Receiver::receive_with_timeout(
char* data, int buffl, int* outsize,
boost::asio::ip::udp::socket *socket,
boost::asio::ip::udp::endpoint &sender_endpoint, int msec_tout)
{
bool timer_overflow = false;
bool read_result = false;
deadline_timer timer( socket->get_io_service() );
timer.expires_from_now( boost::posix_time::milliseconds(msec_tout) );
timer.async_wait( boost::bind(&handle_timeout, &timer_overflow,
boost::asio::placeholders::error) );
socket->async_receive_from(
boost::asio::buffer(data, buffl), sender_endpoint,
boost::bind(&handle_receive_from, &read_result,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred, outsize));
socket->get_io_service().reset();
while ( socket->get_io_service().run_one())
{
if ( read_result )
{
timer.cancel();
}
else if ( timer_overflow )
{
//not to be used on Windows XP, Windows Server 2003, or earlier
socket->cancel();
// Update: added run_one()
socket->get_io_service().run_one();
}
}
// Update: added run_one()
socket->get_io_service().run_one();
return read_result;
}
When the timer exceeds the 20 seconds, the error message "Operation canceled" is returned, but it is difficult to get any other information about what is going on.
Can anyone identify a problem or give me some hints to get some more information about what is going wrong? Any help is appreciated.
Okay, what you're doing is that when you call receive_with_timeout, you're setting up the two asynchronous requests (one for the recv, one for the timeout). When the first one completes, you cancel the other.
However, you never invoke ioservice::run_one() again to allow it's callback to complete. When you cancel an operation in boost::asio, it invokes the handler, usually with an error code indicating that the operation has been aborted or canceled. In this case, I believe you have a handler dangling once you destroy the deadline service, since it has a pointer onto the stack for it to store the result.
The solution is to call run_one() again to process the canceled callback result prior to exiting the function. You should also check the error code being passed to your timeout handler, and only treat it as a timeout if there was no error.
Also, in the case where you do have a timeout, you need to execute run_one so that the async_recv_from handler can execute, and report that it was canceled.
After a clean installation with Xubuntu 12.04 instead of an old install with Ubuntu 10.04, everything now works as expected. Maybe it is because the new install runs a newer kernel, probably with improved networking? Anyway, a re-install with a newer version of the distribution solved my problem.
If anyone else gets unexpected network behavior with an older kernel, I would advice to try it on a system with a newer kernel installed.

Boost async_read doesn't give me an "end of frame" flag

I'm still working on some kind of client for communication with an IP Camera. Now I have the following issue:
I send a request to the camera ( a RTSP DESCRIBEin particular ). Now I get it's answer which looks like this:
RTSP/1.0 200 OK
CSeq: 2
Date: Thu, Jan 01 1970 00:31:41 GMT
Content-Base: rtsp://192.168.0.42/mpeg4?mode=Live&stream=-1&buffer=0&seek=0&fps=100& metainfo=/
Content-Type: application/sdp
Content-Length: 517
This is the header of the answer, followed by a so called Session Description which has the size shown in the field Content-Length. Actually I don't care much for the Session Description , I'm just interested in the Content-Base field. But still, since there is some communication following on the same socket, I need to get rid of all the data.
For receiving'm using the async_read calls from boost::asio.
My code looks ( simplified ) like this:
CommandReadBuffer::CallbackFromAsyncWrite()
{
boost::asio::async_read_until(*m_Socket, m_ReceiveBuffer,"\r\n\r\n",
boost::bind(&CommandReadBuffer::handle_rtsp_describe, this->shared_from_this(),
boost::asio::placeholders::error,boost::asio::placeholders::bytes_transferred));
}
This one reads at least the header ( shown above ) since its terminated by a blank line. As usual for async_write it just reads some more of the data, but nevermind. Now to the next callback function:
void CommandReadBuffer::handle_rtsp_describe(const boost::system::error_code& err,size_t bytesTransferred)
{
std::istream response_stream(&m_ReceiveBuffer);
std::string header;
// Just dump the data on the console
while (std::getline(response_stream, header))
{
// Normally I would search here for the desired content-base field
std::cout << header << "\n";
}
boost::asio::async_read(*m_Socket, m_ReceiveBuffer, boost::asio::transfer_at_least(1),
boost::bind(&CommandReadBuffer::handle_rtsp_setup, this->shared_from_this(),
boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));
}
Now this works fine as well, if I print out the number of received bytes it's always 215.
Now we go on to the critical callback:
void CommandReadBuffer::handle_rtsp_setup(const boost::system::error_code& err, size_t bytesTransferred)
{
std::cout << "Error: " << err.message() << "\n";
if (!err)
{
// Write all of the data that has been read so far.
std::cout << &m_ReceiveBuffer;
// Continue reading remaining data until EOF.
m_DeadlineTimer->async_wait(boost::bind(&CommandReadBuffer::handleTimeout, this->shared_from_this(),boost::asio::placeholders::error));
boost::asio::async_read(*m_Socket, m_ReceiveBuffer, boost::asio::transfer_at_least(1),
boost::bind(&CommandReadBuffer::handle_rtsp_setup, this->shared_from_this(),
boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));
}
else if (err != boost::asio::error::eof)
{
std::cout << "Error: " << err.message() << "\n";
}
else
{
std::cout << "End of Frame " << err.message() << "\n";
}
}
This part reads 220 Bytes. If I look at console Output from this call and compare it with the actualy payload of the frame ( as seen in Wireshark ) I can see that all data has been received. Now I would actually assume that async_read would set me the eof error. But instead the return code of error is success and so it calls async_read again. This time there is no data to be received and it never calls the callback function ( since there will be no more incoming data ).
Now I actually don't know how I could determine that all data has been sent. Actually I would expect the error flag to be set.
Now this is very similar to the implementation of the Boost Example for an Async HTTP client. Also it is done the same way in the Example Boost Async HTTP Client. I implemented this in another call and there it actually works.
Now in my opinion it should make no difference for the async_read call wether it is HTTP or RTSP - end of frame is end of frame, if there is no more data to read.
I'm also aware that according to the boost documentation I am using
void async_read(
AsyncReadStream & s,
basic_streambuf< Allocator > & b,
CompletionCondition completion_condition,
ReadHandler handler);
which means the function will continue until
The supplied buffer is full (that is, it has reached maximum size).
The completion_condition function object returns 0.
So if there is no more data to read, it just continues.
But I also tried the overloaded function without the CompletionCondition parameter, which should return when an error occurs ( EOF !!! ) - But this just won't callback either...
Any suggestions? I just don't get what I'm doing wrong...
I have written an RTSP client and server library using boost asio and can offer the following advice:
The RTSP message syntax is generic: there is no need for different DESCRIBE and SETUP handlers. In general
write an RTSP request
to read the response do a boost::asio::async_read_until("\r\n\r\n")
then check for the Content-Length header
if content_length > 0 do a boost::asio::transfer_at_least(content_length)
Further, why are you expecting an EOF? The connection is still open: the server is waiting for either another SETUP or a PLAY request and typically won't close the connection until the RTSP TCP connection has been timed out, which has a default value of 60 seconds according to RFC2326.
If in your application, you have completed interaction with the RTSP server, close the connection after you have read the response.