I have been working on a http server, when I am trying to read the body part of the request It just gets stuck in there, here are snippets of my code
This part reads until headers and calls the parser
void Connection::start_operation() {
parser_ = new Parser(shared_from_this());
if (sock_.is_open()) {
//! TODO limit header size
asio::async_read_until(
sock_, request_stream_, "\r\n\r\n",
[me = shared_from_this()](system::error_code ec, const size_t bytes) {
if (!ec && bytes) {
std::unique_ptr<char> req(new char[bytes]);
me->input_stream.read(req.get(), bytes);
me->parser_->parser_init(std::string(req.get()));
} else {
error::print(ec);
return;
}
});
}
}
The parser makes the call to read_body on parsing Content-Length Field
template <typename Callback>
void Connection::internal_read_body(std::string &req, size_t bytes, Callback call) {
std::unique_ptr<char> buff_space(new char[BUFFER_SIZE]);
if (sock_.is_open()) {
std::cout << "Here" << std::endl;
sock_.async_read_some(
boost::asio::buffer(buff_space.get(), BUFFER_SIZE),
[me_ = shared_from_this(), bytes, &req, buff_space = std::move(buff_space), call](
boost::system::error_code ec, const size_t &bytes_read) mutable {
std::cout << "$$$ " << buff_space.get() << std::endl;
req.append(buff_space.get(), bytes_read);
bytes -= bytes_read;
if (bytes > 0)
me_->internal_read_body(req, bytes, call);
else
call();
});
}
}
void Connection::read_left_over(std::string &req, size_t &bytes) {
std::string extra_data;
input_stream >> extra_data;
bytes -= size_t(extra_data.size());
req.append(std::move(extra_data));
}
template <typename Callback>
void read_body(std::string &req, size_t bytes, Callback call) {
// reserve so much space in the string
req.reserve(bytes);
// reading from remaining data in the streambuf
read_left_over(req, bytes);
ioc_.dispatch(
[this, &req, bytes, call] {
internal_read_body(req, bytes, call);
});
}
The code is stuck in the async_read_some after printing "Here", I'm not sure what I am doing wrong here, or how to debug through this issue. Any help or hint on the cause of error would be appreciated. Thanks!
Related
I'm working on a RS485 communication class and I'm trying to make a function that reads until a certain char is on the line, but with a time out. The problem is that my system timer immediately returns, doesn't matter which time out I enter. I tried changing the timer to be a member variable of the class, so it doesn't go out of scope, but that wasn't the problem. I tried different implementations of timers (deadline_timer mostly) but that didn't help. If I remove the timer from the code, then the read succeeds, but when I add it, even if I give it a timeout of 10 seconds (which should be waay more than enough), it will respond with an immediate timeout.
I tried making a simple version of the class here, but I guess that the options mostly depend on the type of machine you're talking to:
class RS485CommunicationLayer final {
public:
RS485CommunicationLayer(
const std::string& path,
/* options */
): io(), port(io), timer(port.get_io_service()) {
open(/* options */);
};
std::size_t write(const char* const buffer, const size_t size) {
/*impl*/
}
// THIS FUNCTION --v
void readUntil(std::vector<char>& buffer, char delim,std::chrono::microseconds timeout) {
boost::optional<boost::system::error_code> timer_result;
boost::optional<boost::system::error_code> read_result;
port.get_io_service().reset();
timer.expires_from_now(timeout);
boost::asio::async_read_until(port, asio::dynamic_buffer(buffer), delim, [&read_result] (const boost::system::error_code& error, size_t) { read_result.reset(error); });
timer.async_wait([&timer_result] (const boost::system::error_code& error) { timer_result.reset(error); });
while (port.get_io_service().run_one())
{
if (read_result)
timer.cancel();
else if (timer_result) {
port.cancel();
}
}
if (read_result)
throw boost::system::system_error(*read_result);
};
private:
asio::io_context io;
asio::serial_port port;
boost::asio::system_timer timer;
void open(/*args*/) {
port.open(path);
/*set options*/
}
};
Edit:
I also tried the following implementation after finding out that run_for() exists. But then the buffer stays empty weirdly enough.
void RS485CommunicationLayer::readUntil(std::vector<char>& buffer, char delim, std::chrono::microseconds timeout) {
boost::optional<boost::system::error_code> read_result;
boost::asio::async_read_until(port, asio::dynamic_buffer(buffer), delim, [&read_result] (const boost::system::error_code& error, size_t) { read_result.reset(error); });
port.get_io_service().run_for(timeout);
if (read_result)
throw boost::system::system_error(*read_result);
}
First off, get_io_service() indicates a Very Old(TM) boost version. Also, it just returns io.
Secondly, why so complicated? I don't even really have the energy to see whether there is a subtle problem with the run_one() loop (it looks fine at a glance).
I'd simplify:
size_t readUntil(std::vector<char>& buffer, char delim,
std::chrono::microseconds timeout) {
error_code read_result;
size_t msglen = 0;
io.reset();
asio::system_timer timer(io, timeout);
asio::async_read_until(port, asio::dynamic_buffer(buffer), delim,
[&](error_code ec, size_t n) {
timer.cancel();
read_result = ec;
msglen = n;
});
timer.async_wait([&](error_code ec) { if (!ec) port.cancel(); });
io.run();
if (read_result)
boost::throw_with_location(boost::system::system_error(read_result),
read_result.location());
return msglen;
}
You can just cancel the complementary IO object from the respective completion handlers.
The timer is per-op and local to the readUntil, so it doesn't have to be a member.
Let's also throw in the write side, which is all of:
size_t write(char const* const data, const size_t size) {
return asio::write(port, asio::buffer(data, size));
}
And I can demo it working:
Live On Coliru
#include <boost/asio.hpp>
#include <iomanip>
#include <iostream>
namespace asio = boost::asio;
using boost::system::error_code;
using namespace std::chrono_literals;
class RS485CommunicationLayer final {
public:
RS485CommunicationLayer(std::string const& path) : io(), port(io) { open(path); };
size_t write(char const* const data, const size_t size) {
return asio::write(port, asio::buffer(data, size));
}
size_t readUntil(std::vector<char>& buffer, char delim,
std::chrono::microseconds timeout) {
error_code read_result;
size_t msglen = 0;
io.reset();
asio::system_timer timer(io, timeout);
asio::async_read_until(port, asio::dynamic_buffer(buffer), delim,
[&](error_code ec, size_t n) {
timer.cancel();
read_result = ec;
msglen = n;
});
timer.async_wait([&](error_code ec) { if (!ec) port.cancel(); });
io.run();
if (read_result)
boost::throw_with_location(boost::system::system_error(read_result),
read_result.location());
return msglen;
}
private:
asio::io_context io;
asio::serial_port port;
void open(std::string path) {
port.open(path);
/*set options*/
}
void close();
};
int main(int argc, char** argv) {
RS485CommunicationLayer comm(argc > 1 ? argv[1] : "");
comm.write("Hello world\n", 12);
for (std::vector<char> response_buffer;
auto len = comm.readUntil(response_buffer, '\n', 100ms);) //
{
std::cout << "Received " << response_buffer.size() << " bytes, next "
<< quoted(std::string_view(response_buffer.data(), len - 1))
<< std::endl;
// consume
response_buffer.erase(begin(response_buffer), begin(response_buffer) + len);
}
}
Demo locally with a socat PTS tunnel:
socat -d -d pty,raw,echo=0 pty,raw,echo=0
And throwing dictionaries at the other end:
while true; do cat /etc/dictionaries-common/words ; done | pv > /dev/pts/10
C++ lambda made me confusing.
bind functions for async read.
template<typename T>
class connection
: public boost::enable_shared_from_this<connection<T>>
, boost::noncopyable
{
public:
using err = boost::system::error_code;
protected:
boost::asio::ip::tcp::socket socket_;
...
// completion function
// read until meet '\n'
size_t on_read_completion(const err& error, size_t bytes)
{
if (error) { return 0; }
bool found = std::find(read_buffer_, read_buffer_ + bytes, '\n') < read_buffer_ + bytes;
return found ? 0 : 1;
}
// read message
void on_read(const err& error, size_t bytes)
{
if (!started_) { return; }
if (error)
{
std::cout << "[ERROR] async_read " << error.message();
stop();
}
std::string msg(read_buffer_, bytes);
// handle received message
on_message(msg);
}
This is async read() function.
void read()
{
if (!socket_.is_open())
{
std::cout << "[ERROR] socket_ is not open\n";
return;
}
std::fill_n(read_buffer_, MAX_MSG, '\0');
// it works
#if 0
async_read(socket_, boost::asio::buffer(read_buffer_),
boost::bind(&connection::on_read_completion, this->shared_from_this(), _1, _2),
boost::bind(&connection::on_read, this->shared_from_this(), _1, _2));
#else // not works
async_read(socket_, boost::asio::buffer(read_buffer_),
[this](const err& error, size_t bytes)->size_t
{
if (error) { return 0; }
bool found = std::find(read_buffer_, read_buffer_ + bytes, '\n') < read_buffer_ + bytes; return found ? 0 : 1;
},
[this](const err& error, size_t bytes)->void
{
if (!started_) { return; }
if (error)
{
std::cout << "[ERROR] async_read\n" << error.message();
return;
}
std::string msg(read_buffer_, bytes);
on_message(msg);
});
#endif
}
The problem is lambda [capture] that is not containing shared_ptr itself. So when it call on_read_completion(), lambda capture this is not pointing shared_from_this() that I want.
I need a wise advice for this. Help me please.
Oh, I found the what is the problem.
...
async_read(socket_, boost::asio::buffer(read_buffer_),
[this](const err& error, size_t bytes)->size_t
{
if (error) { return 0; }
bool found = std::find(read_buffer_, read_buffer_ + bytes, '\n') < read_buffer_ + bytes;
return found ? 0 : 1;
},
[this, self = std::move(this->shared_from_this())](const err& error, size_t bytes)->void
{
if (!started_) { return; }
if (error)
{
std::cout << "[ERROR] async_read\n" << error.message() << "\n";
return;
}
std::string msg(read_buffer_, bytes);
self->on_message(msg);
});
...
it worked!
But if you have tips for this please comment! Thank you.
You already figured out that this needs to be explicitly captured in addition to the shared pointer, the latter just for the lifetime guarantee.
However your original question is slightly more interesting than you knew! There's a subtle difference between lambda and bind expressions. If you use bind to bind to a generic handler, it may preserve asio_handler_invoke (or more modern get_associated_executor) semantics under the ADL rule.
This is rarely a concern at the end-user level, but if you mean to provide a generic library of composable operations you may care about this kind of detail for failing to preserve the invocation semantics could lead to bugs (e.g. when a composed handler is bound to a strand executor).
As a sidenote, in the case you showed, consider using the overload of async_read_until that take a string(view) or regex.
I am using Asio to handle my network class.
I have been troubling for a while and I can't get the reason why.
First, the code :
// .h
#pragma once
#include <asio.hpp>
class Connection : public std::enable_shared_from_this<Connection>
{
public:
static std::shared_ptr<Connection> create(asio::io_context& IoContext);
asio::ip::tcp::socket& getSocket();
void start();
private:
Connection(asio::io_context& IoContext);
void startReading();
void sendPacket(std::string Packet);
void handle_read(const asio::error_code& error, size_t bytes);
void handle_write(const asio::error_code& error, size_t bytes);
void disconnect();
asio::ip::tcp::socket socket;
char packet[4096];
bool started;
};
class Network
{
public:
Network(asio::io_context& IoContext, unsigned short Port);
void startServer();
private:
void startAccept();
void handle_accept(std::shared_ptr<Connection> connection, const asio::error_code& error);
asio::io_context& ioContext;
asio::ip::tcp::acceptor acceptor;
bool started;
std::vector<std::shared_ptr<Connection>> connections;
};
// .cpp
Connection::Connection(asio::io_context& IoContext)
: socket(IoContext)
, started(false)
, packet("")
{
}
std::shared_ptr<Connection> Connection::create(asio::io_context& IoContext)
{
return std::shared_ptr<Connection>(new Connection(IoContext));
}
asio::ip::tcp::socket& Connection::getSocket()
{
return socket;
}
void Connection::start()
{
std::cout << "Connection::start();" << std::endl;
if (!started)
{
startReading();
started = true;
}
}
void Connection::sendPacket(std::string Packet)
{
socket.async_send(asio::buffer(Packet), [this](const asio::error_code& error, size_t bytes)
{ handle_write(error, bytes); });
}
void Connection::startReading()
{
socket.async_receive(asio::buffer(packet), [this](const asio::error_code& error, size_t bytes)
{ handle_read(error, bytes); });
}
void Connection::handle_write(const asio::error_code& error, size_t bytes)
{
std::cout << "WRITE : " << error.message() << " size : " << bytes << std::endl;
}
void Connection::handle_read(const asio::error_code& error, size_t bytes)
{
if (error)
{
std::cout << "error:" << error.message();
disconnect();
}
else
{
if (bytes > 0)
{
packet[bytes] = 0;
std::string response = ...;
sendPacket(response);
}
}
}
Network::Network(asio::io_context& IoContext, unsigned short Port)
: ioContext(IoContext)
, acceptor(ioContext, asio::ip::tcp::endpoint(asio::ip::tcp::v4(), Port))
, started(false)
{
std::cout << "Server::Server();" << std::endl;
}
void Network::startServer()
{
std::cout << "Server::startServer();" << std::endl;
if (!started)
{
startAccept();
started = true;
}
}
void Network::startAccept()
{
std::cout << "Server::startAccept();" << std::endl;
std::shared_ptr<Connection> connection = Connection::create(ioContext);
connections.push_back(connection);
asio::error_code er;
acceptor.async_accept(connection->getSocket(), std::bind(&Network::handle_accept, this, connection, er));
}
void Network::handle_accept(std::shared_ptr<Connection> connection, const asio::error_code& error)
{
std::cout << "Server::handle_accept();" << std::endl;
if (!error)
{
std::cout << "Ok" << std::endl;
connection->start();
}
startAccept();
}
(I removed unnecessary function from the cpp in order to make it more condensed, don't be surprised if some are missing)
Basically, it crashes when I sendPacket(something) from the handle_read function.
I found two fixes for it :
create and use a member std::string, then the code in handle_read(...) becomes
if (bytes > 0)
{
packet[bytes] = 0;
std::string response = ...;
sendPacket(my_member_string);
}
or I create a std::string* in sendPacket(...) :
void Connection::sendPacket(std::string Packet)
{
std::string *copy;
socket.async_send(asio::buffer(copy), [this](const asio::error_code& error, size_t bytes)
{ handle_write(error, bytes); });
}
without deleting it (if I do, it crashes the same way).
So, my questions : why is response causing a crash ? I read a lot of same issue talking about "lifetime", could you tell me more about it ? What is really happening in this code with response's life ?
Also, what is the clean way to fix this bug, if it is not one of the above ?
By the way, I am not using Boost.
Thank you by advance
You should not capture [this] pointer within your lambdas in ..._async functions. Because lambda will outlive the this pointer and this will point to invalid address within lambda call.
Use [self=shared_from_this()] and work with self within lambda, so the object will live with shared pointer through async calls.
I apologise for the length of code posted here. I am trying to create a class that uses boost::process to spawn a process, feed it some data on its stdin and to capture all its stdout & stderr.
The subprocess' stdin may be lengthy as might the stdout; The target machine does not have vast amounts of memory so each of these needs to be handled in chunks.
I have read endless examples of usage of boost::process, but have found nothing that answers all these questions together. I have tried combining these examples without success. I'm obviously missing something. I would be grateful for any help.
What happens is that the child process is successfully spawned but nothing happens. The parent process halt on the line marked THUS *.
The class defined:
class CommandProcessor {
public:
explicit CommandProcessor(const std::string &executable_path, bool slow) :
executable_path_(executable_path), slow_(slow), in_(io_service_, ::dup(STDIN_FILENO)), out_(io_service_, ::dup(STDOUT_FILENO)), err_(io_service_, ::dup(STDERR_FILENO)) {
}
private:
void begin_write_stdin();
void end_write_stdin(const boost::system::error_code &ec, std::size_t bytes_transferred);
void begin_read_stdout();
void end_read_stdout(const boost::system::error_code &ec, std::size_t bytes_transferred);
void begin_read_stderr();
void end_read_stderr(const boost::system::error_code &ec, std::size_t bytes_transferred);
public:
void execute_command(const Command& command);
private:
boost::filesystem::path executable_path_;bool slow_;
boost::asio::io_service io_service_;
boost::asio::posix::stream_descriptor in_;
boost::asio::posix::stream_descriptor out_;
boost::asio::posix::stream_descriptor err_;
std::string stdout_;
std::string stderr_;
std::string stdin_buffer_;
std::array<char, 4096> stdout_buffer_;
std::array<char, 4096> stderr_buffer_;
std::vector<std::string>::const_iterator stdin_it_;
std::vector<std::string>::const_iterator stdin_end_;
};
The code (for brevity I include only the bits giving me trouble):
void CommandProcessor::begin_write_stdin() {
if (stdin_buffer_.size() == 0) {
for (; stdin_it_ != stdin_end_; stdin_it_++) {
if (stdin_buffer_.size() + stdin_it_->size() > 4096) {
break;
}
stdin_buffer_ += *stdin_it_;
}
}
if (stdin_buffer_.size() == 0) {
return;
}
in_.async_write_some(boost::asio::buffer(stdin_buffer_),
boost::bind(&CommandProcessor::end_write_stdin, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));
}
void CommandProcessor::end_write_stdin(const boost::system::error_code &ec, std::size_t bytes_transferred __attribute__((unused))) {
if (!ec) {
stdin_it_++;
if (stdin_it_ != stdin_end_) {
begin_write_stdin();
}
}
in_.close();
}
void CommandProcessor::begin_read_stdout() {
out_.async_read_some(boost::asio::buffer(stdout_buffer_),
boost::bind(&CommandProcessor::end_read_stdout, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));
}
void CommandProcessor::end_read_stdout(const boost::system::error_code &ec, std::size_t bytes_transferred __attribute__((unused))) {
if (!ec) {
stdout_ += stdout_buffer_.data();
begin_read_stdout();
}
out_.close();
}
void CommandProcessor::begin_read_stderr() {
err_.async_read_some(boost::asio::buffer(stderr_buffer_), boost::bind(&CommandProcessor::end_read_stderr, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));
}
void CommandProcessor::end_read_stderr(const boost::system::error_code &ec, std::size_t bytes_transferred __attribute__((unused))) {
if (!ec) {
stderr_ += stderr_buffer_.data();
begin_read_stderr();
}
err_.close();
}
void CommandProcessor::execute_command(const Command& command) {
boost::process::context ctx;
ctx.stdin_behavior = boost::process::capture_stream();
ctx.stdout_behavior = boost::process::capture_stream();
ctx.stderr_behavior = boost::process::capture_stream();
stdin_it_ = command.for_stdin_.begin();
stdin_end_ = command.for_stdin_.end();
boost::process::child child(boost::process::launch((executable_path_ / command.executable_name_).string(), command.executable_name_ + command.arguments_, ctx));
boost::process::pistream &child_stdout(child.get_stdout());
**** Halts in next statement
in_.assign(child_stdout.handle().release());
boost::process::pistream &child_stderr(child.get_stderr());
err_.assign(child_stderr.handle().release());
boost::process::postream &child_stdin = child.get_stdin();
out_.assign(child_stdin.handle().release());
begin_read_stdout();
begin_read_stderr();
begin_write_stdin();
boost::process::status child_status(child.wait());
if (child_status.exited()) {
if (child_status.exit_status() == 0) {
throw ProcessorException((boost::format("Exec status %d on %s") % child_status.exit_status() % (executable_path_ / command.executable_name_).string()).str());
}
} else {
throw ProcessorException((boost::format("Exec failure on %s") % (executable_path_ / command.executable_name_).string()).str());
}
}
Is there a way to read from a streambuf without removing the bytes?
I'm reading a 'message size' field from the buffer to check if the whole message was received.
If not, I'm posting another async read to get it, but the handler then has no way to know how long the message was supposed to be - because the size field was removed.
Any help appreciated!
E.g.
boost::asio::streambuf _buffer;
void onReceive(const boost::system::error_code& e, std::size_t bytesTransferred)
{
if(e) return;
if(_buffer.size() > 0)
{
// Partial message was previously received, but I don't know how long.
}
else
{
_buffer.commit(bytesTransferred);
/* Read the size (and remove it from the stream) */
unsigned short size = 0;
std::istream in(&_buffer);
in.read((char*)&size, sizeof(unsigned short);
/* Got the whole message? */
if(_buffer.size() > size)
{
/* Yes. */
}
else
{
/* No - read the rest. */
boost::asio::async_read(/*...*/);
}
}
}
You can use read_async to initiate a read using the size of the message header and then adjust it in a 'completion condition' callback, like so:
typedef boost::system::error_code error_code;
template <typename Stream, typename Message>
void MessageReader<Stream, Message>::startRead()
{
readBuffer = allocateMsg();
async_read(stream,
boost::asio::buffer(readBuffer.get(), sizeof(*readBuffer)),
boost::bind(&MessageReader<Stream, Message>::bytesToRead, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred),
boost::bind(&MessageReader<Stream, Message>::readDone, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
template <typename Stream, typename Message>
size_t MessageReader<Stream, Message>::bytesToRead(const error_code& error,
size_t bytes_read)
{
size_t result;
if (error)
result = 0; // error - stop reading
else if (bytes_read < sizeof(CmnMessageHeader))
result = sizeof(CmnMessageHeader) - bytes_read; // read rest of header
else if (readBuffer->header.byteCount > sizeof(*readBuffer))
result = 0; // bad byte count
else
result = readBuffer->header.byteCount - bytes_read; // read message body
return result;
}
template <typename Stream, typename Message>
void MessageReader<Stream, Message>::readDone(const error_code& error,
size_t bytes_read)
{
if (error)
{
if (error.value() == boost::system::errc::no_such_file_or_directory)
{
notifyStop();
}
else if (error.value() != boost::system::errc::operation_canceled)
{
notifyStop();
}
// else the operation was cancelled, thus no stop notification is needed and
// we can merely return
}
else if (bytes_read != readBuffer->header.byteCount)
{
LOG4CXX_ERROR(logger, "Message byte count mismatch");
notifyStop();
}
else
{
handleMsg(readBuffer);
startRead();
}
}
EDIT: Added typedef for error_code.
I did this yesterday. So i thought i'd offer my solution...
#include <iostream>
#include <sstream>
#include <algorithm>
#include <iterator>
#include <boost/asio.hpp>
#include <boost/asio/streambuf.hpp>
void ReadFromStreambuf()
{
boost::asio::streambuf mybuffer;
// write some data to the buffer
std::ostream o2buffer (&mybuffer);
o2buffer << "hello stackoverflow";
// get buffer size
size_t nBufferSize = boost::asio::buffer_size(mybuffer.data());
// get const buffer
std::stringstream ssOut;
boost::asio::streambuf::const_buffers_type constBuffer = mybuffer.data();
// copy const buffer to stringstream, then output
std::copy(
boost::asio::buffers_begin(constBuffer),
boost::asio::buffers_begin(constBuffer) + nBufferSize,
std::ostream_iterator<char>(ssOut)
);
std::cout << ssOut.str() << "\n";
}
int main(int argc, char const *argv[])
{
ReadFromStreambuf();
return 0;
}
There are two approaches you can adopt:
Issue a single read to read the number of bytes for the size (say 4), the issue a read for the required size.
Use the read some call, and buffer the bytes in your code, say in a vector and analyse that way.
I would go for option 2, it does mean copying the buffer, however I would hazard that it is cheaper than multiple read some calls.