Regarding this post:
Why do I need strand per connection when using boost::asio?
I'm focusing on this statement regarding async calls:
"However, it is not safe for multiple threads to make calls concurrently"
This example:
http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/example/cpp11/chat/chat_client.cpp
If I refer to main as "thread 1" and the spawned thread t as "thread 2", then it seems like thread 1 is calling async_write (assuming no write_in_progress) while thread 2 is calling async_read. What am I missing?
In the official chat example, chat_client::write() defers work to the io_service via io_service::post(), which will:
request that the io_service execute the given handler via a thread that is currently invoking the poll(), poll_one(), run(), or run_one() function on the io_service
not allow the given handler to be invoked within the calling function (e.g. chat_client::write())
As only one thread is running the io_service, and all socket read, write, and close operations are only initiated from handlers that have been posted to the io_service, the program satisfies the thread-safety requirement for socket.
class chat_client
{
void write(const chat_message& msg)
{
// The nullary function `handler` is created, but not invoked within
// the calling function. `msg` is captured by value, allowing `handler`
// to append a valid `msg` object to `write_msgs_`.
auto handler = [this, msg]()
{
bool write_in_progress = !write_msgs_.empty();
write_msgs_.push_back(msg);
if (!write_in_progress)
{
do_write();
}
};
// Request that `handler` be invoked within the `io_service`.
io_service_.post(handler);
}
};
Related
I have a problem where two threads are called like this, one after another.
new boost::thread( &SERVER::start_receive, this);
new boost::thread( &SERVER::run_io_service, this);
Where the first thread calls this function.
void start_receive()
{
udp_socket.async_receive(....);
}
and the second thread calls,
void run_io_service()
{
io_service.run();
}
and sometimes the io_service thread ends up finishing before the start_receive() thread and then the server cannot receive packets.
I thought about putting a sleep function between the two threads to wait a while for the start_receive() to complete and that works but I wondered if there was another sure fire way to make this happen?
When you call io_service.run(), the thread will block, dispatching posted handlers until either:
There are no io_service::work objects associated with the io_service, or
io_service.stop() is called.
If either of these happens, the io_service enters the stopped state and will refuse to dispatch any more handlers in future until its reset() method is called.
Every time you initiate an asynchronous operation on an io object associated with the io_service, an io_service::work object is embedded in the asynchronous handler.
For this reason, point (1) above cannot happen until the asynchronous handler has run.
this code therefore will guarantee that the async process completes and that the asserts pass:
asio::io_service ios; // ios is not in stopped state
assert(!ios.stopped());
auto obj = some_io_object(ios);
bool completed = false;
obj.async_something(..., [&](auto const& ec) { completed = true; });
// nothing will happen yet. There is now 1 work object associated with ios
assert(!completed);
auto ran = ios.run();
assert(completed);
assert(ran == 1); // only 1 async op waiting for completion.
assert(ios.stopped()); // io_service is exhausted and no work remaining
ios.reset();
assert(!ios.stopped()); // io_service is ready to run again
If you want to keep the io_service running, create a work object:
boost::asio::io_service svc;
auto work = std::make_shared<boost::asio::io_service::work>(svc);
svc.run(); // this will block as long as the work object is valid.
The nice thing about this approach is that the work object above will keep the svc object "running", but not block any other operations on it.
I am trying to wrap my head around resource management in boost::asio. I am seeing callbacks called after the corresponding sockets are already destroyed. A good example of this is in the boost::asio official example: http://www.boost.org/doc/libs/1_60_0/doc/html/boost_asio/example/cpp11/chat/chat_client.cpp
I am particularly concerned with the close method:
void close()
{
io_service_.post([this]() { socket_.close(); });
}
If you call this function and afterwards destruct chat_client instance that holds socket_, socket_ will be destructed before the close method is called on it. Also any pending async_* callbacks can be called after the chat_client has been destroyed.
How would you correctly handle this?
You can do socket_.close(); almost any time you want, but you should keep in mind some moments:
If you have threads, this call should be wrapped with strand or you can crash. See boost strand documentation.
Whenever you do close keep in mind that
io_service can already have queued handlers. And they will be called anyway with old state/error code.
close can throw an exception.
close does NOT includes ip::tcp::socket destruction. It
just closes the system socket.
You must manage object lifetime
yourself to ensure objects will be destroyed only when there is no
more handlers. Usually this is done with enable_shared_from_this
on your Connection or socket object.
Invoking socket.close() does not destroy the socket. However, the application may need to manage the lifetime of objects for which the operation and completion handlers depend upon, but this is not necessarily the socket object itself. For instance, consider a client class that holds a buffer, a socket, and has a single outstanding read operation with a completion handler of client::handle_read(). One can close() and explicitly destroy the socket, but the buffer and client instance must remain valid until at least the handler is invoked:
class client
{
...
void read()
{
// Post handler that will start a read operation.
io_service_.post([this]() {
async_read(*socket, boost::asio::buffer(buffer_);
boost::bind(&client::handle_read, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
});
}
void handle_read(
const boost::system::error_code& error,
std::size_t bytes_transferred
)
{
// make use of data members...if socket_ is not used, then it
// is safe for socket to have already been destroyed.
}
void close()
{
io_service_.post([this]() {
socket_->close();
// As long as outstanding completion handlers do not
// invoke operations on socket_, then socket_ can be
// destroyed.
socket_.release(nullptr);
});
}
private:
boost::asio::io_service& io_service_;
// Not a typical pattern, but used to exemplify that outstanding
// operations on `socket_` are not explicitly dependent on the
// lifetime of `socket_`.
std::unique_ptr<boost::asio::socket> socket_;
std::array<char, 512> buffer_;
...
}
The application is responsible for managing the lifetime of objects upon which the operation and handlers are dependent. The chat client example accomplishes this by guaranteeing that the chat_client instance is destroyed after it is no longer in use, by waiting for the io_service.run() to return within the thread join():
int main(...)
{
try
{
...
boost::asio::io_service io_service;
chat_client c(...);
std::thread t([&io_service](){ io_service.run(); });
...
c.close();
t.join(); // Wait for `io_service.run` to return, guaranteeing
// that `chat_client` is no longer in use.
} // The `chat_client` instance is destroyed.
catch (std::exception& e)
{
...
}
}
One common idiom is to managing object lifetime is to have the I/O object be managed by a single class that inherits from enable_shared_from_this<>. When a class inherits from enable_shared_from_this, it provides a shared_from_this() member function that returns a valid shared_ptr instance managing this. A copy of the shared_ptr is passed to completion handlers, such as a capture-list in lambdas or passed as the instance handle to bind(), causing the lifetime of the I/O object to be extended to at least as long as the handler. See the Boost.Asio asynchronous TCP daytime server tutorial for an example using this approach.
Is calling asio::io_service::poll() or poll_one() in a nested or recursive fashion (ie. from within a handler) valid?
A really basic test seems to imply that this works (I've only done the test on one platform) but I want to be sure that calling poll() again from within a handler is considered valid behavior.
I couldn't find any relevant information in the asio docs, so I'm hoping that someone with a bit more experience with asio's inner workings could verify this with an explanation or references.
Basic test:
struct NestedHandler
{
NestedHandler(std::string name, asio::io_service * service) :
name(name),
service(service)
{
// empty
}
void operator()()
{
std::cout << " { ";
std::cout << name;
std::cout << " ...calling poll again... ";
service->poll();
std::cout << " } ";
}
std::string name;
asio::io_service * service;
};
struct DefaultHandler
{
DefaultHandler(std::string name) :
name(name)
{
// empty
}
void operator()()
{
std::cout << " { ";
std::cout << name;
std::cout << " } ";
}
std::string name;
};
int main()
{
asio::io_service service;
service.post(NestedHandler("N",&service));
service.post(DefaultHandler("A"));
service.post(DefaultHandler("B"));
service.post(DefaultHandler("C"));
service.post(DefaultHandler("D"));
std::cout << "asio poll" << std::endl;
service.poll();
return 0;
}
// Output:
asio poll
{ N ...calling poll again... { A } { B } { C } { D } }
It is valid.
For the family of functions that process the io_service, run() is the only one with restrictions:
The run() function must not be called from a thread that is currently calling one of run(), run_one(), poll() or poll_one() on the same io_service object.
However, I am inclined to think that the documentation should also include the same remark for run_one(), as a nested call can result in it blocking indefinitely for either of the following cases[1]:
the only work in the io_service is the handler currently being executed
for non I/O completion port implementations, the only work was posted from within the current handler and the io_service has a concurrency hint of 1
For Windows I/O completion ports, demultiplexing is performed in all threads servicing the io_service using GetQueuedCompletionStatus(). At a high-level, threads calling GetQueuedCompletionStatus() function as if they are part of a thread pool, allowing the OS to dispatch work to each thread. As no single thread is responsible for demultiplexing operations to other threads, nested calls to poll() or poll_one() do not affect operation dispatching for other threads. The documentation states:
Demultiplexing using I/O completion ports is performed in all threads that call io_service::run(), io_service::run_one(), io_service::poll() or io_service::poll_one().
For all other demultiplexing mechanisms systems, a single thread servicing io_service is used to demultiplex I/O operations. The exact demultiplexing mechanism can be found in the Platform-Specific Implementation Notes:
Demultiplexing using [/dev/poll, epoll, kqueue, select] is performed in one of the threads that calls io_service::run(), io_service::run_one(), io_service::poll() or io_service::poll_one().
The implementation for the demultiplexing mechanism differs slightly, but at a high-level:
the io_service has a main queue from which threads consume ready-to-run operations to perform
each call to process the io_service creates a private queue on the stack that is used to manage operations in a lock-free manner
synchronization with the main queue eventually occurs, where a lock is acquired and the private queue operations are copied into the main queue, informing other threads, and allowing them to consume from the main queue.
When the io_service is constructed, it may be provided a concurrency hint, suggesting how many threads the implementation should allow to run concurrently. When non-I/O completion port implementations are provided a concurrency hint of 1, they are optimized to use the private queue as much as possible and defer synchronization with the main queue. For example, when a handler is posted via post():
if invoked from outside of a handler, then the io_service guarantees thread safety so it locks the main queue before enqueueing the handler.
if invoked from within a handler, the posted handler is enqueued into the private queue, deferring deferring synchronization with the main queue until necessary.
When a nested poll() or poll_one() is invoked, it becomes necessary for the private queue to be copied into the main queue, as operations to be performed will be consumed from the main queue. This case is explicitly checked within the implementation:
// We want to support nested calls to poll() and poll_one(), so any handlers
// that are already on a thread-private queue need to be put on to the main
// queue now.
if (one_thread_)
if (thread_info* outer_thread_info = ctx.next_by_key())
op_queue_.push(outer_thread_info->private_op_queue);
When either no concurrency hint or any value other than 1 is provided, then posted handlers are synchronized into the main queue each time. As the private queue does not need to be copied, nested poll() and poll_one() calls will function as normal.
1. In the networking-ts draft, it is noted that run_one() must not be called from a thread that is currently calling run().
Is calling asio::io_service::poll() or poll_one() in a nested or
recursive fashion (ie. from within a handler) valid?
Sytaxically, this is valid. But, its not good bacause in every handler you should run poll(). Also, your stack trace will grow to very big sizes, and you can get big problems with the stack.
In the 5th tutorial, of which the code I have given at bottom of the question, asio documentation introduced the output comes as follows :
Timer 2: 0
Timer 1: 1
Timer 2: 2
Timer 1: 3
Timer 2: 4
.
.
.
After the first one it is as expectable, with the sequence.
But even though Timer1 is wrapped in the strand first, why does Timer 2 starts running first ?
#include <iostream>
#include <asio.hpp>
#include <boost/bind.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
class printer
{
public:
printer(asio::io_service& io)
: strand_(io),
timer1_(io, boost::posix_time::seconds(1)),
timer2_(io, boost::posix_time::seconds(1)),
count_(0)
{
timer1_.async_wait(strand_.wrap(boost::bind(&printer::print1, this)));
timer2_.async_wait(strand_.wrap(boost::bind(&printer::print2, this)));
}
~printer()
{
std::cout << "Final count is " << count_ << "\n";
}
void print1()
{
if (count_ < 10)
{
std::cout << "Timer 1: " << count_ << "\n";
++count_;
timer1_.expires_at(timer1_.expires_at() + boost::posix_time::seconds(1));
timer1_.async_wait(strand_.wrap(boost::bind(&printer::print1, this)));
}
}
void print2()
{
if (count_ < 10)
{
std::cout << "Timer 2: " << count_ << "\n";
++count_;
timer2_.expires_at(timer2_.expires_at() + boost::posix_time::seconds(1));
timer2_.async_wait(strand_.wrap(boost::bind(&printer::print2, this)));
}
}
private:
asio::strand strand_;
asio::deadline_timer timer1_;
asio::deadline_timer timer2_;
int count_;
};
int main()
{
asio::io_service io;
printer p(io);
asio::thread t(boost::bind(&asio::io_service::run, &io));
io.run();
t.join();
system("PAUSE");
return 0;
}
A strand is used to provide serial execution of handlers. Also, under certain conditions, it provides a guarantee on the order of invocation of handlers posted or dispatched through the strand. The example does not meet these conditions. Furthermore, there is no guarantee that one will observe the alternating pattern between the completion handlers.
IO Objects, such as timers, are not wrapped by strands, completion handlers are. A strand can be thought of as being associated with a FIFO queue of handlers. If a handler queue has no handlers currently posted into an io_service, then it will pop one handler from itself and post it into the associated io_service. This flow guarantees that handlers posted into the same strand will not be invoked concurrently.
strand.post() enqueues a handler into the strand.
strand.dispatch() will run the handler if the current caller is running within the context of the strand. Otherwise, it will enqueue the handler as if by post().
strand.wrap() return a new completion handler that, when invoked, will dispatch() the wrapped handler into the strand. Essentially, wrap() defers the dispatching of a handler into the strand.
Given completion handlers a and b, if a is enqueued before b, then a will be invoked before b. This is the fundamental guarantee to which all scenarios can be reduced. The scenarios in which a is guaranteed before b are documented as followed:
strand.post(a) happens before strand.post(b). As post() does not attempt to invoke the provided handler within post(), a is enqueued before b.
strand.post(a) happens before strand.dispatch(b), where strand.dispatch(b) is performed outside of a strand. As strand.dispatch(b) occurs outside of a strand, b is queued as if by post(). Thus, this reduces down to strand.post(a) happening before strand.post(b).
strand.dispatch(a) happens before strand.post(b), where strand.dispatch(a) occurs outside of a strand. As strand.dispatch(a) occurs outside of a strand, a is queued as if by post(). Thus, this reduces down to strand.post(a) happening before strand.post(b).
strand.dispatch(a) happens before strand.dispatch(b), where both are performed outside of the strand. As neither occur within a strand, both handlers are enqueued as if by post(). Thus, this reduces down to strand.post(a) happening before strand.post(b).
The io_service makes no guarantees about the invocation order of handlers. Additionally, the handler returned from strand.wrap() does not run within the context of a strand. The example code simplifies to:
auto wrapped_print1 = strand.wrap(&print1);
auto wrapped_print2 = strand.wrap(&print2);
timer1_.async_wait(wrapped_print1);
timer2_.async_wait(wrapped_print2);
If the async_wait operations complete at the same time, the wrapped_print1 and wrapped_print2 completion handlers will be posted into the io_service for deferred invocation. As the io_service makes no guarantees on the invocation order, it may choose to invoke wrapped_print1 first, or it may choose to invoke wrapped_print2 first. Both wrapped_print handlers are being invoked outside of the context of the strand in an unspecified order, resulting in print1() and print2() being enqueued into the strand in an unspecified order.
The unspecified order in which wrapped_print are invoked is why one is not guaranteed to observe an alternating pattern between the print1 and print2 handlers in the original example. However, given the current implementation of the io_service's internal scheduler, one will observe such a pattern.
Can anyone tell me under what conditions boost::asio's io_service::run() method will return? The documentation documentation for io_service::run() seems to suggest that as long as there is work to be done or handlers to be dispatched, run() won't return.
The reason I'm asking this is that we have a legacy https client that contacts a server and executes http POST's. The separation of concerns in the client is a bit different than what we'd like so we're changing a few things about it, but we're running into problems.
Right now, the client basically has a mis-named connect() call that effectively drives the entire protocol conversation with the server. The connect() call starts off by creating a boost::asio::ip::tcp::resolver object and calling ::async_resolve() on it. This starts a chain where new asio calls are made from within asio callbacks.
void connect()
{
m_resolver.async_resolve( query, bind( &clientclass::resolve_callback, this ) );
thread = new boost::thread( bind( &boost::asio::io_service::run, m_io_service ) );
}
void resolve_callback( error_code & e, resolver::iterator i )
{
if (!e)
{
tcp::endpoint = *i;
m_socket.lowest_layer().async_connect(endpoint, bind(&clientclass::connect_callback,this,_1,++i));
}
}
void connect_callback( error_code & e, resolve::iterator i )
{
if (!e)
{
m_socket.lowest_layer().async_handshake(boost::asio::ssl::stream_base::client,
bind(&clientclass::handshake_callback,this,_1,++i));
}
}
void handshake_callback( error_code &e )
{
if (!e)
{
mesg = format_hello_message();
http_send( mesg, bind(&clientlass::hello_resp_handler,this,_1,_2) );
}
}
void http_send( stringstream & mesg, reply_handler handler )
{
async_write(m_socket, m_request_buffer, bind(&clientclass::write_complete_callback,this,_1,handler));
}
void write_comlete_callback( error_code &e, reply_handler handler )
{
if (!e)
{
async_read_until(m_socket,m_reply_buffer,"\r\n\r\n", bind(&clientclass::handle_reply,this,handler));
}
}
...
Anyways, this continues through the protocol until the protocol conversation is done. From the code here you can see that while connect() is running on the main thread, all of the subsequent callbacks and requests are coming back on the worker thread that is created in connect(). This is 'working' code.
When I try to break this chain up and expose it via an external interface, it stops working. In particular, I'm having the call handle_handshake() call outside of the clientclass object. Then http_send() is part of the interface (or is called by the external interface) and it creates a new worker thread to call io_service::run(). What happens is even though async_write() has been called and even though write_complete_callback() hasn't returned, io_service::run() exits. It exits without error and claims that no handlers were dispatched, but there's still 'work' to be done?
So what I'm wondering is what is io_service::run()'s definition of 'work'? Is it a pending request? Why is it that io_service::run() never returns during this chain of requests and responses in the existing code, but when I try to start the thread up again and start a new chain, it returns almost immediately before it's finished its work?
The definition of work in the context of the run() call is any pending asynchronous operations on that io_service object. This includes the invocations of the handlers in response to an operation. So, if a handler for one operation starts another operation, there is always work available.
In addition, there is an io_service::work class that can be used to create work on an io_service that never completes until the object is destroyed.
When a single chain completes, the io_service has completed all asynchronous operations, and all of the handler's have been invoked without starting a new operation, so it returns. Until you call io_service::reset(), further calls to run() will return without executing any operations.