C++ Design: Multiple TCP clients, boost asio and observers - c++

In my system, I have a juggle a bunch of TCP clients and I am bit confused on how to design it [most of my experience is in C, hence the insecurity]. I am using boost ASIO for managing connection. These are the components I have
A TCPStream class : thin wrapper over boost asio
an IPC protocol, which implement a protocol over TCP:
basically Each message starts with a type and length field
so we can read the individual messages out of the stream.
Connection classes which handle the messages
Observer class which monitors connections
I am writing pseudo C++ code to be concise. I think you will get the idea
class TCPStream {
boost::asio::socket socket_;
public:
template <typename F>
void connect (F f)
{
socket_.connect(f);
}
template <typename F>
void read (F f)
{
socket_.read(f);
}
};
class IpcProtocol : public TCPStream {
public:
template <typename F
void read (F f)
{
TCPStream::read(
[f] (buffer, err) {
while (msg = read_indvidual_message(buffer)) {
// **** this is a violation of how this pattern is
// supposed to work. Ideally there should a callback
// for individual message. Here the same callback
// is called for N no. of messages. But in our case
// its the same callback everytime so this should be
// fine - just avoids some function calls.
f(msg);
};
};
)
}
};
Lets say I have a bunch of TCP connections and there are a handler class
for each of the connection. Lets name it Connection1, Connection2 ...
class Connection {
virtual int type() = 0;
};
class Connection1 : public Connection {
shared_ptr<IpcProtocol> ipc_;
int type ()
{
return 1;
}
void start ()
{
ipc_.connect([self = shared_from_this()](){ self->connected(); });
ipc_.read(
[self = shared_from_this()](msg, err) {
if (!err)
self->process(msg);
} else {
self->error();
}
});
}
void connected ()
{
observer.notify_connected(shared_from_this());
}
void error ()
{
observer.notify_error(shared_from_this());
}
};
This pattern repeats for all connections one way or other.
messages are processed by the connection class itself. But it will let know of
other events [connect, error] to an observer. The reason -
Restart the connection, everytime it disconnect
Bunch of guys needs to know if the connection is established so that they can
send initial request/confguration to server.
There are things that needs be done based on connection status of muliple connections
Eg: if connection1 and connection2 are established, then start connection3 etc.
I added a middle Observer class is there so that the observers do have to directly connect to the connection everytime it is restarted. Each time connection breaks, the connection class is deleted and new one is created.
class Listeners {
public:
virtual void notify_error(shared_ptr<Connection>) = 0;
virtual void notify_connect(shared_ptr<Connection>) = 0;
virtual void interested(int type) = 0;
};
class Observer {
std::vector<Listeners *> listeners_;
public:
void notify_connect(shared_ptr<Connection> connection)
{
for (listener : listeners_) {
if (listener->interested(connection->type())) {
listener->notify_error(connection);
}
}
}
};
Now a rough prototype of this works. But I was wondering if this class design
any good. There are multiple streaming servers which will continuously produce states and send it to my module to program the state in h/w. This needs to be extensible as more clients will be added in future.
Threading
The legacy code had one thread per TCP connection and this worked fine. Here I am trying to handle multiple connections on same thread. Still there will be multiple threads calling ioservice. So the observer will run on multiple threads. I am planning to have a mutex per Listener, so that listeners wont get multiple events concurrently.

HTTP Implements a protocol over TCP so the HTTP Server asio examples are a good starting point for your design, especially: HTTP Server 2, HTTP Server 3 and HTTP Server 4.
Note: that connection lifetime is likely to be an issue, especially since you intend to use class member functions as handlers, see the question and answers here: How to design proper release of a boost::asio socket or wrapper thereof.

Related

C++ GRPC ClientAsyncReaderWriter: how to check if data is available for read?

I have bidirectional streaming async grpc client that use ClientAsyncReaderWriter for communication with server. RPC code looks like:
rpc Process (stream Request) returns (stream Response)
For simplicity Request and Response are bytes arrays (byte[]). I send several chunks of data to server, and when server accumulate enough data, server process this data and send back the response and continue accumulating data for next responses. After several responses, the server send final response and close connection.
For async client I using CompletionQueue. Code looks like:
...
CompletionQueue cq;
std::unique_ptr<Stub> stub;
grpc::ClientContext context;
std::unique_ptr<grpc::ClientAsyncReaderWriter<Request,Response>> responder = stub->AsyncProcess(&context, &cq, handler);
// thread for completition queue
std::thread t(
[]{
void *handler = nullptr;
bool ok = false;
while (cq_.Next(&handler, &ok)) {
if (can_read) {
// how do you know that it is read data available
// Do read
} else {
// do write
...
Request request = prepare_request();
responder_->Write(request, handler);
}
}
}
);
...
// wait
What is the proper way to async reading? Can I try to read if it no data available? Is it blocking call?
Sequencing Read() calls
Can I try to read if it no data available?
Yep, and it's going to be case more often than not. Read() will do nothing until data is available, and only then put its passed tag into the completion queue. (see below for details)
Is it blocking call?
Nope. Read() and Write() return immediately. However, you can only have one of each in flight at any given moment. If you try to send a second one before the previous has completed, it (the second one) will fail.
What is the proper way to async reading?
Each time a Read() is done, start a new one. For that, you need to be able to tell when a Read() is done. This is where tags come in!
When you call Read(&msg, tag), or Write(request, tag),you are telling grpc to put tag in the completion queue associated with that responder once that operation has completed. grpc doesn't care what the tag is, it just hands it off.
So the general strategy you will want to go for is:
As soon as you are ready to start receiving messages:
call responder->Read() once with some tag that you will recognize as a "read done".
Whenever cq_.Next() gives you back that tag, and ok == true:
consume the message
Queue up a new responder->Read() with that same tag.
Obviously, you'll also want to do something similar for your calls to Write().
But since you still want to be able to lookup the handler instance from a given tag, you'll need a way to pack a reference to the handler as well as information about which operation is being finished in a single tag.
Completion queues
Lookup the handler instance from a given tag? Why?
The true raison d'être of completion queues is unfortunately not evident from the examples. They allow multiple asynchronous rpcs to share the same thread. Unless your application only ever makes a single rpc call, the handling thread should not be associated with a specific responder. Instead, that thread should be a general-purpose worker that dispatches events to the correct handler based on the content of the tag.
The official examples tend to do that by using pointer to the handler object as the tag. That works when there's a specific sequence of events to expect since you can easily predict what a handler is reacting to. You often can't do that with async bidirectional streams, since any given completion event could be a Read() or a Write() finishing.
Example
Here's a general outline of what I personally consider to be a clean way to go about all that:
// Base class for async bidir RPCs handlers.
// This is so that the handling thread is not associated with a specific rpc method.
class RpcHandler {
// This will be used as the "tag" argument to the various grpc calls.
struct TagData {
enum class Type {
start_done,
read_done,
write_done,
// add more as needed...
};
RpcHandler* handler;
Type evt;
};
struct TagSet {
TagSet(RpcHandler* self)
: start_done{self, TagData::Type::start_done},
read_done{self, TagData::Type::read_done},
write_done{self, TagData::Type::write_done} {}
TagData start_done;
TagData read_done;
TagData write_done;
};
public:
RpcHandler() : tags(this) {}
virtual ~RpcHandler() = default;
// The actual tag objects we'll be passing
TagSet tags;
virtual void on_ready() = 0;
virtual void on_recv() = 0;
virtual void on_write_done() = 0;
static void handling_thread_main(grpc::CompletionQueue* cq) {
void* raw_tag = nullptr;
bool ok = false;
while (cq->Next(&raw_tag, &ok)) {
TagData* tag = reinterpret_cast<TagData*>(raw_tag);
if(!ok) {
// Handle error
}
else {
switch (tag->evt) {
case TagData::Type::start_done:
tag->handler->on_ready();
break;
case TagData::Type::read_done:
tag->handler->on_recv();
break;
case TagData::Type::write_done:
tag->handler->on_write_done();
break;
}
}
}
}
};
void do_something_with_response(Response const&);
class MyHandler final : public RpcHandler {
public:
using responder_ptr =
std::unique_ptr<grpc::ClientAsyncReaderWriter<Request, Response>>;
MyHandler(responder_ptr responder) : responder_(std::move(responder)) {
// This lock is needed because StartCall() can
// cause the handler thread to access the object.
std::lock_guard lock(mutex_);
responder_->StartCall(&tags.start_done);
}
~MyHandler() {
// TODO: finish/abort the streaming rpc as appropriate.
}
void send(const Request& msg) {
std::lock_guard lock(mutex_);
if (!sending_) {
sending_ = true;
responder_->Write(msg, &tags.write_done);
} else {
// TODO: add some form of synchronous wait, or outright failure
// if the queue starts to get too big.
queued_msgs_.push(msg);
}
}
private:
// When the rpc is ready, queue the first read
void on_ready() override {
std::lock_guard l(mutex_); // To synchronize with the constructor
responder_->Read(&incoming_, &tags.read_done);
};
// When a message arrives, use it, and start reading the next one
void on_recv() override {
// incoming_ never leaves the handling thread, so no need to lock
// ------ If handling is cheap and stays in the handling thread.
do_something_with_response(incoming_);
responder_->Read(&incoming_, &tags.read_done);
// ------ If responses is expensive or involves another thread.
// Response msg = std::move(incoming_);
// responder_->Read(&incoming_, &tags.read_done);
// do_something_with_response(msg);
};
// When has been sent, send the next one is there is any
void on_write_done() override {
std::lock_guard lock(mutex_);
if (!queued_msgs_.empty()) {
responder_->Write(queued_msgs_.front(), &tags.write_done);
queued_msgs_.pop();
} else {
sending_ = false;
}
};
responder_ptr responder_;
// Only ever touched by the handler thread post-construction.
Response incoming_;
bool sending_ = false;
std::queue<Request> queued_msgs_;
std::mutex mutex_; // grpc might be thread-safe, MyHandler isn't...
};
int main() {
// Start the thread as soon as you have a completion queue.
auto cq = std::make_unique<grpc::CompletionQueue>();
std::thread t(RpcHandler::handling_thread_main, cq.get());
// Multiple concurent RPCs sharing the same handling thread:
MyHandler handler1(serviceA->MethodA(&context, cq.get()));
MyHandler handler2(serviceA->MethodA(&context, cq.get()));
MyHandlerB handler3(serviceA->MethodB(&context, cq.get()));
MyHandlerC handler4(serviceB->MethodC(&context, cq.get()));
}
If you have a keen eye, you will notice that the code above stores a bunch (1 per event type) of redundant this pointers in the handler. It's generally not a big deal, but it is possible to do without them via multiple inheritance and downcasting, but that's starting to be somewhat beyond the scope of this question.

Pass data between threads using boost.signals2

I apologize for the ambiguous title, but I'll try to elaborate further here:
I have an application which includes (among others) a control, and TCP server classes.
Communication between the TCP and control class is done via this implementation:
#include <boost/signals2.hpp>
// T - Observable object type
// S - Function signature
template <class T, typename S> class observer {
using F = std::function<S>;
public:
void register_notifier(T &obj, F f)
{
connection_ = obj.connect_notifier(std::forward<F>(f));
}
protected:
boost::signals2::scoped_connection connection_;
};
// S - Function signature
template <typename S> class observable {
public:
boost::signals2::scoped_connection connect_notifier(std::function<S> f)
{
return notify.connect(std::move(f));
}
protected:
boost::signals2::signal<S> notify;
};
Where the TCP server class is the observable, and the control class is the observer.
The TCP server is running on a separate thread from the control class, and uses boost::asio::async_read. Whenever a message is received, the server object sends a notification via the 'notify' member, thus triggering the callback registered in the control class, and then waits to read the next message.
The problem is that I need to somehow safely and efficiently store the data currently stored in the TCP server buffer and pass it to the control class before it's overridden by the next message.
i.e. :
inline void ctl::tcp::server::handle_data_read(/* ... */)
{
if (!error) {
/* .... */
notify(/* ? */); // Using a pointer to the buffer
// would obviously fail as it
// is overridden in the next read
}
/* .... */
}
Those were my ideas for a solution so far:
Allocating heap memory and passing a pointer to it using
unique_ptr, but I'm not sure if boost.signals2 is move-aware.
Use an
unordered map (shared between the objects) that maps an integer index to a unique_ptr of the
data type (std::unordered_map<int, std::unique_ptr<data_type>>),
then only pass the index of the element, and 'pop' it in the control
class callback, but it feels like an overkill.
What I'm really looking for is an idea for a simple and efficient solution to pass the TCP buffer contents for each message between the threads.
Note I'm also open for suggestions to redesign my communication method between the objects if it's completely wrong.

Asynchronous model in grpc c++

My team is designing a scalable solution with micro-services architecture and planning to use gRPC as the transport communication between layers. And we've decided to use async grpc model. The design that example(greeter_async_server.cc) provides doesn't seem viable if I scale the number of RPC methods, because then I'll have to create a new class for every RPC method, and create their objects in HandleRpcs() like this.
Pastebin (Short example code).
void HandleRpcs() {
new CallDataForRPC1(&service_, cq_.get());
new CallDataForRPC2(&service_, cq_.get());
new CallDataForRPC3(&service, cq_.get());
// so on...
}
It'll be hard-coded, all the flexibility will be lost.
I've around 300-400RPC methods to implement and having 300-400 classes will be cumbersome and inefficient when I'll have to handle more than 100K RPC requests/sec and this solution is a very bad design. I can't bear the overhead of creation of objects this way on every single request. Can somebody kindly provide me a workaround for this. Can async grpc c++ not be simple like its sync companion?
Edit: In favour of making the situation more clear, and for those who might be struggling to grasp the flow of this async example, I'm writing what I've understood so far, please make me correct if wrong somewhere.
In async grpc, every time we have to bind a unique-tag with the completion-queue so that when we poll, the server can give it back to us when the particular RPC will be hit by the client, and we infer from the returned unique-tag about the type of the call.
service_->RequestRPC2(&ctx_, &request_, &responder_, cq_, cq_,this); Here we're using the address of the current object as the unique-tag. This is like registering for our RPC call on the completion queue. Then we poll down in HandleRPCs() to see if the client hits the RPC, if so then cq_->Next(&tag, &OK) will fill the tag. The polling code snippet:
while (true) {
GPR_ASSERT(cq_->Next(&tag, &ok));
GPR_ASSERT(ok);
static_cast<CallData*>(tag)->Proceed();
}
Since, the unique-tag that we registered into the queue was the address of the CallData object so we're able to call Proceed(). This was fine for one RPC with its logic inside Proceed(). But with more RPCs each time we'll have all of them inside the CallData, then on polling, we'll be calling the only one Proceed() which will contain logic to (say) RPC1(postgres calls), RPC2(mongodb calls), .. so on. This is like writing all my program inside one function. So, to avoid this, I used a GenericCallData class with the virtual void Proceed() and made derived classes out of it, one class per RPC with their own logic inside their own Proceed(). This is a working solution but I want to avoid writing many classes.
Another solution I tried was keeping all RPC-function-logics out of the proceed() and into their own functions and maintaining a global std::map<long, std::function</*some params*/>> . So whenever I register an RPC with unique-tag onto the queue, I store its corresponding logic function (which I'll surely hard code into the statement and bind all the parameters required), then the unique-tag as key. On polling, when I get the &tag I do a lookup in the map for this key and call the corresponding saved function. Now, there's one more hurdle, I'll have to do this inside the function logic:
// pseudo code
void function(reply, responder, context, service)
{
// register this RPC with another unique tag so to serve new incoming request of the same type on the completion queue
service_->RequestRPC1(/*params*/, new_unique_id);
// now again save this new_unique_id and current function into the map, so when tag will be returned we can do lookup
map.emplace(new_unique_id, function);
// now you're free to do your logic
// do your logic
}
You see this, code has spread into another module now, and it's per RPC based.
Hope it clears the situation.
I thought if somebody could have implemented this type of server in a more easy way.
This post is pretty old by now but I have not seen any answer or example regarding this so I will show how I solved it to any other readers. I have around 30 RPC calls and was looking for a way of reducing the footprint when adding and removing RPC calls. It took me some iterations to figure out a good way to solve it.
So my interface for getting RPC requests from my (g)RPC library is a callback interface that the recepiant need to implement. The interface looks like this:
class IRpcRequestHandler
{
public:
virtual ~IRpcRequestHandler() = default;
virtual void onZigbeeOpenNetworkRequest(const smarthome::ZigbeeOpenNetworkRequest& req,
smarthome::Response& res) = 0;
virtual void onZigbeeTouchlinkDeviceRequest(const smarthome::ZigbeeTouchlinkDeviceRequest& req,
smarthome::Response& res) = 0;
...
};
And some code for setting up/register each RPC method after the gRPC server is started:
void ready()
{
SETUP_SMARTHOME_CALL("ZigbeeOpenNetwork", // Alias that is used for debug messages
smarthome::Command::AsyncService::RequestZigbeeOpenNetwork, // Generated gRPC service method for async.
smarthome::ZigbeeOpenNetworkRequest, // Generated gRPC service request message
smarthome::Response, // Generated gRPC service response message
IRpcRequestHandler::onZigbeeOpenNetworkRequest); // The callback method to call when request has arrived.
SETUP_SMARTHOME_CALL("ZigbeeTouchlinkDevice",
smarthome::Command::AsyncService::RequestZigbeeTouchlinkDevice,
smarthome::ZigbeeTouchlinkDeviceRequest,
smarthome::Response,
IRpcRequestHandler::onZigbeeTouchlinkDeviceRequest);
...
}
This is all that you need to care about when adding and removing RPC methods.
The SETUP_SMARTHOME_CALL is a home-cooked macro which looks like this:
#define SETUP_SMARTHOME_CALL(ALIAS, SERVICE, REQ, RES, CALLBACK_FUNC) \
new ServerCallData<REQ, RES>( \
ALIAS, \
std::bind(&SERVICE, \
&mCommandService, \
std::placeholders::_1, \
std::placeholders::_2, \
std::placeholders::_3, \
std::placeholders::_4, \
std::placeholders::_5, \
std::placeholders::_6), \
mCompletionQueue.get(), \
std::bind(&CALLBACK_FUNC, requestHandler, std::placeholders::_1, std::placeholders::_2))
I think the ServerCallData class looks like the one from gRPCs examples with a few modifications. ServerCallData is derived from a non-templete class with an abstract function void proceed(bool ok) for the CompletionQueue::Next() handling. When ServerCallData is created, it will call the SERVICE method to register itself on the CompletionQueue and on every first proceed(ok) call, it will clone itself which will register another instance. I can post some sample code for that as well if someone is interested.
EDIT: Added some more sample code below.
GrpcServer
class GrpcServer
{
public:
explicit GrpcServer(std::vector<grpc::Service*> services);
virtual ~GrpcServer();
void run(const std::string& sslKey,
const std::string& sslCert,
const std::string& password,
const std::string& listenAddr,
uint32_t port,
uint32_t threads = 1);
private:
virtual void ready(); // Called after gRPC server is created and before polling CQ.
void handleRpcs(); // Function that polls from CQ, can be run by multiple threads. Casts object to CallData and calls CallData::proceed().
std::unique_ptr<ServerCompletionQueue> mCompletionQueue;
std::unique_ptr<Server> mServer;
std::vector<grpc::Service*> mServices;
std::list<std::shared_ptr<std::thread>> mThreads;
...
}
And the main part of the CallData object:
template <typename TREQUEST, typename TREPLY>
class ServerCallData : public ServerCallMethod
{
public:
explicit ServerCallData(const std::string& methodName,
std::function<void(ServerContext*,
TREQUEST*,
::grpc::ServerAsyncResponseWriter<TREPLY>*,
::grpc::CompletionQueue*,
::grpc::ServerCompletionQueue*,
void*)> serviceFunc,
grpc::ServerCompletionQueue* completionQueue,
std::function<void(const TREQUEST&, TREPLY&)> callback,
bool first = false)
: ServerCallMethod(methodName),
mResponder(&mContext),
serviceFunc(serviceFunc),
completionQueue(completionQueue),
callback(callback)
{
requestNewCall();
}
void proceed(bool ok) override
{
if (!ok)
{
delete this;
return;
}
if (callStatus() == ServerCallMethod::PROCESS)
{
callStatus() = ServerCallMethod::FINISH;
new ServerCallData<TREQUEST, TREPLY>(callMethodName(), serviceFunc, completionQueue, callback);
try
{
callback(mRequest, mReply);
}
catch (const std::exception& e)
{
mResponder.Finish(mReply, Status::CANCELLED, this);
return;
}
mResponder.Finish(mReply, Status::OK, this);
}
else
{
delete this;
}
}
private:
void requestNewCall()
{
serviceFunc(
&mContext, &mRequest, &mResponder, completionQueue, completionQueue, this);
}
ServerContext mContext;
TREQUEST mRequest;
TREPLY mReply;
ServerAsyncResponseWriter<TREPLY> mResponder;
std::function<void(ServerContext*,
TREQUEST*,
::grpc::ServerAsyncResponseWriter<TREPLY>*,
::grpc::CompletionQueue*,
::grpc::ServerCompletionQueue*,
void*)>
serviceFunc;
std::function<void(const TREQUEST&, TREPLY&)> callback;
grpc::ServerCompletionQueue* completionQueue;
};
Although the thread is old I wanted to share a solution I am currently implementing. It mainly consists templated classes inheriting CallData to be scalable. This way, each new rpc will only require specializing the templates of the required CallData methods.
Calldata header:
class CallData {
protected:
enum Status { CREATE, PROCESS, FINISH };
Status status;
virtual void treat_create() = 0;
virtual void treat_process() = 0;
public:
void Proceed();
};
CallData Proceed implementation:
void CallData::Proceed() {
switch (status) {
case CREATE:
status = PROCESS;
treat_create();
break;
case PROCESS:
status = FINISH;
treat_process();
break;
case FINISH:
delete this;
}
}
Inheriting from CallData header (simplified):
template <typename Request, typename Reply>
class CallDataTemplated : CallData {
static_assert(std::is_base_of<google::protobuf::Message, Request>::value,
"Request and reply must be protobuf messages");
static_assert(std::is_base_of<google::protobuf::Message, Reply>::value,
"Request and reply must be protobuf messages");
private:
Service,Cq,Context,ResponseWriter,...
Request request;
Reply reply;
protected:
void treat_create() override;
void treat_process() override;
public:
...
};
Then, for specific rpc's in theory you should be able to do things like:
template<>
void CallDataTemplated<HelloRequest, HelloReply>::treat_process() {
...
}
It's a lot of templated methods but preferable to creating a class per rpc from my point of view.

How to design proper release of a boost::asio socket or wrapper thereof

I am making a few attempts at making my own simple asynch TCP server using boost::asio after not having touched it for several years.
The latest example listing I can find is:
http://www.boost.org/doc/libs/1_54_0/doc/html/boost_asio/tutorial/tutdaytime3/src.html
The problem I have with this example listing is that (I feel) it cheats and it cheats big, by making the tcp_connection a shared_ptr, such that it doesn't worry about the lifetime management of each connection. (I think) They do this for brevity, since it is a small tutorial, but that solution is not real world.
What if you wanted to send a message to each client on a timer, or something similar? A collection of client connections is going to be necessary in any real world non-trivial server.
I am worried about the lifetime management of each connection. I figure the natural thing to do would be to keep some collection of tcp_connection objects or pointers to them inside tcp_server. Adding to that collection from the OnConnect callback and removing from that collection OnDisconnect.
Note that OnDisconnect would most likely be called from an actual Disconnect method, which in turn would be called from OnReceive callback or OnSend callback, in the case of an error.
Well, therein lies the problem.
Consider we'd have a callstack that looked something like this:
tcp_connection::~tcp_connection
tcp_server::OnDisconnect
tcp_connection::OnDisconnect
tcp_connection::Disconnect
tcp_connection::OnReceive
This would cause errors as the call stack unwinds and we are executing code in a object that has had its destructor called...I think, right?
I imagine everyone doing server programming comes across this scenario in some fashion. What is a strategy for handling it?
I hope the explanation is good enough to follow. If not let me know and I will create my own source listing, but it will be very large.
Edit:
Related
) Memory management in asynchronous C++ code
IMO not an acceptable answer, relies on cheating with shared_ptr outstanding on receive calls and nothing more, and is not real world. what if the server wanted to say "Hi" to all clients every 5 minutes. A collection of some kind is necessary. What if you are calling io_service.run on multiple threads?
I am also asking on the boost mailing list:
http://boost.2283326.n4.nabble.com/How-to-design-proper-release-of-a-boost-asio-socket-or-wrapper-thereof-td4693442.html
Like I said, I fail to see how using smart pointers is "cheating, and cheating big". I also do not think your assessment that "they do this for brevity" holds water.
Here's a slightly redacted excerpt¹ from our code base that exemplifies how using shared_ptrs doesn't preclude tracking connections.
It shows just the server side of things, with
a very simple connection object in connection.hpp; this uses the enable_shared_from_this
just the fixed size connection_pool (we have dynamically resizing pools too, hence the locking primitives). Note how we can do actions on all active connections.
So you'd trivially write something like this to write to all clients, like on a timer:
_pool.for_each_active([] (auto const& conn) {
send_message(conn, hello_world_packet);
});
a sample listener that shows how it ties in with the connection_pool (which has a sample method to close all connections)
Code Listings
connection.hpp
#pragma once
#include "xxx/net/rpc/protocol.hpp"
#include "log.hpp"
#include "stats_filer.hpp"
#include <memory>
namespace xxx { namespace net { namespace rpc {
struct connection : std::enable_shared_from_this<connection>, protected LogSource {
typedef std::shared_ptr<connection> ptr;
private:
friend struct io;
friend struct listener;
boost::asio::io_service& _svc;
protocol::socket _socket;
protocol::endpoint _ep;
protocol::endpoint _peer;
public:
connection(boost::asio::io_service& svc, protocol::endpoint ep)
: LogSource("rpc::connection"),
_svc(svc),
_socket(svc),
_ep(ep)
{}
void init() {
_socket.set_option(protocol::no_delay(true));
_peer = _socket.remote_endpoint();
g_stats_filer_p->inc_value("asio." + _ep.address().to_string() + ".sockets_accepted");
debug() << "New connection from " << _peer;
}
protocol::endpoint endpoint() const { return _ep; }
protocol::endpoint peer() const { return _peer; }
protocol::socket& socket() { return _socket; }
// TODO encapsulation
int handle() {
return _socket.native_handle();
}
bool valid() const { return _socket.is_open(); }
void cancel() {
_svc.post([this] { _socket.cancel(); });
}
using shutdown_type = boost::asio::ip::tcp::socket::shutdown_type;
void shutdown(shutdown_type what = shutdown_type::shutdown_both) {
_svc.post([=] { _socket.shutdown(what); });
}
~connection() {
g_stats_filer_p->inc_value("asio." + _ep.address().to_string() + ".sockets_disconnected");
}
};
} } }
connection_pool.hpp
#pragma once
#include <mutex>
#include "xxx/threads/null_mutex.hpp"
#include "xxx/net/rpc/connection.hpp"
#include "stats_filer.hpp"
#include "log.hpp"
namespace xxx { namespace net { namespace rpc {
// not thread-safe by default, but pass e.g. std::mutex for `Mutex` if you need it
template <typename Ptr = xxx::net::rpc::connection::ptr, typename Mutex = xxx::threads::null_mutex>
struct basic_connection_pool : LogSource {
using WeakPtr = std::weak_ptr<typename Ptr::element_type>;
basic_connection_pool(std::string name = "connection_pool", size_t size)
: LogSource(std::move(name)), _pool(size)
{ }
bool try_insert(Ptr const& conn) {
std::lock_guard<Mutex> lk(_mx);
auto slot = std::find_if(_pool.begin(), _pool.end(), std::mem_fn(&WeakPtr::expired));
if (slot == _pool.end()) {
g_stats_filer_p->inc_value("asio." + conn->endpoint().address().to_string() + ".connections_dropped");
error() << "dropping connection from " << conn->peer() << ": connection pool (" << _pool.size() << ") saturated";
return false;
}
*slot = conn;
return true;
}
template <typename F>
void for_each_active(F action) {
auto locked = [=] {
using namespace std;
lock_guard<Mutex> lk(_mx);
vector<Ptr> locked(_pool.size());
transform(_pool.begin(), _pool.end(), locked.begin(), mem_fn(&WeakPtr::lock));
return locked;
}();
for (auto const& p : locked)
if (p) action(p);
}
constexpr static bool synchronizing() {
return not std::is_same<xxx::threads::null_mutex, Mutex>();
}
private:
void dump_stats(LogSource::LogTx tx) const {
// lock is assumed!
size_t empty = 0, busy = 0, idle = 0;
for (auto& p : _pool) {
switch (p.use_count()) {
case 0: empty++; break;
case 1: idle++; break;
default: busy++; break;
}
}
tx << "usage empty:" << empty << " busy:" << busy << " idle:" << idle;
}
Mutex _mx;
std::vector<WeakPtr> _pool;
};
// TODO FIXME use null_mutex once growing is no longer required AND if
// en-pooling still only happens from the single IO thread (XXX-2535)
using server_connection_pool = basic_connection_pool<xxx::net::rpc::connection::ptr, std::mutex>;
} } }
listener.hpp
#pragma once
#include "xxx/threads/null_mutex.hpp"
#include <mutex>
#include "xxx/net/rpc/connection_pool.hpp"
#include "xxx/net/rpc/io_operations.hpp"
namespace xxx { namespace net { namespace rpc {
struct listener : std::enable_shared_from_this<listener>, LogSource {
typedef std::shared_ptr<listener> ptr;
protocol::acceptor _acceptor;
protocol::endpoint _ep;
listener(boost::asio::io_service& svc, protocol::endpoint ep, server_connection_pool& pool)
: LogSource("rpc::listener"), _acceptor(svc), _ep(ep), _pool(pool)
{
_acceptor.open(ep.protocol());
_acceptor.set_option(protocol::acceptor::reuse_address(true));
_acceptor.set_option(protocol::no_delay(true));
::fcntl(_acceptor.native(), F_SETFD, FD_CLOEXEC); // FIXME use non-racy socket factory?
_acceptor.bind(ep);
_acceptor.listen(32);
}
void accept_loop(std::function<void(connection::ptr conn)> on_accept) {
auto self = shared_from_this();
auto conn = std::make_shared<xxx::net::rpc::connection>(_acceptor.get_io_service(), _ep);
_acceptor.async_accept(conn->_socket, [this,self,conn,on_accept](boost::system::error_code ec) {
if (ec) {
auto tx = ec == boost::asio::error::operation_aborted? debug() : warn();
tx << "failed accept " << ec.message();
} else {
::fcntl(conn->_socket.native(), F_SETFD, FD_CLOEXEC); // FIXME use non-racy socket factory?
if (_pool.try_insert(conn)) {
on_accept(conn);
}
self->accept_loop(on_accept);
}
});
}
void close() {
_acceptor.cancel();
_acceptor.close();
_acceptor.get_io_service().post([=] {
_pool.for_each_active([] (auto const& sp) {
sp->shutdown(connection::shutdown_type::shutdown_both);
sp->cancel();
});
});
debug() << "shutdown";
}
~listener() {
}
private:
server_connection_pool& _pool;
};
} } }
¹ download as gist https://gist.github.com/sehe/979af25b8ac4fd77e73cdf1da37ab4c2
While others have answered similarly to the second half of this answer, it seems the most complete answer I can find, came from asking the same question on the Boost Mailing list.
http://boost.2283326.n4.nabble.com/How-to-design-proper-release-of-a-boost-asio-socket-or-wrapper-thereof-td4693442.html
I will summarize here in order to assist those that arrive here from a search in the future.
There are 2 options
1) Close the socket in order to cancel any outstanding io and then post a callback for the post-disconnection logic on the io_service and let the server class be called back when the socket has been disconnected. It can then safely release the connection. As long as there was only one thread that had called io_service::run, then other asynchronous operations will have been already been resolved when the callback is made. However, if there are multiple threads that had called io_service::run, then this is not safe.
2) As others have been pointing out in their answers, using the shared_ptr to manage to connections lifetime, using outstanding io operations to keep them alive, is viable. We can use a collection weak_ptr to the connections in order to access them if we need to. The latter is the tidbit that had been omitted from other posts on the topic which confused me.
The way that asio solves the "deletion problem" where there are outstanding async methods is that is splits each async-enabled object into 3 classes, eg:
server
server_service
server_impl
there is one service per io_loop (see use_service<>). The service creates an impl for the server, which is now a handle class.
This has separated the lifetime of the handle and the lifetime of the implementation.
Now, in the handle's destructor, a message can be sent (via the service) to the impl to cancel all outstanding IO.
The handle's destructor is free to wait for those io calls to be queued if necessary (for example if the server's work is being delegated to a background io loop or thread pool).
It has become a habit with me to implement all io_service-enabled objects this way as it makes coding with aiso very much simpler.
Connection lifetime is a fundamental issue with boost::asio. Speaking from experience, I can assure you that getting it wrong causes "undefined behaviour"...
The asio examples use shared_ptr to ensure that a connection is kept alive whilst it may have outstanding handlers in an asio::io_service. Note that even in a single thread, an asio::io_service runs asynchronously to the application code, see CppCon 2016: Michael Caisse "Asynchronous IO with Boost.Asio" for an excellent description of the precise mechanism.
A shared_ptr enables the lifetime of a connection to be controlled by the shared_ptr instance count. IMHO it's not "cheating and cheating big"; but an elegant solution to complicated problem.
However, I agree with you that just using shared_ptr's to control connection lifetimes is not a complete solution since it can lead to resource leaks.
In my answer here: Boost async_* functions and shared_ptr's, I proposed using a combination of shared_ptr and weak_ptr to manage connection lifetimes. An HTTP server using a combination of shared_ptr's and weak_ptr's can be found here: via-httplib.
The HTTP server is built upon an asynchronous TCP server which uses a collection of (shared_ptr's to) connections, created on connects and destroyed on disconnects as you propose.

Overloadable boost::asio::basic_stream_socket

Developing a network application, I have a Connection class that manages sending and receiving messages on the network. I'm using boost::asio.
I now want to let the Connection class handle connections both over TCP, and over local UNIX stream sockets. However, the template-design of boost confuses me. AFAICT, there's no shared base-class between local::stream_protocol::socket and ip::tcp::socket.
How would I go about creating a Connection that encapsulates the network-semantics such that other code don't have to deal with the details of what protocol is used?
I.E. I want to implemented something like:
class Connection() {
Connection(ip::tcp::endpoint& ep);
Connection(local::stream_protocol::endpoint& ep);
void send(Buffer& buf);
}
How would I achieve this?
After some pondering, my current solution is to make the send and recv functions of Connection virtual, and create a template-subclass of Connection, roughly:
template <typename Protocol>
class ConnectionImpl : public Connection {
typedef typename Protocol::socket Socket;
typedef typename Protocol::endpoint EndPoint;
Socket _socket;
public:
ConnectionImpl(boost::asio::io_service& ioSvc, const EndPoint& addr)
: Connection(ioSvc), _socket(ioSvc) {
_socket.connect(addr);
}
void trySend() {
// Initiate async send on _socket here
}
void tryRead() {
// Initiate async recv on _socket here
}
}
Is there a way to avoid the need to subclass and use of virtual functions?
AFAICT, there's no shared base-class between
local::stream_protocol::socket and ip::tcp::socket.
There is explicitly no base class for all socket objects on purpose, the documentation describes the rationale quite well
Unsafe and error prone aspects of the BSD socket API not included. For
example, the use of int to represent all sockets lacks type safety.
The socket representation in Boost.Asio uses a distinct type for each
protocol, e.g. for TCP one would use ip::tcp::socket, and for UDP one
uses ip::udp::socket
Use boost::asio:generic::stream_protocol::socket instead. When you call async_connect()/connect(), it will extract the family and protocol from the remote endpoint and then pass them to the socket() syscall to create the correct socket.
boost::asio::generic::stream_protocol::socket socket_{io_service};
if (use_unix_socket) {
boost::asio::local::stream_protocol::endpoint unix_endpoint{"/tmp/socketpath.sock"};
socket_.async_connect(unix_endpoint, [](boost::system::error_code ec){
}};
}
else {
boost::asio::ip::tcp::endpoint tcp_endpoint{...};
socket_.async_connect(tcp_endpoint, [](boost::system::error_code ec){
}};
}
And there is the code from boost::asio::basic_socket:
template <typename ConnectHandler>
BOOST_ASIO_INITFN_RESULT_TYPE(ConnectHandler,
void (boost::system::error_code))
async_connect(const endpoint_type& peer_endpoint,
BOOST_ASIO_MOVE_ARG(ConnectHandler) handler)
{
// If you get an error on the following line it means that your handler does
// not meet the documented type requirements for a ConnectHandler.
BOOST_ASIO_CONNECT_HANDLER_CHECK(ConnectHandler, handler) type_check;
if (!is_open())
{
boost::system::error_code ec;
const protocol_type protocol = peer_endpoint.protocol();
if (this->get_service().open(this->get_implementation(), protocol, ec))
{
detail::async_result_init<
ConnectHandler, void (boost::system::error_code)> init(
BOOST_ASIO_MOVE_CAST(ConnectHandler)(handler));
this->get_io_service().post(
boost::asio::detail::bind_handler(
BOOST_ASIO_MOVE_CAST(BOOST_ASIO_HANDLER_TYPE(
ConnectHandler, void (boost::system::error_code)))(
init.handler), ec));
return init.result.get();
}
}
return this->get_service().async_connect(this->get_implementation(),
peer_endpoint, BOOST_ASIO_MOVE_CAST(ConnectHandler)(handler));
}