how to do boost::asio::spawn with io_service-per-CPU? - c++

Question

My server is based on boost spawn echo server.
The server runs fine on single-core machine, not even one crash for several months. Even when it takes 100% CPU it still works fine.
But I need to handle more client requests, now I use multi-core machine. To use all the CPUs I run io_service on several thread, like this:
#include <boost/asio/io_service.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/spawn.hpp>
#include <boost/asio/steady_timer.hpp>
#include <boost/asio/write.hpp>
#include <boost/thread/thread.hpp>
#include <iostream>
#include <memory>
#include <thread>
using namespace std;
using boost::asio::ip::tcp;
class session : public std::enable_shared_from_this<session>{
public:
explicit session(tcp::socket socket)
: socket_(std::move(socket)),
timer_(socket_.get_io_service()),
strand_(socket_.get_io_service())
{}
void go()
{
auto self(shared_from_this());
boost::asio::spawn(strand_, [this, self](boost::asio::yield_context yield)
{
try {
char data[1024] = {'3'};
for( ; ;) {
timer_.expires_from_now(std::chrono::seconds(10));
std::size_t n = socket_.async_read_some(boost::asio::buffer(data, sizeof(data)), yield);
// do something with data
// write back something
boost::asio::async_write(socket_, boost::asio::buffer(data, sizeof(data)), yield);
}
} catch(...) {
socket_.close();
timer_.cancel();
}
});
boost::asio::spawn(strand_, [this, self](boost::asio::yield_context yield)
{
while(socket_.is_open()) {
boost::system::error_code ignored_ec;
timer_.async_wait(yield[ignored_ec]);
if(timer_.expires_from_now() <= std::chrono::seconds(0))
socket_.close();
}
});
}
private:
tcp::socket socket_;
boost::asio::steady_timer timer_;
boost::asio::io_service::strand strand_;
};
int main(int argc, char* argv[]) {
try {
boost::asio::io_service io_service;
boost::asio::spawn(io_service, [&](boost::asio::yield_context yield)
{
tcp::acceptor acceptor(io_service,
#define PORT "7788"
tcp::endpoint(tcp::v4(), std::atoi(PORT)));
for( ; ;) {
boost::system::error_code ec;
tcp::socket socket(io_service);
acceptor.async_accept(socket, yield[ec]);
if(!ec)
// std::make_shared<session>(std::move(socket))->go();
io_service.post(boost::bind(&session::go, std::make_shared<session>(std::move(socket))));
}
});
// ----------- this works fine on single-core machine ------------
{
// io_service.run();
}
// ----------- this crashes (with multi core) ----------
{
auto thread_count = std::thread::hardware_concurrency(); // for multi core
boost::thread_group threads;
for(auto i = 0; i < thread_count; ++i)
threads.create_thread(boost::bind(&boost::asio::io_service::run, &io_service));
threads.join_all();
}
} catch(std::exception& e) {
std::cerr << "Exception: " << e.what() << "\n";
}
return 0;
}
The code works fine on single-core maching, but crashes all the time on 2-core/4-core/8-core machine. From the crash dump I don't see anything related to my code, just about something with boost::spawn and some randomly named lambda.
So I want to try this: Run io_service per CPU.
I found some demo, but it uses async function:
void server::start_accept()
{
new_connection_.reset(new connection(
io_service_pool_.get_io_service(), request_handler_));
acceptor_.async_accept(new_connection_->socket(),
boost::bind(&server::handle_accept, this,
boost::asio::placeholders::error));
}
void server::handle_accept(const boost::system::error_code& e)
{
if (!e)
{
new_connection_->start();
}
start_accept();
}
The io_service_pool_.get_io_service() randomly pickup an io_service, but my code uses spawn
boost::asio::spawn(io_service, ...
How to spawn with random io_service?

Answer 1

Seems I was asking the wrong question, spawn cannot work with multiple io_service, but the socket can. I modified the code to this:
int main(int argc, char* argv[]) {
try {
boost::asio::io_service io_service;
boost::asio::io_service::work work(io_service);
auto core_count = std::thread::hardware_concurrency();
// io_service_pool.hpp and io_service_pool.cpp from boost's example
io_service_pool pool(core_count);
boost::asio::spawn(io_service, [&](boost::asio::yield_context yield)
{
#define PORT "7788"
tcp::acceptor acceptor(io_service, tcp::endpoint(tcp::v4(), std::atoi(PORT)));
for( ; ;) {
boost::system::error_code ec;
boost::asio::io_service& ios = pool.get_io_service();
tcp::socket socket(ios);
acceptor.async_accept(socket, yield[ec]);
if(!ec)
ios.post(boost::bind(&session::go, std::make_shared<session>(std::move(socket))));
}
});
{ // run all io_service
thread t([&] { pool.run(); });
t.detach();
io_service.run();
}
} catch(std::exception& e) {
std::cerr << "Exception: " << e.what() << "\n";
}
return 0;
}
Now the server doesn't crash anymore. But I still have no idea what could cause the crash if I use a single io_service for all threads.