Fastest way to close a websocket and free up file discriptors - c++

I am using boost beast for making websocket connections, a process of mine may have a large number of streams/connections and while terminating the process I am calling blocking close of each websocket in destructor using :
if ( m_ws.next_layer().next_layer().is_open() )
{
boost::system::error_code ec;
m_ws.close(boost::beast::websocket::close_code::normal, ec);
}
Having a lot of websockets, makes terminating the process blocking for a long time, is there a way to force terminate(delete) a connection and free up underlying resources faster? Thanks in advance.

As I told you in the comments, closing the connection would be a fast operation on sockets but it doesn't take time and block the thread. In your case, I don't know how much work your program does to close each socket, but keep in mind that if your main thread ends, which means that the process has ended, the SO releases all the resources that it has been using, without closing each socket, I use this technic, and the WebSocket clients detect the end of the connections, but I close the socket when I have some problems with the protocol or the remote endpoint has been disconnected abruptly.
It could be useful that you share your code to see what other activities your code is doing.
By the way, let me share with you my code, where I have no problem to close websocket:
void wscnx::close(beast::websocket::close_code reason, const boost::system::error_code &ec)
{
// std::cout << "\nwscnx::close\n";
if (is_ssl && m_wss->is_open())
m_wss->close(beast::websocket::normal);
else if (!is_ssl && m_ws->is_open())
m_wss->close(beast::websocket::normal);
if (!m_server.expired())
m_server.lock()->cnx_closed(m_cnx_info, ec);
}
In my case, I'm using asynchronous methods to read and synchronous methods
to write, I'm not using an asynchronous method to write to avoid the scenery of two simultaneous write operations.
Also, it's important to say that I'm using the asynchronous way to accept a new connection.
The code to accept the socket connection, where you can set the TIMES OUTS to write and read, instead of using timers.
void wscnx::accept_tcp()
{
m_ws->set_option(
websocket::stream_base::timeout::suggested(
beast::role_type::server));
m_ws->set_option(websocket::stream_base::decorator(
[](websocket::response_type &res)
{
res.set(http::field::server,
std::string(BOOST_BEAST_VERSION_STRING) +
" websocket-server-async");
}));
//std::cout << "wscnx::[" << m_id << "]:: TCP async_handshake\n";
m_ws->async_accept(
[self{shared_from_this()}](const boost::system::error_code &ec)
{
if (ec)
{
self->close(beast::websocket::close_code::protocol_error, ec);
return;
}
// self->read_tcp();
self->read();
});
}
The code to read:
void wscnx::read()
{
if (!is_ssl && !m_ws->is_open())
return;
else if (is_ssl && !m_wss->is_open())
return;
auto f_read = [self{shared_from_this()}](const boost::system::error_code &ec, std::size_t bytes_transferred)
{
boost::ignore_unused(bytes_transferred);
// This indicates that the session was closed
if (ec == websocket::error::closed)
{
self->close(beast::websocket::close_code::normal, ec);
return;
}
if (ec)
{
self->close(beast::websocket::close_code::abnormal, ec);
return;
}
std::string data = beast::buffers_to_string(self->m_rx_buffer.data());
self->m_rx_buffer.consume(bytes_transferred);
if (!self->m_server.expired())
{
std::string_view vdata(data.c_str());
self->m_server.lock()->on_data_rx(self->m_cnx_info.id, vdata, self->cnx_info());
}
self->read();
};//lambda
if (!is_ssl)
m_ws->async_read(m_rx_buffer, f_read);
else
m_wss->async_read(m_rx_buffer, f_read);
}
The code to write:
void wscnx::write(std::string_view data, bool close_on_write)
{
std::unique_lock<std::mutex> u_lock(m_mtx_write);
if ( (!is_ssl && !m_ws->is_open()) || (is_ssl && !m_wss->is_open()))
return;
boost::system::error_code ec;
size_t bytes_transferred{0};
if (is_ssl)
bytes_transferred = m_wss->write(net::buffer(data), ec);
else
bytes_transferred = m_ws->write(net::buffer(data), ec);
boost::ignore_unused(bytes_transferred);
// This indicates that the session was closed
if (ec == websocket::error::closed)
{
// std::cout << "[wscnx::[" << self->m_id << "]::on wirite] Error: " << ec.message() << "\n";
close(beast::websocket::close_code::normal, ec);
return;
}
if (ec)
{
// std::cout << "[wscnx::[" << self->m_id << "]::on wirite] Error READING: " << ec.message() << "\n";
close(beast::websocket::close_code::abnormal, ec);
return;
}
if (close_on_write)
close(beast::websocket::close_code::normal, ec);
}
If you want to see the whole code, this is the Link. The project is still in the developer phase, but it works.

Related

async_read_until: insufficient buffer space or queue is full

I'm converting an application from using Juce asynchronous i/o to asio. The first part is to rewrite the code that receives traffic from another application on the same machine (it's a Lightroom Lua plugin that sends \n delimited messages on port 58764). Whenever I successfully connect to that port with my C++ program, I get a series of error codes, all the same:
An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.
Can someone point out my error? I can see that the socket is successfully opened. I've reduced this from my full program to a minimal example. I also tried it with connect instead of async_connect and had the same problem.
#include <iostream>
#include "asio.hpp"
asio::io_context io_context_;
asio::ip::tcp::socket socket_{io_context_};
void loop_me()
{
asio::streambuf streambuf{};
while (true) {
if (!socket_.is_open()) {
return;
}
else {
asio::async_read_until(socket_, streambuf, '\n',
[&streambuf](const asio::error_code& error_code, std::size_t bytes_transferred) {
if (error_code) {
std::cerr << "Socket error " << error_code.message() << std::endl;
return;
}
// Extract up to the first delimiter.
std::string command{buffers_begin(streambuf.data()),
buffers_begin(streambuf.data()) + bytes_transferred};
std::cout << command << std::endl;
streambuf.consume(bytes_transferred);
});
}
}
}
int main()
{
auto work_{asio::make_work_guard(io_context_)};
std::thread io_thread_;
std::thread run_thread_;
io_thread_ = std::thread([] { io_context_.run(); });
socket_.async_connect(asio::ip::tcp::endpoint(asio::ip::address_v4::loopback(), 58764),
[&run_thread_](const asio::error_code& error) {
if (!error) {
std::cout << "Socket connected in LR_IPC_In\n";
run_thread_ = std::thread(loop_me);
}
else {
std::cerr << "LR_IPC_In socket connect failed " << error.message() << std::endl;
}
});
std::this_thread::sleep_for(std::chrono::seconds(1));
socket_.close();
io_context_.stop();
if (io_thread_.joinable())
io_thread_.join();
if (run_thread_.joinable())
run_thread_.join();
}
You are trying to start an infinite number of asynchronous read operations at the same time. You shouldn't start a new asynchronous read until the previous one finished.
async_read_until returns immediately, even though the data hasn't been received yet. That's the point of "async".

Asio two way communication with peristence socket

I have this requirement where my app have to connect to another app via sockets and will have to maintain persistent connection for quiet long time. My app will be a TCP client and the other is a TCP server. My app will send commands and the server will respond accordingly.
The problem am facing right now is how to read the whole data from server a string and return for app which will issue the next command. Reading synchronously (with asio::read) looked like a good option up until I observed socket hanging up until I terminate the server. Looking at the documentation I found that the library is correctly working.
his function is used to read a certain number of bytes of data from a stream. The call will block until one of the following conditions is true:
1. The supplied buffers are full. That is, the bytes transferred is equal to the sum of the buffer sizes.
2. An error occurred.
The problem is I don't know correct buffer size as the response from the server varies. So If I put a too small buffer it returns fine but missing some data. If I put too big it will hang forever until server quits.
So I thought I would do the async reading. It works only once and I don't know how to make it fetch data until whole data it read.
here is the relevant async code
#define ASIO_STANDALONE 1
#include <iostream>
#include <asio.hpp>
int main()
{
asio::io_context context;
size_t reply_length;
size_t length = 1024;
std::vector<char> buffer;
//create socket
asio::ip::tcp::socket socket(context);
socket.connect(asio::ip::tcp::endpoint(asio::ip::address::from_string("127.0.0.1"), 8088));
std::string dataOut = "list --files"; //some command to write
std::error_code error;
asio::write(socket, asio::buffer(dataOut), error);
if (!error)
{
std::cout << "Receiving...!" << std::endl;
buffer.resize(length);
asio::async_read(socket, asio::buffer(buffer), [&buffer, &context](const asio::error_code &ec, std::size_t bytes_transferred) {
std::copy(buffer.begin(), buffer.end(), std::ostream_iterator<char>(std::cout, ""));
std::cout << "\nRead total of:" << bytes_transferred << "\n";
context.run();
});
}
else
{
std::cout << "send failed: " << error.message() << std::endl;
}
context.run();
}
Searching didn't help much solving my issue.
So my question is, how can I read all the data in a persistent socket with asio? Am not using boost.
You need to loop async_read calls. If you don't want your client to hang on read operation you can define the smallest possible buffer i.e. 1 byte.
Define function which takes socket, buffer and two additional parameters according to async_read's handler signature, and this function calls itself with async_read to make the loop of async_read calls - it reads until some error occures:
void onRead (
asio::ip::tcp::socket& socket,
std::array<char,1>& buf,
const system::error_code& ec,
std::size_t bytes)
{
if (ec)
{
if (ec == asio::error::eof && bytes == 1)
std::cout << buf[0];
return;
}
std::cout << buf[0];
asio::async_read(socket,asio::buffer(buf),
std::bind(onRead, std::ref(socket), std::ref(buf),
std::placeholders::_1, // error code
std::placeholders::_2)); // transferred bytes
}
and the changes in main:
std::array<char,1> buf;
asio::write(socket, asio::buffer(dataOut), error);
if (!error)
{
std::cout << "Receiving...!" << std::endl;
asio::async_read(socket, asio::buffer(buf),
std::bind(onRead, std::ref(socket), std::ref(buf),
std::placeholders::_1,
std::placeholders::_2));
context.run();
}
else
{
std::cout << "send failed: " << error.message() << std::endl;
}
(I am using Boost, so you should replace system::error_code on asio::error_code).

boost-beast websocket server that also accept http connections

I need to implement a simple asynchronous websocket server using boost beast that can accept both websocket and standard http connections.
I've tried something like this:
...
// ws is a boost::beast::websocket::stream<boost::asio::ip::tcp::socket>
ws.async_accept_ex(
[](boost::beast::websocket::response_type& res)
{
res.set(boost::beast::http::field::server, "MyServer");
},
[self](boost::beast::error_code e)
{
if (e) self->ReadHttp();
else self->ReadWs();
}
);
...
void ReadHttp()
{
auto self(shared_from_this());
ws.next_layer().async_read_some(
boost::asio::buffer(data, max_length),
[self](boost::system::error_code ec, std::size_t length)
{
if (!self->ws.next_layer().is_open() || ec==boost::asio::error::eof || ec == boost::asio::error::connection_reset)
// handle disconnection
else if (ec)
// handle error
else
{
std::string s(self->data, length);
cout << "HTTP rx: " << s << endl;
self->ReadHttp();
}
}
);
}
void ReadWs()
{
auto self(shared_from_this());
ws.async_read(
rxData,
[self](boost::beast::error_code ec, std::size_t /*length*/)
{
if(ec == boost::beast::websocket::error::closed)
// handle disconnection
else if ( ec )
// handle error
else
{
std::string s((std::istreambuf_iterator<char>(&self->rxData)), std::istreambuf_iterator<char>());
cout << "WS rx: " << s << endl;
self->rxData.consume(self->rxData.size());
self->Read();
}
}
);
}
but when an HTTP client connects, the server misses the first message sent.
Obviously, this is not the correct approach :-)
Can anyone help me with this?
Thanks
The advanced-server and advanced-server-flex examples demonstrate how to build a server that handles normal HTTP requests and that also handles the WebSocket upgrade request:
https://github.com/boostorg/beast/tree/e23ecc8ac903b303b9d1a9824b97c092cb3c09bd/example/advanced/server
https://github.com/boostorg/beast/tree/e23ecc8ac903b303b9d1a9824b97c092cb3c09bd/example/advanced/server-flex

async_send data not sent

[disclaimer] I am new to boost.
Looking into boost::asio and tried to create a simple asynchronous TCP server with the following functionality:
Listen for connections on port 13
When connected, receive data
If data received == time, then return current datetime, else return a predefined string ("Something else was requested")
The problem:
Although, I accept the connection and receive the data, when transmitting data using async_send, although I receive no error and the value of bytes_transferred is correct, I receive empty data on the client side.
If I try to transmit the data from within handle_accept (instead of handle_read), this works fine.
The implementation:
I worked on the boost asio tutorial found here:
Instantiate a tcp_server object, that basically initiates the acceptor and starts listening. as shown below:
int main()
{
try
{
boost::asio::io_service io_service;
tcp_server server(io_service);
io_service.run();
}
catch (std::exception& e)
{
std::cerr << e.what() << std::endl;
}
return 0;
}
and in tcp_server:
class tcp_server
{
public:
tcp_server(boost::asio::io_service& io_service)
: acceptor_(io_service, tcp::endpoint(tcp::v4(), 13))
{
start_accept();
}
private:
void start_accept()
{
using std::cout;
tcp_connection::pointer new_connection =
tcp_connection::create(acceptor_.get_io_service());
acceptor_.async_accept(new_connection->socket(),
boost::bind(&tcp_server::handle_accept, this, new_connection,
boost::asio::placeholders::error));
cout << "Done";
}
...
}
Once a connection is accepted, I am handling it as shown below:
void handle_accept(tcp_connection::pointer new_connection,
const boost::system::error_code& error)
{
if (!error)
{
new_connection->start();
}
start_accept();
}
Below is the tcp_connection::start() method:
void start()
{
boost::asio::async_read(socket_, boost::asio::buffer(inputBuffer_),
boost::bind(&tcp_connection::handle_read, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
/* the snippet below works here - but not in handle_read
outputBuffer_ = make_daytime_string();
boost::asio::async_write(socket_, boost::asio::buffer(outputBuffer_),
boost::bind(&tcp_connection::handle_write, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));*/
}
and in handle_read:
void handle_read(const boost::system::error_code& error, size_t bytes_transferred)
{
outputBuffer_ = make_daytime_string();
if (strcmp(inputBuffer_, "time"))
{
/*this does not work - correct bytes_transferred but nothing shown on receiving end */
boost::asio::async_write(socket_, boost::asio::buffer(outputBuffer_),
boost::bind(&tcp_connection::handle_write, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
else
{
outputBuffer_ = "Something else was requested";//, 128);
boost::asio::async_write(socket_, boost::asio::buffer(outputBuffer_),
boost::bind(&tcp_connection::handle_write, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
}
The handle_write is shown below:
void handle_write(const boost::system::error_code& error,
size_t bytes_transferred)
{
if (!error)
{
std::cout << "Bytes transferred: " << bytes_transferred;
std::cout << "Message sent: " << outputBuffer_;
}
else
{
std::cout << "Error in writing: " << error.message();
}
}
Note the following regarding handle_write (and this is the really strange thing):
There is no error
The bytes_transferred variable has the correct value
outputBuffer_ has the correct value (as set in handle_read)
Nevertheless, the package received at the client side (Packet Sender) is empty (as far as data is concerned).
The complete code is shared here.
Complete test program (c++14). Note the handling of asynchronous buffering when responding to a receive - there may be a send already in progress.
#include <boost/asio.hpp>
#include <thread>
#include <future>
#include <vector>
#include <array>
#include <memory>
#include <mutex>
#include <condition_variable>
#include <iterator>
#include <iostream>
namespace asio = boost::asio;
asio::io_service server_service;
asio::io_service::work server_work{server_service};
bool listening = false;
std::condition_variable cv_listening;
std::mutex management_mutex;
auto const shared_query = asio::ip::tcp::resolver::query(asio::ip::tcp::v4(), "localhost", "8082");
void client()
try
{
asio::io_service client_service;
asio::ip::tcp::socket socket(client_service);
auto lock = std::unique_lock<std::mutex>(management_mutex);
cv_listening.wait(lock, [] { return listening; });
lock.unlock();
asio::ip::tcp::resolver resolver(client_service);
asio::connect(socket, resolver.resolve(shared_query));
auto s = std::string("time\ntime\ntime\n");
asio::write(socket, asio::buffer(s));
socket.shutdown(asio::ip::tcp::socket::shutdown_send);
asio::streambuf sb;
boost::system::error_code sink;
asio::read(socket, sb, sink);
std::cout << std::addressof(sb);
socket.close();
server_service.stop();
}
catch(const boost::system::system_error& se)
{
std::cerr << "client: " << se.code().message() << std::endl;
}
struct connection
: std::enable_shared_from_this<connection>
{
connection(asio::io_service& ios)
: strand_(ios)
{
}
void run()
{
asio::async_read_until(socket_, buffer_, "\n",
strand_.wrap([self = shared_from_this()](auto const&ec, auto size)
{
if (size == 0 )
{
// error condition
boost::system::error_code sink;
self->socket_.shutdown(asio::ip::tcp::socket::shutdown_receive, sink);
}
else {
self->buffer_.commit(size);
std::istream is(std::addressof(self->buffer_));
std::string str;
while (std::getline(is, str))
{
if (str == "time") {
self->queue_send("eight o clock");
}
}
self->run();
}
}));
}
void queue_send(std::string s)
{
assert(strand_.running_in_this_thread());
s += '\n';
send_buffers_pending_.push_back(std::move(s));
nudge_send();
}
void nudge_send()
{
assert(strand_.running_in_this_thread());
if (send_buffers_sending_.empty() and not send_buffers_pending_.empty())
{
std::swap(send_buffers_pending_, send_buffers_sending_);
std::vector<asio::const_buffers_1> send_buffers;
send_buffers.reserve(send_buffers_sending_.size());
std::transform(send_buffers_sending_.begin(), send_buffers_sending_.end(),
std::back_inserter(send_buffers),
[](auto&& str) {
return asio::buffer(str);
});
asio::async_write(socket_, send_buffers,
strand_.wrap([self = shared_from_this()](auto const& ec, auto size)
{
// should check for errors here...
self->send_buffers_sending_.clear();
self->nudge_send();
}));
}
}
asio::io_service::strand strand_;
asio::ip::tcp::socket socket_{strand_.get_io_service()};
asio::streambuf buffer_;
std::vector<std::string> send_buffers_pending_;
std::vector<std::string> send_buffers_sending_;
};
void begin_accepting(asio::ip::tcp::acceptor& acceptor)
{
auto candidate = std::make_shared<connection>(acceptor.get_io_service());
acceptor.async_accept(candidate->socket_, [candidate, &acceptor](auto const& ec)
{
if (not ec) {
candidate->run();
begin_accepting(acceptor);
}
});
}
void server()
try
{
asio::ip::tcp::acceptor acceptor(server_service);
asio::ip::tcp::resolver resolver(server_service);
auto first = resolver.resolve(shared_query);
acceptor.open(first->endpoint().protocol());
acceptor.bind(first->endpoint());
acceptor.listen();
begin_accepting(acceptor);
auto lock = std::unique_lock<std::mutex>(management_mutex);
listening = true;
lock.unlock();
cv_listening.notify_all();
server_service.run();
}
catch(const boost::system::system_error& se)
{
std::cerr << "server: " << se.code().message() << std::endl;
}
int main()
{
using future_type = std::future<void>;
auto stuff = std::array<future_type, 2> {{std::async(std::launch::async, client),
std::async(std::launch::async, server)}};
for (auto& f : stuff) f.wait();
}
There are multiple issues in this code. Some of them may be responsible for your problem:
TCP has no definition of packets, so there's no guarantee that you will ever receive time at once in handle_read. You need a statemachine for that and to respect the bytes_transferred info. If you only have received a part of the message you need to continue at the correct offset. Or you can use asio utility functions, like reading exactly a length of bytes or reading a line.
In addition the last point, you shouldn't really compare the received data with strcmp. That will only work if the remote also sends a null terminator over the connection - does it?
You don't check whether an error happend, although that might manifest itself in other errors.
You are possibly issueing multiple concurrent async writes if you receive multiple data fragments in a shart timespan. This is not valid in asio.
More important, you mutate the send buffer (outputBuffer_) while the send is in progress. This will pretty much lead to undefined behavior. asio might try to write a piece of memory which is no longer valid.
I have solved the problem with the collective help of the comments provided in the question. The behavior that I was experiencing was because of the functionality of async_read. More specifically in the boost asio documentation it reads:
This function is used to asynchronously read a certain number of bytes
of data from a stream. The function call always returns immediately.
The asynchronous operation will continue until one of the following
conditions is true:
The supplied buffers are full. That is, the bytes transferred is equal to the sum of the buffer sizes.
An error occurred.
The inputBuffer_ I was using to read the input, was a 128 char array. The client I was using, would only transfer the real data (without padding), and therefore the async_read would not return until the connection was closed by the client (or 128 bytes of data were transferred). When the connection was closed by the client there was no way to send back the requested data. This is also the reason that it was working with #Arunmu's simple python tcp client (because he was sending 128 bytes of data always).
To fix the issues, I made the following changes (the full working code is supplied here for reference):
In tcp_connection::start: I am now using async_read_until to read the incoming data (and use \n as a delimiter). The input is stored in a boost::asio::streambuf. async_read is guaranteed to return once the delimiter has been found, or an error has occurred. So there is no chance to issue multiple async_write concurrently.
In handle_read: I have included error checking, which made it much simpler to debug.

Set affinity of c++ std::thread

I am trying to make precise time measurements in Windows when UDP datagrams arrive. Reading documentation from Microsoft I have decided to use QueryPerformanceCounter. In the same documentation it suggests that you set the affinity and performance of the timing thread. I am using Boost and c++ 11 to implement my async UDP server:
void receiveEvent()
{
listening_socket.async_receive_from(
boost::asio::buffer(event_buffer, max_length), signaling_endpoint,
[this](boost::system::error_code ec, std::size_t bytes_recvd)
{
if (!ec && bytes_recvd > 0)
{
LARGE_INTEGER prectime;
::QueryPerformanceCounter(&prectime);
std::cout << prectime.QuadPart << std::endl;
} else if (ec == boost::asio::error::operation_aborted) {
std::cout << "socket canceled" << std::endl;
return;
}
receiveEvent();
});
}
listening_socket(io, boost::asio::ip::udp::endpoint(boost::asio::ip::udp::v4(), port));
receiveEvent();
async_thread = std::thread(boost::bind(&boost::asio::io_service::run, &io));
// TODO set thread affinity using SetProcessAffinityMask
// TODO set thread priority using SetPriorityClass
How do I use std::thread async_thread with SetProcessAffinityMask and SetPriorityClass?