Do websocat handle closing handshake properly with a websocketpp server? - c++

I'm running a websocket server with websocketpp. For applicative reasons, the server may close the connection. When I connect to the server with websocat and the server closes the connection it seems that the closing handshake times out, and the on_close handler is called.
Do you know if websocat handles the closing handshake properly ? (it would be weird if it didn't)
Note : I didn't test with another websocket client.
Am I doing something wrong when closing the connection ?
Why is the on_close handler is called when the closing handshake times out ?
The server code:
#include <thread>
#include <websocketpp/config/asio.hpp>
#include <websocketpp/server.hpp>
#include <iostream>
typedef websocketpp::server<websocketpp::config::asio> server;
using websocketpp::lib::placeholders::_1;
void on_open_connection(server * srv, websocketpp::connection_hdl hdl) {
using namespace std::chrono_literals;
using namespace websocketpp::close::status;
std::this_thread::sleep_for(3s);
auto conn = srv->get_con_from_hdl(hdl);
conn->close(normal, get_string(normal));
}
void on_close(websocketpp::connection_hdl hdl) {
std::cout << "On close" << std::endl;
}
int main() {
server closing_server;
closing_server.init_asio();
closing_server.set_open_handler(std::bind(&on_open_connection, &closing_server, ::_1));
closing_server.set_close_handler(std::bind(&on_close, ::_1));
closing_server.listen(9002);
closing_server.start_accept();
closing_server.run();
}

Related

How to handle ping request on client (boost::beast::websocket)

Imagine that you have some websocket client, that downloading some data in loop like this:
#include <boost/asio.hpp>
#include <boost/beast.hpp>
#include "nlohmann/json.hpp"
namespace beast = boost::beast;
namespace websocket = beast::websocket;
using tcp = boost::asio::ip::tcp;
class Client {
public:
Client(boost::asio::io_context &ctx) : ws_{ctx}, ctx_{ctx} {
ws_.set_option(websocket::stream_base::timeout::suggested(boost::beast::role_type::client));
#define HOST "127.0.0.1"
#define PORT "8000"
boost::asio::connect(ws_.next_layer(), tcp::resolver{ctx_}.resolve(HOST, PORT));
ws_.handshake(HOST ":" PORT, "/api/v1/music");
#undef HOST
#undef PORT
}
~Client() {
if (ws_.is_open()) {
ws_.close(websocket::normal);
}
}
nlohmann::json NextPacket(std::size_t offset) {
nlohmann::json request;
request["offset"] = offset;
ws_.write(boost::asio::buffer(request.dump()));
beast::flat_buffer buffer;
ws_.read(buffer);
return nlohmann::json::parse(std::string_view{reinterpret_cast<const char *>(buffer.data().data()), buffer.size()});
}
private:
boost::beast::websocket::stream<boost::asio::ip::tcp::socket> ws_;
boost::asio::io_context &ctx_;
};
// ... some function
int main() {
boost::asio::io_context context;
boost::asio::executor_work_guard<boost::asio::io_context::executor_type> guard{context.get_executor()};
std::thread{[&context]() { context.run(); }}.detach();
static constexpr std::size_t kSomeVeryBigConstant{1'000'000'000};
Client client{context};
std::size_t offset{};
while (offset < kSomeVeryBigConstant) {
offset += client.NextPacket(offset)["offset"].get<std::size_t>();
// UPDATE:
userDefinedLongPauseHere();
}
}
On the server side we have ping requests with some frequency. Were should I handle ping requests? As I understand it, control_callback controls calls to ping, pong and close functions, not requests. With the read or read_async functions, I also cannot catch the ping request.
Beast responds to pings with pongs automatically, as described here: https://github.com/boostorg/beast/issues/899#issuecomment-346333014
Whenever you call read(), it can process a ping and send a pong without you knowing about that.

Boost inflate algorithm decompress

I am developing a simple test code using Websocket client using c++ boost. A server I get response from says I need to decompress messages using inflate algorithm. I found out there is deflate option in boost Websocket library but it did not work. Please let me know how to convert data to decompressed string.
#include <iostream>
#include <string>
#include <boost/beast/core.hpp>
#include <boost/beast/websocket.hpp>
#include <boost/asio/connect.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/beast/websocket/ssl.hpp>
#include <boost/asio/ssl.hpp>
#include <chrono>
using tcp = boost::asio::ip::tcp;
namespace websocket = boost::beast::websocket;
int main()
{
std::ostringstream stream;
std::string host = "real.okex.com";
auto const port = "8443";
auto const path = "/ws/v3";
boost::beast::multi_buffer buffer;
boost::asio::io_context ioc;
boost::asio::ssl::context ctx{boost::asio::ssl::context::sslv23};
tcp::resolver resolver{ioc};
websocket::stream<boost::asio::ssl::stream<boost::asio::ip::tcp::socket>> wss{ioc, ctx};
ctx.set_verify_mode(boost::asio::ssl::verify_none);
tcp::resolver::results_type results = resolver.resolve(host, port);
boost::asio::connect(wss.next_layer().next_layer(), results.begin(), results.end());
// SSL handshake
wss.next_layer().handshake(boost::asio::ssl::stream_base::client);
// websocket handshake
wss.handshake(host, path);
std::cout << "connected" << std::endl;
// send request to the websocket
wss.write(boost::asio::buffer("{'op':'subscribe', 'args':['spot/ticker:ETH-USDT']}"));
// read message
wss.read(buffer);
std::cout << buffer.size() << std::endl;
buffer.consume(buffer.size());
/*
stream << boost::beast::buffers(buffer.data());
buffer.consume(buffer.size());
std::string incoming = stream.str();
std::cout << incoming << std::endl;
*/
}
Thanks !
I struggled for a long time, then I figured, what if I try with a different server?
That helped. I took echo_compressed/server.py from Autobahn:
wget 'https://github.com/crossbario/autobahn-python/raw/master/examples/twisted/websocket/echo_compressed/server.py'
virtualenv venv && . venv/bin/activate && pip install autobahn twisted
python server.py
That starts a WS server on port 9000. It's not using SSL though, so I disabled that in the code (see #ifdef SSL below).
Now the key is to set the permessage_deflate extension option before WS handshake:
websocket::permessage_deflate opt;
opt.client_enable = true; // for clients
opt.server_enable = true; // for servers
s.set_option(opt);
Also noted that some servers require the port name be present in the Host header when not running on standard ports:
s.handshake(host + ":" + port, path);
Now reading works just fine and deflates as you'd expect, e.g. write it to response.txt:
beast::multi_buffer buffer;
s.read(buffer);
{
std::ofstream ofs("response.txt", std::ios::binary);
std::copy(
net::buffers_begin(buffer.data()),
net::buffers_end(buffer.data()),
std::ostreambuf_iterator<char>(ofs));
}
Or, when replacing the multi_buffer with an Asio streambuf, it's easy to just stream it:
net::streambuf buffer;
s.read(buffer);
std::cout << &buffer;
Proof That It Was Deflating
Inspecting the traffic with tcpdump/Wireshark shows this. Also, the Autobahn logging confirms it:
2020-06-22 02:12:05+0200 [-] Log opened.
2020-06-22 02:12:05+0200 [-] WebSocketServerFactory starting on 9000
2020-06-22 02:12:05+0200 [-] Starting factory <autobahn.twisted.websocket.WebSocketServerFactory object at 0x7f7af3fa5710>
2020-06-22 02:12:05+0200 [-] Site starting on 8080
2020-06-22 02:12:05+0200 [-] Starting factory <twisted.web.server.Site instance at 0x7f7af3850910>
2020-06-22 02:12:11+0200 [-] WebSocket connection request by tcp4:127.0.0.1:48658
2020-06-22 02:12:11+0200 [-] WebSocket extensions in use: [PerMessageDeflate(is_server = True, server_no_context_takeover = False, client_no_context_takeover = False, server_max_window_bits = 15, client_max_window_bits = 15, mem_level = 8)]
The Problem With That Server (real.okex.com)
I don't know what about it, really, but it seems that server is not sending standard responses. Perhaps someone else can tell. Writing the responses to a file did NOT result in a file that looks like it is zlib compressed.
Other tools tried ALSO fail to decode the data:
zlib-flate -uncompress < response.txt
Same with a python oneliner:
python -c 'import zlib; import sys; sys.stdout.write(zlib.decompress(sys.stdin.read()))' < response.txt
Full Listing
As I tested it with:
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/ssl.hpp>
#include <boost/asio.hpp>
#include <boost/beast.hpp>
#include <boost/beast/websocket.hpp>
#include <boost/beast/websocket/ssl.hpp>
#include <iostream>
#include <string>
#include <fstream>
namespace net = boost::asio;
namespace ssl = net::ssl;
namespace beast = boost::beast;
namespace http = beast::http;
namespace websocket = beast::websocket;
using tcp = net::ip::tcp;
//#define SSL
#ifdef SSL
using stream_t = websocket::stream<ssl::stream<tcp::socket>>;
#else
using stream_t = websocket::stream<tcp::socket/*, true*/>;
#endif
int main(int argc, char** argv) {
if (argc<4) {
std::cerr << "Usage: " << argv[0] << " host port path\n";
return 1;
}
std::string host = argc>=2? argv[1] : "real.okex.com";
auto const port = argc>=3? argv[2] : "8443";
auto const path = argc>=3? argv[3] : "/ws/v3";
net::io_context ioc;
ssl::context ctx{ ssl::context::sslv23 };
tcp::resolver resolver{ ioc };
#ifdef SSL
stream_t s{ ioc, ctx };
#else
stream_t s{ ioc };
#endif
ctx.set_verify_mode(ssl::verify_none);
tcp::resolver::results_type results = resolver.resolve(host, port);
net::connect(
beast::get_lowest_layer(s),
//s.next_layer().next_layer(),
results.begin());
#ifdef SSL
// SSL handshake
s.next_layer().handshake(ssl::stream_base::client);
#endif
// websocket handshake
websocket::permessage_deflate opt;
opt.client_enable = true; // for clients
opt.server_enable = true; // for servers
s.set_option(opt);
s.handshake(host + ":" + port, path);
std::cout << "connected" << std::endl;
// send request to the websocket
s.write(net::buffer("{'op':'subscribe', 'args':['spot/ticker:ETH-USDT']}"));
{
net::streambuf buffer;
s.read(buffer);
std::cout << &buffer << std::endl;
}
}
Then I ran with
In the protocol upgrade response, The websocket server should have included a field "Sec-WebSocket-Extensions" which tell the client to use Compression Extensions for WebSocket.
But lots of websocket servers of the crypto exchanges like okex/huobi don't do this. You have to deflate the message in your application code.
You can think of this as moving the deflate/inflate from the protocol layer up to the application layer.

Websocketpp simple HTTP client

I use the excellent websocketpp library to provide a Websockets (and HTTP) server in a C++ application. I also need a HTTP client in the same app to connect to REST APIs. I have been attempting this in websocketpp also, but so far I have had little success. The following preliminary attempt gives me this log output:
[2015-03-06 18:01:18] [connect] Successful connection
[2015-03-06 18:01:18] [error] Server handshake response error: websocketpp.processor:20 (Invalid HTTP status.)
[2015-03-06 18:01:18] [disconnect] Failed: Invalid HTTP status.
This suggests my http_ handler method may need something more. Any advice would be appreciated. The websocketpp docs and examples don't seem to include a simple HTTP client.
#define _WEBSOCKETPP_CPP11_STL_
#include <websocketpp/config/asio_client.hpp>
#include <websocketpp/client.hpp>
#include <websocketpp/common/thread.hpp>
namespace {
using namespace websocketpp;
typedef client<websocketpp::config::asio_client> client;
class Client {
public:
Client(void){
client_.init_asio();
client_.set_http_handler(bind(&Client::http_,this,_1));
}
std::string get(const std::string& url) {
websocketpp::lib::error_code error;
client::connection_ptr con = client_.get_connection(url,error);
if(error) std::runtime_error("Unable to connnect.\n url: "+url+"\n message: "+error.message());
client_.connect(con);
websocketpp::lib::thread asio_thread(&client::run, &client_);
asio_thread.join();
return data_;
}
private:
void http_(connection_hdl hdl){
std::cout<<"Connected\n";
data_ = "http payload";
}
client client_;
std::string data_;
};
}
int main(void){
Client client;
client.get("http://google.com/");
}
WebSocket++'s HTTP handling features are a convenience feature designed to allow WebSocket servers to serve HTTP responses in a limited capacity. WebSocket++ is not intended for use as a generic HTTP library and does not contain the ability to play the role of a (non-WebSocket) HTTP client.
Using a separate library (such as cpp-netlib) for HTTP client functionality is a good solution.
If you're trying to do both WebSocket and HTTP in C++ there's a great library called Beast that has BOTH of these things! Its open source and builds on Boost.Asio:
https://github.com/vinniefalco/Beast/
Here's some example code:
Use HTTP to request the root page from a website and print the response:
#include <beast/http.hpp>
#include <boost/asio.hpp>
#include <iostream>
#include <string>
int main()
{
// Normal boost::asio setup
std::string const host = "boost.org";
boost::asio::io_service ios;
boost::asio::ip::tcp::resolver r(ios);
boost::asio::ip::tcp::socket sock(ios);
boost::asio::connect(sock,
r.resolve(boost::asio::ip::tcp::resolver::query{host, "http"}));
// Send HTTP request using beast
beast::http::request_v1<beast::http::empty_body> req;
req.method = "GET";
req.url = "/";
req.version = 11;
req.headers.replace("Host", host + ":" + std::to_string(sock.remote_endpoint().port()));
req.headers.replace("User-Agent", "Beast");
beast::http::prepare(req);
beast::http::write(sock, req);
// Receive and print HTTP response using beast
beast::streambuf sb;
beast::http::response_v1<beast::http::streambuf_body> resp;
beast::http::read(sock, sb, resp);
std::cout << resp;
}
Establish a WebSocket connection, send a message and receive the reply:
#include <beast/to_string.hpp>
#include <beast/websocket.hpp>
#include <boost/asio.hpp>
#include <iostream>
#include <string>
int main()
{
// Normal boost::asio setup
std::string const host = "echo.websocket.org";
boost::asio::io_service ios;
boost::asio::ip::tcp::resolver r(ios);
boost::asio::ip::tcp::socket sock(ios);
boost::asio::connect(sock,
r.resolve(boost::asio::ip::tcp::resolver::query{host, "80"}));
// WebSocket connect and send message using beast
beast::websocket::stream<boost::asio::ip::tcp::socket&> ws(sock);
ws.handshake(host, "/");
ws.write(boost::asio::buffer("Hello, world!"));
// Receive WebSocket message, print and close using beast
beast::streambuf sb;
beast::websocket::opcode op;
ws.read(op, sb);
ws.close(beast::websocket::close_code::normal);
std::cout << to_string(sb.data()) << "\n";
}
I did not know how to prevent the websocketpp client from asking for a Upgrade: connection so I ended up using cpp-netlib for a HTTP client instead.

Continuously streaming data with Boost::ASIO without closing socket

I'm trying to build an application that:
a) Runs an FDM (flight dynamics model) internally, and manages the flight data
b) Accepts connections on a TCP socket
c) Serves the flight data over said socket.
I've currently managed to get a simple string sent over TCP on my a local socket using the examples/tutorials on the Boos::ASIO website here: http://www.boost.org/doc/libs/1_56_0_b1/doc/html/boost_asio/tutorial.html
My trouble is simply that the connection closes after a single string is written, and I don't know how to keep the connection open and continuously send the data until the simulation is finished (or a stop signal is sent from the listening application).
I also have the FDM working (currently using JSBSim, and borrowing heavily from their included sample code), and can print flight data to stdout no problem. The Boost:ASIO documentation shows a few examples on how to build a server that constantly listens for messages received from the client, but none that send a stream of data out.
To send a stream of data out, you can use the free functions with boost::asio::streambuf.
Here's a simple demo that sends its own source to each client:
#include <boost/asio.hpp>
#include <boost/make_shared.hpp>
#include <boost/function.hpp>
#include <boost/asio/posix/stream_descriptor.hpp>
#include <fstream>
#include <iostream>
namespace io = boost::asio;
namespace ip = io::ip;
using boost::system::error_code;
using boost::make_shared;
using ip::tcp;
void start_accept(io::io_service& svc, tcp::acceptor& acc) {
// per-connection lifetimes:
auto sock = make_shared<tcp::socket>(svc);
acc.async_accept(*sock, [sock,&svc,&acc](error_code ec) {
if (!ec)
{
std::cout << "connection from " << sock->remote_endpoint() << "\n";
// copy source file to buffer data
auto data = make_shared<io::streambuf>();
std::ostream(data.get()) << std::ifstream("main.cpp").rdbuf();
// now write the whole story
io::async_write(*sock, *data, [sock,data/*keep alive*/](error_code ec, size_t transferred){});
// accept new connections too
start_accept(svc, acc);
}
});
}
int main()
{
io::io_service svc;
tcp::acceptor acc(svc, tcp::endpoint(ip::address(), 6767));
start_accept(svc, acc);
svc.run();
}
Note that for simplicity I put the full buffer in memory first, assuming you can do that (you said "and can print flight data to stdout no problem"). So, you could just write different things to the stream (the ostream line in my example).

Socket validation using Poco c++ libraries

I'm working on an application that needs to perform network communication and decided to use the poco c++ libraries. After going through the network tutorial I can't seem to find any forms of validation on establishing a network connection.
In the following example a client tries to connect to a server using a tcp socket stream:
#include "Poco/Net/SocketAddress.h"
#include "Poco/Net/StreamSocket.h"
#include "Poco/Net/SocketStream.h"
#include "Poco/StreamCopier.h"
#include <iostream>
int main(int argc, char** argv)
{
Poco::Net::SocketAddress sa("www.appinf.com", 80);
Poco::Net::StreamSocket socket(sa);
Poco::Net::SocketStream str(socket);
str << "GET / HTTP/1.1\r\n"
"Host: www.appinf.com\r\n"
"\r\n";
str.flush();
Poco::StreamCopier::copyStream(str, std::cout);
return 0;
}
However, I couldn't find any information related to:
Error checking(what if www.appinf.com is unavailable or doesn't exist for that matter)
The type of exception these calls may raise
The only mention is that a SocketStream may hang if the receive timeout is not set for the socket when using formated inputs.
How can I check if a host is alive and may set up a tcp connection, implement a method such as:
void TCPClient::connectTo(std::string host, bool& connected, unsigned int port) {
std::string hi = "hi";
Poco::Net::SocketAddress clientSocketAddress(host, port);
Poco::Net::StreamSocket clientStreamSocket;
// try to connect and avoid hang by setting a timeout
clientStreamSocket.connect(clientSocketAddress, timeout);
// check if the connection has failed or not,
// set the connected parameter accordingly
// additionally try to send bytes over this connection
Poco::Net::SocketStream clientSocketStream(clientStreamSocket);
clientSocketStream << hi << std::endl;
clientSocketStream.flush();
// close the socket stream
clientSocketStream.close();
// close stream
clientStreamSocket.shutdown();
}