I use the excellent websocketpp library to provide a Websockets (and HTTP) server in a C++ application. I also need a HTTP client in the same app to connect to REST APIs. I have been attempting this in websocketpp also, but so far I have had little success. The following preliminary attempt gives me this log output:
[2015-03-06 18:01:18] [connect] Successful connection
[2015-03-06 18:01:18] [error] Server handshake response error: websocketpp.processor:20 (Invalid HTTP status.)
[2015-03-06 18:01:18] [disconnect] Failed: Invalid HTTP status.
This suggests my http_ handler method may need something more. Any advice would be appreciated. The websocketpp docs and examples don't seem to include a simple HTTP client.
#define _WEBSOCKETPP_CPP11_STL_
#include <websocketpp/config/asio_client.hpp>
#include <websocketpp/client.hpp>
#include <websocketpp/common/thread.hpp>
namespace {
using namespace websocketpp;
typedef client<websocketpp::config::asio_client> client;
class Client {
public:
Client(void){
client_.init_asio();
client_.set_http_handler(bind(&Client::http_,this,_1));
}
std::string get(const std::string& url) {
websocketpp::lib::error_code error;
client::connection_ptr con = client_.get_connection(url,error);
if(error) std::runtime_error("Unable to connnect.\n url: "+url+"\n message: "+error.message());
client_.connect(con);
websocketpp::lib::thread asio_thread(&client::run, &client_);
asio_thread.join();
return data_;
}
private:
void http_(connection_hdl hdl){
std::cout<<"Connected\n";
data_ = "http payload";
}
client client_;
std::string data_;
};
}
int main(void){
Client client;
client.get("http://google.com/");
}
WebSocket++'s HTTP handling features are a convenience feature designed to allow WebSocket servers to serve HTTP responses in a limited capacity. WebSocket++ is not intended for use as a generic HTTP library and does not contain the ability to play the role of a (non-WebSocket) HTTP client.
Using a separate library (such as cpp-netlib) for HTTP client functionality is a good solution.
If you're trying to do both WebSocket and HTTP in C++ there's a great library called Beast that has BOTH of these things! Its open source and builds on Boost.Asio:
https://github.com/vinniefalco/Beast/
Here's some example code:
Use HTTP to request the root page from a website and print the response:
#include <beast/http.hpp>
#include <boost/asio.hpp>
#include <iostream>
#include <string>
int main()
{
// Normal boost::asio setup
std::string const host = "boost.org";
boost::asio::io_service ios;
boost::asio::ip::tcp::resolver r(ios);
boost::asio::ip::tcp::socket sock(ios);
boost::asio::connect(sock,
r.resolve(boost::asio::ip::tcp::resolver::query{host, "http"}));
// Send HTTP request using beast
beast::http::request_v1<beast::http::empty_body> req;
req.method = "GET";
req.url = "/";
req.version = 11;
req.headers.replace("Host", host + ":" + std::to_string(sock.remote_endpoint().port()));
req.headers.replace("User-Agent", "Beast");
beast::http::prepare(req);
beast::http::write(sock, req);
// Receive and print HTTP response using beast
beast::streambuf sb;
beast::http::response_v1<beast::http::streambuf_body> resp;
beast::http::read(sock, sb, resp);
std::cout << resp;
}
Establish a WebSocket connection, send a message and receive the reply:
#include <beast/to_string.hpp>
#include <beast/websocket.hpp>
#include <boost/asio.hpp>
#include <iostream>
#include <string>
int main()
{
// Normal boost::asio setup
std::string const host = "echo.websocket.org";
boost::asio::io_service ios;
boost::asio::ip::tcp::resolver r(ios);
boost::asio::ip::tcp::socket sock(ios);
boost::asio::connect(sock,
r.resolve(boost::asio::ip::tcp::resolver::query{host, "80"}));
// WebSocket connect and send message using beast
beast::websocket::stream<boost::asio::ip::tcp::socket&> ws(sock);
ws.handshake(host, "/");
ws.write(boost::asio::buffer("Hello, world!"));
// Receive WebSocket message, print and close using beast
beast::streambuf sb;
beast::websocket::opcode op;
ws.read(op, sb);
ws.close(beast::websocket::close_code::normal);
std::cout << to_string(sb.data()) << "\n";
}
I did not know how to prevent the websocketpp client from asking for a Upgrade: connection so I ended up using cpp-netlib for a HTTP client instead.
Related
Imagine that you have some websocket client, that downloading some data in loop like this:
#include <boost/asio.hpp>
#include <boost/beast.hpp>
#include "nlohmann/json.hpp"
namespace beast = boost::beast;
namespace websocket = beast::websocket;
using tcp = boost::asio::ip::tcp;
class Client {
public:
Client(boost::asio::io_context &ctx) : ws_{ctx}, ctx_{ctx} {
ws_.set_option(websocket::stream_base::timeout::suggested(boost::beast::role_type::client));
#define HOST "127.0.0.1"
#define PORT "8000"
boost::asio::connect(ws_.next_layer(), tcp::resolver{ctx_}.resolve(HOST, PORT));
ws_.handshake(HOST ":" PORT, "/api/v1/music");
#undef HOST
#undef PORT
}
~Client() {
if (ws_.is_open()) {
ws_.close(websocket::normal);
}
}
nlohmann::json NextPacket(std::size_t offset) {
nlohmann::json request;
request["offset"] = offset;
ws_.write(boost::asio::buffer(request.dump()));
beast::flat_buffer buffer;
ws_.read(buffer);
return nlohmann::json::parse(std::string_view{reinterpret_cast<const char *>(buffer.data().data()), buffer.size()});
}
private:
boost::beast::websocket::stream<boost::asio::ip::tcp::socket> ws_;
boost::asio::io_context &ctx_;
};
// ... some function
int main() {
boost::asio::io_context context;
boost::asio::executor_work_guard<boost::asio::io_context::executor_type> guard{context.get_executor()};
std::thread{[&context]() { context.run(); }}.detach();
static constexpr std::size_t kSomeVeryBigConstant{1'000'000'000};
Client client{context};
std::size_t offset{};
while (offset < kSomeVeryBigConstant) {
offset += client.NextPacket(offset)["offset"].get<std::size_t>();
// UPDATE:
userDefinedLongPauseHere();
}
}
On the server side we have ping requests with some frequency. Were should I handle ping requests? As I understand it, control_callback controls calls to ping, pong and close functions, not requests. With the read or read_async functions, I also cannot catch the ping request.
Beast responds to pings with pongs automatically, as described here: https://github.com/boostorg/beast/issues/899#issuecomment-346333014
Whenever you call read(), it can process a ping and send a pong without you knowing about that.
I am trying to send HTTPS request to a server and receive the page contents by only using Boost.Asio(not Network.Ts or Beast or others) by these code :
#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
#include <iostream>
int main() {
boost::system::error_code ec;
using namespace boost::asio;
// what we need
io_service svc;
ssl::context ctx(ssl::context::method::tlsv1);
ssl::stream<ip::tcp::socket> ssock(svc, ctx);
ip::tcp::endpoint endpoint(boost::asio::ip::make_address("157.90.94.153",ec),443);
ssock.lowest_layer().connect(endpoint);
ssock.handshake(ssl::stream_base::handshake_type::client);
// send request
std::string request("GET /index.html HTTP/1.1\r\n\r\n");
boost::asio::write(ssock, buffer(request));
// read response
std::string response;
do {
char buf[1024];
size_t bytes_transferred = ssock.read_some(buffer(buf), ec);
if (!ec) response.append(buf, buf + bytes_transferred);
} while (!ec);
// print and exit
std::cout << "Response received: '" << response << "'\n";
}
But I keep getting 405 Not Allowed on my local PC and 400 Bad Request on Coliru.
What did I do wrong?
... "GET /index.html HTTP/1.1\r\n\r\n"
This is not a valid HTTP/1.1 request. It must at least also contain a Host field and the value of the field must match the servers expectation, i.e.
"GET /index.html HTTP/1.1\r\nHost: example.com\r\n\r\n"
In general, HTTP might look easy but is actually complex and has several pitfalls. If you really need to do HTTP by your own please study the standard.
I am developing a simple test code using Websocket client using c++ boost. A server I get response from says I need to decompress messages using inflate algorithm. I found out there is deflate option in boost Websocket library but it did not work. Please let me know how to convert data to decompressed string.
#include <iostream>
#include <string>
#include <boost/beast/core.hpp>
#include <boost/beast/websocket.hpp>
#include <boost/asio/connect.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/beast/websocket/ssl.hpp>
#include <boost/asio/ssl.hpp>
#include <chrono>
using tcp = boost::asio::ip::tcp;
namespace websocket = boost::beast::websocket;
int main()
{
std::ostringstream stream;
std::string host = "real.okex.com";
auto const port = "8443";
auto const path = "/ws/v3";
boost::beast::multi_buffer buffer;
boost::asio::io_context ioc;
boost::asio::ssl::context ctx{boost::asio::ssl::context::sslv23};
tcp::resolver resolver{ioc};
websocket::stream<boost::asio::ssl::stream<boost::asio::ip::tcp::socket>> wss{ioc, ctx};
ctx.set_verify_mode(boost::asio::ssl::verify_none);
tcp::resolver::results_type results = resolver.resolve(host, port);
boost::asio::connect(wss.next_layer().next_layer(), results.begin(), results.end());
// SSL handshake
wss.next_layer().handshake(boost::asio::ssl::stream_base::client);
// websocket handshake
wss.handshake(host, path);
std::cout << "connected" << std::endl;
// send request to the websocket
wss.write(boost::asio::buffer("{'op':'subscribe', 'args':['spot/ticker:ETH-USDT']}"));
// read message
wss.read(buffer);
std::cout << buffer.size() << std::endl;
buffer.consume(buffer.size());
/*
stream << boost::beast::buffers(buffer.data());
buffer.consume(buffer.size());
std::string incoming = stream.str();
std::cout << incoming << std::endl;
*/
}
Thanks !
I struggled for a long time, then I figured, what if I try with a different server?
That helped. I took echo_compressed/server.py from Autobahn:
wget 'https://github.com/crossbario/autobahn-python/raw/master/examples/twisted/websocket/echo_compressed/server.py'
virtualenv venv && . venv/bin/activate && pip install autobahn twisted
python server.py
That starts a WS server on port 9000. It's not using SSL though, so I disabled that in the code (see #ifdef SSL below).
Now the key is to set the permessage_deflate extension option before WS handshake:
websocket::permessage_deflate opt;
opt.client_enable = true; // for clients
opt.server_enable = true; // for servers
s.set_option(opt);
Also noted that some servers require the port name be present in the Host header when not running on standard ports:
s.handshake(host + ":" + port, path);
Now reading works just fine and deflates as you'd expect, e.g. write it to response.txt:
beast::multi_buffer buffer;
s.read(buffer);
{
std::ofstream ofs("response.txt", std::ios::binary);
std::copy(
net::buffers_begin(buffer.data()),
net::buffers_end(buffer.data()),
std::ostreambuf_iterator<char>(ofs));
}
Or, when replacing the multi_buffer with an Asio streambuf, it's easy to just stream it:
net::streambuf buffer;
s.read(buffer);
std::cout << &buffer;
Proof That It Was Deflating
Inspecting the traffic with tcpdump/Wireshark shows this. Also, the Autobahn logging confirms it:
2020-06-22 02:12:05+0200 [-] Log opened.
2020-06-22 02:12:05+0200 [-] WebSocketServerFactory starting on 9000
2020-06-22 02:12:05+0200 [-] Starting factory <autobahn.twisted.websocket.WebSocketServerFactory object at 0x7f7af3fa5710>
2020-06-22 02:12:05+0200 [-] Site starting on 8080
2020-06-22 02:12:05+0200 [-] Starting factory <twisted.web.server.Site instance at 0x7f7af3850910>
2020-06-22 02:12:11+0200 [-] WebSocket connection request by tcp4:127.0.0.1:48658
2020-06-22 02:12:11+0200 [-] WebSocket extensions in use: [PerMessageDeflate(is_server = True, server_no_context_takeover = False, client_no_context_takeover = False, server_max_window_bits = 15, client_max_window_bits = 15, mem_level = 8)]
The Problem With That Server (real.okex.com)
I don't know what about it, really, but it seems that server is not sending standard responses. Perhaps someone else can tell. Writing the responses to a file did NOT result in a file that looks like it is zlib compressed.
Other tools tried ALSO fail to decode the data:
zlib-flate -uncompress < response.txt
Same with a python oneliner:
python -c 'import zlib; import sys; sys.stdout.write(zlib.decompress(sys.stdin.read()))' < response.txt
Full Listing
As I tested it with:
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/ssl.hpp>
#include <boost/asio.hpp>
#include <boost/beast.hpp>
#include <boost/beast/websocket.hpp>
#include <boost/beast/websocket/ssl.hpp>
#include <iostream>
#include <string>
#include <fstream>
namespace net = boost::asio;
namespace ssl = net::ssl;
namespace beast = boost::beast;
namespace http = beast::http;
namespace websocket = beast::websocket;
using tcp = net::ip::tcp;
//#define SSL
#ifdef SSL
using stream_t = websocket::stream<ssl::stream<tcp::socket>>;
#else
using stream_t = websocket::stream<tcp::socket/*, true*/>;
#endif
int main(int argc, char** argv) {
if (argc<4) {
std::cerr << "Usage: " << argv[0] << " host port path\n";
return 1;
}
std::string host = argc>=2? argv[1] : "real.okex.com";
auto const port = argc>=3? argv[2] : "8443";
auto const path = argc>=3? argv[3] : "/ws/v3";
net::io_context ioc;
ssl::context ctx{ ssl::context::sslv23 };
tcp::resolver resolver{ ioc };
#ifdef SSL
stream_t s{ ioc, ctx };
#else
stream_t s{ ioc };
#endif
ctx.set_verify_mode(ssl::verify_none);
tcp::resolver::results_type results = resolver.resolve(host, port);
net::connect(
beast::get_lowest_layer(s),
//s.next_layer().next_layer(),
results.begin());
#ifdef SSL
// SSL handshake
s.next_layer().handshake(ssl::stream_base::client);
#endif
// websocket handshake
websocket::permessage_deflate opt;
opt.client_enable = true; // for clients
opt.server_enable = true; // for servers
s.set_option(opt);
s.handshake(host + ":" + port, path);
std::cout << "connected" << std::endl;
// send request to the websocket
s.write(net::buffer("{'op':'subscribe', 'args':['spot/ticker:ETH-USDT']}"));
{
net::streambuf buffer;
s.read(buffer);
std::cout << &buffer << std::endl;
}
}
Then I ran with
In the protocol upgrade response, The websocket server should have included a field "Sec-WebSocket-Extensions" which tell the client to use Compression Extensions for WebSocket.
But lots of websocket servers of the crypto exchanges like okex/huobi don't do this. You have to deflate the message in your application code.
You can think of this as moving the deflate/inflate from the protocol layer up to the application layer.
I need to write a command line client for playing tic-tac-toe over a server.
the server accepts http requests and sends back json to my client. i am looking for a quick way to send a http request and receive the json as a string using boost libraries.
example http request = "http://???/newGame?name=david"
example json response = "\"status\":\"okay\", \"id\":\"game-23\", \"letter\":2"
The simplest thing that fits the description:
Live On Coliru
#include <boost/asio.hpp>
#include <iostream>
int main() {
boost::system::error_code ec;
using namespace boost::asio;
// what we need
io_service svc;
ip::tcp::socket sock(svc);
sock.connect({ {}, 8087 }); // http://localhost:8087 for testing
// send request
std::string request("GET /newGame?name=david HTTP/1.1\r\n\r\n");
sock.send(buffer(request));
// read response
std::string response;
do {
char buf[1024];
size_t bytes_transferred = sock.receive(buffer(buf), {}, ec);
if (!ec) response.append(buf, buf + bytes_transferred);
} while (!ec);
// print and exit
std::cout << "Response received: '" << response << "'\n";
}
This receives the full response. You can test it with a dummy server:(also Live On Coliru):
netcat -l localhost 8087 <<< '"status":"okay", "id":"game-23", "letter":2'
This will show that the request is received, and the response will be written out by our client code above.
Note that for more ideas you could look at the examples http://www.boost.org/doc/libs/release/doc/html/boost_asio/examples.html (although they focus on asynchronous communications, because that's the topic of the Asio library)
I'm working on an application that needs to perform network communication and decided to use the poco c++ libraries. After going through the network tutorial I can't seem to find any forms of validation on establishing a network connection.
In the following example a client tries to connect to a server using a tcp socket stream:
#include "Poco/Net/SocketAddress.h"
#include "Poco/Net/StreamSocket.h"
#include "Poco/Net/SocketStream.h"
#include "Poco/StreamCopier.h"
#include <iostream>
int main(int argc, char** argv)
{
Poco::Net::SocketAddress sa("www.appinf.com", 80);
Poco::Net::StreamSocket socket(sa);
Poco::Net::SocketStream str(socket);
str << "GET / HTTP/1.1\r\n"
"Host: www.appinf.com\r\n"
"\r\n";
str.flush();
Poco::StreamCopier::copyStream(str, std::cout);
return 0;
}
However, I couldn't find any information related to:
Error checking(what if www.appinf.com is unavailable or doesn't exist for that matter)
The type of exception these calls may raise
The only mention is that a SocketStream may hang if the receive timeout is not set for the socket when using formated inputs.
How can I check if a host is alive and may set up a tcp connection, implement a method such as:
void TCPClient::connectTo(std::string host, bool& connected, unsigned int port) {
std::string hi = "hi";
Poco::Net::SocketAddress clientSocketAddress(host, port);
Poco::Net::StreamSocket clientStreamSocket;
// try to connect and avoid hang by setting a timeout
clientStreamSocket.connect(clientSocketAddress, timeout);
// check if the connection has failed or not,
// set the connected parameter accordingly
// additionally try to send bytes over this connection
Poco::Net::SocketStream clientSocketStream(clientStreamSocket);
clientSocketStream << hi << std::endl;
clientSocketStream.flush();
// close the socket stream
clientSocketStream.close();
// close stream
clientStreamSocket.shutdown();
}