Reading a serialized struct at the receiver end boost asio - c++

I am new to boost and networking ;). I am making a client server application with boost::asio, I need to pass structs as messages so used boost::asio::serialization for it :
test.h
#pragma once
#include <boost/archive/binary_oarchive.hpp>
#include <boost/serialization/serialization.hpp>
struct Test
{
public:
int a;
int b;
template<typename archive> void serialize(archive& ar, const unsigned version) {
ar & a;
ar & b;
}
};
client side sending:
void send_asynchronously(tcp::socket& socket) {
Test info;
info.a = 1;
info.b = 2;
{
std::ostream os(&buf);
boost::archive::binary_oarchive out_archive(os);
out_archive << info;
}
async_write(socket, buf, on_send_completed);
}
On the receiver side, I read the data into a boost::asio::buffer, I want to know a way to parse this buffer and extract the object on server side. Please help.

You don't show enough code to know how you declared buf or managed the lifetime.
I'm assuming you used boost::asio::streambuf buf; and it has static storage duration (namespace scope) or is a class member (but you didn't show a class).
Either way, whatever you have you can do "the same" in reverse to receive.
Here's a shortened version (that leaves out the async so we don't have to make guesses about the lifetimes of things like I mentioned above);
Connect
Let's connect to an imaginary server (we can make one below) at port 3001 on localhost:
asio::io_context ioc;
asio::streambuf buf;
tcp::socket s(ioc, tcp::v4());
s.connect({{}, 3001});
Serialize
Basically what you had:
{
std::ostream os(&buf);
boost::archive::binary_oarchive oa(os);
Test req {13,31};
oa << req;
}
Note the {} scope around the stream/archive make sure the archive is completed before sending.
Send
/*auto bytes_sent =*/ asio::write(s, buf);
Receive
Let's assume our server sends back another Test object serialized in the same way¹.
Reading into the buffer, assuming no framing we'll just "read until the end of the stream":
boost::system::error_code ec;
/*auto bytes_received =*/ asio::read(s, buf, ec);
if (ec && ec != asio::error::eof) {
std::cout << "Read error: " << ec.message() << "\n";
return 1;
}
In real life you want timeouts and limits to the amount of data read. Often your protocol will add framing where you know what amount of data to read or what boundary marker to expect.
Deserialize
Test response; // uninitialized
{
std::istream is(&buf);
boost::archive::binary_iarchive ia(is);
ia >> response;
}
Full Demo
Live On Coliru
#include <boost/asio.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
#include <boost/serialization/serialization.hpp>
#include <iostream>
namespace asio = boost::asio;
using tcp = boost::asio::ip::tcp;
struct Test {
int a,b;
template<typename Ar> void serialize(Ar& ar, unsigned) { ar & a & b; }
};
int main() {
asio::io_context ioc;
asio::streambuf buf;
tcp::socket s(ioc, tcp::v4());
s.connect({{}, 3001});
///////////////////
// send a "request"
///////////////////
{
std::ostream os(&buf);
boost::archive::binary_oarchive oa(os);
Test req {13,31};
oa << req;
}
/*auto bytes_sent =*/ asio::write(s, buf);
/////////////////////
// receive "response"
/////////////////////
boost::system::error_code ec;
/*auto bytes_received =*/ asio::read(s, buf, ec);
if (ec && ec != asio::error::eof) {
std::cout << "Read error: " << ec.message() << "\n";
return 1;
}
Test response; // uninitialized
{
std::istream is(&buf);
boost::archive::binary_iarchive ia(is);
ia >> response;
}
std::cout << "Response: {" << response.a << ", " << response.b << "}\n";
}
Using netcat to mock a server with a previously generated response Test{42,99} (base64 encoded here):
base64 -d <<<"FgAAAAAAAABzZXJpYWxpemF0aW9uOjphcmNoaXZlEgAECAQIAQAAAAAAAAAAKgAAAGMAAAA=" | nc -N -l -p 3001
It prints:
Response: {42, 99}
¹ on the same architecture and compiled with the same version of boost, because Boost's binary archives are not portable. The live demo is good demonstration of this

Related

Boost asio - is it possible to reuse a socket?

After I:
created a tcp-socket
put it in listening
received an Incoming "message" from the Client
"processed" the incoming accomplice and closed the socket...
Whether that it is possible to put this socket on listening again? Experiment shows that no - the accept() function - throws an error.
Is it possible, somehow, to "reset" a closed socket to its original state? In order not to release the previously allocated memory for this socket and not to create a new socket, which in the end will also have to be deleted.
PS:
my_socket_p = new boost::asio::ip::tcp::socket(io_context);
my_acceptor.accept(my_socket_p );
The socket you put in listening mode is typically not the socket that you close when you're done handling a request.
Asio makes this distinction more explicit than in the underlying sockets API. I discussed this before here: Design rationale behind that separate acceptor class exists in ASIO (see also e.g. What does it mean to "open" an acceptor?).
Simple example of a typical server that accepts requests and echoes them back reversed:
Live On Coliru
#include <boost/asio.hpp>
#include <iostream>
namespace asio = boost::asio;
using asio::ip::tcp;
int main() {
boost::asio::io_context ioc;
// listen on port 7878
tcp::acceptor acc(ioc, {{}, 7878});
acc.listen();
while (true) {
tcp::socket conn = acc.accept();
auto peer = conn.remote_endpoint();
try {
std::string request;
read_until(conn, asio::dynamic_buffer(request), "\n");
reverse(begin(request), end(request) - 1); // reverse until line-end
write(conn, asio::buffer("reversed: " + request));
conn.close();
} catch(boost::system::system_error const& se) {
std::cerr << "Error with peer " << peer << ": " << se.code().message() << std::endl;
}
}
}
With example clients:
netcat 127.0.0.1 7878 <<< "Hello world!"
netcat 127.0.0.1 7878 <<< "Bye world!"
Prints
reversed: !dlrow olleH
reversed: !dlrow eyB
Re-using?
There are other overloads of accept that take a socket by reference, and yes they can be re-used:
tcp::socket conn(ioc); // reused
while (true) {
acc.accept(conn);
auto peer = conn.remote_endpoint();
With the rest of the code un-changed: Live On Coliru
More Typical
More typically code would be asynchronous and the "connection handling" would be in some other class, e.g. Session. In such cases conn would be moved into Session. This is also explicitly documented as proper use, e.g.:
Following the move, the moved-from object is in the same state as if constructed using the basic_socket(const executor_type&) constructor.
So, you could write the code as such:
tcp::socket conn(ioc); // reused
while (true) {
acc.accept(conn);
std::make_shared<Session>(std::move(conn))->run();
}
You can see it Live On Coliru again, although I'll include the far more idiomatic version using the move-accept handler overload of async_accept instead here:
Live On Coliru
#include <boost/asio.hpp>
#include <iostream>
namespace asio = boost::asio;
using asio::ip::tcp;
using boost::system::error_code;
struct Session : std::enable_shared_from_this<Session> {
Session(tcp::socket s) : conn_(std::move(s)) {}
void run() { do_reverse_echo(); }
private:
void do_reverse_echo() {
async_read_until( //
conn_, asio::dynamic_buffer(buffer_), "\n",
[this, self = shared_from_this()](error_code ec, size_t) {
if (ec) {
std::cerr << "Error with peer " << peer_ << ": " << ec.message() << std::endl;
return;
}
reverse(begin(buffer_), end(buffer_) - 1); // reverse until line-end
buffer_ = "reversed: " + buffer_;
async_write(conn_, asio::buffer(buffer_), asio::detached);
});
}
tcp::socket conn_;
tcp::endpoint peer_{conn_.remote_endpoint()};
std::string buffer_;
};
struct Listener {
Listener(asio::any_io_executor ex, uint16_t port) : acc(ex, {{}, port}) {
acc.listen();
accept_loop();
}
private:
tcp::acceptor acc;
void accept_loop() {
acc.async_accept([this](error_code ec, tcp::socket conn) {
if (ec) {
std::cerr << "Stopping listener: " << ec.message() << std::endl;
return;
}
accept_loop();
std::make_shared<Session>(std::move(conn))->run();
});
}
};
int main() {
boost::asio::io_context ioc;
Listener s(ioc.get_executor(), 7878);
ioc.run();
}
Still with the same output, obviously.

Boost Beast Read Conent By Portions

I am trying to understand how can I limit the amount of data that is read from the internet by calling 'read_some' function in boost beast.
The starting point is the incremental read example in the beast's docs.
From the docs I understood that the really read data is stored in the flat_buffer.
I make the following experiment:
Set max flat_buffer's size to 1024
Connect to a relatively large (several KB) html page
Call read_some one time
Turn the internet off
Try to read the page to the end
Since buffer's capacity is not large enough to store the entire page, my experiment should fail - I should not be able to read the entire page. Nevertheless, it finishes successfully. That means that there exists some additional buffer where the read data is stored. But what is it made for and how can I limit its size?
UPD
Here is my source code:
#include <boost/beast/core.hpp>
#include <boost/beast/http.hpp>
#include <boost/beast/version.hpp>
#include <boost/asio/strand.hpp>
#include <cstdlib>
#include <functional>
#include <iostream>
#include <memory>
#include <string>
namespace beast = boost::beast; // from <boost/beast.hpp>
namespace http = beast::http; // from <boost/beast/http.hpp>
namespace net = boost::asio; // from <boost/asio.hpp>
using namespace http;
template<
bool isRequest,
class SyncReadStream,
class DynamicBuffer>
void
read_and_print_body(
std::ostream& os,
SyncReadStream& stream,
DynamicBuffer& buffer,
boost::beast::error_code& ec ) {
parser<isRequest, buffer_body> p;
read_header( stream, buffer, p, ec );
if ( ec )
return;
while ( !p.is_done()) {
char buf[512];
p.get().body().data = buf;
p.get().body().size = sizeof( buf );
read_some( stream, buffer, p, ec );
if ( ec == error::need_buffer )
ec = {};
if ( ec )
return;
os.write( buf, sizeof( buf ) - p.get().body().size );
}
}
int main(int argc, char** argv)
{
try
{
// Check command line arguments.
if(argc != 4 && argc != 5)
{
std::cerr <<
"Usage: http-client-sync <host> <port> <target> [<HTTP version: 1.0 or 1.1(default)>]\n" <<
"Example:\n" <<
" http-client-sync www.example.com 80 /\n" <<
" http-client-sync www.example.com 80 / 1.0\n";
return EXIT_FAILURE;
}
auto const host = argv[1];
auto const port = argv[2];
auto const target = argv[3];
int version = argc == 5 && !std::strcmp("1.0", argv[4]) ? 10 : 11;
// The io_context is required for all I/O
net::io_context ioc;
// These objects perform our I/O
boost::asio::ip::tcp::resolver resolver(ioc);
beast::tcp_stream stream(ioc);
// Look up the domain name
auto const results = resolver.resolve(host, port);
// Make the connection on the IP address we get from a lookup
stream.connect(results);
// Set up an HTTP GET request message
http::request<http::string_body> req{http::verb::get, target, version};
req.set(http::field::host, host);
req.set(http::field::user_agent, BOOST_BEAST_VERSION_STRING);
// Send the HTTP request to the remote host
http::write(stream, req);
// This buffer is used for reading and must be persisted
beast::flat_buffer buffer;
boost::beast::error_code ec;
read_and_print_body<false>(std::cout, stream, buffer, ec);
}
catch(std::exception const& e)
{
std::cerr << "Error: " << e.what() << std::endl;
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
The operating system's TCP IP stack obviously needs to buffer data, so that's likely where it gets buffered.
The way to test your desired scenario:
Live On Coliru
#include <boost/beast.hpp>
#include <iostream>
#include <thread>
namespace net = boost::asio;
namespace beast = boost::beast;
namespace http = beast::http;
using net::ip::tcp;
void server()
{
net::io_context ioc;
tcp::acceptor acc{ioc, {{}, 8989}};
acc.listen();
auto conn = acc.accept();
http::request<http::string_body> msg(
http::verb::get, "/", 11, std::string(20ull << 10, '*'));
msg.prepare_payload();
http::request_serializer<http::string_body> ser(msg);
size_t hbytes = write_header(conn, ser);
// size_t bbytes = write_some(conn, ser);
size_t bbytes = write(conn, net::buffer(msg.body(), 1024));
std::cout << "sent " << hbytes << " header and " << bbytes << "/"
<< msg.body().length() << " of body" << std::endl;
// closes connection
}
namespace {
template<bool isRequest, class SyncReadStream, class DynamicBuffer>
auto
read_and_print_body(
std::ostream& /*os*/,
SyncReadStream& stream,
DynamicBuffer& buffer,
boost::beast::error_code& ec)
{
struct { size_t hbytes = 0, bbytes = 0; } ret;
http::parser<isRequest, http::buffer_body> p;
//p.header_limit(8192);
//p.body_limit(1024);
ret.hbytes = read_header(stream, buffer, p, ec);
if(ec)
return ret;
while(! p.is_done())
{
char buf[512];
p.get().body().data = buf;
p.get().body().size = sizeof(buf);
ret.bbytes += http::read_some(stream, buffer, p, ec);
if(ec == http::error::need_buffer)
ec = {};
if(ec)
break;
//os.write(buf, sizeof(buf) - p.get().body().size);
}
return ret;
}
}
void client()
{
net::io_context ioc;
tcp::socket conn{ioc};
conn.connect({{}, 8989});
beast::error_code ec;
beast::flat_buffer buf;
auto [hbytes, bbytes] = read_and_print_body<true>(std::cout, conn, buf, ec);
std::cout << "received hbytes:" << hbytes << " bbytes:" << bbytes
<< " (" << ec.message() << ")" << std::endl;
}
int main()
{
std::jthread s(server);
std::this_thread::sleep_for(std::chrono::seconds(1));
std::jthread c(client);
}
Prints
sent 41 header and 1024/20480 of body
received 1065 bytes of message (partial message)
Side Notes
You start your question with:
I am trying to understand how can I limit the amount of data that is read from the internet
That's built in to Beast
by calling 'read_some' function in boost beast.
To just limit the total amount of data read, you don't have to use read_some in a loop (http::read by definition already does exactly that).
E.g. with the above example, if you replace 20ull<<10 (20 KiB) with 20ull<<20 (20 MiB) you will exceed the default size limit:
http::request<http::string_body> msg(http::verb::get, "/", 11,
std::string(20ull << 20, '*'));
Prints Live On Coliru
sent 44 header and 1024/20971520 of body
received hbytes:44 bbytes:0 (body limit exceeded)
You can also set your own parser limits:
http::parser<isRequest, http::buffer_body> p;
p.header_limit(8192);
p.body_limit(1024);
Which prints Live On Coliru:
sent 41 header and 1024/20480 of body
received hbytes:41 bbytes:0 (body limit exceeded)
As you can see it even knows to reject the request after just reading the headers, using the content-length information from the headers.

c++ Sending struct over network

I'm working with Intel SGX which has predefined structures. I need to send these structures over a network connection which is operated by using boost::asio.
The structure that needs to be send has the following format:
typedef struct _ra_samp_request_header_t{
uint8_t type; /* set to one of ra_msg_type_t*/
uint32_t size; /*size of request body*/
uint8_t align[3];
uint8_t body[];
} ra_samp_request_header_t;
For the sending and receiving, the methods async_write and async_async_read_some are used
boost::asio::async_write(socket_, boost::asio::buffer(data_, max_length),
boost::bind(&Session::handle_write, this,
boost::asio::placeholders::error));
socket_.async_read_some(boost::asio::buffer(data_, max_length),
boost::bind(&Session::handle_read, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
whereas data_ is defined as
enum { max_length = 1024 };
char data_[max_length];
My first approach was to transform the single structure elements into strings and store them in a vector<string> which is then further transformed into char* whereas each element is separated by \n.
But when assembling the received char* on the receiver side back to the original structure I run into some troubles.
Is this really the way this should be done or is there a nicer more sufficient way of transfering the structure
Do you need it to be portable?
If not:
simplistic approach
using Boost Serialization
If it needs to be portable
complicate the simplistic approach with ntohl and htonl calls etc.
use Boost Serialization with EOS Portable Archives
1. simplistic approach
Just send the struct as POD data (assuming it is actually POD, which given the code in your question is a fair assumption as the struct is clearly not C++).
A simple sample that uses synchronous calls on 2 threads (listener and client) shows how the server sends a packet to the client which the client receives correctly.
Notes:
using async calls is a trivial change (change write and read into async_write and async_write, which just makes control flow a bit less legible unless using coroutines)
I showed how I'd use malloc/free in a (exceptio) safe manner in C++11. You may want to make a simple Rule-Of-Zero wrapper instead in your codebase.
Live On Coliru
#include <boost/asio.hpp>
#include <cstring>
namespace ba = boost::asio;
using ba::ip::tcp;
typedef struct _ra_samp_request_header_t{
uint8_t type; /* set to one of ra_msg_type_t*/
uint32_t size; /*size of request body*/
uint8_t align[3];
uint8_t body[];
} ra_samp_request_header_t;
#include <iostream>
#include <thread>
#include <memory>
int main() {
auto unique_ra_header = [](uint32_t body_size) {
static_assert(std::is_pod<ra_samp_request_header_t>(), "not pod");
auto* raw = static_cast<ra_samp_request_header_t*>(::malloc(sizeof(ra_samp_request_header_t)+body_size));
new (raw) ra_samp_request_header_t { 2, body_size, {0} };
return std::unique_ptr<ra_samp_request_header_t, decltype(&std::free)>(raw, std::free);
};
auto const& body = "There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which looks reasonable.";
auto sample = unique_ra_header(sizeof(body));
std::strncpy(reinterpret_cast<char*>(+sample->body), body, sizeof(body));
ba::io_service svc;
ra_samp_request_header_t const& packet = *sample;
auto listener = std::thread([&] {
try {
tcp::acceptor a(svc, tcp::endpoint { {}, 6767 });
tcp::socket s(svc);
a.accept(s);
std::cout << "listener: Accepted: " << s.remote_endpoint() << "\n";
auto written = ba::write(s, ba::buffer(&packet, sizeof(packet) + packet.size));
std::cout << "listener: Written: " << written << "\n";
} catch(std::exception const& e) {
std::cerr << "listener: " << e.what() << "\n";
}
});
std::this_thread::sleep_for(std::chrono::milliseconds(10)); // make sure listener is ready
auto client = std::thread([&] {
try {
tcp::socket s(svc);
s.connect(tcp::endpoint { {}, 6767 });
// this is to avoid the output to get intermingled, only
std::this_thread::sleep_for(std::chrono::milliseconds(200));
std::cout << "client: Connected: " << s.remote_endpoint() << "\n";
enum { max_length = 1024 };
auto packet_p = unique_ra_header(max_length); // slight over allocation for simplicity
boost::system::error_code ec;
auto received = ba::read(s, ba::buffer(packet_p.get(), max_length), ec);
// we expect only eof since the message received is likely not max_length
if (ec != ba::error::eof) ba::detail::throw_error(ec);
std::cout << "client: Received: " << received << "\n";
(std::cout << "client: Payload: ").write(reinterpret_cast<char const*>(packet_p->body), packet_p->size) << "\n";
} catch(std::exception const& e) {
std::cerr << "client: " << e.what() << "\n";
}
});
client.join();
listener.join();
}
Prints
g++ -std=gnu++11 -Os -Wall -pedantic main.cpp -pthread -lboost_system && ./a.out
listener: Accepted: 127.0.0.1:42914
listener: Written: 645
client: Connected: 127.0.0.1:6767
client: Received: 645
client: Payload: There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which looks reasonable.
1b. simplistic with wrapper
Because for Boost Serialization it would be convenient to have such a wrapper anyways, let's rewrite that using such a "Rule Of Zero" wrapper:
Live On Coliru
namespace mywrappers {
struct ra_samp_request_header {
enum { max_length = 1024 };
// Rule Of Zero - https://rmf.io/cxx11/rule-of-zero
ra_samp_request_header(uint32_t body_size = max_length) : _p(create(body_size)) {}
::ra_samp_request_header_t const& get() const { return *_p; };
::ra_samp_request_header_t& get() { return *_p; };
private:
static_assert(std::is_pod<::ra_samp_request_header_t>(), "not pod");
using Ptr = std::unique_ptr<::ra_samp_request_header_t, decltype(&std::free)>;
Ptr _p;
static Ptr create(uint32_t body_size) {
auto* raw = static_cast<::ra_samp_request_header_t*>(::malloc(sizeof(::ra_samp_request_header_t)+body_size));
new (raw) ::ra_samp_request_header_t { 2, body_size, {0} };
return Ptr(raw, std::free);
};
};
}
2. using Boost Serialization
Without much ado, here's a simplistic way to implement serialization in-class for that wrapper:
friend class boost::serialization::access;
template<typename Ar>
void save(Ar& ar, unsigned /*version*/) const {
ar & _p->type
& _p->size
& boost::serialization::make_array(_p->body, _p->size);
}
template<typename Ar>
void load(Ar& ar, unsigned /*version*/) {
uint8_t type = 0;
uint32_t size = 0;
ar & type & size;
auto tmp = create(size);
*tmp = ::ra_samp_request_header_t { type, size, {0} };
ar & boost::serialization::make_array(tmp->body, tmp->size);
// if no exceptions, swap it out
_p = std::move(tmp);
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
Which then simplifies the test driver to this - using streambuf:
auto listener = std::thread([&] {
try {
tcp::acceptor a(svc, tcp::endpoint { {}, 6767 });
tcp::socket s(svc);
a.accept(s);
std::cout << "listener: Accepted: " << s.remote_endpoint() << "\n";
ba::streambuf sb;
{
std::ostream os(&sb);
boost::archive::binary_oarchive oa(os);
oa << sample;
}
auto written = ba::write(s, sb);
std::cout << "listener: Written: " << written << "\n";
} catch(std::exception const& e) {
std::cerr << "listener: " << e.what() << "\n";
}
});
std::this_thread::sleep_for(std::chrono::milliseconds(10)); // make sure listener is ready
auto client = std::thread([&] {
try {
tcp::socket s(svc);
s.connect(tcp::endpoint { {}, 6767 });
// this is to avoid the output to get intermingled, only
std::this_thread::sleep_for(std::chrono::milliseconds(200));
std::cout << "client: Connected: " << s.remote_endpoint() << "\n";
mywrappers::ra_samp_request_header packet;
boost::system::error_code ec;
ba::streambuf sb;
auto received = ba::read(s, sb, ec);
// we expect only eof since the message received is likely not max_length
if (ec != ba::error::eof) ba::detail::throw_error(ec);
std::cout << "client: Received: " << received << "\n";
{
std::istream is(&sb);
boost::archive::binary_iarchive ia(is);
ia >> packet;
}
(std::cout << "client: Payload: ").write(reinterpret_cast<char const*>(packet.get().body), packet.get().size) << "\n";
} catch(std::exception const& e) {
std::cerr << "client: " << e.what() << "\n";
}
});
All other code is unchanged from the above, see it Live On Coliru. Output unchanged, except packet sizes grew to 683 on my 64-bit machine using Boost 1.62.
3. complicate the simplistic approach
I'm not in the mood to demo this. It feels like being a C programmer instead of a C++ programmer. Of course there are clever ways to avoid writing the endian-ness twiddling etc. For a modern approach see e.g.
I can't find the sample talk/blog post about using Fusion to generate struct portable serialization for your classes (might add later)
https://www.youtube.com/watch?v=zvfPK4ot9uA
4. use EAS Portable Archive
Is a simple drop-in exercise using the code of 3.

Boost not working over network

I've made a simple module for transferring data. Most of my code is based on code I found on SO
When I run the instances on my local machine, everything works fine. But if I try running this over a local area network, I get the error "Can't Assign Requested Address".
Note: My "instances basically involve running ./server 1 0 and ./server 1 1 so, they;re waiting for data. And then ./server 0 to send it over.
Here's the code
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <iostream>
#include <boost/serialization/string.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/asio.hpp>
static std::string const server_ip = "10.0.0.4";
static std::string const client_ip = "10.0.0.5";
using std::cout;
using std::endl;
struct Location {
double rent[6];
double cost[7];
std::string name, group;
std::string locationOfObjectFile;
int locationNo;
template <typename Ar> void serialize(Ar &ar, unsigned) {
ar &rent;
ar &cost;
ar &name;
ar &group;
ar &locationOfObjectFile;
ar &locationNo;
}
};
struct Player {
int currentPosition;
double currentMoney;
template <typename Ar> void serialize(Ar &ar, unsigned) {
ar &currentPosition;
ar &currentMoney;
}
};
struct Monopoly {
std::vector<Location> locations;
std::vector<Player> players;
std::string currency;
template <typename Ar> void serialize(Ar &ar, unsigned) {
ar &locations;
ar &players;
ar &currency;
}
};
Location l1;
Player p1;
Monopoly game, game2;
void readData(int x) {
boost::asio::io_service io_service;
uint16_t port = x;
boost::asio::ip::tcp::acceptor acceptor(
io_service, boost::asio::ip::tcp::endpoint(boost::asio::ip::address::from_string(server_ip), port));
/* code */
boost::asio::ip::tcp::socket socket(io_service);
acceptor.accept(socket);
std::cout << "connection from " << socket.remote_endpoint() << std::endl;
// read header
size_t header;
boost::asio::read(socket, boost::asio::buffer(&header, sizeof(header)));
std::cout << "body is " << header << " bytes" << std::endl;
// read body
boost::asio::streambuf buf;
const size_t rc = boost::asio::read(socket, buf.prepare(header));
buf.commit(header);
std::cout << "read " << rc << " bytes" << std::endl;
// deserialize
std::istream is(&buf);
boost::archive::text_iarchive ar(is);
ar &game2;
cout << game2.locations[0].rent[1] << endl;
cout << game2.players[0].currentPosition << "how cool is this?";
socket.close();
}
void sendData() {
for (int i = 0; i <= 1; i++) {
boost::asio::streambuf buf;
std::ostream os(&buf);
boost::archive::text_oarchive ar(os);
ar &game;
boost::asio::io_service io_service;
boost::asio::ip::tcp::socket socket(io_service);
short port = i + 1234;
socket.connect(boost::asio::ip::tcp::endpoint(boost::asio::ip::address::from_string(client_ip), port));
const size_t header = buf.size();
std::cout << "buffer size " << header << " bytes" << std::endl;
// send header and buffer using scatter
std::vector<boost::asio::const_buffer> buffers;
buffers.push_back(boost::asio::buffer(&header, sizeof(header)));
buffers.push_back(buf.data());
const size_t rc = boost::asio::write(socket, buffers);
std::cout << "wrote " << rc << " bytes" << std::endl;
;
socket.close();
}
}
int main(int argc, char **argv) {
l1.name = "soemthig";
l1.group = 2;
p1.currentMoney = 300;
p1.currentPosition = 422;
for (int i = 0; i < 7; ++i) {
l1.cost[i] = i;
/* code */
}
for (int i = 0; i < 6; ++i) {
l1.rent[i] = 2 * i;
/* code */
}
l1.locationOfObjectFile = "ajhsdk/asdc.obj";
l1.locationNo = 5;
game.locations.push_back(l1);
game.players.push_back(p1);
game.currency = "dollar";
cout << game.currency;
if (atoi(argv[1]) ==
1) // argv[0]=0 implies server, argv[0]==1 implies client while argv[1] specifies 1st or second client
{
cout << "reading data";
if (atoi(argv[2]) == 0) {
readData(1234);
/* code */
} else {
readData(1235);
}
} else {
cout << "writing data";
sendData();
}
}
Here is the error message stack trace:
libc++abi.dylib: terminating with uncaught exception of type boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector‌​<boost::system::system_error> >: bind: Can't assign requested address dollarreading dataAbort trap: 6 and 10.0.0.4 is the IP address of the computer that is supposed to send the data.
You get this when the connection is refused.
This, in turn, could happen when the client is not listening on the required interface.
Make sure that the IP addresses are actually the public IP addresses on the network and the machines can reach eachother. E.g. use netstat -tlpn (or similar on your OS) to ascertain that the clients are listening:
tcp 0 0 192.168.2.136:1234 0.0.0.0:* LISTEN 18186/valgrind.bin
tcp 0 0 192.168.2.136:1235 0.0.0.0:* LISTEN 18187/valgrind.bin
Now, try to connect using e.g. netcat from the server machine:
netcat 192.168.2.136 1234
This will probably crash the client, but it will also tell you whether connecting was possible.
If not, then either the address is not reachable, the client is not listening on the right interface, the firewall is filtering your traffic etc.
PS. I've made your code self contained and running live on Coliru. Please do this; making your code selfcontained make it possible for people to actually fix things

How to create a boost ssl iostream?

I'm adding HTTPS support to code that does input and output using boost tcp::iostream (acting as an HTTP server).
I've found examples (and have a working toy HTTPS server) that do SSL input/output using boost::asio::read/boost::asio::write, but none that use iostreams and the << >> operators. How do I turn an ssl::stream into an iostream?
Working code:
#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
#include <boost/foreach.hpp>
#include <iostream>
#include <sstream>
#include <string>
using namespace std;
using namespace boost;
using boost::asio::ip::tcp;
typedef boost::asio::ssl::stream<boost::asio::ip::tcp::socket> ssl_stream;
string HTTPReply(int nStatus, const string& strMsg)
{
string strStatus;
if (nStatus == 200) strStatus = "OK";
else if (nStatus == 400) strStatus = "Bad Request";
else if (nStatus == 404) strStatus = "Not Found";
else if (nStatus == 500) strStatus = "Internal Server Error";
ostringstream s;
s << "HTTP/1.1 " << nStatus << " " << strStatus << "\r\n"
<< "Connection: close\r\n"
<< "Content-Length: " << strMsg.size() << "\r\n"
<< "Content-Type: application/json\r\n"
<< "Date: Sat, 09 Jul 2009 12:04:08 GMT\r\n"
<< "Server: json-rpc/1.0\r\n"
<< "\r\n"
<< strMsg;
return s.str();
}
int main()
{
// Bind to loopback 127.0.0.1 so the socket can only be accessed locally
boost::asio::io_service io_service;
tcp::endpoint endpoint(boost::asio::ip::address_v4::loopback(), 1111);
tcp::acceptor acceptor(io_service, endpoint);
boost::asio::ssl::context context(io_service, boost::asio::ssl::context::sslv23);
context.set_options(
boost::asio::ssl::context::default_workarounds
| boost::asio::ssl::context::no_sslv2);
context.use_certificate_chain_file("server.cert");
context.use_private_key_file("server.pem", boost::asio::ssl::context::pem);
for(;;)
{
// Accept connection
ssl_stream stream(io_service, context);
tcp::endpoint peer_endpoint;
acceptor.accept(stream.lowest_layer(), peer_endpoint);
boost::system::error_code ec;
stream.handshake(boost::asio::ssl::stream_base::server, ec);
if (!ec) {
boost::asio::write(stream, boost::asio::buffer(HTTPReply(200, "Okely-Dokely\n")));
// I really want to write:
// iostream_object << HTTPReply(200, "Okely-Dokely\n") << std::flush;
}
}
}
It seems like the ssl::stream_service would be the answer, but that is a dead end.
Using boost::iostreams (as suggested by accepted answer) is the right approach; here's the working code I've ended up with:
#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
#include <boost/iostreams/concepts.hpp>
#include <boost/iostreams/stream.hpp>
#include <sstream>
#include <string>
#include <iostream>
using namespace boost::asio;
typedef ssl::stream<ip::tcp::socket> ssl_stream;
//
// IOStream device that speaks SSL but can also speak non-SSL
//
class ssl_iostream_device : public boost::iostreams::device<boost::iostreams::bidirectional> {
public:
ssl_iostream_device(ssl_stream &_stream, bool _use_ssl ) : stream(_stream)
{
use_ssl = _use_ssl;
need_handshake = _use_ssl;
}
void handshake(ssl::stream_base::handshake_type role)
{
if (!need_handshake) return;
need_handshake = false;
stream.handshake(role);
}
std::streamsize read(char* s, std::streamsize n)
{
handshake(ssl::stream_base::server); // HTTPS servers read first
if (use_ssl) return stream.read_some(boost::asio::buffer(s, n));
return stream.next_layer().read_some(boost::asio::buffer(s, n));
}
std::streamsize write(const char* s, std::streamsize n)
{
handshake(ssl::stream_base::client); // HTTPS clients write first
if (use_ssl) return boost::asio::write(stream, boost::asio::buffer(s, n));
return boost::asio::write(stream.next_layer(), boost::asio::buffer(s, n));
}
private:
bool need_handshake;
bool use_ssl;
ssl_stream& stream;
};
std::string HTTPReply(int nStatus, const std::string& strMsg)
{
std::string strStatus;
if (nStatus == 200) strStatus = "OK";
else if (nStatus == 400) strStatus = "Bad Request";
else if (nStatus == 404) strStatus = "Not Found";
else if (nStatus == 500) strStatus = "Internal Server Error";
std::ostringstream s;
s << "HTTP/1.1 " << nStatus << " " << strStatus << "\r\n"
<< "Connection: close\r\n"
<< "Content-Length: " << strMsg.size() << "\r\n"
<< "Content-Type: application/json\r\n"
<< "Date: Sat, 09 Jul 2009 12:04:08 GMT\r\n"
<< "Server: json-rpc/1.0\r\n"
<< "\r\n"
<< strMsg;
return s.str();
}
void handle_request(std::iostream& s)
{
s << HTTPReply(200, "Okely-Dokely\n") << std::flush;
}
int main(int argc, char* argv[])
{
bool use_ssl = (argc <= 1);
// Bind to loopback 127.0.0.1 so the socket can only be accessed locally
io_service io_service;
ip::tcp::endpoint endpoint(ip::address_v4::loopback(), 1111);
ip::tcp::acceptor acceptor(io_service, endpoint);
ssl::context context(io_service, ssl::context::sslv23);
context.set_options(
ssl::context::default_workarounds
| ssl::context::no_sslv2);
context.use_certificate_chain_file("server.cert");
context.use_private_key_file("server.pem", ssl::context::pem);
for(;;)
{
ip::tcp::endpoint peer_endpoint;
ssl_stream _ssl_stream(io_service, context);
ssl_iostream_device d(_ssl_stream, use_ssl);
boost::iostreams::stream<ssl_iostream_device> ssl_iostream(d);
// Accept connection
acceptor.accept(_ssl_stream.lowest_layer(), peer_endpoint);
std::string method;
std::string path;
ssl_iostream >> method >> path;
handle_request(ssl_iostream);
}
}
#Guy's suggestion (using boost::asio::streambuf) should work, and it's probably the easiest to implement. The main drawback to that approach is that everything you write to the iostream will be buffered in memory until the end, when the call to boost::asio::write() will dump the entire contents of the buffer onto the ssl stream at once. (I should note that this kind of buffering can actually be desirable in many cases, and in your case it probably makes no difference at all since you've said it's a low-volume application).
If this is just a "one-off" I would probably implement it using #Guy's approach.
That being said -- there are a number of good reasons that you might rather have a solution that allows you to use iostream calls to write directly into your ssl_stream. If you find that this is the case, then you'll need to build your own wrapper class that extends std::streambuf, overriding overflow(), and sync() (and maybe others depending on your needs).
Fortunately, boost::iostreams provides a relatively easy way to do this without having to mess around with the std classes directly. You just build your own class that implements the appropriate Device contract. In this case that's Sink, and the boost::iostreams::sink class is provided as a convenient way to get most of the way there. Once you have a new Sink class that encapsulates the process of writing to your underlying ssl_stream, all you have to do is create a boost::iostreams::stream that is templated to your new device type, and off you go.
It will look something like the following (this example is adapted from here, see also this related stackoverflow post):
//---this should be considered to be "pseudo-code",
//---it has not been tested, and probably won't even compile
//---
#include <boost/iostreams/concepts.hpp>
// other includes omitted for brevity ...
typedef boost::asio::ssl::stream<boost::asio::ip::tcp::socket> ssl_stream;
class ssl_iostream_sink : public sink {
public:
ssl_iostream_sink( ssl_stream *theStream )
{
stream = theStream;
}
std::streamsize write(const char* s, std::streamsize n)
{
// Write up to n characters to the underlying
// data sink into the buffer s, returning the
// number of characters written
boost::asio::write(*stream, boost::asio::buffer(s, n));
}
private:
ssl_stream *stream;
};
Now, your accept loop might change to look something like this:
for(;;)
{
// Accept connection
ssl_stream stream(io_service, context);
tcp::endpoint peer_endpoint;
acceptor.accept(stream.lowest_layer(), peer_endpoint);
boost::system::error_code ec;
stream.handshake(boost::asio::ssl::stream_base::server, ec);
if (!ec) {
// wrap the ssl stream with iostream
ssl_iostream_sink my_sink(&stream);
boost::iostream::stream<ssl_iostream_sink> iostream_object(my_sink);
// Now it works the way you want...
iostream_object << HTTPReply(200, "Okely-Dokely\n") << std::flush;
}
}
That approach hooks the ssl stream into the iostream framework. So now you should be able to do anything to iostream_object in the above example, that you would normally do with any other std::ostream (like stdout). And the stuff that you write to it will get written into the ssl_stream behind the scenes. Iostreams has built-in buffering, so some degree of buffering will take place internally -- but this is a good thing -- it will buffer until it has accumulated some reasonable amount of data, then it will dump it on the ssl stream, and go back to buffering. The final std::flush, should force it to empty the buffer out to the ssl_stream.
If you need more control over internal buffering (or any other advanced stuff), have a look at the other cool stuff available in boost::iostreams. Specifically, you might start by looking at stream_buffer.
Good luck!
I think what you want to do is use stream buffers (asio::streambuf)
Then you can do something like (untested code written on the fly follows):
boost::asio::streambuf msg;
std::ostream msg_stream(&msg);
msg_stream << "hello world";
msg_stream.flush();
boost::asio::write(stream, msg);
Similarly your read/receive side can read into a stream buffer in conjunction with std::istream so you can process your input using various stream functions/operators.
Asio reference for streambuf
Another note is I think you should check out the asio tutorials/examples. Once you do you'll probably want to change your code to work asynchronously rather than the synchronous example you're showing above.
ssl::stream could be wrapped with boost::iostreams / bidirectional to mimic similar behaviours as tcp::iostream. flushing output before further reading seems cannot be avoided.
#include <regex>
#include <string>
#include <iostream>
#include <boost/iostreams/stream.hpp>
#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
namespace bios = boost::iostreams;
namespace asio = boost::asio;
namespace ssl = boost::asio::ssl;
using std::string;
using boost::asio::ip::tcp;
using boost::system::system_error;
using boost::system::error_code;
int parse_url(const std::string &s,
std::string& proto, std::string& host, std::string& path)
{
std::smatch m;
bool found = regex_search(s, m, std::regex("^(http[s]?)://([^/]*)(.*)$"));
if (m.size() != 4)
return -1;
proto = m[1].str();
host = m[2].str();
path = m[3].str();
return 0;
}
void get_page(std::iostream& s, const string& host, const string& path)
{
s << "GET " << path << " HTTP/1.0\r\n"
<< "Host: " << host << "\r\n"
<< "Accept: */*\r\n"
<< "Connection: close\r\n\r\n" << std::flush;
std::cout << s.rdbuf() << std::endl;;
}
typedef ssl::stream<tcp::socket> ssl_socket;
class ssl_wrapper : public bios::device<bios::bidirectional>
{
ssl_socket& sock;
public:
typedef char char_type;
ssl_wrapper(ssl_socket& sock) : sock(sock) {}
std::streamsize read(char_type* s, std::streamsize n) {
error_code ec;
auto rc = asio::read(sock, asio::buffer(s,n), ec);
return rc;
}
std::streamsize write(const char_type* s, std::streamsize n) {
return asio::write(sock, asio::buffer(s,n));
}
};
int main(int argc, char* argv[])
{
std::string proto, host, path;
if (argc!= 2 || parse_url(argv[1], proto, host, path)!=0)
return EXIT_FAILURE;
try {
if (proto != "https") {
tcp::iostream s(host, proto);
s.expires_from_now(boost::posix_time::seconds(60));
get_page(s, host, path);
} else {
asio::io_service ios;
tcp::resolver resolver(ios);
tcp::resolver::query query(host, "https");
tcp::resolver::iterator endpoint_iterator =
resolver.resolve(query);
ssl::context ctx(ssl::context::sslv23);
ctx.set_default_verify_paths();
ssl_socket socket(ios, ctx);
asio::connect(socket.lowest_layer(), endpoint_iterator);
socket.set_verify_mode(ssl::verify_none);
socket.set_verify_callback(ssl::rfc2818_verification(host));
socket.handshake(ssl_socket::client);
bios::stream<ssl_wrapper> ss(socket);
get_page(ss, host, path);
}
} catch (const std::exception& e) {
std::cout << "Exception: " << e.what() << "\n";
}
}