Continuously streaming data with Boost::ASIO without closing socket - c++

I'm trying to build an application that:
a) Runs an FDM (flight dynamics model) internally, and manages the flight data
b) Accepts connections on a TCP socket
c) Serves the flight data over said socket.
I've currently managed to get a simple string sent over TCP on my a local socket using the examples/tutorials on the Boos::ASIO website here: http://www.boost.org/doc/libs/1_56_0_b1/doc/html/boost_asio/tutorial.html
My trouble is simply that the connection closes after a single string is written, and I don't know how to keep the connection open and continuously send the data until the simulation is finished (or a stop signal is sent from the listening application).
I also have the FDM working (currently using JSBSim, and borrowing heavily from their included sample code), and can print flight data to stdout no problem. The Boost:ASIO documentation shows a few examples on how to build a server that constantly listens for messages received from the client, but none that send a stream of data out.

To send a stream of data out, you can use the free functions with boost::asio::streambuf.
Here's a simple demo that sends its own source to each client:
#include <boost/asio.hpp>
#include <boost/make_shared.hpp>
#include <boost/function.hpp>
#include <boost/asio/posix/stream_descriptor.hpp>
#include <fstream>
#include <iostream>
namespace io = boost::asio;
namespace ip = io::ip;
using boost::system::error_code;
using boost::make_shared;
using ip::tcp;
void start_accept(io::io_service& svc, tcp::acceptor& acc) {
// per-connection lifetimes:
auto sock = make_shared<tcp::socket>(svc);
acc.async_accept(*sock, [sock,&svc,&acc](error_code ec) {
if (!ec)
{
std::cout << "connection from " << sock->remote_endpoint() << "\n";
// copy source file to buffer data
auto data = make_shared<io::streambuf>();
std::ostream(data.get()) << std::ifstream("main.cpp").rdbuf();
// now write the whole story
io::async_write(*sock, *data, [sock,data/*keep alive*/](error_code ec, size_t transferred){});
// accept new connections too
start_accept(svc, acc);
}
});
}
int main()
{
io::io_service svc;
tcp::acceptor acc(svc, tcp::endpoint(ip::address(), 6767));
start_accept(svc, acc);
svc.run();
}
Note that for simplicity I put the full buffer in memory first, assuming you can do that (you said "and can print flight data to stdout no problem"). So, you could just write different things to the stream (the ostream line in my example).

Related

C++, Poco Socketstream to Websocket for JSON packet transfer

I'm working on a project that is relying fairly heavily on packet latency. The architect for the overall system wants to use the Poco::Net::Websocket protocol as the transport layer between the different nodes. We have been running the application with ZMQ till this point, but there's too much overhead and we are not seeing the speeds we need. So I'm tasked with converting the ZMQ portion of the system over to WebSockets.
I've never used WebSockets before and I'm having issues understanding what's going on. I know WebSockets were originally designed as a way to speed up communication between the web browser and the page server decreasing latency through the use of full-duplex bi-directional communication.
There are a lot of tutorials for javascript, nodejs, and even python, but it's more limited when you're talking about C++, which is what I'm working with. The examples that I have found are geared more towards browser requests, which I'm not surprised by. Rather than general TCP packet transfer, which is what our use case would be.
Here are my thoughts and questions.
Seeing how I couldn't really understand what was happening, I decided to start with a Poco::Net::SocketStream configuration first. I chose to do it this way, based on the thought that Websockets are nothing more than a TCP socket inserted into a higher-level protocol framework. I figured transitioning would be easier than trying from scratch. So I spun up a server and client application using standard Poco socket libraries.
I used the example here as a base. I just ended up moving everything to a single file. I also spun up a Poco::Net::TCPServer as well, a sample found here.
I modified both to manage the fact that I'm sending JSON packets between the client and server. I took snippets of our original code to format the JSON packets and encode them with CBOR. I ended up keeping the zmq::message_t format that I was originally using, as I ran into issues receiving the packets on the other end. So as it wasn't the major issue I just skipped that part for now. So I'm aware the ZMQ portions of the code below aren't necessary, a small problem for another time.
Here is the Client code for reference.
#include "Poco/Net/HTTPRequest.h"
#include "Poco/Net/HTTPResponse.h"
#include "Poco/Net/HTTPMessage.h"
#include "Poco/Net/WebSocket.h"
#include "Poco/Net/HTTPClientSession.h"
#include <iostream>
#include "Poco/Net/SocketAddress.h"
#include "Poco/Net/StreamSocket.h"
#include "Poco/Net/SocketStream.h"
#include "Poco/Net/WebSocket.h"
#include <jsoncons/json.hpp>
#include <jsoncons/json.hpp>
#include <jsoncons_ext/cbor/cbor.hpp>
#include <jsoncons_ext/jsonpath/json_query.hpp>
#include <jsoncons_ext/jsonpath/jsonpath.hpp>
#include <iostream>
#include <string>
#include <zmq.hpp>
using namespace jsoncons;
using Poco::Net::HTTPClientSession;
using Poco::Net::HTTPMessage;
using Poco::Net::HTTPRequest;
using Poco::Net::HTTPResponse;
using Poco::Net::WebSocket;
class ClientHandler
{
private:
std::string host;
int port;
// IP endpoint/socket address (consists of host addr and port #)
Poco::Net::SocketAddress socketAddr;
// Interface to a TCP stream socket
// Poco::Net::StreamSocket socket;
Poco::Net::Socket socket;
// Stream for reading from / writing to a socket (accepts a socket)
Poco::Net::SocketStream stream;
public:
ClientHandler(std::string h, int p) : host(h), port(p), socketAddr(h, p), socket(), stream(socket)
{
std::cout << "Host: " << host << "\tPort: " << port << std::endl;
}
// Connect to the initialized socket address' hostname and port
bool connected()
{
std::cout << "Creating a connection with ["
<< socketAddr.host().toString()
<< "] through port [" << socketAddr.port() << "] ...";
try
{
// socket.connect(socketAddr);
// std::cout << "Success!" << std::endl;
}
catch (Poco::Exception err)
{
std::cout << std::endl;
std::cout << "Socket connection error: [" << err.displayText() << "]" << std::endl;
return false;
}
return true;
}
// Send a message to the connected server
bool sendMessage()
{
std::cout << std::endl;
const char channel[] = "fool";
std::string channelPort = std::to_string(5577);
try
{
std::string message;
std::cout << "Enter a message to send to the server: ";
std::cin >> message;
json msg = json(json_object_arg, {{"$op", "create_channel"}, {"$svc", "zmq"}, {"channel_name", channel}, {"port", channelPort}});
std::vector<uint8_t> buffer;
jsoncons::cbor::encode_cbor(msg, buffer);
zmq::const_buffer msg_out = zmq::buffer(buffer);
if (message.compare("exit") != 0)
{
std::cout << "Sending the message to the server!\n\t";
// socket.sendBytes(msg_out.data(), msg_out.size());
return true;
}
else
return false;
}
catch (Poco::Exception err)
{
std::cout << "Data send error: [" << err.displayText() << "]" << std::endl;
return false;
}
}
};
int main(int argc, char **argv)
{
int port = 2001;
std::string hostname = "10.0.12.97";
// Handle the server-client connection and send some JSON
try
{
ClientHandler handler(hostname, port);
if (handler.connected())
while (handler.sendMessage())
;
}
catch (Poco::Exception err)
{
std::cout << "Handler Error -> " << err.displayText() << std::endl;
}
return 0;
}
So, How do I get from this ^ to Websockets?
I have been reading and digging through the documentation for Poco Websockets. At first, I thought I needed to figure out the HTTPClientSession, HTTPRequest, and HTTPResponse objects I saw in so many examples. However, looking at Documentation for Poco Websockets constructors, it appeared all I needed to do was pass it a socket connection. So spent some time trying to figure that out, but after going back to the documentation it appears that it specifies that I need to send it a socket that is already a Websocket.
Creates a WebSocket from another Socket, which must be a WebSocket,
That's what I get for not actually reading it. So I spent some more time reading through the constructors and it appears that I do in fact have to send at least 3 arguments.
WebSocket
WebSocket(
HTTPClientSession & cs,
HTTPRequest & request,
HTTPResponse & response
);
Creates a client-side WebSocket, using the given HTTPClientSession and HTTPRequest for the initial handshake (HTTP Upgrade request).
So I'm now back to HTTPClientSession, HTTPRequest, and HTTPResponse objects.
I think I understand HTTPClientSession. Here I should provide the Server name and port. So updating the code above should look something like this
...
Poco::Net::HTTPClientSession cs;
public:
ClientHandler(std::string h, int p) : host(h), port(p), cs(h, p)
...
HTTPResponse looks to be a pointer to a response object for the WebSocket to return the HTTPRequest response. Looks to contain a status code, date, reason(not sure what reason is)
The HTTPRequest is a point of interest. I'm lost as to what I need to format this as. There are three arguments for the request.
Method - this looks like nothing more than telling the server that you are requesting something or looking to hand something over.
HTTP_POST - I would assume this is what I want?
HTTP_GET
URI - for HTTP_GET this would be the path of the resource I'm looking to retrieve. And for Post, this would be the path that I would like to post something to on the server
Version - looks to be self-explanatory, as this is the HTTP version I would like to use.
but here is the rub of it, I do not want to post/get anything that's associated with a URI, I'm just looking to transfer a JSON packet and manage it on the other end. I've found a sample code of a client that uses this information, but I don't understand why it's using HTTP_GET, and the reason for the /?encode=text reference. I can't seem to find details anywhere on it. That leads me to the set value of the request object, don't understand what the "set()" call is for or how it would work in my situation.
With the new information, I updated my constructor to this.
...
Poco::Net::WebSocket *ws;
public:
ClientHandler(std::string h, int p) : host(h), port(p)
{
std::cout << "TEST" << std::endl;
// Poco::Net::HTTPClientSession cs(h, p);
Poco::Net::HTTPClientSession cs(host, port);
Poco::Net::HTTPRequest req(HTTPRequest::HTTP_GET, "/?encoding=text", HTTPMessage::HTTP_1_1);
// req.set("origin", "http://www.websocket.org");
Poco::Net::HTTPResponse resp;
ws = new WebSocket(cs, req, resp);
std::cout << "Host: " << host << "\tPort: " << port << std::endl;
}
...
However, I enter the constructor but never leave it. It gets hung up when I attempt to create the WebSocket. I'm assuming it doesn't like something. I tried the HTTPClientSesseion cs two ways. and leaving the req.set() in as well. Just hangs.
Could someone lend me some insight as to how I can get Poco::Net::Websocket running? and some information so I can understand what's going on a bit better. So when I attempt the server-side I will have some more information that will help.

Asio Peer to Peer Network programming

I was digging through the Asio documention for sockets but I couldn't find anything useful on how I can handle the following situation:
I assume to have a lot of servers in a peer to peer network (up to 1000).
Servers will have to communicate regularly with each other so I do not want to open a new client connection to send a message to another server every time this is needed (huge overhead).
At the same time, creating n threads that each correspond to a client -> server connection is also not really viable.
I'll implement different communication schemes (all-to-all, star and tree) so 1, log(n) and n of the servers will have to instantiate those n socket clients to create a connection to the other servers.
Is there a good way I can simply do (pseudocode).
pool = ConnectionPool.create(vector<IP>);
pool.sendMessage(ip, message);
I know on the server side I can use an async connection. However, I don't really know how to handle it from the "client" (sender) perspective in C++/Asio.
Tl:DR;
Which APIs and classes am I supposed to use when I want to "send" messages to N servers without having to open N connections every time I do that and neither using N threads".
Yes, each process will need a server side (to receive messages from any of the n participants) and one client side (to send messages to any of the n participants). However, as far as I could find in Asio, the only way to send messages to k of the n participants is by creating k threads with k connections
Then you must not have looked in the right place, or not very far at all.
A core tenet async IO is multiplexing IO on a single thread (all of the kqueue/epoll/select/IO completion ports etc abstractions are geared towards that goal).
Here's an absolutely lazy-coded demonstration that shows:
single threaded everything
a listener that accepts unbounded clients (we could easily add additional listeners)
we connect to a collection of "peers"
on a heartbeat interval we send all the peers a heartbeat message
for (auto& peer : peers)
async_write(peer, buffer(message), [ep=peer.remote_endpoint(ec)](error_code ec, size_t xfr) {
std::cout << "(sent " << xfr << " bytes to " << ep << "(" << ec.message() << ")" << std::endl;
});
additionally it handles asynchronous process signals (INT, TERM) to shutdown all the async operations
"Live¹" On Coliru
#include <boost/asio.hpp>
#include <list>
#include <iostream>
using std::tuple;
using namespace std::literals;
template <typename T>
static auto reference_eq(T const& obj) {
return [p=&obj](auto& ref) { return &ref == p; };
}
int main() {
using namespace boost::asio; // don't be this lazy please
using boost::system::error_code;
using ip::tcp;
io_context ioc;
tcp::acceptor listener(ioc, {{}, 6868});
listener.set_option(tcp::acceptor::reuse_address(true));
listener.listen();
using Loop = std::function<void()>;
std::list<tcp::socket> clients, peers;
// accept unbounded clients
Loop accept_loop = [&] {
listener.async_accept([&](error_code const& ec, tcp::socket s) {
if (!ec) {
std::cout << "New session " << s.remote_endpoint() << std::endl;
clients.push_back(std::move(s));
accept_loop();
}
});
};
tcp::resolver resoler(ioc);
for (auto [host,service] : {
tuple{"www.example.com", "http"},
{"localhost", "6868"},
{"::1", "6868"},
// ...
})
{
auto& p = peers.emplace_back(ioc);
async_connect(p, resoler.resolve(host,service), [&,spec=(host+":"s+service)](error_code ec, auto...) {
std::cout << "For " << spec << " (" << ec.message() << ")";
if (!ec)
std::cout << " " << p.remote_endpoint();
else
peers.remove_if(reference_eq(p));
std::cout << std::endl;
});
}
std::string const& message = "heartbeat\n";
high_resolution_timer timer(ioc);
Loop heartbeat = [&]() mutable {
timer.expires_from_now(2s);
timer.async_wait([&](error_code ec) {
std::cout << "heartbeat " << ec.message() << std::endl;
if (ec)
return;
for (auto& peer : peers)
async_write(peer, buffer(message), [ep=peer.remote_endpoint(ec)](error_code ec, size_t xfr) {
std::cout << "(sent " << xfr << " bytes to " << ep << "(" << ec.message() << ")" << std::endl;
});
heartbeat();
});
};
signal_set sigs(ioc, SIGINT, SIGTERM);
sigs.async_wait([&](error_code ec, int sig) {
if (!ec) {
std::cout << "signal: " << strsignal(sig) << std::endl;
listener.cancel();
timer.cancel();
} });
accept_loop();
heartbeat();
ioc.run_for(10s); // max time for Coliru, or just `run()`
}
Prints (on my system):
New session 127.0.0.1:46730
For localhost:6868 (Success) 127.0.0.1:6868
For ::1:6868 (Connection refused)
For www.example.com:http (Success) 93.184.216.34:80
heartbeat Success
(sent 10 bytes to 93.184.216.34:80(Success)
(sent 10 bytes to 127.0.0.1:6868(Success)
heartbeat Success
(sent 10 bytes to 93.184.216.34:80(Success)
(sent 10 bytes to 127.0.0.1:6868(Success)
heartbeat Success
(sent 10 bytes to 93.184.216.34:80(Success)
(sent 10 bytes to 127.0.0.1:6868(Success)
^Csignal: Interrupt
heartbeat Operation canceled
Note how the one client ("New session") is our own peer connection on localhost:6868 :)
Of course, in real life you would have a class to represent a client session, perhaps have queues for messages pending sending, and optionally run on multiple threads (using strands to synchronize access to shared objects).
OTHER SAMPLES
If you really wish to avoid an explicit collection of clients, see this very similar demo: How to pass a boost asio tcp socket to a thread for sending heartbeat to client or server which
also starts from single-threaded, but adds a thread pool for strand demonstration purposes)
It has a heartbeat timer per session meaning that each session can have their own frequency
¹ it's not working on coliru because of limited access to network. A loop-back only version without resolver use works: Live On Coliru
Since you stated you want to use a TCP i.e. connection based protocol, you can use the async ASIO API and could rely on 1 thread, because async i.e. reactor pattern call do not block.
Your server would use boost::asio::async_write to a boost::asio::ip::tcp::socket, which is equal to one TCP connection happening. The callback you give async_write as a parameter will be called when you are done sending, but async_write would return immediatly. Receiving would be similar to a client. In order to get a TCP connection to a incoming client you would have to use a boost::asio::ip::tcp::resolver which opens new TCP connections/sockets for you by listening via boost::asio::ip::tcp::resolver::async_resolve in the client and boost::asio::ip::tcp::acceptor initialized with a boost::asio::ip::tcp::endpoint and boost::asio::ip::tcp::acceptor::async_accept on server side. Actually you would need 2, one for IPv4 and for IPv6 each.
Since you would have some state with a TCP connection on server side, you would ordinary have to track in a central place, but to avoid this contention and ease the pattern, its common to use a class which inherits std::enable_shared_from_this, which will give a std::shared_pointer of itself into the callback to std::async_write so that, between sending and receiving, where the thread is not blocked in the usual sense, it would not be forgotten i.e. deleted.
For reading I recommend boost::asio::async_read_until and in general a boost::asio::streambuf.
By this 1 thread that runs boost::asio::io_context::run in a loop would suffice, it would unblock every-time one of the many connections need processing of the received stuff or something new to be sent has to be generated.
The general project is a bit out of scope, it would help if you could narrow your question a bit, or better read the talks and examples. I have written something similiar as you indent, a resilient overlay network: https://github.com/Superlokkus/code

Sending UDP data from local port to local port

I'm trying to send data to my own local port using UDP packet.
For this I use boost::asio
#include <iostream>
#include <boost/asio.hpp>
#include <boost/system/error_code.hpp>
using boost::asio::ip::udp;
using boost::asio::ip::address;
boost::asio::io_service _ioServiceOut;
std::shared_ptr<udp::socket> _socketOut;
std::shared_ptr<udp::endpoint> _endpointOut;
static int PortOut = 32676;
static std::string Loopback = "127.0.0.1";
int main()
{
_endpointOut = std::make_shared<udp::endpoint>(address::from_string(Loopback), PortOut);
_socketOut = std::make_shared<udp::socket>(_ioServiceOut, *_endpointOut.get());
_socketOut->send_to(
boost::asio::buffer("0"), *(_endpointOut.get())
);
return 0;
}
The problem is, it sends from 32676 to 32676 and my other application that is listening to this port, can't receive the message then.
When writin a similar application in C#, the Net.Socket assigns a random port as the outcoming one.
How can I achieve the same effect with boost::asio?
So, thanks to tkausl I figured out the problem is the Output socked initialization. If you want your system to assing you a free port, just create a new endpoint setting the port to 0.
_socketOut = std::make_shared<udp::socket>(_ioServiceOut, new udp::endpoint(address::from_string(Loopback), 0));

How can I choose the port to which I send the response

I am currently trying to communicate with a PLC by using Modbus/TCP but even thought I can read the Modbus frame send by the PLC I must give the Modbus response to port 502 and the stream I use send to the port used by the PLC to send the frame. I tried using two stream to receive and send but when i close the first one the PLC get that as a timed out communication and then refuse the second connection.
If it can help you here is the code I use but for now it barely does anything else than allowing me to test the connection.
#define _WIN32_WINNT 0x0501
#include <iostream>
#include <boost/asio.hpp>
#include <boost/system/config.hpp>
#include <string>
using namespace std;
using boost::asio::ip::tcp;
int main()
{
boost::asio::io_service io_service;
tcp::endpoint endpoint(tcp::v4(), 502);
tcp::acceptor acceptor(io_service, endpoint);
while (1)
{
int i =0;
string buff;
char buf[30];
tcp::iostream stream;
tcp::iostream s("192.168.10.150", "502");
acceptor.accept(*stream.rdbuf());
getline(stream, buff);
cout<<buff<<endl;
s <<buff;
}
}
If you have any suggestion.

Socket validation using Poco c++ libraries

I'm working on an application that needs to perform network communication and decided to use the poco c++ libraries. After going through the network tutorial I can't seem to find any forms of validation on establishing a network connection.
In the following example a client tries to connect to a server using a tcp socket stream:
#include "Poco/Net/SocketAddress.h"
#include "Poco/Net/StreamSocket.h"
#include "Poco/Net/SocketStream.h"
#include "Poco/StreamCopier.h"
#include <iostream>
int main(int argc, char** argv)
{
Poco::Net::SocketAddress sa("www.appinf.com", 80);
Poco::Net::StreamSocket socket(sa);
Poco::Net::SocketStream str(socket);
str << "GET / HTTP/1.1\r\n"
"Host: www.appinf.com\r\n"
"\r\n";
str.flush();
Poco::StreamCopier::copyStream(str, std::cout);
return 0;
}
However, I couldn't find any information related to:
Error checking(what if www.appinf.com is unavailable or doesn't exist for that matter)
The type of exception these calls may raise
The only mention is that a SocketStream may hang if the receive timeout is not set for the socket when using formated inputs.
How can I check if a host is alive and may set up a tcp connection, implement a method such as:
void TCPClient::connectTo(std::string host, bool& connected, unsigned int port) {
std::string hi = "hi";
Poco::Net::SocketAddress clientSocketAddress(host, port);
Poco::Net::StreamSocket clientStreamSocket;
// try to connect and avoid hang by setting a timeout
clientStreamSocket.connect(clientSocketAddress, timeout);
// check if the connection has failed or not,
// set the connected parameter accordingly
// additionally try to send bytes over this connection
Poco::Net::SocketStream clientSocketStream(clientStreamSocket);
clientSocketStream << hi << std::endl;
clientSocketStream.flush();
// close the socket stream
clientSocketStream.close();
// close stream
clientStreamSocket.shutdown();
}