I am setting up an HTTP server in C++ using the uWebSockets library and I would like to add a middleware to serve static files, similar to what app.use(express.static(path.join(__dirname, 'public'))); does in Express.js.
The static files reside in the public folder. The middleware should make the server load files under the path http://localhost:8087/css/bootstrap.min.css, not http://localhost:8087/public/css/bootstrap.min.css, thus re-routing root to public.
How could one do this in C++ using the uWebSockets library? I already inspected the uWS::App struct, however I find there nothing related to the path or to serving static files.
Here is an example an HTTP server:
#include <uWebSockets/App.h>
#include <rapidjson/rapidjson.h>
#include "util/AsyncFileReader.h"
#include "util/AsyncFileStreamer.h"
#include <iostream>
#include <string>
void get_home(uWS::HttpResponse<false> *res, uWS::HttpRequest *req) {
res->writeHeader("Content-Type", "text/html; charset=utf8");
// res->writeStatus(uWS::HTTP_200_OK);
// res->end("Hello! This is <b>Sergei's C++ web server</b>.");
AsyncFileReader page_contents("./public/home.html");
res->end(page_contents.peek(0));
}
int main() {
int port{8087};
// HTTP
uWS::App app = uWS::App();
app.get("/", get_home);
app.listen(port, [&port](auto *token) {
if (token) {
std::cout << "Listening on port " << port << std::endl;
}
})
.run();
return 0;
}
There is an example with this exactly
I eventually ended up adding a directory watch and updating the html files if saved (a few changes in codebase) but i guess thats a different thing
#include "helpers/AsyncFileReader.h"
#include "helpers/AsyncFileStreamer.h"
#include "helpers/Middleware.h"
AsyncFileStreamer asyncFileStreamer("htmls"); // htmls is a relative folder path to static files
app.get("/*", gethome); // note the *
void get_home(auto *res, auto *req) {
//void get_home(uWS::HttpResponse<false> *res, uWS::HttpRequest *req) {
serveFile(res, req); // essentially res->writeStatus(uWS::HTTP_200_OK);
asyncFileStreamer.streamFile(res, req->getUrl());
}
Please note serveFile() function also needs to take care of different Content-Type header setting for images
example mentioned:
https://github.com/uNetworking/uWebSockets/blob/master/examples/HttpServer.cpp
To expand the answer of #amit.user105387, here is how a working solution could look like:
#include <uWebSockets/App.h>
#include <rapidjson/rapidjson.h>
#include "util/AsyncFileReader.h"
#include "util/AsyncFileStreamer.h"
#include "util/Middleware.h"
#include <iostream>
#include <string>
void get_home(uWS::HttpResponse<false> *res, uWS::HttpRequest *req) {
// res->writeHeader("Alt-Svc", "h2=\"localhost:8087\"public\"");
res->writeHeader("Content-Type", "text/html; charset=utf8");
AsyncFileReader page_contents("./public/html/home.html");
res->end(page_contents.peek(0));
}
void PrintCachedFiles(std::map<std::string_view, AsyncFileReader *> dict) {
for (auto it = dict.begin(); it != dict.end(); it++) {
std::cout << it->first << std::endl;
}
}
int main() {
const int port{8087};
const std::string public_folder{"./public"};
AsyncFileStreamer public_files("./public");
// HTTP
uWS::App app = uWS::App();
app.get("/", [&public_files](auto res, auto req){
std::cout << "root folder = "<< public_files.root << std::endl;
PrintCachedFiles(public_files.asyncFileReaders);
res->writeHeader("Content-Type", "text/html; charset=utf8");
public_files.streamFile(res, "/html/home.html");
res->writeStatus(uWS::HTTP_200_OK)->end();
});
app.get("/css/:path", [&public_files](auto res, auto req){
public_files.streamFile(res, req->getUrl());
res->writeStatus(uWS::HTTP_200_OK)->end();
});
app.get("/js/:path", [&public_files](auto res, auto req){
public_files.streamFile(res, req->getUrl());
res->writeStatus(uWS::HTTP_200_OK)->end();
});
app.get("/img/:path/*", [&public_files](auto res, auto req){
serveFile(res, req);
public_files.streamFile(res, req->getUrl());
res->writeStatus(uWS::HTTP_200_OK)->end();
});
app.get("/*", [&public_files](auto res, auto req){
res->writeStatus(uWS::HTTP_200_OK)->end("404: Page not found.");
});
app.listen(port, [&port](auto *token) {
if (token) {
std::cout << "Listening on port " << port << std::endl;
}
})
.run();
return 0;
}
Related
I wrote a some code that should send GET request and get response.
It works for ip-api.com and returns me json file.
But for api.vk.com it returns html as that:
<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
<hr><center>kittenx</center>
</body>
</html>
The most interesting thing is that the program returns the correct link, after opening which the desired GET request will be executed.
main.cpp:
#include <iostream>
#include "client.hpp"
#include "json.hpp"
std::string get_token(const std::string &);
int main()
{
std::string token = get_token("data/token1");
std::string query = "https://api.vk.com/method/groups.getMembers?access_token=" + token + "&v=5.13&group_id=klubauto";
std::cout << query << "\n\n\n";
Client client(url);
client.send_request(query);
std::string response = client.get_response();
std::cout << response << std::endl;
return 0;
}
client.hpp:
#pragma once
#include <string>
#include <boost/beast.hpp>
#include <boost/asio/connect.hpp>
#include <boost/asio/ip/tcp.hpp>
namespace http = boost::beast::http;
class Client
{
public:
Client();
Client(const std::string &api);
~Client();
void send_request(const std::string &arguments);
std::string get_response();
private:
boost::asio::io_context io;
boost::asio::ip::tcp::resolver resolver;
boost::asio::ip::tcp::socket socket;
std::string url;
};
client.cpp
#include "client.hpp"
/*
* Constructors
*/
Client::Client() : url("google.com"), resolver(io), socket(io)
{
boost::asio::connect(socket, resolver.resolve(url, "80"));
}
Client::Client(const std::string &api) : url(api), resolver(io), socket(io)
{
boost::asio::connect(socket, resolver.resolve(url, "80"));
}
/*
* Destructor
*/
Client::~Client()
{
socket.shutdown(boost::asio::ip::tcp::socket::shutdown_both);
}
/*
* Send request
*/
void Client::send_request(const std::string &arguments)
{
http::request<http::string_body> req(http::verb::get, arguments, 11);
req.set(http::field::host, url);
req.set(http::field::user_agent, BOOST_BEAST_VERSION_STRING);
http::write(socket, req);
}
/*
* Get response
*/
std::string Client::get_response()
{
std::string response;
{
boost::beast::flat_buffer buffer;
http::response<http::dynamic_body> res;
http::read(socket, buffer, res);
response = boost::beast::buffers_to_string(res.body().data());
}
return response;
}
I would like to receive a json file in the response variable, please tell me how to achieve this?
Like I commented, that's how HTTP works: Servers can redirect to a better/new location.
I assume the prime reason for this is because your connection is not HTTPS, and that's what the end-points require. So, fix that first.
Next, your query includes the base URL, which is another error.
Live Demo
#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
#include <boost/beast.hpp>
#include <string>
namespace http = boost::beast::http;
namespace ssl = boost::asio::ssl;
using boost::asio::ip::tcp;
class Client {
public:
Client(const std::string& host = "google.com") : _host(host) {
_ctx.set_default_verify_paths();
connect(_socket.lowest_layer(),
tcp::resolver{_io}.resolve(_host, "https"));
_socket.handshake(ssl::stream_base::client);
}
void send_request(const std::string& query)
{
http::request<http::string_body> req(http::verb::get, query, 11);
req.set(http::field::host, _host);
req.set(http::field::user_agent, BOOST_BEAST_VERSION_STRING);
http::write(_socket, req);
}
std::string get_response() {
http::response<http::string_body> res;
boost::beast::flat_buffer buffer;
http::read(_socket, buffer, res);
return std::move(res.body());
}
private:
boost::asio::io_context _io;
ssl::context _ctx{ssl::context::sslv23_client};
ssl::stream<tcp::socket> _socket{_io, _ctx};
std::string _host;
};
#include <iostream>
#include <boost/json.hpp>
#include <boost/json/src.hpp> // for COLIRU header-only
namespace json = boost::json;
std::string get_token(const std::string&) { return ""; }
int main()
{
Client client("api.vk.com");
client.send_request("/method/groups.getMembers?access_token=" +
get_token("data/token1") + "&v=5.13&group_id=klubauto");
std::cout << json::parse(client.get_response()) << std::endl;
}
Coliru doesn't allow public network access:
terminate called after throwing an instance of 'boost::wrapexcept<boost::system::system_error>'
what(): resolve: Service not found
But on my machine it correctly says:
{"error":{"error_code":5,"error_msg":"User authorization failed: no access_token passed.","request_params":[{"key":"v","value":"5.13"},{"key":"group_id","value":"klubauto"},{"key":"method","value":"groups.getMembers"},{"key":"oauth","value":"1"}]}}
Note I included quite a number of simplifications along the way.
I have started with this example so won't post all the code. My objective is to download a large file without blocking my main thread. The second objective is to get notifications so I can update a progress bar. I do have the code working a couple of ways. First is to just ioc.run(); and let it go to work, I get the file downloaded. But I can not find anyway to start the session without blocking.
The second way I can make the calls down to http::async_read_some and the call works but I can not get a response that I can use. I don't know if there is a way to pass a lambda that captures.
The #if 0..#else..#endif switches the methods. I'm sure there is a simple way but I just can not see it. I'll clean up the code when I get it working, like setting the local file name. Thanks.
std::size_t on_read_some(boost::system::error_code ec, std::size_t bytes_transferred)
{
if (ec);//deal with it...
if (!bValidConnection) {
std::string_view view((const char*)buffer_.data().data(), bytes_transferred);
auto pos = view.find("Content-Length:");
if (pos == std::string_view::npos)
;//error
file_size = std::stoi(view.substr(pos+sizeof("Content-Length:")).data());
if (!file_size)
;//error
bValidConnection = true;
}
else {
file_pos += bytes_transferred;
response_call(ec, file_pos);
}
#if 0
std::cout << "in on_read_some caller\n";
http::async_read_some(stream_, buffer_, file_parser_, std::bind(
response_call,
std::placeholders::_1,
std::placeholders::_2));
#else
std::cout << "in on_read_some inner\n";
http::async_read_some(stream_, buffer_, file_parser_, std::bind(
&session::on_read_some,
shared_from_this(),
std::placeholders::_1,
std::placeholders::_2));
#endif
return buffer_.size();
}
The main, messy but.....
struct lambda_type {
bool bDone = false;
void operator ()(const boost::system::error_code ec, std::size_t bytes_transferred) {
;
}
};
int main(int argc, char** argv)
{
auto const host = "reserveanalyst.com";
auto const port = "443";
auto const target = "/downloads/demo.msi";
int version = argc == 5 && !std::strcmp("1.0", argv[4]) ? 10 : 11;
boost::asio::io_context ioc;
ssl::context ctx{ ssl::context::sslv23_client };
load_root_certificates(ctx);
//ctx.load_verify_file("ca.pem");
auto so = std::make_shared<session>(ioc, ctx);
so->run(host, port, target, version);
bool bDone = false;
auto const lambda = [](const boost::system::error_code ec, std::size_t bytes_transferred) {
std::cout << "data lambda bytes: " << bytes_transferred << " er: " << ec.message() << std::endl;
};
lambda_type lambda2;
so->set_response_call(lambda);
ioc.run();
std::cout << "not in ioc.run()!!!!!!!!" << std::endl;
so->async_read_some(lambda);
//pseudo message pump when working.........
for (;;) {
std::this_thread::sleep_for(250ms);
std::cout << "time" << std::endl;
}
return EXIT_SUCCESS;
}
And stuff I've added to the class session
class session : public std::enable_shared_from_this<session>
{
using response_call_type = void(*)(boost::system::error_code ec, std::size_t bytes_transferred);
http::response_parser<http::file_body> file_parser_;
response_call_type response_call;
//
bool bValidConnection = false;
std::size_t file_pos = 0;
std::size_t file_size = 0;
public:
auto& get_result() { return res_; }
auto& get_buffer() { return buffer_; }
void set_response_call(response_call_type the_call) { response_call = the_call; }
I've updated this as I finally put it to use and I wanted the old method where I could download to a file or a string. Link to how asio works, great talk.
CppCon 2016 Michael Caisse Asynchronous IO with BoostAsio
As for my misunderstanding of how to pass a lambda, here is Adam Nevraumont's answer
There are two ways to compile this using a type to select the method. Both are shown at the beginning of main. You can construct either a file downloader or string downloader by selecting the type of beast parser. The parsers don't have the same constructs so an if constexpr compile time conditions are used. And I checked, a release build of the downloader is about 1K so pretty light weight for what it does. In the case of a small string you don't have to handle the call backs. either pass an empty lambda or add the likes of:
if(response_call)
response_call(resp_ok, test);
This looks to be a pretty clean way to get the job done so I've updated this post as of 11/27/2202.
The code:
//
// Copyright (c) 2016-2019 Vinnie Falco (vinnie dot falco at gmail dot com)
//
// Distributed under the Boost Software License, Version 1.0. (See accompanying
// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
//
// Official repository: https://github.com/boostorg/beast
//------------------------------------------------------------------------------
//
// Example: HTTP SSL client, synchronous, usable in a thread with a message pump
// Added code to use from a message pump
// Also useable as body to a file download, or body to string
//
//------------------------------------------------------------------------------
#include <boost/beast/core.hpp>
#include <boost/beast/http.hpp>
#include <boost/beast/ssl.hpp>
#include <boost/beast/version.hpp>
#include <boost/asio/connect.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/ssl/error.hpp>
#include <boost/asio/ssl/stream.hpp>
#include <cstdlib>
#include <iostream>
#include <string>
#include <fstream>
//the boost shipped certificates
#include <boost/../libs/beast/example/common/root_certificates.hpp>
//TODO add your ssl libs as you would like
#ifdef _M_IX86
#pragma comment(lib, "libcrypto.lib")
#pragma comment(lib, "libssl.lib")
#elif _M_X64
#pragma comment(lib, "libcrypto-3-x64.lib")
#pragma comment(lib, "libssl-3-x64.lib")
#endif
namespace downloader {
namespace beast = boost::beast; // from <boost/beast.hpp>
namespace http = beast::http; // from <boost/beast/http.hpp>
namespace net = boost::asio; // from <boost/asio.hpp>
namespace ssl = net::ssl; // from <boost/asio/ssl.hpp>
using tcp = net::ip::tcp; // from <boost/asio/ip/tcp.hpp>
//specialization if using < c++17; see both 'if constexpr' below.
//this is not needed otherwise
//namespace detail {
// template<typename Type>
// void open_file(http::parser < false, Type>& p, const char* name, boost::system::error_code& file_open_ec) { }
// template<>
// void open_file(http::parser<false, http::file_body>& p, const char* name, boost::system::error_code& file_open_ec) {
// p.get().body().open(name, boost::beast::file_mode::write, file_open_ec);
// }
// template<typename Type>
// std::string get_string(http::parser < false, Type>& p) { return std::string{}; }
// template<>
// std::string get_string(http::parser<false, http::string_body>& p) {
// return p.get().body();
// }
//} //namespace detail
enum responses {
resp_null,
resp_ok,
resp_done,
resp_error,
};
using response_call_type = std::function< void(responses, std::size_t)>;
template<typename ParserType>
struct download {
//as these can be set with array initialization
const char* target_ = "/";
const char* filename_ = "test.txt";
const char* host_ = "lakeweb.net";
std::string body_;
using response_call_type = std::function< void(responses, std::size_t)>;
response_call_type response_call;
boost::asio::io_context ioc_;
ssl::context ctx_{ ssl::context::sslv23_client };
ssl::stream<tcp::socket> stream_{ ioc_, ctx_ };
tcp::resolver resolver_{ ioc_ };
boost::beast::flat_buffer buffer_;
uint64_t file_size_{};
int version{ 11 };
void set_response_call(response_call_type the_call) { response_call = the_call; }
uint64_t get_file_size() { return file_size_; }
void stop() { ioc_.stop(); }
bool stopped() { return ioc_.stopped(); }
std::string get_body() { return std::move(body_); }
void run() {
try {
// TODO should have a timer in case of a hang
load_root_certificates(ctx_);
// Set SNI Hostname (many hosts need this to handshake successfully)
if (!SSL_set_tlsext_host_name(stream_.native_handle(), host_)) {
boost::system::error_code ec{ static_cast<int>(::ERR_get_error()), boost::asio::error::get_ssl_category() };
throw boost::system::system_error{ ec };
}
//TODO resolve is depreciated, use endpoint
auto const results = resolver_.resolve(host_, "443");
boost::asio::connect(stream_.next_layer(), results.begin(), results.end());
stream_.handshake(ssl::stream_base::client);
// Set up an HTTP GET request message
http::request<http::string_body> req{ http::verb::get, target_, version };
req.set(http::field::host, host_);
req.set(http::field::user_agent, "mY aGENT");
// Send the HTTP request to the remote host
http::write(stream_, req);
// Read the header
boost::system::error_code file_open_ec;
http::parser<false, ParserType> p;
p.body_limit((std::numeric_limits<std::uint32_t>::max)());
//detail::open_file(p, filename_, file_open_ec);
//or => c++17
if constexpr (std::is_same_v<ParserType, http::file_body>)
p.get().body().open(filename_, boost::beast::file_mode::write, file_open_ec);
http::read_header(stream_, buffer_, p);
file_size_ = p.content_length().has_value() ? p.content_length().value() : 0;
//Read the body
uint64_t test{};
boost::system::error_code rec;
for (;;) {
test += http::read_some(stream_, buffer_, p, rec);
if (test >= file_size_) {
response_call(resp_done, 0);
break;
}
response_call(resp_ok, test);
}
// Gracefully close the stream
boost::system::error_code ec;
stream_.shutdown(ec);
if (ec == boost::asio::error::eof)
{
// Rationale:
// http://stackoverflow.com/questions/25587403/boost-asio-ssl-async-shutdown-always-finishes-with-an-error
ec.assign(0, ec.category());
}
if (ec)
throw boost::system::system_error{ ec };
//value = detail::get_string(p);
//or => c++17
if constexpr (std::is_same_v<ParserType, http::string_body>)
body_ = p.get().body();
}
catch (std::exception const& e)
{
std::cerr << "Error: " << e.what() << std::endl;
response_call(resp_error, -1);
}
ioc_.stop();
}
};
}//namespace downloadns
//comment to test with string body
#define THE_FILE_BODY_TEST
int main(int argc, char** argv)
{
using namespace downloader;
#ifdef THE_FILE_BODY_TEST
download<http::file_body> dl{"/Nasiri%20Abarbekouh_Mahdi.pdf", "test.pdf"};
#else //string body test
download<http::string_body> dl{ "/robots.txt" };
#endif
responses dl_response{ resp_null };
size_t cur_size{};
auto static const lambda = [&dl_response, &dl, &cur_size](responses response, std::size_t bytes_transferred) {
if ((dl_response = response) == resp_ok) {
cur_size += bytes_transferred;
size_t sizes = dl.get_file_size() - cur_size;//because size is what is left
//drive your progress bar from here in a GUI app
}
};
dl.set_response_call(lambda);
std::thread thread{ [&dl]() { dl.run(); } };
//thread has started, now the pseudo message pump
bool quit = false; //true: as if a cancel button was pushed; won't finish download
for (int i = 0; ; ++i) {
switch (dl_response) { //ad hoc as if messaged
case resp_ok:
std::cout << "from sendmessage: " << cur_size << std::endl;
dl_response = resp_null;
break;
case resp_done:
std::cout << "from sendmessage: done" << std::endl;
dl_response = resp_null;
break;
case resp_error:
std::cout << "from sendmessage: error" << std::endl;
dl_response = resp_null;
}//switch
if (!(i % 5))
std::cout << "in message pump, stopped: " << std::boolalpha << dl.stopped() << std::endl;
std::this_thread::sleep_for(std::chrono::milliseconds(100));
if (quit && i == 10) //the cancel message
dl.stop();
if (!(i % 20) && dl.stopped()) {//dl job was quit or error or finished
std::cout << "dl is stopped" << std::endl;
break;
}
}
#ifdef THE_FILE_BODY_TEST
std::cout << "file written named: 'test.txt'" << std::endl;
#else
std::string res = dl.get_body();
std::cout << "body retrieved:\n" << res << std::endl;
#endif
if (thread.joinable())//in the case a thread was never started
thread.join();
std::cout << "exiting, program all done" << std::endl;
return EXIT_SUCCESS;
}
I strongly recommend against using the low-level [async_]read_some function instead of using http::[async_]read as intended with http::response_parser<http::buffer_body>
I do have an example of that - which is a little bit complicated by the fact that it also uses Boost Process to concurrently decompress the body data, but regardless it should show you how to use it:
How to read data from Internet using muli-threading with connecting only once?
I guess I could tailor it to your specific example given more complete code, but perhaps the above is good enough? Also see "Relay an HTTP message" in libs/beast/example/doc/http_examples.hpp which I used as "inspiration".
Caution: the buffer arithmetic is not intuitive. I think this is unfortunate and should not have been necessary, so pay (very) close attention to these samples for exactly how that's done.
I created sample app, for sending/receiving messages between node.js app which is running as socket.io server and C++ client, Below is my code of C++ client side:
sio::client io;
socket::ptr current_socket;
string w = "harshil";
io.set_open_listener([&]() {
io.socket()->emit("message", w); // Can able to send message to server
});
io.socket()->on("server", sio::socket::event_listener([&](event &e)
{
cout << __LINE__ << endl; // Can not print line :(
}));
io.connect("http://127.0.0.1:8081");
Over here you can see, that client can able to send message to server, but it can not receive message based on "server" event name, Can some one help me on it?
To those of you who stuck as me, then below sample code will be helpful for them.
Created different sample code for node.js acts as socket.io server and main.cpp file acts as client,
'use strict';
const express = require('express');
const app = express();
const serverHttp = require('http').Server(app);
const io = require('socket.io')(serverHttp);
const port = 8081;
io.on('connection', function (socket) {
socket.on('message', function (data) {
console.log("key received!!!" + data);
socket.emit('server', 'hello socket io');
console.log("sent server msg");
});
});
serverHttp.listen(port, function() {
console.log("init!!!");
});
Sample server app, which receives request from client and emit message to client.
#include "sio_client.h"
#include <unistd.h>
#include <functional>
#include <iostream>
#include <thread>
#include <mutex>
#include <condition_variable>
#include <string>
#define HIGHLIGHT(__O__) std::cout<<"\e[1;31m"<<__O__<<"\e[0m"<<std::endl
#define EM(__O__) std::cout<<"\e[1;30;1m"<<__O__<<"\e[0m"<<std::endl
#define MAIN_FUNC int main(int argc ,const char* args[])
using namespace sio;
using namespace std;
std::mutex _lock;
std::condition_variable_any _cond;
bool connect_finish = false;
class connection_listener
{
sio::client &handler;
public:
connection_listener(sio::client& h):
handler(h)
{
}
void on_connected()
{
_lock.lock();
_cond.notify_all();
connect_finish = true;
_lock.unlock();
}
void on_close(client::close_reason const& reason)
{
std::cout<<"sio closed "<<std::endl;
exit(0);
}
void on_fail()
{
std::cout<<"sio failed "<<std::endl;
exit(0);
}
};
socket::ptr current_socket;
void bind_events()
{
current_socket->on("server", sio::socket::event_listener_aux([&](string const& name, message::ptr const& data, bool isAck,message::list &ack_resp)
{
_lock.lock();
cout << name << endl;
cout << data->get_string() << endl;
_lock.unlock();
}));
}
MAIN_FUNC
{
sio::client h;
connection_listener l(h);
h.set_open_listener(std::bind(&connection_listener::on_connected, &l));
h.set_close_listener(std::bind(&connection_listener::on_close, &l,std::placeholders::_1));
h.set_fail_listener(std::bind(&connection_listener::on_fail, &l));
h.connect("http://127.0.0.1:8081");
_lock.lock();
if(!connect_finish)
{
cout << "wait\n";
_cond.wait(_lock);
}
_lock.unlock();
current_socket = h.socket();
string nickname;
while (nickname.length() == 0) {
HIGHLIGHT("Type your nickname:");
getline(cin, nickname);
}
current_socket->emit("message", nickname);
bind_events();
sleep(10);
h.sync_close();
h.clear_con_listeners();
return 0;
}
Sample client app, which emits message to server, and receives message from server,
I created a TCPServer using the Poco::Net::TCPServer framework, that uses a unix domain socket and it seems to work. However if I close the server and start it again I get this error:
Net Exception: Address already in use: /tmp/app.SocketTest
What is the right way to deal with this error?
Are the TCPServerConnections, TCPServerConnectionFactory and sockets
automatically cleaned-up or do I need to implement their destructors or destroy them manually?
EDIT
I have two questions here. The first is answered by using remove() on the socket-file. The other question is, if the clean-up in the Poco::Net::TCPServer framework is automatic or if it has to be manually implemented to prevent memory-leak?
Here is the code for the TCPServer:
#include "Poco/Util/ServerApplication.h"
#include "Poco/Net/TCPServer.h"
#include "Poco/Net/TCPServerConnection.h"
#include "Poco/Net/TCPServerConnectionFactory.h"
#include "Poco/Util/Option.h"
#include "Poco/Util/OptionSet.h"
#include "Poco/Util/HelpFormatter.h"
#include "Poco/Net/StreamSocket.h"
#include "Poco/Net/ServerSocket.h"
#include "Poco/Net/SocketAddress.h"
#include "Poco/File.h"
#include <fstream>
#include <iostream>
using Poco::Net::ServerSocket;
using Poco::Net::StreamSocket;
using Poco::Net::TCPServer;
using Poco::Net::TCPServerConnection;
using Poco::Net::TCPServerConnectionFactory;
using Poco::Net::SocketAddress;
using Poco::Util::ServerApplication;
using Poco::Util::Option;
using Poco::Util::OptionSet;
using Poco::Util::HelpFormatter;
class UnixSocketServerConnection: public TCPServerConnection
/// This class handles all client connections.
{
public:
UnixSocketServerConnection(const StreamSocket& s):
TCPServerConnection(s)
{
}
void run()
{
try
{
/*char buffer[1024];
int n = 1;
while (n > 0)
{
n = socket().receiveBytes(buffer, sizeof(buffer));
EchoBack(buffer);
}*/
std::string message;
char buffer[1024];
int n = 1;
while (n > 0)
{
n = socket().receiveBytes(buffer, sizeof(buffer));
buffer[n] = '\0';
message += buffer;
if(sizeof(buffer) > n && message != "")
{
EchoBack(message);
message = "";
}
}
}
catch (Poco::Exception& exc)
{
std::cerr << "Error: " << exc.displayText() << std::endl;
}
std::cout << "Disconnected." << std::endl;
}
private:
inline void EchoBack(std::string message)
{
std::cout << "Message: " << message << std::endl;
socket().sendBytes(message.data(), message.length());
}
};
class UnixSocketServerConnectionFactory: public TCPServerConnectionFactory
/// A factory
{
public:
UnixSocketServerConnectionFactory()
{
}
TCPServerConnection* createConnection(const StreamSocket& socket)
{
std::cout << "Got new connection." << std::endl;
return new UnixSocketServerConnection(socket);
}
private:
};
class UnixSocketServer: public Poco::Util::ServerApplication
/// The main application class.
{
public:
UnixSocketServer(): _helpRequested(false)
{
}
~UnixSocketServer()
{
}
protected:
void initialize(Application& self)
{
loadConfiguration(); // load default configuration files, if present
ServerApplication::initialize(self);
}
void uninitialize()
{
ServerApplication::uninitialize();
}
void defineOptions(OptionSet& options)
{
ServerApplication::defineOptions(options);
options.addOption(
Option("help", "h", "display help information on command line arguments")
.required(false)
.repeatable(false));
}
void handleOption(const std::string& name, const std::string& value)
{
ServerApplication::handleOption(name, value);
if (name == "help")
_helpRequested = true;
}
void displayHelp()
{
HelpFormatter helpFormatter(options());
helpFormatter.setCommand(commandName());
helpFormatter.setUsage("OPTIONS");
helpFormatter.setHeader("A server application to test unix domain sockets.");
helpFormatter.format(std::cout);
}
int main(const std::vector<std::string>& args)
{
if (_helpRequested)
{
displayHelp();
}
else
{
// set-up unix domain socket
Poco::File socketFile("/tmp/app.SocketTest");
SocketAddress unixSocket(SocketAddress::UNIX_LOCAL, socketFile.path());
// set-up a server socket
ServerSocket svs(unixSocket);
// set-up a TCPServer instance
TCPServer srv(new UnixSocketServerConnectionFactory, svs);
// start the TCPServer
srv.start();
// wait for CTRL-C or kill
waitForTerminationRequest();
// Stop the TCPServer
srv.stop();
}
return Application::EXIT_OK;
}
private:
bool _helpRequested;
};
int main(int argc, char **argv) {
UnixSocketServer app;
return app.run(argc, argv);
}
You don't need to worry about deallocating memory. All is done by library.
TCPServer srv(new UnixSocketServerConnectionFactory, svs);
^^^
Instance of UnixSocketServerConnectionFactory is deleted by TCPServer according to poco ref
The server takes ownership of the TCPServerConnectionFactory and
deletes it when it's no longer needed.
TCPServerConnection* createConnection(const StreamSocket& socket)
{
std::cout << "Got new connection." << std::endl;
return new UnixSocketServerConnection(socket);
^^^
}
instances of UnixSocketServerConnection are deleted by Poco library code as well:
As soon as the run() method returns, the server connection object is
destroyed and the connection is automatically closed.
The problem with Poco::File was that the destructor of Poco::File cannot remove file, you have to do it explicitly by remove method.
We've made good progress in getting GRPC running under RHEL 7.
Our application has one rather complicated structure with three levels of nesting with the outer level implementing a "oneof" keyword.
We find that all our other structures run fine, but this one gives us an RPC failure with code=14.
We've simplified this part of the application as much as possible so it can hopefully be recompiled and run easily.
Here's the .proto file, updated to accommodate Uli's question:
syntax = "proto3";
option java_multiple_files = true;
option java_package = "io.grpc.examples.debug";
option java_outer_classname = "DebugProto";
option objc_class_prefix = "DEBUG";
package DEBUGpackage;
service DEBUGservice {
rpc DEBUG_val_container_get (input_int32_request) returns (outer_container) {}
}
message input_int32_request {
int32 ival = 1;
}
message inner_container {
repeated uint32 val_array = 1;
}
message middle_container {
inner_container vac = 1;
}
message other_container {
int32 other_val = 1;
}
message outer_container {
oneof reply {
middle_container r1 = 1;
other_container r2 = 2;
}
}
(Please note that the java lines in this prototype code are just in there because they are in the GRPC website examples. Our code is entirely C++, with no java. Don't know if that means we can do without some of these "option java..." lines).
Here's our client source code:
#include <iostream>
#include <memory>
#include <string>
#include <grpc++/grpc++.h>
#include <grpc/support/log.h>
#include <thread>
#include <unistd.h>
#include "debug.grpc.pb.h"
using grpc::Channel;
using grpc::ClientAsyncResponseReader;
using grpc::ClientContext;
using grpc::CompletionQueue;
using grpc::Status;
using DEBUGpackage::input_int32_request;
using DEBUGpackage::inner_container;
using DEBUGpackage::middle_container;
using DEBUGpackage::outer_container;
using DEBUGpackage::DEBUGservice;
class DEBUGClient {
public:
explicit DEBUGClient(std::shared_ptr<Channel> channel)
: stub_(DEBUGservice::NewStub(channel)) {}
void DEBUG_val_container_get() {
std::cout << "in DEBUG_val_container_get" << std::endl;
// Data we are sending to the server
input_int32_request val;
val.set_ival(0);
AsyncClientCall* call = new AsyncClientCall;
call->response_reader = stub_->AsyncDEBUG_val_container_get(&call->context, val, &cq_);
call->response_reader->Finish(&call->reply_, &call->status, (void*)call);
}
void AsyncCompleteRpc() {
void* got_tag;
bool ok = false;
while (cq_.Next(&got_tag, &ok)) {
AsyncClientCall* call = static_cast<AsyncClientCall*>(got_tag);
GPR_ASSERT(ok);
if (call->status.ok()) {
if (call->reply_.has_r1()) {
std::cout << call << " DEBUG received: "
<< call->reply_.r1().vac().val_array(0) << std::endl;
}
}
else {
std::cout << call << " RPC failed" << std::endl;
std::cout << " RPC failure code = " << call->status.error_code() << std::endl;
std::cout << " RPC failure message = " << call->status.error_message() << std::endl;
}
delete call;
}
}
private:
struct AsyncClientCall {
outer_container reply_;
ClientContext context;
Status status;
std::unique_ptr<ClientAsyncResponseReader<outer_container>> response_reader;
};
std::unique_ptr<DEBUGservice::Stub> stub_;
CompletionQueue cq_;
};
int main(int argc, char** argv) {
DEBUGClient DEBUG0(grpc::CreateChannel("172.16.17.46:50050", grpc::InsecureChannelCredentials()));
std::thread thread0_ = std::thread(&DEBUGClient::AsyncCompleteRpc, &DEBUG0);
DEBUG0.DEBUG_val_container_get();
sleep(1);
std::cout << "Press control-c to quit" << std::endl << std::endl;
thread0_.join(); //blocks forever
return 0;
}
And, here's our server source code:
#include <memory>
#include <iostream>
#include <string>
#include <thread>
#include <grpc++/grpc++.h>
#include <grpc/support/log.h>
#include "debug.grpc.pb.h"
#include <time.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
using grpc::Server;
using grpc::ServerAsyncResponseWriter;
using grpc::ServerBuilder;
using grpc::ServerContext;
using grpc::ServerCompletionQueue;
using grpc::Status;
using DEBUGpackage::inner_container;
using DEBUGpackage::input_int32_request;
using DEBUGpackage::middle_container;
using DEBUGpackage::outer_container;
using DEBUGpackage::DEBUGservice;
std::string save_server_address;
class ServerImpl final {
public:
~ServerImpl() {
server_->Shutdown();
cq_->Shutdown();
}
void Run() {
std::string server_address("0.0.0.0:50050");
ServerBuilder builder;
builder.AddListeningPort(server_address, grpc::InsecureServerCredentials());
builder.RegisterService(&service_);
cq_ = builder.AddCompletionQueue();
server_ = builder.BuildAndStart();
std::cout << "Server listening on " << server_address << std::endl;
save_server_address = server_address;
HandleRpcs();
}
private:
class CallData {
public:
virtual void Proceed() = 0;
};
class DebugGetCallData final : public CallData{
public:
DebugGetCallData(DEBUGservice::AsyncService* service, ServerCompletionQueue* cq)
: service_(service), cq_(cq), responder_(&ctx_), status_(CREATE) {
Proceed();
}
void Proceed() {
if (status_ == CREATE) {
status_ = PROCESS;
service_->RequestDEBUG_val_container_get(&ctx_, &request_, &responder_, cq_, cq_, this);
} else if (status_ == PROCESS) {
new DebugGetCallData(service_, cq_);
char *portchar;
portchar = (char *) save_server_address.c_str();
long cq_addr = (long) cq_;
int cq_addr32 = (int) (cq_addr & 0xfffffff);
srand(cq_addr32);
fprintf(stderr, "%s task started\n", portchar); fflush(stderr);
unsigned int return_val = 10;
inner_container ic;
ic.add_val_array(return_val);
middle_container reply_temp;
reply_temp.set_allocated_vac(&ic);
reply_.set_allocated_r1(&reply_temp);
fprintf(stderr, "%s %s task done\n", portchar, "val_container_get"); fflush(stderr);
status_ = FINISH;
responder_.Finish(reply_, Status::OK, this);
} else {
GPR_ASSERT(status_ == FINISH);
}
}
private:
DEBUGservice::AsyncService* service_;
ServerCompletionQueue* cq_;
ServerContext ctx_;
input_int32_request request_;
outer_container reply_;
ServerAsyncResponseWriter<outer_container> responder_;
enum CallStatus { CREATE, PROCESS, FINISH };
CallStatus status_;
};
void HandleRpcs() {
new DebugGetCallData(&service_, cq_.get());
void* tag;
bool ok;
while (true) {
GPR_ASSERT(cq_->Next(&tag, &ok));
GPR_ASSERT(ok);
static_cast<CallData*>(tag)->Proceed();
}
}
std::unique_ptr<ServerCompletionQueue> cq_;
DEBUGservice::AsyncService service_;
std::unique_ptr<Server> server_;
};
int main() {
ServerImpl server;
server.Run();
return 0;
}
The output when I run it looks like this:
[fossum#netsres46 debug]$ DEBUG_client2
in DEBUG_val_container_get
0xb73ff0 RPC failed
RPC failure code = 14
RPC failure message = Endpoint read failed
Press control-c to quit
We ran the server under gdb, and found a place in the generated
file "debug.pb.cc" where if we just comment out one line, it all starts working.
Here's the pertinent piece of the generated file "debug.pb.cc":
middle_container::~middle_container() {
// ##protoc_insertion_point(destructor:DEBUGpackage.middle_container)
SharedDtor();
}
void middle_container::SharedDtor() {
if (this != internal_default_instance()) {
delete vac_; // comment out this one line, to make the problem go away
}
}
The "delete vac_" line appears to be an attempt to delete storage that either has already been deleted, or is about to be deleted somewhere else. Please, can someone look into this? [The files below are still the files we use to generate this code, and to debug the problem to this point]
I have no idea whether I've uncovered a bug in GRPC, or whether I've coded something wrong.
The issue is that you are allocated middle_container reply_tmp on the stack in your server. As a result it gets destructed as soon as you pass out of the scope. At that time, you have called Finish but not yet waited for its result. Since this is an async server, the data must remain alive until you've received the tag back for it. This is why manually editing your destructor works in your case; you're basically nullifying the destructor (and leaking memory as a result).