Ive currently been messing around with boost trying it out. When i try to make a simple multi threaded echo server it exits when receiving with error code 3. I have looked over the documentation many times and still no luck. I know it is probably something very simple i'm overlooking. I have decent experience with winsock but i would like to learn the boost library.
here is the code thats failing i took out the
typedef boost::shared_ptr<tcp::socket> socket_ptr;
boost::asio::io_service io;
boost::array<char, 512> buf;
void startserver ( std::string host, std::string port )
{
tcp::acceptor a (io, tcp::endpoint(tcp::v4(), atoi(port.c_str())));
for(;;)
{
socket_ptr sock (new tcp::socket(io));
a.accept(*sock);
std::cout << sock->remote_endpoint() << std::endl;
boost::thread t (boost::bind(session, sock));
}
}
void session ( socket_ptr sock )
{
sock->send(boost::asio::buffer("welcome"),0,er);
size_t len;
for(;;)
{
len = sock->receive(boost::asio::buffer(buf));
sock->send(boost::asio::buffer(buf,len),0,er);
}
}
I can connect to it fine with netcat and it receives the welcome message but right when it goes to receive it crashs. Ive tried catching an error using boost::system::error_code on each one but nothing was returned
There are too many issues. Check asio documentation for correct examples. Some of issues:
Creating boost::thread object t and then immediately exit scope. This deattaches thread and it not controllable now; or, as mentioned Joachim Pileborg it can terminate (im not very familiar with boost::threads, so correct me if i wrong).
Right after this you starting new acceptor. You should hold only 1 acceptor per listening port.
No point to create thread for this at all, it is ASIO, use async ;)
receive does not wait data, it just fetch packet data ASIO already had (what is not true in this case)
Check examples at boost site, i think your case is blocking tcp echo server
It's most likely because the thread goes out of scope. From the manual page of the boost::thread destructor:
If the thread is joinable calls to std::terminate. Destroys *this.
This means that when the thread is started it might run for a little while before the thread in startserver gets control again and the thread object is destructed and your thread is terminated.
Related
I am trying to create a concurrent c++ TCP server using threads. In particular I was wondering if I could use std::async to accept connections and serve each one in its own thread.
So far I have created a rough mockup but can't really tell if I am on the correct path.
void networking::TCP_Server::acceptConnection() {
std::string stringToSend{"This is a test string to be replied to"};
int new_fd = accept(listeningFD, nullptr, nullptr);
send(new_fd, stringToSend.c_str(), stringToSend.size(), 0);
sleep(3);
std::cout << ("End of thread");
}
///LISTEN FOR CONNECTIONS ON listeningFD
///CREATE A LIST OF FILE DESCRIPTORS FOR POLL fds[]
(fds[i].fd == listeningFD) {
do {
std::cout << ("New incoming connection - %d\n", new_fd);
std::async(std::launch::async, acceptConnection)
} while (new_fd != -1);
} /* End of existing connection is readable */
} /* End of loop through pollable descriptors */
I am connecting at the same time to the server with two clients and would expect for the loop to run through both new connections and create a thread for each one. As of now it is as it runs in deferred mode, one gets accepted, the other waits until the first finishes.
Any ideas?
(Pardon any mistakes in the code)
std::async returns a std::future which the code doesn't save into a variable, hence its destructor is called immediately. std::future::~future() blocks the calling thread until the future becomes ready.
You may like to use (detached) std::thread instead of std::async.
There are more scalable strategies to handle many clients. I highly recommend reading old but instructive The C10K problem.
You may also like to get familar with Asio C++ Library.
I'd like to use the TTheadedServer in a separate thread to have control on when to stop/start it. My application needs only 1 controlling thread and one processing thread. I don't expect to have more than one client as I'm using thrift as a relay. TSimpleServer is not thread-safe, so I dropped that option.
I made a little minimal example to check whether it's thread-safe, and used clang's thread-sanitizer to make sure it's thread-safe. Here's the example
std::shared_ptr<MyHandler> handler = std::make_shared<MyHandler>();
int port = 9090;
th::stdcxx::shared_ptr<th::TProcessor> processor(new HandlerProcessor(handler));
th::stdcxx::shared_ptr<tht::TServerTransport> serverTransport(new tht::TServerSocket(port));
th::stdcxx::shared_ptr<tht::TTransportFactory> transportFactory(
new tht::TBufferedTransportFactory());
th::stdcxx::shared_ptr<thp::TProtocolFactory> protocolFactory(new thp::TBinaryProtocolFactory());
ths::TThreadedServer server(processor, serverTransport, transportFactory, protocolFactory);
// start in another thread
std::thread t(&ths::TThreadedServer::serve, &server);
t.detach();
std::this_thread::sleep_for(std::chrono::seconds(5));
// stop in this thread
server.stop();
std::this_thread::sleep_for(std::chrono::seconds(5));
So what I simply do is start the server with serve() in another thread, then wait for some time, and stop it. I ran this with thread sanitizer, and got a few thread safety warnings. I mention 2 here:
First: thrift/lib/cpp/src/thrift/transport/TServerSocket.cpp:244, at:
interruptableChildren_ = enable;
Second: thrift/lib/cpp/src/thrift/transport/TServerSocket.cpp:654, at:
if (-1 == send(notifySocket, cast_sockopt(&byte), sizeof(int8_t), 0)) {
GlobalOutput.perror("TServerSocket::notify() send() ", THRIFT_GET_SOCKET_ERROR);
}
So is what I'm doing correct? And is TThreadedServer controller thread-safe? Thread-sanitizer doesn't seem to think so, although the test program works with no problems.
I'm using Thrift 0.12.0.
It's thread-safe, but it might have multi-threading bugs that will never manifest in practice. For example, in case of interruptableChildren_ - the flow is that you might configure its value in the main thread but then it's read by the acceptor thread (where TServerSocket::acceptImpl runs). In theory, you write and read from the unprotected variable. In practice, you never change it after you launch your server with
std::thread t(&ths::TThreadedServer::serve, &server);
line, so no data races will occur.
I am guessing notify() case is similar.
I am writing a server that accepts data from a device and processes it. Everything works fine unless there is an interruption in the network (i.e., if I unplug the Ethernet cable, then reconnect it). I'm using read_until() because the protocol that the device uses terminates the packet with a specific sequence of bytes. When the data stream is interrupted, read_until() blocks, as expected. However when the stream starts up again, it remains blocked. If I look at the data stream with Wireshark, the device continues transmitting and each packet is being ACK'ed by the network stack. But if I look at bytes_readable it is always 0. How can I detect the interruption and how to re-establish a connection to the data stream? Below is a code snippet and thanks in advance for any help you can offer. [Go easy on me, this is my first Stack Overflow question....and yes I did try to search for an answer.]
using boost::asio::ip::tcp;
boost::asio::io_service IOservice;
tcp::acceptor acceptor(IOservice, tcp::endpoint(tcp::v4(), listenPort));
tcp::socket socket(IOservice);
acceptor.accept(socket);
for (;;)
{
len = boost::asio::read_until(socket, sbuf, end);
// Process sbuf
// etc.
}
Remember, the client initiates a connection, so the only thing you need to achieve is to re-create the socket and start accepting again. I will keep the format of your snippet but I hope your real code is properly encapsulated.
using SocketType = boost::asio::ip::tcp::socket;
std::unique_ptr<SocketType> CreateSocketAndAccept(
boost::asio::io_service& io_service,
boost::asio::ip::tcp::acceptor& acceptor) {
auto socket = std::make_unique<boost::asio::ip::tcp::socket>(io_service);
boost::system::error_code ec;
acceptor.accept(*socket.get(), ec);
if (ec) {
//TODO: Add handler.
}
return socket;
}
...
auto socket = CreateSocketAndAccept(IOservice, acceptor);
for (;;) {
boost::system::error_code ec;
auto len = boost::asio::read_until(*socket.get(), sbuf, end, ec);
if (ec) // you could be more picky here of course,
// e.g. check against connection_reset, connection_aborted
socket = CreateSocketAndAccept(IOservice, acceptor);
...
}
Footnote: Should go without saying, socket needs to stay in scope.
Edit: Based on the comments bellow.
The listening socket itself does not know whether a client is silent or whether it got cut off. All operations, especially synchronous, should impose a time limit on completion. Consider setting SO_RCVTIMEO or SO_KEEPALIVE (per socket, or system wide, for more info How to use SO_KEEPALIVE option properly to detect that the client at the other end is down?).
Another option is to go async and implement a full fledged "shared" socket server (BOOST example page is a great start).
Either way, you might run into data consistency issues and be forced to deal with it, e.g. when the client detects an interrupted connection, it would resend the data. (or something more complex using higher level protocols)
If you want to stay synchronous, the way I've seen things handled is to destroy the socket when you detect an interruption. The blocking call should throw an exception that you can catch and then start accepting connections again.
for (;;)
{
try {
len = boost::asio::read_until(socket, sbuf, end);
// Process sbuf
// etc.
}
catch (const boost::system::system_error& e) {
// clean up. Start accepting new connections.
}
}
As Tom mentions in his answer, there is no difference between inactivity and ungraceful disconnection so you need an external mechanism to detect this.
If you're expecting continuous data transfer, maybe a timeout per connection on the server side is enough. A simple ping could also work. After accepting a connection, ping your client every X seconds and declare the connection dead if he doesn't answer.
I am trying to integrate a websocketpp server into a multithreaded project. Everything works fine in a single thread approach, but I encountered a problem when creating a separate boost::thread for endpoint.listen() that would run in the background (so it does not disrupt the execution of the main thread). I have tried the code with Boost v1.46.1 and v1.50.0 on Ubuntu 12.04 64-bit with the newest build of websocketpp. Below is a code sample and an explanation of my approach.
#include <websocketpp/websocketpp.hpp>
#include <boost/thread.hpp>
#include <boost/bind.hpp>
#include <exception>
using websocketpp::server;
class echo_server_handler : public server::handler {
public:
void on_message(connection_ptr con, message_ptr msg) {
con->send(msg->get_payload(),msg->get_opcode());
std::cout << "Got message: " << msg->get_payload() << std::endl;
}
};
int main(int argc, char* argv[]) {
unsigned short port = 9002;
try {
server::handler::ptr h(new echo_server_handler());
server echo_endpoint(h);
echo_endpoint.alog().unset_level(websocketpp::log::alevel::ALL);
echo_endpoint.elog().unset_level(websocketpp::log::elevel::ALL);
echo_endpoint.alog().set_level(websocketpp::log::alevel::CONNECT);
echo_endpoint.alog().set_level(websocketpp::log::alevel::DISCONNECT);
echo_endpoint.elog().set_level(websocketpp::log::elevel::RERROR);
echo_endpoint.elog().set_level(websocketpp::log::elevel::FATAL);
std::cout << "Starting WebSocket echo server on port " << port << std::endl;
//Getting pointer to the right function
void(websocketpp::role::server<websocketpp::server>::*f)(uint16_t,size_t) =
&websocketpp::role::server<websocketpp::server>::listen;
std::cout << "Starting WSServer thread... \n" << std:endl;
boost::shared_ptr<boost::thread> ptr(new boost::thread(boost::bind(f, &echo_endpoint, port, 1)));
//ptr->join();
} catch (std::exception& e) {
std::cerr << "Exception: " << e.what() << std::endl;
}
//Simulating processing in the main thread
while(true) {std::cout << "Main thread processing..."<<std::endl; sleep(5);}
return 0;
}
If I compile the code with ptr->join(); the listening thread works fine, but it makes the main thread sleep. If I leave ptr->join() out and let the listening thread run in background, I encounter an error after the thread creation:
/usr/local/boost_1_50_0/libbin/include/boost/thread/pthread/recursive_mutex.hpp:105:
void boost::recursive_mutex::lock(): Assertion
`!pthread_mutex_lock(&m)' failed.
I'm not very experienced with threading or boost threads, and quite new with websocketpp, so I'm not sure if I'm doing something wrong here. If there are any better (and working) ways to tackle this issue, I would love to see some examples. I have been trying to figure out this problem for a long time now, so any help would be priceless. Thanks in advance!
Check out also: gdb stacktrace and valgrind results
Edit:
The "while(true)" in the code sample is there just to simulate the processing in the main thread. I'm integrating a websocket server in a big project that has different types of socket connections, events, data processing, client synchronization etc. running in the background. The websocket connection provides just another way to connect to the server using a web client instead a native one. The main thread creates all the necessary stuff, but I can't really affect in which order they are created, so the websocket server must be started in its own thread.
You create all the objects within the scope of try/catch. When you leave this scope, these objects get destroyed.
So, either move the object definitions out of try/catch, or move while(true) loop into it.
Why are you creating the boost::thread on the heap, when it could be on the stack?
You don't need to use boost::bind with boost::thread, so it should be simply:
boost::thread t(f, &echo_endpoint, port, 1);
Much simpler, no?
As for your program's behaviour. If you call ptr->join() there then the main thread waits for the other thread to finish, which never happens, so of course it sleeps. If you don't join it then ptr and echo_endpoint and h all go out of scope. The other thread will then be trying to use objects which no longer exist.
As #IgorR. said, you should put the while loop inside the try-catch block, so the work in the main loop can happen before the other thread and the objects it uses go out of scope.
From Boost 1.50 the boost::thread destructor matches the behaviour of std::thread i.e. it calls terminate() if the thread is joinable when its destructor runs. This is to prevent the sort of error you have, where the thread continues running even though the boost::thread handle referring to it and other stack objects no longer exist. If you want it to keep running you must detach it explicitly (but in your program that would still be wrong, because the echo_endpoint and h objects would still cease to exist and the thread would still try to use them.) So before a boost::thread object goes out of scope you should either join it or detach it.
In our application we use Boost libraries (and ASIO for network communications).
Recently, we discovered that if we're sending our data from different threads via same socket, our client application is receiving garbaged data.
Small test to highlight the issue:
#include <stdio.h>
#include <boost/thread.hpp>
#include <boost/asio.hpp>
void send_routine(boost::shared_ptr<boost::asio::ip::tcp::socket> s, char c)
{
std::vector<char> data(15000, c);
data.push_back('\n');
for (int i=0; i<1000; i++)
boost::asio::write(*s, boost::asio::buffer(&data[0], data.size()));
}
int main()
{
using namespace boost::asio;
using namespace boost::asio::ip;
try {
io_service io_service;
io_service::work work(io_service);
const char* host = "localhost";
const char* service_name = "18000";
tcp::resolver resolver(io_service);
tcp::resolver::query query(tcp::v4(), host, service_name);
tcp::resolver::iterator iterator = resolver.resolve(query);
auto socket = boost::shared_ptr<tcp::socket>(new tcp::socket(io_service));
socket->connect(*iterator);
boost::thread t1(send_routine, socket, 'A');
boost::thread t2(send_routine, socket, 'B');
boost::thread t3(send_routine, socket, 'C');
t1.join();
t2.join();
t3.join();
}
catch (std::exception& e) {
printf("FAIL: %s\n", e.what());
}
return 0;
}
So, we create socket here, connect to localhost:18000 and start 3 threads which will write to the socket.
In different terminal window, I run nc -l -p 18000 | tee out.txt | sort | uniq | wc -l. I expect 3 as output, but it returns more then 100 "different strings" in the network stream (so, data is corrupted). But it works with small buffer sizes (if we'll change 15000 to 80, for example).
So, the question is: is it a correct behavior of ASIO library? And another: how to fix it? Should I use mutex inside my send_routine function (or there is another solution)?
write and async_write are not thread safe in the manner you are using them. The canonical way to approach this is to queue your messages, then write them out one at a time.
Yes there is another solution !
Strands: Use Threads Without Explicit Locking. Be care that strands only provides "atomic" access to socket for the "event handlers", of course you need to use asio "event handlers" which is not the case of your code. In other words you need to use boost::asio::async_write instead of boost::asio::write.
Well according to the documentation tcp::socket is not thread safe when shared between multiple threads.
So you either do a synchronisation like you suggested with boost::mutex or you use async write. The io_service the work for you.
You might have two problems, the threading issue could be solve for example by having one thread dedicated to writing and a queue where all threads post there response. You can also change your design to an asynchronous one and use the write_some() function and let the threading be done by the io_service::run(), which can be run by more than one thread.
Second, you might have a protocol problem, if a client expects the answers to it's questions in the same order.
hth
Torsten