I'd like to use the TTheadedServer in a separate thread to have control on when to stop/start it. My application needs only 1 controlling thread and one processing thread. I don't expect to have more than one client as I'm using thrift as a relay. TSimpleServer is not thread-safe, so I dropped that option.
I made a little minimal example to check whether it's thread-safe, and used clang's thread-sanitizer to make sure it's thread-safe. Here's the example
std::shared_ptr<MyHandler> handler = std::make_shared<MyHandler>();
int port = 9090;
th::stdcxx::shared_ptr<th::TProcessor> processor(new HandlerProcessor(handler));
th::stdcxx::shared_ptr<tht::TServerTransport> serverTransport(new tht::TServerSocket(port));
th::stdcxx::shared_ptr<tht::TTransportFactory> transportFactory(
new tht::TBufferedTransportFactory());
th::stdcxx::shared_ptr<thp::TProtocolFactory> protocolFactory(new thp::TBinaryProtocolFactory());
ths::TThreadedServer server(processor, serverTransport, transportFactory, protocolFactory);
// start in another thread
std::thread t(&ths::TThreadedServer::serve, &server);
t.detach();
std::this_thread::sleep_for(std::chrono::seconds(5));
// stop in this thread
server.stop();
std::this_thread::sleep_for(std::chrono::seconds(5));
So what I simply do is start the server with serve() in another thread, then wait for some time, and stop it. I ran this with thread sanitizer, and got a few thread safety warnings. I mention 2 here:
First: thrift/lib/cpp/src/thrift/transport/TServerSocket.cpp:244, at:
interruptableChildren_ = enable;
Second: thrift/lib/cpp/src/thrift/transport/TServerSocket.cpp:654, at:
if (-1 == send(notifySocket, cast_sockopt(&byte), sizeof(int8_t), 0)) {
GlobalOutput.perror("TServerSocket::notify() send() ", THRIFT_GET_SOCKET_ERROR);
}
So is what I'm doing correct? And is TThreadedServer controller thread-safe? Thread-sanitizer doesn't seem to think so, although the test program works with no problems.
I'm using Thrift 0.12.0.
It's thread-safe, but it might have multi-threading bugs that will never manifest in practice. For example, in case of interruptableChildren_ - the flow is that you might configure its value in the main thread but then it's read by the acceptor thread (where TServerSocket::acceptImpl runs). In theory, you write and read from the unprotected variable. In practice, you never change it after you launch your server with
std::thread t(&ths::TThreadedServer::serve, &server);
line, so no data races will occur.
I am guessing notify() case is similar.
Related
I'm referring to this sample program from the Beast repository: https://www.boost.org/doc/libs/1_67_0/libs/beast/example/http/server/fast/http_server_fast.cpp
I've made some changes to the code to check the ability to process multiple requests simultaneously.
boost::asio::io_context ioc{1};
tcp::acceptor acceptor{ioc, {address, port}};
std::list<http_worker> workers;
for (int i = 0; i < 10; ++i)
{
workers.emplace_back(acceptor, doc_root);
workers.back().start();
}
ioc.run();
My understanding with the above is that I will now have 10 worker objects to run I/O, i.e. handle incoming connections.
So, my first question is the above understanding correct?
Assuming that the above is correct, I've made some changes to the lambda (handler) passed to the tcp::acceptor:
void accept()
{
// Clean up any previous connection.
boost::beast::error_code ec;
socket_.close(ec);
buffer_.consume(buffer_.size());
acceptor_.async_accept(
socket_,
[this](boost::beast::error_code ec)
{
if (ec)
{
accept();
}
else
{
boost::system::error_code ec2;
boost::asio::ip::tcp::endpoint endpoint = socket_.remote_endpoint(ec2);
// Request must be fully processed within 60 seconds.
request_deadline_.expires_after(
std::chrono::seconds(60));
std::cerr << "Remote Endpoint address: " << endpoint.address() << " port: " << endpoint.port() << "\n";
read_request();
}
});
}
And also in process_request():
void process_request(http::request<request_body_t, http::basic_fields<alloc_t>> const& req)
{
switch (req.method())
{
case http::verb::get:
std::cerr << "Simulate processing\n";
std::this_thread::sleep_for(std::chrono::seconds(30));
send_file(req.target());
break;
default:
// We return responses indicating an error if
// we do not recognize the request method.
send_bad_response(
http::status::bad_request,
"Invalid request-method '" + req.method_string().to_string() + "'\r\n");
break;
}
}
And here's my problem: If I send 2 simultaneous GET requests to my server, they're being processed sequentially, and I know this because the 2nd "Simulate processing" statement is printed ~30 seconds after the previous one which would mean that execution gets blocked on the first thread.
I've tried to read the documentation of boost::asio to better understand this, but to no avail.
The documentation for acceptor::async_accept says:
Regardless of whether the asynchronous operation completes immediately or not, the handler will not be >invoked from within this function. Invocation of the handler will be performed in a manner equivalent to >using boost::asio::io_service::post().
And the documentation for boost::asio::io_service::post() says:
The io_service guarantees that the handler will only be called in a thread in which the run(), >run_one(), poll() or poll_one() member functions is currently being invoked.
So, if 10 workers are in the run() state, then why would the two requests get queued?
And also, is there a way to workaround this behavior without adapting to a different example? (e.g. https://www.boost.org/doc/libs/1_67_0/libs/beast/example/http/server/async/http_server_async.cpp)
io_context does not create threads internally to execute the tasks, but rather uses the threads that call io_context::run explicitly. In the example the io_context::run is called just from one thread (main thread). So you have just one thread for task executions, which (thread) gets blocked in sleep and there is no other thread to execute other tasks.
To make this example work you have to:
Add more thread into the pool (like in the second example you referred to)
size_t const threads_count = 4;
std::vector<std::thread> v;
v.reserve(threads_count - 1);
for(size_t i = 0; i < threads_count - 1; ++i) { // add thraed_count threads into the pool
v.emplace_back([&ioc]{ ioc.run(); });
}
ioc.run(); // add the main thread into the pool as well
Add synchronization (for example, using strand like in the second example) where it is needed (at least for socket reads and writes), because now your application is multi-threaded.
UPDATE 1
Answering to the question "What is the purpose of a list of workers in the Beast example (the first one that referred) if in fact io_context is only running on one thread?"
Notice, regardless of thread count IO operations here are asynchronous, meaning http::async_write(socket_...) does not block the thread. And notice, that I explain here the original example (not your modified version). One worker here deals with one round-trip of 'request-response'. Imagine the situation. There are two clients client1 and client2. Client1 has poor internet connection (or requests a very big file) and client2 has the opposite conditions. Client1 makes request. Then client2 makes request. So if there was just one worker client2 would had to wait until client1 finished the whole round-trip 'request-response`. But, because there are more than one workers client2 gets response immediately not waiting the client1 (keep in mind IO does not block your single thread). The example is optimized for situation where bottleneck is IO but not the actual work. In your modified example you have quite the opposite situation - the work (30s) is very expensive compared to IO. For that case better use the second example.
I am trying to create a concurrent c++ TCP server using threads. In particular I was wondering if I could use std::async to accept connections and serve each one in its own thread.
So far I have created a rough mockup but can't really tell if I am on the correct path.
void networking::TCP_Server::acceptConnection() {
std::string stringToSend{"This is a test string to be replied to"};
int new_fd = accept(listeningFD, nullptr, nullptr);
send(new_fd, stringToSend.c_str(), stringToSend.size(), 0);
sleep(3);
std::cout << ("End of thread");
}
///LISTEN FOR CONNECTIONS ON listeningFD
///CREATE A LIST OF FILE DESCRIPTORS FOR POLL fds[]
(fds[i].fd == listeningFD) {
do {
std::cout << ("New incoming connection - %d\n", new_fd);
std::async(std::launch::async, acceptConnection)
} while (new_fd != -1);
} /* End of existing connection is readable */
} /* End of loop through pollable descriptors */
I am connecting at the same time to the server with two clients and would expect for the loop to run through both new connections and create a thread for each one. As of now it is as it runs in deferred mode, one gets accepted, the other waits until the first finishes.
Any ideas?
(Pardon any mistakes in the code)
std::async returns a std::future which the code doesn't save into a variable, hence its destructor is called immediately. std::future::~future() blocks the calling thread until the future becomes ready.
You may like to use (detached) std::thread instead of std::async.
There are more scalable strategies to handle many clients. I highly recommend reading old but instructive The C10K problem.
You may also like to get familar with Asio C++ Library.
I am implementing multithreaded C++ program for Linux platform where I need a functionality similar to WaitForMultipleObjects().
While searching for the solution I observed that there are articles that describe how to achieve WaitForMultipleObjects() functionality in Linux with examples but those examples does not satisfy the scenario that I have to support.
The scenario in my case is pretty simple. I have a daemon process in which the main thread exposes a method/callback to the outside world for example to a DLL. The code of the DLL is not under my control. The same main thread creates a new thread "Thread 1". Thread 1 has to execute kind of an infinite loop in which it would wait for a shutdown event (daemon shutdown) OR it would wait on the data available event being signaled through the exposed method/callback mentioned above.
In short the thread would be waiting on shutdown event and data available event where if shutdown event is signaled the wait would satisfy and the loop would be broken or if data available event is signaled then also wait would satisfy and thread would do business processing.
In windows, it seems very straight forward. Below is the MS Windows based pseudo code for my scenario.
//**Main thread**
//Load the DLL
LoadLibrary("some DLL")
//Create a new thread
hThread1 = __beginthreadex(..., &ThreadProc, ...)
//callback in main thread (mentioned in above description) which would be called by the DLL
void Callbackfunc(data)
{
qdata.push(data);
SetEvent(s_hDataAvailableEvent);
}
void OnShutdown()
{
SetEvent(g_hShutdownEvent);
WaitforSingleObject(hThread1,..., INFINITE);
//Cleanup here
}
//**Thread 1**
unsigned int WINAPI ThreadProc(void *pObject)
{
while (true)
{
HANDLE hEvents[2];
hEvents[0] = g_hShutdownEvent;
hEvents[1] = s_hDataAvailableEvent;
//3rd parameter is set to FALSE that means the wait should satisfy if state of any one of the objects is signaled.
dwEvent = WaitForMultipleObjects(2, hEvents, FALSE, INFINITE);
switch (dwEvent)
{
case WAIT_OBJECT_0 + 0:
// Shutdown event is set, break the loop
return 0;
case WAIT_OBJECT_0 + 1:
//do business processing here
break;
default:
// error handling
}
}
}
I want to implement the same for Linux. According to my understanding when it would come to Linux, it has totally different mechanism where we need to register for signals. If the termination signal arrives, the process would come to know that it is about to shutdown but before that it is necessary for the process to wait for the running thread to gracefully shutdown.
The correct way to do this in Linux would be using condition variables. While this is not the same as WaitForMultipleObjects in Windows, you will get the same functionality.
Use two bools to determine whether there is data available or a shutdown must occur.
Then have the shutdown function and the data function both set the bools accordingly, and signal the condition variable.
#include <pthread.h>
pthread_cond_t cv = PTHREAD_COND_INITIALIZER;
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_t hThread1; // this isn't a good name for it in linux, you'd be
// better with something line "tid1" but for
// comparison's sake, I've kept this
bool shutdown_signalled;
bool data_available;
void OnShutdown()
{
//...shutdown behavior...
pthread_mutex_lock(&mutex);
shutdown_signalled = true;
pthread_mutex_unlock(&mutex);
pthread_cond_signal(&cv);
}
void Callbackfunc(...)
{
// ... whatever needs to be done ...
pthread_mutex_lock(&mutex);
data_available = true;
pthread_mutex_unlock(&mutex);
pthread_cond_signal(&cv);
}
void *ThreadProc(void *args)
{
while(true){
pthread_mutex_lock(&mutex);
while (!(shutdown_signalled || data_available)){
// wait as long as there is no data available and a shutdown
// has not beeen signalled
pthread_cond_wait(&cv, &mutex);
}
if (data_available){
//process data
data_available = false;
}
if (shutdown_signalled){
//do the shutdown
pthread_mutex_unlock(&mutex);
return NULL;
}
pthread_mutex_unlock(&mutex); //you might be able to put the unlock
// before the ifs, idk the particulars of your code
}
}
int main(void)
{
shutdown_signalled = false;
data_available = false;
pthread_create(&hThread1, &ThreadProc, ...);
pthread_join(hThread1, NULL);
//...
}
I know windows has condition variables as well, so this shouldn't look too alien. I don't know what rules windows has about them, but on a POSIX platform the wait needs to be inside of a while loop because "spurious wakeups" can occur.
If you wish to write unix or linux specific code, you have differenr APIs available:
pthread: provides threads, mutex, condition variables
IPC (inter process comunication) mechanisms : mutex, semaphore, shared memory
signals
For threads, the first library is mandatory (there are lower level syscalls on linux, but it's more tedious). For events, the three may be used.
The system shutdown event generate termination (SIG_TERM) and kill (SIG_KILL) signals broadcasted to all the relevant processes. Hence an individual daemon shutdown can also be initiated this way. The goal of the game is to catch the signals, and initiate process shutdown. The important points are:
the signal mechanism is made in such a way that it is not necessary to wait for them
Simply install a so called handler using sigaction, and the system will do the rest.
the signal is set to the process, and any thread may intercept it (the handler may execute in any context)
You need therefore to install a signal handler (see sigaction(2)), and somehow pass the information to the other threads that the application must terminate.
The most convenient way is probably to have a global mutex protected flag which all your threads will consult regularily. The signal handler will set that flag to indicate shutdown. For the worker thread, it means
telling the remote host that the server is closing down,
close its socket on read
process all the remaining received commands/data and send answers
close the socket
exit
For the main thread, this will mean initiating a join on the worker thread, then exit.
This model should not interfer with the way data is normally processed: a blocking call to select or poll will return the error EINTR if a signal was caught, and for a non blocking call, the thread is regularily checking the flag, so it does work too.
Ive currently been messing around with boost trying it out. When i try to make a simple multi threaded echo server it exits when receiving with error code 3. I have looked over the documentation many times and still no luck. I know it is probably something very simple i'm overlooking. I have decent experience with winsock but i would like to learn the boost library.
here is the code thats failing i took out the
typedef boost::shared_ptr<tcp::socket> socket_ptr;
boost::asio::io_service io;
boost::array<char, 512> buf;
void startserver ( std::string host, std::string port )
{
tcp::acceptor a (io, tcp::endpoint(tcp::v4(), atoi(port.c_str())));
for(;;)
{
socket_ptr sock (new tcp::socket(io));
a.accept(*sock);
std::cout << sock->remote_endpoint() << std::endl;
boost::thread t (boost::bind(session, sock));
}
}
void session ( socket_ptr sock )
{
sock->send(boost::asio::buffer("welcome"),0,er);
size_t len;
for(;;)
{
len = sock->receive(boost::asio::buffer(buf));
sock->send(boost::asio::buffer(buf,len),0,er);
}
}
I can connect to it fine with netcat and it receives the welcome message but right when it goes to receive it crashs. Ive tried catching an error using boost::system::error_code on each one but nothing was returned
There are too many issues. Check asio documentation for correct examples. Some of issues:
Creating boost::thread object t and then immediately exit scope. This deattaches thread and it not controllable now; or, as mentioned Joachim Pileborg it can terminate (im not very familiar with boost::threads, so correct me if i wrong).
Right after this you starting new acceptor. You should hold only 1 acceptor per listening port.
No point to create thread for this at all, it is ASIO, use async ;)
receive does not wait data, it just fetch packet data ASIO already had (what is not true in this case)
Check examples at boost site, i think your case is blocking tcp echo server
It's most likely because the thread goes out of scope. From the manual page of the boost::thread destructor:
If the thread is joinable calls to std::terminate. Destroys *this.
This means that when the thread is started it might run for a little while before the thread in startserver gets control again and the thread object is destructed and your thread is terminated.
I am trying to integrate a websocketpp server into a multithreaded project. Everything works fine in a single thread approach, but I encountered a problem when creating a separate boost::thread for endpoint.listen() that would run in the background (so it does not disrupt the execution of the main thread). I have tried the code with Boost v1.46.1 and v1.50.0 on Ubuntu 12.04 64-bit with the newest build of websocketpp. Below is a code sample and an explanation of my approach.
#include <websocketpp/websocketpp.hpp>
#include <boost/thread.hpp>
#include <boost/bind.hpp>
#include <exception>
using websocketpp::server;
class echo_server_handler : public server::handler {
public:
void on_message(connection_ptr con, message_ptr msg) {
con->send(msg->get_payload(),msg->get_opcode());
std::cout << "Got message: " << msg->get_payload() << std::endl;
}
};
int main(int argc, char* argv[]) {
unsigned short port = 9002;
try {
server::handler::ptr h(new echo_server_handler());
server echo_endpoint(h);
echo_endpoint.alog().unset_level(websocketpp::log::alevel::ALL);
echo_endpoint.elog().unset_level(websocketpp::log::elevel::ALL);
echo_endpoint.alog().set_level(websocketpp::log::alevel::CONNECT);
echo_endpoint.alog().set_level(websocketpp::log::alevel::DISCONNECT);
echo_endpoint.elog().set_level(websocketpp::log::elevel::RERROR);
echo_endpoint.elog().set_level(websocketpp::log::elevel::FATAL);
std::cout << "Starting WebSocket echo server on port " << port << std::endl;
//Getting pointer to the right function
void(websocketpp::role::server<websocketpp::server>::*f)(uint16_t,size_t) =
&websocketpp::role::server<websocketpp::server>::listen;
std::cout << "Starting WSServer thread... \n" << std:endl;
boost::shared_ptr<boost::thread> ptr(new boost::thread(boost::bind(f, &echo_endpoint, port, 1)));
//ptr->join();
} catch (std::exception& e) {
std::cerr << "Exception: " << e.what() << std::endl;
}
//Simulating processing in the main thread
while(true) {std::cout << "Main thread processing..."<<std::endl; sleep(5);}
return 0;
}
If I compile the code with ptr->join(); the listening thread works fine, but it makes the main thread sleep. If I leave ptr->join() out and let the listening thread run in background, I encounter an error after the thread creation:
/usr/local/boost_1_50_0/libbin/include/boost/thread/pthread/recursive_mutex.hpp:105:
void boost::recursive_mutex::lock(): Assertion
`!pthread_mutex_lock(&m)' failed.
I'm not very experienced with threading or boost threads, and quite new with websocketpp, so I'm not sure if I'm doing something wrong here. If there are any better (and working) ways to tackle this issue, I would love to see some examples. I have been trying to figure out this problem for a long time now, so any help would be priceless. Thanks in advance!
Check out also: gdb stacktrace and valgrind results
Edit:
The "while(true)" in the code sample is there just to simulate the processing in the main thread. I'm integrating a websocket server in a big project that has different types of socket connections, events, data processing, client synchronization etc. running in the background. The websocket connection provides just another way to connect to the server using a web client instead a native one. The main thread creates all the necessary stuff, but I can't really affect in which order they are created, so the websocket server must be started in its own thread.
You create all the objects within the scope of try/catch. When you leave this scope, these objects get destroyed.
So, either move the object definitions out of try/catch, or move while(true) loop into it.
Why are you creating the boost::thread on the heap, when it could be on the stack?
You don't need to use boost::bind with boost::thread, so it should be simply:
boost::thread t(f, &echo_endpoint, port, 1);
Much simpler, no?
As for your program's behaviour. If you call ptr->join() there then the main thread waits for the other thread to finish, which never happens, so of course it sleeps. If you don't join it then ptr and echo_endpoint and h all go out of scope. The other thread will then be trying to use objects which no longer exist.
As #IgorR. said, you should put the while loop inside the try-catch block, so the work in the main loop can happen before the other thread and the objects it uses go out of scope.
From Boost 1.50 the boost::thread destructor matches the behaviour of std::thread i.e. it calls terminate() if the thread is joinable when its destructor runs. This is to prevent the sort of error you have, where the thread continues running even though the boost::thread handle referring to it and other stack objects no longer exist. If you want it to keep running you must detach it explicitly (but in your program that would still be wrong, because the echo_endpoint and h objects would still cease to exist and the thread would still try to use them.) So before a boost::thread object goes out of scope you should either join it or detach it.