I did check the example reverse_client_cb with the reverse_worker as a worker.
The worker is getting called and executed but not a single callback ( complete,fail,etc) is getting called in the client.
Below is a sample worker and client that I wrote to check the same, but, same problem.
Is there any configuration that's need to be done in gearman or in the way the worker and clients are executed?
sampleworker.cc
void* myfunc(gearman_job_st *job, void *data,size_t *size,gearman_return_t *ret)
{
printf("\nmyfunc called\n");
*ret=GEARMAN_SUCCESS;
char *result=(char*)calloc(30,sizeof(char));
strcpy(result,"work completed");
*size=30;
return result;
}
int main(int args,char* argv[])
{
gearman_worker_st worker;
gearman_worker_create(&worker);
gearman_worker_add_server(&worker,"localhost",0);
gearman_return_t ret = gearman_worker_add_function(&worker,"sample",0,myfunc,NULL);
while(1) gearman_worker_work(&worker);
gearman_worker_free(&worker);
return 0;
}
sampleclient.cc
static gearman_return_t complete(gearman_task_st *task)
{
printf("Completed: %s %.*s\n", gearman_task_job_handle(task),
(int)gearman_task_data_size(task), (char *)gearman_task_data(task));
return GEARMAN_SUCCESS;
}
static gearman_return_t fail(gearman_task_st *task)
{
printf("Failed: %s\n", gearman_task_job_handle(task));
return GEARMAN_SUCCESS;
}
int main(int args,char* argv[])
{
gearman_client_st client;
gearman_return_t ret;
gearman_client_create(&client);
gearman_client_add_server(&client,"localhost",0);
gearman_task_st task,*task2;
gearman_client_add_task(&client,&task,NULL,"sample",NULL,argv[1],(size_t)strlen(argv[1]),&ret);
gearman_client_set_complete_fn(&client, &complete);
gearman_client_set_fail_fn(&client, &fail);
gearman_client_run_tasks(&client);
gearman_client_free(&client);
return 0;
}
usage
./sampleworker
./sampleclient Hello
"myfunc called" getting printed and the client is also waiting till the worker completes the job. But nothing is printed from complete or fail functions
got it.
I removed the following libs which was installed and now I'm getting all the callbacks.
libgearman-client-async-perl - Asynchronous client for gearman distributed job system
libgearman-client-perl - Client for gearman distributed job system.
in my opinion. your made some mistake in function invoking sequence.
gearman_client_set_complete_fn(&client, &complete);
gearman_client_set_fail_fn(&client, &fail);
// the function must be called after set_complete_fn
gearman_client_add_task
gearman_client_run_tasks(&client);
Related
I want to run boost::asio::io_service.run() in a background thread. So when I need it post() func into.
This is main func:
int main(int /*argc*/, char** /*argv*/)
{
std::string message = "hello";
logg = new logger_client(filename,ip,13666);
logg->start();
while (true)
logg->add_string(message);
return 0;
}
And some relevant funcs from logger_client:
std::auto_ptr<boost::asio::io_service::work> work;
logger_client::logger_client(std::string& filename,std::string& ip, uint16_t port) : work(new boost::asio::io_service::work(io_service))
{
}
void logger_client::start()
{
ios_thread = new boost::thread(boost::bind(&io_service.run,&io_service));
}
void print_nothing()
{
printf("%s\n","lie");
}
void logger_client::add_string(std::string& message)
{
io_service.post(boost::bind(print_nothing));
//io_service.post(strand->wrap(boost::bind(&logger_client::add_string_imp,this,message)));
//io_service.run();
}
When i run this, my program eats 2Gb less than a minute. If i remove endless work and change to this:
void logger_client::add_string(std::string& message)
{
io_service.post(boost::bind(print_nothing));
//io_service.post(strand->wrap(boost::bind(&logger_client::add_string_imp,this,message)));
io_service.run();
}
Program works just fine. But I don't want to invoke async operations on this (main) thread. What am i doing wrong?
UPDATE
I added sleep(1sec) in while(true) loop and memory is no longer growing. But this is not a solution. Because if I call run() after post() (i.e. use main thread for processing handles) and even add five more threads with while(true) loops memory is not growing. So why main thread is so much better than newly created? I also tried thread pool for io_service::run - did not help.
io_service.run will exit unless there are pending operations.
Therefore, your ios_thread will exit immediately.
The solution is to use io_service::work.
In addition, endless loop spam like this
while (true)
logg->add_string(message);
is not a good idea, maybe add some sleep(), to slow it down a bit and keep it under control.
I would like to create a c++ webserver that will perform a task for each user that lands on my website. Since the task might be computationally heavy (for now just a long sleep), I'd like to handle each user on a different thread. I'm using mongoose to set up a webserver.
The different processes (in my code below just one, aka server1) are set up correctly and seem to function correctly. However, the threads seem to be queuing one after the other so if 2 users hit the end point, the second user must wait until the first user finishes. What am I missing? Do the threads run out of scope? Is there a "thread-manager" that I should be using?
#include "../../mongoose.h"
#include <unistd.h>
#include <iostream>
#include <stdlib.h>
#include <thread>
//what happens whenever someone lands on an endpoint
void myEvent(struct mg_connection *conn){
//long delay...
std::thread mythread(usleep, 2*5000000);
mythread.join();
mg_send_header(conn, "Content-Type", "text/plain");
mg_printf_data(conn, "This is a reply from server instance # %s",
(char *) conn->server_param);
}
static int ev_handler(struct mg_connection *conn, enum mg_event ev) {
if (ev == MG_REQUEST) {
myEvent(conn);
return MG_TRUE;
} else if (ev == MG_AUTH) {
return MG_TRUE;
} else {
return MG_FALSE;
}
}
static void *serve(void *server) {
for (;;) mg_poll_server((struct mg_server *) server, 1000);
return NULL;
}
int main(void) {
struct mg_server *server1;
server1 = mg_create_server((void *) "1", ev_handler);
mg_set_option(server1, "listening_port", "8080");
mg_start_thread(serve, server1);
getchar();
return 0;
}
Long running requests should be handled like this:
static void thread_func(struct mg_connection *conn) {
sleep(60); // simulate long processing
conn->user_data = "done"; // Production code must not do that.
// Other thread must never access connection
// structure directly. This example is just
// for demonstration.
}
static int ev_handler(struct mg_connection *conn, enum mg_event ev) {
switch (ev) {
case MG_REQUEST:
conn->user_data = "doing...";
spawn_thread(thread_func, conn);
return MG_MORE; // Important! Signal Mongoose we are not done yet
case MG_POLL:
if (conn->user_data != NULL && !strcmp(conn->user_data, "done")) {
mg_printf(conn, "HTTP/1.0 200 OK\n\n Done !");
return MG_TRUE; // Signal we're finished. Mongoose can close this connection
}
return MG_FALSE; // Still not done
Caveat: I'm not familiar with mongoose
My assumptions:
The serve function is polling for incoming connections
If the thread executing mg_poll_server is the same thread that triggers the call to ev_handler then your problem is the fact that ev_handler calls myEvent which starts a long running operation and blocks the thread (i.e., by calling join). In this case you're also blocking the thread which is handling the incoming connections (i.e., A subsequent client must wait for the first client to finish their work), which seems is the behavior you describe seeing.
I'm not sure what the real task is supposed to do so I can't say for sure how you should fix this. Perhaps in your use-case it may be possible to call detach otherwise you might keep track of executing threads and defer calling join on them until the server is shutdown.
James Adkison is absolutely right. So, if instead the beginning of the code looks like this:
void someFunc(struct mg_connection *conn){
usleep(2*5000000);
std::cout << "hello!" << std::endl;
std::cout<< "This finished from server instance #"<<conn<<std::endl;
mg_send_header(conn, "Content-Type", "application/json");
mg_printf_data(conn, "{\"message\": \"This is a reply from server instance # %s\"}",
// (char *) conn->server_param);
}
void myEvent(struct mg_connection *conn){
std::thread mythread(someFunc,conn);
mythread.detach();
std::cout<< "This is a reply from server instance #"<<(char *) conn->server_param<<std::endl;
}
static int ev_handler(struct mg_connection *conn, enum mg_event ev) {
if (ev == MG_REQUEST) {
myEvent(conn);
return MG_TRUE;
} else if (ev == MG_AUTH) {
//.... exactly as before
//....
then the program works. Basically the difference is replacing .join() with .detach(). someFunc is running now in parallel for 2 users -- so that's great!. Thanks!
I'm trying to create a multithread server application using mongoose web server library.
I have main thread serving connections and sending requests to processors that are working in their own threads. Then processors place results into queue and queue observer must send results back to clients.
Sources are looking that way:
Here I prepare the data for processors and place it to queue.
typedef std::pair<struct mg_connection*, const char*> TransferData;
int server_app::event_handler(struct mg_connection *conn, enum mg_event ev)
{
Request req;
if (ev == MG_AUTH)
return MG_TRUE; // Authorize all requests
else if (ev == MG_REQUEST)
{
req = parse_request(conn);
task_queue->push(TransferData(conn,req.second));
mg_printf(conn, "%s", ""); // (1)
return MG_MORE; // (2)
}
else
return MG_FALSE; // Rest of the events are not processed
}
And here I'm trying to send the result back. This function is working in it's own thread.
void server_app::check_results()
{
while(true)
{
TransferData res;
if(!res_queue->pop(res))
{
boost::this_thread::sleep_for(boost::chrono::milliseconds(100));
continue;
}
mg_printf_data(res.first, "%s", res.second); // (3)
}
}
The problem is a client doesn't receive anything from the server.
If I run check_result function manualy in the event_handler after placing a task into the queue and then pass computed result back to event_handler, I'm able to send it to client using mg_printf_data (with returning MG_TRUE). Any other way - I'm not.
What exactly should I change in this sources to make it works?
Ok... It looks like I've solved it myself.
I'd been looking into mongoose.c code and an hour later I found the piece of code below:
static void write_terminating_chunk(struct connection *conn) {
mg_write(&conn->mg_conn, "0\r\n\r\n", 5);
}
static int call_request_handler(struct connection *conn) {
int result;
conn->mg_conn.content = conn->ns_conn->recv_iobuf.buf;
if ((result = call_user(conn, MG_REQUEST)) == MG_TRUE) {
if (conn->ns_conn->flags & MG_HEADERS_SENT) {
write_terminating_chunk(conn);
}
close_local_endpoint(conn);
}
return result;
}
So I've tried to do mg_write(&conn->mg_conn, "0\r\n\r\n", 5); after line (3) and now it's working.
If there is something more difficult than debugging a multithreaded app that is trying to describe the bug itself.
I have two boost::threads (application and display).
Both use the same asio::io_service to do their work.
The display thread has a std::set of type window* which is a class I use to wrap winapi window management.
I use a custom message queue to communicate these two threads.
One of these messages (terminate) is used to notify the display thread that it must not "post" any more methods and that it must call thread_group.remove_thread and remove itself.
The thread has a variable (state) that flags the state of the thread (running, paused, terminated).
If it is running it "posts" it's update() method that iterates an std::set and calls the update method in each window* it contains.
If it is terminated, it clears the std::set, removes itself from the thread_group and doesn't post any more work.
The problem: Once a while, when trying to close the app, the thread's update method gets ran after the thread got "terminated" and the std::set got cleared. Then the update method tries to iterate the std::set and a SIGSEGV takes place. This only happens 1 every 10 runs of the application and I'm having a hard time trying to guess what's wrong.
I'll try to post the relevant code, if more is needed I'll try to add it.
int main(int argc, char **argv)
{
boost::asio::io_service ios;
boost::asio::strand strand(ios);
boost::thread_group threads;
owl::system::pump pump;
application app(&threads, &strand, &pump);
owl::system::display display(&strand, &pump);
ios.run();
threads.join_all();
return 0;
}
...
void display::on_terminate()
{
close_all_windows();
}
...
void display::close_all_windows()
{
windows.move_first();
while (!windows.eof())
{
window* win = windows.value();
win->destroy();
delete win;
windows.move_next();
}
windows.clear();
check_no_window();
}
...
void display::on_update()
{
if (windows.size())
{
windows.move_first();
while (!windows.eof())
{
windows.value()->update();
windows.move_next(); // Here happens the SIGSEGV
}
}
}
The class display inherits the class subsystem that manages thread execution. This is the relevant code involving the execution of on_update()
void subsystem::do_update()
{
message* msg;
size_t message_count = messages.size();
for (size_t i=0; i<message_count; i++)
{
msg = messages[i];
process_message(msg);
strand->dispatch(strand->wrap(boost::bind(&message::deallocate, msg)));
}
switch (state)
{
case running:
{
on_update();
}
break;
case paused:
{
// Do not update. Just check the queue and sleep
sleep(10);
}
break;
case terminated:
{
do_terminate();
return;
}
break;
}
strand->post(strand->wrap(boost::bind(&subsystem::check_for_messages, this)));
}
void subsystem::check_for_messages()
{
messages.clear();
pump->get_messages(this, messages);
ios->post(boost::bind(&subsystem::do_update, this));
}
The SIGSEGV occurs exactly when trying to increment the std::set iterator.
Child process PID: 2272
Program received signal SIGSEGV, Segmentation fault.
In std::_Rb_tree_increment(std::_Rb_tree_node_base const*) ()
stl_tree.h:269
I'm using gSoap to write a webservice. It's running as a console application. In all gSoap examples I see, that requests are dispatched in infinite loop like for(;;;) even in multi-threaded version.
But how can I make my webservice to terminate gracefully when, say, user presses space on the console?
Preferably:
stop accepting new connections;
Serve existing ones;
Exit from application
The only solution I came up so far is using timeouts
soap->recv_timeout = 20;
soap->send_timeout = 20;
soap->connect_timeout = 5;
soap->accept_timeout = 5;
Then all blocking functions return periodically. But this is not ideal for me, because I want to be able to terminate the app quickly even if there is an ongoing transmission, but at the same time, don't want to compromise reliability on a slow/flaky connection (it's an embedded device connected via GPRS).
The section 7.2.4 How to Create a Multi-Threaded Stand-Alone Service in the documentation has example code for writing an accept loop. You need to write your own accept loop and add signal handling so it responds to Ctrl-C.
stop accepting new connections:
Leave the loop so you stop calling accept.
Serve existing ones:
The threads need to inform you when they are finished, so you can exit when the number of active clients is zero. (boost::thead_group has a join_all which does exactly that.)
Exit from application:
What you need to do is register signal handler so when you terminate your application using Ctrl + C, it calls you registered function where you can gracefully terminates.
e.g
class gsoap_test {
public:
void start() {
running_ = true;
while(running_) {
//gsoap threads
}
//stop and cleanup
}
void stop() {
running_ = false;
}
private:
bool running_;
};
//global variable
gsoap_test gsoap;
void sighandler(int sig)
{
std::cout<< "Signal caught..." << std::endl;
//Stop gracefully here
gsoap.stop();
exit(0);
}
int main(int argc, char** argv) {
//register signal
signal(SIGABRT, &sighandler);
signal(SIGTERM, &sighandler);
signal(SIGINT, &sighandler);
gsoap.start();
return EXIT_SUCCESS;
}