Seems like all the examples always show running the same io_service in all threads.
Can you start multiple io_services? Here is what I would like to do:
Start io_service A in the main thread for handling user input...
Start another io_service B in another thread that then can start a bunch of worker
threads all sharing io_service B.
Users on io_service A can "post" work on io_service B so that it gets done on the worker pool but no work is to be done on io_service A, i.e. the main thread.
Is this possible? Does this make sense?
Thanks
In my experience, it really depends on the application if an io_service per cpu or one per process is better performing. There was a discussion on the asio-users mailing list a few years ago on this very topic.
The Boost.Asio documentation has some great examples showing these two techniques in the HTTP Server 2 and HTTP Server 3 examples. But keep in mind the second HTTP server just shows how to use this technique, not when or why to use it. Those questions will need to be answered by profiling your application.
In general, you should use the following order when creating applications using Boost.Asio
Single threaded
Thread pool with a single io_service
Multiple io_service objects with some sort of CPU affinity
Good question!
Yes, it is possible for one. In an application I'm currently working on I have broken up the application into separate components responsible for different aspects of the system. Each component runs in its own thread, has its own set of timers, does its own network I/O using asio. From a testability/design perspective, it seems more clean to me, since no component can interfere with another, but I stand to be corrected. I suppose I could rewrite everything passing in the io service as a parameter, but currently haven't found the need to do so.
So coming back to your question, you can do whatever you want, IMO it's more a case of try it out and change it if you run into any issues.
Also, you might want to take a look at what Sam Miller pointed out in a different post WRT handling user input ( that is if you're using a console): https://stackoverflow.com/questions/5210796/boost-asio-how-to-write-console-server
Related
My boost server accidentally stopped accepting incoming connections because some other guy from my team created yet another server using boost acceptor in a different thread (using different port)? Is it normal and how make them two servers work independently and not messing with each other?
SOLVED: acceptors had nothing to do with it, the guy started an infinite loop somewhere that blocked other components. I guess that is what happens when the team is working uncoordinated :( Sorry guys, sehe's the best as always
We're using multiple acceptors with a single io_service just fine, as by design.
Also, we're sharing out work across multiple other io_service instances, using the same sockets, just fine, as by design.
What could be happening in your code base would be antipatterns: if people call stop() on your io_service instance then yes that would wreak havoc on any other async operations also queued on the same instance.
So, in general, the idea would be to avoid using stop() or similar "life-time" operations on a shared io_service instance. The only appropriate time for such a call would be during a forced shutdown sequence, but really graceful shutdown should let all active connections shutdown and the pending work to be drained, so that threads running io_service::run would spontaneously complete anyways.
See also:
Why would we need many acceptors in the Boost.ASIO?
How to design proper release of a boost::asio socket or wrapper thereof
I am working on a RPC framework, I want to use a multi io_service design to decouple the io_objects that perform the IO (front-end) from the the threads that perform the RPC work (the back-end).
The front-end should be single threaded and the back-end should have a thread pool. I was considering a design to get the front-end and back-end to synchronise using a condition variables. However, it seems boost::thread and boost::asio do not comingle --i.e., it seems condition variable async_wait support is not available. I have a question open on this matter here.
It occured to me that io_service::post() might be used to synchronise the two io_service objects. I have attached a diagram below, I just want to know if I understand the post mechanism correctly, and weather this is a sensible implementation.
I assume that you use "a single io_service and a thread pool calling io_service::run()"
Also I assume that your frond-end is single-threaded just to avoid a race condition writing from multiple threads to the same socket.
The same goal can be achieved using io_service::strand (tutorial).Your front-end can be MT synchronized by io_service::strand. All posts from back-end to front-end (and handlers from front-end to front-end like handle_connect etc.) should be wrapped by strand, something like this:
back-end -> front-end:
io_service.post(front_end.strand.wrap(
boost::bind(&Front_end::send_response, front_end_ptr)));
or front-end -> front-end:
socket.async_connect(endpoint, strand.wrap(
boost::bind(&Front_end::handle_connect, shared_from_this(),
boost::asio::placeholders::error)));
And all posts from front-end to back-end shouldn't be wrapped by strand.
If you back-end is a thread pool calling any of the io_service::run(), io_service::run_one(), io_service::poll(), io_service::poll_one() functions and your handler(s) require access to shared resources then you still have to take care to lock those shared resources somehow in the handler's themselves.
Given the limited amount of information posted in the question, I would assume this would work fine given the caveat above.
However, when posting there is some measurable overhead for setting up the necessary completion ports and waiting -- overhead you could avoid using a different implementation of your back end "queue".
Without knowing the exact details of what you need to accomplish, I would suggest that you look into thread building blocks for pipelines or perhaps more simply for a concurrent queue.
Currently I´m not sure, I try to make a high-performance server, I got a 6Core CPU, so if I would use the "io_service_per_cpu" design, I have 6 io_service´s.
I already heard that the threadpool design isn´t the best one, but I´m not sure about that.
What knowledge do you have? Someone already made a Stress test with each, or something else?
In my experience it is vastly easier to approach asynchronous application design with the following order:
single thread and a single io_service
multiple threads, each invoking io_service::run() from a single io_service. Use strands for handlers that require access to shared data structures.
io_service per cpu
The motivation for changing between each of these designs should be done after profiling your application. Note that the HTTP Server 2 example only shows how to use an io_service per CPU, it does not show you when or why to use such a design.
Another good way to approach this: start up multiple copies of your process and bind each one to a different core using your OS'es. For FreeBSD, use cpuset. The OS is going to do a better job than any userland code will. Then you need to use an external load balancer to distribute load across your server instances. Extra points for binding a NIC interrupts to a particular CPU.
I've begun using Boost.ASIO for some simple network programming, my understanding of the library is not a great deal, so please bear with me and my newbie question.
At the moment in my project I only have 1 io_service object. Which use for all the async I/O operations etc.
My understanding is that one can create multiple threads and pass the run method of an io_service instance to the thread to provide more threads to the io_service.
My question: Is it good design to have multiple io_service objects? say for example have 2 distinct io_service instances, each with 2 threads associated, do they somehow know about each other (and hence cooperate with each), or if not would they negatively affect each other?
My intention is to have 1 io_service for socket based I/O and another for serial based (tty) I/O.
We use multiple io_service's because some of the components in our application need to run all their worker threads at certain fixed priorities, different for each component. Thus each component is given its own io_service, and each component has its own pool of threads executing run().
Other designs I could think of would be if a different number of threads in the pool is required for each IO, or, more relevant to your case, is if the pool cannot be shared because, for example, if your network IO can take out every thread and leave your serial IO waiting.
IIRC, during Michael Caisse's Boostcon ASIO talk (which is worth watching anyway), I believe this question is explicitly asked by an audience member and ok'd as a potential solution. I take from that that it's not wrong per se, and can be used that way according to your design.
This discussion may be enlightening:
http://thread.gmane.org/gmane.comp.lib.boost.asio.user/1300
I don't have the code right here, but why would you use multiple io_services?
I thought it used one io_service and multiple threads executing run on
that one io_service.
IIUC, each io_service owns a select/epoll/whatever queue, so having multiple
io_services is akin to having multiple independent select/epoll loops. In some
situations, eg. large numbers of sockets and multiple CPUs, this might help.
Something I'm less sure about is with multiple threads all running
io_service::run (with the same io_service). I think this just means the
handlers are run concurrently, while the select/epoll/etc. loop is 'shared'.
I think this is best for when your handlers are relatively long-running
operations.
I am thinking of writing a server application - along the lines of mySQL or Apache.
The main requirements are:
Clients will communicate with the server via TCP/IP (sockets)
The server will spawn a new child process to handle requests (ala Apache)
Ideally, I would like to use the BOOST libraries rather than attempt to reinvent my own. There must be code somewhere that does most of what I am trying to do - so I can use it (or atleast part of it as my starting point) can anyone point me to a useful link?
In the (hopefully unlikely) event that there is no code I can use as a starting point, can someone point out the most appropriate BOOST libraries to use - and a general guideline on how to proceeed.
My main worry is how to know when one of the children has crashed. AFAIK, there are two ways of doing this:
Using heartbeats between the parent and children (this quickly becomes messy, and introduces more things that could go wrong)
Somehow wrap the spawning of the process with a timeout parameter - but this is a dumb approach, because if a child is carrying out time intensive work, the parent may incorrectly think that the child has died
What is the best practises of making the parent aware that a child has died?
[Edit]
BTW, I am developing/running/deploying on Linux
On what platform (Windows/Linux/both)? Processes on Windows are considered more heavy-weight than on Linux, so you may indeed consider threads.
Also, I think it is better (like Apache does) not to spawn a process for each request but to have a process pool, so you save the cost of creating a process, especially on Windows.
If you are on Linux, can waitpid() be useful for you? You can use it in the non-blocking mode to check recurrently with some interval whether one of the child processes terminated
I can say for sure that Pion is your only stable option.
I have never used it but I intend to, and the API looks very clean.
As for the Boost libraries you would need:
Boost.Asio
Boost.Threading
Boost.Spirit (or something similar to parse the HTTP protocol)
Boost.IPC
What about using threads (which are supported by Boost) rather than forking the process? This would allow you to make queries about the state of a child and, imho, threads are simpler to handle than forking.
Generally Boost.Asio is good point to begin with.
But several points to be aware of:
Boost.Asio is very good library but it is not very fork aware, so don't try to share Asio
event loop between several fork processes - this would not work (i.e. - if boost::asio::io_service was created before fork - don't use it in more then one process after it)
Also it does not allow you to release file handler from boost::asio::XX::socket
so only way is to call dup and then pass it to child process.
But to be honest? I don't think you'll find any network event loop library that is
fork aware (maybe with exception of CppCMS's booster.aio that I had written
to be fork aware by myself).
Waiting for children is quite simple you can define a signal handler with sigaction
on SIGCHLD signal that is send then child crashes or exits.
So all you need to do is handle this signal and in main loop call waitpid when such
signal received.
With asio you can use "self-pipe" trick to wake the loop from sleep from signal handler.
First, take a look at CPPCMS. It might already fit your needs.
Now, as pointed by others, boost::asio is a good starting point but is really the basics of the task.
Maybe you'll be more interested in the works being done about server-code based on boost::asio : cpp-netlib (that is made to be submitted in boost once done) The author's blog.
I've made an FOSS library for creating C++ applications in a modular way. It's hosted at
https://github.com/chilabot/chila
here's my blog: http://chilatools.blogspot.com/view/sidebar
It's specially suited for generic server creation (that was my motivation for constructing it), but I think it can be used for any kind of application.
The part that has to be deployed with the final binary is LGPL, so it can be used with commercial applications.