The active object design pattern as I understand is tying up a (private/dedicated) thread life time with an object and making it work on independent data. From some of the documentation I read , the evolution of this kind of paradigm was because of two reasons , first , managing raw threads would be pain and second more threads contending for the shared resource doesn't scale well using mutex and locks. while I agree with the first reason , I do not fully comprehend the second . Making an object active just makes the object independent but the problems like contention for lock/mutex is still there (as we still have shared queue/buffer), the object just delegated the sharing responsibility onto the message queue. The only advantage of this design pattern as i see is the case where I had to perform long asynch task on the shared object (now that i am just passing message to a shared queue , the threads no longer have to block for long on mutex/locks but they still will blocka and contend for publishing messages/task). Other than this case could someone tell more scenarios where this kind of design pattern will have other advantages.
The second question I have is (I just started digging around design patterns) , what is the conceptual difference between , active object , reactor and proactor design pattern . How do you decide in which design pattern is more efficient and suits your requirements more. It would be really nice if someone can demonstrate certain examples showing how the three design patterns will behave and which one has comparative advantage/disadvantage in different scenarios.
I am kind of confused as I have used active object (which used shared thread-safe buffer) and boost::asio(Proactor) both to do similar kind of async stuff , I would like to know if any one has more insights on applicability of different patterns when approaching a problem.
The ACE website has some very good papers on the Active Object, Proactor en Reactor design patterns. A short summary of their intents:
The Active Object design pattern decouples method execution
from method invocation to enhance concurrency and
simplify synchronized access to an object that resides in its
own thread of control. Also known as: Concurrent Object, Actor.
The Proactor pattern supports the demultiplexing and dispatching
of multiple event handlers, which are triggered by the completion
of asynchronous events. This pattern is heavily used in Boost.Asio.
The Reactor design pattern handles service requests that are delivered
concurrently to an application by one or more clients. Each service
in an application may consist of several methods and is represented by
a separate event handler that is responsible for dispatching service-specific
requests. Also known as: Dispatcher, Notifier.
Related
I want to understand what is true-asio way to use shared data?
reading the asio and the beast examples, the only example of using shared data is http_crawl.cpp. (perhaps I missed something)
in that example the shared object is only used to collect statistics for sessions, that is the sessions do not read that object's data.
as a result I have three questions:
Is it implied that interaction with shared data in asio-style is an Active Object? i.e. should mutexes be avoided?
whether the statement will be correct that for reading the shared data it is also necessary to use "requests" to Active Object, and also no mutexes?
has anyone tried to evaluate the overhead of "requests" to Active Object, compared to using mutexes?
Is it implied that interaction with shared data in asio-style is an Active Object? i.e. should mutexes be avoided?
Starting at the end, yes mutexes should be avoided. This is because all service handlers (initiations and completions) will be executed on the service thread(s) which means that blocking in a handler will block all other handlers.
Whether that leads to Active Object seems to be a choice to me. Yes, a typical approach would be like Active Object (see e.g. boost::asio and Active Object), where operations queue for the data.
However, other approaches are viable and frequently seen, like e.g. the data being moving with their task(s) e.g. through a task flow.
whether the statement will be correct that for reading the shared data it is also necessary to use "requests" to Active Object, and also no mutexes?
Yes, synchronization needs to happen for shared state, regardless of the design pattern chosen (although some design pattern reduce sharing alltogether).
The Asio approach is using strands, which abstract away the scheduling from the control flow. This gives the service the option to optimize for various cases (e.g. continuation on the same strand, the case where there's only one service thread anyway etc.).
has anyone tried to evaluate the overhead of "requests" to Active Object, compared to using mutexes?
Lots of people and lots of times. Often are wary of trying Asio because "it uses locking internally". If you know what you're doing, throughput can be excellent, which goes for most patterns and industrial-strength frameworks.
Specific benchmarks depend heavily on specific implementation choices. I'm pretty sure you can find examples on github, blogs and perhaps even on this site.
(perhaps I missed something)
You're missing the fact that all IO objects are not thread-safe, which means that they themselves are shared data for any composed asynchronous operation (chain)
My current application owns multiple «activatable» objects*. My intent is to "run" all those object in the same io_context and to add the necessary protection in order to toggle from single to multiple threads (to make it scalable)
If these objects were completely independent from each others, the number of threads running the associated io_context could grow smoothly. But since those objects need to cooperate, the application crashes in multithread despite the strand in each object.
Let's say we have objects of type A and type B, all of them served by the same io_context. Each of those types run asynchronous operations (timers and sockets - their handlers are surrounded with bind_executor(strand, handler)), and can build a cache based on information received via sockets and posted operations to them. Objects of type A needs to get information cached from multiple instances of B in order to perform their own work.
Would it be possible to access this information by using strands (without adding explicit mutex protection) and if yes how ?
If not, what strategy could be adopted to achieve the scalability?
I already tried playing with futures but that strategy leads unsurprisingly to deadlocks.
Thanx
(*) Maybe I'm wrong in the terminology: objects get a reference to an io_context and own their own strand, so I think they are activatable, because they don't own really a running thread
You're mixing vague words a bit. "Activatable", "Strandify", "inter coorporating". They're all close to meaningful concepts, yet, narrowly avoid binding to any precise meaning.
Deconstructing
Let's simplify using more precise concepts.
Let's say we have objects of type A and type B, all of them served by the same io_context
I think it's more fruitful to say "types A and B have associated executors". When you make sure all operations on A and B operate from that executor and you make sure that executor serializes access, then you basically get the Active Object pattern.
[can build a cache based on information received via sockets] and posted operations to them
That's interesting. I take that to mean you don't directly call members of the class, unless they defer the actual execution to the strand. This, again, would be the Active Object.
However, your symptoms suggest that not all operations are "posted to them". Which implies they run on arbitrary threads, leading to your problem.
Would it be possible to access this information by using strands (without adding explicit mutex protection) and if yes how ?
The key to your problems is here. Data dependencies. It's also, ;ole;y going to limit the usefulness of scaling, unless of course the generation of information to retrieve from other threads is a computationally expensive operation.
However, in the light of the phrase _"to get information cached from multiple instances of B'" suggests that in fact, the data is instantaneous, and you'll just be paying synchronization costs for accessing across threads.
Questions
Q. Would it be possible to access this information by using strands (without adding explicit mutex protection) and if yes how ?
Technically, yes. By making sure all operations go on the strand, and the objects become true active objects.
However, there's an important caveat: strands aren't zero-cost. Only in certain contexts they can be optimized (e.g. in immediate continuations or when the execution context has no concurrency).
But in all other contexts, they end up synchronizing at similar cost as mutexes. The purpose of a strand is not to remove the lock contention. Instead it rather allows one to declaratively specify the synchronization requirements for tasks, so that so that the same code can be correctly synchronized regardless of the methods of async completion (using callbacks, futures, coroutines, awaitables, etc) or the chosen execution context(s).
Example: I recently uncovered a vivid illustration of the cost of strand synchronization even in a simple context (where serial execution was already implicitly guaranteed) here:
sehe mar 15, 23:08 Oh cool. The strands were unnecessary. I add them for safety until I know it's safe to go without. In this case the async call chains form logical strands (there are no timers or full duplex sockets going on, so it's all linear). That... improves the situation :)
Now it's 3.5gbps even with the 1024 byte server buffer
The throughput increased ~7x from just removing the strand.
Q. If not, what strategy could be adopted to achieve the scalability?
I suspect you really want caches that contain shared_futures. So that the first retrieval puts the future for the result in cache, where subsequent retrievals get the already existing shared future immediately.
If you make sure your cache lookup datastructure is threadsafe, likely with a reader/writer lock (shared_mutex), you will be free to access it with minimal overhead from any actor, instead of requiring to go through individual strands of each producer.
Keep in mind that awaiting futures is a blocking operation. So, if you do that from tasks posted on the execution context, you may easily run out of threads. In such cases it maybe better to provide async_get in terms of boost::asio::async_result or boost::asio::async_completion so you can wait in non-blocking fashion.
Is there a way for a thread-pool to cancel a task underway? Better yet, is there a safe alternative for on-demand cancelling opaque function calls in thread_pools?
Killing the entire process is a bad idea and using native handle to perform pthread_cancel or similar API is a last resort only.
Extra
Bonus if the cancellation is immediate, but it's acceptable if the cancellation has some time constraint 'guarantees' (say cancellation within 0.1 execution seconds of the thread in question for example)
More details
I am not restricted to using Boost.Thread.thread_pool or any specific library. The only limitation is compatibility with C++14, and ability to work on at least BSD and Linux based OS.
The tasks are usually data-processing related, pre-compiled and loaded dynamically using C-API (extern "C") and thus are opaque entities. The aim is to perform compute intensive tasks with an option to cancel them when the user sends interrupts.
While launching, the thread_id for a specific task is known, and thus some API can be sued to find more details if required.
Disclaimer
I know using native thread handles to cancel/exit threads is not recommended and is a sign of bad design. I also can't modify the functions using boost::this_thread::interrupt_point, but can wrap them in lambdas/other constructs if that helps. I feel like this is a rock and hard place situation, so alternate suggestions are welcome, but they need to be minimally intrusive in existing functionality, and can be dramatic in their scope for the feature-set being discussed.
EDIT:
Clarification
I guess this should have gone in the 'More Details' section, but I want it to remain separate to show that existing 2 answers are based o limited information. After reading the answers, I went back to the drawing board and came up with the following "constraints" since the question I posed was overly generic. If I should post a new question, please let me know.
My interface promises a "const" input (functional programming style non-mutable input) by using mutexes/copy-by-value as needed and passing by const& (and expecting thread to behave well).
I also mis-used the term "arbitrary" since the jobs aren't arbitrary (empirically speaking) and have the following constraints:
some which download from "internet" already use a "condition variable"
not violate const correctness
can spawn other threads, but they must not outlast the parent
can use mutex, but those can't exist outside the function body
output is via atomic<shared_ptr> passed as argument
pure functions (no shared state with outside) **
** can be lambda binding a functor, in which case the function needs to makes sure it's data structures aren't corrupted (which is the case as usually, the state is a 1 or 2 atomic<inbuilt-type>). Usually the internal state is queried from an external db (similar architecture like cookie + web-server, and the tab/browser can be closed anytime)
These constraints aren't written down as a contract or anything, but rather I generalized based on the "modules" currently in use. The jobs are arbitrary in terms of what they can do: GPU/CPU/internet all are fair play.
It is infeasible to insert a periodic check because of heavy library usage. The libraries (not owned by us) haven't been designed to periodically check a condition variable since it'd incur a performance penalty for the general case and rewriting the libraries is not possible.
Is there a way for a thread-pool to cancel a task underway?
Not at that level of generality, no, and also not if the task running in the thread is implemented natively and arbitrarily in C or C++. You cannot terminate a running task prior to its completion without terminating its whole thread, except with the cooperation of the task.
Better
yet, is there a safe alternative for on-demand cancelling opaque
function calls in thread_pools?
No. The only way to get (approximately) on-demand preemption of a specific thread is to deliver a signal to it (that is is not blocking or ignoring) via pthread_kill(). If such a signal terminates the thread but not the whole process then it does not automatically make any provision for freeing allocated objects or managing the state of mutexes or other synchronization objects. If the signal does not terminate the thread then the interruption can produce surprising and unwanted effects in code not designed to accommodate such signal usage.
Killing the entire process is a bad idea and using native handle to
perform pthread_cancel or similar API is a last resort only.
Note that pthread_cancel() can be blocked by the thread, and that even when not blocked, its effects may be deferred indefinitely. When the effects do occur, they do not necessarily include memory or synchronization-object cleanup. You need the thread to cooperate with its own cancellation to achieve these.
Just what a thread's cooperation with cancellation looks like depends in part on the details of the cancellation mechanism you choose.
Cancelling a non cooperative, not designed to be cancelled component is only possible if that component has limited, constrained, managed interactions with the rest of the system:
the ressources owned by the components should be managed externally (the system knows which component uses what resources)
all accesses should be indirect
the modifications of shared ressources should be safe and reversible until completion
That would allow the system to clean up resource, stop operations, cancel incomplete changes...
None of these properties are cheap; all the properties of threads are the exact opposite of these properties.
Threads only have an implied concept of ownership apparent in the running thread: for a deleted thread, determining what was owned by the thread is not possible.
Threads access shared objects directly. A thread can start modifications of shared objects; after cancellation, such modifications that would be partial, non effective, incoherent if stopped in the middle of an operation.
Cancelled threads could leave locked mutexes around. At least subsequent accesses to these mutexes by other threads trying to access the shared object would deadlock.
Or they might find some data structure in a bad state.
Providing safe cancellation for arbitrary non cooperative threads is not doable even with very large scale changes to thread synchronization objects. Not even by a complete redesign of the thread primitives.
You would have to make thread almost like full processes to be able to do that; but it wouldn't be called a thread then!
Our (Windows native C++) app is composed of threaded objects and managers. It is pretty well written, with a design that sees Manager objects controlling the lifecycle of their minions. Various objects dispatch and receive events; some events come from Windows, some are home-grown.
In general, we have to be very aware of thread interoperability so we use hand-rolled synchronization techniques using Win32 critical sections, semaphores and the like. However, occasionally we suffer thread deadlock during shut-down due to things like event handler re-entrancy.
Now I wonder if there is a decent app shut-down strategy we could implement to make this easier to develop for - something like every object registering for a shutdown event from a central controller and changing its execution behaviour accordingly? Is this too naive or brittle?
I would prefer strategies that don't stipulate rewriting the entire app to use Microsoft's Parallel Patterns Library or similar. ;-)
Thanks.
EDIT:
I guess I am asking for an approach to controlling object life cycles in a complex app where many threads and events are firing all the time. Giovanni's suggestion is the obvious one (hand-roll our own), but I am convinced there must be various off-the-shelf strategies or frameworks, for cleanly shutting down active objects in the correct order. For example, if you want to base your C++ app on an IoC paradigm you might use PocoCapsule instead of trying to develop your own container. Is there something similar for controlling object lifecycles in an app?
This seems like a special case of the more general question, "how do I avoid deadlocks in my multithreaded application?"
And the answer to that is, as always: make sure that any time your threads have to acquire more than one lock at a time, that they all acquire the locks in the same order, and make sure all threads release their locks in a finite amount of time. This rule applies just as much at shutdown as at any other time. Nothing less is good enough; nothing more is necessary. (See here for a relevant discussion)
As for how to best do this... the best way (if possible) is to simplify your program as much as you can, and avoid holding more than one lock at a time if you can possibly help it.
If you absolutely must hold more than one lock at a time, you must verify your program to be sure that every thread that holds multiple locks locks them in the same order. Programs like helgrind or Intel thread checker can help with this, but it often comes down to simply eyeballing the code until you've proved to yourself that it satisfies this constraint. Also, if you are able to reproduce the deadlocks easily, you can examine (using a debugger) the stack trace of each deadlocked thread, which will show where the deadlocked threads are forever-blocked at, and with that information, you can that start to figure out where the lock-ordering inconsistencies are in your code. Yes, it's a major pain, but I don't think there is any good way around it (other than avoiding holding multiple locks at once). :(
One possible general strategy would be to send an "I am shutting down" event to every manager, which would cause the managers to do one of three things (depending on how long running your event-handlers are, and how much latency you want between the user initiating shutdown, and the app actually exiting).
1) Stop accepting new events, and run the handlers for all events received before the "I am shutting down" event. To avoid deadlocks you may need to accept events that are critical to the completion of other event handlers. These could be signaled by a flag in the event or the type of the event (for example). If you have such events then you should also consider restructuring your code so that those actions are not performed through event handlers (as dependent events would be prone to deadlocks in ordinary operation too.)
2) Stop accepting new events, and discard all events that were received after the event that the handler is currently running. Similar comments about dependent events apply in this case too.
3) Interrupt the currently running event (with a function similar to boost::thread::interrupt()), and run no further events. This requires your handler code to be exception safe (which it should already be, if you care about resource leaks), and to enter interruption points at fairly regular intervals, but it leads to the minimum latency.
Of course you could mix these three strategies together, depending on the particular latency and data corruption requirements of each of your managers.
As a general method, use an atomic boolean to indicate "i am shutting down", then every thread checks this boolean before acquiring each lock, handling each event etc. Can't give a more detailed answer unless you give us a more detailed question.
I'd like a large list so I can reference this for ideas. Some answers already have been enlightening .
What are some concurrency models? I heard of message passing where there is no memory shared. Futures which returns an object right away (so it doesn't block) and allows you to dereference the original function returns value later when you need it blocking if the results are not ready yet. I heard of coroutines, software transactional memory and random others.
I searched for a list or a wiki and couldn't find any good ones (many did not list the 3 I mentioned above) and many results gave me a complicated description explaining how it works rather then what it does or how it is to be used.
What are some concurrency models and what is a simple description of what they do? One per answer.
Actor Model
I heard of message passing where there is no memory shared.
Is it about Erlang-style Actors?
Scala uses this idea in its Actors framework (thus, in Scala its not a part of the language, just a library) and it looks quite sexy!
In a few words Actors are objects that have no shared data at all, but can use async messages for interaction. Actors can be located on one or different hosts and use interesting error handling policy (when error happened - actor just dies).
You should read more on this in Erlang and Scala docs, its really straightforward and progressive approach!
Chapters 3, 17, 17.11:
http://www.scala-lang.org/sites/default/files/linuxsoft_archives/docu/files/ScalaByExample.pdf
https://en.wikipedia.org/wiki/Actor_model
COM Threading (Concurrency) Model
Single-Threaded Apartments
Multi-Threaded Apartments
Mixed Model Development
COM objects can be used in multiple threads of a process. The terms
"Single- threaded Apartmen*t" (STA) and
"*Multi-threaded Apartment" (MTA) are
used to create a conceptual framework
for describing the relationship
between objects and threads, the
concurrency relationships among
objects, the means by which method
calls are delivered to an object, and
the rules for passing interface
pointers among threads. Components and
their clients choose between the
following two apartment models
presently supported by COM:
Single-threaded Apartment model (STA):
One or more threads in a process use
COM and calls to COM objects are
synchronized by COM. Interfaces are
marshaled between threads. A
degenerate case of the single-threaded
apartment model, where only one thread
in a given process uses COM, is called
the single-threading model. Previous
Microsoft information and
documentation has sometimes referred
to the STA model simply as the
"apartment model." Multi-threaded
Apartment model (MTA): One or more
threads use COM and calls to COM
objects associated with the MTA are
made directly by all threads
associated with the MTA without any
interposition of system code between
caller and object. Because multiple
simultaneous clients may be calling
objects more or less simultaneously
(simultaneously on multi-processor
systems), objects must synchronize
their internal state by themselves.
Interfaces are not marshaled between
threads. Previous Microsoft
information and documentation has
sometimes referred to this model as
the "free-threaded model." Both the
STA model and the MTA model can be
used in the same process. This is
sometimes referred to as a
"mixed-model" process.
Other models according to Wikipedia
There are several models of concurrent
computing, which can be used to
understand and analyze concurrent
systems. These models include:
Actor model
Object-capability model for security
Petri nets
Process calculi such as
Ambient calculus
Calculus of Communicating Systems (CCS)
Communicating Sequential Processes (CSP)
π-calculus
Futures
A future is a place-holder for the
undetermined result of a (concurrent)
computation. Once the computation
delivers a result, the associated
future is eliminated by globally
replacing it with the result value.
That value may be a future on its own.
Whenever a future is requested by a
concurrent computation, i.e. it tries
to access its value, that computation
automatically synchronizes on the
future by blocking until it becomes
determined or failed.
There are four kinds of futures:
concurrent futures stand for the result of a concurrent computation,
lazy futures stand for the result of a computation that is only performed on request,
promised futures stand for a value that is promised to be delivered later by explicit means,
failed futures represent the result of a computation that terminated with an exception.
Software transactional memory
In computer science, software
transactional memory (STM) is a
concurrency control mechanism
analogous to database transactions for
controlling access to shared memory in
concurrent computing. It is an
alternative to lock-based
synchronization. A transaction in this
context is a piece of code that
executes a series of reads and writes
to shared memory. These reads and
writes logically occur at a single
instant in time; intermediate states
are not visible to other (successful)
transactions. The idea of providing
hardware support for transactions
originated in a 1986 paper and patent
by Tom Knight[1]. The idea was
popularized by Maurice Herlihy and J.
Eliot B. Moss[2]. In 1995 Nir Shavit
and Dan Touitou extended this idea to
software-only transactional memory
(STM)[3]. STM has recently been the
focus of intense research and support
for practical implementations is
growing.
There's also map/reduce.
The idea is to spawn many instances of a sub problem and to combine the answers when they're done. A simple example would be matrix multiplication, which is the sum of several dot products. You spawn a worker thread for each dot product, and when all the threads are finished you sum the result.
This is how GPUs, functional languages such as LISP/Scheme/APL, and some frameworks (Google's Map/Reduce) handle concurrency.
Coroutines
In computer science, coroutines are
program components that generalize
subroutines to allow multiple entry
points for suspending and resuming
execution at certain locations.
Coroutines are well-suited for
implementing more familiar program
components such as cooperative tasks,
iterators, infinite lists and pipes.
There's also non-blocking concurrency such as compare-and-swap and load-link/store-conditional instructions. For example, compare-and-swap (cas) could be defined as so:
bool cas( int new_value, int current_value, int * location );
This operation will then attempt to set the value at location to the value passed in new_value, but only if the value in location is the same as current_value. This only requires one instruction and is usually how blocking concurrency (mutexes/semaphores/etc.) are implemented.
IPC (including MPI and RMI)
Hi,
in the wiki pages you can find that MPI (message passing interface) is a methods of a general IPC technique: http://en.wikipedia.org/wiki/Inter-process_communication
Another interesting approach is a Remote procedure call. For example Java's RMI enables you
to focus only on your application domain and communication patterns. It's an "application level" concurrency.
http://www.oracle.com/technetwork/java/javase/tech/index-jsp-136424.html
There a various design patterns/tools available to aid in shared memory model prallelization. Apart from the mentioned futures one can also take advantage of:
1. Thread pool pattern - focuses on task distribution between fixed number of threads: http://en.wikipedia.org/wiki/Thread_pool_pattern
2. Scheduler pattern - controls the threads execution according to a chosen scheduling policy http://en.wikipedia.org/wiki/Scheduler_pattern
3. Reactor pattern - to embed a single threaded application in a parallel environment http://en.wikipedia.org/wiki/Reactor_pattern
4. OpenMP (allows to parallelize part of code by means of preprocessor pragmas)
Regards,
Marcin
Parallel Random Access Machine (PRAM) is useful for complexity/tractability isues (please refer to a nice book for details).
About models you will also find something here (by Blaise Barney)
How about tuple space?
A tuple space is an implementation of the associative memory paradigm
for parallel/distributed computing. It provides a repository of tuples
that can be accessed concurrently. As an illustrative example,
consider that there are a group of processors that produce pieces of
data and a group of processors that use the data. Producers post their
data as tuples in the space, and the consumers then retrieve data from
the space that match a certain pattern. This is also known as the
blackboard metaphor. Tuple space may be thought as a form of
distributed shared memory.
LMAX's disruptor pattern keeps data in place and assures only one thread (consumer or producer) is owner of a data item (=queue slot) at a time.