Managing agent thread pools in Clojure - clojure

Is there a way to control the thread pools which handle the functions which get sent to agents? As I understand things, if I send-off, underneath the hood I'm using an unbounded thread pool. I would like to, say, run some functions on one thread pool and other functions on another. The reason for this is say I have a some functions which do IO and which are also less important. I'd throw these on some bounded thread pool and wouldn't worry if there was excessive blocking and they stacked up since they're, well, less important. The main thing is that I wouldn't want their crappy IO blocking to say have an effect on some more important functions which are running on another thread pool.
I'm basing the question off of something similar I did with thread pools in Akka and I'm just wondering I can accomplish the same thing with Clojure.

For Clojure versions up to 1.4:
You cannot replace the built-in agent send and send-off thread pools. They are hard-coded in Agent.java.
The send pool (used for computation) is fixed size = 2 + Runtime.getRuntime().availableProcessors().
The send-off pool (also used for futures) is a cached thread pool and will grow without bound. This allows an arbitrary number of background tasks to wait for I/O. The cached threads will be reused and discarded if they've been idle for one minute.
If you want to manage work on your own thread pool, you'll need to dip into java.util.concurrent or use a Clojure wrapper lib.
For Clojure 1.5 (upcoming):
You can supply your own ExecutorService using (send-via executor a f), the default threadpools are not hard-wired anymore. See Agent.java in Clojure 1.5+ for details.

The Clojure library Claypoole is designed for exactly this. It lets you define threadpools and use (and reuse) them for futures, pmaps, and so on.

Amit Rathore (of Runa inc), has published a library (called medusa) for managing thread pools. It looks like a fairly close match for what you are looking for.
http://s-expressions.com/2010/06/08/medusa-0-1-a-supervised-thread-pool-for-clojure-futures-2/

Related

Node C++ module vs libuv thread pool size

I've written a Nodejs C++ module that makes use of NAN's AsyncWorker to expose async module functionality. Works great. However, I understand that AsyncWorker makes use of libuv's thread pool, which defaults to just 4 threads.
While this (or a #-of-cores based limitation) might make sense for CPU-heavy functions, some of my exposed functions may run relatively long, even though they don't use the CPU (network activity, etc). Therefore the thread pool might get all used up even though no computation-intensive work is going on.
The easy solution is to increase the thread pool size (UV_THREADPOOL_SIZE). However, I am concerned that this thread pool is used for other things as well, which might suffer from a performance hit due to too much parallelization (the libuv documentation states, "The threadpool is global and shared across all event loops...").
Is my concern valid? Is there a way to make use of a separate, larger, thread pool only for certain AsyncWorker's that are long-running but not CPU-intenstive, while leaving the common thread-pool untouched?

Notify caller that a thread has finished

I am trying to use the multithreading features in the C++11 standard library and have the following situation envisioned.
I have a parent class which maintains a queue of thread. So something like:
std::queue<MyMTObject *> _my_threads;
The class MyMTObject contains the std::thread object.
The queue has a fixed size of 5 and the class initially starts with the queue being full.
As I have jobs to process I launch threads and I remove them from the queue. What I would like is to get a notification when the job is finished along with the pointer to the MyMTObject, so that I can reinsert them into the queue and make them available again.
I have basically 2 questions:
1: Is this a sound idea? I know I have not specified specifics but broadly speaking. I will, of course, control all access to the queue with a mutex.
2: Is there a way to implement this notification mechanism without using external libraries like Qt or boost.
For duplicates, I did look on the site but could not find anything that was suitable to manage a collection of threads.
I'm not sure if I need to mention this, but std::thread objects can't be re-used. Generally, the only reason you keep a std::thread reference is to std::thread::join the thread. If you don't plan to join the thread later (e.g. dispatch to threads and wait for completion), it's generally advised to std::thread::detach it.
If you're trying to keep threads for a thread pool, it's probably easier to have each thread block on the std::queue and pull objects from the queue to work on. This is relatively easy to implement using a std::mutex and a std::condition_variable. It generally gives good throughput, but to get finer control over scheduling you can do things like keep a seperate std::queue for each thread.
Detaching the threads and creating a work queue also has the added benefit that it avoids redundantly requesting the operating system create new threads which adds overhead and increases overall resource usage.
You could try to deploy some version of Reactor pattern I think. So, you could start one additional control thread that cleans after these workers. Now, you create a ThreadSafeQueue that will be used to communicate events from worker threads to control thread. This queue should be implemented in such a way that you can select on it and wait for any activity on the other end (some thread terminates and calls queue.push for example).
All in all I think it's quite elegant solution. I does add an overhead of an additional thread, but this thread will be mostly sleeping and waking up only once a while to clean up after the worker.
There is no elegant way to do this in Posix, and C++ threading model is almost a thin wrapper on Posix.
You can join a specific thread (one at a time), or you can wait on futures - again, one future at a time.
The best you can do to avoid looping is to employ a conditional variable, and make all threads singal on it (as well as indicating which one just exited by setting some sort of per-thread flag) just before they are about to exit. The 'reaper' would notice the signal and check the flags.
The issue is that this solution requires thread cooperation. But I know not of any better.

How to have a long waiting thread in Intel TBB?

I want to create a thread or task (more than one to be exact) that goes and does some non CPU intensive work that will take a lot of time because of external causes, such a HTTP request or a file IO operation from a slow disk. I could do this with async await in C# and would be exactly what i am trying to do here. Spawn a thread or task and let it do it's own thing while i continue with execution of the program and simply let it return the result whenever ready. The problem with TBB i have is that all tasks i can make think they are created for a CPU intensive work.
Is what TBB calls GUI Thread what i want in this case ? I would need more than one, is that possible ? Can you point me to the right direction ? Should i look for another library that provides threading and is available for multiple OS ?
Any I/O blocking activity is poorly modeled by a task -- since tasks are meant to run to completion, it's just not what tasks are for. You will not find any TBB task-based approach that circumvents this. Since what you want is a thread, and you want it to work more-or-less nicely with other TBB code you already have, just use TBB's native thread class to solve the problem as you would with any other threading API. You won't need to set priority or anything else on this TBB-managed thread, because it'll get to its blocking call and then not take up any further time until the resource is available.
About the only thing I can think of specifically in TBB is that a task can be assigned a priority. But this isn't the same thing as a thread priority. TBB task priorities only dictate when a task will be selected from the ready pool, but like you said - once the task is running, it's expected to be working hard. The way to do use this to solve the problem you mentioned is to break your IO work into segments, then submit them into the work pool as a series of (dependent) low-priority tasks. But I don't think this gets to your real problem ...
The GUI Thread you mentioned is a pattern in the TBB patterns document that says how to offload a task and then wait for a callback to signal that it's complete. It's not altogether different from an async. I don't think this solves your problem either.
I think the best way for you here is to make an OS-level thread. That's pthreads on Linux or windows threads on Windows. Then you'll want to call this on it: http://msdn.microsoft.com/en-us/library/windows/desktop/ms686277(v=vs.85).aspx ... if you happen to be in C++11, you could use a std::thread to create the thread and then call thread::native_handle to get a handle to call the Windows API to set the priority.

How to add tasks to a tango (D) ThreadPool asynchroniously ?

I am comparing a task queue/thread pool pattern system to an n-threads system in D. I'm really new to the D programming language but have worked with threads in C, Java, and Python before. I'm using the Tango library, and I'm building a webserver as an example.
I decided to use tango.core.ThreadPool as my thread pool, as my project is focused on ease of use and performance between traditional threading and task queues.
The documentation shows that I have 3 options:
ThreadPool.wait() - Blocks the current thread while the pool consumes tasks from the queue.
ThreadPool.shutdown() - Finishes the tasks in the pool but not the ones in the queue.
ThreadPool.finish() - Finishes all tasks in the pool and queue, but then accept no more.
None of these things are what I want. It is my understanding that your list of tasks should be able to grow in these systems. The web server is very simple and naïve; I just want it to try its best at scaling to many concurrent requests, even if its resource management only consists of consuming things in the task queue as quickly as possible.
I suspect that it's because the main thread needs to join the other threads, but I'm a bit rusty on my threading knowledge.
what about void append(JobD job, Args args) ? from the docs it works like the Executor.execute(Runnable) form java (submit a task to be run some time in the future)
note that here it is a LIFO queue instead of the expected FIFO queue so allocate enough workers
I discovered that the way I was constructing my delegate contributed to blocking in some part of the code. Instead of closing over the object returned by SocketServer.accept, I now pass that object as a parameter to my delegate. I don't know why this was the solution, but the program now works as expected. I heard that closures in D version 1 are broken; maybe this has something to do with it.

Possible frameworks/ideas for thread managment and work allocation in C++

I am developing a C++ application that needs to process large amount of data. I am not in position to partition data so that multi-processes can handle each partition independently. I am hoping to get ideas on frameworks/libraries that can manage threads and work allocation among worker threads.
Manage threads should include at least below functionality.
1. Decide on how many workers threads are required. We may need to provide user-defined function to calculate number of threads.
2. Create required number of threads.
3. Kill/stop unnecessary threads to reduce resource wastage.
4. Monitor healthiness of each worker thread.
Work allocation should include below functionality.
1. Using callback functionality, the library should get a piece of work.
2. Allocate the work to available worker thread.
3. Master/slave configuration or pipeline-of-worker-threads should be possible.
Many thanks in advance.
Your question essentially boils down to "how do I implement a thread pool?"
Writing a good thread pool is tricky. I recommend hunting for a library that already does what you want rather than trying to implement it yourself. Boost has a thread-pool library in the review queue, and both Microsoft's concurrency runtime and Intel's Threading Building Blocks contain thread pools.
With regard to your specific questions, most platforms provide a function to obtain the number of processors. In C++0x this is std::thread::hardware_concurrency(). You can then use this in combination with information about the work to be done to pick a number of worker threads.
Since creating threads is actually quite time consuming on many platforms, and blocked threads do not consume significant resources beyond their stack space and thread info block, I would recommend that you just block worker threads with no work to do on a condition variable or similar synchronization primitive rather than killing them in the first instance. However, if you end up with a large number of idle threads, it may be a signal that your pool has too many threads, and you could reduce the number of waiting threads.
Monitoring the "healthiness" of each thread is tricky, and typically platform dependent. The simplest way is just to check that (a) the thread is still running, and hasn't unexpectedly died, and (b) the thread is processing tasks at an acceptable rate.
The simplest means of allocating work to threads is just to use a single shared job queue: all tasks are added to the queue, and each thread takes a task when it has completed the previous task. A more complex alternative is to have a queue per thread, with a work-stealing scheme that allows a thread to take work from others if it has run out of tasks.
If your threads can submit tasks to the work queue and wait for the results then you need to have a scheme for ensuring that your worker threads do not all get stalled waiting for tasks that have not yet been scheduled. One option is to spawn a new thread when a task gets blocked, and another is to run the not-yet-scheduled task that is blocking a given thread on that thread directly in a recursive manner. There are advantages and disadvantages with both these schemes, and with other alternatives.