I am trying to do use the http-kit client library in clojure to do synchronous posts returning promises. Is there any way to limit the number of threads doing the actual post?
All the examples I could find of using the inbuilt thread pool use the lower level primitive function called request but they were all for http/get.
Thanks
I'm assuming you've seen http://http-kit.org/client.html#sync
My question is do you want to do a synchronous POST, or limit the number of threads? You can do a sync POST with 100 threads, it just so happens you're main thread will wait for the request to return.
Maybe more importantly, why do you want to limit the number of threads?
Also, see https://github.com/http-kit/http-kit/blob/master/src/org/httpkit/client.clj, specifically request. You can handle it a map of arguments, like
{:url "http://yoursite.com" :worker-pool my-thread-pool-executor}
my-thread-pool-executor has to extend ExecutorService.
Specifically, you need to overload submit which is what the RespListener uses in http-kit. You could make submit synchronous with your own ExecutorService implementation so it runs on the same thread.
Related
I have a single threaded asynchronous tcp server written using boost asio. Each incoming request will go through several processing steps (synchronous and asynchronous) and finally send back the response using async write.
For small loads with 10 concurrent requests, it works decently. However, when I test using a parallelism of 100, things start worsening. Response latency starts increasing as time progresses. So, I want to try with some multi-threaded processing for handling requests.
I am looking for a decent example / help on creating and running multiple threads for asynchronous reading/writing to clients. I have the following doubts:
Should I use a single IOS object and call its run method in all of the threads of the thread pool, or should I use a separate IOS per thread?
If I use a single IOS, is there a possibility that part of the tcp data goes to one thread, while another part going to another thread and so on.. Is this understanding correct?
Is there any other better way?
Thanks for any help and pointers here.
Without seeing your code I can only guess what goes wrong. Most probably you're running long actions inside async completion handlers. The completion handlers should be fast - get the data, hand it off for further processing, done.
As a first priority, I would go full-asynchronous and run all processing in a thread pool. You can find an example here, where a new thread is started for every new client, which you can replace with a thread pool.
Use a single io_service. A single io_service can handle a lot of parallelism, provided you don't delay it inside completion handlers. This simplifies the implementation because you don't have to worry about completion handlers running in parallel, which will happen if you run multiple IOS in multiple threads.
Q1: Should I use a single IOS object and call its run method in all of the threads of the thread pool, or should I use a separate IOS per thread?
Either you can
HTTP Server 2 - IOS per thread
HTTP Server 3 - single IOS with thread pool
Q2: If I use a single IOS, is there a possibility that part of the tcp data goes to one thread, while another part going to another thread and so on.. Is this understanding correct?
Yes, there is a race condition, but boost.asio support strand to avoid it.
Q3: Is there any other better way?
To me, not find a better way, if you find, tell me or past here, thank you.
BTW, as #rustyx said, your program is blocked at sync calls, turn to full-asynchronous calls will help.
I have started working with 'cpp rest sdk'. The key point I understood it that all requests and response are asynchronous using the means of PPL. This is fine to boost scalability and performance of the application.
The question I have is weather is any way to request and wait for a response in a 'synchronous' fashion.
I found the following web page which seems to work fine and it claims to make call in synchronous way. Is it so?
Does a call to task::get() guarantees that the function returns when the response is ready to be read?
The major idea of asynchronous programming is to chain all parts you want to have executed sequentially. If you want the program to wait until the sequence is finished, you can end the chain with .wait()
client.request(web::http::methods::GET, U("/foo.html"))
.then(/*some lambda*/)
.then(/*some lambda*/) //and so on
.wait(); //stop here until the chain is executed
Similarly, you can also use get() which also calls wait() but further returns the result of the task.
I am making a Messenger bot and I am using Ring as my http framework.
Sometime I want to apply delays between messages sent by the bot. My expectation would be that it is safe to use Thread/sleep because this will make the active thread sleep and not the entire server. Is that so, or should I resort to clojure/core.async?
This is the code I would be writing without async:
(match [reply]
; The bot wants to send a message (text, images, videos etc.) after n milliseconds
[{:message message :delay delay}]
(do
(Thread/sleep interval delay)
(facebook/send-message sender-id message))
; More code would follow...
A link to Ring code where its behaviour in this sense is clear would be appreciated, as well as any other with explanation on the matter.
Ring is the wrong thing to ask this question about: ring is not an http server, but rather an abstraction over http servers. Ring itself does not have a fixed threading model: all it really cares about is that you have a function from request to response.
What really makes this decision is which ring adapter you use. By far the most common is ring-jetty-adapter, which is a jetty http handler that delegates to your function through ring. And jetty does indeed have a single thread for each request, so that you can sleep in one thread without impacting others (but as noted in another answer, threads are not free, so you don't want to do a ton of this regularly).
But there are other ring handlers with different threading models. For example, aleph includes a ring adapter based on netty, which uses java.nio for non-blocking IO in a small, limited threadpool; in that case, sleeping on a "request thread" is very disruptive.
Assuming you're talking about code in a handler, Thread/sleep in Ring does make the thread for the request sleep. If you have multiple requests you are burning up expensive server threads.
The reason why Ring blocks is because the (non-async) model is based on function composition, where the result of one function is the output for another. So they have to wait, where exactly I can pinpoint this in the code I don't know.
Putting it in a go-block is better, because then you are not blocking server threads. It can return the response while you send the message. Do note that you cannot use results from the go block.
If you also want a response asynchronously (without blocking a server thread) you can for example use Pedestal.
For most servers synchronous handlers are sufficient, but if you are using Thread/sleeps AND want a response I would recommend using asynchronous Ring handlers or Pedestal or another framework.
I'm making an application that will continually send CFHTTP requests to a server to search for items, as well as sending further CFHTTP requests to perform actions on any returned results.
The issue I'm having is that the server has a maximum threshold of 3 requests per second and even when I try to implement a sleep call every 4 milliseconds it doesn't work properly as, although it delays, the CFHTTP requests can queue up if it takes them a couple of seconds to return so that it then tries to send multiple in the same second triggering the threshold to be exceeded.
Is there a way I can ensure that there are never more than 3 active CFHTTP requests?
I think you are going to need to implement some sort of logging widget as part of your process. The log will keep track of request frequency. If the threshold is not met, then you would just skip over that iteration of your CFHTTP call. I don't mean a file log or a database log, but something implemented in the application or even request scope depending on your implementation. There is no way to throttle CFHTTP itself. It is basically a very simplistic wrapper around a Java HTTP library which then goes straight to the underlying operating system.
If you're limiting concurrent requests, then first part of this answer applies. If you're looking to limit the number of requests per second, then the bit at the end applies. The question kind of asks both things.
If I understand correctly, you've got a number of threads (either as requests CF is processing or threads CF has created itself) which all need to make calls to the same rate-limited domain. What you need is a central way of co-ordinating access, combined with a nice way of controlling program execution.
I don't know of any native limits that CF might support (I'd be happy to be proven wrong) so you're likely to have to implement your own. The cheap'n'nasty way to do this is to increment and decrement a allowed_conenctions variable in a long-lived scope such as appliation. The downsides are that you have to implement checking all over the place and that if there are no spare connections, you'll have to wait somehow.
Really what you have is a resource pool (of allowed HTTP connections) and I'm guessing that you want your code to wait until a connection is free. CF does this kind of thing already for database connections.
In your case, there isn't really a need to keep anything in a pool (as HTTP connections aren't long-lived), other than a permit to use the resource. Java provides a class which ought to provide what you're after, the Semaphore.
I've not tried it but in theory, something like the snippet below ought to work:
//Application.cfc:onApplicationStart()
application.http_pool = CreateObject("java","java.util.concurrent.Semaphore").init(3)
//Meanwhile, elsewhere in your code
application.http_pool.acquire()
//Make my HTTP call
application.http_pool.release()
You could even wrap the HTTP object to provide this functionality without having to use the acquire/release each time, which would make it more reliable.
EDIT
It you're looking to limit rates, look at guava's RateLimiter which has the same general interface as Semaphore above, but implements rate limiting for you. You'd need to add guava to ColdFusion's classpath, or use JavaLoader or use CF10 which has classloading facilities built-in.
This is somewhat related to this question, but I think I need to know a little bit more. I've been trying to get my head around how to do this for a few days (whilst working on other parts), but the time has come for me to bite the bullet and get multi-threaded. Also, I'm after a bit more information than the question linked.
Firstly, about multi-threading. As I have been testing my code, I've not bothered with any multi-threading. It's just a console application that starts a connection to a test server and everything else is then handled. The main loop is this:
while(true)
{
Root::instance().performIO(); // calls io_service::runOne();
}
When I write my main application, I'm guessing this solution won't be acceptable (as it would have to be called in the message loop which, whilst possible, would have issues when the message queue blocks waiting for a message. You could change it so that the message-loop doesn't block, but then isn't that going to whack the CPU usage through the roof?)
The solution it seems is to throw another thread at it. Okay, fine. But then I've read that io_service::run() returns when there is no work to do. What is that? Is that when there's no data, or no connections? If at least one connection exists does it stay alive? If so, that's not so much of a problem as I only have to start up a new thread when the first connection is made and I'm happy if it all stops when there is nothing going on at all. I guess I am confused by the definition of 'no work to do'.
Then I have to worry about synchronizing my boost thread with my main GUI thread. So, I guess my questions are:
What is the best-practice way of using boost::asio in a client application with regard to threads and keeping them alive?
When writing to a socket from the main thread to the IO thread, is synchronization achieved using boost::asio::post, so that the call happens later in the io_service?
When data is received, how do people get the data back to the UI thread? In the past when I used completion ports, I made a special event that could post the data back to the main UI thread using a ::SendMessage. It wasn't elegant, but it worked.
I'll be reading some more today, but it would be great to get a heads up from someone who has done this already. The Boost::asio documentation isn't great, and most of my work so far has been based on a bit of the documentation, some trial/error, some example code on the web.
1) Have a look at io_service::work. As long as an work object exists io_service::run will not return. So if you start doing your clean up, destroy the work object, cancel any outstanding operations, for example an async_read on a socket, wait for run to return and clean up your resources.
2) io_service::post will asynchronously execute the given handler from a thread running the io_service. A callback can be used to get the result of the operation executed.
3) You needs some form of messaging system to inform your GUI thread of the new data. There are several possibilities here.
As far as your remark about the documention, I thing Asio is one of the better documented boost libraries and it comes with clear examples.
boost::io_service::run() will return only when there's nothing to do, so no async operations are pending, e.g. async accept/connection, async read/write or async timer wait. so before calling io_service::run() you first have to start any async op.
i haven't got do you have console or GUI app? in any case multithreading looks like a overkill. you can use Asio in conjunction with your message loop. if it's win32 GUI you can call io_service::run_one() from you OnIdle() handler. in case of console application you can setup deadline_timer that regularly checks (every 200ms?) for user input and use it with io_service::run(). everything in single thread to greatly simplify the solution
1) What is the best-practice way of using
boost::asio in a client application
with regard to threads and keeping
them alive?
As the documentation suggests, a pool of threads invoking io_service::run is the most scalable and easiest to implement.
2) When writing to a socket from the main
thread to the IO thread, is
synchronization achieved using
boost::asio::post, so that the call
happens later in the io_service?
You will need to use a strand to protect any handlers that can be invoked by multiple threads. See this answer as it may help you, as well as this example.
3) When data is received, how do people
get the data back to the UI thread? In
the past when I used completion ports,
I made a special event that could post
the data back to the main UI thread
using a ::SendMessage. It wasn't
elegant, but it worked.
How about providing a callback in the form of a boost::function when you post an asynchronous event to the io_service? Then the event's handler can invoke the callback and update the UI with the results.
When data is received, how do people get the data back to the UI thread? In the past when I used completion ports, I made a special event that could post the data back to the main UI thread using a ::SendMessage. It wasn't elegant, but it worked
::PostMessage may be more appropriate.
Unless everything runs in one thread these mechanisms must be used to safely post events to the UI thread.