I need to create a REST data source in two cases :
when there is nothing in the local browser cache - the standard way through ds.dataBind()
when there is something - by preloading the previously cached json result in it.
The browser cache may by any browser cache. I prefer localForage.
Is there any way through public API to push a json array into the REST ds after its creation and before databind() in order to prevent any inital GET/databind call?
There is nothing built-in for the igDataSource to handle cached data over requests. My suggestion would be to utilize the jQuery.ajaxSetup to intercept the requests and use cached data if such exists in localStorage or anywhere else.
$.ajaxSetup({
beforeSend: function (jqXHR, settings) {
// return from local storage instead of initiating the request
}
});
I have come to the conclusion that it is not possible to do it this way (on the level of $.ajax) because returning from beforeSend does not cancel the request. On the other hand aborting the request (jqXHR.abort()) aborts the whole request pipeline which aborts the execution of all other $.ajax callbacks which is a dead end - the whole dataSource pipeline is then aborted which stops me from getting any result.
The only solution for the moment is to create different type of dataSource (JSON ds) during the creation of the grid (in my case these are the ds for the combos).
Update
It is not at all impossible, but the pipeline consisting methods _remoteData->_processRequest->_successCallBack->CompleteCallBack must be abstracted away in a state-machine like class. The problem comes from the fact that the state-machine is implemented via $.ajax which is not really conceived with that kind of scenario in mind and hacking it is not a good idea.
If there is a lightweight js state-machine library it can be done.
Related
I want to invoke CefV8Context::Eval function and get the returned value in browser process's UI thread. But the CEF3 C++ API Docs states that V8 handles can only be accessed from the thread on which they are created. Valid threads for creating a V8 handle include the render process main thread (TID_RENDERER) and WebWorker threads. Is that means I should use the inter-process communication (CefProcessMessage) to invoke that method and get the return value? If so, how to do this in synchronous mode?
Short answer: CefFrame::ExecuteJavaScript for simple requests will work. For more complex ones, you have to give up one level of synchronousness or use a custom message loop.
What I understand you want to do is to execute some Javascript code as part of your native App's UI Thread. There are two possibilities:
It's generic JS code, doesn't really access any variables or functions in your JS, and as such has not context. This means Cef can just spin up a new V8 context and execute your code - see CefFrame::ExecuteJavaScript(). To quote the examples on CEF's JS Integration link:
CefRefPtr browser = ...;
CefRefPtr frame = browser->GetMainFrame();
frame->ExecuteJavaScript("alert('ExecuteJavaScript works!');",
frame->GetURL(), 0);
It's JS code with a context. In this case, read on.
Yes - CEF is designed such that only the RenderProcess has access to the V8 engine, you'll have to use a CefProcessMessage to head over to the Browser and do the evaluation there. You sound like you already know how to do that. I'll link an answer of mine for others who don't and may stumble upon this later: Background process on the native function at Chromium Embedded Framework
The CEFProcessMessage from Browser to Render processes is one place where the request has to be synchronized.
So after your send your logic over to the render process, you'll need to do the actual execution of the javascript code. That, thankfully, is quite easy - the same JS integration link goes on to say:
Native code can execute JS functions by using the ExecuteFunction()
and ExecuteFunctionWithContext() methods
The best part - the execution seems to be synchronous (I say seems to, since I can't find concrete docs on this). The usage in the examples illustrates this:
if (callback_func_->ExecuteFunctionWithContext(callback_context_, NULL, args, retval, exception, false)) {
if (exception.get()) {
// Execution threw an exception.
} else {
// Execution succeeded.
}
}
You'll notice that the second line assumes that the first has finished execution and that the results of said execution are available to it. So, The CefV8Value::ExecuteFunction() call is by nature synchronous.
So the question boils down to - How do I post a CefProcessMessage from Browser to Renderer process synchronously?. Unfortunately, the class itself is not set up to do that. What's more, the IPC Wiki Page explicitly disallows it:
Some messages should be synchronous from the renderer's perspective.
This happens mostly when there is a WebKit call to us that is supposed
to return something, but that we must do in the browser. Examples of
this type of messages are spell-checking and getting the cookies for
JavaScript. Synchronous browser-to-renderer IPC is disallowed to
prevent blocking the user-interface on a potentially flaky renderer.
Is this such a big deal? Well, I don't really know since I've not come across this need - to me, it's ok since the Browser's message loop will keep spinning away waiting for something to do, and receive nothing till your renderer sends a process message with the results of JS. The only way the browser gets something else to do is when some interaction happens, which can't since the renderer is blocking.
If you really definitely need synchronousness, I'd recommend that you use your custom MessageLoop which calls CefDoMessageLoopWork() on every iteration. That way, you can set a flag to suspend loop work until your message is received from renderer. Note that CefDoMessageLoopWork() and CefRunMessageLoop() are mutually exclusive and cannot work with each other - you either manage the loop yourself, or let CEF do it for you.
That was long, and covers most of what you might want to do - hope it helps!
My web app is based on (embedded) Jetty 9. The code that runs inside Jetty (i.e. from the *.war file) has the need to, at times, execute an HTTP request back into Jetty and itself, completely asynchronously to "real" HTTP requests coming from the network.
I know what you will say, but this is the situation I ended up with after merging multiple disparate products into one container and presently cannot avoid it. A stop-gap measure is in place - it actually sends a network HTTP request back to itself (presently using Jetty client, but that does not matter). However, not only that adds more overhead, it also does not allow us to pass actual object references we'd like to be able to pass via, say, request attributes.
Desire is to be able to do something like constructing new HttpServletRequest and HttpServletResponse pair and use a dispatcher to "include" (or similar) the other servlet we presently can only access via the network. We've built "dummy" implementations of those, but the this fails in Jetty's dispatcher line 120 with a null pointer exception:
public void include(ServletRequest request, ServletResponse response) throws ServletException, IOException
{
Request baseRequest=(request instanceof Request)?((Request)request):HttpChannel.getCurrentHttpChannel().getRequest();
... because this is not an instance of Jetty's Request class and getCurrentHttpChannel() returns null because the thread is a worker thread, not an http serving one and does not have Jetty's thread locals set up.
I am contemplating options, but would like some guidance if anyone can offer it. Some things I am thinking of:
Actually use Jetty's Request class as a base. Currently not visible to the web app (a container class, would have to play with classpath and class loaders perhaps. May still be impossible (don't know what to expect there).
Play with Jetty's thread locals, attempt to tell Jetty to set up current thread as necessary. Don't know where to begin. UPDATE Tried to setServerClasses([]) and then set the current HttpChannel to the one I 'stole' from another thread. Failed misearably: java.lang.IllegalAccessError: tried to access method org.eclipse.jetty.server.HttpChannel.setCurrentHttpChannel(Lorg/eclipse/jetty/server/HttpChannel;)V from class ...
Ideally, find a better/proper way of feeding a "top" request in without going via the network. Ideally would execute on the same thread, but I would be less concerned with that.
Remember that, unfortunately, I cannot avoid this at this time. I would much rather invoke code directly, but I cannot, as the code I had to add into mine is too big to handle at this time and too dependent on some third party filters I can't even modify (and only work as filters, on real requests).
Please help!
I'm making an application that will continually send CFHTTP requests to a server to search for items, as well as sending further CFHTTP requests to perform actions on any returned results.
The issue I'm having is that the server has a maximum threshold of 3 requests per second and even when I try to implement a sleep call every 4 milliseconds it doesn't work properly as, although it delays, the CFHTTP requests can queue up if it takes them a couple of seconds to return so that it then tries to send multiple in the same second triggering the threshold to be exceeded.
Is there a way I can ensure that there are never more than 3 active CFHTTP requests?
I think you are going to need to implement some sort of logging widget as part of your process. The log will keep track of request frequency. If the threshold is not met, then you would just skip over that iteration of your CFHTTP call. I don't mean a file log or a database log, but something implemented in the application or even request scope depending on your implementation. There is no way to throttle CFHTTP itself. It is basically a very simplistic wrapper around a Java HTTP library which then goes straight to the underlying operating system.
If you're limiting concurrent requests, then first part of this answer applies. If you're looking to limit the number of requests per second, then the bit at the end applies. The question kind of asks both things.
If I understand correctly, you've got a number of threads (either as requests CF is processing or threads CF has created itself) which all need to make calls to the same rate-limited domain. What you need is a central way of co-ordinating access, combined with a nice way of controlling program execution.
I don't know of any native limits that CF might support (I'd be happy to be proven wrong) so you're likely to have to implement your own. The cheap'n'nasty way to do this is to increment and decrement a allowed_conenctions variable in a long-lived scope such as appliation. The downsides are that you have to implement checking all over the place and that if there are no spare connections, you'll have to wait somehow.
Really what you have is a resource pool (of allowed HTTP connections) and I'm guessing that you want your code to wait until a connection is free. CF does this kind of thing already for database connections.
In your case, there isn't really a need to keep anything in a pool (as HTTP connections aren't long-lived), other than a permit to use the resource. Java provides a class which ought to provide what you're after, the Semaphore.
I've not tried it but in theory, something like the snippet below ought to work:
//Application.cfc:onApplicationStart()
application.http_pool = CreateObject("java","java.util.concurrent.Semaphore").init(3)
//Meanwhile, elsewhere in your code
application.http_pool.acquire()
//Make my HTTP call
application.http_pool.release()
You could even wrap the HTTP object to provide this functionality without having to use the acquire/release each time, which would make it more reliable.
EDIT
It you're looking to limit rates, look at guava's RateLimiter which has the same general interface as Semaphore above, but implements rate limiting for you. You'd need to add guava to ColdFusion's classpath, or use JavaLoader or use CF10 which has classloading facilities built-in.
I have a Django view that does some pretty heavy processing and takes around 20-30 seconds to return a result.
Sometimes the user will end up closing the browser window (terminating the connection) before the request completes -- in that case, I'd like to be able to detect this and stop working. The work I do is read-only on the database so there isn't any issue with transactions.
In PHP the connection_aborted function does exactly this. Is this functionality available in Django?
Here's example code I'd like to write:
def myview(request):
while not connection_aborted():
# do another bit of work...
if work_complete:
return HttpResponse('results go here')
Thanks.
I don't think Django provides it because it basically can't. More than Django itself, this depends on the way Django interfaces with your web server. All this depends on your software stack (which you have not specified). I don't think it's even part of the FastCGI and WSGI protocols!
Edit: I'm also pretty sure that Django does not start sending any data to the client until your view finishes execution, so it can't possibly know if the connection is dead. The underlying socket won't trigger an error unless the server tries to send some data back to the user.
That connection_aborted method in PHP doesn't do what you think it does. It will tell you if the client disconnected but only if the buffer has been flushed, i.e. some sort of response is sent from the server back to the client. The PHP versions wouldn't even work as you've written if above. You'd have to add a call to something like flush within your loop to have the server attempt to send data.
HTTP is a stateless protocol. It's designed to not have either the client or the server dependent on each other. As a result the state of either is only known when there is a connection is created, and that only occurs when there's some data to send one way or another.
Your best bet is to do as #MattH suggested and do this through a bit of AJAX, and if you'd like you can integrate something like Node.js to make client "check-ins" during processing. How to set that up properly is beyond my area of expertise, though.
So you have an AJAX view that runs a query that takes 20-30 seconds to process requested in the background of a rendered page and you're concerned about wasted resources for when someone cancels the page load.
I see that you've got options in three broad categories:
Live with it. Improve the situation by caching the results in case the user comes back.
Make it faster. Throw more space at a time/space trade-off. Maintain intermediate tables. Precalculate the entire thing, etc.
Do something clever with the browser fast-polling a "is it ready yet?" query and the server cancelling the query if it doesn't receive a nag within interval * 2 or similar. If you're really clever, you could return progress / ETA to the nags. However, this might not have particularly useful behaviour when the system is under load or your site is being accessed over limited bandwidth.
I don't think you should go for option 3 because it's increasing complexity and resource usage for not much gain.
We are building a REST service that will take about 5 minutes to execute. It will be only called a few times a day by an internal app. Is there an issue using a REST (ie: HTTP) request that takes 5 minutes to complete?
Do we have to worry about timeouts? Should we be starting the request in a separate thread on the server and have the client poll for the status?
This is one approach.
Create a new request to perform ProcessXYZ
POST /ProcessXYZRequests
201-Created
Location: /ProcessXYZRequest/987
If you want to see the current status of the request:
GET /ProcessXYZRequest/987
<ProcessXYZRequest Id="987">
<Status>In progress</Status>
<Cancel method="DELETE" href="/ProcessXYZRequest/987"/>
</ProcessXYZRequest>
when the request is finished you would see something like
GET /ProcessXYZRequest/987
<ProcessXYZRequest>
<Status>Completed</Status>
<Results href="/ProcessXYZRequest/Results"/>
</ProcessXYZRequest>
Using this approach you can easily imagine what the following requests would give
GET /ProcessXYZRequests/Pending
GET /ProcessXYZRequests/Completed
GET /ProcessXYZRequests/Failed
GET /ProcessXYZRequests/Today
Assuming that you can configure HTTP timeouts using whatever framework you choose, then you could request via a GET and just hang for 5 mins.
However it may be more flexible to initiate an execution via a POST, get a receipt (a number/id whatever), and then perform a GET using that 5 mins later (and perhaps retry given that your procedure won't take exactly 5 mins every time). If the request is still ongoing then return an appropriate HTTP error code (404 perhaps, but what would you return for a GET with a non-existant receipt?), or return the results if available.
As Brian Agnew points out, 5 minutes is entirely manageable, if somewhat wasteful of resources, if one can control timeout settings. Otherwise, at least two requests must be made: The first to get the result-producing process rolling, and the second (and third, fourth, etc., if the result takes longer than expected to compile) to poll for the result.
Brian Agnew and Darrel Miller both suggest similar approaches for the two(+)-step approach: POST a request to a factory endpoint, starting a job on the server, and later GET the result from the returned result endpoint.
While the above is a very common solution, and indeed adheres to the letter of the REST constraints, it smells very much of RPC. That is, rather than saying, "provide me a representation of this resource", it says "run this job" (RPC) and then "provide me a representation of the resource that is the result of running the job" (REST). EDIT: I'm speaking very loosely here. To be clear, none of this explicitly defies the REST constraints, but it does very much resemble dressing up a non-RESTful approach in REST's clothing, losing out on its benefits (e.g. caching, idempotency) in the process.
As such, I would rather suggest that when the client first attempts to GET the resource, the server should respond with 202 "Accepted" (http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.3), perhaps with "try back in 5 minutes" somewhere in the response entity. Thereafter, the client can poll the same endpoint to GET the result, if available (otherwise return another 202, and try again later).
Some additional benefits of this approach are that single-use resources (such as jobs) are not unnecessarily created, two separate endpoints need not be queried (factory and result), and likewise the second endpoint need not be determined from parsing the response from the first, thus simpler. Moreover, results can be cached, "for free" (code-wise). Set the cache expiration time in the result header according to how long the results are "valid", in some sense, for your problem domain.
I wish I could call this a textbook example of a "resource-oriented" approach, but, perhaps ironically, Chapter 8 of "RESTful Web Services" suggests the two-endpoint, factory approach. Go figure.
If you control both ends, then you can do whatever you want. E.g. browsers tend to launch HTTP requests with "connection close" headers so you are left with fewer options ;-)
Bear in mind that if you've got some NAT/Firewalls in between you might have some drop connections if they are inactive for some time.
Could I suggest registering a "callback" procedure? The client issues the request with a "callback end-point" to the server, gets a "ticket". Once the server finishes, it "callbacks" the client... or the client can check the request's status through the ticket identifier.