In Python Flask I'm sending a POST request with some network tasks.
All works fine, but the Flask website stalls (freezes) as long as the task is working on it.
For example animated.gif also freezes as long as the task is lasting.
I tried app.run(threaded=True), but that doesn't change.
Any suggestion?
If I understand correctly, setting theaded=True, only means that it will process multiple request but each request is still handled by a single thread. If you can't cut down on the processing time, you could try better ways for dealing with the long running task on the UI.
Related
I'm running Django 4.0.5 + Django Rest Framework + Nginx + Gunicorn
Sometimes, I'm going to need to handle some POST requests with a lot of data to process.
The user will wait for a "ok" or "fail" response and a list of ids resulting from the process.
Everything works fine so far for mid size body requests (this is subjective), but when I get into big ones, the process will take 1min+.
It's in these cases when I get a 500 error response from DRF, but my process in the background will keep running till the end (but user will not know it finished successfully).
I was doing some investigation and changed the Gunicorn timeout parameter (to 180), but didn't change the behavior in the service.
Is there a way to set a timeout larger than 60s at the #api_view or somewhere else?
Use celery async tasks to process such requests as background tasks.
I'm trying to execute a long running function (ex: sleep(30)) after a Django view returns a response. I've tried implementing the solutions suggested to similar questions:
How to execute code in Django after response has been sent
Execute code in Django after response has been sent to the
client
However, the client's page load only completes after the long running function completes running when using a WSGI server like gunicorn.
Now that Django supports asynchronous views is it possible to run a long running query asynchronously?
Obviously, I am looking for a solution regarding the same issue, to open a view which should start a background task and send a response to the client without waiting until started task is finished.
As far as I understand yet this is not one of the objectives of async view in Django. The problem is that all executed code is connected the the worker started to handle the http request. If the response is sent back to the client the worker cannot handle any other code / task anymore started in the view asynchronous. Therefore, all async functions require an "await" in front of. Consequently, the view will only send its response to the client if the awaited function is finished.
As I understand all background tasks must be pushed in a queue of tasks where another worker can catch each new task. There are several solution for this, like Djangp Channels or Django Q. However, I am not sure what is the most lightweighted solution.
I am making a Django Rest framework based server and in one of the request, I get an audio file from front-end, on which I need to run some ML based algorithm(I have script for same) and respond back to user with the result. Problem is that this request might take 5-10 seconds to execute. I am trying to understand following things:
Will Celery help me reduce the workload on server, as in any case I need to wait for the result of the ML Algo and respond back to user.
Should I create a different server to handle this type of request? Will that be a better approach?
Also, is my flow of doing things correct. First, Upload the file to some cloud platform for storage and serialize the instance to get the url of file. Second run the script using celery and wait for the result. Third, Respond back with the result.
Thanks for helping.
So I'm trying to accomplish the following. User browses webpage and at the sime time there is a task running in the background. When the task completes it should return args where one of args is flag: True in order to trigger a javascript and javascript shows a modal form.
I tested it before without async tasks and it works, but now with celery it just stores results in database. I did some research on tornado-celery and related stuff but some of components like tornado-redis is not mantained anymore so it would not be vise in my opinion to use that.
So what are my options, thanks?
If I understand you correctly, then you want to communicate something from the server side back to the client. You generally have three options for that:
1) Make a long pending request to the server - kinda bad. Jumping over the details, it will bog down your web server if not configured to handle that, it will make your site score low on performance tests and if the request fails, everything fails.
2) Poll the server with numerous requests with a time interval (0.2 s, something like that) - better. It will increase the traffic, but the requests will be tiny and will not interfere with the site's performance very much. If you instate a long interval to not load the server with pointless requests, then the users will see the data with a bit of a delay. On the upside this will not fail (if written correctly) even if the connection is interrupted.
3) Websockets where the server can just hit the client with any message whenever needed - nice, but takes some time to get used to. If you want to try, you can use django-channels which is a nice library for Django websockets.
If I did not understand you correctly and this is not the problem at hand and you are figuring how to get data back from a Celery task to Django, then you can store the Celery task ID-s and use the ID-s to first check, if the task is completed and then query the data from Celery.
I have a rails 4 application which has some API methods and those methods consumes time for computation and generating huge JSON response for clients. The problem is that these requests block the entire app. And only one 1 user(request) can be served at the same time. The request runs for a long time to generate JSON response. When the first request is running, and the application receives new request's then all those new request's are failed.. How to solve this.. Unfortunately rails doesn't this automatically..
I have gone through similar threads on SO, but was not able to find solution for Rails 4 application.Please share your experience and guide me in the right direction to solve this issue.
Thanks!
The best practice for such long running API calls is to make them asynchronous. That way the user thread is not blocked.
The two popular gems to run background jobs are
1) https://github.com/collectiveidea/delayed_job
2) https://github.com/resque/resque