I am implementing server sent events using flask. If I use time.sleep inside my function, the sse doesn't return anything and the request stays as pending in the browser. If I don't use sleep, there would be overload of responses in the browser, so I need to use some delay. Why is time.sleep blocking the request? Is there another way I can add time delay here?
def get_message():
time.sleep(1.0)
s="xyz" #some function here for our business logic
return s
#app.route('/stream')
def stream():
def eventStream():
while True:
yield 'data: {}\n\n'.format(get_message())
return Response(eventStream(), mimetype="text/event-stream")
Related
I have the following async consumer:
class MyAsyncSoncumer(AsyncWebsocketConsumer):
async def send_http_request(self):
async with aiohttp.ClientSession(
timeout=aiohttp.ClientTimeout(total=60) # We have 60 seconds total timeout
) as session:
await session.post('my_url', json={
'key': 'value'
})
async def connect(self):
await self.accept()
await self.send_http_request()
async def receive(self, text_data=None, bytes_data=None):
print(text_data)
Here, on connect method, I first accept the connection, then call a method that issues an http request with aiohttp, which has a 60 second timeout. Lets assume that the url we're sending the request to is inaccessible. My initial understanding is, as all these methods are coroutines, while we are waiting for the response to the request, if we receive a message, receive method would be called and we could process the message, before the request finishes. However, in reality, I only start receiveing messages after the request times out, so it seems like the consumer is waiting for the send_http_request to finish before being able to receive messages.
If I replace
await self.send_http_request()
with
asyncio.create_task(self.send_http_request())
I can reveive messages while the request is being made, as I do not await for it to finish on accept method.
My understanding was that in the first case also, while awaiting for the request, I would be able to receive messages as we are using different coroutines here, but that is not the case. Could it be that the whole consumer instance works as a single coroutine? Can someone clarify what's happenning here?
Consumers in django channels each run thier own runloop (async Task). But this is per consumer not per message, so if you are handling a message and you await something then the entire runloop for that websocket connection is awaiting.
REST API service has a limit of requests (say a maximum of 100 requests per minute). In Django, I am trying to allow USERs to access such API and retrieve data in real-time to update SQL tables. Therefore there is a problem that if multiple users are trying to access the API, the limit of requests is likely to be exceeded.
Here is a code snippet as an example of how I currently perform requests - each user will add a list of objects he wants to request and run request_engine().start(object_list) to access the API. I use multithreading to speed up requests. I also allow retrying failed API requests via setting a limit on the number of requests for each request object upper_limit.
As I understand there should be some queue for API requests. I anticipate there must be a more elegant solution for this, however, I could not find any similar examples. How can one implement/rewrite this for multiUSER usage with Django?
import requests
from multiprocessing.dummy import Pool as ThreadPool
N=50 # number of threads
upper_limit=1 # limit on the number of requests for a single object
class request_engine():
def __init__(self):
pass
def start(self,objs):
self.objs={obj:{'status':0,'data':None} for obj in objs}
done=False
while not done:
self.parallel_requests()
done=all(_['status']>upper_limit or _['status']==-1 for obj,_ in self.objs.items())
return dict(self.objs)
def single_request(self,request_obj):
URL = f"https://reqres.in/api/users?page={request_obj}"
r = requests.get(url = URL)
if r.ok:
res = r.json()
self.objs[request_obj]['status']=-1
self.objs[request_obj]['data']=res
else:
self.objs[request_obj]['status']+=1
def parallel_requests(self):
objs=[obj for obj,_ in self.objs.items() if _['status']!=-1 and _['status']<=upper_limit]
pool = ThreadPool(N)
pool.map(self.single_request, objs)
pool.close()
pool.join()
objs=[1,2,3,4,5,6,7,7,8,234,124,24,535,6,234,24,4,1,3,4,5,4,3,5,3,1,5,2,3,5,3]
result=request_engine().start(objs)
print([_['status'] for obj,_ in result.items()])
# status corresponds to the number of unsuccessful requests
# status=-1 implies success of the request
Thanks in advance.
Currently my Flask app only processes one request at one time. Any request has to wait for previous request to finish before being processed and it is not a good user experience.
While I do not want to increase the number of requests the Flask app can processed at one time, how is it possible to return a Server Busy message immediately when the next request comes in before the previous request finishes?
I have tried out the threading method below, but can only get both 'Server busy message' and "Proper return message" after 10 seconds.
import threading
from contextlib import ExitStack
busy = threading.Lock()
#app.route("/hello")
def hello():
if busy.acquire(timeout = 1):
return 'Server busy message'
with ExitStack() as stack:
stack.callback(busy.release)
# simulate heavy processing
time.sleep(10)
return "Proper return message"
Relatively long-running task is delegated to celery workers, which are running separately, on another server.
However, results are added back to the relational database (table updated according to a task_descr.id as a key, see below), worker uses ignore_result.
Task requested from Flask application:
task = app.celery.send_task('tasks.mytask', [task_descr.id, attachments])
The problem is that tasks are requested while transaction is not yet closed on the Flask side. This causes race condition, because sometimes celery worker completes the task before the end of transaction in Flask app.
What is the proper way to send tasks after successful transaction only?
Or should the worker check task_descr.id availability before attempting conditional UPDATE and retry the task (this feels as too complex arrangement)?
Answer to Run function after a certain type of model is committed discusses similar situation, but here task sending is explicit, so no need to listen to the updates/inserts in some model.
One of the ways is Per-Request After-Request Callbacks, thanks to Armin Ronacher:
from flask import g
def after_this_request(func):
if not hasattr(g, 'call_after_request'):
g.call_after_request = []
g.call_after_request.append(func)
return func
#app.after_request
def per_request_callbacks(response):
for func in getattr(g, 'call_after_request', ()):
response = func(response)
return response
In my case the usage is in the form of a nested function:
task_desc = ...
attachments = ...
#...
#after_this_request
def send_mytask(response):
if response.status_code in {200, 302}:
task = app.celery.send_task('tasks.mytask', [task_descr.id, attachments])
return response
Not ideal, but works. My tasks are only for successfully served request, so I do not care of 500s or other error conditions.
I'll first explain the architecture of my system and then move to the question:
I have a REST API which is used as my API gateway. This server is build using Flask. I also have RabbitMQ cluster, and a client I wrote that listens to a specific queue and executes the tasks its getting.
Until now, all of my requests were asynchronous, so once a request has reached to the API gateway, a callback_uri field with URL to POST the results to provided as part of the request, and the API gateway was just responsible for sending the task to RabbitMQ and the worker processed the task, and at the end POSTed the results back to the callback URL.
My question is:
I want a new endpoint to be synchronous in the sense of, that the processing will be done still by the same worker as before, but I want to get the results back to the API gateway to return to the user and release the connection.
My current solution:
I'm sending a unique callback_uri as part of the request to the worker as before, but now the specific endpoint is implemented by my API gateway and allow both POST and GET methods, so the worker can POST the results once it finished, and my API gateway keeps polling the callback URL until a result is available and then return the result to the client.
Is there any other preferred option other than having a busy-waiting HTTP worker polling its own endpoint to get the results? but still be synchronous so the connection released only when the results become available?
Code for illustration only:
#app.route('/long_task', methods=['POST'])
#sync_request
def long_task():
try:
if request.get_json() is None:
return ERROR_MSG_NO_JSON, 400
create_and_send_request_to_rabbitmq()
return '', 200
except Exception as ex:
return ERROR_MSG_NO_DATA, 400
def sync_request(func):
def call(*args, **kwargs):
create_callback_uri()
result = func(*args, **kwargs)
status_code = result[1]
if status_code == 200:
result = get_callback_result()
return result
return call
def get_callback_result():
callback_uri = request.get_json()['callback_uri']
has_answer = False
headers = {'content-type': 'application/json'}
empty_response = {}
content = json.dumps(empty_response)
try:
with Timeout(seconds=SYNC_REQUEST_TIMEOUT_SECONDS):
while not has_answer:
response = requests.get(callback_uri, headers=headers)
if response.status_code == 200:
has_answer = True
content = response.content
else:
time.sleep(0.2)
except TimeoutException:
log.debug('Timed out on sync request for request %s ' % request)
return content, 200
So if I understand you correctly you want your backend to wait for the response from some worker (via RabbitMQ). You can achieve that by implementing rpc over rabbitmq. The key idea is to use the correlation id.
But definitely the most efficient way would be to run the client over websockets (or raw tcp socket if it is not a browser) and notify him directly when the job is done. That way you don't lock resources (client connection, rabbitmq queues) and you avoid performance hit (rpc).