I have a serverless database cluster that spins down when it's out of use. Cold starts take a minute or so, during which django just waits and waits to send the http request object back.
Before I code something complicated, I'm looking for recommendations for an existing middleware/signal integration that would allow me to render an interim view while django attempts to get the database connection (for instance, if the database query takes longer than a second to complete, render to this view instead.)
You could create a custom middleware that tests for DB connectivity on every request. Bear in mind that this will attempt to create a new DB connection on every request
from django.db import connection
from django.db.utils import OperationalError
from django.shortcuts import render_to_response
def db_is_up_middleware(get_response):
def middleware(request):
try:
c = connection.cursor()
except OperationalError:
return render_to_response('your_template.html')
else:
return get_response(request)
return middleware
Partial solution:
I reduced the RESTful gateway's query timeout to 3 seconds. At the end of timeout, I return a 504 Error with a nice message that tells the user that the server has gone to sleep but it will be booting back up shortly. Inside the text/html response of the 504 Error, I included a refresh meta tag, so browsers will automatically try to reload the view.
I took all database calls off the public-facing site and put them behind an authentication layer. This means that authenticating will be the step most likely to time out, and users will (I hope) naturally try to reauthenticate after receiving the 504 Error above.
I added an AJAX jquery call in document.ready() to pull a random database record. This query will time out and nothing will happen (as intended). It forces the database server to begin booting up immediately whenever a user hits a public-facing page.
Related
I have a Flask app for http and web socket (Flask-SocketIO) communication between client and server using gevent. I also use server side session with Flask-Session extension for the app. I run a background task using SocketIO.start_background_task. And from this task, I need to access session information which will be used to emit message using socketio. I get error when accessing session from the task "RuntimeError: Working outside of request context." This typically means that you attempted to use functionality that needed an active HTTP request.
Socket IO instance is created as below-
socket_io = SocketIO(app, async_mode='gevent', manage_session=False)
Is there any issue with this usage. How this issue could be addressed?
Thanks
This is not related to Flask-SocketIO, but to Flask. The background task that you started does not have request and application contexts, so it does not have access to the session (it doesn't even know who the client is).
Flask provides the copy_current_request_context decorator duplicate the request/app contexts from the request handler into a background task.
The example from the Flask documentation uses gevent.spawn() to start the task, but this would be the same for a task started with start_background_task().
import gevent
from flask import copy_current_request_context
#app.route('/')
def index():
#copy_current_request_context
def do_some_work():
# do some work here, it can access flask.request or
# flask.session like you would otherwise in the view function.
...
gevent.spawn(do_some_work)
return 'Regular response'
I want to abort a request to an API which takes longer than 5 seconds. I know the client-side code can have a timeout of 5 seconds and that will ensure it doesn't wait any longer than that to get a response, but the Django server would still be processing the request even though the client won't be receiving it at the end.
As far as I know, even if I configure this on nginx or gunicorn, those will still act as clients for the Django server app and the same problem will occur - that nginx will not wait for more than 5 seconds, but Django will continue processing the request.
Is there any way I can somehow abort this request processing in Django (application server code level) itself?
For context on the specific use case:
This API parses a file and we have a size limit of 5 MB on that file. Even then, for some reason, sometimes, the file parsing takes more than 20-30 seconds, which is why we want to abort the request for this API if it exceeds a certain threshold.
You would have to raise a specific exception after a certain time.
In order to make sure you always return the same error you could use a custom error handler.
from rest_framework.views import exception_handler
def custom_exception_handler(exc, context):
if isinstance(exc, MyTimeoutException):
return Response(
{'detail': 'Your request timed out.'},
status=status.HTTP_408_REQUEST_TIMEOUT)
response = exception_handler(exc, context)
return response
REST_FRAMEWORK = {
'EXCEPTION_HANDLER': 'my_project.my_app.utils.custom_exception_handler'
}
I make my project on Django, it has Gunicorn on WSGI, Daphne on ASGI. ASGI server needed only for handling Websocket protocol. Using Channels in Django for Websocket routing and handling. Nginx on static and proxy. Database is Mysql.
Generally: is there a way to synchronise variable values in memory between ASGI and WSGI app without writing to database?
TLDR:
HTTP (wsgi) works for major interacting with database (for now, creating instances of models).
Websocket (asgi) is planned to work with user controls (for now, connect to rooms, in future, would be in-game controls? rotate piece etc. The project is Tetris multiplayer, where users can create rooms, for example, for 2 or 4 players (parallel tetris fields), when created other players can connect into that rooms.)
'Under the hood' there is 'engine' (some data is stored in memory when the server runs):
# engine/status.py
active_rooms = {}
when creating a new room, HTTP controller (from views.py) calls function:
import engine.status as status
from engine.Room import Room
def create_room(id, size):
new_room = Room(size)
...
status.active_rooms[id] = new_room
...
So, it writes a new key-value pair into dict (status.active_rooms), whers key is number(id), value is instance of class 'Room'.
When other player clicks on eg.'connect' button in room, Javascript on client sends special message by Websocket protosol.
Websocket handler calls function:
def make_connect(data):
id = data['room_id']
...
if int(id) not in status.active_rooms:
msg = 'No room'
return {'type': 'info', 'msg': msg}
else:
msg = 'Room exists'
...
so it checks if exists the room with this id in memory.
The problem is:
The dict is always empty when check! Seems like ASGI and WSGI apps have each own instance of 'engine'.
It means, client can not see the actual status on server.
I tried to make dumps into database, but the class has some specific fields which can not be pickled.
My idea now is, to make 'creating rooms' with ASGI app (thru Websocket not HTTP).
Maybe i am missing something? Is there some another way to share data between ASGI and WSGI apps?
Just for information: i managed to make WS parallel request at the same time where goes HTTP request.
WSGI application writes to DB, ASGI application creates objects in memory with specific keys which could be used to access data from DB.
On next WS requests, ASGI reads keys from these objects in memory and call function which loads data from DB. Overall, ASGI and WSGI do not use identical environment, but using unique keys which were the same in first parallel HTTP and WS requests, ASGI can access data which was received by WSGI
I have a scenario where I have to upload a file from flask app to a third party API. I have wrapped all API requests in Flask to control API usage. For this I have redirected the traffic from main route towards the api wrapper route with http 307 status to preserve the request body and in the API wrapper I have used request to post into third party API endpoint.
The problem is only file < 100KB gets send through the redirection request, having a file larger than 100 KB gets somehow terminated in the sending phase.
Is there any limit in the 307 redirection and payload size?
I tried debugging by watching the network timing stack trace, from there it seems the request is dropped in the sending phase.
Main Blueprint
#main.route('/upload/',methods=['POST','GET'])
def upload():
#for ajax call
if request.method == 'POST'
return redirect(url_for('api.file_push'),code=307)
else:
return render_template('file-upload.html')
API Blueprint
#api.route('/upload/',methods=['POST'])
def file_push():
upload_file = request.files['file']
filename = urllib.parse.quote(upload_file.filename)
toUpload = upload_file.read()
result=requests.post(apiInterfaces.FILE_UPLOAD_INTERFACE+'/'+filename,files{'file':toUpload})
return result
Yes, I can directly send post request to API endpoint from main route but I don't want to, it will destroy my system design and architecture.
I assume you're using Python, and possibly requests so this answer will be based on what I've learned figuring this out (debugging with a colleague). I filed a bug report with psf/requests. There is a related answer here which confirms my suspicions.
It seems that when you initiate a PUT request using requests (and urllib3), the entire request is sent before a response from the server is looked at, but some servers can send a HTTP 307 during this time. One of two things happen:
the server closes the connection by sending the response, even if the client has not finished sending the entire file. In this case, the client might see a closed connection and you won't have a response you can use for redirect (this happens with urllib3>1.26.10 (roughly)) but requests is not handling this situation correctly
the server sends the response and you re-upload the file to the second location (urllib3==1.26.3 has this behavior when using requests). Technically, there is a bug in urllib3 and it should have failed, but silently lets you upload...
However, it seems that if you are expecting a redirect, the most obvious solution might be to send a null byte via PUT first, get a valid response back for the new URL [don't follow redirects], and then use that response to do the PUT of the full file. With requests, it's probably something like
>>> import requests
>>> from io import BytesIO
>>> data = BytesIO(b'\x00')
>>> response = request.put(url, data=data, allow_redirects=False)
>>> request.put(response.headers['Location'], data=fp, allow_redirects=False)
and at that point, you'll be ok (assuming you only expect a single redirect here).
When server caches a page, I thought it worked like this, but apparently I'm wrong.
server stores http response contents in a cache-store with a key (that's generated for the endpoint)
one could create separate cache key for certain aspects of requests (such as user language)
when there's a request for the same key, and it's stored in cache-store (such as redis), webserver returns the cached response instead of recreating the response
I think the above understanding is flawed because,,
when I hit the endpoint with axios, first request is hit on the server, but the second request doesn't even hit the endpoint, it seems axios is reusing the previous response it received.
So, if I clear all cache store between two requests, the second request doesn't get to the request handler (view function in django)
for http request (browser reload) does work as I described in the above, it hits the web server and gets the cached response.
In django, I'm using from django.views.decorators.cache import cache_page
But the question is more of general working of web response cache
General web caching is explained very well here * https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching
I guess certain client never reaches out to server when cache-control timeout or expires is set, and some client does reach out to server..