I'm currently running a Django application with SESSION_ENGINE configured as django.contrib.sessions.backends.db. I'd like to change this to django.contrib.sessions.backends.cached_db for a performance boost.
Can I make this change without destroying the existing sessions?
Yes, you can make this change without logged in users suddenly finding themselves being logged out. That's because cached_db checks memcache first for the key and if it cannot be found in it, goes to the database. Thus making this change will not cause a loss of session data. Fragment of code from cached_db
def load(self):
try:
data = self._cache.get(self.cache_key)
except Exception:
# Some backends (e.g. memcache) raise an exception on invalid
# cache keys. If this happens, reset the session. See #17810.
data = None
if data is None:
# Duplicate DBStore.load, because we need to keep track
# of the expiry date to set it properly in the cache.
However please note that cached sessions backends are a bit over rated. Depending on the middleware that you have, the session object may be updated very often, as often as every request if only to change the expire date. In that case you will find that the database is being written to all the time. Which means the cached value has to be discarded too.
You should be able to. cached_db backend is just a write-through cache to a database backed, persistent, db backend which speeds up your read queries. It will not speed up your write queries, so you should try and find out how much you are reading and writing the session data.
Your Django SECRET_KEY setting determines your session key hashing parametrs along with Session settings that determine the cache you will use for sessions and session your TTLs, so if you are not changing those variables, you should be good.
Related
I'm specifying a default SESSION_COOKIE_AGE in the main settings.py file; however, I'd like admin users to be able to overwrite this value. This value should persist across server restarts.
My initial reaction was to store this in a single-row in a DB table and then write some middleware that would adjust the user's expiry date on each request to match the user-supplied SESSION_COOKIE_AGE; however, that seems like a lot of redundant db queries. Then I went down a signals rabbit hole and am considering creating two signals (post_save and connection_created) to handle: user updates to the user-supplied SESSION_COOKIE_AGE value stored in the DB as well as ensuring SESSION_COOKIE_AGE is updated upon the server starting. And finally I see all of these projects like django-constance, etc. to dynamically update settings constants at runtime.
Any suggestions on the most reliable / least error-prone architecture for this?
In my settings.py, I have a database configured. I have a page that issues a sql query against the database defined in settings.py every 10 seconds.
I have been using it for 1 year now and never had any issues with it.
My database admin ran a login audit on our database. As it turns out, each individual single sql query has a unique login to the database. His audit took 5 minutes to run just today and it is because of my django application logging in.
I was pretty surprised to find out that every query that is issues has a unique login attempt to the database.
Is there anyway to create a "session" for the backend database in settings.py. I really feel that the application should have a single login and use that session to issue commands.
Did I miss a setting to do this?
Sql login audit:
Consider setting CONN_MAX_AGE in DATABASES setting.
From the docs
Persistent connections avoid the overhead of re-establishing a connection to the database in each request. They’re controlled by the CONN_MAX_AGE parameter which defines the maximum lifetime of a connection. It can be set independently for each database.
The default value is 0, preserving the historical behavior of closing the database connection at the end of each request. To enable persistent connections, set CONN_MAX_AGE to a positive integer of seconds. For unlimited persistent connections, set it to None.
When a user logs in some details are saved to the session, lets say using key='user_settings'
When some updates happens within the system I need to loop through all users and delete the 'user_settings' key from each users session. That way I will trigger loading the details fresh from the database.
How can I loop through all sessions and remove the key from each of the sessions?
Looping
Even if your site has only a few thousand users, you might find that the django_sessions table has a few million entries in it. This happens if the sessions haven't been cleaned up. If you loop through them and delete the session entries you will find the process to be very slow. Even 100 sessions would result in a 100 queries being executed.
Using the Session Model.
You can apply filters on django.contrib.sessions.models.Session just as you would on any other model. Thus the following query would clear out all the sessions in one go.
from django.contrib.sessions.models import Session
Session.objects.all().delete()
You could choose to leave out some users
Session.objects.filter(session_key__in = [ ... ]).delete()
It does not involve a data retrieval or looping.
./manage.py
You could also use ./manage.py clearsessions to clean up expired sessions. Unexpired ones will be retained.
At the database level
Perhaps the fasted is to delete all the sessions is with an sql query. Where as retrieval and delete will be very very slow and involves millions of queries.
In the case of databases that support TRUNCATE
TRUNCATE django_session
For those that don't
DELETE FROM django_session
Lastly bear in mind that sessions do expire from time to time, so there isn't a lot of harm in clearing an entire session object instead of just deleting a specific key
You can access and update session data via the SessionStore object:
from django.contrib.sessions.models import Session, SessionStore
for session in Session.objects.all():
store = SessionStore(session.session_key)
del store['user_settings']
store.save()
You can delete the key from the session like any other dictionary.
del request.session['your key']
for more info check this out
How do I delete a session key in Django after it is used once?
I have a pandas dataframe with a loose wrapper class around it that provides metadata for my django/DRF application. The application is basically a user friendly (non programmer) way to do some data analysis and validation. Between requests I want to be able to save the state of the dataframe so I can have a series of interactions with the data but it does not need to be saved in a database ( It only needs to survive as long as the browser session ). From this it was logical to check out django's session framework, but from what I've heard session data should be lightweight and the dataframe object does not json serialize.
Because I dont have a ton of users, and I want the app to feel like a desktop site, I was thinking of using the django cache as a way to keep the dataframe object in memory. So putting the data in the cache would go something like this
>>> from django.core.cache import caches
>>> cache1 = caches['default']
>>> cache1.set(request.session._get_session_key, dataframe_object)
and then the same except using get in the following requests to access.
Is this a good way to do handle this workflow or is there another system I should use to keep rather large data(5mb to 100mb) in memory?
If you are running your application on a modern server then 100mb is not a huge amount of memory. However if you have more than a couple dozen simultaneous users, each requiring 100mb of cache then this could add up to more memory than your server can handle. Your cache and server should be configured appropriately and you may want to limit the total number of cached dataframes in your python code.
Since it does appear that Django needs to serialize session data your choice is to either use sessions with PickleSerializer or to use the cache. According to documentation, PickleSerializer is not recommended for security reasons so your choice to use the cache is a good one.
The default cache backend in Django does not share entries across processes so you would get better memory and time efficiency by installing memcached and enabling the memcached.MemcachedCache backend.
I have an authentication backend based off a legacy database. When someone logs in using that database and there isn't a corresponding User record, I create one. What I'm wondering is if there is some way to alert the Django system to this fact, so that for example I can redirect the brand-new user to a different page.
The only thing I can think of is adding a flag to the users' profile record called something like is_new which is tested once and then set to False as soon as they're redirected.
Basically, I'm wondering if someone else has figured this out so I don't have to reinvent the wheel.
I found the easiest way to accomplish this is to do exactly as you've said. I had a similar requirement on one of my projects. We needed to show a "Don't forget to update your profile" message to any new member until they had visit their profile for the first time. The client wanted it quickly so we added a 'visited_profile' field to the User's profile and defaulted that to False.
We settled on this because it was super fast to implement, didn't require tinkering with the registration process, worked with existing users, and didn't require extra queries every page load (since the user and user profile is retrieved on every page already). Took us all of 10 minutes to add the field, run the South migration and put an if tag into the template.
There's two methods that I know of to determine if an object has been created:
1) When using get_or_create a tuple is returned of the form (obj, created) where created is a boolean indicating obviously enough whether the object was created or not
2) The post_save signal passes a created paramater, also a boolean, also indicating whether the object was created or not.
At the simplest level, you can use either of these two hooks to set a session var, that you can then check and redirect accordingly.
If you can get by with it, you could also directly redirect either after calling get_or_create or in the post_save signal.
You can use a file-based cache to store the users that aren't yet saved to the database. When the user logs in for the second time, you can look in the cache, find the user object, and save it to the database for good.
Here's some info on django caching: http://docs.djangoproject.com/en/dev/topics/cache/?from=olddocs
PS: don't use Memcached because it will delete all information in the situation of a computer crash or shut down.