I would like to have a global variable in my django app that stores resulting list of objects that I later use in some functions and I don't want to evaluate queryset more that once, i do it like this:
from app.models import StopWord
a = list(StopWord.objects.values_list('word', flat=True))
...
def some_func():
... (using a variable) ...
This seems ok to me but the problem is that syncdb and test command throw an exception:
django.db.utils.DatabaseError: (1146, "Table 'app_stopword' doesn't exist")
I don't know how to get rid of this, may be I am on a wrong way?
It sounds like the app that StopWord is a part of is either not in your installed apps setting, or you haven't run syncdb to generate the table.
Storing a 'global value' can be simulated by using the django cache framework.
# there is more to it then this - read the documentation
# settings.py needs to be configured.
from django.core.cache import cache
class StopWord(models.Model):
... # field definitions
#classmethod
def get_all_words(cls):
key = 'StopWord.AllCachedWords.Key'
words = cache.get(key)
if words is None:
words = list(StopWord.objects.values_list('word', flat=True))
cache.set(key, words)
return words
#elsewhere
from app.models import StopWord
for word in StopWord.get_all_words():
# do something
The above also handles a sort of cache invalidation. Your settings should set a default timeout, or you can set your own timeout as a 3rd parameter to cache.set(). This ensures that while you avoid most database calls, the cache will be refreshed every so often so new stopwords can be used without restarting the application.
Don't initialize queries in a global scope. Bind None to the name, then write a function that first checks if the value is None and if so generates the data, and then returns the value.
Related
I would like to view the queries run inside a block of code, ideally getting it as a list of strings.
Of course there are similar SO questions and answers, but they do not address my three specific requirements:
Works for queries other than SELECT.
Works when the code is not in DEBUG mode.
The code executes normally, i.e. any production code runs as production code.
What I have so far is a transaction inside a DEBUG=True override, that is instantly rolled back after the queries are collected.
from contextlib import contextmanager
from django.conf import settings
from django.db import connections
from django.db import transaction
from django.test.utils import override_settings
#contextmanager
#override_settings(DEBUG=True)
def print_query():
class OhNo(Exception):
pass
queries = []
try:
with transaction.atomic():
yield
for connection in connections:
queries.extend(connections[connection].queries)
print(queries)
raise OhNo
except OhNo:
pass
def do_special_debug_thing():
print('haha yes')
with print_query():
Foo.objects.update(bar=1)
if settings.DEBUG:
do_special_debug_thing()
There are two problems with that snippet:
That DEBUG override doesn't do anything. The context manager prints out [].
If the DEBUG override is effective, then do_special_debug_thing is called, which I do not want to happen.
So, as far as I know, there is no way to collect all queries made inside a block of code, including those that are SELECT statements, while DEBUG is off. What ways are there to achieve this?
If you would like to do that only once, getting queries separately and put them in a list could help you.
update = Foo.objects.filter(bar=1)
query = str(update.query)
print(query)
My goal is to cache a view until an event occurs where the view's cache would need to expire, otherwise cache for 1 hour. This is what I have in urls.py
url(r'^get_some_stuff/$', cache_page(60 * 60, key_prefix='get_some_stuff')(views.StuffView.as_view())),
And this works fine. Now I'm trying to fetch the cached view to verify that there's something there and I tried this:
from django.core.cache import cache
cache.get('get_some_stuff')
But this returns None. I was hoping to do something like this:
from django.core.cache import cache
#relevant event happened
cache.delete('get_some_stuff')
What's the right way to handle cache?
I've tried passing the uri path:
cache.get('/api/get_some_stuff/')
And I still get None returned.
>>> cache.has_key('/api/get_some_stuff/')
False
>>> cache.has_key('/api/get_some_stuff')
False
>>> cache.has_key('get_some_stuff')
False
I've reviewed the suggested answer and it does not solve the underlying issue at all. It does not appear to be as trivial as passing the uri routing path as the key since keys are somewhat abstracted within django.
Here is a code snippet from Relekang about expire #cache_page
from django.core.cache import cache
from django.core.urlresolvers import reverse
from django.http import HttpRequest
from django.utils.cache import get_cache_key
def expire_page_cache(view, args=None):
"""
Removes cache created by cache_page functionality.
Parameters are used as they are in reverse()
"""
if args is None:
path = reverse(view)
else:
path = reverse(view, args=args)
request = HttpRequest()
request.path = path
key = get_cache_key(request)
if cache.has_key(key):
cache.delete(key)
Django's Cache framework only allows to cache data for predefined time and to clear expired cache data you may need to use django-signals to notify some receiver function which clears cache.
And cache.get, cache.has_key, cache.delete requires complete cache_key to be passed not the url or key-prefix. As django takes care of the keys we don't have much control to get or delete data.
If you are using database caching then use raw sql query to delete cache record from the database table when it's stale. write a query which says delete from cache_table with cache_key like ('%1:views.decorators.cache.cache_page%')
I faced same issues with per-view caching and I went with low-level cache api. I cached final result querysets using cache.set() and good part is you can set your own key and play with it.
I recently wrote shopping cart code that depends on the Session object. It seemed the reasonable way to store data for anonymous users.
While doing a bunch of testing, I ran into an annoying problem--when users sign in part way through the checkout process (or simply while browsing for other products), Django issues a new session_key and I lose access to my session data.
Is there a way to keep the old session data? Or is my design approach wrong?
Try writing your own SessionBackend that inherits from existing one and overrides the cycle_key method.
1 In your settings.py:
SESSION_ENGINE = 'my_app.session_backend'
2 my_app.session_backend.py:
from django.contrib.sessions.backends.db import SessionStore as DbSessionStore
class SessionStore(DbSessionStore):
def cycle_key(self):
pass
cycle_key is beeing called in login view after authentication.
Let me now if it works ;)
Instead of disabling the cycle_key() (which is a security measure to avoid session fixation vulnerabilities), you could consider restoring the values through a decorator at the login and logout views. See:
https://stackoverflow.com/a/41849076/146289
I'm trying to do something similar. Django can change the session_key to mitigate session fixation vulnerabilities, so it's not suitable for a foreign key. I want something more permanent. So I'll just put the permanent identifier in request.session['visitor_id']:
from django.utils.crypto import get_random_string
import string
VALID_KEY_CHARS = string.ascii_lowercase + string.digits
def example_view(request):
if not request.session.get('visitor_id'):
self.request.session['visitor_id'] = get_random_string(32, VALID_KEY_CHARS)
# Now code the rest of the view, using the visitor_id instead of
# session_key for keys in your model.
# ...
Some additional features were added to a django application, and as a result the upload_to function also expanded.
Since django by default stores filenames in the database, and files on disk, no harm has been done - new files are named using a new upload_to function, while old files continue to work.
However, this is messy - we end up in a situation with files like
/media/userID/oldfilename.pdf
and
/media/app/userID/projectID/newfilename.pdf
Is there a way to bulk rename those files? I guess it could be done by iterating through the database, checking if the path in FileField matches the result of current upload_to, and if not, rename.. it seems like a common problem, so perhaps there is a more generic way out there?
The simple solution is to write a custom Django management command. You can run the command using Django's standard manage.py.
Something like this:
from django.core.management.base import BaseCommand, CommandError
from example.models import YourModel
class Command(BaseCommand):
args = ''
help = ''
def handle(self, *args, **options):
# Get all objects
objects = YourModel.objects.all()
for object in objects: # For each object
# If old file path:
if not 'userID/projectID' in objects.filefield.name:
# Move the file, eg usign shutil http://docs.python.org/library/shutil.html#shutil.move
# Update filefield
# save object
I'm trying to find a way to cache the results of a query that won't change with frequency. For example, categories of products from an e-commerce (cellphones, TV, etc).
I'm thinking of using the template fragment caching, but in this fragment, I will iterate over a list of these categories. This list is avaliable in any part of the site, so it's in my base.html file. Do I have always to send the list of categories when rendering the templates? Or is there a more dynamic way to do this, making the list always available in the template?
Pop your cached query into Django's cache:
from django.core.cache import cache
cache.set('key', queryset)
Then create a context processor to add the value of the cache to all templates:
# myproject/myapp/context_processors.py
from django.core.cache import cache
def cached_queries():
return {'cache', cache.get('key')}
Then add your context processor in your Django settings file:
TEMPLATE_CONTEXT_PROCESSORS += (
'myproject.myapp.context_processors.cached_queries'
)
Now you will be able to access the cache variable in all generic templates and all templates which have a requests context, which a template is given if this is done in the view:
return render_to_response('my_template.html',
my_data_dictionary,
context_instance=RequestContext(request))
When to Set the Cache
It depends on what is contained in the cache. However a common problem is that Django only really gets to execute Python whenever a page request is sent, and this is often not where you want to do this kind of work.
An alternative is to create a custom management command for a particular app. You can then either run this manually when necessary, or more commonly set this to run as a cron job.
To create a management command you must create a class decended from Command inside of a management/commands directory located inside of an app:
# myproject/myapp/management/commands/update_cache.py
from django.core.management.base import NoArgsCommand
from django.core.cache import cache
class Command(NoArgsCommand):
help = 'Refreshes my cache'
def handle_noargs(self, **options):
cache.set('key', queryset)
The name of this file is important as this will be the name of the command. In this case you can now call this on the command line:
python manage.py update_cache
You can also use johnny-cache for automatic caching of querysets. It will (by default) cache all querysets, but you can force it not to cache some.