Django templatetag "order of processing" - django

I am trying to write a set of template tags that allow you to easily specify js and css files from within the template files themselves. Something along the lines of {% requires global.css %}, and later in the request, {% get_required_css %}.
I have this mostly working, but there are a couple of issues. We'll start with the 'timing' issues.
Each template tag is made up of two steps, call/init and render. Every call/init happens before any render procedure is called. In order to guarantee that all of the files are queued before the {% get_required_css %} is rendered, I need to build my list of required files in the call/init procedures themselves.
So, I need to collect all of the files into one bundle per request. The context dict is obviously the place for this, but unfortunately, the call/init doesn't have access to the context variable.
Is this making sense? Anyone see a way around this (without resorting to a hack-y global request object)?
Another possibility to store these in a local dict but they would still need to be tied to the request somehow... possibly some sort of {% start_requires %} tag? But I have no clue how to make that work either.

I've come up with a way to do this which more suits your needs. It will have a bit more load on the server, but proper caching can help to alleviate most of that. Below I've outlined a way that should work if the CSS includes are the same for each path. You'll need to create a single view to include all of these files, but you can actually optimize your CSS using this method, making only a single CSS call for each page.
import md5
class LoadCss(template.Node):
def __init__(self, tag_name, css):
self.css = css
self.tag_name = tag_name
def render(self, context):
request = context['request']
md5key = md5.new(request.path).hexdigest()
if md5key not in request.session:
request.session[md5key] = list()
## This assumes that this method is being called in the correct output order.
request.session[md5key].append(self.css)
return '<!-- Require %s -->' % self.css
def do_load_css(parser, token):
tag_name, css = token.split_contents()
return LoadCss(tag_name, key)
register.tag('requires', do_load_css)
class IncludeCss(template.Node):
def __init__(self, tag_name):
self.tag_name = tag_name
def render(self, context):
request = context['request']
md5key = md5.new(request.path).hexdigest()
return '<link rel="stylesheet" href="/path/to/css/view/%s">' % md5key
def do_include_css(parser, token):
return IncludeCss(token)
register.tag('get_required_css', do_include_css)
views.py:
from django.conf import settings
from django.views.decorators.cache import cache_page
import os
#cache_page(60 * 15) ## 15 Minute cache.
def css_view(request, md5key):
css_requires = request.session.get(md5key, list())
output = list()
for css in css_requires:
fname = os.path.join(settings.MEDIA_ROOT, 'css', css) ## Assumes MEDIA_ROOT/css/ is where the CSS files are.
f = open(fname, 'r')
output.append(f.read())
HttpResponse(''.join(output), mimetype="text/css")
This allows you to store the CSS information in the context, then in the session, and serve the output from a view (with caching to make it faster). This will, of course, have a bit more server overhead.
If you need to vary the CSS on more than just the path, then you can simply modify the md5 lines to suit your needs. You have access to the entire request object, and the context, so almost everything should be in there.
Beware: On second review, this may cause a race condition if the browser fetches the CSS before the session has been populated. I do not believe Django works that way, but I don't feel like looking it up right now.

Related

Wagtail add functions to models.py

i'm trying to make a custom plotly-graphic on a wagtail homepage.
I got this far. I'm overriding the wagtail Page-model by altering the context returned to the template. Am i doing this the right way, is this possible in models.py ?
Thnx in advanced.
from django.db import models
from wagtail.models import Page
from wagtail.fields import RichTextField
from wagtail.admin.panels import FieldPanel
import psycopg2
from psycopg2 import sql
import pandas as pd
import plotly.graph_objs as go
from plotly.offline import plot
class CasPage(Page):
body = RichTextField(blank=True)
content_panels = Page.content_panels + [
FieldPanel('body'),
]
def get_connection(self):
try:
return psycopg2.connect(
database="xxxx",
user="xxxx",
password="xxxx",
host="xxxxxxxxxxxxx",
port=xxxxx,
)
except:
return False
conn = get_connection()
cursor = conn.cursor()
strquery = (f'''SELECT t.datum, t.grwaarde - LAG(t.grwaarde,1) OVER (ORDER BY datum) AS
gebruiktgas
FROM XXX
''')
data = pd.read_sql(strquery, conn)
fig1 = go.Figure(
data = data,
layout=go.Layout(
title="Gas-verbruik",
yaxis_title="aantal M3")
)
output = plotly.plot(fig1, output_type='div', include_plotlyjs=False)
# https://stackoverflow.com/questions/32626815/wagtail-views-extra-context
def get_context(self, request):
context = super(CasPage, self).get_context(request)
context['output'] = output
return context
Kind of the right track. You should move all the plot code into its own method though. At the moment, it runs the plot code when the site initialises then stays stored in memory.
There's three usual ways to get the plot to the rendered page then.
As you've done with context
As a property or method of the page class
As a template tag called from the template
The first two have more or less the same effect, except the 2nd makes the property available anywhere, not just the template. The context method runs before the page starts rendering, the other two happen during that process. I guess the only real difference there is that if you're using template caching, the context will always run each time the page is loaded, the other two only run when the cache is invalid, or if the code is escaped out of the cache (for fragment caching).
To call the plot as a property of your page class, you'd just pull out the code into a def with the #property decorator:
class CasPage(Page):
....
#property
def plot(self):
try:
conn = psycopg2.connect(
database="xxxx",
user="xxxx",
password="xxxx",
host="xxxxxxxxxxxxx",
port=xxxxx,
)
cursor = conn.cursor()
strquery = (f'''SELECT t.datum, t.grwaarde - LAG(t.grwaarde,1) OVER (ORDER BY datum) AS
gebruiktgas FROM XXX''')
data = pd.read_sql(strquery, conn)
fig1 = go.Figure(
data = data,
layout=go.Layout(
title="Gas-verbruik",
yaxis_title="aantal M3")
)
return plotly.plot(fig1, output_type='div', include_plotlyjs=False)
except Exception as e:
print(f"{type(e).__name__} at line {e.__traceback__.tb_lineno} of {__file__}: {e}")
return None
^ I haven't tried this code ... it should work as is, but no guarantees I didn't make a typo ;)
Now you can access your plot with {{ self.plot }} in the template.
If you want to stick with context, then you'd stay with the def above but just amend your output line to
context['output'] = self.plot
Template tags are more useful when they're being used in StructBlocks and not part of a page class like this, or where you have code that you want to re-use in multiple templates.
Then you'd move all that plot code into a template tag file, register it and call it in the template with {% plot %}. Wagtail template tags work the same as Django: https://docs.djangoproject.com/en/4.1/howto/custom-template-tags/
Is the plot data outside of the site database? If not, you could probably get the data via the ORM if it was defined as a model. If so, it's probably worth writing a view (or stored procedure if you want to pass parameters) on the db server and calling that rather than hard coding the SQL into your python.
The other consideration is the page load time - if the dataset is big, this could take a while and prevent the page from loading. You'd probably want a front-end solution in that case.

Django Sitetree returning "expected str instance, LazyTitle found" in {% sitetree_page_title from menu %}

Django version: 3.2.9 & 4.0.4
Python version:3.8.10
OS: Ubuntu 20.04lts
I have a sitetree that is using a context variable in the title. When going to that path and loading the template with {% sitetree_page_title from menu %} it returns sequence item 1: expected str instance, LazyTitle found.
I didn't find any information on this exact issue, but it seems that when using context data in the title a LazyTitle object is generated. The object is not being converted to a string so the template engine is bombing.
I will include the relevant code below. I was able to get around the issue by editing sitetreeapp.py and wrapping the return of get_current_page_title() in str(), but that feels excessive.
The app in question is a small test app I'm working to layout an example CRUD app with the features our company needs. It does nothing fancy. The app isn't 100% complete yet so if anything looks out of place, it could be I haven't got to that point, but this portion should be working fine.
The sitetree in question is loaded dynamically via the config.ready method. Menu items without context variables are working without issue. I have verified that the context variable in question is available within the template. Hopefully it's something simple that I am overlooking. Any input is appreciated. I should also note that, while I have used StackOverflow for many years, I haven't posted much so please forgive my post formatting.
sitetree.py - note insert and list are working fine
from sitetree.utils import tree, item
sitetrees = [[
# setup the base tree
tree('app_model_test', items=[
# Then define items and their children with `item` function.
item('App Test', 'test_app:app_test_home',
hint='Test CRUD App',
children=[
item('Search Results', 'test_app:app_test_list',
in_menu=False, in_sitetree=False),
item('Insert Record', 'test_app:app_test_insert',
hint="Insert a new record",
),
item('Update Record {{ object.ssn }}', 'test_app:app_test_update object.id',
in_menu=False, in_sitetree=False),
])
]),
], ]
urls.py
app_name = 'test_app'
urlpatterns = [
path('insert/', views.app_test_form, name='app_test_insert'),
path('update/<int:id>/', views.app_test_form, name='app_test_update'),
path('list/', views.appTestListView.as_view(), name='app_test_list'),
]
views.py - relevant pieces
def app_test_form(request, action='add', id=0):
if id > 0:
test_obj = get_object_or_404(appTestModel, id=id)
form = appTestForm(instance=test_obj)
if request.method == 'GET':
if id == 0:
form = appTestForm()
test_obj = None
else:
if id == 0:
form = appTestForm(request.POST)
if form.is_valid():
form.save()
return redirect('/list')
# this will have to be somewhere else and contain whatever app apps are
# installed to get all the menu entries
menus = ['app_model_test']
context = {'form': form,
'title': 'app Test',
'action': action,
'menus': menus,
'object': test_obj}
return render(request, 'CRUD_base.html', context)
dyn_tree_register.py
from sitetree.sitetreeapp import register_dynamic_trees, compose_dynamic_tree
from sitetree.utils import tree, item
from . import sitetree as app_test_tree
register_dynamic_trees(
[
compose_dynamic_tree(*app_test_tree.sitetrees),
],
# Line below tells sitetree to drop and recreate cache, so that all newly registered
# dynamic trees are rendered immediately.
reset_cache=True
)
apps.py
from django.apps import AppConfig
class AppModelTestConfig(AppConfig):
default_auto_field = 'django.db.models.BigAutoField'
name = 'app_model_test'
def ready(self):
from . import dyn_tree_register
and lastly the relevant template portion:
{% load sitetree %}
{% for menu in menus %}
<h1 class="title" id="page-title">{% sitetree_page_title from menu %}</h1>
{% endfor %}
As I mentioned, I can make this work by modifying sitetreeapp.py and wrapping the return of get_current_page_title in str():
def get_current_page_title(self, tree_alias: str, context: Context) -> str:
"""Returns resolved from sitetree title for current page.
:param tree_alias:
:param context:
"""
return str(self.get_current_page_attr('title_resolved', tree_alias, context))
Edit - 2022-04-13
As a temporary workaround I created a wrapper template tag that calls the sitetree_page_title and then renders it and wraps the output in str(). This seems hackish to me so I appreciate any insight. Short term this will get me by but I would rather not put something like this into production as I feel there's a bug on my side rather than within sitetree otherwise more people would have run into this.
custom tag and template.Node class
#register.tag()
def app_str_sitetree_page_title(parser, token):
'''
There is an issue causing sitetree's page title function to return a LazyPageTitle object which is not being converted to a string.
This function is a temporary fix until the issue is resolved to force the return to a string.
'''
from sitetree.templatetags import sitetree
ret = sitetree.sitetree_page_title(parser, token)
return StrSitetreePageTitle(ret)
class StrSitetreePageTitle(template.Node):
'''
This is the render wrapper to ensure a string is returned from sitetree_page_title
Like app_str_sitetree_page_title this is temporary
'''
def __init__(self, title_obj) -> None:
self.title_obj = title_obj
def render(self, context):
title_render = self.title_obj.render(context)
return str(title_render)

using two templates from one view

I am trying to present content from a view in two ways: html and csv download. The only way I was able to do it was to use 2 different views, one for html presentation and one for csv. This duplicates my code and I am looking for a more elegant solution.
Any suggestions?
Here is the sample code:
# views.py
[...]
def member_list(request):
member_list = Member.objects.all()
return render_to_response("member_list.html",
{'member_list':member_list)
def member_csv_list(request):
member_list = Member.objects.all()
csv_list = HttpResponse(content_type='text/csv')
csv_list['Content-Disposition'] = 'attachment; filename="member_list.csv"'
writer = csv.writer(csv_list)
writer.writerow(['Name', 'Member Type', 'Rooms'])
for member in member_list:
fields = [member.name, member.member_type, member.room]
writer.writerow(fields)
return member_list
You can use a parameter in your url and implement a view like
def myview(request) :
type = request.GET.get('type', 'html')
# do processing
if type == 'html':
# return html
else if type == 'csv':
# return csv
If you access a url like http://yourserver/myview?type=csv it will render the csv part of the view. When the url http://yourserver/myview is accessed it will return the html part of the view.
Rohan's answer is absolutely the right paradigm. For an excellent tutorial-style introduction to this topic, cf. Multiple Templates in Django.
Here are a few quotes (all credit goes to Scott Newman).
To serve a printable version of an article, for example, we can add ?printable to the end of the URL.
To make it work, we'll add an extra step in our view to check the URL for this variable. If it exists, we'll load up a printer-friendly template file. If it doesn't exist, we'll load the normal template file.
def detail(request, pid):
'''
Accepts a press release ID and returns the detail page
'''
p = get_object_or_404(PressRelease, id=pid)
if request.GET.has_key('printable'):
template_file = 'press/detail_printable.html'
else:
template_file = 'press/detail.html'
t = loader.get_template(template_file)
c = Context({'press': p})
return HttpResponse(t.render(c))
He continues with template overrides and different templates by domain names. All this is excellent.

Altering one query parameter in a url (Django)

I have a search page that takes a variety of parameters. I want to create a new URL by just altering one parameter in the query. Is there an easy way to do this - something like:
# example request url
http://example.com/search?q=foo&option=bar&option2=baz&change=before
# ideal template code
{% url_with change 'after' %}
# resulting url
http://example.com/search?q=foo&option=bar&option2=baz&change=after
So this would take the request url, alter one query parameter and then return the new url. Similar to what can be achieved in Perl's Catalyst using $c->uri_with({change => 'after'}).
Or is there a better way?
[UPDATED: removed references to pagination]
I did this simple tag which doesn't require any extra libraries:
#register.simple_tag
def url_replace(request, field, value):
dict_ = request.GET.copy()
dict_[field] = value
return dict_.urlencode()
Use as:
<a href="?{% url_replace request 'param' value %}">
It wil add 'param' to your url GET string if it's not there, or replace it with the new value if it's already there.
You also need the RequestContext request instance to be provided to your template from your view. More info here:
http://lincolnloop.com/blog/2008/may/10/getting-requestcontext-your-templates/
So, write a template tag around this:
from urlparse import urlparse, urlunparse
from django.http import QueryDict
def replace_query_param(url, attr, val):
(scheme, netloc, path, params, query, fragment) = urlparse(url)
query_dict = QueryDict(query).copy()
query_dict[attr] = val
query = query_dict.urlencode()
return urlunparse((scheme, netloc, path, params, query, fragment))
For a more comprehensive solution, use Zachary Voase's URLObject 2, which is very nicely done.
Note:
The urlparse module is renamed to urllib.parse in Python 3.
I improved mpaf's solution, to get request directly from tag.
#register.simple_tag(takes_context = True)
def url_replace(context, field, value):
dict_ = context['request'].GET.copy()
dict_[field] = value
return dict_.urlencode()
This worked pretty well for me. Allows you to set any number of parameters in the URL. Works nice for a pager, while keeping the rest of the query string.
from django import template
from urlobject import URLObject
register = template.Library()
#register.simple_tag(takes_context=True)
def url_set_param(context, **kwargs):
url = URLObject(context.request.get_full_path())
path = url.path
query = url.query
for k, v in kwargs.items():
query = query.set_param(k, v)
return '{}?{}'.format(path, query)
Then in the template:
<a href="{% url_set_param page=last %}">
There are a number of template tags for modifying the query string djangosnippets.org:
http://djangosnippets.org/snippets/553/
http://djangosnippets.org/snippets/826/
http://djangosnippets.org/snippets/1243/
I would say those are the most promising looking. One point in all of them is that you must be using django.core.context_processors.request in your TEMPLATE_CONTEXT_PROCESSORS.
You can try https://github.com/dcramer/django-paging
In addition to the snippets mentioned by Mark Lavin, Here's a list of other implementations I could find for a Django template tag which modifies the current HTTP GET query string.
On djangosnippets.org:
#2237 Manipulate URL query strings using context variables using a template tag by JHsaunders
#2332 Querystring Builder - create urls with GET params by jibberia
my favorite: #2413 Yet another query string template tag by atms
#2428 Add GET parameters from current request by naktinis
On PyPI:
django-spurl by Jamie Matthews
django-urltags by Calloway Project/Corey Oordt
the add_query_param filter in django-rest-framework by Tom Christie
On GitHub:
update_querystring by David Gouldin

Invalidating Memcached Keys on save() in Django

I've got a view in Django that uses memcached to cache data for the more highly trafficked views that rely on a relatively static set of data. The key word is relatively: I need invalidate the memcached key for that particular URL's data when it's changed in the database. To be as clear as possible, here's the meat an' potatoes of the view (Person is a model, cache is django.core.cache.cache):
def person_detail(request, slug):
if request.is_ajax():
cache_key = "%s_ABOUT_%s" % settings.SITE_PREFIX, slug
# Check the cache to see if we've already got this result made.
json_dict = cache.get(cache_key)
# Was it a cache hit?
if json_dict is None:
# That's a negative Ghost Rider
person = get_object_or_404(Person, display = True, slug = slug)
json_dict = {
'name' : person.name,
'bio' : person.bio_html,
'image' : person.image.extra_thumbnails['large'].absolute_url,
}
cache.set(cache_key)
# json_dict will now exist, whether it's from the cache or not
response = HttpResponse()
response['Content-Type'] = 'text/javascript'
response.write(simpljson.dumps(json_dict)) # Make sure it's all properly formatted for JS by using simplejson
return response
else:
# This is where the fully templated response is generated
What I want to do is get at that cache_key variable in it's "unformatted" form, but I'm not sure how to do this--if it can be done at all.
Just in case there's already something to do this, here's what I want to do with it (this is from the Person model's hypothetical save method)
def save(self):
# If this is an update, the key will be cached, otherwise it won't, let's see if we can't find me
try:
old_self = Person.objects.get(pk=self.id)
cache_key = # Voodoo magic to get that variable
old_key = cache_key.format(settings.SITE_PREFIX, old_self.slug) # Generate the key currently cached
cache.delete(old_key) # Hit it with both barrels of rock salt
# Turns out this doesn't already exist, let's make that first request even faster by making this cache right now
except DoesNotExist:
# I haven't gotten to this yet.
super(Person, self).save()
I'm thinking about making a view class for this sorta stuff, and having functions in it like remove_cache or generate_cache since I do this sorta stuff a lot. Would that be a better idea? If so, how would I call the views in the URLconf if they're in a class?
URLConf should point to any callable. There's no strict requirement to make it point to function exactly. You could implement base class with your cache methods then extend it:
class RealView(BaseViewWithCacheMethods):
def __call__(self, request):
if request.is_ajax():
return self.ajax_view()
return self.html_view()
URLConf definition would be something like that:
from django.conf.urls.defaults import *
from views import RealView
urlpattrens = patterns('',
(r'^$', RealView()),
)