i'm trying to make a custom plotly-graphic on a wagtail homepage.
I got this far. I'm overriding the wagtail Page-model by altering the context returned to the template. Am i doing this the right way, is this possible in models.py ?
Thnx in advanced.
from django.db import models
from wagtail.models import Page
from wagtail.fields import RichTextField
from wagtail.admin.panels import FieldPanel
import psycopg2
from psycopg2 import sql
import pandas as pd
import plotly.graph_objs as go
from plotly.offline import plot
class CasPage(Page):
body = RichTextField(blank=True)
content_panels = Page.content_panels + [
FieldPanel('body'),
]
def get_connection(self):
try:
return psycopg2.connect(
database="xxxx",
user="xxxx",
password="xxxx",
host="xxxxxxxxxxxxx",
port=xxxxx,
)
except:
return False
conn = get_connection()
cursor = conn.cursor()
strquery = (f'''SELECT t.datum, t.grwaarde - LAG(t.grwaarde,1) OVER (ORDER BY datum) AS
gebruiktgas
FROM XXX
''')
data = pd.read_sql(strquery, conn)
fig1 = go.Figure(
data = data,
layout=go.Layout(
title="Gas-verbruik",
yaxis_title="aantal M3")
)
output = plotly.plot(fig1, output_type='div', include_plotlyjs=False)
# https://stackoverflow.com/questions/32626815/wagtail-views-extra-context
def get_context(self, request):
context = super(CasPage, self).get_context(request)
context['output'] = output
return context
Kind of the right track. You should move all the plot code into its own method though. At the moment, it runs the plot code when the site initialises then stays stored in memory.
There's three usual ways to get the plot to the rendered page then.
As you've done with context
As a property or method of the page class
As a template tag called from the template
The first two have more or less the same effect, except the 2nd makes the property available anywhere, not just the template. The context method runs before the page starts rendering, the other two happen during that process. I guess the only real difference there is that if you're using template caching, the context will always run each time the page is loaded, the other two only run when the cache is invalid, or if the code is escaped out of the cache (for fragment caching).
To call the plot as a property of your page class, you'd just pull out the code into a def with the #property decorator:
class CasPage(Page):
....
#property
def plot(self):
try:
conn = psycopg2.connect(
database="xxxx",
user="xxxx",
password="xxxx",
host="xxxxxxxxxxxxx",
port=xxxxx,
)
cursor = conn.cursor()
strquery = (f'''SELECT t.datum, t.grwaarde - LAG(t.grwaarde,1) OVER (ORDER BY datum) AS
gebruiktgas FROM XXX''')
data = pd.read_sql(strquery, conn)
fig1 = go.Figure(
data = data,
layout=go.Layout(
title="Gas-verbruik",
yaxis_title="aantal M3")
)
return plotly.plot(fig1, output_type='div', include_plotlyjs=False)
except Exception as e:
print(f"{type(e).__name__} at line {e.__traceback__.tb_lineno} of {__file__}: {e}")
return None
^ I haven't tried this code ... it should work as is, but no guarantees I didn't make a typo ;)
Now you can access your plot with {{ self.plot }} in the template.
If you want to stick with context, then you'd stay with the def above but just amend your output line to
context['output'] = self.plot
Template tags are more useful when they're being used in StructBlocks and not part of a page class like this, or where you have code that you want to re-use in multiple templates.
Then you'd move all that plot code into a template tag file, register it and call it in the template with {% plot %}. Wagtail template tags work the same as Django: https://docs.djangoproject.com/en/4.1/howto/custom-template-tags/
Is the plot data outside of the site database? If not, you could probably get the data via the ORM if it was defined as a model. If so, it's probably worth writing a view (or stored procedure if you want to pass parameters) on the db server and calling that rather than hard coding the SQL into your python.
The other consideration is the page load time - if the dataset is big, this could take a while and prevent the page from loading. You'd probably want a front-end solution in that case.
I am fairly new to django and I have the problem of creating full access for a site. The user has to give some additional information to get full access after signing up. I want the full access to automatically expire after 6 months. I defined a custom user model with the extra condition:
models.py
from django.db import models
from django.contrib.auth.models import AbstractUser
class CustomUser(AbstractUser):
full_name = models.CharField(blank=True, max_length=255)
has_full_access = models.BooleanField(default=False)
#some other stuff
After typing in some data for getting full access, the user gets redirected to this view which sets the boolean to true:
views.py
def data_gathered_done(request):
current_user = CustomUser.objects.get(id=request.user.id)
current_user.has_full_access = True
current_user.save()
#some other stuff
I want this boolean field to automatically reset to it's default (False) 6 months after the full access has been granted. How can I do that?
I'd do it with a property on the Model.
from datetime import datetime, timedelta
from django.db import models
from django.contrib.auth.models import AbstractUser
expire_after = timedelta(days=180)
class CustomUser(AbstractUser):
full_name = models.CharField(blank=True, max_length=255)
full_access_since = models.DatetimeField(auto_add_now=True)
#some other stuff
#property
def has_full_access(self):
return datetime.now() - expire_after < self.full_access_since
Then you can use the Boolean normally
from django.http import HttpResponseForbidden
user = CostumUser.objects.get(pk=123)
if not user.has_full_access:
return HttpResponseForbidden()
I'm a little late to this question, but I had a somewhat similar problem recently where I needed a boolean "lock" that "expired" after a 90 minutes. I didn't want to install any third-party dependencies or packages to do this.
The scenario: When a user accesses an "edit mode" view/template from a given model instance's detail view, I need to lock out all other users to prevent concurrency issues.
However after X minutes, I want others to be able to edit so I needed the UI menu options to revert back.
(Note: In my case I have to deal with concurrency at the database level as well, but this solution deals with the UI.) However, the logic could be extended to handle other time-based access issues within a site or webapp.
If I handled this only client side (say with AJAX), a user might lock a model and potentially their computer blows up, hence no AJAX fires to unlock. Has to be back-end. Like the answer above, a function that checks timestamps on the model seems like the way to go, but then again I have users all over the world - how do I deal with daylight savings and different timezones?
My solution was to use a non-DST timezone as a time constant so I didn't have to worry about that. Who cares what timezone I'm benchmarking - it's just a back-end method that checks durations.
models.py
class SomeProduct(models.Model):
name = models.CharField()
description = models.TextField()
lock = models.BooleanField(default=False)
timestamp = models.DateTimeField(null=True, blank=True, auto_now_add=False)
def __str__(self):
return str(self.name)
views.py
import datetime
import pytz
def update_product_view(request, slug): # This view shows forms and locks out other editors
qs = SomeProduct.objects.get(slug=slug):
if qs.lock == False:
qs.lock = True
now = datetime.datetime.now(pytz.timezone('US/Hawaii')) #Hawaii time is constant, no DST
qs.timestamp = now
qs.save()
elif qs.lock == True:
now = datetime.datetime.now(pytz.timezone('US/Hawaii'))
qs.timestamp = now
qs.save()
else:
pass
# Forms and other view logic here...
def product_view(request, slug): # This view unlocks the model if enough time has passed
qs = SomeProduct.objects.get(slug=slug):
if qs is not None:
try: # in case no timestamp has been set
now = datetime.datetime.now(pytz.timezone('US/Hawaii'))
then = qs.timestamp
delta = (now-then).total_seconds() # compare the difference
minutes = 60 #seconds
if delta > 90*minutes:
qs.lock = False # if 90 or more minutes have passed, unlock the model
qs.save()
else:
pass
except:
pass
else:
pass
# context and other view logic here...
template
{% if obj.lock == True %}
# adjust edit options or hide buttons accordingly
{% else obj.lock == False %}
# show button that leads to edit view url
{% endif %}
This is a pretty simplified version of my code, but the basics are there. I also have some JS on the front end that informs the user with a timeclock, exit edit mode URL that unlocks the model, etc. Your needs may vary. If anybody has some perspective on how I can make this better or any "gotchas" I'd love to learn something so please share. For now this works!
I recently upgraded from Django Mezzanine from 1.4 to 3.1.4. The transition has been smooth except an error with models which extend the Mezzanine Page class. When I call the get_FOO_display property on any choice field, I get the short name with & between each character. For instance, if I have the test class:
from mezzanine.pages.models import Page
class TestModel(Page):
CHOICES = (
('ab', "Aardvarks and Bubblegum"),
('cd', "Coocoos and Diphtheria"),
)
prop = models.CharField(max_length=2, choices=CHOICES)
I get the following in the Django shell:
In [1]: from project.models import TestModel
In [2]: test = TestModel(prop="ab")
In [3]: test.get_prop_display()
Out[3]: u'a & b'
If I my model simply extends models.Model instead of Page, get_prop_display() works as expected and I get Out[3]: Aardvarks and Bubblegum
Any insights are appreciated.
update
It has been fixed.
Ref to the code:
def contribute_to_class(self, cls, name):
def _get_FIELD_display(self, field):
value = getattr(self, field.attname)
value = force_text(" & ".join([dict(field.choices).get(v, v)
for v in value]), strings_only=True)
return value
setattr(cls, '_get_FIELD_display', _get_FIELD_display)
super(MultiChoiceField, self).contribute_to_class(cls, name)
MultiChoiceField unconditionally overrides the _get_FIELD_display method of the model it resides in. In your code, that model is Page and the field is Page.in_menus.
Comparing with Django's logic, the above code may cause incorrect behavior when Page or MultiChoiceField is used in your model.
Perhaps it's a bug, and here I've raised an issue. It's fixed now.
It seems like it should be easy to run "explain" directly off of a queryset in Django, but I don't see anything obvious for how to do it, and "explain" is a difficult thing to search for in the docs.
Well, there seems to be nothing out there except a toolbar so I wrote my own mixin to give me an explain() method on my querysets:
from django.db import connections
from django.db.models.query import QuerySet
class QuerySetExplainMixin:
def explain(self):
cursor = connections[self.db].cursor()
cursor.execute('explain %s' % str(self.query))
return cursor.fetchall()
QuerySet.__bases__ += (QuerySetExplainMixin,)
Hopefully this is useful to others.
QuerySet.explain(), available in Django 2.1.0 and above, is now the official way to explain queries.
Just a slight modification to guidoism's answer. This prevents getting a ProgrammingError: syntax error at or near ... error caused by the parameters not being correctly escaped in the raw query:
from django.db import connections
from django.db.models.query import QuerySet
class QuerySetExplainMixin:
def explain(self):
cursor = connections[self.db].cursor()
query, params = self.query.sql_with_params()
cursor.execute('explain %s' % query, params)
return '\n'.join(r[0] for r in cursor.fetchall())
QuerySet.__bases__ += (QuerySetExplainMixin,)
To use, simply invoke explain() at the end of your queryset, e.g.:
print SomeModel.objects.filter(...).explain()
There is a lot of documentation on how to serialize a Model QuerySet but how do you just serialize to JSON the fields of a Model Instance?
You can easily use a list to wrap the required object and that's all what django serializers need to correctly serialize it, eg.:
from django.core import serializers
# assuming obj is a model instance
serialized_obj = serializers.serialize('json', [ obj, ])
If you're dealing with a list of model instances the best you can do is using serializers.serialize(), it gonna fit your need perfectly.
However, you are to face an issue with trying to serialize a single object, not a list of objects. That way, in order to get rid of different hacks, just use Django's model_to_dict (if I'm not mistaken, serializers.serialize() relies on it, too):
from django.forms.models import model_to_dict
# assuming obj is your model instance
dict_obj = model_to_dict( obj )
You now just need one straight json.dumps call to serialize it to json:
import json
serialized = json.dumps(dict_obj)
That's it! :)
To avoid the array wrapper, remove it before you return the response:
import json
from django.core import serializers
def getObject(request, id):
obj = MyModel.objects.get(pk=id)
data = serializers.serialize('json', [obj,])
struct = json.loads(data)
data = json.dumps(struct[0])
return HttpResponse(data, mimetype='application/json')
I found this interesting post on the subject too:
http://timsaylor.com/convert-django-model-instances-to-dictionaries
It uses django.forms.models.model_to_dict, which looks like the perfect tool for the job.
There is a good answer for this and I'm surprised it hasn't been mentioned. With a few lines you can handle dates, models, and everything else.
Make a custom encoder that can handle models:
from django.forms import model_to_dict
from django.core.serializers.json import DjangoJSONEncoder
from django.db.models import Model
class ExtendedEncoder(DjangoJSONEncoder):
def default(self, o):
if isinstance(o, Model):
return model_to_dict(o)
return super().default(o)
Now use it when you use json.dumps
json.dumps(data, cls=ExtendedEncoder)
Now models, dates and everything can be serialized and it doesn't have to be in an array or serialized and unserialized. Anything you have that is custom can just be added to the default method.
You can even use Django's native JsonResponse this way:
from django.http import JsonResponse
JsonResponse(data, encoder=ExtendedEncoder)
It sounds like what you're asking about involves serializing the data structure of a Django model instance for interoperability. The other posters are correct: if you wanted the serialized form to be used with a python application that can query the database via Django's api, then you would wan to serialize a queryset with one object. If, on the other hand, what you need is a way to re-inflate the model instance somewhere else without touching the database or without using Django, then you have a little bit of work to do.
Here's what I do:
First, I use demjson for the conversion. It happened to be what I found first, but it might not be the best. My implementation depends on one of its features, but there should be similar ways with other converters.
Second, implement a json_equivalent method on all models that you might need serialized. This is a magic method for demjson, but it's probably something you're going to want to think about no matter what implementation you choose. The idea is that you return an object that is directly convertible to json (i.e. an array or dictionary). If you really want to do this automatically:
def json_equivalent(self):
dictionary = {}
for field in self._meta.get_all_field_names()
dictionary[field] = self.__getattribute__(field)
return dictionary
This will not be helpful to you unless you have a completely flat data structure (no ForeignKeys, only numbers and strings in the database, etc.). Otherwise, you should seriously think about the right way to implement this method.
Third, call demjson.JSON.encode(instance) and you have what you want.
If you want to return the single model object as a json response to a client, you can do this simple solution:
from django.forms.models import model_to_dict
from django.http import JsonResponse
movie = Movie.objects.get(pk=1)
return JsonResponse(model_to_dict(movie))
If you're asking how to serialize a single object from a model and you know you're only going to get one object in the queryset (for instance, using objects.get), then use something like:
import django.core.serializers
import django.http
import models
def jsonExample(request,poll_id):
s = django.core.serializers.serialize('json',[models.Poll.objects.get(id=poll_id)])
# s is a string with [] around it, so strip them off
o=s.strip("[]")
return django.http.HttpResponse(o, mimetype="application/json")
which would get you something of the form:
{"pk": 1, "model": "polls.poll", "fields": {"pub_date": "2013-06-27T02:29:38.284Z", "question": "What's up?"}}
.values() is what I needed to convert a model instance to JSON.
.values() documentation: https://docs.djangoproject.com/en/3.0/ref/models/querysets/#values
Example usage with a model called Project.
Note: I'm using Django Rest Framework
from django.http import JsonResponse
#csrf_exempt
#api_view(["GET"])
def get_project(request):
id = request.query_params['id']
data = Project.objects.filter(id=id).values()
if len(data) == 0:
return JsonResponse(status=404, data={'message': 'Project with id {} not found.'.format(id)})
return JsonResponse(data[0])
Result from a valid id:
{
"id": 47,
"title": "Project Name",
"description": "",
"created_at": "2020-01-21T18:13:49.693Z",
}
I solved this problem by adding a serialization method to my model:
def toJSON(self):
import simplejson
return simplejson.dumps(dict([(attr, getattr(self, attr)) for attr in [f.name for f in self._meta.fields]]))
Here's the verbose equivalent for those averse to one-liners:
def toJSON(self):
fields = []
for field in self._meta.fields:
fields.append(field.name)
d = {}
for attr in fields:
d[attr] = getattr(self, attr)
import simplejson
return simplejson.dumps(d)
_meta.fields is an ordered list of model fields which can be accessed from instances and from the model itself.
Here's my solution for this, which allows you to easily customize the JSON as well as organize related records
Firstly implement a method on the model. I call is json but you can call it whatever you like, e.g.:
class Car(Model):
...
def json(self):
return {
'manufacturer': self.manufacturer.name,
'model': self.model,
'colors': [color.json for color in self.colors.all()],
}
Then in the view I do:
data = [car.json for car in Car.objects.all()]
return HttpResponse(json.dumps(data), content_type='application/json; charset=UTF-8', status=status)
Use list, it will solve problem
Step1:
result=YOUR_MODELE_NAME.objects.values('PROP1','PROP2').all();
Step2:
result=list(result) #after getting data from model convert result to list
Step3:
return HttpResponse(json.dumps(result), content_type = "application/json")
Use Django Serializer with python format,
from django.core import serializers
qs = SomeModel.objects.all()
serialized_obj = serializers.serialize('python', qs)
What's difference between json and python format?
The json format will return the result as str whereas python will return the result in either list or OrderedDict
To serialize and deserialze, use the following:
from django.core import serializers
serial = serializers.serialize("json", [obj])
...
# .next() pulls the first object out of the generator
# .object retrieves django object the object from the DeserializedObject
obj = next(serializers.deserialize("json", serial)).object
All of these answers were a little hacky compared to what I would expect from a framework, the simplest method, I think by far, if you are using the rest framework:
rep = YourSerializerClass().to_representation(your_instance)
json.dumps(rep)
This uses the Serializer directly, respecting the fields you've defined on it, as well as any associations, etc.
It doesn't seem you can serialize an instance, you'd have to serialize a QuerySet of one object.
from django.core import serializers
from models import *
def getUser(request):
return HttpResponse(json(Users.objects.filter(id=88)))
I run out of the svn release of django, so this may not be in earlier versions.
ville = UneVille.objects.get(nom='lihlihlihlih')
....
blablablab
.......
return HttpResponse(simplejson.dumps(ville.__dict__))
I return the dict of my instance
so it return something like {'field1':value,"field2":value,....}
how about this way:
def ins2dic(obj):
SubDic = obj.__dict__
del SubDic['id']
del SubDic['_state']
return SubDic
or exclude anything you don't want.
This is a project that it can serialize(JSON base now) all data in your model and put them to a specific directory automatically and then it can deserialize it whenever you want... I've personally serialized thousand records with this script and then load all of them back to another database without any losing data.
Anyone that would be interested in opensource projects can contribute this project and add more feature to it.
serializer_deserializer_model
Let this is a serializers for CROPS, Do like below. It works for me, Definitely It will work for you also.
First import serializers
from django.core import serializers
Then you can write like this
class CropVarietySerializer(serializers.Serializer):
crop_variety_info = serializers.serialize('json', [ obj, ])
OR you can write like this
class CropVarietySerializer(serializers.Serializer):
crop_variety_info = serializers.JSONField()
Then Call this serializer inside your views.py
For more details, Please visit https://docs.djangoproject.com/en/4.1/topics/serialization/
serializers.JSONField(*args, **kwargs) and serializers.JSONField() are same. you can also visit https://www.django-rest-framework.org/api-guide/fields/ for JSONField() details.