So I've been using a Postgres database with the extension Postgis installed, and then using a Django setup to draw spatial data on a Leaflet OpenStreetMap. It has been a bit difficult translating my sql queries to the database functionality that Django is using whenever you're communicating with your database. Often I seem to be missing Postgis functions such as ST_LineCrossingDirection or ST_FrechetDistance.
How do I get to use those functions along with .annotate and .filter from Django without having to write custom sql queries and executing those?
I've tried to look into F() and Func() from Django as well, but I don't think that solves my issue as it seem to be using built in aggregate functions. I also tried to execute RawSQL in an annotate function to perform the function ST_LineCrossingDirection but it would require me to write a WHERE clause and the condition is something I'm not aware of until I get to the filter() call where I'm using intersects() between two geometries.
Anyway .. If anyone knows how to use what I assume is missing Postgis functions please let me know. Because my code is getting quite messy and ineffective.
Thanks, and all help is appreciated!
You should take a look at GeoFunc, it's the GeoDjango equivalent of Django's Func.
from django.contrib.gis.db.models.functions import GeoFunc
class LineCrossingDirection(GeoFunc):
function='ST_LineCrossingDirection'
class FrechetDistance(GeoFunc):
function='ST_FrechetDistance'
Related
You can query Django's JSONField, either by direct lookup, or by using annotations. Now I realize if you annotate a field, you can all sorts of complex queries, but for the very basic query, which one is actually the preferred method?
Example: Lets say I have model like so
class Document(models.Model):
data = JSONField()
And then I store an object using the following command:
>>> Document.objects.create(data={'name': 'Foo', 'age': 24})
Now, the query I want is the most basic: Find all documents where data__name is 'Foo'. I can do this 2 ways, one using annotation, and one without, like so:
>>> from django.db.models.expressions import RawSQL
>>> Document.objects.filter(data__name='Foo')
>>> Document.objects.annotate(name = RawSQL("(data->>'name')::text", [])).filter(name='Foo')
So what exactly is the difference? And if I can make basic queries, why do I need to annotate? Provided of course I am not going to make complex queries.
There is no reason whatsoever to use raw SQL for queries where you can use ORM syntax. For someone who is conversant in SQL but less experienced with Django's ORM, RawSQL might provide an easier path to a certain result than the ORM, which has its own learning curve.
There might be more complex queries where the ORM runs into problems or where it might not give you the exact SQL query that you need. It is in these cases that RawSQL comes in handy – although the ORM is getting more feature-complete with every iteration, with
Cast (since 1.10),
Window functions (since 2.0),
a constantly growing array of wrappers for database functions
the ability to define custom wrappers for database functions with Func expressions (since 1.8) etc.
They are interchangable so it's matter of taste. I think Document.objects.filter(data__name='Foo') is better because:
It's easier to read
In the future, MariaDB or MySql can support JSON fields and your code will be able to run on both PostgreSQL and MariaDB.
Don't use RawSQL as a general rule. You can create security holes in your app.
I have old database and need custom select.
in sql i can use:
SELECT x(geometry) FROM table
but I dont know how to force x() function in django select.
You can create a custom function using Expressions.
class GeoX(Func):
function = 'X'
MyModel.objects.annotate(x=GeoX('field'))
This will result in the geometry X function being invoked and it's result being annotated to a field labeled x in your model. However this is the inferior solution. The better solution is to install geodjango. That provides full access to almost every single function in PostGIS. Also works with spatialite and msyql spatial extensions.
I managed to solve problem using Func()
Func(F('geometry'), function='x')
I have the following function to determine who downloaded a certain book:
#cached_property
def get_downloader_info(self):
return self.downloaders.select_related('user').values(
'user__username', 'user__full_name')
Since I'm only using two fields, does it make sense to use .defer() on the remaining fields?
I tried to use .only(), but I get an error that some fields are not JSON serializable.
I'm open to all suggestions, if any, for optimizing this queryset.
Thank you!
Before you try every possible optimization, you should get your hands on the SQL query generated by the ORM (you can print it to stdout or use something like django debug toolbar) and see what is slow about it. After that I suggest you run that query with EXPLAIN ANALYZE and find out what is slow about that query. If the query is slow because lot of data has to be transfer than it makes lot of sense to use only or defer. Using only and defer (or values) gives you better performances only if you need to retrieve lot of data, but it does not make your database job much easier (unless you really have to read a lot of data of course).
Since you are using Django and Postgresql, you can get a psql session with manage.py dbshell and get query timings with \timing
I love being able to write quick and dirty query strings right into the URL of the Django admin. Like: /admin/myapp/mymodel/?pub_date__year=2011
AND statements are just as easy: /admin/myapp/mymodel/?pub_date__year=2011&author=Jim
I'm wondering if it's possible to issue an 'OR' statement via the URL. Anyone heard of such functionality?
Django < 1.4 doesn't support OR queries. Sometimes it is possible to translate OR queries to __in - queries which are supported (they are equivalent to OR queries but only for single field values).
You can also upgrade to django development version: it has more versatile list_filter implementation (see https://docs.djangoproject.com/en/dev//ref/contrib/admin/#django.contrib.admin.ModelAdmin.list_filter ) which can be used for providing advanced admin filters (including OR-queries).
The & is not a logical AND, even though it seems to be acting that way in your case. I'm pretty certain there is no way to create a logical OR in the GET query string.
i'm triyng to make a full text search with postgresql and django So I've created a function search_client(text) which returns a list of clients. To call it from the DB i use something like this:
SELECT * FROM search_client('something')
and i'm not really sure how to call it from django. i know i could do something like
cursor = connection.cursor()
cursor.execute("SELECT * FROM search_client('something')")
result = cursor.fetchall()
but that will only return a list of values, and i'd like to have a list of objects, like when i use the "filter()" method.
Any ideas?? thanks for your time!
If your goal is a full-featured search engine, have a look at django-haystack. It rocks.
As for your question, the new (Django 1.2) raw method might work:
qs = MyModel.objects.raw("SELECT * FROM search_client('something')")
If you're using Django 1.2, you can use the raw() ORM method to execute custom SQL but get back Django models. If you're not, you can still execute the SQL via the extra() method on default QuerySet, and pump it into a custom method to either then go pull the real ORM records, or make new, temporary, objects
First, you probably don't want to do this. Do you have proof that your database function is actually faster?
Implement this in Python first. When you can prove that your Python implementation really is the slowest part of your transaction, then you can try a stored procedure.
Second, you have the extra method available in Django.
http://docs.djangoproject.com/en/1.2/ref/models/querysets/#django.db.models.QuerySet.extra
Note that compute-intensive database procedures are often slow.