I am using Django 1.7 and Postgres 9.4. I would like to create an index using C collation on a VARCHAR column in my database.
This is because C-collated indexes can apparently make SELECT... LIKE queries much faster, and I'm noticing that these queries are slow in my database, even though they are using a varchar_pattern_ops index.
From the Django models file, is it possible to specify that a particular index should use C collation? The documentation doesn't mention this, so I assume not.
If I create the index manually in Postgres and specify C collation, can I then continue to use the Django ORM with startswith queries and use the index? In other words, will these queries from Django automatically specify C collation, or will they continue to ask for UTF-8 collation, thus bypassing the index?
Perhaps I could get round this by specifying that the whole database should use C collation, but I'm not sure. Any advice would be appreciated.
Related
My Django JSON field contains a list of values like ["N05BB01", "R06AX33"].
atc_code = JSONField(default=list())
I would like to filter this field, for 'does any string in list contain "N05"?'.
like
mymodel.objects.filter(?????)
In this case if you are not using SQLite or Oracle, you can use contains:
mymodel.objects.filter(atc_code__contains='N05')
Which generates this SQL:
SELECT * FROM "mymodel" WHERE UPPER("mymodel"."atc_code"::text) LIKE UPPER(%N05%)
Relational-based answer
Usually, such an approach (with a list of values in JSONField) can be possible when the relational structure is used in the wrong way.
The best approach here:
Create a new model that describes your atc_code entity. For example AtcCode
Depends on the meaning of the atc_code and its relation to the MainEnity use ForeignKeyField or ManyToManyField
Utilize all pros from the relational database and powerful Django ORM with such built-in features as filtering, adding, removing, querying with any database backend.
It will work on any supported database. A relational database will work faster when you are using relations properly.
My recommendation is to use JSONField when you have a really unstructured object.
I started investigating why my Django Model.objects.filter(condition = variable).order_by(textcolumn) queries do not yield objects in correct order. And found out that it is database (Postgresql) issue.
In my earlier question (Postgresql sorting language specific characters (collation)) i figured out (with a lot of help from zero323 in actually getting it to work) that i can specify collation per database query like this:
SELECT nimi COLLATE "et_EE" FROM test ORDER BY nimi ASC;
But as much as i can see, order_by only accepts field names as arguments.
I was wondering, that if it is somehow possible to extend that functionality to include also the collation parameter? Is it possible to hack it in somehow using mixins or whatnot? Or is feature request the only way to do this right now?
I wish it would work something like this:
Model.objects.filter(condition = variable).order_by(*fieldnames, collation = 'et_EE')
Edit1:
Apparently im not the only one to ask for this:
https://groups.google.com/forum/#!msg/django-developers/0iESVnawNAY/JefMfAm7nQMJ
Alan
As #olau menioned in the comment, since Django 3.2 Collate utility is available. For older Django versions see the original information below the following code sample:
# Starting with Django 3.2:
Test.objects.order_by(Collate('nimi', 'et_EE'))
Since Django 1.8 order_by() accepts not only field names but also query expressions.
In another answer I gave an example of how you can override the default collation for a column. The useful query expression here is Func(), which you may subclass or use directly:
nimi_et = Func(
'nimi',
function='et_EE',
template='(%(expressions)s) COLLATE "%(function)s"')
Test.objects.order_by(nimi_et.asc())
Yet, note that the resulting SQL will be more like:
SELECT nimi FROM test ORDER BY nimi COLLATE "et_EE" ASC;
That is, the collation is overridden in ORDER BY clause rather than in SELECT clause. However, if you need to use it in a WHERE clause, you can use Func() in annotate().
allright. It seems that right now the Raw Queries are the only way to do this.
But There is django ticket open which will hopefully be closed/resolved sometime soon.
Can Django support Oracle nested tables or varrays or collections in some manner? Asking just for completeness as our project is reworking the data model, attempting to move away from EAV organization, but I don't like creating a bucket load of dependent supporting tables for each main entity.
e.g.
(not the proper Oracle syntax, but gets the idea across)
Events
eventid
report_id
result_tuple (result_type_id, result_value)
anomaly_tuple(anomaly_type_id, anomaly_value)
contributing_factors_tuple(cf_type_id, cf_value)
etc,
where the can be multiple rows of the tuples for one eventid
each of these tuples can, of course exist as separate tables, but this seems to be more concise. If it 's something Django can't do, or I can't modify the model classes to do easily, then perhaps just having django create the extra tables is the way to go.
--edit--
I note that django-hstore is doing something very similar to what I want to do, but using postgresql's hstore capability. Maybe I can branch off of that for an Oracle nested table implementation. I dunno...I'm pretty new to python and django, so my reach may exceed my grasp in this case.
Querying a nested table gives you a cursor to traverse the tuples, one member of which is yet another cursor, so you can get the rows from the nested table.
I'm reading that I can use raw SQL in Django and have Django actually build my models from the results.
However I'm wondering what happens if I use joins in the raw SQL. How will Django know what models to use?
(Are there any other issues I should be aware of?)
It's not the joins that matter, but the column names. You could, for example, do the following:
SELECT table.id, other_table.name AS name from table join other_table using (id)
and pass that into your table model. Django would then treat the names from other_table as though they were names from table and give your normal table instances. I can't imagine why you would want to do that though...
The important thing to remember is that Django is using a very simple mapping from your SQL to its model structure. You can subvert it if you want, but you'll probably end up with some hard to maintain code.
My initial question was here and was related to the postgres backend. Postgres subquery, ordering by subquery
Now my problem has moved onwards to the Django ORM layer. I essentially want to order a query by a postgres function ('idx', taken from the above stackoverflow work)
I've gone through trying to use model.objects.extra(order_by ) or simply order_by but I believe both of these need the order_by parameter to be an attribute or a field known to Django.
I'm trying to think how to solve this without having to revert to using an entirely raw SQL query through a model manager.
You can use extra to add the function result to your query and then order by it. Something like:
MyModel.objects.extra(select={'idx': 'idx(foo, bar)'}, order_by=['idx'])