I need to make a get query like:
obj = Current.objects.get(Code='M01.C0001')
But the query giving "Multiple Objects Returned' error because of the database has another record with similar unicode string 'M01.Ç0001'
[<obj: M01.Ç0001>, <obj: M01.C0001>]
I try to fetch data with field lookup functions, but it does not work anyway.
I googled around but I didn't find a way to temporarily set the Collation for this query.
Is it possible to temporarily set collation during executing a get query in Django 1.3?
SOLUTION:
I solved my problem with using raw django query with adding COLLATE to sql string.
obj = Current.objects.raw("SELECT * FROM Current WHERE Code = 'M01.C0001' COLLATE utf8_bin;")
Collation is a database property, so you cannot do that.
Change collation to database.
Related
I have a model with some HStoreField attributes and I can't seem to use Django's ORM HStoreField to query those values using LIKE.
When doing Model.objects.filter(hstoreattr__values__contains=['text']), the queryset only contains rows in which hstoreattr has any value that matches text exactly.
What I'm looking for is a way to search by, say, te instead of text and those same rows be returned as well. I'm aware this is possible in a raw PostgreSQL query but I'm looking for a solution that uses Django ORM.
If you want to check value of particular key in every object if it contains 'te', you can do:
Model.objects.filter(hstoreattr__your_key__icontains='te')
If you want to check if any key in your hstore field contains 'te', you will need to create your own lookup in django, because by default django won't do such thing. Refer to custom lookups in django docs for more info.
As far as I can remember, you cannot filter in values. If you want to filter in values, you have to pass a column and value you are referencing to. When you want it to be case insensitive use __icontains.
Although you cannot filter by all values, you can filter by all keys. Just like you showed in your code.
If you want to search for 'text' in all objects in key named let's say 'fo' - just do smth like this:
Model.objects.filter(hstoreattr__icontains={'fo': 'text'})
In one of the django apps we use two database engine A and B, both are the same database but with different schemas. We have a table called C in both schemas but using db routing it's always made to point to database B. We have formed a valuelist queryset from one of the models in A, tried to pass the same in table C using filter condition __in but it always fetches empty though there are matching records. When we convert valueslist queryset to a list and use it in table C using filter condition __in it works fine.
Not working
data = modelindbA.objects.values_list('somecolumn',flat=True)
info = C.objects.filter(somecolumn__in=data).values_list
Working
data = modelindbA.objects.values_list('somecolumn',flat=True)
data = list(data)
info = C.objects.filter(somecolumn__in=data).values_list
I have read django docs and other SO questions, couldn't find anything relative. My guess is that since both models are in different database schemas the above is not working. I need assistance on how to troubleshoot this issue.
When you use a queryset with __in, Django will construct a single SQL query that uses a subquery for the __in clause. Since the two tables are in different databases, no rows will match.
By contrast, if you convert the first queryset to a list, Django will go ahead and fetch the data from the first database. When you then pass that data to the second query, hitting the second database, it will work as expected.
See the documentation for the in field lookup for more details:
You can also use a queryset to dynamically evaluate the list of values instead of providing a list of literal values.... This queryset will be evaluated as subselect statement:
SELECT ... WHERE blog.id IN (SELECT id FROM ... WHERE NAME LIKE '%Cheddar%')
Because values_list method returns django.db.models.query.QuerySet, not a list.
When you use it with same schema the orm optimise it and should make just one query, but when schemas are different it fails.
Just use list().
I would even recommend to use it for one schema since it can decrease complexity of query and work better on big tables.
Is it possible to make query with a collation different from database table have?
Using extra() is a little messy. Something similar can now be achieved with Func() expression (since Django 1.8):
username_ci = Func(
'username',
function='utf8_general_ci',
template='(%(expressions)s) COLLATE "%(function)s"')
This can be used in annotate():
User.objects.annotate(uname_ci=username_ci).filter(uname_ci='joeblow').exists()
Or in order_by() to override default collation rules when sorting:
User.objects.order_by(username_ci)
Now, it still may seem messy, but if you look at the docs and code of Func(), you will discover that it is very easy to subclass it and make a reusable collation setter.
I used this trick with Postgres database.
Here is how you can use a specific collation instead of the default collation for a given table/column. I'm assuming you always want that to be the case insensitive utf8_general_ci, but you can easily change that in the code or add it as a variable.
Note the use of the params kwarg instead of the db literal function. Params exists for the exact same purpose.
def iexact(**kw):
fields = [['%s=%%s collate utf8_general_ci'%field,value] for (field,value) in kw.items()]
return dict(where=[f[0] for f in fields], params=[f[1] for f in fields])
if User.objects.extra(**iexact(username='joeblow')).exists():
status = "Found a user with this username!"
I solve this using bit of a hack;
Django's extra method is just like raw method, they both using the query statetment directly;
MyModel.objects.extra(where=["name LIKE '%%" + name + "%%' COLLATE utf8_general_ci"])
But like this sql injection is possible. We need to escape name variable. I searched a lot for a function which just escapes a string for db. Found one in MySQL-python package but it can't escape unicode strings. Also package has literal method in connection but to use it we need an instance (maybe it is for db characteristic).
At last I used Django's db.connection.cursor.
from django.db import connection
cursor = connection.cursor()
name = cursor.db.connection.literal(name)[1:-1] # [1:-1] excluding quotes
With this way we also need an instance but I suppose this not require a db connection. And I suppose this method db independent. If I am wrong please correct me.
This above solution works. In case of getting the reverse order the following snippet
sort_value = sort.strip()
if sort_value in ['name', '-name']:
sort = Func('name', function='C', template='(%(expressions)s) COLLATE "%(function)s"')
if sort_value in ['-name']:
f_res = queryset.order_by(sort).reverse()
else:
f_res = queryset.order_by(sort)
return f_res
Is it possible to prevent multiple querys when i use django ORM ? Example:
product = Product.objects.get(name="Banana")
for provider in product.providers.all():
print provider.name
This code will make 2 SQL querys:
1 - SELECT ••• FROM stock_product WHERE stock_product.name = 'Banana'
2 - SELECT stock_provider.id, stock_provider.name FROM stock_provider INNER JOIN stock_product_reference ON (stock_provider.id = stock_product_reference.provider_id) WHERE stock_product_reference.product_id = 1
I confess, i use Doctrine (PHP) for some projects. With doctrine it's possible to specify joins when retrieve the object (relations are populated in object, so no need to query database again for get attribute relation value).
Is it possible to do the same with Django's ORM ?
PS: I hop my question is comprehensive, english is not my primary language.
In Django 1.4 or later, you can use prefetch_related. It's like select_related but allows M2M relations and such.
product = Product.objects.prefetch_related('providers').get(name="Banana")
You still get two queries, though. From the docs:
prefetch_related, on the other hand, does a separate lookup for each relationship, and does the ‘joining’ in Python.
As for packing this down into a single query, Django won't do it like Doctrine because it doesn't do that much post-processing of the result set (Django would have to remove all the redundant column data, since you'll get a row per provider and each of these rows will have a copy of all of product's fields).
So if you want to pack this down to one query, you're going to have to turn it around and run the query on the Provider table (I'm guessing at your schema):
providers = Provider.objects.filter(product__name="Banana").select_related('product')
This should pack it down to one query, but you won't get a single product ORM object out of it, instead needing to get the product fields via providers[k].product.
You can use prefetch_related, sometimes in combination with select_related, to get all related objects in a single query: https://docs.djangoproject.com/en/1.5/ref/models/querysets/#prefetch-related
I'm trying to create a custom field which would automatically add COLLATE information into the WHERE part of SQL query:
class IgnoreDiacriticsField(models.TextField):
def get_prep_lookup(self, lookup_type, value):
if lookup_type == 'exact':
return ' "' + self.get_prep_value(value) + '" COLLATE utf8_general_ci'
when I perform a query like this:
result = ModelClass.objects.filter(field='value')
then nothing is found, even though the query (print result.query) is valid and matches several rows. Am I doing something wrong?
The reason why I'm adding the collation iformation is that I want perform queries on those fields and ignore any diacritics.
Are you using MySQL 1.2.1p2 by any chance? From the Django documentation
If you're using MySQLdb 1.2.1p2, Django's standard CharField class
will return unicode strings even with utf8_bin collation. However,
TextField fields will be returned as an array.array instance (from
Python's standard array module). There isn't a lot Django can do about
that, since, again, the information needed to make the necessary
conversions isn't available when the data is read in from the
database. This problem was fixed in MySQLdb 1.2.2, so if you want to
use TextField with utf8_bin collation, upgrading to version 1.2.2 and
then dealing with the bytestrings (which shouldn't be too difficult)
as described above is the recommended solution.