Can I order the results of QuerySet by values inside of HStoreField, for example I've got model:
class Product(model.Models):
name = CharField(max_length=100)
properties = HStoreField()
And I want to store some properties of my product in HStoreField like:
{ 'discount': '10', 'color': 'white'}
In view I want to order the resulting QuerySet by discount.
The above answer does not work. Order transforms were never implemented for HStoreField, see https://code.djangoproject.com/ticket/24747.
But the suggestion in https://code.djangoproject.com/ticket/24592 works. Here is some more detail.
from django.contrib.gis.db.models import TextField, HStoreField, Model
from django.db.models import F, Func, Value
class MyThing(Model):
name: TextField()
keys: HStoreField()
things = [MyThing(name='foo'),
MyThing(name='bar'),
MyThing(name='baz')]
things[0].keys['movie'] = 'Jaws'
things[1].keys['movie'] = 'Psycho'
things[2].keys['movie'] = 'The Birds'
things[0].keys['rating'] = 5
things[1].keys['rating'] = 4
things[2].keys['year'] = '1963'
# Informal search
MyThing.objects\
.filter(keys__has_key='rating')\
.order_by(Func(F('keys'), Value('movie'),
function='',
arg_joiner=' -> ',
output_field=TextField()))
The formal search is exactly as described in the second link above. Use the imports in the above snippet with that code.
This might work (I cannot test right now):
.order_by("properties -> 'discount'")
But be aware that HSTORE values are strings, so you do not get numeric order but instead items are ordered as strings.
Do get proper numeric order, you should extract the properties->discount key as separate column and cast it to integer.
Related
Is it possible to filter a Django queryset by model property?
i have a method in my model:
#property
def myproperty(self):
[..]
and now i want to filter by this property like:
MyModel.objects.filter(myproperty=[..])
is this somehow possible?
Nope. Django filters operate at the database level, generating SQL. To filter based on Python properties, you have to load the object into Python to evaluate the property--and at that point, you've already done all the work to load it.
I might be misunderstanding your original question, but there is a filter builtin in python.
filtered = filter(myproperty, MyModel.objects)
But it's better to use a list comprehension:
filtered = [x for x in MyModel.objects if x.myproperty()]
or even better, a generator expression:
filtered = (x for x in MyModel.objects if x.myproperty())
Riffing off #TheGrimmScientist's suggested workaround, you can make these "sql properties" by defining them on the Manager or the QuerySet, and reuse/chain/compose them:
With a Manager:
class CompanyManager(models.Manager):
def with_chairs_needed(self):
return self.annotate(chairs_needed=F('num_employees') - F('num_chairs'))
class Company(models.Model):
# ...
objects = CompanyManager()
Company.objects.with_chairs_needed().filter(chairs_needed__lt=4)
With a QuerySet:
class CompanyQuerySet(models.QuerySet):
def many_employees(self, n=50):
return self.filter(num_employees__gte=n)
def needs_fewer_chairs_than(self, n=5):
return self.with_chairs_needed().filter(chairs_needed__lt=n)
def with_chairs_needed(self):
return self.annotate(chairs_needed=F('num_employees') - F('num_chairs'))
class Company(models.Model):
# ...
objects = CompanyQuerySet.as_manager()
Company.objects.needs_fewer_chairs_than(4).many_employees()
See https://docs.djangoproject.com/en/1.9/topics/db/managers/ for more.
Note that I am going off the documentation and have not tested the above.
Looks like using F() with annotations will be my solution to this.
It's not going to filter by #property, since F talks to the databse before objects are brought into python. But still putting it here as an answer since my reason for wanting filter by property was really wanting to filter objects by the result of simple arithmetic on two different fields.
so, something along the lines of:
companies = Company.objects\
.annotate(chairs_needed=F('num_employees') - F('num_chairs'))\
.filter(chairs_needed__lt=4)
rather than defining the property to be:
#property
def chairs_needed(self):
return self.num_employees - self.num_chairs
then doing a list comprehension across all objects.
I had the same problem, and I developed this simple solution:
objects = [
my_object
for my_object in MyModel.objects.all()
if my_object.myProperty == [...]
]
This is not a performatic solution, it shouldn't be done in tables that contains a large amount of data. This is great for a simple solution or for a personal small project.
PLEASE someone correct me, but I guess I have found a solution, at least for my own case.
I want to work on all those elements whose properties are exactly equal to ... whatever.
But I have several models, and this routine should work for all models. And it does:
def selectByProperties(modelType, specify):
clause = "SELECT * from %s" % modelType._meta.db_table
if len(specify) > 0:
clause += " WHERE "
for field, eqvalue in specify.items():
clause += "%s = '%s' AND " % (field, eqvalue)
clause = clause [:-5] # remove last AND
print clause
return modelType.objects.raw(clause)
With this universal subroutine, I can select all those elements which exactly equal my dictionary of 'specify' (propertyname,propertyvalue) combinations.
The first parameter takes a (models.Model),
the second a dictionary like:
{"property1" : "77" , "property2" : "12"}
And it creates an SQL statement like
SELECT * from appname_modelname WHERE property1 = '77' AND property2 = '12'
and returns a QuerySet on those elements.
This is a test function:
from myApp.models import myModel
def testSelectByProperties ():
specify = {"property1" : "77" , "property2" : "12"}
subset = selectByProperties(myModel, specify)
nameField = "property0"
## checking if that is what I expected:
for i in subset:
print i.__dict__[nameField],
for j in specify.keys():
print i.__dict__[j],
print
And? What do you think?
i know it is an old question, but for the sake of those jumping here i think it is useful to read the question below and the relative answer:
How to customize admin filter in Django 1.4
It may also be possible to use queryset annotations that duplicate the property get/set-logic, as suggested e.g. by #rattray and #thegrimmscientist, in conjunction with the property. This could yield something that works both on the Python level and on the database level.
Not sure about the drawbacks, however: see this SO question for an example.
Given the following Django models:
class Room(models.Model):
name = models.CharField(max_length=20)
class Beacon(models.Model):
room = models.ForeignKey(Room)
uuid = models.UUIDField(default=uuid.uuid4)
major = models.PostiveIntegerField(max_value=65536)
minor = models.PositiveIntegerField(max_value=65536)
The Beacon model is a bluetooth beacon relationship to the room.
I want to select all Rooms that match a given uuid, major, minor combination.
The catch is, that I want to order the rooms by the beacon that is nearest to me. Because of this, I need to be able to assign a value to each beacon dynamically, and then sort by it.
Is this possible with the Django ORM? In Django 1.8?
NOTE - I will know the ordering of the beacons beforehand, I will be using the order they are passed in the query string. So the first beacon (uuid, major, minor) passed should match the first room that is returned by the Room QuerySet
I am envisioning something like this, though I know this won't work:
beacon_order = [
beacon1 = 1,
beacon0 = 2,
beacon3 = 3,
]
queryset = Room.objects.annotate(beacon_order=beacon_order).\
order_by('beacon_order')
If you already know the order of the beacons, there's no need to sort within the QuerySet itself. Take an ordered list called beacon_list, which contains the beacons' primary keys in order, e.g. the item at index 0 is the closest beacon's primary key, the item at index 1 is the second closest beacon's PK, etc. Then use a list comprehension:
ordered_rooms = [Room.objects.get(pk=x) for x in beacon_list]
You don't have to use the PK either, you can use anything which identifies the given object in the database, e.g. the name field.
Looks like this works:
from django.db.models import Case, Q, When
beacons = request.query_params.getlist('beacon[]')
query = Q()
order = []
for pos, beacon in enumerate(beacons):
uuid, major, minor = beacon.split(':')
query |= Q(
beacon__uuid=uuid,
beacon__major=major,
beacon__minor=minor,
)
order.append(When(
beacon__uuid=uuid,
beacon__major=major,
beacon__minor=minor,
then=pos,
))
rooms = Room.objects.filter(query).order_by(Case(*order))
With the following models:
class Item(models.Model):
name = models.CharField(max_length=255)
attributes = models.ManyToManyField(ItemAttribute)
class ItemAttribute(models.Model):
attribute = models.CharField(max_length=255)
string_value = models.CharField(max_length=255)
int_value = models.IntegerField()
I also have an Item which has 2 attributes, 'color': 'red', and 'size': 3.
If I do any of these queries:
Item.objects.filter(attributes__string_value='red')
Item.objects.filter(attributes__int_value=3)
I will get Item returned, works as I expected.
However, if I try to do a multiple query, like:
Item.objects.filter(attributes__string_value='red', attributes__int_value=3)
All I want to do is an AND. This won't work either:
Item.objects.filter(Q(attributes__string_value='red') & Q(attributes__int_value=3))
The output is:
<QuerySet []>
Why? How can I build such a query that my Item is returned, because it has the attribute red and the attribute 3?
If it's of any use, you can chain filter expressions in Django:
query = Item.objects.filter(attributes__string_value='red').filter(attributes__int_value=3')
From the DOCS:
This takes the initial QuerySet of all entries in the database, adds a filter, then an exclusion, then another filter. The final result is a QuerySet containing all entries with a headline that starts with “What”, that were published between January 30, 2005, and the current day.
To do it with .filter() but with dynamic arguments:
args = {
'{0}__{1}'.format('attributes', 'string_value'): 'red',
'{0}__{1}'.format('attributes', 'int_value'): 3
}
Product.objects.filter(**args)
You can also (if you need a mix of AND and OR) use Django's Q objects.
Keyword argument queries – in filter(), etc. – are “AND”ed together. If you need to execute more complex queries (for example, queries with OR statements), you can use Q objects.
A Q object (django.db.models.Q) is an object used to encapsulate a
collection of keyword arguments. These keyword arguments are specified
as in “Field lookups” above.
You would have something like this instead of having all the Q objects within that filter:
** import Q from django
from *models import Item
#assuming your arguments are kwargs
final_q_expression = Q(kwargs[1])
for arg in kwargs[2:..]
final_q_expression = final_q_expression & Q(arg);
result = Item.objects.filter(final_q_expression)
This is code I haven't run, it's out of the top of my head. Treat it as pseudo-code if you will.
Although, this doesn't answer why the ways you've tried don't quite work. Maybe it has to do with the lookups that span relationships, and the tables that are getting joined to get those values. I would suggest printing yourQuerySet.query to visualize the raw SQL that is being formed and that might help guide you as to why .filter( Q() & Q()) is not working.
In sqlalchemy, I define a function whose argument is a dictionary may contain multiple key-values.I want to query according to the key-values. Here is my program:
def get_contact_conditions(kwds):
for fieldName, fieldValue in kwds.items():
condition = fieldName + "=:fieldName"
contact = session.query(Contact).filter(condition).params(fieldName = fieldValue).all()
session.commit()
return contact
this situation above is when the dictionary has only one key-values, but maybe there are two or three in it, then how to code condition and the values in params(). In another word how to code query clause.
Thank you!
If the only comparison you need is "=" then this should work:
def get_contact_conditions(kwds):
contact = session.query(Contact).filter_by(**kwds).all()
return contact
You can read about it here:
http://docs.sqlalchemy.org/en/rel_0_7/orm/query.html#sqlalchemy.orm.query.Query.filter_by
If you need arbitrary operators(<,>,in_,like,etc.) you can pass in a list of sqlalchemy clauses as arguments to the and_ function and pass the result to filter(). These expressions are generated when you compare a model class attribute to something, like:
Contact.id < 5
So you could do something like this:
from sqlalchemy.sql import and_
def get_contacts(*exps):
return session.query(Contact).filter(and_(*exps)).all()
# All contacts with an id less than 5 and a name that starts with user.
contacts = get_contacts(Contact.id < 5, Contact.name.like('user%'))
But you should consider that at this point the function is getting closer to just calling session.query directly.
You should read the sqlalchemy docs more to learn about the operators/comparisons that are available on model columns/relations. There are some really useful ones.
I have two tables like so:
class Collection(models.Model):
name = models.CharField()
class Image(models.Model):
name = models.CharField()
image = models.ImageField()
collection = models.ForeignKey(Collection)
I'd like to retrieve the first image out of every collection. I have attempted:
image_list = Image.objects.order_by('collection.id').distinct('collection.id')
but it didn't work out the way I expected :(
Any ideas?
Thanks.
Don't use dots to separate fields that span relations in Django; the double-underscore convention is used instead -- it means "follow this relation to get to this field"
this is more correct:
image_list = Image.objects.order_by('collection__id').distinct('collection__id')
However, it probably doesn't do what you want.
The concept of "first" doesn't always apply in relational databases the way you seem to be using it. For all of the records in the image table with the same collection id, there is no record which is 'first' or 'last' -- they're all just records. You could put another field on that table to define a specific order, or you could order by id, or alphabetically by name, but none of those will happen by default.
What will probably work best for you is to get the list of collections with one query, and then get a single item per collection, in separate queries:
collection_ids = Image.objects.values_list('collection', flat=True).distinct()
image_list = [
Image.objects.filter(collection__id=c)[0] for c in collection_ids
]
If you want to apply an order to the Images, to define which is 'first', then modify it like this:
collection_ids = Image.objects.values_list('collection', flat=True).distinct()
image_list = [
Image.objects.filter(collection__id=c).order_by('-id')[0] for c in collection_ids
]
You could also write raw SQL -- MySQL aggregation has the interesting property that fields which are not aggregated over can still appear in the final output, and essentially take a random value from the set of matching records. Something like this might work:
Image.objects.raw("SELECT image.* FROM app_image GROUP BY collection_id")
This query should get you one image from each collection, but you will have no control over which one is returned.
As written in my comment, you cannot use specific fields with distinct under MySQL. However, you can achieve the same result with the following:
from itertools import groupby
all_images = Image.objects.order_by('collection__id')
images_by_collection = groupby(all_images, lambda image: image.collection_id)
image_list = sum([group for key, group in images_by_collection], [])
Unfortunately, this results in a "bigger" query to the DB (all images are retrieved).
dict([(c.id, c.image_set.all()[0]) for c in Collection.objects.all()])
That will create a dictionary of the first image (by default ordering) in each collection, keyed by the collection's id. Be aware, though, that this will generate 1+N queries, where N is the total number of collection objects.
To get around that, you'll either need to wait for Django 1.4 and prefetch_related or use something like django-batch-select.
First get the distinct result, then do your filters.
I think you should try this one.
image_list = Image.objects.distinct()
image_list = image_list.order_by('collection__id')