I have two classes, Portfolio, and PortfolioImage.
class PortfolioImage(models.Model):
portfolio = models.ForeignKey('Portfolio', related_name='images')
...
class Portfolio(models.Model):
def num_images(self):
return self.images.count()
I want to write a "non-empty portfolio" manager for Portfolio, so that I can do:
queryset = Portfolio.nonempty.all()
I've tried doing something like this, but I don't think this is even close:
class NonEmptyManager(models.Manager):
def get_query_set(self):
return super(NonEmptyManager, self).get_query_set().filter(num_images > 0)
I don't really know where to start, and I'm finding the documentation a bit lacking in this area.
Any ideas? Thanks,
First of all according to documentation you cannot use model methods for lookup with filter/exclude clause. Then also you cannot use python operators (> in your case) with filter/exclude.
To resolve your task if you are using Django 1.1beta:
from django.db.models import Count
#...
def get_query_set(self):
return super(NonEmptyManager,self).get_query_set()\
.annotate(num_images=Count('images'))\
.filter(num_images__gt=0)
But this solution has some limitations.
Another way for Django >= 1.0:
def get_query_set(self):
return super(NonEmptyManager,self).get_query_set()\
.filter(images__isnull=True)
Related
I have an endpoint that can follow this format:
www.example.com/ModelA/2/ModelB/5/ModelC?word=hello
Model C has a FK to B, which has a FK to A. I should only ever see C's that correspond to the same A and B at one time. In the above example, we should...
filter C by those with an FK to B id = 5 and A id = 2
also filter C by the field 'word' that contains hello.
I know how to use the filter_queryset() method to accomplish #1:
class BaseModelCViewSet(GenericViewSet):
queryset = ModelC.objects.all()
class ModelCViewSet(BaseModelCViewSet, mixins.RetrieveModelMixin, mixins.ListModelMixin):
def filter_queryset(self, queryset):
return queryset.filter(ModelB=self.kwargs["ModelB"], ModelB__ModelA=self.kwargs["ModelA"])
I also know how to use a Filterset class to filter by fields on ModelC to accomplish #2
class ModelCFilterSet(GeoFilterSet):
word = CharFilter(field_name='word', lookup_expr='icontains')
But I can only get one or the other to work. If I add filterset_class = ModelCFilterSet to ModelCViewSet then it no longer does #1, and without it, it does not do #2.
How do I accomplish both? Ideally I want all of this in the ModelCFilterSet
Note - As hinted by the use of GeoFilterSet I will (later on) be using DRF to add a GIS query, this is just a simplified example. So I think that restricts me to using FilterSet classes in some manner.
I'm not sure this would be of help in your situation, but I often use nested urls in DRF in a way that would be convenient to perform the task. I use a library called drf-nested-routers that does part of the job, namely keeps track of the relations by the provided ids. Let me show an example:
# views.py
from rest_framework import exceptions, viewsets
class ModelBViewSet(viewsets.ModelViewSet):
# This is a viewset for the nested part that depends on ModelA
queryset = ModelB.objects.order_by('id').select_related('model_a_fk_field')
serializer_class = ModelBSerializer
filterset_class = ModelBFilterSet # more about it below
def get_queryset(self, *args, **kwargs):
model_a_entry_id = self.kwargs.get('model_a_pk')
model_a_entry = ModelA.objects.filter(id=model_a_entry_id).first()
if not model_a_entry:
raise exceptions.NotFound("MAYDAY")
return self.queryset.filter(model_c_fk_field=model_a_entry)
class ModelAViewSet(viewsets.ModelViewSet):
queryset = ModelA.objects.order_by('id')
serializer_class = ModelASerializer
# urls.py
from rest_framework_nested import routers
router = routers.SimpleRouter()
router.register('model-a', ModelAViewSet, basename='model_a')
model_a_router = routers.NestedSimpleRouter(router, 'model-a', lookup='model_a')
model_a_router.register('model-b', ModelBViewSet, basename='model_b')
...
In this case I can make a query like www.example.com/ModelA/2/ModelB/ that will only return the entries of ModelB that point to the object of ModelA with id 2. Likewise, www.example.com/ModelA/2/ModelB/5 will return only the corresponding object of ModelB in case it is related to ModelA-id2. A further level for ModelC would act correspondingly.
Sticking to the example, by now we have filtered the entries of ModelB related to a particular object of ModelA, that is we have received the relevant queryset. Next, we have to search for a particular subset within this queryset, and here's where FilterSet comes in play. The easiest way to customise its behaviour is by writing specific methods.
# filters.py
import django_filters
class ModelBFilterSet(django_filters.FilterSet):
word = django_filters.CharFilter(
method="get_word",
)
def get_word(self, queryset, name, value):
return queryset.filter(word__icontains=value)
In fact, you don't even have to use a method here; the way you pasted would work as well (word = CharFilter(field_name='word', lookup_expr='icontains')), I just wanted to point out that there is such an option too.
The filter starts its job with the queryset that has already been processed by the viewset and now it will just narrow down our sample using the given parameter.
I haven't tried this with a three-level nested URL, only checked on the example of two levels, but I think the third level should act in the same way.
Figured it out. So creating a filter_queryset() method overwrites the one that is in the GenericAPIView class (which I already knew).
However - that overwritten class is also responsible for using the FilterSet class that I defined. So by overwriting it, I also "broke" the FilterSet.
Solution was adding a super() to call the original class before the one I wrote:
def filter_queryset(self, queryset):
queryset = super().filter_queryset(queryset)
return queryset.filter(asset=self.kwargs["asset"], asset__program=self.kwargs["program"])
So I'm pretty new to Django, I started playing yesterday and have been playing with the standard polls tutorial.
Context
I'd like to be able to filter the active questions based on the results of a custom method (in this case it is the Question.is_open() method (fig1 below).
The problem as I understand it
When I try and access only the active questions using a filter like
questions.objects.filter(is_open=true) it fails. If I understand correctly this relies on a queryset exposed via a model manager which can only filter based on records within the sql database.
My questions
1) Am I approaching this problem in most pythonic/django/dry way ? Should I be exposing these methods by subclassing the models.Manager and generating a custom queryset ? (that appears to be the consensus online).
2) If I should be using a manager subclass with a custom queryset, i'm not sure what the code would look like. For example, should I be using sql via a cursor.execute (as per the documentation here, which seems very low level) ? Or is there a better, higher level way of achieving this in django itself ?
I'd appreciate any insights into how to approach this.
Thanks
Matt
My models.py
class Question(models.Model):
question_text = models.CharField(max_length=200)
pub_date = models.DateTimeField('date published',default=timezone.now())
start_date = models.DateTimeField('poll start date',default=timezone.now())
closed_date = models.DateTimeField('poll close date', default=timezone.now() + datetime.timedelta(days=1))
def time_now(self):
return timezone.now()
def was_published_recently(self):
return self.pub_date >= timezone.now() - datetime.timedelta(days=1)
def is_open(self):
return ((timezone.now() > self.start_date) and (timezone.now() < self.closed_date))
def was_opened_recently(self):
return self.start_date >= timezone.now() - datetime.timedelta(days=1) and self.is_open()
def was_closed_recently(self):
return self.closed_date >= timezone.now() - datetime.timedelta(days=1) and not self.is_open()
def is_opening_soon(self):
return self.start_date <= timezone.now() - datetime.timedelta(days=1)
def closing_soon(self):
return self.closed_date <= timezone.now() - datetime.timedelta(days=1)
[Update]
Just as a follow-up. I've subclassed the default manager with a hardcoded SQL string (just for testing), however, it fails as it's not an attribute
class QuestionManager(models.Manager):
def get_queryset(self):
return super().get_queryset()
def get_expired(self):
from django.db import connection
with connection.cursor() as cursor:
cursor.execute("""
select id, question_text, closed_date, start_date, pub_date from polls_question
where ( polls_question.start_date < '2017-12-24 00:08') and (polls_question.closed_date > '2017-12-25 00:01')
order by pub_date;""")
result_list = []
for row in cursor.fetchall():
p = self.model(id=row[0], question=row[1], closed_date=row[2], start_date=row[3], pub_date=row[4])
result_list.append(p)
return result_list
I'm calling the method with active_poll_list = Question.objects.get_expired()
but I get the exception
Exception Value:
'Manager' object has no attribute 'get_expired'
I'm really not sure I understand why this doesn't work. It must be my misunderstanding of how I should invoke a method that returns a queryset from the manager.
Any suggestions would be much appreciated.
Thanks
There are so many things in your question and I'll try to cover as many as possible.
When you're trying to get a queryset for a model, you can use only the field attributes as lookups. That means in your example that you can do:
Question.objects.filter(question_text='What's the question?')
or:
Question.objects.filter(question_text__icontains='what')
But you can't query a method:
Question.objects.filter(is_open=True)
There is no field is_open. It is a method of the model class and it can't be used when filtering a queryset.
The methods you have declared in the class Question might be better decorated as properties (#property) or as cached properties. For the later import this:
from django.utils.functional import cached_property
and decorate the methods like this:
#cached_property
def is_open(self):
# ...
This will make the calculated value avaiable as property, not as method:
question = Question.objects.get(pk=1)
print(question.is_open)
When you specify default value for time fields you very probably want this:
pub_date = models.DateTimeField('date published', default=timezone.now)
Pay attention - it is just timezone.now! The callable should be called when an entry is created. Otherwise the method timezone.now() will be called the first time the django app starts and all entries will have that time saved.
If you want to add extra methods to the manager, you have to assign your custom manager to the objects:
class Question(models.Model):
# the fields ...
objects = QuestionManager()
After that the method get_expired will be available:
Question.objects.get_expired()
I hope this helps you to understand some things that went wrong in your code.
Looks like i'd missed off brackets when defining Question.objects
It's still not working, but I think I can figure it out from here.
Suppose I have a model, MyModel, with a property method that uses another model's queryset.
class OtherModel(models.Model)
...
class MyModel(models.Model):
simple_attr = models.CharField('Yada yada')
#property
def complex_attr(self):
list_other_model = OtherModel.objects.all()
...
# Complex algorithm using queryset from 'OtherModel' and simple_attr
return result
This causes my get_queryset() method on MyModel to query the database to generate the list_other_model variable every time for every single row.
Which causes my MyModel ListView to generate hundreds of SQL queries. Not efficient.
How can I architect a Manager or get_queryset method to cache the variable list_other_model for each row when using MyModel.objects.all()?
I hope my question makes sense--I'm on my sixth shot of espresso, and still haven't found a way to reduce the db queries.
Not sure if this is the best way to do it, but it works.
If someone posts a better answer, I'll accept theirs.
class OtherModel(models.Model)
...
class MyModelManager(models.Manager):
def get_queryset(self):
self.model.list_other_model = OtherModel.objects.all()
return super(MyModelManager, self).get_queryset()
class MyModel(models.Model):
simple_attr = models.CharField('Yada yada')
list_other_model = None
objects = MyModelManager()
#property
def complex_attr(self):
...
# Complex algorithm using queryset from 'OtherModel' and simple_attr
return result
I have a private boolean flag on my model, and a custom manager that overwrites the get_query_set method, with a filter, removing private=True:
class myManager(models.Manager):
def get_query_set(self):
qs = super(myManager, self).get_query_set()
qs = qs.filter(private=False)
return qs
class myModel(models.Model):
private = models.BooleanField(default=False)
owner = models.ForeignKey('Profile', related_name="owned")
#...etc...
objects = myManager()
I want the default queryset to exclude the private models be default as a security measure, preventing accidental usage of the model showing private models.
Sometimes, however, I will want to show the private models, so I have the following on the manager:
def for_user(self, user):
if user and not user.is_authenticated():
return self.get_query_set()
qs = super(myManager, self).get_query_set()
qs = qs.filter(Q(owner=user, private=True) | Q(private=False))
return qs
This works excellently, with the limitation that I can't chain the filter. This becomes a problem when I have a fk pointing the myModel and use otherModel.mymodel_set. otherModel.mymodel_set.for_user(user) wont work because mymodel_set returns a QuerySet object, rather than the manager.
Now the real problem starts, as I can't see a way to make the for_user() method work on a QuerySet subclass, because I can't access the full, unfiltered queryset (basically overwriting the get_query_set) form the QuerySet subclass, like I can in the manager (using super() to get the base queryset.)
What is the best way to work around this?
I'm not tied to any particular interface, but I would like it to be as djangoy/DRY as it can be. Obviously I could drop the security and just call a method to filter out private tasks on each call, but I really don't want to have to do that.
Update
manji's answer below is very close, however it fails when the queryset I want isn't a subset of the default queryset. I guess the real question here is how can I remove a particular filter from a chained query?
Define a custom QuerySet (containing your custom filter methods):
class MyQuerySet(models.query.QuerySet):
def public(self):
return self.filter(private=False)
def for_user(self, user):
if user and not user.is_authenticated():
return self.public()
return self.filter(Q(owner=user, private=True) | Q(private=False))
Define a custom manager that will use MyQuerySet (MyQuerySet custom filters will be accessible as if they were defined in the manager[by overriding __getattr__]):
# A Custom Manager accepting custom QuerySet
class MyManager(models.Manager):
use_for_related_fields = True
def __init__(self, qs_class=models.query.QuerySet):
self.queryset_class = qs_class
super(QuerySetManager, self).__init__()
def get_query_set(self):
return self.queryset_class(self.model).public()
def __getattr__(self, attr, *args):
try:
return getattr(self.__class__, attr, *args)
except AttributeError:
return getattr(self.get_query_set(), attr, *args)
Then in the model:
class MyModel(models.Model):
private = models.BooleanField(default=False)
owner = models.ForeignKey('Profile', related_name="owned")
#...etc...
objects = myManager(MyQuerySet)
Now you can:
¤ access by default only public models:
MyModel.objects.filter(..
¤ access for_user models:
MyModel.objects.for_user(user1).filter(..
Because of (use_for_related_fields = True), this same manager wil be used for related managers. So you can also:
¤ access by default only public models from related managers:
otherModel.mymodel_set.filter(..
¤ access for_user from related managers:
otherModel.mymodel_set.for_user(user).filter(..
More informations: Subclassing Django QuerySets & Custom managers with chainable filters (django snippet)
To use the chain you should override the get_query_set in your manager and place the for_user in your custom QuerySet.
I don't like this solution, but it works.
class CustomQuerySet(models.query.QuerySet):
def for_user(self):
return super(CustomQuerySet, self).filter(*args, **kwargs).filter(private=False)
class CustomManager(models.Manager):
def get_query_set(self):
return CustomQuerySet(self.model, using=self._db)
If you need to "reset" the QuerySet you can access the model of the queryset and call the original manager again (to fully reset). However that's probably not very useful for you, unless you were keeping track of the previous filter/exclude etc statements and can replay them again on the reset queryset. With a bit of planning that actually wouldn't be too hard to do, but may be a bit brute force.
Overall manji's answer is definitely the right way to go.
So amending manji's answer you need to replace the existing "model"."private" = False with ("model"."owner_id" = 2 AND "model"."private" = True ) OR "model"."private" = False ). To do that you will need to walk through the where object on the query object of the queryset to find the relevant bit to remove. The query object has a WhereNode object that represents the tree of the where clause, with each node having multiple children. You'd have to call the as_sql on the node to figure out if it's the one you are after:
from django.db import connection
qn = connection.ops.quote_name
q = myModel.objects.all()
print q.query.where.children[0].as_sql(qn, connection)
Which should give you something like:
('"model"."private" = ?', [False])
However trying to do that is probably way more effort than it's worth and it's delving into bits of Django that are probably not API-stable.
My recommendation would be to use two managers. One that can access everything (an escape hatch of sort), the other with the default filtering applied. The default manager is the first one, so you need to play around with the ordering depending on what you need to do. Then restructure your code to know which one to use - so you don't have the problem of having the extra private=False clause in there already.
I have these models:
def Foo(Models.model):
size = models.IntegerField()
# other fields
def is_active(self):
if check_condition:
return True
else:
return False
def Bar(Models.model):
foo = models.ForeignKey("Foo")
# other fields
Now I want to query Bars that are having active Foo's as such:
Bar.objects.filter(foo.is_active())
I am getting error such as
SyntaxError at /
('non-keyword arg after keyword arg'
How can I achieve this?
You cannot query against model methods or properties. Either use the criteria within it in the query, or filter in Python using a list comprehension or genex.
You could also use a custom manager. Then you could run something like this:
Bar.objects.foo_active()
And all you have to do is:
class BarManager(models.Manager):
def foo_active(self):
# use your method to filter results
return you_custom_queryset
Check out the docs.
I had similar problem: I am using class-based view object_list and I had to filter by model's method. (storing the information in database wasn't an option because the property was based on time and I would have to create a cronjob and/or... no way)
My answer is ineffective and I don't know how it's gonna scale on larger data; but, it works:
q = Model.objects.filter(...)...
# here is the trick
q_ids = [o.id for o in q if o.method()]
q = q.filter(id__in=q_ids)
You can't filter on methods, however if the is_active method on Foo checks an attribute on Foo, you can use the double-underscore syntax like Bar.objects.filter(foo__is_active_attribute=True)