Django-tables2: how to order_by accessor - django

Say we have a column like:
num_member = tables.Column(accessor = 'members.count', verbose_name = 'number of members' )
When I tried to sort this in the template, it raises:
Field Error: Cannot resolve keyword u'count' into field
I read the document and it says we can use order_by by passing in some sort of accessor, but how exactly do we do this please?

For function like Model's property method, you can access it directly using accessor. For example:
Class MyModel(models.Model):
data= models.CharField(max_length=255)
#property
def print_function(self):
return 'hello world'
#Table class
class MyTable(tables.Table):
data= tables.Column(accessor='print_function')
class Meta:
model = MyModel
fields = ('data')
Using the above method, you can show different kinds of data in table using accessor:
Class SomeModel(models.Model):
some_data= models.CharField(max_length=255)
data= models.ManyToManyField(MyModel)
#property
def count_function(self):
some_data= self.data.objects.count() #returns count of the objects
return some_data
#Table class
class SomeTable(tables.Table):
data= tables.Column(accessor='count_function')
class Meta:
model = SomeModel
fields = ('data')
And accessor can be used for directly accessing related foreignkey model's field value like:
Class SomeModel(models.Model):
somedata= models.ForeignKey(MyModel)
#Table class
class MyTable(tables.Table):
data= tables.Column(accessor='somedata.data')
class Meta:
model = SomeModel
fields = ('data')
EDIT
Lets give an example of how order_by can be used:
#Class Based view, I am subclassing ListView and SingleTableView here for example
class MyView(ListView, SingleTableView):
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
context['table'].order_by = '-last_updated' #last_updated is a datetimefield in model
return context
In the above code, what I have done is that, I am changing the order of the table data in context which will later be rendered in template.

Fairly old question, however, I have been confronted with the same problem today: I couldn't order my tables if the accessor was a property (or 0-argument-method) and not a model field.
After not finding anything in the docs and inspecting the source code, it turned out that tables2 will pass the ordering to the database if its data is a QuerySet, but otherwise it will do a Python list sort with an appropriate key:
# django_tables2/tables.py -> class TableData
def order_by(self, aliases):
# ...
if hasattr(self, "queryset"):
translate = lambda accessor: accessor.replace(Accessor.SEPARATOR, QUERYSET_ACCESSOR_SEPARATOR)
if accessors:
self.queryset = self.queryset.order_by(*(translate(a) for a in accessors))
else:
self.list.sort(key=OrderByTuple(accessors).key)
I assume that this can not be trivially solved by using a try-except instead of the if-else because an exception would only be raised once the queryset is evaluated which only happens later.
Solution: whenever your sort-parameter is not a model field, turn the QuerySet into a list before handing it to the table. For many cases in django, this will be as simple as overriding get_queryset:
def get_queryset(self):
qs = super(ViewName, self).get_queryset()
return list(qs)
This should work best if your accessor is a cached_property on the model of your table, e.g.:
from django.utils.functional import cached_property
#cached_property
def member_count(self):
# do the heavy stuff here in the model
return whatever
Then, in the table:
num_member = tables.Column(
accessor='members_count',
verbose_name='number of members'
)

Related

Why does a queryset applied in a ModelForm not inherit a queryset from a ModelManager?

I have a custom queryset on a model manager:
class TenantManager(models.Manager):
def get_queryset(self):
return super().get_queryset().filter(myfield=myvalue)
class TenantModel(TenantModelMixin, models.Model):
objects = TenantManager()
class Meta:
abstract = True
I use the abstract TenantModel as a mixin with another model to apply the TenantManager. E.g.
class MyModel(TenantModel):
This works as expected, applying the TenantManager filter every time MyModel.objects.all() is called when inside a view.
However, when I create a ModelForm with the model, the filter is not applied and all results (without the filter are returned. For example:
class AddPersonForm(forms.ModelForm):
class Meta:
model = MyModel
fields = ('person', )
Why is this and how to I ensure the ModelManager is applied to the queryset in ModelForm?
Edit
#Willem suggests the reason is forms use ._base_manager and not .objects (although I can not find this in the Django source code), however the docs say not to filter this kind of manager, so how does one filter form queries?
Don’t filter away any results in this type of manager subclass
This
manager is used to access objects that are related to from some other
model. In those situations, Django has to be able to see all the
objects for the model it is fetching, so that anything which is
referred to can be retrieved.
If you override the get_queryset() method and filter out any rows,
Django will return incorrect results. Don’t do that. A manager that
filters results in get_queryset() is not appropriate for use as a base
manager.
You can do it in two ways:
First: When creating the form instance, add the queryset for the desired field.
person_form = AddPersonForm()
person_form.fields["myfield"].queryset = TenantModel.objects.filter(myfield="myvalue")
Second: Override the field's queryset in the AddPersonForm itself.
class AddPersonForm(forms.ModelForm):
class Meta:
model = MyModel
fields = ('person', )
def __init__(self, *args, **kwargs):
super(AddPersonForm, self).__init__(*args, **kwargs)
self.fields['myfield'].queryset = TenantModel.objects.filter(myfield="myvalue")
I'm not sure why your code doesn't properly works. Probably you haven't reload django app. You could load queryset in __init__ of your form class
class AddPersonForm(forms.ModelForm):
person = forms.ModelMultipleChoiceField(queryset=None)
class Meta:
model = MyOtherModel
fields = ('person', )
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.fields['person'].queryset = MyModel.objects.all()

Django-filter: filtering by model property

I read on several places that it is not possible to filter Django querysets using properties because Django ORM would have no idea how to convert those into SQL.
However, once the data are fetched and loaded into memory, it shall be possible to filter them in Python using those properties.
And my question: is there any library that allows the querysets to be filtered by properties in memory? And if not, how exactly must the querysets be tampered with so that this becomes possible? And how to include django-filter into this?
Have you got difficult property or not?
If not you can rewrite it to queryset like this:
from django.db import models
class UserQueryset(models.Manager):
def get_queryset(self):
return super().get_queryset().annotate(
has_profile=models.Exists(Profile.objects.filter(user_id=models.OuterRef('id')))
)
class User(models.Model):
objects = UserQueryset
class Profile(models.Model):
user = models.OneToOneField(User, related_name='profile')
# When you want to filter by has profile just use it like has field has profile
user_with_profiles = User.objects.filter(has_profile=True)
Maybe it is not what you want, but it can help you in some cases
django-filter wants and assumes that you are using querysets. Once you take a queryset and change it into a list, then whatever is downstream needs to be able to handle just a list or just iterate through the list, which is no longer a queryset.
If you have a django_filters.FilterSet like:
class FooFilterset(django_filters.FilterSet):
bar = django_filters.Filter('updated', lookup_expr='exact')
my_property_filter = MyPropertyFilter('property')
class Meta:
model = Foo
fields = ('bar', 'my_property_filter')
then you can write MyPropertyFilter like:
class MyPropertyFilter(django_filters.Filter):
def filter(self, qs, value):
return [row for row in qs if row.baz == value]
At this point, anything downstream of MyProperteyFilter will have a list.
Note: I believe the order of fields should have your custom filter, MyPropertyFilter last, because then it will always be processed after the normal queryset filters.
So, you have just broken the "queryset" API, for certain values of broken. At this point, you'll have to work through the errors of whatever is downstream. If whatever is after the FilterSet requires a .count member, you can change MyPropertyFilter like:
class MyPropertyFilter(django_filters.Filter):
def filter(self, qs, value):
result = [row for row in qs if row.baz == value]
result.count = len(result)
return result
You're in uncharted territory, and you'll have to hack your way through.
Anyways, I've done this before and it isn't horrible. Just take the errors as they come.
Since filtering by non-field attributes such as property inevitably converts the QuerySet to list (or similar), I like to postpone it and do the filtering on object_list in get_context_data method. To keep the filtering logic inside the filterset class, I use a simple trick. I've defined a decorator
def attr_filter(func):
def wrapper(self, queryset, name, value, force=False, *args, **kwargs):
if force:
return func(self, queryset, name, value, *args, **kwargs)
else:
return queryset
return wrapper
which is used on django-filter non-field filtering methods. Thanks to this decorator, the filtering basically does nothing (or skips) the non-field filtering methods (because of force=False default value).
Next, I defined a Mixin to be used in the view class.
class FilterByAttrsMixin:
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
filtered_list = self.filter_qs_by_attributes(self.object_list, self.filterset)
context.update({
'object_list': filtered_list,
})
return context
def filter_qs_by_attributes(self, queryset, filterset_instance):
if hasattr(filterset_instance.form, 'cleaned_data'):
for field_name in filter_instance.filters:
method_name = f'attr_filter_{field_name}'
if hasattr(filterset_instance, method_name):
value = filterset_instance.form.cleaned_data[field_name]
if value:
queryset = getattr(filterset_instance, filter_method_name)(queryset, field_name, value, force=True)
return queryset
It basically just returns to your filterset and runs all methods called attr_filter_<field_name>, this time with force=True.
In summary, you need to:
Inherit the FilterByAttrsMixin in your view class
call your filtering method attr_filter_<field_name>
use attr_filter decorator on the filtering method
Simple example (given that I have model called MyModel with property called is_static that I want to filter by:
model:
class MyModel(models.Model):
...
#property
def is_static(self):
...
view:
class MyFilterView(FilterByAttrsMixin, django_filters.views.FilterView):
...
filterset_class = MyFiltersetClass
...
filter:
class MyFiltersetClass(django_filters.FilterSet):
is_static = django_filters.BooleanFilter(
method='attr_filter_is_static',
)
class Meta:
model = MyModel
fields = [...]
#attr_filter
def attr_filter_is_static(self, queryset, name, value):
return [instance for instance in queryset if instance.is_static]
Take a look at django-property-filter package. This is an extension to django-filter and provides functionality to filter querysets by class properties.
Short example from the documentation:
from django_property_filter import PropertyNumberFilter, PropertyFilterSet
class BookFilterSet(PropertyFilterSet):
prop_number = PropertyNumberFilter(field_name='discounted_price', lookup_expr='gte')
class Meta:
model = NumberClass
fields = ['prop_number']

Custom columns in django_tables2

I've had a search around for this but haven't had much luck so looking for a bit of help. I'm trying to add some extra columns to a table defined by a model, using function definitions in the model. Here's what my code looks like now:
# models.py
class MyModel(models.Model):
my_field = models.TextField()
def my_function(self):
# Return some calculated value based on the entry
return my_value
# tables.py
class MyTable(tables.Table):
my_extra_column = tables.Column(....)
class Meta:
model = MyModel
# views.py
table = MyTable(MyModel.objects.all())
RequestConfig(request).configure(table)
return render(request, ....)
My question is can I access my_function in the entries passed to MyTable so I can show the result of my_function in the custom my_extra_column column? I assume I need to be using accessors, but I can't see how I can access the queryset data using this. Thanks!
I figured it out in the end, it was actually not too hard after all :)
So using my example above, in order to add a custom column using a function in the associated model you just use accessors ...
# models.py
class MyModel(models.Model):
my_field = models.TextField()
my_field_2 = models.IntegerField()
def my_function(self):
# Return some calculated value based on the entry
return my_value
# tables.py
class MyTable(tables.Table):
my_extra_column = tables.Column(accessor='my_function',
verbose_name='My calculated value')
class Meta:
fields = ['my_field', 'my_field_2', 'my_extra_column']
model = MyModel
The trouble comes if and when you want to be able to sort this data, because the function won't translate into any valid field in MyModel. So you could either disable sorting on this column using ordering=False or specify a set using order_by=('field', 'field2')

Django Rest Framework - Incorrect source object on nested serialization when using proxy model

I have a question concerning using proxy models with the Django Rest Framework and nested serialization.
My proxy models are as follows:
class MyField(Field):
class Meta:
proxy = True
def field_type_name(self):
# logic that computes the field type name here
return "the result"
class MyForm(Form):
class Meta:
proxy = True
The Field model is defined in another app that I've included in my project. I wanted to add my own method to it without modifying the model so I made a proxy.
These are the serializers for the proxy models:
class MyFieldSerializer(serializers.HyperlinkedModelSerializer):
field_type = serializers.ChoiceField(source='field_type_name',
choices=form_fields.NAMES)
class Meta:
model = MyField
fields = ('url', 'field_type',)
class MyFormSerializer(serializers.HyperlinkedModelSerializer):
fields = MyFieldSerializer(many=True)
class Meta:
model = MyForm
fields = ('url', 'fields')
And the viewsets:
class MyFieldViewSet(viewsets.ModelViewSet):
queryset = MyField.objects.all()
serializer_class = MyFieldSerializer
class MyFormViewSet(viewsets.ModelViewSet):
queryset = MyForm.objects.all()
serializer_class = MyFormSerializer
urls.py:
router.register(r'fields', views.MyFieldViewSet)
router.register(r'forms', views.MyFormViewSet)
If I go to /fields/ it works fine. The method I added in the proxy model is executed correctly.
[
{
"url": "http://127.0.0.1:8000/fields/1/",
"field_type": "the result",
},
{ ...
But if I go to /forms/ I get the following error:
AttributeError at /forms/
'Field' object has no attribute 'field_type_name'
/Users/..../lib/python2.7/site-packages/rest_framework/fields.py in get_component
"""
Given an object, and an attribute name,
return that attribute on the object.
"""
if isinstance(obj, dict):
val = obj.get(attr_name)
else:
**val = getattr(obj, attr_name)**
if is_simple_callable(val):
return val()
return val
▼ Local vars
Variable Value
attr_name u'field_type_name'
obj <Field: Cools2>
As you can see the obj is Field instead of MyField which is why it's not able to call field_type_name. This only happens on the nested serialization. If anyone has a suggestion on how I can best fix this I'd greatly appreciate it.
EDIT:
Based on Kevin's response I'm editing the proxy models to try to fix this.
Here are the base models for reference:
class Form(AbstractForm):
pass
class Field(AbstractField):
form = models.ForeignKey("Form", related_name="fields")
Here is my attempt to fix the problem (using examples from Django proxy model and ForeignKey):
class MyField(Field):
class Meta:
proxy = True
def field_type_name(self):
# logic that computes the field type name here
return "the result"
# this works
#property
def form(self):
return MyForm.objects.get(id=self.form_id)
class MyForm(Form):
class Meta:
proxy = True
# this does not work
#property
def fields(self):
qs = super(MyForm, self).fields
qs.model = MyField
return qs
Now I can get MyForm from MyField but not MyField from MyForm (the reverse):
>>> MyField.objects.get(pk=1).form
<MyForm: Cool Form>
>>> MyForm.objects.get(pk=1).fields.all()
[]
I
This is because your model Form (or MyForm) isn't configured to return MyField objects when you access the field attribute on the form. It's not configured to substitute your proxied-version.
Try it yourself, open ./manage.py shell and try to read the fields related manager, it will return a collection of Field objects.
>>> form = MyForm.objects.all()[0].fields.all()
(Btw, I have to guess on the actual model structure since the original Field and Form models weren't included in your example).
If it's a read-only field, you could use serializers.SerializerMethodField to add a method to the serializer (your field_type_name(). If you want to be able to edit it, you're better off writing your own field sub-class that handles the conversion.

Django REST Framework: adding additional field to ModelSerializer

I want to serialize a model, but want to include an additional field that requires doing some database lookups on the model instance to be serialized:
class FooSerializer(serializers.ModelSerializer):
my_field = ... # result of some database queries on the input Foo object
class Meta:
model = Foo
fields = ('id', 'name', 'myfield')
What is the right way to do this? I see that you can pass in extra "context" to the serializer, is the right answer to pass in the additional field in a context dictionary?
With that approach, the logic of getting the field I need would not be self-contained with the serializer definition, which is ideal since every serialized instance will need my_field. Elsewhere in the DRF serializers documentation it says "extra fields can correspond to any property or callable on the model". Are "extra fields" what I'm talking about?
Should I define a function in Foo's model definition that returns my_field value, and in the serializer I hook up my_field to that callable? What does that look like?
Happy to clarify the question if necessary.
I think SerializerMethodField is what you're looking for:
class FooSerializer(serializers.ModelSerializer):
my_field = serializers.SerializerMethodField('is_named_bar')
def is_named_bar(self, foo):
return foo.name == "bar"
class Meta:
model = Foo
fields = ('id', 'name', 'my_field')
http://www.django-rest-framework.org/api-guide/fields/#serializermethodfield
You can change your model method to property and use it in serializer with this approach.
class Foo(models.Model):
. . .
#property
def my_field(self):
return stuff
. . .
class FooSerializer(ModelSerializer):
my_field = serializers.ReadOnlyField(source='my_field')
class Meta:
model = Foo
fields = ('my_field',)
Edit: With recent versions of rest framework (I tried 3.3.3), you don't need to change to property. Model method will just work fine.
With the last version of Django Rest Framework, you need to create a method in your model with the name of the field you want to add. No need for #property and source='field' raise an error.
class Foo(models.Model):
. . .
def foo(self):
return 'stuff'
. . .
class FooSerializer(ModelSerializer):
foo = serializers.ReadOnlyField()
class Meta:
model = Foo
fields = ('foo',)
if you want read and write on your extra field, you can use a new custom serializer, that extends serializers.Serializer, and use it like this
class ExtraFieldSerializer(serializers.Serializer):
def to_representation(self, instance):
# this would have the same as body as in a SerializerMethodField
return 'my logic here'
def to_internal_value(self, data):
# This must return a dictionary that will be used to
# update the caller's validation data, i.e. if the result
# produced should just be set back into the field that this
# serializer is set to, return the following:
return {
self.field_name: 'Any python object made with data: %s' % data
}
class MyModelSerializer(serializers.ModelSerializer):
my_extra_field = ExtraFieldSerializer(source='*')
class Meta:
model = MyModel
fields = ['id', 'my_extra_field']
i use this in related nested fields with some custom logic
My response to a similar question (here) might be useful.
If you have a Model Method defined in the following way:
class MyModel(models.Model):
...
def model_method(self):
return "some_calculated_result"
You can add the result of calling said method to your serializer like so:
class MyModelSerializer(serializers.ModelSerializer):
model_method_field = serializers.CharField(source='model_method')
p.s. Since the custom field isn't really a field in your model, you'll usually want to make it read-only, like so:
class Meta:
model = MyModel
read_only_fields = (
'model_method_field',
)
If you want to add field dynamically for each object u can use to_represention.
class FooSerializer(serializers.ModelSerializer):
class Meta:
model = Foo
fields = ('id', 'name',)
def to_representation(self, instance):
representation = super().to_representation(instance)
if instance.name!='': #condition
representation['email']=instance.name+"#xyz.com"#adding key and value
representation['currency']=instance.task.profile.currency #adding key and value some other relation field
return representation
return representation
In this way you can add key and value for each obj dynamically
hope u like it
This worked for me.
If we want to just add an additional field in ModelSerializer, we can
do it like below, and also the field can be assigned some val after
some calculations of lookup. Or in some cases, if we want to send the
parameters in API response.
In model.py
class Foo(models.Model):
"""Model Foo"""
name = models.CharField(max_length=30, help_text="Customer Name")
In serializer.py
class FooSerializer(serializers.ModelSerializer):
retrieved_time = serializers.SerializerMethodField()
#classmethod
def get_retrieved_time(self, object):
"""getter method to add field retrieved_time"""
return None
class Meta:
model = Foo
fields = ('id', 'name', 'retrieved_time ')
Hope this could help someone.
class Demo(models.Model):
...
#property
def property_name(self):
...
If you want to use the same property name:
class DemoSerializer(serializers.ModelSerializer):
property_name = serializers.ReadOnlyField()
class Meta:
model = Product
fields = '__all__' # or you can choose your own fields
If you want to use different property name, just change this:
new_property_name = serializers.ReadOnlyField(source='property_name')
As Chemical Programer said in this comment, in latest DRF you can just do it like this:
class FooSerializer(serializers.ModelSerializer):
extra_field = serializers.SerializerMethodField()
def get_extra_field(self, foo_instance):
return foo_instance.a + foo_instance.b
class Meta:
model = Foo
fields = ('extra_field', ...)
DRF docs source
Even though, this is not what author has wanted, it still can be considered useful for people here:
If you are using .save() ModelSerializer's method, you can pass **kwargs into it. By this, you can save multiple dynamic values.
i.e. .save(**{'foo':'bar', 'lorem':'ipsum'})
Add the following in serializer class:
def to_representation(self, instance):
representation = super().to_representation(instance)
representation['package_id'] = "custom value"
return representation