wagtail search_fields on snippet with foreign key - django

I have a snippet which is a proxy of one of my standard django models.
search_fields works fine when filtering on standard fields, the problem is I can't seem to get foreign keys to work.
This page has an example on the bottom that shows how to create searchable snippets:
https://docs.wagtail.org/en/stable/topics/snippets.html
The main model has a field called "day" which is a foreign key to a Day-table. A day has a calendar_year, which I would like to be able to filter on while searching in the wagtail snippets area. in the def str method I'm able to display the name in the list, the search is the problem here.
Suggestions?
#register_snippet
class EventSnippet(index.Indexed, Event):
# We make a proxy model just to be able to add to this file or potentially if we want custom methods on it.
panels = [
FieldPanel('name'),
]
search_fields = [
index.SearchField('day__calendar_year', partial_match=True), # This prompts an error
index.SearchField('name', partial_match=True),
]
class Meta:
proxy = True
def __str__(self):
return f"{self.name} {self.day.calendar_year}"
When running python manage.py update_index i get the following warning:
EventSnippet.search_fields contains non-existent field 'day__calendar_year

You can't use complex lookups with double-underscores inside SearchField - search queries work by populating a central table (the search index) in advance with the data you're going to be searching on, which means you can't do arbitrary lookups and transformations on it like you would with a standard database query.
However, you can use any method or attribute in SearchField - not just database fields - so you could add a method that returns the year, and use that:
#register_snippet
class EventSnippet(index.Indexed, Event):
# ...
def get_year(self):
return self.day.calendar_year
search_fields = [
index.SearchField('get_year', partial_match=True),
index.SearchField('name', partial_match=True),
]

Related

Wagtail - selected_related on fieldpanels?

I've registred a snippet which has a foreign key to a somewhat huge table in wagtail 3. When attempting to click on an entry to see the detail view which displays the fieldpanels, I have about 5-10 seconds of loading which tells me massive amounts of queries are being made. Is there any way to use select_related() on a fieldpanel field?
In this particular case I'd like to use select_related on "day"
#register_snippet
class EventSnippet(index.Indexed, Event):
panels = [
FieldPanel('name'),
FieldPanel('day'),
]
search_fields = [
index.SearchField('name', partial_match=True),
index.RelatedFields('day', [index.SearchField('calendar_year')]),
]
class Meta:
proxy = True

How to ignore special characters from the search field in Django

The model is something like
class Product(BaseModel):
name = models.CharField(db_column='name', max_length=200, blank=False, null=False, unique=True)
View is
class ProductViewSet(BaseViewSet):
queryset = Product.objects.all()
...
filterset_class = ProductFilter
The filter is
class ProductFilter(django_filters.FilterSet):
search = django_filters.CharFilter(field_name='name', lookup_expr='icontains')
class Meta:
model = Product
fields = []
Now.. if the name field has a value something like "This is a/sample" and search text is "asample". I would like to return that row.
Thanks in advance.
If the question is only for one special character ie. '/' then you can create a custom filter method with Replace like this :
class ProductFilter(django_filters.FilterSet):
def filter_without_special_chars(self, queryset, field, value):
return queryset.annotate(search_field=Replace('name', Value('/'), Value('')).filter(search_field__icontains=value)
search = django_filters.CharFilter(method='filter_without_special_chars')
class Meta:
model = Product
fields = []
You can also do this for multiple special characters BUT it won't be the optimal solution, I would suggest you user ElasticSearch (or something similar) for that.
For multiple char replacement the function would look something like this :
def filter_without_special_chars(self, queryset, field, value):
return queryset.annotate(sf1=Replace('name', Value('!'), Value('')),
sf2=Replace('sf1', Value('%'), Value(''))).filter(sf2__icontains=value)
Use PostGreSQL, which currently supports the 'unaccent' extension. This makes searching for 'año' possible when only typing 'ano'.
Best thing is, you can decide whether to use this extension for every filter by, for example using
Person.objects.filter(first_name__unaccent__icontains=search)
Switch your database to PostgreSQL and add the unaccent extension as follows:
Part of answer from #SaeX in another thread:
How can I activate the unaccent extension on an already existing model
A migration file needs to be manually made and applied.
First, create an empty migration:
./manage.py makemigrations myapp --empty
Then open the file and add UnaccentExtension to operations:
from django.contrib.postgres.operations import UnaccentExtension
class Migration(migrations.Migration):
dependencies = [
(<snip>)
]
operations = [
UnaccentExtension()
]
Now apply the migration using ./manage.py migrate.
If you'd get following error during that last step:
django.db.utils.ProgrammingError: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
... then temporarily allow superuser rights to your user by performing postgres# ALTER ROLE <user_name> SUPERUSER; and its NOSUPERUSER counterpart. pgAdminIII can do this, too.
Now enjoy the unaccent functionality using Django:
>>> Person.objects.filter(first_name__unaccent=u"Helène")
[<Person: Michels Hélène>]
Again, part of this answer belongs to #SaeX
IMPORTANT
But for me his answer still didn't work, so don't forget to
add the line django.contrib.postgresin INSTALLED_APPS (settings.py)

Not getting fields and their values which are ManyToManyField type

I have a django model named Persona:
class Persona(models.model):
name=models.CharField(max_length=100,primary_key=True)
pages_visited = models.ManyToManyField(Page)
items_searched = models.ManyToManyField(ItemsSearched)
visits = models.IntegerField(null=True,blank=True)
connect = models.CharField(max_length=True,null=True,blank=True)
image = models.ForeignKey('Image',on_delete=models.CASCADE)
def __str__(self):
return self.name
I have an object for this model:
<QuerySet [<Persona: aman>]>
Now when i am trying to get the values of all fields for this object i can see all fields and their corresponding values except fields which are ManyToManyField type.
I get the following result when i execute this : Persona.objects.filter(name='aman').values()
<QuerySet [{'visits': None, 'image_id': 3, 'name': 'aman', 'connect': 'call'}]>
I cannot see 'items_searched' and 'pages_visited' fields and their corresponding values though when i log into admin i can see them.
These are the images which show you my execution:
Command Prompt
Admin Panel
To access m2m filed in django there is .all() keyword refer this
p1 = Persona.objects.filter(name='aman')
p1_m2m = p1.pages_visited.all()
which will give the result you wanted
I believe what you're looking for is Inlines.
In your admin file, add
from .models import Page # Edit this import according to your project structure
class PagesVisitedInline(admin.TabularInline):
model = Page
class PersonaAdmin(admin.ModelAdmin):
[...] # admin code goes here
inlines = [PagesVisitedInline, ]
You could use StackedInline aswell if you prefer, read more here.

Django Rest Framework Ordering on a SerializerMethodField

I have a Forum Topic model that I want to order on a computed SerializerMethodField, such as vote_count. Here are a very simplified Model, Serializer and ViewSet to show the issue:
# models.py
class Topic(models.Model):
"""
An individual discussion post in the forum
"""
title = models.CharField(max_length=60)
def vote_count(self):
"""
count the votes for the object
"""
return TopicVote.objects.filter(topic=self).count()
# serializers.py
class TopicSerializer(serializers.ModelSerializer):
vote_count = serializers.SerializerMethodField()
def get_vote_count(self, obj):
return obj.vote_count()
class Meta:
model = Topic
# views.py
class TopicViewSet(TopicMixin, viewsets.ModelViewSet):
queryset = Topic.objects.all()
serializer_class = TopicSerializer
Here is what works:
OrderingFilter is on by default and I can successfully order /topics?ordering=title
The vote_count function works perfectly
I'm trying to order by the MethodField on the TopicSerializer, vote_count like /topics?ordering=-vote_count but it seems that is not supported. Is there any way I can order by that field?
My simplified JSON response looks like this:
{
"id": 1,
"title": "first post",
"voteCount": 1
},
{
"id": 2,
"title": "second post",
"voteCount": 8
},
{
"id": 3,
"title": "third post",
"voteCount": 4
}
I'm using Ember to consume my API and the parser is turning it to camelCase. I've tried ordering=voteCount as well, but that doesn't work (and it shouldn't)
This is not possible using the default OrderingFilter, because the ordering is implemented on the database side. This is for efficiency reasons, as manually sorting the results can be incredibly slow and means breaking from a standard QuerySet. By keeping everything as a QuerySet, you benefit from the built-in filtering provided by Django REST framework (which generally expects a QuerySet) and the built-in pagination (which can be slow without one).
Now, you have two options in these cases: figure out how to retrieve your value on the database side, or try to minimize the performance hit you are going to have to take. Since the latter option is very implementation-specific, I'm going to skip it for now.
In this case, you can use the Count function provided by Django to do the count on the database side. This is provided as part of the aggregation API and works like the SQL COUNT function. You can do the equivalent Count call by modifying your queryset on the view to be
queryset = Topic.objects.annotate(vote_count=Count('topicvote_set'))
Replacing topicvote_set with your related_name for the field (you have one set, right?). This will allow you to order the results based on the number of votes, and even do filtering (if you want to) because it is available within the query itself.
This would require making a slight change to your serializer, so it pulls from the new vote_count property available on objects.
class TopicSerializer(serializers.ModelSerializer):
vote_count = serializers.IntegerField(read_only=True)
class Meta:
model = Topic
This will override your existing vote_count method, so you may want to rename the variable used when annotating (if you can't replace the old method).
Also, you can pass a method name as the source of a Django REST framework field and it will automatically call it. So technically your current serializer could just be
class TopicSerializer(serializers.ModelSerializer):
vote_count = serializers.IntegerField(read_only=True)
class Meta:
model = Topic
And it would work exactly like it currently does. Note that read_only is required in this case because a method is not the same as a property, so the value cannot be set.
Thanks #Kevin Brown for your great explanation and answer!
In my case I needed to sort a serializerMethodField called total_donation which is the sum of donations from the UserPayments table.
UserPayments has:
User as a foreignKey
sum which is an IntegerField
related_name='payments'
I needed to get the total donations per User but only donations that have a status of 'donated', not 'pending'. Also needed to filter out the payment_type coupon, which is related through two other foreign keys.
I was dumbfounded how to join and filter those donations and then be able to sort it via ordering_fields.
Thanks to your post I figured it out!
I realized it needed to be part of the original queryset in order to sort with ordering.
All I needed to do was annotate the queryset in my view, using Sum() with filters inside like so:
class DashboardUserListView(generics.ListAPIView):
donation_filter = Q(payments__status='donated') & ~Q(payments__payment_type__payment_type='coupon')
queryset = User.objects.annotate(total_donated=Sum('payments__sum', filter=donation_filter ))
serializer_class = DashboardUserListSerializer
pagination_class = DashboardUsersPagination
filter_backends = [filters.OrderingFilter]
ordering_fields = ['created', 'last_login', 'total_donated' ]
ordering = ['-created',]
I will put it here because the described case is not the only one.
The idea is to rewrite the list method of your Viewset to order by any of your SerializerMethodField(s) also without moving your logic from the Serializer to the ModelManager (especially when you work with several complex methods and/or related models)
def list(self, request, *args, **kwargs):
response = super().list(request, args, kwargs)
ordering = request.query_params.get('ordering')
if "-" in ordering:
response.data['results'] = sorted(response.data['results'], key=lambda k: (k[ordering.replace('-','')], ), reverse=True)
else:
response.data['results'] = sorted(response.data['results'], key=lambda k: (k[ordering], ))
return response

tastypie with django-simple-history - display model history as rest API

I would like to share django model history (created by django-simple-history) using tastypie.
Problem is, how to prepare ModelResource for this purpose.
Access to model history is by model.history manager. So access to all changes of model we can gain by model.history.all()
What i would like to obtain? For example. I have django model Task and the API endpoints:
http://127.0.0.1/api/v1/task - display all tasks list
http://127.0.0.1/api/v1/task/1 - display details for choosen task
http://127.0.0.1/api/v1/task/1/history - display history of task no. 1
First two links presents default behavior of ModelResource. what i have till now?
class TaskResource(ModelResource):
class Meta:
# it displays all available history entries for all task objects
queryset = Task.history.all()
resource_name = 'task'
def prepend_urls(self):
return [
url(r"^(?P<resource_name>%s)/(?P<pk>\w[\w/-]*)/history$" % (self._meta.resource_name,),
self.wrap_view('get_history'),
name="api_history"),
]
def get_history(self, request, **kwargs):
#...
get_history should return bundle with history entries.. but how this method should look?
I guess, i need to create bundle with needed data, but don't know how exactly should i do that.
Does someeone have experience with simple-history and tastypie to present some simple example?
It seems, solution was simpler than i thought. Maybe someone use this in feature:
class TaskHistoryResource(ModelResource):
class Meta:
queryset = Task.history.all()
filtering = { 'id' = ALL }
class TaskResource(ModelResource):
history = fields.ToManyField(AssetTypeHistoryResource, 'history')
class Meta:
# it displays all available history entries for all task objects
queryset = Task.history.all()
resource_name = 'task'
def prepend_urls(self):
return [
url(r"^(?P<resource_name>%s)/(?P<pk>\w[\w/-]*)/history$" %(self._meta.resource_name,),
self.wrap_view('get_history'),
name="api_history"),
]
def get_history(self, request, **kwargs):
try:
bundle = self.build_bundle(data={'pk': kwargs['pk']}, request=request)
obj = self.cached_obj_get(bundle=bundle, **self.remove_api_resource_names(kwargs))
except ObjectDoesNotExist:
return HttpGone()
except MultipleObjectsReturned:
return HttpMultipleChoices("More than one resource is found at this URI.")
history_resource = TaskHistoryResource()
return history_resource.get_list(request, id=obj.pk)
A bit changed solution from:
http://django-tastypie.readthedocs.org/en/latest/cookbook.html#nested-resources
Basically, there was need to create additional resource with history entries. get_history method creates and returns instance of it with appropriate filter on id field (in django-simple-history id field contain id of major object. Revision primary key names history_id)
Hope, that will help someone.