I'm using django-haystack to power search for my application and I have boolean fields i would like to filter my models by. However, when I try doing this, my search query fails.
The search backend i'm using is elasticsearch
Posting some of your code here would be helpful in figuring out what is going wrong for you, as it's hard to troubleshoot with nothing.
How I implemented this (using whoosh rather than elasticsearch, but the django code should be the same/similar at any rate) was like so:
created my own searchform class (if you haven't already done this, look at Haystack Docs - Creating your own form
from django import forms
from haystack.forms import SearchForm
class PaidPropertySearchForm(SearchForm):
q = forms.CharField(required=False, label=('Town Area'))
#othersearchtermshere
furnished = forms.BooleanField(required=False)
def search(self):
sqs = super(PaidPropertySearchForm, self).search()
#other search filtering here
if self.is_valid() and self.cleaned_data['furnished']:
sqs = sqs.filter(furnished__contains=self.cleaned_data['furnished'])
return sqs
relevant model field is simply:
class Properties (models.Model):.
furnished = models.BooleanField()
and my searchindexes.py:
class PropertyIndex(indexes.SearchIndex, indexes.Indexable):
#other indexed terms
furnished = indexes.BooleanField(model_attr='furnished', default='false')
Here it's worth noting that there are some issues between haystack and the whoosh backend with boolean values, that required me to specify the defult index value as a string 'false' rather than the boolean value False. If i didn't do this or didn't set a default, for some reason ALL boolean values were indexed as True, regardless of what the model had stored. I don't know if that issue would affect elasticsearch, so you may not need
default='false'
in your search index
Hopefully this helps get you on the right track if you haven't already figured it out.
sqs = sqs.filter(boolean_field=True)
did not work for me either (using Haystack 2.1.0 and Elasticsearch 1.2.1) but
sqs = sqs.filter(boolean_field=1)
did.
Related
with the new django-filter v2 coming out. I was wondering how i could replicate the below functionality with it.
Before my filterset was as so
class PostFilterSet(filters.FilterSet):
date = filters.DateFilter(
name='date',
lookup_expr=DATETIME_LOOKUPS,
method='filter_date')
class Meta:
# Meta Options here
def filter_date(self, queryset, name, value):
# filter logic
What this would essentially let me do on the api is make filter request like this 'https://example.com/api/post/?date__gt=exampledate' but with the latest update i can no longer keep the list of DATETIME_LOOKUPS and have to use the Lookup choice filter which changed the request to 'https://example.com/api/post/?date=exampledate&date_lookup=exact' is there a way to make it work like it did before in a painless way ?
I have the following model:
class Credit(models.Model):
user = models.ForeignKey(User)
...
and a ListView
class CreditListView(ListView):
paginate_by = 10
model = Credit
...
if I want to filter the credits by users in side CreditListView:
def get_queryset(self):
users = User.objects.filter(...)[:10]
credits = Credits.objects.filter(user__in=users)
return credits
I will get a NotSupportedError exception:
(1235, "This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery'")
Try this :
users = User.objects.filter(...)[:10]
users = list(users)
credits = Credits.objects.filter(user__in=users)
Django model returns a list which exactly isn't a Python list but an extended form of that. But that has some limitations. Hence, we need to convert that into a Python understandable list.
The problem was this line:
users = User.objects.filter(...)[:10]
It doesn't like the limit within the subquery, I thought I have tried removing it, could be django server didn't restart properly.
(1235, "This version of MySQL doesn't yet support 'LIMIT &
IN/ALL/ANY/SOME subquery'")
Pagination is responsible for the LIMIT and your queryset for the IN. You might be able to get away by rewriting the query with include/exclude or by using index ranges instead of pagination. If not, use raw SQL to get your queryset.
In general using MySQL (with MyISAM in particular) is a very painful idea, because apart from its problems as a database, it does not support 20-25% of Django's ORM. If you can not switch to PostgeSQL, try using InnoDB.
I am writing a simple library app (kind of just a book database at the moment) and I have a Haystack search page as the home page. I am trying to add multiple search fields to the page, so you can search specifically in the title, author etc. instead of just one generic "keyword" search. I have been trying to make this work with Haystack's "faceting" feature, but it seems to be more oriented around refining a search based on given, strict categories.
Can you use Haystack facets to provide multiple search fields? Or is Haystack just not cut out for this kind of job? If so, what should I be using instead?
If you need more context, the current project is available on GitHub.
Try ModelSearchForm. This form adds new fields to form. It iterates through all registered models for the current SearchSite and provides a checkbox for each one. If no models are selected, all types will show up in the results.
custom form example from documentation and can be converted to ModelSearchForm simply inherting from ModelSearcForm
from django import forms
from haystack.forms import SearchForm
class DateRangeSearchForm(SearchForm):
start_date = forms.DateField(required=False)
end_date = forms.DateField(required=False)
def search(self):
# First, store the SearchQuerySet received from other processing.
sqs = super(DateRangeSearchForm, self).search()
if not self.is_valid():
return self.no_query_found()
# Check to see if a start_date was chosen.
if self.cleaned_data['start_date']:
sqs = sqs.filter(pub_date__gte=self.cleaned_data['start_date'])
# Check to see if an end_date was chosen.
if self.cleaned_data['end_date']:
sqs = sqs.filter(pub_date__lte=self.cleaned_data['end_date'])
return sqs
DateRange SearchForm is a custom form and has more flexibility as it has more control on developer side. The simplest way to go about creating your own form is to inherit from SearchForm (or the desired parent) and extend the search method. By doing this, you save yourself most of the work of handling data correctly and stay API compatible with the SearchView.
More help can be from this question django haystack custom form
I'm building a basic time logging app right now and I have a todo model that uses django-taggit. My Todo model looks like this:
class Todo(models.Model):
project = models.ForeignKey(Project)
description = models.CharField(max_length=300)
is_done = models.BooleanField(default=False)
billable = models.BooleanField(default=True)
date_completed = models.DateTimeField(blank=True, null=True)
completed_by = models.ForeignKey(User, blank=True, null=True)
tags = TaggableManager()
def __unicode__(self):
return self.description
I'm trying to get a list of unique tags for all the Todos in a project and I have managed to get this to work using a set comprehension, however for every Todo in the project I have to query the database to get the tags. My set comprehension is:
unique_tags = { tag.name.lower() for todo in project.todo_set.all() for tag in todo.tags.all() }
This works just fine, however for every todo in the project it runs a separate query to grab all the tags. I was wondering if there is any way I can do something similar to prefetch_related in order to avoid these duplicate queries:
unique_tags = { tag.name.lower() for todo in project.todo_set.all().prefetch_related('tags') for tag in todo.tags.all() }
Running the previous code gives me the error:
'tags' does not resolve to a item that supports prefetching - this is an invalid parameter to prefetch_related().
I did see that someone asked a very similar question here: Optimize django query to pull foreign key and django-taggit relationship however it doesn't look like it ever got a definite answer. I was hoping someone could help me out. Thanks!
Taggit now supports prefetch_related directly on tag fields (in version 0.11.0 and later, released 2013-11-25).
This feature was introduced in this pull request. In the test case for it, notice that after prefetching tags using .prefetch_related('tags'), there are 0 additional queries for listing the tags.
Slightly hackish soution:
ct = ContentType.objects.get_for_model(Todo)
todo_pks = [each.pk for each in project.todo_set.all()]
tagged_items = TaggedItem.objects.filter(content_type=ct, object_id__in=todo_pks) #only one db query
unique_tags = set([each.tag for each in tagged_items])
Explanation
I say it is hackish because we had to use TaggedItem and ContentType which taggit uses internally.
Taggit doesn't provide any method for your particular use case. The reason is because it is generic. The intention for taggit is that any instance of any model can be tagged. So, it makes use of ContentType and GenericForeignKey for that.
The models used internally in taggit are Tag and TaggedItem. Model Tag only contains the string representation of the tag. TaggedItem is the model which is used to associate these tags with any object. Since the tags should be associatable with any object, TaggedItem uses model ContentType.
The apis provided by taggit like tags.all(), tags.add() etc internally make use of TaggedItem and filters on this model to give you the tags for a particular instance.
Since, your requirement is to get all the tags for a particular list of objects we had to make use of the internal classes used by taggit.
Use django-tagging and method usage_for_model
def usage_for_model(self, model, counts=False, min_count=None, filters=None):
"""
Obtain a list of tags associated with instances of the given
Model class.
If ``counts`` is True, a ``count`` attribute will be added to
each tag, indicating how many times it has been used against
the Model class in question.
If ``min_count`` is given, only tags which have a ``count``
greater than or equal to ``min_count`` will be returned.
Passing a value for ``min_count`` implies ``counts=True``.
To limit the tags (and counts, if specified) returned to those
used by a subset of the Model's instances, pass a dictionary
of field lookups to be applied to the given Model as the
``filters`` argument.
"""
A slightly less hackish answer than akshar's, but only slightly...
You can use prefetch_related as long as you traverse the tagged_item relations yourself, using the clause prefetch_related('tagged_items__tag'). Unfortunately, todo.tags.all() won't take advantage of that prefetch - the 'tags' manager will still end up doing its own query - so you have to step over the tagged_items relation there too. This should do the job:
unique_tags = { tagged_item.tag.name.lower()
for todo in project.todo_set.all().prefetch_related('tagged_items__tag')
for tagged_item in todo.tagged_items.all() }
I have a single model in my django app that I want to create a search form for. Is there a way to search all the fields in the model at once with the same search string? I've looked into xapian and solr but they seem like a lot of overhead for searching over 1 model. I want to be able to say something like:
results = Assignment.objects.filter(any_column = search_string)
I realize there might not be something that concise but right now the only option I can come up with other than using a search app is to check each field separately and concatenate the results together.
Once you have all the field names you can create Q objects using kwarg expansion and use reduce() along with operator.or_ to turn them into a single query.
qgroup = reduce(operator.or_, (Q(**{fieldname: value}) for fieldname in fieldnames))
asgns = Assignment.objects.filter(qgroup)
Old question, but for further reference I am adding this:
In django 1.10 SearchVector class was added.
Usage from the docs:
Searching against a single field is great but rather limiting. The Entry instances we’re searching belong to a Blog, which has a tagline field. To query against both fields, use a SearchVector:
>>> from django.contrib.postgres.search import SearchVector
>>> Entry.objects.annotate(
... search=SearchVector('body_text', 'blog__tagline'),
... ).filter(search='Cheese')
[<Entry: Cheese on Toast recipes>, <Entry: Pizza Recipes>]