Accent insensitive search django sqlite - django

I am using sqlite and django. I am trying to search for accented strings stored in the sqlite database using non-accented search query.
For example: search for "Rio Grande" when the database contains "Río Grande".
I found this SO post SQLite accent-insensitive search that mentions to first create a collation and then call the search query using COLLATE NOACCENTS
This is exactly what I need. However I am not sure how to do this using django models. Any suggestions?
As suggested in this Postgres specific post on django documentation: PostgreSQL specific lookups I tried the following:
myObject.objects.filter(locationTuple__unaccent__contains='Rio Grande')
but got the following error:
FieldError: Unsupported lookup 'unaccent' for CharField or join on the field not permitted. which is understandable because this lookup might not work for sqlite.


I think this is the closest thing you're going to find, however, it needs some development work:
https://github.com/djcoin/django-unaccent
I'm jealous as well, as my workplace isn't on PostgreSQL either (yet).

Related

Django upgrade filter/prefetch_related behaviour change?

I'm in the process of upgrading from Django 1.8.19 to 1.11.15 and I've found a piece of code which is breaking.
In particular this query is doing something different to what it did before.
project_groups = brand.project_groups.prefetch_related(
'project', 'project__score', 'project__themes').filter(
project=projects
).distinct()
Previously (in Django 1.8), according to the output of "project_groups.query" it produced SQL including:
... projectgroup.project_id IN [projects query]
Now it produces SQL reading:
... projectgroup.project_id = [projects query]
This breaks as the [projects query] returns more than one row. So I get a:
ProgrammingError: more than one row returned by a subquery used as an expression
The only changes I've made to the code for this upgrade are to models and migrations to use ArrayField and HStoreField from django.contrib.postgres.fields instead of the django_hstore equivalents.
My guess is that the original code was wrong, but worked due to a bug in Django (filter/prefetch_related) which has now been fixed. Is that likely to be correct? If this is in fact a new bug in Django I don't want to write code which relies on it!

There was a change to field lookups behaviour of Django between 1.8 and 1.9 that explains this - you can see the details at Django ticket #25284.
In Django 1.8 a query such as Model.objects.filter(related_id = RelatedModel.objects.all()) used to result in an implicit __in lookup so the SQL query contained related_id IN (SELECT id FROM ...). But in Django 1.9 the "IN" is changed to an "=", which causes the query to break in MySql and Postgres. The change was classed a bug fix as the implicit "IN" behaviour was undocumented and probably accidental.
You should be able to fix the query fairly easily by adding an explicit lookup type to the field lookup such as .filter(project__in=projects) - see Django Documentation: Field Lookups

django json field: which one?

I am looking for a JSON field for Django.
I have found mainly 2 jsonfield app and I am not sure which one I should use.
The main difference I see is that the first one does not have the native JSON datatype support for PostgreSQL anymore.
It has been removed recently (https://github.com/bradjasper/django-jsonfield/commit/15957c9dab18c546ae5c119f8a6057e5db6b2135). It was related to this issue https://github.com/bradjasper/django-jsonfield/issues/57
but I am not sure if it's the right approach since JSONB is also coming soon with PostgreSQL 9.4. I think it's better to use the native datatype when using PostgreSQL. What do you think?
1) https://github.com/bradjasper/django-jsonfield
2) https://bitbucket.org/schinckel/django-jsonfield/

Since Django 1.9 JSON support is back again with JSONField:
https://docs.djangoproject.com/en/1.9/ref/contrib/postgres/fields/

django haystack 2.0 contains query

Even though doc says
The contains filter became the new default filter as of Haystack v2.X (the default in Haystack v1.X was exact). This changed because exact caused problems and was unintuitive for new people trying to use Haystack. contains is a much more natural usage. http://django-haystack.readthedocs.org/en/latest/searchqueryset_api.html
I find my Solr returns exact match.
I tried subclassing SearchForm and overriding search function to use
filter(content__contains=...) as suggested by django haystack how do I find substrings in words?
But it doesn't work.
How do I do the contains search?
Using Ngram suggested in Django-Haystack with Solr contains search is the only way to do it?
I'm using solr 3.6 if that matters.

Django Haystack similarity search

I'm a Django newbie doing a primitive website. I installed haystack and Whoosh as its search engine cause it was the simplest thing to do. It works fine, but there is a problem and I don't know how to Google it. I have some categories on my site and I have indexed their names to search. So, when a user enters "Computing" it finds the computing category and links to it. But there is a problem. If a user enters "Comp" into search field, it doesn't find "Computing" at all. Is this something that can be configured and how?
EDIT:
What else have I tried? Installing haystack 2.0, following this tutorial, installing solr instead of whoosh, trying Ngram fields, rebuilding indexes 10 times, rewriting search_indexes.py. Everything. Doesn't work. If I type in Comp, it doesn't find Computing. Is there anything else I could do? I have noticed that in the tutorial above, everything works like a charm instantly.

When you do the usual:
SearchQuerySet().filter(title='Computing')
in Haystack 1.x, it filters on everything exactly matching 'Computing'.
You can change that behaviour by using Haystack's Field Lookups, for example, using 'contains' will filter on anything containing the given string (Computing, Utingcomp, Comp):
SearchQuerySet().filter(title__contains='Comp')
In Haystack 2.x, the default filter is 'contains', so it should behave as you would expect it to "out-of-the-box"

Check out the documentation on autocomplete. You need to setup your indices to support Ngram's, but this should be exactly what you need.
from haystack.query import SearchQuerySet
SearchQuerySet().autocomplete(content_auto='old')
# Result match things like 'goldfish', 'cuckold' & 'older'.

So, if I'm understanding, what you're looking for is the equivalent of 'LIKE' in SQL.
The problem is search engines that back Haystack aren't like an RDBMS.
The low level implementation of this filter will involve using wildcard characters but most of the Haystack backends don't support a leading wildcard, something required for an icontains/endswith filter. However, since most backends support trailing wildcards, Haystack 2.x includes a startswith filter. The only case this doesn't handle is searching for the end of a word, which doesn't look to be possible.
So, if you have indexed:
"Look at our great discounts in Computer section"
Then the following Haystack query DO match:
SearchQuerySet().filter(title__startswith='comp')
# match!
Notice the difference between Django vs. Haystack startswith filters. Django startswith will match at the beginning of the complete sentence (i.e. a CharField), but the Haystack one will match at the beginning of a token (i.e. each word in a complete sentence).
Hope it helps!

Django admin URL query string "OR". Is it possible?

I love being able to write quick and dirty query strings right into the URL of the Django admin. Like: /admin/myapp/mymodel/?pub_date__year=2011
AND statements are just as easy: /admin/myapp/mymodel/?pub_date__year=2011&author=Jim
I'm wondering if it's possible to issue an 'OR' statement via the URL. Anyone heard of such functionality?

Django < 1.4 doesn't support OR queries. Sometimes it is possible to translate OR queries to __in - queries which are supported (they are equivalent to OR queries but only for single field values).
You can also upgrade to django development version: it has more versatile list_filter implementation (see https://docs.djangoproject.com/en/dev//ref/contrib/admin/#django.contrib.admin.ModelAdmin.list_filter ) which can be used for providing advanced admin filters (including OR-queries).

The & is not a logical AND, even though it seems to be acting that way in your case. I'm pretty certain there is no way to create a logical OR in the GET query string.