I am implementing a search function for my API to return the object's properties when requested. So far I have tried to use Full Text Search which is usable but it has these annoying things: words have to be spelled correctly for the results to be returned and partial search, for example "appl" instead of "apple", won't work. I have also tried Trigram Similarity but it failed for long sentences. How do I implement a search function that is both accurate and fuzzy in Django?
This works
This won't work
This is my views.py
from django.shortcuts import render
from rest_framework.response import Response
from rest_framework import status
from rest_framework.decorators import api_view
from .models import Object_Locations
from .serializers import Object_LocationsSerializer
from django.contrib.postgres.search import SearchVector, SearchQuery
def index(request):
return render(request, 'main/base.html', {})
#api_view(['GET',])
def LocationsList(request):
if request.method == 'GET':
vector = SearchVector('name', 'desc', 'catergory')
query = request.GET.get('search')
if query:
locations = Object_Locations.objects.annotate(search=vector,).filter(search=SearchQuery(query))
else:
locations = Object_Locations.objects.all()
serializer = Object_LocationsSerializer(locations, many=True)
return Response(serializer.data)
From the SearchQuery documentation :
SearchQuery translates the terms the user provides into a search query object that the database compares to a search vector. By default, all the words the user provides are passed through the stemming algorithms, and then it looks for matches for all of the resulting terms.
If search_type is 'plain', which is the default, the terms are treated as separate keywords. If search_type is 'phrase', the terms are treated as a single phrase. If search_type is 'raw', then you can provide a formatted search query with terms and operators.
In the code above the SearchQuery with the search phrase "a blac husky" is translated to a SQL code like:
... ## plainto_tsquery('a blac husky') ...
So if you want to have results with similar search phrases it's possible to combine query terms:
from django.contrib.postgres.search import SearchQuery, SearchVector
from django.shortcuts import render
from rest_framework import status
from rest_framework.decorators import api_view
from rest_framework.response import Response
from .models import Object_Locations
from .serializers import Object_LocationsSerializer
def index(request):
return render(request, 'main/base.html', {})
#api_view(['GET',])
def LocationsList(request):
if request.method == 'GET':
vector = SearchVector('name', 'desc', 'catergory')
query_terms = request.GET.get('search')
query_raw = ' | '.join(query_terms.split())
query = SearchQuery(query_raw, search_type='raw')
if query.value:
locations = Object_Locations.objects.annotate(search=vector).filter(search=query)
else:
locations = Object_Locations.objects.all()
serializer = Object_LocationsSerializer(locations, many=True)
return Response(serializer.data)
In the above code the SearchQuery with the search phrase "a blac husky" is translated to a SQL code like:
... ## plainto_tsquery('a | blac | husky') ...
It's possible to use logical operation to combine SearchQuery as described in the documentation I linked.
If you want to deepen further you can read an article that I wrote on the subject:
"Full-Text Search in Django with PostgreSQL"
Related
I don't want to run count queries on views where count is not needed.
How can I turn it off?
I found the following workaround in another post on stackoverflow.
The count query is not fired, but I can see pages that have no data.
(I can see up to ?page=10000, even though there are only about 10 pages.)
#settings.py
REST_FRAMEWORK = {
"DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
"PAGE_SIZE": 20,
...
}
import sys
from django.core.paginator import Paginator
from django.utils.functional import cached_property
from rest_framework.pagination import PageNumberPagination
class CustomPaginatorClass(Paginator):
#cached_property
def count(self):
return sys.maxsize
class CustomPagination(PageNumberPagination):
django_paginator_class = CustomPaginatorClass
You can raise a HTTP 404 in case the page does not contain any elements with:
from rest_framework.exceptions import NotFound
class CustomPagination(PageNumberPagination):
django_paginator_class = CustomPaginatorClass
def paginate_queryset(self, queryset, request, view=None):
data = super().paginate_queryset(queryset, request, view=view)
if not data:
raise NotFound('No data found for this page')
return data
This will fetch the paginated data with one query, and then we check if there is at least one element. If that is not the case, we know that the page we are using should not exist.
I am trying to use REST API in django for retrieve some data in json format.
When i hit this url:
http://192.168.2.87:8000/locker/123/
It gives me output like this (from Database)
{"id": 1, "locker_id": 123, "locker_user_name": "taimur"}
But if i want to get the output by passing the parameters like this
http://192.168.2.87:8000/locker/?locker_id=123&locker_user_name=taimur&id=1
views.py
from postman, How can i do this??
from django.shortcuts import render, HttpResponse, get_object_or_404
from django.http import JsonResponse
from .models import Locker
from .serializers import LockerSerializer
from rest_framework.response import Response
from rest_framework import status
from rest_framework.views import APIView
def locker_data_response(request, locker_id):
if request.method == 'GET':
locker_information = get_object_or_404(Locker, locker_id = locker_id)
print(locker_information)
locker_information_serialize = LockerSerializer(locker_information)
print(locker_information_serialize)
return JsonResponse(locker_information_serialize.data)
urls.py
from django.urls import path, re_path
from . import views
urlpatterns = [
re_path('(?P<locker_id>[0-9]+)/$', views.locker_data_response, name='locker_data_response'),
]
You get them from the request object:
def locker_data_response(request):
if request.method == 'GET':
locker_id = request.data.get('locker_id') # this will return None if not found
locker_user_name = request.data.get('locker_user_name')
locker_information = get_object_or_404(Locker, locker_id=locker_id)
print(locker_information)
locker_information_serialize = LockerSerializer(locker_information)
print(locker_information_serialize)
return JsonResponse(locker_information_serialize.data)
And the url will change to:
locker/$
[EDIT: Sorry, if you are using drf you should use data rather than GET]
[EDIT 2: If you want to use it like this, you will also need to change the url and signature of the view]
[EDIT 3: Added the code in the correct place in view]
If your url is something like domain/search/?q=haha, Then you would use request.GET.get('q', '').
q is the parameter you want, And '' is the default value if q isn't found.
If you are instead just configuring your URLconf, Then your captures from the regex are passed to the function as arguments (or named arguments).
Such as:
(r'^user/(?P<username>\w{0,50})/$', views.profile_page,),
Then in your views.py you would have
def profile_page(request, username):
# Rest of the method
I am having a hard time configuring Swagger UI. Here are the very explanatory docs: https://django-rest-swagger.readthedocs.io/en/latest/
YAML docstrings are deprecated. Does somebody know how to configure Swagger UI (query parameters, etc) from within the python code?
If it's impossible for some strange reason. Is there any working alternative or is it the best for me to just go and write, api documentation by hand?
OK, found it. This is not ideal solution - but needed this for frontend (web and mobile) devs - and it do the job.
Basically the newest DRF and Swagger uses from rest_framework.schemas import SchemaGenerator for providing the docs for Swagger.
So I needed to little extend it:
# -*- coding: utf-8 -*-
import urlparse
import coreapi
from rest_framework.schemas import SchemaGenerator
class ParamsSchemaGenerator(SchemaGenerator):
def get_link(self, path, method, callback, view):
"""
Return a `coreapi.Link` instance for the given endpoint.
"""
fields = self.get_path_fields(path, method, callback, view)
fields += self.get_serializer_fields(path, method, callback, view)
fields += self.get_pagination_fields(path, method, callback, view)
fields += self.get_filter_fields(path, method, callback, view)
fields += self.get_docs_fields(path, method, callback, view) # this is the extended line;
if fields and any([field.location in ('form', 'body') for field in fields]):
encoding = self.get_encoding(path, method, callback, view)
else:
encoding = None
if self.url and path.startswith('/'):
path = path[1:]
return coreapi.Link(
url=urlparse.urljoin(self.url, path),
action=method.lower(),
encoding=encoding,
fields=fields
)
# and this is fully custom additional docs method;
def get_docs_fields(self, path, method, callback, view):
fields = []
if hasattr(view, 'docs_fields'):
for field in view.docs_fields:
field = coreapi.Field(
name=field.get('name'),
location=field.get('query'),
required=field.get('required'),
type=field.get('type'),
description=field.get('description')
)
fields.append(field)
return fields
Then i need to define a function that will return the schemas with the generator defined above:
# -*- coding: utf-8 -*-
# monkey patching FTW!
from rest_framework import exceptions
from rest_framework.permissions import AllowAny
from rest_framework.renderers import CoreJSONRenderer
from rest_framework.response import Response
from rest_framework.views import APIView
from rest_framework_swagger import renderers
from kolomnie.core.schema.generator import ParamsSchemaGenerator
def get_params_swagger_view(title=None, url=None):
"""
Returns schema view which renders Swagger/OpenAPI.
(Replace with DRF get_schema_view shortcut in 3.5)
"""
class SwaggerSchemaView(APIView):
_ignore_model_permissions = True
exclude_from_schema = True
permission_classes = [AllowAny]
renderer_classes = [
CoreJSONRenderer,
renderers.OpenAPIRenderer,
renderers.SwaggerUIRenderer
]
def get(self, request):
generator = ParamsSchemaGenerator(title=title, url=url)
schema = generator.get_schema(request=request)
if not schema:
raise exceptions.ValidationError(
'The schema generator did not return a schema Document'
)
return Response(schema)
return SwaggerSchemaView.as_view()
This is how i put it in the urls:
if settings.DEBUG:
api_views = get_params_swagger_view(title='Some API')
And now little more magic, I defined a mixin for the view which stores the documentation fields:
# -*- coding: utf-8 -*-
class DocsMixin(object):
"""
This mixin can be used to document the query parameters in GET
if there's no other way to do it. Please refer to the: ParamsSchemaGenerator.get_links
for more information;
"""
docs_fields = []
And this is how I use it:
class BaseSearchResultsView(generics.GenericAPIView, SearchDocs):
....
Where SearchDocs is this:
class SearchDocs(DocsMixin):
"""
Documents the get query in search;
"""
docs_fields = [
{
'name': 'q',
'location': 'query',
'required': False,
'type': 'string',
'description': 'The base query for the search;',
},
...
Just find out that I do not need mixin :) Just docs_fields defined on the view.
Probably this will not fulfill all your needs - but think it's a good start :)
Happy coding!
How to return complex information in a single api under Django rest framework?
Assuming I have a model:
class Apple(models.Model):
color
size
shape
With a single api: /api/get-apples, I want to return a json like below:
{"red": {"round":[0.1,0.2,0.3],"spigold":[0.3,0.4,0.5]},
"yellow":{"round":[0.1,0.2,0.4],"spigold":[0.2,0.4,0.5]}}
What will be the best way to achieve this?
Create a serializers.py in your app's folder and add this code into it.
from rest_framework import serializers
class AppleSerializer(serializers.ModelSerializer):
class Meta:
model = Apple
fields = ('color', 'size', 'shape',)
In your views.py:
from rest_framework.generics import ListAPIView
from .serializers import AppleSerializer
class get_apples(ListAPIView):
serializer_class = AppleSerializer
def get_queryset(self):
# Here create your queryset.
queryset = Apple.objects.all()
return queryset
In your urls.py:
url(r'^api/get_apples/', views.get_apples.as_view(), name='get_apples'),
And you are good to go.
Output will be like this.
Let's say you have 2 apples.
{{"color": "red", "size": "blabla", "shape": "round"},{...(another apple json)}}
I'd edit my previous answer but I think it is a good example of using serializers with covering view,urls and serializer parts. Therefore, I didn't want to delete it :)
Here is how to return a complex json structure.
As I mentioned before, as far as I know, we can't do something like that by using rest framework's serializers class because it need comparison and grouping. I'll use rest_framework's api_view structure.
Also, I didn't understand types of size and shape in your model and what is your desired output. Therefore, this might be wrong but you'll get the idea anyway.
from rest_framework.decorators import api_view
from django.http import HttpResponse
import json
#api_view(['GET'])
def get_apples(request):
# define your queryset here as you want.
basket = Apple.objects.all()
data = {} # empty dictionary
for apple in basket:
if data.get(apple.color, None) is not None: # if same color of apple exists in basket
# check if shape exists or not.
if data.get(apple.color).get(apple.shape, None) is not None:
data.get(apple.color).get(apple.shape).append(apple.size)
else:
data.get(apple.color)[apple.shape] = [apple.size]
else:
data[apple.color] = {apple.shape: [apple.size]}
return HttpResponse(json.dumps(data), content_type='application/json; charset=utf-8')
I didn't test this code but probably this will work. Let me know if this works or not!
I'm using the DefaultRouter provided by DRF because I need a root api view. However, the items on that view aren't in any logical order. I looked into the source and discovered that each entry is just put into a dictionary (which inherently isn't ordered).
class DefaultRouter(SimpleRouter):
"""
The default router extends the SimpleRouter, but also adds in a default
API root view, and adds format suffix patterns to the URLs.
"""
include_root_view = True
include_format_suffixes = True
root_view_name = 'api-root'
def get_api_root_view(self):
"""
Return a view to use as the API root.
"""
api_root_dict = {}
list_name = self.routes[0].name
for prefix, viewset, basename in self.registry:
api_root_dict[prefix] = list_name.format(basename=basename)
class APIRoot(views.APIView):
_ignore_model_permissions = True
def get(self, request, format=None):
ret = {}
for key, url_name in api_root_dict.items():
ret[key] = reverse(url_name, request=request, format=format)
return Response(ret)
return APIRoot.as_view()
I'd like to order the items on the root api view alphabetically and could easily do that by modifying the source. But I was wondering, have any of you come up with solutions to order the root api items without modifying the source code?
Along the lines of what you suggest and of the first point in Denis Cornehi's answer, here is an extension of DefaultRouter that orders the urls by their base_names:
# myapp/routers.py
from rest_framework import routers
from rest_framework import views
from rest_framework.response import Response
from rest_framework.reverse import reverse
import operator
import collections
class OrderedDefaultRouter(routers.DefaultRouter):
def get_api_root_view(self):
"""
Return a view to use as the API root but do it with ordered links.
"""
api_root_dict = {}
list_name = self.routes[0].name
for prefix, viewset, basename in self.registry:
api_root_dict[prefix] = list_name.format(basename=basename)
class APIRoot(views.APIView):
_ignore_model_permissions = True
def get(self, request, format=None):
ret = {}
for key, url_name in api_root_dict.items():
ret[key] = reverse(url_name, request=request, format=format)
sorted_ret = collections.OrderedDict(sorted(ret.items(), key=operator.itemgetter(0)))
return Response(sorted_ret)
return APIRoot.as_view()
I see two ways here:
as you suggested, override the router, change the APIRoot view (returning an OrderedDict should be enough). In this case I addiotinally would raise an issue with DRF to (perhaps) change it for everyone.
Extend/Override the JSONRenderer to allow sorting the keys when dumping the JSON. Then extend the BrowsableAPIRenderer to set this property. Or just sort all responses. (and, again, perhaps this is interesting for every user of DRF).
With DRF 3.11.2 [not sure from which older version the below will work] you can also do
router = DefaultRouter()
# Register routes ...
router.registry.sort(key=lambda x: x[0])