Django search objects by multiple keys - django

I am trying to prepare search form where user is able to type 1, 2 or all (3 in this case) search filters.
Lets say that search filters are:
last name, phone and address. I am trying to filter queryset by:
if filterForm.is_valid():
last_name = filterForm.cleaned_data.get('last_name')
phone= filterForm.cleaned_data.get('phone')
address = filterForm.cleaned_data.get('address')
if last_name is None and phone is None and address is None:
pass
#we dont do search id db
else:
clients = Client.objects.filter(Q(last_name__contains=last_name) | Q(phone=phone) | Q(address__contains=address))
Each search key may be blank.
Unfortunately, it returns more results then expected. When I type in search filter "Example" as last name field, it returns all fields with this last name + many others rows.
Any idea how to fix this search issue?

I believe that your search returns more results than expected when any of the search keys are blank since a blank key will match any row with a value.
By only filtering on keys that contains a value it should work better.
Here is one example of how it can be done:
if filterForm.is_valid():
last_name = filterForm.cleaned_data.get('last_name')
phone= filterForm.cleaned_data.get('phone')
address = filterForm.cleaned_data.get('address')
import operator
predicates = []
if last_name:
predicates.append(Q(last_name__contains=last_name))
if phone:
predicates.append(Q(phone=phone))
if address:
predicates.append(Q(address__contains=address))
if len(predicates) == 0:
# Nothing to search for
pass
else:
clients = Client.objects.filter(reduce(operator.or_, predicates))
The code above will dynamically add filters that shall be added to the query. The usage of oprator.or_ will concatenate the statements with OR (=at least one statement needs to be satisfied). If you instead want all statements to be satisfied you can use operator.and_ instead.

Related

Django - creating and saving multiple object in a loop, with ForeignKeys

I am having trouble creating and saving objects in Django. I am very new to Django so I'm sure I'm missing something very obvious!
I am building a price comparison app, and I have a Search model:
Search - all searches carried out, recording best price, worst price, product searched for, time of search etc. I have successfully managed to save these searches to a DB and am happy with this model.
The two new models I am working with are:
Result - this is intended to record all search results returned, for each search carried out. I.e. Seller 1 £100, Seller 2 £200, Seller 3, £300. (One search has many search results).
'Agent' - a simple table of Agents that I compare prices at. (One Agent can have many search Results).
class Agent(models.Model):
agent_id = models.AutoField(primary_key=True)
agent_name = models.CharField(max_length=30)
class Result(models.Model):
search_id = models.ForeignKey(Search, on_delete=models.CASCADE) # Foreign Key of Search table
agent_id = models.ForeignKey(Agent, on_delete=models.CASCADE) # Foreign Key of Agent table
price = models.FloatField()
search_position = models.IntegerField().
My code that is creating and saving the objects is here:
def update_search_table(listed, product):
if len(listed) > 0:
search = Search(product=product,
no_of_agents=len(listed),
valid_search=1,
best_price=listed[0]['cost'],
worst_price=listed[-1]['cost'])
search.save()
for i in range(len(listed)):
agent = Agent.objects.get(agent_name = listed[i]['company'])
# print(agent.agent_id) # Prints expected value
# print(search.search_id) # Prints expected value
# print(listed[i]['cost']) # Prints expected value
# print(i + 1) # Prints expected value
result = Result(search_id = search,
agent_id = agent,
price = listed[i]['cost'],
position = i + 1)
search.result_set.add(result)
agent.result_set.add(result)
result.save()
Up to search.save() is working as expected.
The first line of the for loop is also correctly retrieving the relevant Agent.
The rest of it is going wrong (i.e. not saving any Result objects to the Result table). What I want to achieve is, if there are 10 different agent results returned, create 10 Result objects and save each one. Link each of those 10 objects to the Search that triggered the results, and link each of those 10 objects to the relevant Agent.
Have tried quite a few iterations but not sure where I'm going wrong.
Thanks

Filter multiple Django model fields with variable number of arguments

I'm implementing search functionality with an option of looking for a record by matching multiple tables and multiple fields in these tables.
Say I want to find a Customer by his/her first or last name, or by ID of placed Order which is stored in different model than Customer.
The easy scenario which I already implemented is that a user only types single word into search field, I then use Django Q to query Order model using direct field reference or related_query_name reference like:
result = Order.objects.filter(
Q(customer__first_name__icontains=user_input)
|Q(customer__last_name__icontains=user_input)
|Q(order_id__icontains=user_input)
).distinct()
Piece of a cake, no problems at all.
But what if user wants to narrow the search and types multiple words into search field.
Example: user has typed Bruce and got a whole lot of records back as a result of search.
Now he/she wants to be more specific and adds customer's last name to search.So the search becomes Bruce Wayne, after splitting this into separate parts I'm having Bruce and Wayne. Obviously I don't want to search Orders model because order_id is a single-word instance and it's sufficient to find customer at once so for this case I'm dropping it out of query at all.
Now I'm trying to match customer by both first AND last name, I also want to handle the scenario where the order of provided data is random, to properly handle Bruce Wayne and Wayne Bruce, meaning I still have customers full name but the position of first and last name aren't fixed.
And this is the question I'm looking answer for: how to build query that will search multiple fields of model not knowing which of search words belongs to which table.
I'm guessing the solution is trivial and there's for sure an elegant way to create such a dynamic query, but I can't think of a way how.
You can dynamically OR a variable number of Q objects together to achieve your desired search. The approach below makes it trivial to add or remove fields you want to include in the search.
from functools import reduce
from operator import or_
fields = (
'customer__first_name__icontains',
'customer__last_name__icontains',
'order_id__icontains'
)
parts = []
terms = ["Bruce", "Wayne"] # produce this from your search input field
for term in terms:
for field in fields:
parts.append(Q(**{field: term}))
query = reduce(or_, parts)
result = Order.objects.filter(query).distinct()
The use of reduce combines the Q objects by ORing them together. Credit to that part of the answer goes to this answer.
The solution I came up with is rather complex, but it works exactly the way I wanted to handle this problem:
search_keys = user_input.split()
if len(search_keys) > 1:
first_name_set = set()
last_name_set = set()
for key in search_keys:
first_name_set.add(Q(customer__first_name__icontains=key))
last_name_set.add(Q(customer__last_name__icontains=key))
query = reduce(and_, [reduce(or_, first_name_set), reduce(or_, last_name_set)])
else:
search_fields = [
Q(customer__first_name__icontains=user_input),
Q(customer__last_name__icontains=user_input),
Q(order_id__icontains=user_input),
]
query = reduce(or_, search_fields)
result = Order.objects.filter(query).distinct()

Django QuerySet to search to different fields

I have the following code:
result = Invoice.objects.filter(
Q(client__in=Client.objects.filter(
Q(first_name__icontains=search_param) | Q(last_name__icontains=search_param))))
class Client(models.Model):
first_name = models.CharField(max_length=30)
last_name = models.CharField(max_length=30)
If both first and last name are typed, the search results in nothing. However, if only one of both is typed it works and that I can not understand since that is an OR statement with contains.
My logic is:
Joe Doe contains Joe, therefore the first condition matches and it should return the value found.
You'll need to split the search_param and search on each entry. Since Django does not execute a query until it is evaluated, you can simply append filters to your queryset.
result = Invoice.objects.all()
for chars in search_param.split():
result = result.filter(
Q(client__in=Client.objects.filter(
Q(first_name__icontains=chars)
| Q(last_name__icontains=chars)
)
)
# This should show all who matched any name entered,
# even if you enter last name first.
print(results)

How To perform search on a specific field rather than all fields in search_fields

In Django, if I want to search something, I'd specify the fields in search_fields. What if I want to perform a search on one specific field? I want to implement something like a drop-down to select the fields to perform the search on.(e.g. all, last name, first name, phone number as the drop-down options)
I did not fully understand what you mean but.
suppose when user searched you sent search argument and search field name to your view. _search_term and _search_field
if _search_term:
result = model_cls.objects.all()
args = Q()
Q1 = Q(**{'{0}__icontains'.format(_search_field):_search_term})
args = args | Q1
result = result.filter(*(args,) )
See how I created Q object without knowing field name beforehand.
Result will be your filtered rows, I used icontains, you can change it, you can even let user change it.

Django - filter queryset disregarding spaces

I have a model that contains phone numbers external_number, stored as a char field:
models.py
class PhoneRecord(models.Model):
def __unicode__(self):
return "Call to %s (%s)" % (self.external_number, self.call_date.strftime("%c"))
INBOUND = "I"
OUTBOUND = "O"
DIRECTION_CHOICES = (
(INBOUND, "Inbound"),
(OUTBOUND, "Outbound"),
)
external_number = models.CharField(max_length=20)
call_date = models.DateTimeField()
external_id = models.CharField(max_length=20)
call_duration = models.TimeField()
call_direction = models.CharField(max_length=1, choices=DIRECTION_CHOICES, default=INBOUND)
call = models.FileField(upload_to='calls/%Y/%m/%d')
The form is cleaning and storing the data using the UKPhoneNumberField from https://github.com/g1smd/django-localflavor-UK-forms/blob/master/django/localflavor/uk/forms.py
This means that the number is stored in the database in the format 01234 567890 i.e. with a space in.
I have created a filter using django-filters which works well, when searching for partial phone number except that it doesn't filter correctly when the search term doesn't include the space. i.e.
search for 01234 returns the record for the example above
search for 567890 returns the record for the example above
search for 01234 567890 returns the record for the example above
search for 01234567890 does not return the record for the example above
Now, I could subject the form on the filter to the same restrictions (i.e. UKPhoneNumberField as the input screen, but that then takes away the ability to search for partial phone numbers.
I have also explored the possibility of using something like django-phonenumber-field that will control both the model and the form, but the validation provided by UKPhoneNumberField allows me to validate based on the type of number entered (e.g. mobile or geographical).
What I would ideally like is either
My filter to ignore spaces that are either input by the user in their search query, or any spaces that are in the stored value. Is this even possible?
Apply the validation provided by UKPhoneNumberField to another field type without having to analyse and re-write all the regular expressions provided.
Some other UK Phone number validation I have not found yet!
You could setup the filter to do something like this:
phone_number = "01234 567890"
parts = phone_number.split(" ")
PhoneRecord.objects.filter(
external_number__startswith="{} ".format(parts[0]),
external_number__endswith=" {}".format(parts[1]),
)
That way the filter is looking for the first half of the number with the space and then the second half of the number with the space as well. The only records that would be returned would be ones that had the value of "01234 567890"
I ended up adding a custom filter with a field_class = UKPhoneFilter. This means I can't search on partial numbers, but that was removed as a requirement from the project.
You should think about normalizing the data before you save them into your DB.
You could just use django-phonenumber-field which does just that.
In regarding your problem you can always use django's regex field postfix to query over regular expression.
e.g. MyModel.objects.filter(myfiel__regex=r'[a-Z]+')