For example, the query is:
a = request.GET['country']
b = request.GET['name']
c = request.GET['title']
...
result = models.Data.objects.filter('country' = a, 'name' = b, 'title' = c, ...)
What should I do if one of these a b c is "Any"?
I mean if I receive from the frontend, a="Any", how should I effectively free this limit in filter?
You can make a function to build a Q object from a parameter:
def _get_filter(name, value):
return Q() if value == 'Any' else Q((name, value))
and combine them with &:
q = (
_get_filter('country', request.GET['country']) &
_get_filter('name', request.GET['name']) &
_get_filter('title', request.GET['title'])
)
result = models.Data.objects.filter(q)
If possible, I’d also get the source of the request (frontend?) to send a better value to indicate “Any”, like an empty string.
You don't have to do everything in a single query. The query is not run until after everything is done.
You can use:
result = models.Data.objects.all()
if a != 'any':
result = result.filter('country' = a)
if b != 'any':
result = result.filter('name' = b)
if c != 'any':
result = result.filter('title' = c)
...
The query won't run against the database until everything is finished, so this really doesn't add any extra queries.
Related
I have some server side processing of some data (client-side library = jQuery DataTables)
I am using POST as my ajax method. In my Flask webapp, I can access the POST data with request.values
The data type / structure of request.values is werkzeug.datastructures.CombinedMultiDict
If the user wants to sort a column, the request contains a key called action with a value of filter (note the below printouts are obtained with for v in request.values: print v, request.values[v])
...
columns[7][data] role
columns[8][search][regex] false
action filter
columns[10][name]
columns[3][search][value]
...
all the column names are also contained in the request as keys. The columns that have search terms will have the search string as a value for the column name key (as opposed to empty for columns with no search term entered. So, If I want to search for firstname containing bill, I would see the following in my request
columns[7][searchable] true
...
columns[6][name]
firstname bill
columns[0][search][value]
columns[2][searchable] true
...
columns[5][data] phone
role
columns[10][data] registered_on
...
columns[0][searchable] true
email
columns[7][orderable] true
...
columns[2][search][value]
Notice how role and email are empty. So my code below is very non-DRY
rv = request.values
if rv.get('action') == 'filter':
if len(rv.get('firstname')):
q = q.filter(User.firstname.ilike('%{0}%'.format(rv.get('firstname'))))
if len(rv.get('lastname')):
q = q.filter(User.lastname.ilike('%{0}%'.format(rv.get('lastname'))))
if len(rv.get('username')):
q = q.filter(User.username.ilike('%{0}%'.format(rv.get('username'))))
if len(rv.get('email')):
q = q.filter(User.email.ilike('%{0}%'.format(rv.get('email'))))
if len(rv.get('phone')):
q = q.filter(User.phone.ilike('%{0}%'.format(rv.get('phone'))))
if len(rv.get('region')):
q = q.filter(User.region.name.ilike('%{0}%'.format(rv.get('region'))))
if len(rv.get('role')):
q = q.filter(User.role.name.ilike('%{0}%'.format(rv.get('role'))))
if len(rv.get('is_active')):
q = q.filter(User.is_active_ == '{0}'.format(rv.get('is_active')))
if len(rv.get('is_confirmed')):
q = q.filter(User.is_confirmed == '{0}'.format(rv.get('is_confirmed')))
if len(rv.get('registered_on_from')):
fdate = datetime.strptime(rv.get('registered_on_from'), '%Y-%m-%d')
q = q.filter(User.registered_on > fdate)
if len(rv.get('registered_on_to')):
tdate = datetime.strptime(rv.get('registered_on_to'), '%Y-%m-%d')
q = q.filter(User.registered_on < tdate)
I was building the sorting functionality, and I found the following statement that greatly simplified my life (see this answer)
q = q.order_by('{name} {dir}'.format(name=sort_col_name, dir=sort_dir))
I was wondering if there was a way to simplify this set of filtering queries like the above sorting code since I will have to do this for many other models.
This should help:
from sqlalchemy import inspect
from sqlalchemy.sql.sqltypes import String,Boolean
def filter_model_by_request(qry,model,rv):
if rv.get('action') == 'filter':
mapper = inspect(model).attrs # model mapper
col_names = list(set([c.key for c in mapper]) & set(rv.keys()))
# col_names is a list generated by intersecting the request values and model column names
for col_name in col_names:
col = mapper[col_name].columns[0]
col_type = type(col.type)
if col_type == String: # filter for String
qry = qry.filter(col.ilike('%{0}%'.format(rv.get(col_name))))
elif col_type == Boolean: # filter for Boolean
qry = qry.filter(col == '{0}'.format(rv.get(col_name)))
return qry
Example call (I used it with a #app.before_request and a cURL call to verify):
qry = db.session.query(User)
print filter_model_by_request(qry,User,request.values).count()
The date range filtering is not included in the function, add this feature if you wish, your code is fine for that purpose.
side note: be careful with the bigger/smaller operators for the dates. You're excluding the actual requested dates. Use <= or >= to include dates in filtering action. It's always a pitfall for me..
I have the following code:
for e in Contact.objects.filter(contact_owner=batch.user, group=batch.group):
msg = Message.objects.create(
recipient_number=e.mobile,
)
However, I want to do something like the below where first I check if the object has a group if not filter on contact instead else filter on group.
As you can see this is not very DRY. How can this be made better?
if self.group == None:
for e in Contact.objects.filter(contact_owner=batch.user, contact=batch.group):
msg = Message.objects.create(
recipient_number=e.mobile,
)
else:
for e in Contact.objects.filter(contact_owner=batch.user, group=batch.group):
msg = Message.objects.create(
recipient_number=e.mobile,
)
The idea you should consider in your case is taken from
https://docs.djangoproject.com/en/dev/topics/db/queries/#chaining-filters
QuerySets are lazy – the act of creating a QuerySet doesn’t involve any database activity. You can stack filters together all day long, and Django won’t actually run the query until the QuerySet is evaluated
This allows you to build queries brick by brick.
q = Contact.objects.filter(contact_owner=batch.user)
if self.group == None:
q = q.filter(contact=batch.group)
else
q = q.filter(group=batch.group)
for e in q:
msg = Message.objects.create(
recipient_number=e.mobile,
)
You can also use the if operator, if you prefer this kind of shorter writing
q=Contact.objects.filter(contact_owner=batch.user)
q = q.filter(contact=batch.group) if self.group is None else q.filter(group=batch.group)
for e in q:
msg = Message.objects.create(
recipient_number=e.mobile,
)
Does anyone know how I get Django Filter build an OR statement? I'm not sure if I have to use the Q object or not, I thought I need some type of OR pipe but this doesn't seem right:
filter_mfr_method = request.GET.getlist('filter_mfr_method')
for m in filter_mfr_method:
designs = designs.filter(Q(mfr_method = m) | m if m else '')
# Or should I do it this way?
#designs = designs.filter(mfr_method = m | m if m else '')
I would like this to be:
SELECT * FROM table WHERE mfr_method = 1 OR mfr_method = 2 OR mfr_method = 3
EDIT: Here is what Worked
filter_mfr_method = request.GET.getlist('filter_mfr_method')
list = []
for m in filter_mfr_method:
list.append(Q(mfr_method = m))
designs = designs.filter(reduce(operator.or_, list))
What about:
import operator
filter_mfr_method = request.GET.getlist('filter_mfr_method')
filter_params = reduce(operator.or_, filter_mfr_method, Q())
designs = designs.filter(filter_params)
Something I used before:
qry = None
for val in request.GET.getlist('filter_mfr_method'):
v = {'mfr_method': val}
q = Q(**v)
if qry:
qry = qry | q
else:
qry = q
designs = designs.filter(qry)
That is taken from one of my query builders.
I've got this complex filtering mechanism...
d = copy(request.GET)
d.setdefault('sort_by', 'created')
d.setdefault('sort_dir', 'desc')
form = FilterShipmentForm(d)
filter = {
'status': ShipmentStatuses.ACTIVE
}
exclude = {}
if not request.user.is_staff:
filter['user__is_staff'] = False
if request.user.is_authenticated():
exclude['user__blocked_by__blocked'] = request.user
if form.is_valid():
d = form.cleaned_data
if d.get('pickup_city'): filter['pickup_address__city__icontains'] = d['pickup_city']
if d.get('dropoff_city'): filter['dropoff_address__city__icontains'] = d['dropoff_city']
if d.get('pickup_province'): filter['pickup_address__province__exact'] = d['pickup_province']
if d.get('dropoff_province'): filter['dropoff_address__province__exact'] = d['dropoff_province']
if d.get('pickup_country'): filter['pickup_address__country__exact'] = d['pickup_country']
if d.get('dropoff_country'): filter['dropoff_address__country__exact'] = d['dropoff_country']
if d.get('min_price'): filter['target_price__gte'] = d['min_price']
if d.get('max_price'): filter['target_price__lte'] = d['max_price']
if d.get('min_distance'): filter['distance__gte'] = d['min_distance'] * 1000
if d.get('max_distance'): filter['distance__lte'] = d['max_distance'] * 1000
if d.get('available_on'): # <--- RELEVANT BIT HERE ---
filter['pickup_earliest__lte'] = d['available_on'] # basically I want "lte OR none"
filter['pickup_latest__gte'] = d['available_on']
if d.get('shipper'): filter['user__username__iexact'] = d['shipper']
order = ife(d['sort_dir'] == 'desc', '-') + d['sort_by']
shipments = Shipment.objects.filter(**filter).exclude(**exclude).order_by(order) \
.annotate(num_bids=Count('bids'), min_bid=Min('bids__amount'), max_bid=Max('bids__amount'))
And now my client tells me he wants pickup/drop-off dates to be 'flexible' as an option. So I've updated the DB to allow dates to be NULL for this purpose, but now the "available for pickup on" filter won't work as expected. It should include NULL/None dates. Is there an easy fix for this?
Flip the logic and use exclude(). What you really want to do is exclude any data that specifies a date that doesn't fit. If pickup_latest and pickup_earliest are NULL it shouldn't match the exclude query and wont be removed. Eg
exclude['pickup_latest__lt'] = d['available_on']
exclude['pickup_earliest__gt'] = d['available_on']
Most database engines don't like relational comparisons with NULL values. Use <field>__isnull to explicitly check if a value is NULL in the database, but you'll need to use Q objects to OR the conditions together.
Don't think that's actually a django-specific question. Variable 'd' is a python dictionary, no? If so, you can use this:
filter['pickup_latest__gte'] = d.get('available_on', None)
I need select rows with django orm. I need equivalent of such query
select * from order where (user_from = u and f1 not is null) or (user_to = u and f2 not is null)
I try to do this way:
Order.objects.filter(user_from = self).exclude(f1 = None)+Order.objects.filter(user_to = self).exclude(f2 = None)
But no union in orm.. how can be such task done by orm?
(i see one solution to add some fields in my model, but it intresting to solve it without fields adding)
Take a look at Q objects. You can execute something like:
Order.objects.filter(
Q(user_from = u, f1__isnull = False) | Q(user_to = u, f2__isnull = False)
)
Warning: this is untested code. It would be a good idea to see the actual SQL query generated and verify that this is indeed what you need.