I need select rows with django orm. I need equivalent of such query
select * from order where (user_from = u and f1 not is null) or (user_to = u and f2 not is null)
I try to do this way:
Order.objects.filter(user_from = self).exclude(f1 = None)+Order.objects.filter(user_to = self).exclude(f2 = None)
But no union in orm.. how can be such task done by orm?
(i see one solution to add some fields in my model, but it intresting to solve it without fields adding)
Take a look at Q objects. You can execute something like:
Order.objects.filter(
Q(user_from = u, f1__isnull = False) | Q(user_to = u, f2__isnull = False)
)
Warning: this is untested code. It would be a good idea to see the actual SQL query generated and verify that this is indeed what you need.
Related
In a Django search app, I want to query a clickhouse database (using the infi.clickhouse_orm library) for pairs of values such as (a=1 AND b>=1.5) OR (a=2 AND b>=1). In SQL this could be done with
select * from table where a == 1 and b >= 1.5 UNION ALL select * from table where a == 2 and b >= 1
Looking at other exemples I have tried:
With the queryset defined as
qs = TABLE.objects_in(db)
qs_1 = qs.filter(A__eq=1, B__gte=1.5)
qs_2 = qs.filter(A__eq=2, B__gte=1)
The | operator
qs_union = qs_1 | qs_2
which returns
unsupported operand type(s) for |: 'QuerySet' and 'QuerySet'
The UNION operator
qs_union = qs_1.union(qs_2)
which returns
'QuerySet' object has no attribute 'union'
and the Q objects
qs_union = qs.filter(Q(A__eq=1, B__gte=1.5) | Q(A__eq=2, B__gte=1))
which returns
'Q' object has no attribute 'to_sql'
From a clickhouse models, how do you perfom a union of 2, or more, queryset?
Thanks!
Short answer: you should use the Q class of infi.clickhouse_orm.query, like:
from infi.clickhouse_orm.query import Q
The Q-class in info.clickhouse_orm [GitHub] has a to_sql method:
class Q(object):
# ...
def to_sql(self, model_cls):
if self._fovs:
sql = ' {} '.format(self._mode).join(fov.to_sql(model_cls) for fov in self._fovs)
else:
if self._l_child and self._r_child:
sql = '({} {} {})'.format(
self._l_child.to_sql(model_cls), self._mode, self._r_child.to_sql(model_cls))
else:
return '1'
if self._negate:
sql = 'NOT (%s)' % sql
return sql
Since the errors says it can not find the to_sql, it looks like you did not use that Q-class, but Django's Q class.
I want to multiply two values in a queryset how do I achieve this:
This is what I have :
def get_adjustment(self):
Adjustment.objects.filter(employee_month_id=1,adjustment_type=2)
.values('exchange_rate','amount').aggregate(Sum('amount', field="exchange_rate*amount")
You need to use F() objects.
.values('exchange_rate','amount').aggregate(Sum(F('amount') * F('exchange_rate')))
See the query expressions docs.
obj = SomeModel.objects.filter(**something).extra(
select = {'total': 'SUM(one_column * another_column)'},
)
Adjustment.objects.filter(employee_month_id=1,adjustment_type=2)
.values('exchange_rate','amount').extra(select = {'total': 'SUM(exchange_rate * amount)'})
For example, the query is:
a = request.GET['country']
b = request.GET['name']
c = request.GET['title']
...
result = models.Data.objects.filter('country' = a, 'name' = b, 'title' = c, ...)
What should I do if one of these a b c is "Any"?
I mean if I receive from the frontend, a="Any", how should I effectively free this limit in filter?
You can make a function to build a Q object from a parameter:
def _get_filter(name, value):
return Q() if value == 'Any' else Q((name, value))
and combine them with &:
q = (
_get_filter('country', request.GET['country']) &
_get_filter('name', request.GET['name']) &
_get_filter('title', request.GET['title'])
)
result = models.Data.objects.filter(q)
If possible, I’d also get the source of the request (frontend?) to send a better value to indicate “Any”, like an empty string.
You don't have to do everything in a single query. The query is not run until after everything is done.
You can use:
result = models.Data.objects.all()
if a != 'any':
result = result.filter('country' = a)
if b != 'any':
result = result.filter('name' = b)
if c != 'any':
result = result.filter('title' = c)
...
The query won't run against the database until everything is finished, so this really doesn't add any extra queries.
I have some server side processing of some data (client-side library = jQuery DataTables)
I am using POST as my ajax method. In my Flask webapp, I can access the POST data with request.values
The data type / structure of request.values is werkzeug.datastructures.CombinedMultiDict
If the user wants to sort a column, the request contains a key called action with a value of filter (note the below printouts are obtained with for v in request.values: print v, request.values[v])
...
columns[7][data] role
columns[8][search][regex] false
action filter
columns[10][name]
columns[3][search][value]
...
all the column names are also contained in the request as keys. The columns that have search terms will have the search string as a value for the column name key (as opposed to empty for columns with no search term entered. So, If I want to search for firstname containing bill, I would see the following in my request
columns[7][searchable] true
...
columns[6][name]
firstname bill
columns[0][search][value]
columns[2][searchable] true
...
columns[5][data] phone
role
columns[10][data] registered_on
...
columns[0][searchable] true
email
columns[7][orderable] true
...
columns[2][search][value]
Notice how role and email are empty. So my code below is very non-DRY
rv = request.values
if rv.get('action') == 'filter':
if len(rv.get('firstname')):
q = q.filter(User.firstname.ilike('%{0}%'.format(rv.get('firstname'))))
if len(rv.get('lastname')):
q = q.filter(User.lastname.ilike('%{0}%'.format(rv.get('lastname'))))
if len(rv.get('username')):
q = q.filter(User.username.ilike('%{0}%'.format(rv.get('username'))))
if len(rv.get('email')):
q = q.filter(User.email.ilike('%{0}%'.format(rv.get('email'))))
if len(rv.get('phone')):
q = q.filter(User.phone.ilike('%{0}%'.format(rv.get('phone'))))
if len(rv.get('region')):
q = q.filter(User.region.name.ilike('%{0}%'.format(rv.get('region'))))
if len(rv.get('role')):
q = q.filter(User.role.name.ilike('%{0}%'.format(rv.get('role'))))
if len(rv.get('is_active')):
q = q.filter(User.is_active_ == '{0}'.format(rv.get('is_active')))
if len(rv.get('is_confirmed')):
q = q.filter(User.is_confirmed == '{0}'.format(rv.get('is_confirmed')))
if len(rv.get('registered_on_from')):
fdate = datetime.strptime(rv.get('registered_on_from'), '%Y-%m-%d')
q = q.filter(User.registered_on > fdate)
if len(rv.get('registered_on_to')):
tdate = datetime.strptime(rv.get('registered_on_to'), '%Y-%m-%d')
q = q.filter(User.registered_on < tdate)
I was building the sorting functionality, and I found the following statement that greatly simplified my life (see this answer)
q = q.order_by('{name} {dir}'.format(name=sort_col_name, dir=sort_dir))
I was wondering if there was a way to simplify this set of filtering queries like the above sorting code since I will have to do this for many other models.
This should help:
from sqlalchemy import inspect
from sqlalchemy.sql.sqltypes import String,Boolean
def filter_model_by_request(qry,model,rv):
if rv.get('action') == 'filter':
mapper = inspect(model).attrs # model mapper
col_names = list(set([c.key for c in mapper]) & set(rv.keys()))
# col_names is a list generated by intersecting the request values and model column names
for col_name in col_names:
col = mapper[col_name].columns[0]
col_type = type(col.type)
if col_type == String: # filter for String
qry = qry.filter(col.ilike('%{0}%'.format(rv.get(col_name))))
elif col_type == Boolean: # filter for Boolean
qry = qry.filter(col == '{0}'.format(rv.get(col_name)))
return qry
Example call (I used it with a #app.before_request and a cURL call to verify):
qry = db.session.query(User)
print filter_model_by_request(qry,User,request.values).count()
The date range filtering is not included in the function, add this feature if you wish, your code is fine for that purpose.
side note: be careful with the bigger/smaller operators for the dates. You're excluding the actual requested dates. Use <= or >= to include dates in filtering action. It's always a pitfall for me..
I've got this complex filtering mechanism...
d = copy(request.GET)
d.setdefault('sort_by', 'created')
d.setdefault('sort_dir', 'desc')
form = FilterShipmentForm(d)
filter = {
'status': ShipmentStatuses.ACTIVE
}
exclude = {}
if not request.user.is_staff:
filter['user__is_staff'] = False
if request.user.is_authenticated():
exclude['user__blocked_by__blocked'] = request.user
if form.is_valid():
d = form.cleaned_data
if d.get('pickup_city'): filter['pickup_address__city__icontains'] = d['pickup_city']
if d.get('dropoff_city'): filter['dropoff_address__city__icontains'] = d['dropoff_city']
if d.get('pickup_province'): filter['pickup_address__province__exact'] = d['pickup_province']
if d.get('dropoff_province'): filter['dropoff_address__province__exact'] = d['dropoff_province']
if d.get('pickup_country'): filter['pickup_address__country__exact'] = d['pickup_country']
if d.get('dropoff_country'): filter['dropoff_address__country__exact'] = d['dropoff_country']
if d.get('min_price'): filter['target_price__gte'] = d['min_price']
if d.get('max_price'): filter['target_price__lte'] = d['max_price']
if d.get('min_distance'): filter['distance__gte'] = d['min_distance'] * 1000
if d.get('max_distance'): filter['distance__lte'] = d['max_distance'] * 1000
if d.get('available_on'): # <--- RELEVANT BIT HERE ---
filter['pickup_earliest__lte'] = d['available_on'] # basically I want "lte OR none"
filter['pickup_latest__gte'] = d['available_on']
if d.get('shipper'): filter['user__username__iexact'] = d['shipper']
order = ife(d['sort_dir'] == 'desc', '-') + d['sort_by']
shipments = Shipment.objects.filter(**filter).exclude(**exclude).order_by(order) \
.annotate(num_bids=Count('bids'), min_bid=Min('bids__amount'), max_bid=Max('bids__amount'))
And now my client tells me he wants pickup/drop-off dates to be 'flexible' as an option. So I've updated the DB to allow dates to be NULL for this purpose, but now the "available for pickup on" filter won't work as expected. It should include NULL/None dates. Is there an easy fix for this?
Flip the logic and use exclude(). What you really want to do is exclude any data that specifies a date that doesn't fit. If pickup_latest and pickup_earliest are NULL it shouldn't match the exclude query and wont be removed. Eg
exclude['pickup_latest__lt'] = d['available_on']
exclude['pickup_earliest__gt'] = d['available_on']
Most database engines don't like relational comparisons with NULL values. Use <field>__isnull to explicitly check if a value is NULL in the database, but you'll need to use Q objects to OR the conditions together.
Don't think that's actually a django-specific question. Variable 'd' is a python dictionary, no? If so, you can use this:
filter['pickup_latest__gte'] = d.get('available_on', None)