I know you can update all Django records matching a filter by using:
myQuery = myModel.objects.filter(fieldA = 1, FieldB = 2)
myQuery.update(fieldA = 5, FieldB = 6)
But if I want to iterate through the query results and only update certain values, how can I do this?
I have tried:
myQuery = myModel.objects.filter(fieldA = 1, FieldB = 2)
for item in range(myQuery.count())
if (myQuery[item].fieldC) == 10:
myQuery[item].update(fieldC = 100)
This returns AttributeError: 'myModel' object has no attribute 'update'
As you found out yourself, a Model object has no .update(..) method. You can .save(..) [Django-doc] the object, and specify what fields to update with the update_fields=… parameter [Django-doc]:
myQuery = myModel.objects.filter(fieldA=1, FieldB=2)
for item in myQuery:
if item.fieldC == 10:
item.fieldC = 100
item.save(update_fields=['fieldC'])
That being said, the above is very inefficient. Since for n objects, it will make a total of at most n+1 queries.
You can simply move the condition to the filter part here:
myModel.objects.filter(fieldA=1, FieldB=2, fieldC=10).update(fieldC=100)
Related
There is a model
class Fabric(models.Model):
vendor_code = models.CharField(max_length=50)
color = models.CharField(max_length=50)
lot = models.CharField(max_length=50)
I've a list of objects
values = [
{'vendor_code': '123', 'color': 'aodfe', 'lot': 'some lot 1'},
{'vendor_code': '456', 'color': 'adfae', 'lot': 'some lot 2'},
{'vendor_code': '789', 'color': 'dvade', 'lot': 'some lot 3'},
]
There are no ids in dict objects. How can get objects checking for list of field values(for all 3 values per object at same time)?
I know that I can query one by one in loop as:
for item in values:
fabric = Fabric.objects.filter(vendor_code=item['vendor_code'], color=item['color'], lot=item['lot'])
but amount of objects in list can be large. Is there any proper way to get objects at once, if they exists? Or at least to get them with min amount of db hit.
Thanks in advance!
You can use the in (__in) filter like so:
fabrics = Fabric.objects.filter(
vendor_code__in=[value['vendor_code'] for value in values],
color__in=[value['color'] for value in values],
lot__in=[value['lot'] for value in values],
)
This will however iterate the values list 3 times, to only iterate it once use something like this:
vendor_codes = []
colors = []
lots = []
for value in values:
vendor_codes.append(value['vendor_code'])
colors.append(value['color'])
lots.append(value['lot'])
fabrics = Fabric.objects.filter(
vendor_code__in=vendor_codes,
color__in=colors,
lot__in=lots,
)
To filter according to all three values at the same time you will have to use Q objects like this:
q_objects = []
for value in values:
q_objects.append(Q(
vendor_code=value['vendor_code'],
color=value['color'],
lot=value['lot']
)
)
final_q_object = Q()
for q_object in q_objects:
final_q_object.add(q_object, Q.OR)
fabrics = Fabric.objects.filter(final_q_object)
The gist of it is to get this query:
Q(Q(a=i, b=j, c=k)) | Q(Q(a=l, b=m, c=n) | ...)
Final answer after a bit of optimization:
final_query = Q()
for item in values:
final_query.add(
Q(
vendor_code=value['vendor_code'],
color=value['color'],
lot=value['lot']
),
Q.OR
)
fabrics = Fabric.objects.filter(final_query)
You could use the field lookup "in".
# get all the vendor codes in a list
vendor_code_list = []
for v in values:
vendor_code_list.append(v['vendor_code'])
# query all the fabrics
fabrics = Fabric.objects.filter(vendor_code__in=vendor_code_list)
If you want to match exact values and you are sure that your item keys are valid field names you can just:
for item in values:
fabric = Fabric.objects.filter(**item)
otherwise if you want check if your items are contained inside existing items you can:
for item in values:
item_in = {'{}__in'.format(key):val for key, val in item.items()}
fabric = Fabric.objects.filter(**item_in)
I have two tables like below.
These are in 'one(History.testinfoid) to many(Result.testinfoid)' relationship.
(Result table is external database)
class History(models.Model): # default database
idx = models.AutoField(primary_key=True)
scenario_id = models.ForeignKey(Scenario)
executor = models.CharField(max_length=255)
createdate = models.DateTimeField()
testinfoid = models.IntegerField(unique=True)
class Result(models.Model): # external (Result.objects.using('external'))
idx = models.AutoField(primary_key=True)
testinfoid = models.ForeignKey(History, to_field='testinfoid', related_name='result')
testresult = models.CharField(max_length=10)
class Meta:
unique_together = (('idx', 'testinfoid'),)
So, I want to express the Count by 'testresult' field in Result table.
It has some condition such as 'Pass' or 'Fail'.
I want to express a count query set for each condition. like this.
[{'idx': 1, 'pass_count': 10, 'fail_count': 5, 'executor': 'someone', ...} ...
...
{'idx': 10, 'pass_count': 1, 'fail_count': 10, 'executor': 'someone', ...}]
Is it possible?
It is a two level aggregation where the second level should be displayed as table columns - "matrix view".
A) Solution with Python loop to create columns with annotations by the second level ("testresult").
from django.db.models import Count
from collections import OrderedDict
qs = (History.objects
.values('pk', 'executor', 'testinfoid',... 'result__testresult')
.annotate(result_count=Count('pk'))
)
qs = qs.filter(...).order_by(...)
data = OrderedDict()
count_columns = ('pass_count', 'fail_count', 'error_count',
'expected_failure_count', 'unexpected_success_count')
for row in qs:
data.setdefault(row.pk, dict.fromkeys(count_columns, 0)).update(
{(k if k != result_count else row['result__testresult'] + '_count'): v
for k, v in row_items()
if k != 'result__testresult'
}
)
out = list(data.values())
The class OrderedDict is used to preserve order_by().
B) Solution with Subquery in Django 1.11+ (if the result should be a queryset. e.g. to be sorted or filtered finally in an Admin view by clicking, and if a more complicated query is acceptable and number of columns *_count is very low.). I can write a solution with subquery, but I'm not sure if the query will be fast enough with different database backends. Maybe someone other answers.
My question is very similar to this question: How to modify a queryset and save it as new objects?
Say my models has following fields:
class myModel(models.Model):
articleId = models.ForeignKey(otherModel)
myCounter = models.IntegerField
Now, say keeping the articleId as constant, I want to save multiple rows by varying myCounter. This is what I am trying to do:
for x in range(1, 5):
m = myModel()
m.articleId = otherModel.objects.get(pid="some constant")
m.myCounter = x
m.save()
m.id = None
As suggested by the above post (and similar others), I tried setting both 'id' and 'pk' as None. But nothing is helping.
This code is writing just one row in the database and is updating the value of myCounter. How do I commit 4 different rows?
You could use bulk_create for this:
an_article_object = otherModel.objects.create(name="Some Constant")
myModel.objects.bulk_create([
myModel(articleId=an_article_object, myCounter=x) for x in range(1, 5)
])
EDIT:
To fix your issue in a loop:
article = otherModel.objects.create(name="SomeConst")
#or fetch the article object
m = myModel(articleId = article)
for x in range(1, 5):
m.myCounter = x
m.pk = None
m.save()
Model -
myCounter = models.AutoField(primary_key=True)
#myCounter = models.AutoField(primary_key=False)
#you use primary_key = True if you do not want to use default field "id" given by django to your model
Code -
for x in range(1, 5):
m = myModel()
m.articleId = "some constant"
m.save()
If you want to change the same object you could try this:
m = myModel()
for x in range(1, 5):
m.articleId = otherModel.objects.get(pid="some constant")
m.myCounter = x
m.save()
You can use the force_insert parameter to force an INSERT statement. This will raise an error if the insert fails, but it won't silently update an existing object.
m.save(force_insert=True)
I have some server side processing of some data (client-side library = jQuery DataTables)
I am using POST as my ajax method. In my Flask webapp, I can access the POST data with request.values
The data type / structure of request.values is werkzeug.datastructures.CombinedMultiDict
If the user wants to sort a column, the request contains a key called action with a value of filter (note the below printouts are obtained with for v in request.values: print v, request.values[v])
...
columns[7][data] role
columns[8][search][regex] false
action filter
columns[10][name]
columns[3][search][value]
...
all the column names are also contained in the request as keys. The columns that have search terms will have the search string as a value for the column name key (as opposed to empty for columns with no search term entered. So, If I want to search for firstname containing bill, I would see the following in my request
columns[7][searchable] true
...
columns[6][name]
firstname bill
columns[0][search][value]
columns[2][searchable] true
...
columns[5][data] phone
role
columns[10][data] registered_on
...
columns[0][searchable] true
email
columns[7][orderable] true
...
columns[2][search][value]
Notice how role and email are empty. So my code below is very non-DRY
rv = request.values
if rv.get('action') == 'filter':
if len(rv.get('firstname')):
q = q.filter(User.firstname.ilike('%{0}%'.format(rv.get('firstname'))))
if len(rv.get('lastname')):
q = q.filter(User.lastname.ilike('%{0}%'.format(rv.get('lastname'))))
if len(rv.get('username')):
q = q.filter(User.username.ilike('%{0}%'.format(rv.get('username'))))
if len(rv.get('email')):
q = q.filter(User.email.ilike('%{0}%'.format(rv.get('email'))))
if len(rv.get('phone')):
q = q.filter(User.phone.ilike('%{0}%'.format(rv.get('phone'))))
if len(rv.get('region')):
q = q.filter(User.region.name.ilike('%{0}%'.format(rv.get('region'))))
if len(rv.get('role')):
q = q.filter(User.role.name.ilike('%{0}%'.format(rv.get('role'))))
if len(rv.get('is_active')):
q = q.filter(User.is_active_ == '{0}'.format(rv.get('is_active')))
if len(rv.get('is_confirmed')):
q = q.filter(User.is_confirmed == '{0}'.format(rv.get('is_confirmed')))
if len(rv.get('registered_on_from')):
fdate = datetime.strptime(rv.get('registered_on_from'), '%Y-%m-%d')
q = q.filter(User.registered_on > fdate)
if len(rv.get('registered_on_to')):
tdate = datetime.strptime(rv.get('registered_on_to'), '%Y-%m-%d')
q = q.filter(User.registered_on < tdate)
I was building the sorting functionality, and I found the following statement that greatly simplified my life (see this answer)
q = q.order_by('{name} {dir}'.format(name=sort_col_name, dir=sort_dir))
I was wondering if there was a way to simplify this set of filtering queries like the above sorting code since I will have to do this for many other models.
This should help:
from sqlalchemy import inspect
from sqlalchemy.sql.sqltypes import String,Boolean
def filter_model_by_request(qry,model,rv):
if rv.get('action') == 'filter':
mapper = inspect(model).attrs # model mapper
col_names = list(set([c.key for c in mapper]) & set(rv.keys()))
# col_names is a list generated by intersecting the request values and model column names
for col_name in col_names:
col = mapper[col_name].columns[0]
col_type = type(col.type)
if col_type == String: # filter for String
qry = qry.filter(col.ilike('%{0}%'.format(rv.get(col_name))))
elif col_type == Boolean: # filter for Boolean
qry = qry.filter(col == '{0}'.format(rv.get(col_name)))
return qry
Example call (I used it with a #app.before_request and a cURL call to verify):
qry = db.session.query(User)
print filter_model_by_request(qry,User,request.values).count()
The date range filtering is not included in the function, add this feature if you wish, your code is fine for that purpose.
side note: be careful with the bigger/smaller operators for the dates. You're excluding the actual requested dates. Use <= or >= to include dates in filtering action. It's always a pitfall for me..
I have the following queryset -
pk = Jobmst.objects.db_manager('database1').extra(where=['jobmst_alias=%s'], params=[alias]).values('jobmst_id')
Which returns the following -
[{'jobmst_id': 21195}]
edit -
pk = Jobmst.objects.db_manager('database1').extra(where=['jobmst_alias=%s'], params=[alias]).values_list('jobmst_id', flat=True)
Returns the following -
[21195]
Close but I don't want the brackets I want only the integer.
I want it to simply give me the integer value so I can call it in another query -
mst = Jobmst.objects.db_manager('database1').raw("""
SELECT jobmst_id, jobmst_type, jobmst_prntname AS jobmst_prntid, jobmst_active,
evntmst_id, jobmst_evntoffset, jobmst_name, jobmst_mode, jobmst_owner, jobmst_desc,
jobmst_crttm, jobdtl_id, jobmst_lstchgtm, jobmst_runbook, jobcls_id, jobmst_prntname,
jobmst_alias, jobmst_dirty FROM Jobmst WHERE jobmst_id = %s""", [pk])
Which would ideally parse as this -
mst = Jobmst.objects.db_manager('database1').raw("""
SELECT jobmst_id, jobmst_type, jobmst_prntname AS jobmst_prntid, jobmst_active,
evntmst_id, jobmst_evntoffset, jobmst_name, jobmst_mode, jobmst_owner, jobmst_desc,
jobmst_crttm, jobdtl_id, jobmst_lstchgtm, jobmst_runbook, jobcls_id, jobmst_prntname,
jobmst_alias, jobmst_dirty FROM Jobmst WHERE jobmst_id = 21195""")
values returns list of dicts, so you can access value by list index and key name:
try:
pk = Jobmst.objects.db_manager('database1').extra(where=['jobmst_alias=%s'],
params=[alias]).values('jobmst_id')[0]['jobmst_id']
except IndexError:
pk = None
You can use values_list that returns only list of specified fields values and to access the value you need only index:
try:
pk = Jobmst.objects.db_manager('database1').extra(where=['jobmst_alias=%s'],
params=[alias]).values_list('jobmst_id', flat=True)[0]
except IndexError:
pk = None