I'd like to find adjacent items in a queryset given a primary key, as asked here. However I want this to work for any given queryset, that i haven't set the ordering for myself. How can I find the current ordering of a queryset for __gt and __lt filtering?
For example:
queryset = MyModel.objects.all().order_by('some_field')
next_obj = get_next(queryset, after_pk, 10)
...
def get_next(queryset, pk, n):
#ordering is unknown here
obj = queryset.get(pk=pk)
field = queryset.get_order_by_field_name() #???
return queryset.filter(**{field+'__gt': getattr(obj, field)})[:n]
The model may define a default ordering and I can check if the queryset is ordered, but I don't think either are helpful in this case.
You can retrieve the list of columns from order_by clause by accessing queryset.query.order_by property.
From django docs:
The query attribute is an opaque object. It represents the internals of the query construction and is not part of the public API. However, it is safe (and fully supported) to pickle and unpickle the attribute’s contents as described here.
Related
I want to implement a custom ordering backend (using rest_framework.filters.OrderingFilter) and I basically want to check if the field exists for my queryset first before filtering it without evaluating the queryset.
something like
if queryset.is_available_field(my_field):
queryset = queryset.order_by(my_field)
I cannot try / except because I don't know exactly when my queryset will be evaluated.
I found one solution that works for my case but might not cover some edge cases. Also it is not very elegant.
def has_field(queryset, field):
model_fields = [f.name for f in queryset.model._meta.fields]
annotations = list(queryset.query.annotations.keys())
return field in model_fields + annotations
I'm not sure how best to go about this. I have the following #property method on my Modal:
#property
def check_ins(self):
return SurfCheckIn.objects.filter(surf_waiver=self)
However, I would like to return this compited property in my values() of a queryset. I thought of using a SubQuery annotation:
queryset = self.get_queryset().filter(
performance_datetime__year=date.year,
performance_datetime__month=date.month,
performance_datetime__day=date.day,
void_type='0'
).exclude(
surf_code='SPECTATOR'
).order_by(
'performance_datetime'
).annotate(
surf_check_ins=SubQuery()
).values()
But I'm not sure where to take the SubQuery from here? What would be the most sensible approach to retrieve that #property inside an annotation?
I think #RossRogers is on the right track. Using prefetch_related seems like it would work better than an annotation here, since you want to return all of the related objects and not some aggregate or exists check.
To leverage this, you'd do a prefetch_related and update your property to return the value from the reverse relation instead of performing a new query:
#property
def check_ins(self):
# Use reverse relation here. This is the default name based on your model,
# but if the foreign key has a `related_name` set, use that value instead.
return self.surfcheckin_set.all()
Instead of an annotation, you'd prefetch the set and that will prevent a database hit per returned waiver:
# Again, replace surcheckin_set with the `related_name` if set on the foreign key for `surf_waiver`.
queryset = self.get_queryset().filter(
performance_datetime__year=date.year,
performance_datetime__month=date.month,
performance_datetime__day=date.day,
void_type='0'
).exclude(
surf_code='SPECTATOR'
).prefetch_related(
"surfcheckin_set"
).order_by(
'performance_datetime'
).values()
Calling prefetch_related here means you'll perform one query for the whole queryset and relate the values back to the manager for the reverse relation. When the property calls .all() on the reverse relation, it will use this cached query result.
I have 2 models, company and Product.
class Product(Meta):
company = models.ForeignKey(Company, related_name='products', on_delete=models.CASCADE)
form the database I'm trying to get the Company data and the corresponding products.
From the products I want to get only the name and to be ordered descending by updated_at, created_at.
I'm working with Prefetch object and prefetch_related and definitively I have multiple misunderstandings how they work.
def get_queryset(self):
qs = Company.objects.prefetch_related(
Prefetch('products', queryset=Product.objects.only('name').order_by('-updated_at', '-created_at'))).get()
return qs
The error that I receive is:
get() returned more than one Company
Because I closed the prefetch_related method/function with ))) :
I thought get() will act over the Company object and get it using the pk/slug from the url(as get do by default in DetailView). Seems that is not the case.
I'm already using 'products' the related name in the Prefetch object, why in queryset is necessary to tell again the model queryset=Product.objects.... ?
I was looking at the following example in django documentation:
Question.objects.prefetch_related(Prefetch('choice_set')).get().choice_set.all()
If there is 'choice_set' in Prefetch object why is called at the end choice_set.all() ?
Isn't Django attached to the quesryset in prefetch_related the products to the queryset (question.choice_set) ?
I think my problem is that I don't understand the order of execution, and I'm confused how methods are chained, even if are closed by ')'
queryset.get() will only work if the queryset has a single object. If it contains zero or more than one objects, you'll get an error.
You should return a queryset from the get_queryset object. In the class based view, the code that filters on the pk/slug is in get_object.
The prefetch_related method is useful if you want to fetch the products for multiple countries. The way that the Django docs use get() is confusing in my opinion - if the queryset has a single item then prefetch_related is over-complicated.
If you have a single company, then there is no advantage, and the code will be simpler if you fetch the countries separately, e.g. in get_context_data.
def get_context_data(self, **kwargs):
context = super(MyView, self).get_context_data(**kwargs)
context['products'] = Product.objects.filter(company=self.object).Product.objects.only('name').order_by('-updated_at', '-created_at')))
return context
I've removed the only('name') call. It's an optimisation that you probably don't need.
If you really want to use prefetch_related, then remove the get().
qs = Company.objects.prefetch_related(
Prefetch('products', queryset=Product.objects.order_by('-updated_at', '-created_at')))
By specifying the queryset above, you are able to change the order (you could filter it if you like). If you don't want to customize the queryset you can simply do:
Company.objects.prefetch_related('products')
When you use Question.objects.prefetch_related(...), the queryset is still a list of questions. You need to call choice_set.all() on the individual instances to access their choices. This won't cause any additional queries, because Django has already prefetched the choices.
queryset = Question.objects.prefetch_related(Prefetch('choice_set'))
for question in queryset:
print(question) # the question
print(question.choice_set.all()) # the related choices
My goal is to create a clone of a queryset and then insert it into the database.
Following the suggestions of this post, I have the following code:
qs_new = copy.copy(qs)
MyModel.objects.bulk_create(qs_new)
However, with this code I run into duplicate primary key error. As for now, I only can come up with the following work-around:
qs_new = copy.copy(qs)
for x in qs_new:
x.id = None
MyModel.objects.bulk_create(qs_new)
Question: Can I implement this code snippet without going through loop ?
Can't think of a way without loop, but just a suggestion:
# add all fields here except 'id'
qs = qs.values('field1', 'field2', 'field3')
new_qs = [MyModel(**i) for i in qs]
MyModel.objects.bulk_create(new_qs)
Note that bulk_create behaves differently depending on the underlying database. With Postgres you get the new primary keys set:
Support for setting primary keys on objects created using
bulk_create() when using PostgreSQL was added.
https://docs.djangoproject.com/en/1.10/ref/models/querysets/#django.db.models.query.QuerySet.bulk_create
You should, however make sure that the objects you are creating either have no primary keys or only keys that are not taken yet. In the latter case you should run the code that sets the PKs as well as the bulk_create inside transaction.atomic().
Fetching the values explicitly as suggested by Shang Wang might be faster because only the given values are retrieved from the DB instead of fetching everything. If you have foreign key relations or m2m relations you might want to avoid simply throwing the complex instances into bulk_create but instead explicitly naming all attributes that are required when constructing a new MyModel instance.
Here an example:
class MyModel(Model):
name = TextField(...)
related = ForeignKeyField(...)
my_m2m = ManyToManyField(...)
In case of MyModel above, you would want to preserve the ForeignKey relations by specifying related_id and the PK of the related object in the constructor of MyModel, avoiding specifying related.
With m2m relations, you might end up skipping bulk_create altogether because you need each specific new PK, the corresponding original PK (from the instance that was copied) and the m2m relations of that original instance. Then you would have to create new m2m relations with the new PK and these mappings.
# add all fields here except 'id'
qs = qs.values('name', 'related_id')
MyModel.objects.bulk_create([MyModel(**i) for i in qs])
Note for completeness:
If you have overriden save() on your model (or if you are inheriting from 3rd party with custom save methods), it won't be executed and neither will any post_save handlers (yours or 3rd party).
I tried and you need a loop to set the id to None, then it works. so finally it may be like this:
qs_new = copy.copy(qs)
for q in qs_new:
q.id = None
# also, you can set other fields if you need
MyModel.objects.bulk_create(qs_new)
This works for me.
Right now I have a Django queryset that I want to filter by the result of another query set. Right now I am doing this as (and it works):
field = 'content_object__pk'
values = other_queryset.values_list(field, flat=True)
objects = queryset.filter(pk__in=values)
where the field is the name of a foreign key the pk in queryset. The ORM is smart enough to run the above is one query.
I was trying to simplify this to (ie to filter with the object list themselves rather than having to explicitly say pk):
field = 'content_object'
objects = queryset & other_queryset.values_list(field, flat=True)
but this gives the following error:
AssertionError: Cannot combine queries on two different base models.
What is the right way to do this type of filtering?
You can do the next:
result = MyModel.objects.filter(field__in=other_query)
The results will be the objects in the model where the field is a foreign key to the model in the other_query
you can chain queries in Django...
qs = entry.objects.filter(...).filter(...)