Consider the following many-to-one relationship. Users can own many widgets:
class Widget(models.Model):
owner = models.ForeignKey('User')
class User(models.Model):
pass
I have a fairly complicated query that returns me a queryset of Widgets. From that I want to list the distinct values of User.
I know I could loop my Widgets and pull out users with .values('owner') or {% regroup ... %} but the dataset is huge and I'd like to do this at the database so it actually returns Users not Widgets first time around.
My best idea to date is pulling out a .values_list('owner', flat=True) and then doing a User.objects.filter(pk__in=...) using that. That pulls it back to two queries but that still seems like more than it should need to be.
Any ideas?
Use backward relationship:
User.objects.distinct().filter(widget__in=your_widget_queryset)
Related
Let's use these 4 simple models for example. A city can have multiple shops, and a shop can have multiple products, and a product can have multiple images.
models.py
class City(models.Model):
name=models.CharField(max_length=300)
class Shop(models.Model):
name = models.CharField(max_length=300)
city = models.ForeignKey(City, related_name='related_city', on_delete=models.CASCADE)
class Product(models.Model):
name=models.CharField(max_length=300)
description=models.CharField(max_length=5000)
shop=models.ForeignKey(Shop, related_name='related_shop', on_delete=models.CASCADE)
class Image(models.Model):
image=models.ImageField(null=True)
product=models.ForeignKey(Product, related_name='related_product', on_delete=models.CASCADE)
For an eCommerce website, users will be writing keywords and I filter on the products names, to get the matching results. I also want to fetch together the related data of shops, cities and images, relevant to the products I will be showing.
To achieve that I am using .select_related() to retrieve the other objects from the foreignKeys as well.
Now, my question is, what is the best way to send that to the client?
One way is to make a single serializer, that groups all data from all 4 tables in a single JSON. That JSON will look like 1NF, since it will have many repetitions, for example, there will be new row for every image, and every shop that the product can be found, and the 10.000 character long description will be repeated for each row, so this is not such a good idea. More specifically the fields are: (product_id, product_name, product_description, product_image_filepath, product_in_shop_id, shop_in_city_id)
The second approach will use Django queryset caching, which I have no experience at all, and maybe you can give me advice on how to make it efficient.
The second way would be to get Product.objects.filter(by input keywords).select_related().all(), cache this list of id's of products, and return this queryset of the viewset.
Then, from the client's side, I make another GET request, just for the images, and I don't know how to reuse the list of id's of products, that I queried earlier, in the previous viewset / queryset.
How do I fetch only the images that I need, for products that matched the user input keywords, such that I don't need to query the id's of products again?
How will the code look like exactly, for caching this in one viewset, and reusing it again in another viewset?
Then I also need one more GET request to get the list of shops and cities, where the product is available, so it will be fantastic if I can reuse the list of id's I got from the first queryset to fetch the products.
Is the second approach a good idea? If yes, how to implement it exactly?
Or should I stick with the first approach, which I am sceptical is the right way to do this.
I am using PostgreSQL database, and I will probably end up using ElasticSearch for my search engine, to quickly find the matching keywords.
I need to allow users to create and store filters for one of my models. The only decent idea I came up with is something like this:
class MyModel(models.Model):
field1 = models.CharField()
field2 = models.CharField()
class MyModelFilter(models.Model):
owner = models.ForeignKey('User', on_delete=models.CASCADE, verbose_name=_('Filter owner'))
filter = models.TextField(_('JSON-defined filter'), blank=False)
So the filter field store a string like:
{"field1": "value1", "field2": "value2"}.
Then, somewhere in code:
filters = MyModelFilter.objects.filter(owner_id=owner_id)
querysets = [MyModel.objects.filter(**json.loads(filter)) for filter in filters]
result_queryset = reduce(lambda x, y: x|y, querysets)
This is not safe and I need to control available filter keys somehow. On the other hand, it presents full power of django queryset filters. For example, with this code I can filter related models.
So I wonder, is there any better approach to this problem, or maybe a 3rd-party library, that implements same functionality?
UPD:
reduce in code is for filtering with OR condition.
UPD2:
User-defined filters will be used by another part of system to filter newly added model instances, so I really need to store them on server-side somehow (not in cookies or something like that).
SOLUTION:
In the end, I used django-filter to generate filter form, then grabbing it's query data, converting in to json and saving it to the database.
After that, I could deserialize that field and use it in my FilterSet again. One problem that I couldn't solve in a normal way is testing single model in my FilterSet (when model in already fetched and I need to test, it it matches filter) so I ended up doing it manually (by checking each filter condition on model).
Are you sure this is actually what you want to do? Are your end users going to know what a filter is, or how to format the filter?
I suggest that you look into the Django-filter library (https://django-filter.readthedocs.io/).
It will enable you to create filters for your Django models, and then assist you with rendering the filters as forms in the UI.
I have a simple case with 2 models: Item and Category with ManyToMany between them. I want to show a page listing all categories and for each category list of items. I have hundreds of categories so django hits db hundreds of times (when iterating thru categories and calling items.all() for each one). I need to select data from the intermediate table manually and use select_related() to pull item and category for each record - one query instead of hundreds.
I know that introducing 'through' would solve the problem but I don't want to do it now because it may break existing code (using through makes you can't use add, create, or assignment to create relationships - which I want to avoid for now).
So, is it possible at all without creating a model for intermediate table?
You could make a model for your existing table, and just not use it as the through field for the m2m, and make it unmanaged. eg:
class ItemCategory(models.Model):
item = models.ForeignKey('Item')
category = models.ForeignKey('Category')
class Meta:
db_table = 'the_name_of_the_existing_m2m_table'
managed = False
Something like that, anyway.
I'm trying to restrict the selectable values of a 'persons' field in a particular form.
I have a TaskPerson model that has two foreign keys: one for 'task' one for 'person'.
In my form, the persons field should allow the user to select one or more persons, but only those persons which match a certain task.
I've attempted this:
persons = [tp.person for tp in TaskPerson.objects.filter(task=thistask)]
form.fields["persons"].queryset = persons
This list comprehension gives me the correct person objects I require, but my form doesn't display at all, presumably because it gives me only a standard python list.
I had a look over the docs, but I'm not quite sure how to progress. Could someone please advise how I can correctly display my form?
Many thanks
You can easily get a QuerySet of Person objects by following the reverse relationship to TaskPerson
http://docs.djangoproject.com/en/dev/topics/db/queries/#following-relationships-backward
form.fields['field'].queryset = Person.objects.filter(taskperson__task=thistask)
I'm trying to optimise my app by keeping the number of queries to a minimum... I've noticed I'm getting a lot of extra queries when doing something like this:
class Category(models.Model):
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=127, blank=False)
class Project(models.Model):
categories = models.ManyToMany(Category)
Then later, if I want to retrieve a project and all related categories, I have to do something like this :
{% for category in project.categories.all() %}
Whilst this does what I want it does so in two queries. I was wondering if there was a way of joining the M2M field so I could get the results I need with just one query? I tried this:
def category_list(self):
return self.join(list(self.category))
But it's not working.
Thanks!
Which, whilst does what I want, adds an extra query.
What do you mean by this? Do you want to pick up a Project and its categories using one query?
If you did mean this, then unfortunately there is no mechanism at present to do this without resorting to a custom SQL query. The select_related() mechanism used for foreign keys won't work here either. There is (was?) a Django ticket open for this but it has been closed as "wontfix" by the Django developers.
What you want is not seem to possible because,
In DBMS level, ManyToMany relatin is not possible, so an intermediate table is needed to join tables with ManyToMany relation.
On Django level, for your model definition, django creates an ectra table to create a ManyToMany connection, table is named using your two tables, in this example it will be something like *[app_name]_product_category*, and contains foreignkeys for your two database table.
So, you can not even acces to a field on the table with a manytomany connection via django with a such categories__name relation in your Model filter or get functions.