How QuerySets are evaluated in Django? - django

I have following models:
class Product(models.Model):
"""
Basic product
"""
name = models.CharField(max_length=100, db_column='name', unique=True)
url = models.SlugField(max_length=100, db_column="url", unique=True, db_index=True)
description = HTMLField(db_column='description')
category = models.ForeignKey(Category, db_column='category', related_name='products')
class FirstObject(Product):
pass
class FirstProduct(models.Model):
product = models.ForeignKey(FirstObject, db_column='product')
color = models.ForeignKey(Color, db_index=True, db_column='color')
class SecondObject(Product):
pass
class SecondProduct(models.Model):
product = models.ForeignKey(SecondObject, db_column='product')
diameter = models.PositiveSmallIntegerField(db_column='diameter')
In other words I have two different types of products (with different parameters).
I want for particular category (in category can be only one type of product and I know what) select all products with appropriate parameters.
How can this be accomplished efficiently?
If I write Category.objects.get(id=id).products.all() and then use related manager to fetch parameters of particular product, does it mean that database is hitted for every product?
Second approach is to fetch all products in one query and then fetch all parameters.
Then group them in list/dictionary.
What approach is the best? Or maybe there is another approach?
Thank you.

Your schema really does not lend itself well to querying. You will very quickly hit the worst case query behaviour (2 queries for every type of modification to a product. I suggest you have a look at the schema for django-shop-simplevariations and see how they are able to achieve fast lookups (Hint, the schema is structured for prefetch_related to be effective).

Related

Django ORM filter by singular field name

Is it possible to filter a model by some additional/singular version of field name?
For example, using model like (where authors is CharField which contains python-list, e.g. ['Johnny']):
class Movie(models.Model):
title = models.CharField(max_length=100, null=False, blank=False)
actors = models.CharField(max_length=250, null=True, blank=True)
And query something like Movie.objects.filter(input), where input is <QueryDict: {'actor': ["['Johnny']", "['Pablo']"]}>
Anyone has idea how to solve it?
Thanks in advance.
The way you have made the models, you would need to save the actors as a String, for example Johnny, Pablo. Therefore, you would be able to make queries like Movie.objects.all().filter(actors='Johnny, Pablo').
Usually, this should have been done with a many-to-one relationship model, letting you query specific objects, letting you query multiple actors or one natively.

Django - Referential integrity on OneToOneField

I am trying to import a product feed (Product) into Django. I need to maintain a set of selected products (SelectedProduct) which also hold overrides for the product descriptions. I thought the best way to represent this is as a OneToOneField linking SelectedProduct with Product.
class Product(models.Model):
sku = models.CharField(max_length=32, primary_key=True)
title = models.CharField(max_length=200)
description = models.TextField(max_length=2000, blank=True, null=True)
class SelectedProduct(models.Model):
product = models.OneToOneField(Product, db_column='product_sku', on_delete=models.DO_NOTHING)
description = models.TextField(max_length=2000, blank=True, null=True)
For simplicity, each time the product feed is read, I am intending to delete all the products and re-import the whole product feed (within a transaction, so I can rollback if required).
However, I don't want to truncate the SelectedProduct at the same time, since this contains the descriptions which have been overridden. I was hoping that models.DO_NOTHING might help, but it doesn't.
I suppose I either need to temporarily disable the referential integrity while I import the feed (and delete any entries from SelectedProduct which would break the integrity) or I need to represent the relationship differently.
Any advice appreciated please :-)
Note - the above is a simplified representation. I have variants hanging off products too and will have selected variants overriding variant prices.
For the record, I went with the suggestion from #TimNyborg to use update_or_create. The data load now takes roughly 10 times as long, but as it's a batch import of a small-ish dataset the performance isn't an issue and it feels like a better solution overall.

Django M2M with a large table

I have a typical M2M scenario where promotion activities are related to our retailers. However we have a large number of retailers (over 10k) and therefore I can't use the normal multiple select widget.
What I would aim to do is have an 'activity' instance page with a 'retailer' sub-page which would have a table listing all those retailers currently related to the activity. In addition there would be a 'delete' checkbox next to each retailer so they could be removed from the list if necessary. (Naturally, I would also have another search/results page where users could select which retailers they want to add to the list, but I'm sure I can sort that out myself).
Could someone point me in the right direction regarding modelforms and formset factories as I'm not sure where to go from here. It would seem obvious to directly manipulate the app_activity_associated_retailers table but I don't think I can do this with the existing functions. Is there was a pattern for doing this.
class Activity(models.Model):
budget = models.ForeignKey('Budget')
activity_type = models.ForeignKey('ActivityType')
amount = models.DecimalField(max_digits=8, decimal_places=2)
associated_retailers = models.ManyToManyField('Retailer', related_name='associated_retailers')
class Retailer(models.Model):
name = models.CharField(max_length=50)
address01 = models.CharField(max_length=50)
address02 = models.CharField(max_length=50, blank=True)
postcode = models.CharField(max_length=5)
city = models.CharField(max_length=20)
All ManyToManyFields have a through model, whether you define one yourself or not. In your case, it'll have an id, an activity field and a retailer field. You can access the table with Activity.associated_retailers.through -- one "obvious" way is to just expose it as a "model" like
ActivityRetailer = Activity.associated_retailers.through
You can now manipulate these relationships like they were any ol' Django model, so you can i.e. generate querysets like
retailer_records_for_activity = ActivityRetailer.objects.filter(activity_id=1234)
... and you can also create model formsets (complete with that delete checkbox if so configured) for these pseudo-models.

Is this correct setup for a simple cross ref table Django

I am new to django and not very good at building models yet. I wanted to see best way to setup a cross ref table for my project.
In my scenario I want to allow users to search parts based off either manufacturer part number or a resellers part number.
One specific manufacturer part number can be associated to many reseller part numbers but each reseller part number can only be associated to one manufacturer part number.
So for example Manuacturer A makes a widget and calls it w12345. Reseller A calls this widget aw12345, Reseller B calls it bw12345, and Reseller C calls it widgetxyz. If a user searches on anyone of those options they will be brought to the correct part which is w12345 will all its information like price, qty, picture, etc.
I set my models up below, is this the correct way to do this? Or do I need more models?
class Parts(models.Model):
sup_name = models.CharField(max_length=50)
mfr_part = models.CharField(max_length=30)
sup_part = models.CharField(max_length=30)
part_desc = models.CharField(max_length=200)
part_price = models.FloatField(max_length=6)
part_qty = models.IntegerField(max_length=6)
part_img = models.ImageField(upload_to=None, height_field=None, width_field=None, max_length=100)
url = models.URLField()
partxref = models.ForeignKey('PartXRef')
def __unicode__(self):
return self.sup_name
class PartXRef(models.Model):
supplier = models.CharField(max_length=50)
mfr_part_number = models.CharField(max_length=30)
sup_part_number = models.CharField(max_length=30)
def __unicode__(self):
return self.supplier
There are a few things wrong here. The main issue is that the ForeignKey is the wrong way round: there are many possible PartXRefs for each Part, so the FK should live on the former pointing at the latter.
The other issue is that it is a fundamental principle of database design, called normalization, that you should never repeat data; there should be one canonical place where each piece of data lives. In this case, the manufacturer number should live only in Part, and the name of the supplier and the number they've assigned to it should live only in PartXRef.

What is the best way to model a heterogenous many-to-many relationship in Django?

I've searched around for a while, but can't seem to find an existing question for this (although it could be an issue of not knowing terminology).
I'm new to Django, and have been attempting to take a design which should be very expandable over time, and make it work with Django's ORM. Essentially, it's a series of many-to-many relationships using a shared junction table.
The design is a generic game crafting system, which says "if you meet [require], you can create [reward] using [cost] as materials." This allows items to be sold from any number of shops using the same system, and is generic enough to support a wide range of mechanics - I've seen it used successfully in the past.
Django doesn't support multiple M2M relationships sharing the same junction table (apparently since it has no way to work out the reverse relationship), so I seem to have these options:
Let it create its own junction tables, which ends up being six or more, or
Use foreign keys to the junction table in place of a built-in MTM relationship.
The first option is a bit of a mess, since I know I'll eventually have to add additional fields into the junction tables. The second option works pretty well. Unfortunately, because there is no foreign key from the junction table BACK to each of the other tables, I'm constantly fighting the admin system to get it to do what I want.
Here are the affected models:
class Craft(models.Model):
name = models.CharField(max_length=30)
description = models.CharField(max_length=300, blank=True)
cost = models.ForeignKey('Container', related_name="craft_cost")
reward = models.ForeignKey('Container', related_name="craft_reward")
require = models.ForeignKey('Container', related_name="craft_require")
class ShopContent(models.Model):
shopId = models.ForeignKey(Shop)
cost = models.ForeignKey('Container', related_name="shop_cost")
reward = models.ForeignKey('Container', related_name="shop_reward")
require = models.ForeignKey('Container', related_name="shop_require")
description = models.CharField(max_length=300)
class Container(models.Model):
name = models.CharField(max_length=30)
class ContainerContent(models.Model):
containerId = models.ForeignKey(Container, verbose_name="Container")
itemId = models.ForeignKey(Item, verbose_name="Item")
itemMin = models.PositiveSmallIntegerField(verbose_name=u"min amount")
itemMax = models.PositiveSmallIntegerField(verbose_name=u"max amount")
weight = models.PositiveSmallIntegerField(null=True, blank=True)
optionGroup = models.PositiveSmallIntegerField(null=True, blank=True,
verbose_name=u"option group")
Is there a simpler, likely obvious way to get this working? I'm attempting to allow inline editing of ContainerContent information from each related column on the Craft edit interface.
It sounds like you have a sort of "Transaction" that has a name, description, and type, and defines a cost, reward, and requirement. You should define that as a single model, not multiple ones (ShopContent, Craft, etc.).
class Transaction(models.Model):
TYPE_CHOICES = (('Craft', 0),
('Purchase', 1),
)
name = models.CharField(max_length=30)
description = models.CharField(max_length=300, blank=True)
cost = models.ForeignKey('Container')
reward = models.ForeignKey('Container')
require = models.ForeignKey('Container')
type = models.IntegerField(choices = TYPE_CHOICES)
Now Shop etc. can have a single ManyToManyField to Transaction.
Whether or not you use this particular model, the cost, reward and require relationships should all be in one place -- as above, or in OneToOne relationships with Craft, ShopContent etc. As you guessed, you shouldn't have a whole bunch of complex Many-To-Many through tables that are all really the same.
You mention at the bottom of your post that you're
attempting to allow inline editing of ContainerContent information from each related column on the Craft edit interface.
If you're modeling several levels of relationship, and using the admin app, you'll need to either apply some sort of nested inline patch, or use some sort of linking scheme like the one I use in my recent question, How do I add a link from the Django admin page of one object to the admin page of a related object?
I am smelling something is too complicated here, but I might be wrong. As a start,
is this any better? (ContainerContent will be figured out later)
class Cost(models.Model):
name = models.CharField(max_length=30)
class Reward(models.Model):
name = models.CharField(max_length=30)
class Require(models.Model):
name = models.CharField(max_length=30)
class Craft(models.Model):
name = models.CharField(max_length=30)
description = models.CharField(max_length=300, blank=True)
cost = models.ForeignKey(Cost)
reward = models.ForeignKey(Reward)
require = models.ForeignKey(Require)
class Shop(models.Model):
name = models.CharField(max_length=30)
crafts = models.ManyToMany(Craft, blank=True)