i have three classes
Product have many Descriptions and each model have many stores
what i want to do
select all products but store.qty value > 0
I've tried
pr = Product.objects.all().exclude(Product__Product_description__qty > 0)
how can i do that ?
class Product
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=255)
class Product_description
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=255)
product = models.ForeignKey(Product)
class Store
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=255)
desc = models.ForeignKey(Product_description)
qty = models.IntegerField()
pr = Product.objects.filter(Product_description__qty__lte = 0)
Or if you really must use exclude:
pr = Product.objects.exclude(Product_description__qty__gt = 0)
all() is not necessary in either case; you just end up building an untriggered proxy that goes into building the filter/exclude queryset afterward. It wastes memory and CPU, but otherwise does nothing. Only the .delete() operator requires a working all() queryset, but it's a special case designed explicitly to avoid the accidental destruction of datasets.
The Django Queryset API documentation is very readable.
Django convention is to name your class ProductDescription.
This seems like a backward hierarchy. Why would stores have "product descriptions?" Isn't that metadata on the product itself, and what you care about is that the stores have a certain quantity of product? Or are these product variants, i.e you want to find all the products for which stores have at least one green or blue or orange one? Something tells me that your project needs a careful re-think.
Related
Please see EDIT1 below, as well.
Using Django 3.0.6 and python3.8, given following models
class Plants(models.Model):
plantid = models.TextField(primary_key=True, unique=True)
class Pollutions(models.Model):
pollutionsid = models.IntegerField(unique=True, primary_key=True)
year = models.IntegerField()
plantid = models.ForeignKey(Plants, models.DO_NOTHING, db_column='plantid')
pollutant = models.TextField()
releasesto = models.TextField(blank=True, null=True)
amount = models.FloatField(db_column="amount", blank=True, null=True)
class Meta:
managed = False
db_table = 'pollutions'
unique_together = (('plantid', 'releasesto', 'pollutant', 'year'))
class Monthp(models.Model):
monthpid = models.IntegerField(unique=True, primary_key=True)
year = models.IntegerField()
month = models.IntegerField()
plantid = models.ForeignKey(Plants, models.DO_NOTHING, db_column='plantid')
power = models.IntegerField(null=False)
class Meta:
managed = False
db_table = 'monthp'
unique_together = ('plantid', 'year', 'month')
I'd like to annotate - based on a foreign key relationship and a fiter a value, particulary - to each plant the amount of co2 and the Sum of its power for a given year. For sake of debugging having replaced Sum by Count using the following query:
annotated = tmp.all().annotate(
energy=Count('monthp__power', filter=Q(monthp__year=YEAR)),
co2=Count('pollutions__amount', filter=Q(pollutions__year=YEAR, pollutions__pollutant="CO2", pollutions__releasesto="Air")))
However this returns too many items (a wrong number using Sum, respectively)
annotated.first().co2 # 60, but it should be 1
annotated.first().energy # 252, but it should be 1
although my database guarantees - as denoted, that (plantid, year, month) and (plantid, releasesto, pollutant, year) are unique together, which can easily be demonstrated:
pl = annotated.first().plantid
testplant = Plants.objects.get(pk=pl) # plant object
pco2 = Pollutions.objects.filter(plantid=testplant, year=YEAR, pollutant="CO2", releasesto="Air")
len(pco2) # 1, as expected
Why does django return to many results and how can I tell django to limit the elements to annotate to the 'current primary key' in other words to only annotate the elements where the foreign key matches the primary key?
I can achieve what I intend to do by using distinct and Max:
energy=Sum('yearly__power', distinct=True, filter=Q(yearly__year=YEAR)),
co2=Max('pollutions__amount', ...
However the performance is inacceptable.
I have tested to use model_to_dict and appending the wanted values "by hand" to the dict, which works for the values itself, but not for sorting the resulted dict (e.g. by energy) and it is acutally faster than the workaround directly above.
It conceptually strikes to me that the manual approach is faster than letting the database do, what it is intended to do.
Is this a feature limitation of django's orm or am I missing something?
EDIT1:
The behaviour is known as bug since 11 years.
Even others "spent a whole day on this".
I am now trying it with subqueries. However the forein key I am using is not a primary key of its table. So the kind of "usual" approach to use "pk=''" does not work. More clearly, trying:
tmp = Plants.objects.filter(somefilter)
subq1 = Subquery(Yearly.objects.filter(pk=OuterRef('plantid'), year=YEAR)) tmp1 = tmp.all().annotate(
energy=Count(Subquery(subq1))
)
returns
OperationalError at /xyz
no such column: U0.yid
Which definitely makes sense because Plants has no clue what a yid is, it only knows plantids. How do I adjust the subquery to that?
Let's say I have two models:
class Testmodel1():
amount = models.IntegerField(null=True)
contact = models.ForeignKey(Testmodel2)
entry_time = models.DateTimeField()
stage = choicesfiled
class Testmodel2():
name = models.CharField()
mobile_no = models.CharField()
I want to find out the object of Testmodel1 for contact > 3 which is created in the last 24 hours last = arrow.utcnow().shift(hours=-24).date().
I am applying a query:
n1=Testmodel1.objects.filter(entry_time__gte=last, stage=1).annotate(t_count=Count('contact')).filter(t_count__gt=3)
But it seems it's not working. Because I am getting an empty queryset.
Any help would be appreciated.
Only a partial answer. Sorry! Your code looks fine to me, so I'm just trying to find a solution by approaching it from a different direction.
Here's how I structure (sort of) similar code on one of my projects.
from datetime import timedelta, date
....
base_date = date.today()
start_date = base_date + timedelta(days=30)
end_date = base_date
possible_holidays = Holiday.objects.filter(
start_date__lte=start_date, end_date__gte=end_date)
From there, could you just do something like:
if possible_holidays.contact_set.count() > 3:
pass
Does that work?
The problem is your Many-to-One relationship is inverted. This relationship is a parent-child relationship, where a parent can have multiple children, but a children can only have one parent. In database this relationship is stored as a child's ForeignKey field that points to the child's parent.
In your case Testmodel1 is a parent and Testmodel2 is a child (Testmodel1 can have multiple contacts represented by Testmodel2) This means that ForeignKey field should belong to Testmodel2, not Testmodel1.
class Testmodel1():
amount = models.IntegerField(null=True)
entry_time = models.DateTimeField()
stage = choicesfiled
class Testmodel2():
name = models.CharField()
mobile_no = models.ForeignKey()
parent = models.ForeignKey(Testmodel1,
related_name='contacts',
)
With this model structure you can reference Testmodel1's contacts as testmodel1.contacts.all(). Your query then should look like this:
n1 = (Testmodel1.objects
.filter(entry_time__gte=last, stage=1)
.annotate(t_count=Count('contacts'))
.filter(t_count__gt=3)
)
docs reference
So I'm trying to put together a webpage and I am currently have trouble putting together a results page for each user in the web application I am putting together.
Here are what my models look like:
class Fault(models.Model):
name = models.CharField(max_length=255)
severity = models.PositiveSmallIntegerField(default=0)
description = models.CharField(max_length=1024, null=False, blank=False)
recommendation = models.CharField(max_length=1024, null=False, blank=False)
date_added = models.DateTimeField(_('date added'), default=timezone.now)
...
class FaultInstance(models.Model):
auto = models.ForeignKey(Auto)
fault = models.ForeignKey(Fault)
date_added = models.DateTimeField(_('date added'), default=timezone.now)
objects = FaultInstanceManager()
...
class Auto(models.Model):
label = models.CharField(max_length=255)
model = models.CharField(max_length=255)
make = models.CharField(max_length=255)
year = models.IntegerField(max_length=4)
user = models.ForeignKey(AUTH_USER_MODEL)
...
I don't know if my model relationships are ideal, however it made sense it my head. So each user can have multiple Auto objects associated to them. And each Auto can have multiple FaultInstance objects associated to it.
In the results page, I want to list out the all the FaultInstances that a user has across their Autos. And under each listed FaultInstance I will have a list of all the autos that the user owns that has the fault, with its information (here is kind of what I had in mind).
All FaultInstance Listing Ordered by Severity (large number to low number)
FaultInstance:
FaultDescription:
FaultRecommendation:
ListofAutosWithFault:
AutoLabel AutoModel AutoYear ...
AutoLabel AutoModel AutoYear ...
Obviously, do things the correct way would mean that I want to do as much of the list creation in the Python/Django side of things and avoid doing any logic or processing in the template. I am able to create a list per severity with the a model manager as seen here:
class FaultInstanceManager(models.Manager):
def get_faults_by_user_severity(self, user, severity):
faults = defaultdict(list)
qs_faultinst = self.model.objects.select_related().filter(
auto__user=user, fault__severity=severity
).order_by('auto__make')
for result in qs_faultinst:
faults[result.fault].append(result)
faults.default_factory = None
return faults
I still need to specify each severity but I guess if I only have 5 severity levels, I can create a list for each severity level and pass each individual one to template. Any suggestions for this is appreciated. However, thats not my problem. My stopping point right now is that I want to create a summary table at the top of their report which can give the user breakdown of fault instances per make|model|year. I can't think of the proper query or data structure to pass on to the template.
Summary (table of all the FaultInstances with the following column headers):
FaultInstance Make|Model|Year NumberOfAutosAffected
This will let me know metrics for a make or a model or a year (in the example below, its separating faults based on model). I'm listing FaultInstances because I'm only listed Faults that a connected to a user.
For Example
Bad Starter Nissan 1
Bad Tailight Honda 2
Bad Tailight Nissan 1
And I am such a perfectionist that I want to do this while optimizing database queries. If I can create a data structure in my original query that will be easily parsed in template and still get both these sections in my report (maybe a defaultdict of a defaultdict(list)), thats what I want to do. Thanks for the help and hopefully my question is thorough and makes sense.
It makes sense to use related names because it simplifies your query. Like this:
class FaultInstance(models.Model):
auto = models.ForeignKey(Auto, related_name='fault_instances')
fault = models.ForeignKey(Fault, related_name='fault_instances')
...
class Auto(models.Model):
user = models.ForeignKey(AUTH_USER_MODEL, related_name='autos')
In this case you can use:
qs_faultinst = user.fault_instances.filter(fault__severity=severity).order_by('auto__make')
instead of:
qs_faultinst = self.model.objects.select_related().filter(
auto__user=user, fault__severity=severity
).order_by('auto__make')
I can't figure out your summary table, may be you meant:
Fault Make|Model|Year NumberOfAutosAffected
In this case you can use aggregation. But It (grouping) would still be slow if you have enough data. The one easy solution is just to denormalize data by creating extra model and create few signals to update it or you can use cache.
If you have a predefined set of severities then think about this:
class Fault(models.Model):
SEVERITY_LOW = 0
SEVERITY_MIDDLE = 1
SEVERITY_HIGH = 2
...
SEVERITY_CHOICES = (
(SEVERITY_LOW, 'Low'),
(SEVERITY_MIDDLE, 'Middle'),
(SEVERITY_HIGH, 'High'),
...
)
...
severity = models.PositiveSmallIntegerField(default=SEVERITY_LOW,
choices=SEVERITY_CHOICES)
...
In your templates you can just iterate through Fault.SEVERITY_CHOICES.
Update:
Change your models:
Аllocate model into a separate model:
class AutoModel(models.Model):
name = models.CharField(max_length=255)
Change the field model of model Auto :
class Auto(models.Model):
...
auto_model = models.ForeignKey(AutoModel, related_name='cars')
...
Add a model:
class MyDenormalizedModelForReport(models.Model):
fault = models.ForeignKey(Fault, related_name='reports')
auto_model = models.ForeignKey(AutoModel, related_name='reports')
year = models.IntegerField(max_length=4)
number_of_auto_affected = models.IntegerField(default=0)
Add a signal:
def update_denormalized_model(sender, instance, created, **kwargs):
if created:
rep, dummy_created = MyDenormalizedModelForReport.objects.get_or_create(fault=instance.fault, auto_model=instance.auto.auto_model, year=instance.auto.year)
rep.number_of_auto_affected += 1
rep.save()
post_save.connect(update_denormalized_model, sender=FaultInstance)
I wonder how I would model my Products model to auto-create (and that the admin-App would also understand it) variants of a Product based on it's variant-parts.
My Products have;
Colors
Sizes
and can probably get more features in the future.
How would I model my Product class to generate all variants of the Product?
Say I would create a new Product in Colors Red Blue Green and in Sizes XS S M L XL.
class Product(models.Model):
name = models.CharField(max_length=200)
class Color(models.Model):
product = models.ForeignKey(Product)
name = models.CharField(max_length=200)
class Size(models.Model):
product = models.ForeignKey(Product)
name = models.CharField(max_length=200)
class FutureVariant(models.Model):
product = models.ForeignKey(Product)
name = models.CharField(max_length=200)
# etc.
Now when I would need a smart method that when I would auto-create all color-size-[FUTURE VARIANT] for that product.
So I would tell Django;
Create new Product
In the colors Red Blue Green
In the sizes XS S M L XL
And the Product class would go and produce Products with all possible combinations in the products_product table.
I'm almost sure that this has design flaws. But I'm just curious how to put this logic in the ORM, and not to write weird procedural code, which would probably go against the DRY principal.
In Database logic I would think of something like this;
PRODUCTS
- id
- name
PRODUCTS_VARIANTS_COLORS
- id
- name
- html_code
PRODUCTS_VARIANTS_SIZES
- id
- name
PRODUCTS_VARIANTS_TABLES
- table_name
- table_id
PRODUCTS_VARIANTS
- product_id
- variant_table
- variant_id
This way I could make endless variant tables, as long as I would register them in my PRODUCTS_VARIANTS_TABLES and store their name as relevant. PRODUCTS_VARIANTS would hold all the the variants of the product, including combinations of them all. I am also aiming to have a selection-phase where the user can chose (in a HTML checkbox-list) which variants it does and doesn't want.
The problem (I think) is that this would not really comply with a logic in the ORM.
I don't know if you are asking about alternatives or just looking to make your way work, but what about splitting a product from it's attributes?
So instead of having separate models for attributes, you just have an Attribute model. This way you are future-proofing your database so you can easily add more attributes (like if you have products with a height and width instead of just color or size).
class AttributeBase(models.Model):
label = models.CharField(max_length=255) # e.g. color, size, shape, etc.
...
class Attribute(models.Model):
base = models.ForeignKey('AttributeBase', related_name='attributes')
value = models.CharField(max_length=255) # e.g. red, L, round, etc.
internal_value = models.CharField(max_length=255, null=True, blank=True) # other values you may need e.g. #ff0000, etc.
...
class ProductAttribute(Attribute):
product = models.ForeignKey('Product', related_name='attributes')
It now becomes very easy to create all attributes for a product...
class Product(models.Model):
...
def add_all_attributes(self):
for attribute in Attribute.objects.all():
self.attributes.add(attribute)
now when you use product.add_all_attributes() that product will contain every attribute. AND you can even make it add attributes of a certain AttributeBase
def add_all_attributes_for_base(self, label):
base = AttributeBase.objects.get(label=label)
for attribute in base.attributes.all():
self.attributes.add(attribute)
You could write something as:
class Product(models.Model):
#classmethod
def create_variants(cls):
# compute all possible combinations
combinations = ...
for combination in combinations:
Product.objects.create(**combination)
Creating all the combinations would indeed happen through registering the possible variants and their possible values.
Note that ORM is there to help you map Django objects to database records, it doesn't help you with producing the database records (read: Django models) that you wish to save.
There is a catalog of products, the products have properties ... want to filter products by the properties.
class Product(models.Model):
name = models.CharField(verbose_name="Название",max_length=255)
description = models.TextField(verbose_name="Описание")
category = models.ManyToManyField(Category,verbose_name=("Категория"))
class Product_Attribute(models.Model):
product = models.ForeignKey(Product)
option = models.ForeignKey(Attribute_Option)
value = models.CharField(verbose_name=("Value"), max_length=255)
class Attribute_Option(models.Model):
description = models.CharField(verbose_name=("Description"), max_length=100)
name = models.SlugField(verbose_name=("Attribute name"), max_length=100)
I want to filter for two or more attribute
Your original question is extremely ambiguous as to what properties you wish to filter the Product objects on. If you want all Product objects that relate to a specific Attribute_Option.description value, you could use:
attribute_options = Attribute_Option.objects.get(description='foo')
product_attributes = Product_Attribute.objects.select_related('Product').filter(option__in=attribute_options)
results = [p.product for p in product_attributes]
If you simply want all Product objects that have both a specific name and a specific description, you could use:
Product.objects.filter(name='foo', description='bar')
I really have no idea which properties you are trying to filter on though. If you specify that in your question, you might be able to get an answer that matches your specific use-case.