I am in the middle of writing tests and must be missing something basic, because it appears that objects that I create remain isolated.
I have Customers that can hold a number of Items. Inside the test, after creating Items (with a price) belonging to a Customer, I want to sum the prices of the items for a Customer (using a Customer method), but keep getting 0.
The actual code is more complex, so I simplified the details while keeping the structure and logic the same.
The version with the "production" database works fine, so there must be something about the way tests and their databases are run that I am missing.
### models.py
class Customer(models.Model):
name = models.CharField(max_length=20)
def compute_total(self):
total = 0
items = self.item_set.all()
for item in items:
total += item.price
return total
def __str__(self):
return self.name
class Item(models.Model):
customer = models.ForeignKey(Customer, on_delete=models.CASCADE)
name = models.CharField(max_length=20)
price = models.IntegerField()
def __str__(self):
return "%s %s (%s)" % (self.customer, self.name, self.price)
### tests.py
class CustomerTests(TestCase):
def test_add_two_items(self):
ken = Customer(name="Ken")
apple = Item(customer=ken, name="apple", price=10)
banana = Item(customer=ken, name="banana", price=2)
print(apple) # make sure apple is created, and it is.
self.assertEquals(ken.compute_total(), 12)
For the "production" version, I manually add Joe and two fruits as above in the admin, and test it with:
### views.py
def index(request):
joe = Customer.objects.get(name="Joe")
return HttpResponse(joe.compute_total())
Here, the total is what I expect, so how do I achieve this behavior inside a test?
What happens is when you use item_set in compute_total function it queries the database for all items with customer_id of ken.
But there are no items, and there is no ken because you didn't actually persisted anything into the database. You just created some objects.
So what you need to do is:
ken = Customer(name="Ken")
ken.save() # Ken must be persisted before creating his items.
apple = Item(customer=ken, name="apple", price=10)
banana = Item(customer=ken, name="banana", price=2)
apple.save()
banana.save()
First we need to persist ken, so that the database creates an ID for him. Then we create apple and banana with that id as a foreign key, and persist them as well.
Related
In my frontend application I have a panel which shows 6 the most popular products on my site. Searching for the most popular products each request by the number of views can be costly. I think that a good way to improve it will be to create a table in my data base which will store 6 the most popular products and the table will be refreshed for example every one minute. What should I looking for to do so cyclical operation on my django backend?
You can create separate model
models.py
class ProductRecord(models.Model):
product = models.OneToOneField(
'Product'
related_name='stats', on_delete=models.CASCADE)
# Data used for generating a score
num_views = models.PositiveIntegerField(default=0)
num_basket_additions = models.PositiveIntegerField(
default=0)
num_purchases = models.PositiveIntegerField(
default=0)
# Product score - used within search
score = models.FloatField(default=0.00)
def __str__(self):
return _("Record for '%s'") % self.product
rule_engine.py
class Calculator:
weights = {
'num_views': 1,
'num_basket_additions': 3,
'num_purchases': 5
}
def calculate_scores(self):
total_weight = float(sum(self.weights.values()))
weighted_fields = [
self.weights[name] * F(name) for name in self.weights.keys()]
ProductRecord.objects.update(
score=sum(weighted_fields) / total_weight)
receivers.py
Here you'll have multiple signals to receive addition to basket, views, favoriting, and so on..
This is an example
def receive_product_view(sender, product, user, **kwargs):
UserProductView.objects.create(product=product, user=user)
## yyou can do much advanced staff here using signals
Another solution
You can use Redis to save such these data like counts and purchases
Or make aws lambda function for receiving analytical data, all of these solutions depend on your case
This is a pseudo solution. hope it's been beneficial to you
i'm trying to call back unique constraints field , in my project i have to count number of M2M selected
class Booking(models.Model):
room_no = models.ForeignKey(Room,on_delete=models.CASCADE,blank=True,related_name='rooms')
takes_by = models.ManyToManyField(Vistor)
#property
def no_persons(self):
qnt = Booking.objects.filter(takes_by__full_information=self).count()#but this doesnt work
return qnt
Cannot query "some room information": Must be "Vistor" instance.
class Vistor(models.Model):
full_name = models.CharField(max_length=150)
dob = models.DateField(max_length=14)
city = models.ForeignKey(City,on_delete=models.CASCADE)
class Meta:
constraints = [
models.UniqueConstraint(fields=['full_name','dob','city'],name='full_information')
]
def __str__(self):
return f'{self.full_name} - {self.city} - {self.dob}'
it it possible to access full_information through Booking model ? thank you ..
If you want to count the number of Visitors related to that booking, you can count these with:
#property
def no_persons(self):
self.taken_by.count()
This will make an extra query to the database, therefore it is often better to let the database count these in the query. You can thus remove the property, and query with:
from django.db.models import Count
Booking.objects.annotate(
no_persons=Count('takes_by')
)
The Bookings that arise from this QuerySet will have an extra attribute no_persons with the number of related Visitors.
I am trying to update several records when I hit the save button in the admin with a raw sql which is located in models.py (def save(self, *args, **kwargs)
The raw sql is like this as a prototype
cursor=connection.cursor()
cursor.execute("UPDATE sales_ordered_item SET oi_delivery = %s WHERE oi_order_id = %s", ['2011-05-29', '1105212105'])
Unfortunately it does not update all records which meet the criteria. Only one and sometimes more but never all.
With the SQLite Manager and the following SQL everything works great and all the records get updated:
UPDATE sales_ordered_item
SET oi_delivery = '2011-05-29'
WHERE oi_order_id = '1105212105'
I was thinking of using a manager to update the table but I have no idea how this would work when not using static data like '2011-05-29'. Anyways, it would be great to understand in the first place how to hit all records with the raw sql.
Any recommendations how to solve the problems in a different way are also highly appreciated
Here ist the code which I stripped a little to keep it short
# Orders of the customers
class Order(models.Model):
"""
Defines the order data incl. payment, shipping and delivery
"""
# Main Data
o_customer = models.ForeignKey(Customer, related_name='customer',
verbose_name=_u'Customer'), help_text=_(u'Please select the related Customer'))
o_id = models.CharField(_(u'Order ID'), max_length=10, primary_key=True,
help_text=_(u'ID has the format YYMMDDHHMM'))
o_date = models.DateField(_(u'created'))
and more...
# Order Item
class Ordered_item(models.Model):
"""
Defines the ordered item to which order it belongs, pricing is decoupled from the
catalogue to be free of any changes in the pricing. Pricing and description is copied
from the item catalogue as a proposal and can be altered
"""
oi_order = models.ForeignKey(Order, related_name='Order', verbose_name=_(u'Order ID'))
oi_pos = models.CharField(_('Position'), max_length=2, default='01')
oi_quantity = models.PositiveIntegerField(_('Quantity'), default=1)
# Date of the delivery to determine the status of the item: ordered or already delivered
oi_delivery = models.DateField(_(u'Delivery'), null=True, blank=True)
and more ...
def save(self, *args, **kwargs):
# does not hit all records, use static values for test purposes
cursor=connection.cursor()
cursor.execute("UPDATE sales_ordered_item SET oi_delivery = %s WHERE oi_order_id = %s", ['2011-05-29', '1105212105'])
super(Ordered_item, self).save(*args, **kwargs)
This is probably happening because you are not commiting the transaction (See https://docs.djangoproject.com/en/dev/topics/db/sql/#executing-custom-sql-directly)
Add these lines after your cursor.execute:
from django.db import transaction
transaction.commit_unless_managed()
You asked for a manager method.
SalesOrderedItem.objects.filter(oi_order='1105212105').update(oi_delivery='2011-05-29')
should do the job for you!
Edit:
I assume that you have two models (I am guessing this code from your raw SQL):
class OiOrder(models.Model):
pass
class SalesOrderedItem(models.Model):
oi_order = models.ForeignKey(OiOrder)
oi_delivery = models.DateField()
So:
SalesOrderedItem.objects.filter(oi_order='1105212105')
gives you all SalesOrderedItem which have a oi_order of 1105212105.
... update(oi_delivery='2011-05-29')
The update method updates all oi_delivery attributes.
EDIT:
It turns out the real question is - how do I get select_related to follow the m2m relationships I have defined? Those are the ones that are taxing my system. Any ideas?
I have two classes for my django app. The first (Item class) describes an item along with some functions that return information about the item. The second class (Itemlist class) takes a list of these items and then does some processing on them to return different values. The problem I'm having is that returning a list of items from Itemlist is taking a ton of queries, and I'm not sure where they're coming from.
class Item(models.Model):
# for archiving purposes
archive_id = models.IntegerField()
users = models.ManyToManyField(User, through='User_item_rel',
related_name='users_set')
# for many to one relationship (tags)
tag = models.ForeignKey(Tag)
sub_tag = models.CharField(default='',max_length=40)
name = models.CharField(max_length=40)
purch_date = models.DateField(default=datetime.datetime.now())
date_edited = models.DateTimeField(auto_now_add=True)
price = models.DecimalField(max_digits=6, decimal_places=2)
buyer = models.ManyToManyField(User, through='Buyer_item_rel',
related_name='buyers_set')
comments = models.CharField(default='',max_length=400)
house_id = models.IntegerField()
class Meta:
ordering = ['-purch_date']
def shortDisplayBuyers(self):
if len(self.buyer_item_rel_set.all()) != 1:
return "multiple buyers"
else:
return self.buyer_item_rel_set.all()[0].buyer.name
def listBuyers(self):
return self.buyer_item_rel_set.all()
def listUsers(self):
return self.user_item_rel_set.all()
def tag_name(self):
return self.tag
def sub_tag_name(self):
return self.sub_tag
def __unicode__(self):
return self.name
and the second class:
class Item_list:
def __init__(self, list = None, house_id = None, user_id = None,
archive_id = None, houseMode = 0):
self.list = list
self.house_id = house_id
self.uid = int(user_id)
self.archive_id = archive_id
self.gen_balancing_transactions()
self.houseMode = houseMode
def ret_list(self):
return self.list
So after I construct Itemlist with a large list of items, Itemlist.ret_list() takes up to 800 queries for 25 items. What can I do to fix this?
Try using select_related
As per a question I asked here
Dan is right in telling you to use select_related.
select_related can be read about here.
What it does is return in the same query data for the main object in your queryset and the model or fields specified in the select_related clause.
So, instead of a query like:
select * from item
followed by several queries like this every time you access one of the item_list objects:
select * from item_list where item_id = <one of the items for the query above>
the ORM will generate a query like:
select item.*, item_list.*
from item a join item_list b
where item a.id = b.item_id
In other words: it will hit the database once for all the data.
You probably want to use prefetch_related
Works similarly to select_related, but can deal with relations selected_related cannot. The join happens in python, but I've found it to be more efficient for this kind of work than the large # of queries.
Related reading on the subject
What I want is to be able to get this weeks/this months/this years etc. hotest products. So I have a model named ProductStatistics that will log each hit and each purchase on a day-to-day basis. This is the models I have got to work with:
class Product(models.Model):
name = models.CharField(_("Name"), max_length=200)
slug = models.SlugField()
description = models.TextField(_("Description"))
picture = models.ImageField(upload_to=product_upload_path, blank=True)
category = models.ForeignKey(ProductCategory)
prices = models.ManyToManyField(Store, through='Pricing')
objects = ProductManager()
class Meta:
ordering = ('name', )
def __unicode__(self):
return self.name
class ProductStatistic(models.Model):
# There is only 1 `date` each day. `date` is
# set by datetime.today().date()
date = models.DateTimeField(default=datetime.now)
hits = models.PositiveIntegerField(default=0)
purchases = models.PositiveIntegerField(default=0)
product = models.ForeignKey(Product)
class Meta:
ordering = ('product', 'date', 'purchases', 'hits', )
def __unicode__(self):
return u'%s: %s - %s hits, %s purchases' % (self.product.name, str(self.date).split(' ')[0], self.hits, self.purchases)
How would you go about sorting the Products after say (hits+(purchases*2)) the latest week?
This structure isn't set in stone either, so if you would structure the models in any other way, please tell!
first idea:
in the view you could query for today's ProductStatistic, than loop over the the queryset and add a variable ranking to every object and add that object to a list. Then just sort after ranking and pass the list to ur template.
second idea:
create a filed ranking (hidden for admin) and write the solution of ur formula each time the object is saved to the database by using a pre_save-signal. Now you can do ProductStatistic.objects.filter(date=today()).order_by('ranking')
Both ideas have pros&cons, but I like second idea more
edit as response to the comment
Use Idea 2
Write a view, where you filter like this: ProductStatistic.objects.filter(product= aProductObject, date__gte=startdate, date__lte=enddate)
loop over the queryset and do somthing like aProductObject.ranking+= qs_obj.ranking
pass a sorted list of the queryset to the template
Basically a combination of both ideas
edit to your own answer
Your solution isn't far away from what I suggested — but in sql-space.
But another solution:
Make a Hit-Model:
class Hit(models.Model):
date = models.DateTimeFiles(auto_now=True)
product = models.ForeignKey(Product)
purchased= models.BooleanField(default=False)
session = models.CharField(max_length=40)
in your view for displaying a product you check, if there is a Hit-object with the session, and object. if not, you save it
Hit(product=product,
date=datetime.datetime.now(),
session=request.session.session_key).save()
in your purchase view you get the Hit-object and set purchased=True
Now in your templates/DB-Tools you can do real statistics.
Of course it can generate a lot of DB-Objects over the time, so you should think about a good deletion-strategy (like sum the data after 3 month into another model MonthlyHitArchive)
If you think, that displaying this statistics would generate to much DB-Traffic, you should consider using some caching.
I solved this the way I didn't want to solve it. I added week_rank, month_rank and overall_rank to Product and then I just added the following to my ProductStatistic model.
def calculate_rank(self, days_ago=7, overall=False):
if overall:
return self._default_manager.all().extra(
select = {'rank': 'SUM(hits + (clicks * 2))'}
).values()[0]['rank']
else:
return self._default_manager.filter(
date__gte = datetime.today()-timedelta(days_ago),
date__lte = datetime.today()
).extra(
select = {'rank': 'SUM(hits + (clicks * 2))'}
).values()[0]['rank']
def save(self, *args, **kwargs):
super(ProductStatistic, self).save(*args, **kwargs)
t = Product.objects.get(pk=self.product.id)
t.week_rank = self.calculate_rank()
t.month_rank = self.calculate_rank(30)
t.overall_rank = self.calculate_rank(overall=True)
t.save()
I'll leave it unsolved if there is a better solution.