Can someone give me the best approach with an example for the following...
On a page I load the 'Group' object by ID. I also want to list all contacts that belong to that group (with paging).
Because of the paging issue I was thinking of just running a second database query with...
In my view...
group = get_object_or_404(Group, pk=id)
contacts = Contacts.objects.filter(group=x)
But this seems wasteful as I'm already getting the Group why hit the database twice.
See my model.
class GroupManager(models.Manager):
def for_user(self, user):
return self.get_query_set().filter(user=user,)
class Group(models.Model):
name = models.CharField(max_length=60)
modified = models.DateTimeField(null=True, auto_now=True,)
#FK
user = models.ForeignKey(User, related_name="user")
objects = GroupManager()
def get_absolute_url(self):
return reverse('contacts.views.group', args=[str(self.id)])
class Contact(models.Model):
first_name = models.CharField(max_length=60)
last_name = models.CharField(max_length=60)
#FK
group = models.ForeignKey(Group)
This is what select_related is designed for:
Returns a QuerySet that will automatically “follow” foreign-key
relationships, selecting that additional related-object data when it
executes its query. This is a performance booster which results in
(sometimes much) larger queries but means later use of foreign-key
relationships won’t require database queries.
In your case it would be:
Group.objects.select_related().get(pk=group)
Now on each FK lookup, you won't hit the database again.
The next step would be to cache the results using the cache api so that you don't hit the database everytime the next "page" is called. This would be useful if your data isn't time sensitive.
Related
I am working with Django on a database that has additional fields on intermediate models. Since it's a big database, I try to optimize the way the data is loaded. But I have a problem with the extra fields of the association table.
Let's take this example from Django's documentation :
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=128)
def __str__(self):
return self.name
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person, through='Membership')
def __str__(self):
return self.name
class Membership(models.Model):
person = models.ForeignKey(Person, on_delete=models.CASCADE)
group = models.ForeignKey(Group, on_delete=models.CASCADE)
date_joined = models.DateField()
invite_reason = models.CharField(max_length=64)
I would like to retrieve, from each entity of the Group class, all the entities of the Person class and all the fields invite_reason or date_joined.
To retrieve the persons, it goes fast with the QuerySet.prefetch_related attribute that prevents the deluge of database queries that is caused by accessing related objects.
groups = Group.objects.prefetch_related('members')
However, I did not find a solution to retrieve in a constant access time the extra fields invite_reason and date_joined.
I tried prefetching membership_set or a related name in my variable groups but my code doesn't go faster.
# NOT WORKING
groups = Group.objects.prefetch_related('members', 'membership_set')
I also tried using a Prefetch object with a queryset parameter using select_related but it didn't work. Everything I've tried to load all the Membership data into groups at initialization has failed and I end up having a very long runtime retrieving the extra fields from the table.
# TAKES A WHILE BECAUSE NOTHING IS PREFETCHED
for group in groups:
invite_reason_list = group.membership_set.values_list('invite_reason', flat=True)
date_joined_list = group.membership_set.values_list('date_joined', flat=True)
How do I stop the deluge of database queries that is caused by accessing related objects?
When you don't write related_name.all() on prefetching, it does not work as expected. you can get the data like this:
prefetch_membership_set = models.Prefetch('membership_set',
Membership.objects.only(
'date_joined', 'invite_reason'))
groups = Group.objects.prefetch_related(prefetch_membership_set)
for group in groups:
invite_reason_list = []
date_joined_list = []
for membership in group.membership_set.all():
invite_reason_list.append(
membership.invite_reason
)
date_joined_list.append(
membership.date_joined
)
I have a Loan model that has some relationships, one of which is a many to many with a Resource model (the items our borrower is loaning). I'd like to add a derived property to the loan that does a quick check on the status of the loan's various resources. But whenever I try to access the loan's resources from within this property, I just get back None. It works fine for the one to one relationships, just not many to many.
class Loan(models.Model):
borrower = models.ForeignKey('Borrower', on_delete=models.PROTECT)
resources = models.ManyToManyField('Resource', through='LoanedResource', blank=True)
start_date = models.DateTimeField()
due_date = models.DateTimeField()
#property
def is_closed(self):
print(self.borrower) # Works!
print(self.resources) # None :(
print(self.loanedresources_set) # None :(
print(LoanedResource.objects.filter(loan = self.id)) # This works, but I believe it bypasses prefetch_related, so gets really slow.
# Return my calculated value here
If this can't work, anyone have ideas on how to create derived property with many to many relationship that takes advantage of prefetch_related?
Thanks!
You can access the M2M objects as
self.resources.all()
I'm making an inventory app to control the existence of tools in my workshop. Besides knowing how many things I have, I want to know where things are (what tool cart the tool is in ) and who owns the tool cart (Employee). I also need to keep a record of all damaged tools. I've been going about this in the following way:
1.- I have a model called Item that has all common filed for all tools, then I create a new model per tool type with specific field for each tool type i.e.(end-mill-cutters, drill-bits, screws, etc ). these tool Type models all inherit from Item as Multi-table inheritance.
2.- I made the models for my tools carts and its called Carritos( in spanish) this table has a One To One relation ship to Employees( since a carrito can be owned by one person only). It also has a Many To Many relationship to my Item table trough a secondary model called Transaccion, this model handles make the relation between Carrito and Items
this is the Carritos model
class Carritos(models.Model):
no_carrito = models.CharField(max_length=3, unique=True)
empleado = models.OneToOneField(Empleados, on_delete=models.CASCADE)
# empleado = models.ManyToManyField(Empleados, through='Transaccion')
items = models.ManyToManyField(Item, through='Transaccion', related_name='carritos')
f_creacion = models.DateTimeField(auto_now_add=True)
f_actualizacion = models.DateTimeField(auto_now=True)
activo = models.BooleanField(default=True)
def get_absolute_url(self):
return reverse('inventario:carrito')#, kwargs={'pk': self.pk})
class Meta:
verbose_name_plural = "Carritos"
def __str__(self):
return self.no_carrito
class Transaccion(models.Model):
carrito = models.ForeignKey(Carritos, on_delete=models.CASCADE, related_name='items_carrito')
herramienta = models.ForeignKey(Item, on_delete=models.CASCADE, related_name='items_carrito')
cantidad = models.PositiveSmallIntegerField(default=1)
activo = models.BooleanField(default=True)
tipo = models.CharField(max_length=10, choices=CONSUMIBLE, blank=True, null=True)
motivo = models.CharField(max_length=10, blank=True, null=True)
def get_absolute_url(self):
return reverse('inventario:carrito')#, kwargs={'pk': self.pk})
3.- The idea I had to get the carritos logic is to get a list of existing carts in the carts main page and have a button bind to a CreateView CBV to create a new carrito if needed.
This list would also have a button bind to an UpdateView CBV in order to be able to change the employee in case the employee quits and an other button bind to a function that in theory would work as a DetailView to see all data assigned to carrito like (employee assigned to it, carrito number, and all Items in the carrito).
My intention was to be able to add an Item inside this view and have all items listed, I managed to be able to add Items and also managed to display all Items and the amount of those Items the carrito has. I had some issues on how to go about when multiple items of the same kind needed to be added to the carrito (let's say I needed to add 2 cutters exactly the same). But I figured that since I all ready had the Transaccion table, and this table tied Item to Carritos. I could use this to record every items as 1 of each and have an active field as Boolean, this way I could display and aggregate all distinct items and sum totals of every item in my view. It works for displaying the quantities.
The problem I'm currently are having, is if I want to edit a tool type and deactivate one of the items in the transaction model I always get the firs items on the list no matter how I choose to filter it.
My views for carritos creation
# =========================================================================== #
# LOGICA PARA CREAR CARRITOS
# =========================================================================== #
# ===================> Logica relacinado con Cortadores <=====================#
def home_carrito(request):
template_name = 'inventario/carrito/createcarrito.html'
model = Carritos
carritos = Carritos.objects.all()
if carritos:
return render(request, template_name, {'carritos':carritos})
else:
return render(request,template_name)
class CarritoCreate(CreateView):
model = Carritos
fields = [
'no_carrito',
'empleado',
'activo',
]
class ItemCreate(CreateView):
model = Transaccion
fields = [
'carrito',
'herramienta',
]
def detalle_carrito(request, pk):
model = Carritos, Transaccion
template_name = 'inventario/carrito/detalles_carrito.html'
carritos = Carritos.objects.filter(pk=pk)
# GEST ALL TOOLS ASSIGNE TO CARRITO'S PK THAT ARE ACTIVE
# TRY TO GET ALL ACTIVE ITEMS THAT BELONG TO CARRITO = PK AND AGREGATE TOTAL ITEMS PER TYPE
cantidades = Transaccion.objects.values('herramienta__description').annotate(Sum('cantidad')).filter(activo=True, carrito_id=pk)
# GEST ALL TOOLS ASSIGNE TO CARRITO'S PK THAT ARE NOT ACTIVE
eliminados = Transaccion.objects.filter(activo=False,carrito_id=pk)
return render(request,template_name, {'carrito':carritos, 'trans':cantidades, 'eliminados':eliminados})
class CarritoUpdate(UpdateView):
model = Carritos
fields = [
'no_carrito',
'empleado',
'activo',
]
template_name_suffix = '_update_form'
def ItemUpdate(UpdateView):
model = Transaccion
fields = [
'carrito',
'herramienta',
'cantidad',
'tipo',
'motivo',
'activo',
]
template_name_suffix = '_update_form'
def detalle_Items(request, pk):
model = Transaccion
template_name = 'inventario/carrito/test-template.html'
try:
items_update = Transaccion.objects.filter(activo=True, carrito_id=pk, herramienta_id=pk)
except Transaccion.DoesNotExist:
raise Http404()
return render(request, template_name, {'items_update':items_update})
So what I need in the first place is to know if what I'm doing is logical? or make sense. Scond thing I need is to know if there a better way and how?
and finally I need help resolving my issue: I need to get into an updateview for every Item in my Transaccion model and be able to disable or enable that record.
Different people would take different approaches. So far I think what you have built would absolutely work, but what becomes complicated is building the realtime inventories, which looks like its built from the transaction log. What I see as missing from the model is the snapshot of real time inventories available. Because we aren't talking about a data model that changes so frequently - like ad impressions - you can store that upon transaction vice computing it as needed.
For instance, your global inventory of hammers is 5 hammers. One employee adds a hammer to a cart. From here you articulated a couple of different use cases. One is that you need to know that employee XYZ (which implies a specific cart based on the 1-to-1) has that specific hammer. What you'd also like to know is how many hammers you have available? You may also want to understand the turnover of specific assets. Does employee XYZ maintain items in his cart longer than the average employee?
To do this I think you'd need to talk about the API layer which orchestrates that logic and the addition of another object which snapshots actual inventory instead of computing that from the transaction log. Why I bring up the API layer is that it may be a cleaner abstraction to place the logic for orchestrating multiple model changes in that than having the model itself house that logic.
So in short, I think what you've built works - but the logical expression of the use cases you've articulated are handled at the viewset/ modelviewset layer in an API. Because thats where you'll need to prep the data to be loaded into a specific format for visualization. And thats where what is easily serializable becomes the dominant force in model complexity.
I have two Models in Django. The first has the hierarchy of what job functions (positions) report to which other positions, and the second is people and what job function they hold.
class PositionHierarchy(model.Model):
pcn = models.CharField(max_length=50)
title = models.CharField(max_length=100)
level = models.CharField(max_length=25)
report_to = models.ForeignKey('PositionHierachy', null=True)
class Person(model.Model):
first_name = models.CharField(max_length=50)
last_name = models.CharField(max_length=50)
...
position = models.ForeignKey(PositionHierarchy)
When I have a Person record and I want to find the person's manager, I have to do
manager = person.position.report_to.person_set.all()[0]
# Can't use .first() because we haven't upgraded to 1.6 yet
If I'm getting people with a QuerySet, I can join (and avoid a second trip to the database) with position and report_to using Person.objects.select_related('position', 'position__reports_to').filter(...), but is there any way to avoid making another trip to the database to get the person_set? I tried adding 'position__reports_to__person_set' or just position__reports_to__person to the select_related, but that doesn't seem to change the query. Is this what prefetch_related is for?
I'd like to make a custom manager so that when I do a query to get Person records, I also get their PositionHeirarchy and their manager's Person record without more round trips to the database. This is what I have so far:
class PersonWithManagerManager(models.Manager):
def get_query_set(self):
qs = super(PersonWithManagerManager, self).get_query_set()
return qs.select_related(
'position',
'position__reports_to',
).prefetch_related(
)
Yes, that is what prefetch_related() is for. It will require an additional query, but the idea is that it will get all of the related information at once, instead of once per Person.
In your case:
qs.select_related('position__report_to')
.prefetch_related('position__report_to__person_set')
should require two queries, regardless of the number of Persons in the original query set.
Compare this example from the documentation:
>>> Restaurant.objects.select_related('best_pizza')
.prefetch_related('best_pizza__toppings')
When I build a model in Django, I add the following information:
def __unicode__(self):
return self.some_field
class Meta:
ordering = ['some_field_to_order_by']
Is it possible to set ordering based on a sort of if/else scenario?
For example, I have a customer model which, for sake of argument, has these fields:
first_name
last_name
company_name
is_company (boolean)
If the customer is a Company, I only add information to the company_name field and set is_company=True, leaving the other two blank. If the customer is a person, then I add information to the first_name and last_name fields, leaving the company_name blank and the is_company=False.
I want to sort these records by last_name if the is_company field is False and company_name if is_company is True.
Is this possible?
EDIT For an example (per request, sort of)
For my app, this customer table holds information regarding owners of security systems. Sometimes, a security system is installed in a residential setting. In this case, the owner of the system is a person -- thus, I would enter the first_name and last_name into the customer record. Sometimes the system is installed in a commercial setting, therefore the owner is a company. For this I enter only the company_name field, leaving the other fields blank.
Now, when I provide an estimate for a NEW security system installation, I can provide the estimate to a new customer or an existing customer (existing customer, but a new location for them). When it is an existing customer, I have a drop down box that lists all existing customers (ALL RECORDS in the customer table).
THIS is where I want all the records to be ordered properly. As it is now, I get a jumbled mess of hundreds of records making it brutal to find the existing owner.
Hopefully this helps with what I'm trying to achieve.
You can create a custom manager, which will do what you want.
class CompanyFilter(models.Manager):
def get_query_set(self):
qs = super(CompanyFilter, self).get_query_set()
return qs.filter(is_company=True).order_by('company_name')
class PersonFilter(models.Manager):
def get_query_set(self):
qs = super(PersonFilter, self).get_query_set()
return qs.filter(is_company=False).order_by('last_name')
class Contact(models.Model):
# ... your regular fields
companies = CompanyFilter()
people = PersonFilter()
all_companies = Contact.companies.all()
all_people = Contact.people.all()
all_contacts = Contact.objects.all()
1 . assuming that either of the fields is always NULL you can use extra query:
MyModel.objects
.extra(select=dict(name='coalesce(last_name, company_name)'))
.order_by('name')
(COALESCE seems to be supported in all major database engines)
2 . or simply eliminate either last_name or company_name and put last names and company names in to the same field and sort by it.