Django M2M with a large table - django

I have a typical M2M scenario where promotion activities are related to our retailers. However we have a large number of retailers (over 10k) and therefore I can't use the normal multiple select widget.
What I would aim to do is have an 'activity' instance page with a 'retailer' sub-page which would have a table listing all those retailers currently related to the activity. In addition there would be a 'delete' checkbox next to each retailer so they could be removed from the list if necessary. (Naturally, I would also have another search/results page where users could select which retailers they want to add to the list, but I'm sure I can sort that out myself).
Could someone point me in the right direction regarding modelforms and formset factories as I'm not sure where to go from here. It would seem obvious to directly manipulate the app_activity_associated_retailers table but I don't think I can do this with the existing functions. Is there was a pattern for doing this.
class Activity(models.Model):
budget = models.ForeignKey('Budget')
activity_type = models.ForeignKey('ActivityType')
amount = models.DecimalField(max_digits=8, decimal_places=2)
associated_retailers = models.ManyToManyField('Retailer', related_name='associated_retailers')
class Retailer(models.Model):
name = models.CharField(max_length=50)
address01 = models.CharField(max_length=50)
address02 = models.CharField(max_length=50, blank=True)
postcode = models.CharField(max_length=5)
city = models.CharField(max_length=20)

All ManyToManyFields have a through model, whether you define one yourself or not. In your case, it'll have an id, an activity field and a retailer field. You can access the table with Activity.associated_retailers.through -- one "obvious" way is to just expose it as a "model" like
ActivityRetailer = Activity.associated_retailers.through
You can now manipulate these relationships like they were any ol' Django model, so you can i.e. generate querysets like
retailer_records_for_activity = ActivityRetailer.objects.filter(activity_id=1234)
... and you can also create model formsets (complete with that delete checkbox if so configured) for these pseudo-models.

Related

Django - Generic One-to-One Relation

I have a website with different kinds of activities:
Lessons ;
Exercises ;
Quizzes.
So far, each type of activity corresponds to a specific Model and a table in the database. I would like to create an ordered path through these activities. For instance:
Lesson 1
then Exercise 1
then Lesson 2
then Quizz 1
etc.
I am considering to create a model named Activity that would store the following data:
A number: the number of the activity in the path ;
A One-To-One relationship to one given activity (lesson, exercise, quizz etc.).
(1) I have seen that Django offers a GenericForeignKey to handle many-to-one relationship to different kinds of models, like many comments associated to a single lesson or a single exercise. Is there something similar for Generic OneToOne relationship?
(2) I would like to track the progress of my users. To do so, I am considering having a many-to-many relationship called "done_activities" built in my User model and linked to my Activity model. Do you think this is an efficient way of approaching the problem ?
I'm not sure you would need or want self-referential fields in this case. Consider the following structure as an example. I do not propose this 'in stone' as THE solution, but more to spur your own ideas about the solution you need. Please note that I'm leaving out __str__ methods and such for brevity:
ACTIVITY_TYPE = ['Lesson', 'Exercise', 'Quiz']
class Activity(models.Model):
activity_name = models.CharField(max_length=200, blank=True)
activity_type = models.CharField(max_length=100, choices=ACTIVITY_TYPE, blank=True, db_index=True)
activity_desc = models.CharField(max_length=200, blank=True) #description of the lesson, exercise, or quiz
class Program(models.Model):
program_name = models.CharField(max_length=200, blank=True, db_index=True)
description = models.Charfield(max_length=200, blank=True)
class ProgramActivity(models.Model):
program = models.ForeignKey(Program, on_delete=models.CASCADE, your args etc...)
activity = models.ForeignKey(Activity, on_delete=models.CASCADE, your args here etc...)
path_order = models.PositiveSmallIntegerField() # stores a number for the order in the program path
class UserProgram(models.Model):
student = models.ForeignKey(User, etc...) # FK to the user
program = models.ForeignKey(Program, etc...) # connects users to programs
progress = models.DecimalField(max_digits=5, decimal_places=2) # stores percentage of program completed (for example)
In this schema scenario, the following are true:
Activitys are all tracked and stored together on a single table,
organized by type, and each one can have its own description.
Programs are stored on their own model, and
represent a named object that unites all their constituent activities.
ProgramActivity connects activities to specific programs, and
allows you to set the order in the path for that activity relative to the program, and change it easily if you have to. You can easily query activities = ProgramActivity.objects.filter(program=some_program).order_by('path_order') and get a very usable list of a Program's activities.
Finally the UserProgram model records a User's "enrollments" and
progress in each, in this example, by percentage of the program completed.
This is just one possible approach. You may, for example, want to create an activity type table instead of a list dropdown, which may be a more robust way of managing activities over time.

Django: sub-fields within a model field

In my primary class model Deals, I have certain fields as description, price, date_created etc. I now have to add some fields having sub-fields to it. For eg, I'm trying to add an age field to Deals. This age field further has subfields (like score_for_kid, score_for_baby, score_for_old etc), and I want to edit these scores from the admin.
Here is my models.py:
class Deals(models.Model):
description = models.TextField()
price = models.DecimalField(max_digits=7, decimal_places=2)
url = models.URLField(verify_exists=False)
currency = models.CharField(max_length=3)
created_date = models.DateField(auto_now_add=True)
kid_score = models.IntegerField(max_length=2,default=0)
teenager_score = models.IntegerField(max_length=2,default=0)
youth_score = models.IntegerField(max_length=2,default=0)
old_score = models.IntegerField(max_length=2,default=0)
I don't want to store all these sub fields (around 20-25 in 4 different fields) in the model, instead an age field connected to these subfields. Would a ManyToManyField work for this?
The underlying requirement is that when a user selects a subfield (say kids) on the browser, all the objects having higher kid scores are displayed.
I'm very new to Django and any help on this would be great. Thanks.
If I understand your question properly ou need to use ForeignKey fields.
class Deals(models.Model):
description = models.TextField()
price = models.DecimalField(max_digits=7, decimal_places=2)
#...
age = models.ForeignKey(Age)
class Age(models.Model):
kid_score = models.IntegerField(max_length=2,default=0)
teenager_score = models.IntegerField(max_length=2,default=0)
#...
Have a good read of the docs on Models. You might also find it useful to do some reading on relational databases / basic sql.
When you come to edit your objects in the django admin, you'll probably want to use an InlineModelAdmin class.
UPDATE
re-reading your question, it sounds like you might simply want to show / hide these additional fields on the main Deal model. If this is the case then you want to use fieldsets in the admin, with a class 'collapse'. There's an example in the docs.
If you want each Deal record to have multiple kid_score's associated with it then you want a foreign key. If each Deal can only have one kid_score then you need to keep the kid_score (and other) fields in the main model (if this is confusing then definitely do some reading on sql / relational databases).

What is the best way to model a heterogenous many-to-many relationship in Django?

I've searched around for a while, but can't seem to find an existing question for this (although it could be an issue of not knowing terminology).
I'm new to Django, and have been attempting to take a design which should be very expandable over time, and make it work with Django's ORM. Essentially, it's a series of many-to-many relationships using a shared junction table.
The design is a generic game crafting system, which says "if you meet [require], you can create [reward] using [cost] as materials." This allows items to be sold from any number of shops using the same system, and is generic enough to support a wide range of mechanics - I've seen it used successfully in the past.
Django doesn't support multiple M2M relationships sharing the same junction table (apparently since it has no way to work out the reverse relationship), so I seem to have these options:
Let it create its own junction tables, which ends up being six or more, or
Use foreign keys to the junction table in place of a built-in MTM relationship.
The first option is a bit of a mess, since I know I'll eventually have to add additional fields into the junction tables. The second option works pretty well. Unfortunately, because there is no foreign key from the junction table BACK to each of the other tables, I'm constantly fighting the admin system to get it to do what I want.
Here are the affected models:
class Craft(models.Model):
name = models.CharField(max_length=30)
description = models.CharField(max_length=300, blank=True)
cost = models.ForeignKey('Container', related_name="craft_cost")
reward = models.ForeignKey('Container', related_name="craft_reward")
require = models.ForeignKey('Container', related_name="craft_require")
class ShopContent(models.Model):
shopId = models.ForeignKey(Shop)
cost = models.ForeignKey('Container', related_name="shop_cost")
reward = models.ForeignKey('Container', related_name="shop_reward")
require = models.ForeignKey('Container', related_name="shop_require")
description = models.CharField(max_length=300)
class Container(models.Model):
name = models.CharField(max_length=30)
class ContainerContent(models.Model):
containerId = models.ForeignKey(Container, verbose_name="Container")
itemId = models.ForeignKey(Item, verbose_name="Item")
itemMin = models.PositiveSmallIntegerField(verbose_name=u"min amount")
itemMax = models.PositiveSmallIntegerField(verbose_name=u"max amount")
weight = models.PositiveSmallIntegerField(null=True, blank=True)
optionGroup = models.PositiveSmallIntegerField(null=True, blank=True,
verbose_name=u"option group")
Is there a simpler, likely obvious way to get this working? I'm attempting to allow inline editing of ContainerContent information from each related column on the Craft edit interface.
It sounds like you have a sort of "Transaction" that has a name, description, and type, and defines a cost, reward, and requirement. You should define that as a single model, not multiple ones (ShopContent, Craft, etc.).
class Transaction(models.Model):
TYPE_CHOICES = (('Craft', 0),
('Purchase', 1),
)
name = models.CharField(max_length=30)
description = models.CharField(max_length=300, blank=True)
cost = models.ForeignKey('Container')
reward = models.ForeignKey('Container')
require = models.ForeignKey('Container')
type = models.IntegerField(choices = TYPE_CHOICES)
Now Shop etc. can have a single ManyToManyField to Transaction.
Whether or not you use this particular model, the cost, reward and require relationships should all be in one place -- as above, or in OneToOne relationships with Craft, ShopContent etc. As you guessed, you shouldn't have a whole bunch of complex Many-To-Many through tables that are all really the same.
You mention at the bottom of your post that you're
attempting to allow inline editing of ContainerContent information from each related column on the Craft edit interface.
If you're modeling several levels of relationship, and using the admin app, you'll need to either apply some sort of nested inline patch, or use some sort of linking scheme like the one I use in my recent question, How do I add a link from the Django admin page of one object to the admin page of a related object?
I am smelling something is too complicated here, but I might be wrong. As a start,
is this any better? (ContainerContent will be figured out later)
class Cost(models.Model):
name = models.CharField(max_length=30)
class Reward(models.Model):
name = models.CharField(max_length=30)
class Require(models.Model):
name = models.CharField(max_length=30)
class Craft(models.Model):
name = models.CharField(max_length=30)
description = models.CharField(max_length=300, blank=True)
cost = models.ForeignKey(Cost)
reward = models.ForeignKey(Reward)
require = models.ForeignKey(Require)
class Shop(models.Model):
name = models.CharField(max_length=30)
crafts = models.ManyToMany(Craft, blank=True)

Django query with distinct and order_by

I have two models
class Employer(models.Model):
name = models.CharField(max_length=300, blank=False)
id = models.IntegerField()
status = models.IntegerField()
eminence = models.IntegerField(null=False,default=4)
class JobTitle(models.Model)
name = models.CharField(max_length=300, blank=False)
employer = models.ForeignKey(Employer,unique=False,null=True)
activatedate = models.DateTimeField(default=datetime.datetime.now)
I need all employers in the order of whose jobtitle activated last.
Employer.objects.filter(status=1).order_by('eminence','-jobtitle__activatedate')
This query gives me what I want but it returns repeated employers if employer has more than one jobtitle.
I would use distinct() but in Django documents I found that
*Any fields used in an order_by() call are included in the SQL SELECT columns. This can sometimes lead to unexpected results when used in conjunction with distinct(). If you order by fields from a related model, those fields will be added to the selected columns and they may make otherwise duplicate rows appear to be distinct. Since the extra columns don't appear in the returned results (they are only there to support ordering), it sometimes looks like non-distinct results are being returned.*
Although they explained my problem no solution is specified.
Could give me a suggestion how can I group by my employer list without corrupting the API stability?
You can place on the Employer class its latest JobTitle activation date, and then order by this field without using relations.[1] The tradeoff here is a bit of data duplication, and the necessity to update employer's latest job title activation date manually when corresponding JobTitle instance changes.
Also, check this post for another solution which uses annotate().
Related questions:
[1] Queryset API distinct() does not work?

Showing partial tabular inline models in Django

Let's see if I can explain myself, I have this models:
class BillHeader(models.Model):
number = models.CharField(_('Bill number'), max_length=10, unique=True, \
default=__number)
client = models.ForeignKey(ClienteYProveedor, verbose_name=_('Client'))
date = models.DateTimeField(_('Date'), default=datetime.now)
def __unicode__(self):
return str(self.number)
class Meta:
abstract = True
class BillFooter(models.Model):
base_import = models.DecimalField(_('Base import'), max_digits=12, \
decimal_places=2)
class Meta:
abstract = True
class BillBody(models.Model):
description = models.CharField(_('Description'), max_length=200)
amount = models.DecimalField(_('Amount'), max_digits=6, decimal_places=2)
discount = models.DecimalField(_('Discount'), max_digits=4, \
decimal_places=2)
price = models.DecimalField(_('Price'), max_digits=12, decimal_places=2)
unitaryprice = models.DecimalField(_('Unitary Price'), max_digits=12, \
decimal_places=2)
def __unicode__(self):
return self.description
class Meta:
abstract = True
class EmittedBill(BillHeader, BillBody, BillFooter):
pass
class ReceivedBill(BillHeader, BillBody, BillFooter):
pass
When the user adds an Emmited or Received bill I need to show BillHeader as a normal fieldset, but BillBody and BillFooter need to be TabularInline.
If I put those as TabularInline in admin.py, an error rises saying that they need a ForeignKey to the related models. Of course, I can't put those foreign keys, because they are declared at the bottom. I think you guys call this "backwards foreign keys".
My question is this: how can I do this to show TabularInlines in the admin without making a mess?. I can do it without abstract base classes, but then another problem comes, it shows the other ForeignKey in the TabularInline (if you are on EmmitedBills it shows the FK to ReceivedBills in the TabularInline and viceversa) and I couldn't been able to exclude them.
Sorry if this is a stupid question, but I'm not a programmer (it's just a hobby) and I'm really making myself a mess with data models.
I'll explain better:
I have two types of bills, Emitted and Received and both of them show on the admin home (that's why I didn't use a BooleanField to mark them). Both types have the same fields except one, the bill number, which in Emmitted will be autogenerated. Each bill consists on 1 header with number, client and date, 1 or more body inline entries with a description, amount, price, etc. and 1 inline footer, showing the total price without taxes, applied taxes, etc.
Update
I have done everything, but I have a problem, in my new model BillBody has two FK's (EmmitedBill and ReceivedBill) and they show up in the TabularInline. How can I hide them?field.exclude() gives an error.
I don't fully understand your question, but you can use
ForeignKey('ModelName')
instead of
ForeignKey(ModelName)
if the ModelName model isn't already declared. Maybe this solves your problem.
Inline admins (like TabularInline) are only used when you have a one-to-many relation, which is created by a ForeignKey on the many side. If you don't have such a foreign key, then you cannot use a inline admin. Inheritance is definitely different from a ForeignKey.
However, I think your data model is wrong. It seems like you do want to store bills. There are two types of bills, emitted and received bills. Both emitted and received bills do have the same fields. Furthermore, you want that each bill consists of a header with number, client and date, 1 or more body entries, where each entry stores the information you store in BillBody and 1 or more decimals base_number.
A probably better data model for you
class Bill(models.Model):
number = models.CharField(_('Bill number'), max_length=10, unique=True, default=__number)
client = models.ForeignKey(ClienteYProveedor, verbose_name=_('Client'))
date = models.DateTimeField(_('Date'), default=datetime.now)
def get_total_price(self):
return sum([entry.price for entry in self.entries])
class BillEntry(models.Model):
bill = models.ForeignKey(Bill, related_name='entries')
description = models.CharField(_('Description'), max_length=200)
amount = models.DecimalField(_('Amount'), max_digits=6, decimal_places=2)
discount = models.DecimalField(_('Discount'), max_digits=4, decimal_places=2)
price = models.DecimalField(_('Price'), max_digits=12, decimal_places=2)
unitaryprice = models.DecimalField(_('Unitary Price'), max_digits=12, decimal_places=2)
I have left out the __unicode__ methods.
Now you have a foreign key from BillEntry to Bill and you can use a tabular inline. I didn't understand your usage of base_import so I left this out.
Price computation
If your price should always equal something like amount*unitaryprice - discount or amount*(unitaryprice-discount) then you shouldn't put this in a field but either compute it when it is needed, either in Python or in the database. If you want to do this in Python you can use a method similar to get_total_price. If you want to compute them when querying the database then it is a little bit more difficult to get it working with Django.
In the last case, you can have a look at SQL views, but I think this is a little bit too difficult for a beginner. Another option is to use a custom SQL expression:
BillEntry.objects.extra(select={'price': 'amount*unitaryprice-discount'})
This will compute the price for all entries during selection.
Update
If you add two subclasses for emitted and received bills and use multi table inheritance then you can use one foreign key from BillEntry to Bill.
class EmittedBill(Bill):
pass
class ReceivedBill(Bill):
pass
You probably also have to think about the database model generated by Django. Usually, you only want to store elementary data in the database, and not computed data (like you want to do in your footer). So, if prices are computed using some formula and using the unitaryprice, amount etc. you shouldn't store the result of this formula but recompute it when necessary (and eventually cache to avoid re-computations). If you don't do this, chances are that you at some moment update something (for example the amount) and forget to update the computed values (the price) which leads to inconsistencies in your database (and thus bugs in your application). A good database does have constraints so that it is impossible to store inconsistent database without breaking at least one constraint.
I also don't see why you want a separate header and footer per bill. A model isn't the real bill, it stores the information for a bill. If you want to have a visible header and footer, then you should do this in your view layer (the template) and not in the model itself.