I have an existing working site, that takes advantage of Adobe Lightroom's catalog being an sqlite3 db. The model is auto-generated.
Whilst updating the code for the site (to TemplateView and streamlining), I thought I'd get adventurous and pull on one of the class models to get the information about how many times a face (person) appears in the photos.
The model:
class Aglibrarykeywordpopularity(models.Model):
id_local = models.IntegerField(primary_key=True, blank=True, null=False)
occurrences = models.TextField() # This field type is a guess.
popularity = models.TextField() # This field type is a guess.
tag = models.TextField(unique=True) # This field type is a guess.
class Meta:
managed = False
db_table = 'AgLibraryKeywordPopularity'
The 'tag' value is obtainable from another model, where it's called 'id_local':
face_id = list(Aglibrarykeyword.objects.values_list('id_local', flat=True))
The values for the 'face_id' are all present in the previous 'tag' field when you view the db.
However, when using:
for a in face_id:
face_pop.append(Aglibrarykeywordpopularity.objects.get(tag=a).occurrences)
.. it complains that there are no matches. I've substituted a for a number, or string of the number, even adding .0 to the representation, and it still complains no matches. I've tried .filter...first() etc.
slides.models.DoesNotExist: Aglibrarykeywordpopularity matching query does not exist
My current work-around is going to output All() of the objects to lists and obtain the values I want that way - which works, but I'm interested more in why it's not working as it should.
Something I notice is that the model has tag defined as TextField, and when I obtain the values myself from All() they are float numbers. Is it the case that there's a sort of 3 way mismatching of variable types, 1)the model 2)the data 3)the query - so no matter if I put a float number or text in my query, it will never match? i.e. do I have to change the model, migrate it and all should be well? (as I'm no Django expert, does the "managed = False" mean that even if I do this, it will not effect the db data?)
TIA for comments.
Related
few years ego I worked with Odoo framework. and Odoo has very nice feature like this:
partner_id = field.Many2one(Partner)
partner_name = fields.Char(string='Partner name', related='partner_id.name')
basically whenever you would assign different partner_id from Partner table, partner_name would be assigned automatically. Now I started to work with django (absolute newbie), and I can't seem to find a similar functionality.
My question is what could be possible solution for this problem. Maybe there are already established external libraries that has this sort of functionality?
Expected result:
product = models.ForeignKey(Product)
product_color = models.CharField(string='Partner name', related='product.color')
having in mind that product object would have color field and it would be assigned to product_color whenever product field value Product object color value changes. Also what about storing it to database? Would be nice if there was an option to chose between storing it in database or getting it on the fly.
Cheers!
Creating a getter is pretty easy, because you can simply have functions in a Python object behave as a property:
class SampleModel(models.Model):
product = models.ForeignKey(Product)
#property
def product_color(self):
return self.product.color
This does retrieve the property on the fly, which will cause a call to the database.
Duplicating data, is usually a (more severe) antipattern. Synchronizing data, even in two tables in the same database, often turns out harder than one might expect. Even if you would use Django's signal framework for example, then some Django ORM calls can circumvent that (for example .update(..) [Django-doc]). But even if you somehow would cover those cases, then another program that talks to the database could update one of the two fields.
Most databases have triggers that can help. But again, the number of cases to cover are often larger than expected. For example, if the Product that we refer to is removed, then or the foreign key now points to a different Product, then we will need to update that field.
Therefore it is often better, to fetch the name of the related product when we need it. We can do so by (a) defining a property; or (b) make an annotation, for example in the manager.
Defining a property
We can define a property that will load the related product, and fetch the related name, like:
class Order(models.Model):
product = models.ForeignKey(Product, on_delete=models.PROTECT)
#property
def product_name(self):
return self.product.name
Then we can fetch the product name with some_order.product_name. This might not be very efficient if we need to fetch it often, since the relations are, by default, loaded lazily in Django, and thus can result in an N+1 problem.
Annotate the queryset
We can make an annotation that will fetch the name of the product in the same query when we fetch the Order, for example:
from django.db.models import F
class OrderManager(models.Manager):
def get_queryset(self):
return super().get_queryset().annotate(
product_name=F('product__name')
)
class Order(models.Model):
product = models.ForeignKey(Product, on_delete=models.PROTECT)
objects = OrderManager()
Then if we fetch an order. For example with Order.objects.get(pk=1), then that Order object will have an attribute product_name with the name of the product.
I'm building a basic time logging app right now and I have a todo model that uses django-taggit. My Todo model looks like this:
class Todo(models.Model):
project = models.ForeignKey(Project)
description = models.CharField(max_length=300)
is_done = models.BooleanField(default=False)
billable = models.BooleanField(default=True)
date_completed = models.DateTimeField(blank=True, null=True)
completed_by = models.ForeignKey(User, blank=True, null=True)
tags = TaggableManager()
def __unicode__(self):
return self.description
I'm trying to get a list of unique tags for all the Todos in a project and I have managed to get this to work using a set comprehension, however for every Todo in the project I have to query the database to get the tags. My set comprehension is:
unique_tags = { tag.name.lower() for todo in project.todo_set.all() for tag in todo.tags.all() }
This works just fine, however for every todo in the project it runs a separate query to grab all the tags. I was wondering if there is any way I can do something similar to prefetch_related in order to avoid these duplicate queries:
unique_tags = { tag.name.lower() for todo in project.todo_set.all().prefetch_related('tags') for tag in todo.tags.all() }
Running the previous code gives me the error:
'tags' does not resolve to a item that supports prefetching - this is an invalid parameter to prefetch_related().
I did see that someone asked a very similar question here: Optimize django query to pull foreign key and django-taggit relationship however it doesn't look like it ever got a definite answer. I was hoping someone could help me out. Thanks!
Taggit now supports prefetch_related directly on tag fields (in version 0.11.0 and later, released 2013-11-25).
This feature was introduced in this pull request. In the test case for it, notice that after prefetching tags using .prefetch_related('tags'), there are 0 additional queries for listing the tags.
Slightly hackish soution:
ct = ContentType.objects.get_for_model(Todo)
todo_pks = [each.pk for each in project.todo_set.all()]
tagged_items = TaggedItem.objects.filter(content_type=ct, object_id__in=todo_pks) #only one db query
unique_tags = set([each.tag for each in tagged_items])
Explanation
I say it is hackish because we had to use TaggedItem and ContentType which taggit uses internally.
Taggit doesn't provide any method for your particular use case. The reason is because it is generic. The intention for taggit is that any instance of any model can be tagged. So, it makes use of ContentType and GenericForeignKey for that.
The models used internally in taggit are Tag and TaggedItem. Model Tag only contains the string representation of the tag. TaggedItem is the model which is used to associate these tags with any object. Since the tags should be associatable with any object, TaggedItem uses model ContentType.
The apis provided by taggit like tags.all(), tags.add() etc internally make use of TaggedItem and filters on this model to give you the tags for a particular instance.
Since, your requirement is to get all the tags for a particular list of objects we had to make use of the internal classes used by taggit.
Use django-tagging and method usage_for_model
def usage_for_model(self, model, counts=False, min_count=None, filters=None):
"""
Obtain a list of tags associated with instances of the given
Model class.
If ``counts`` is True, a ``count`` attribute will be added to
each tag, indicating how many times it has been used against
the Model class in question.
If ``min_count`` is given, only tags which have a ``count``
greater than or equal to ``min_count`` will be returned.
Passing a value for ``min_count`` implies ``counts=True``.
To limit the tags (and counts, if specified) returned to those
used by a subset of the Model's instances, pass a dictionary
of field lookups to be applied to the given Model as the
``filters`` argument.
"""
A slightly less hackish answer than akshar's, but only slightly...
You can use prefetch_related as long as you traverse the tagged_item relations yourself, using the clause prefetch_related('tagged_items__tag'). Unfortunately, todo.tags.all() won't take advantage of that prefetch - the 'tags' manager will still end up doing its own query - so you have to step over the tagged_items relation there too. This should do the job:
unique_tags = { tagged_item.tag.name.lower()
for todo in project.todo_set.all().prefetch_related('tagged_items__tag')
for tagged_item in todo.tagged_items.all() }
I am gradually replacing a legacy database front end with a django based system. All models are Managed = False, to preserve the original db structure.
However I have struck a problem where a field in a table is computed. The field is defined in (pseudo) sql as full_name = fname|| ||lname.
I can define the full_name field as a charfield; and I can read it no problems, however any attempt to update the record results in an update error on that field.
I can use a #property; but that replicates the functionality in django, rather than displaying the results from the db itself. Using this approach will cause problems with more complex fields (in tables I am yet to get to) that are defined with UDFs.
What is really needed is a 'read_only' or 'computed' attribute on the model itself; what is the best way to achieve this?
Do you just want to define a method in your class? Something like:
def Person(models.Model):
fname=models.CharField(...)
lname=models.CharField(...)
def fullname(self):
return self.fname+" "+self.lname
(not exactly sure what Managed=False means...)
if you are trying to make calculation on a database models and pass the value of a model field to another model field of the same class model, using a defined function then this solution might help you. for example lets assume you have an investment company and you give 20% per month for the capital each user invested, you would want want to pass value from capital model to a function that calculates the percentage interest, and then you will pass that function into another field monthly_payment and get saved in the database.
1) pip install django-computed-property
2) add 'computed_property' to your installed apps in project settings.
3) in your models.py, import computed_property then
class Investment(models.Model):
name = models.CharField(max_length=200)
capital = models.FloatField(null=False)
percentage = models.CharField(max_length=5)
duration = models.CharField(max_length=10, default="1 months")
monthly_payment = computed_property.ComputedFloatField( compute_from='monthly_percentage', null=False, blank=False)
then your function to perform the calculation will go thus
#property
def monthly_percentage(self):
return (20 / 100) * self.capital
Note: what i discovered was if you use the inbuilt django fields be it FloatFiled or IntegerField, this function won't read the amount you are passing in to get your 20% calculations.i hope this works for you as i stated all as they worked for me,cheers.
I posted this question on the django-users list, but haven't had a reply there yet.
I have models that look something like this:
class ProductGroup(models.Model):
name = models.CharField(max_length=10, primary_key=True)
def __unicode__(self): return self.name
class ProductRun(models.Model):
date = models.DateField(primary_key=True)
def __unicode__(self): return self.date.isoformat()
class CatalogItem(models.Model):
cid = models.CharField(max_length=25, primary_key=True)
group = models.ForeignKey(ProductGroup)
run = models.ForeignKey(ProductRun)
pnumber = models.IntegerField()
def __unicode__(self): return self.cid
class Meta:
unique_together = ('group', 'run', 'pnumber')
class Transaction(models.Model):
timestamp = models.DateTimeField()
user = models.ForeignKey(User)
item = models.ForeignKey(CatalogItem)
quantity = models.IntegerField()
price = models.FloatField()
Let's say there are about 10 ProductGroups and 10-20 relevant
ProductRuns at any given time. Each group has 20-200 distinct
product numbers (pnumber), so there are at least a few thousand
CatalogItems.
I am working on formsets for the Transaction model. Instead of a
single select menu with the several thousand CatalogItems for the
ForeignKey field, I want to substitute three drop-down menus, for
group, run, and pnumber, which uniquely identify the CatalogItem.
I'd also like to limit the choices in the second two drop-downs to
those runs and pnumbers which are available for the currently
selected product group (I can update them via AJAX if the user
changes the product group, but it's important that the initial page
load as described without relying on AJAX).
What's the best way to do this?
As a point of departure, here's what I've tried/considered so far:
My first approach was to exclude the item foreign key field from the
form, add the substitute dropdowns by overriding the add_fields
method of the formset, and then extract the data and populate the
fields manually on the model instances before saving them. It's
straightforward and pretty simple, but it's not very reusable and I
don't think it is the right way to do this.
My second approach was to create a new field which inherits both
MultiValueField and ModelChoiceField, and a corresponding
MultiWidget subclass. This seems like the right approach. As
Malcolm Tredinnick put it in
a django-users discussion,
"the 'smarts' of a field lie in the Field class."
The problem I'm having is when/where to fetch the lists of choices
from the db. The code I have now does it in the Field's __init__,
but that means I have to know which ProductGroup I'm dealing with
before I can even define the Form class, since I have to instantiate the
Field when I define the form. So I have a factory
function which I call at the last minute from my view--after I know
what CatalogItems I have and which product group they're in--to
create form/formset classes and instantiate them. It works, but I
wonder if there's a better way. After all, the field should be
able to determine the correct choices much later on, once it knows
its current value.
Another problem is that my implementation limits the entire formset
to transactions relating to (CatalogItems from) a single
ProductGroup.
A third possibility I'm entertaining is to put it all in the Widget
class. Once I have the related model instance, or the cid, or
whatever the widget is given, I can get the ProductGroup and
construct the drop-downs. This would solve the issues with my
second approach, but doesn't seem like the right approach.
One way of setting field choices of a form in a formset is in the form's __init__ method by overwriting the self.fields['field_name'].choices, but since a more dynamic approach is desired, here is what works in a view:
from django.forms.models import modelformset_factory
user_choices = [(1, 'something'), (2, 'something_else')] # some basic choices
PurchaserChoiceFormSet = modelformset_factory(PurchaserChoice, form=PurchaserChoiceForm, extra=5, max_num=5)
my_formset = PurchaserChoiceFormSet(self.request.POST or None, queryset=worksheet_choices)
# and now for the magical for loop
for choice_form in my_formset:
choice_form.fields['model'].choices = user_choices
I wasn't able to find the answer for this but tried it out and it works in Django 1.6.5. I figured it out since formsets and for loops seem to go so well together :)
I ended up sticking with the second approach, but I'm convinced now that it was the Short Way That Was Very Long. I had to dig around a bit in the ModelForm and FormField innards, and IMO the complexity outweighs the minimal benefits.
What I wrote in the question about the first approach, "It's straightforward and pretty simple," should have been the tip-off.
I've essentially got two tables: Page(PK=url) and PageProperty(PK=url+name).
Here is how I have my Models set up:
class Page(model.Model):
url = model.CharField(primary_key=True, max_length=255, db_column='url')
#.....
class PageProperty(model.Model):
# table with compound key (url + name)
url = model.ForeignKey('Page', to_field='url', db_column='url', primary_key=True)
name = model.CharField(primary_key=True, max_length=20)
value = model.TextField()
I have a ModelAdmin set up so I can Inline edit PageProperty(s) from Page. Its a legacy database and I know there's a lot of data in there. But the Admin is only showing ONE of the PagePropertys, not all.
I think you might need to apply the extra option to your TabularInline. Example:
class PagePropertyInline(admin.TabularInline):
model = PageProperty
extra = 3
You could probably do some magic to make the amount of extra items dynamic (such as the number of PageProperty objects for a given Page, but I'll leave that up to you.
I would suggest further reading on InlineModelAdmin options and Formsets.
Because it felt as thought a non-integer primary key was too much against the grain, I ended up buckling down and migrating the schema to use an auto generated integer pk for both tables. After that everything was smooth sailing again and the Inlines worked perfectly.