I have a model (lets call it Entity) that has an attribute (Attribute) that changes over time, but I want to keep a history of how that attribute changes in the database. I need to be able to filter my Entities by the current value of Attribute in its manager. But because Django (as far as I can tell) won't let me do this in one query natively, I have created a database view that produces the latest value of Attribute for every Entity. So my model structure looks something like this:
class Entity(models.Model):
def set_attribute(self, value):
self.attribute_history.create(value=value)
def is_attribute_positive(self, value):
return self.attribute.value > 0
class AttributeEntry(models.Model):
entity = models.ForeignKey(Entity, related_name='attribute_history')
value = models.IntegerField()
time = models.DateTimeField(auto_now_add=True)
class AttributeView(models.Model)
id = models.IntegerField(primary_key=True, db_column='id',
on_delete=models.DO_NOTHING)
entity = models.OneToOneField(Entity, related_name='attribute')
value = models.IntegerField()
time = models.DateTimeField()
class Meta:
managed = False
My database has the view that produces the current attribute, created with SQL like this:
CREATE VIEW myapp_attributeview AS
SELECT h1.*
FROM myapp_attributehistory h1
LEFT OUTER JOIN myapp_attributehistory h2
ON h1.entity_id = h2.entity_id
AND (h1.time < h2.time
OR h1.time = h2.time
AND h1.id < h2.id)
WHERE h2.id IS NULL;
My problem is that if I set the attribute on a model object using set_attribute() checking it with is_attribute_positive() doesn't always work, because Django may be caching that the related AttributeView object. How I can I make Django update its model, at the very least by requerying the view? Can I mark the attribute property as dirty somehow?
PS: the whole reason I'm doing this is so I can do things like Entity.objects.filter(attribute__value__exact=...).filter(...), so if someone knows an easier way to get that functionality, such an answer will be accepted, too!
I understand that the attribute value is modified by another process (maybe not even Django) accessing the same database. If this is not the case you should take a look at django-reversion.
On the other hand if that is the case, you should take a look at second answer of this. It says that commiting transaction invalidate query cache and offer this snippet.
>>> from django.db import transaction
>>> transaction.enter_transaction_management()
>>> transaction.commit() # Whenever you want to see new data
I never directly solved the problem, but I was able to sidestep it by changing is_attribute_positiive() to directly query the database table, instead of the view.
def is_attribute_positive(self, value):
return self.attribute_history.latest().value > 0
So while the view gives me the flexibility of being able to filter queries on Entity, it seems the best thing to do once the object is received is to operate directly on the table-backed model.
Related
I'm building a Django application, and in it I would like to track whenever a particular model was last accessed.
I'm opting for this in order to build a user activity history.
I know Django provides auto_now and auto_now_add, but these do not do what I want them to do. The latter tracks when a model was created, and the former tracks when it was last modified, which is different from when it was last accessed, mind you.
I've tried adding another datetime field to my model's specification:
accessed_on = models.DateTimeField()
Then I try to update the model's access manually by calling the following after each access:
model.accessed_on = datetime.utcnow()
model.save()
But it still won't work.
I've gone through the django documentation for an answer, but couldn't find one.
Help would be much appreciated.
What about creating a model with a field that contains the last save-date. Plus saving the object every time is translated from the DB representation to the python representation?
class YourModel(models.Model):
date_accessed = models.DateTimeField(auto_now=True)
#classmethod
def from_db(cls, db, field_names, values):
obj = super().from_db(db, field_names, values)
obj.save()
return obj
Suppose we have the next model:
class Publications(models.Model):
author = ..........
post = ..........
and we don't want duplicate records to be stored in the database.
This could be done with unique togheter on the model:
Meta:
unique_together = (author, post)
or it could be done in the view with something like:
register_exist = Publications.objects.filter(...).exists()
if register_exist == False:
#Code to save the info
What are the advantages or disadvantages of using these methods?
Meta:
unique_together = (author, post)
Constrain at database level. This make the data always consistent no matter what views input the data.
But the other one:
register_exist = Publications.objects.filter(...).exists()
if register_exist == False:
#Code to save the info
Constrain at application level. There might be a cost to query and check if the record is existing or not. And the data might not be consistent among the application when somebody might add new record without this step (by incident or accident), that make the data no longer consistent anymore.
In a nutshell, the unique_together attribute create a UNIQUE constraint whereas the .filter(..) is used to filter the QuerySet wrt the given conditions.
In other words, If you applied unique_together functionality in your model, you can't break that constraint (technically possible, but) even if you try to do so.
few years ego I worked with Odoo framework. and Odoo has very nice feature like this:
partner_id = field.Many2one(Partner)
partner_name = fields.Char(string='Partner name', related='partner_id.name')
basically whenever you would assign different partner_id from Partner table, partner_name would be assigned automatically. Now I started to work with django (absolute newbie), and I can't seem to find a similar functionality.
My question is what could be possible solution for this problem. Maybe there are already established external libraries that has this sort of functionality?
Expected result:
product = models.ForeignKey(Product)
product_color = models.CharField(string='Partner name', related='product.color')
having in mind that product object would have color field and it would be assigned to product_color whenever product field value Product object color value changes. Also what about storing it to database? Would be nice if there was an option to chose between storing it in database or getting it on the fly.
Cheers!
Creating a getter is pretty easy, because you can simply have functions in a Python object behave as a property:
class SampleModel(models.Model):
product = models.ForeignKey(Product)
#property
def product_color(self):
return self.product.color
This does retrieve the property on the fly, which will cause a call to the database.
Duplicating data, is usually a (more severe) antipattern. Synchronizing data, even in two tables in the same database, often turns out harder than one might expect. Even if you would use Django's signal framework for example, then some Django ORM calls can circumvent that (for example .update(..) [Django-doc]). But even if you somehow would cover those cases, then another program that talks to the database could update one of the two fields.
Most databases have triggers that can help. But again, the number of cases to cover are often larger than expected. For example, if the Product that we refer to is removed, then or the foreign key now points to a different Product, then we will need to update that field.
Therefore it is often better, to fetch the name of the related product when we need it. We can do so by (a) defining a property; or (b) make an annotation, for example in the manager.
Defining a property
We can define a property that will load the related product, and fetch the related name, like:
class Order(models.Model):
product = models.ForeignKey(Product, on_delete=models.PROTECT)
#property
def product_name(self):
return self.product.name
Then we can fetch the product name with some_order.product_name. This might not be very efficient if we need to fetch it often, since the relations are, by default, loaded lazily in Django, and thus can result in an N+1 problem.
Annotate the queryset
We can make an annotation that will fetch the name of the product in the same query when we fetch the Order, for example:
from django.db.models import F
class OrderManager(models.Manager):
def get_queryset(self):
return super().get_queryset().annotate(
product_name=F('product__name')
)
class Order(models.Model):
product = models.ForeignKey(Product, on_delete=models.PROTECT)
objects = OrderManager()
Then if we fetch an order. For example with Order.objects.get(pk=1), then that Order object will have an attribute product_name with the name of the product.
Suppose I have:
from django.db import models
class MyContentClass(models.Model):
content = models.TextField()
another_field = models.TextField()
x = MyContentClass(content="Hello, world!", another_field="More Info")
Is there a more concise way to perform the following logic?
existing = MyContentClass.objects.filter(content=x.content, another_field=x.another_field)
if existing:
x = existing[0]
else:
x.save()
# x now points to an object which is saved to the DB,
# either one we've just saved there or one that already existed
# with the same field values we're interested in.
Specifically:
Is there a way to query for both (all) fields without specifying
each one separately?
Is there a better idiom for either getting the old object or saving the new one? Something like get_or_create, but which accepts an object as a parameter?
Assume the code which does the saving is separate from the code which generates the initial MyContentClass instance which we need to compare to. This is typical of a case where you have a function which returns a model object without also saving it.
You could convert x to a dictionary with
x_data = x.__dict__
Then that could be passed into the object's get_or_create method.
MyContentClass.objects.get_or_create(**x_data)
The problem with this is that there are a few fields that will cause this to error out (eg the unique ID, or the _state Django modelstate field). However, if you pop() those out of the dictionary beforehand, then you'd probably be good to go :)
cleaned_dict = remove_unneeded_fields(x_data)
MyContentClass.objects.get_or_create(**cleaned_dict)
def remove_unneeded_fields(x_data):
unneeded_fields = [
'_state',
'id',
# Whatever other fields you don't want the new obj to have
# eg any field marked as 'unique'
]
for field in unneeded_fields:
del x_data[field]
return x_data
EDIT
To avoid issues associated with having to maintain a whitelist/blacklist of fields you, could do something like this:
def remove_unneeded_fields(x_data, MyObjModel):
cleaned_data = {}
for field in MyObjModel._meta.fields:
if not field.unique:
cleaned_data[field.name] = x_data[field.name]
return cleaned_Data
There would probably have to be more validation than simply checking that the field is not unique, but this might offer some flexibility when it comes to minor model field changes.
I would suggest to create a custom manager for those models and add the functions you want to do with the models (like a custom get_or_create function).
https://docs.djangoproject.com/en/1.10/topics/db/managers/#custom-managers
This would be the cleanest way and involves no hacking. :)
You can create specific managers for specific models or create a superclass with functions you want for all models.
If you just want to add a second manager with a different name, beware that it will become the default manager if you don't set the objects manager first (https://docs.djangoproject.com/en/1.10/topics/db/managers/#default-managers)
In my project I'm using Mongodb with mongoengine ORM(Python),
MongoDB shell version: 3.0.3
pymongo==3.1.1
mongoengine==0.10.1
I Wanted to track events happening to a collection to a log_collection, with original documents fields plus some new fields.
That is when a document in the Original collection is getting updated with an event, there would be a new document added to the log_collection.
class Original(DynamicDocument):
identifier = StringField(required=True, unique=True, primary_key=True)
field1 = StringField()
class LogEvents(DynamicDocument):
pass
For this I tried to make use mongoengeine switch_collection
col = Original.objects.get(id=some_id)
col.switch_collection('log_events', False)
col.new_field = "new_field"
col.save()
When we call the col.save() ,
1) it update to the log_events collection with the primary key field (_id) of the original document, so we cannot log the multiple events pertaining to the same document in the Original collection.
2) Even if we change the primary key field(_id), to a new value and save(),
col = Original.objects.get(id=some_id)
col.switch_collection('log_events', False)
col.id=<new-id>
col.new_field = "new_field"
col.save()
Then as it calls the save method of the Original collection, and there are unique indexes with multiple fields on the Original collection, its not possible to save to DB.
Is there any easy way to log events to the LogEvents collection without much hassle.
What is the use of cascade and cascade_kwargs options with the save method, how can we make use of it..?
What is the use of cascade and cascade_kwargs options with the save method, how can we make use of it..?
Mongoengine Docs:
cascade_save(*args, **kwargs)
Recursively saves any references / generic references on the document
Basically, this determines whether or not referenced documents are automatically saved when saving referencer document.
I don't think it is related to your issue.
A simple option for your kind of problem would be to overload the save() method of the Original class. Something like this:
class Original(DynamicDocument):
identifier = StringField(required=True, unique=True, primary_key=True)
field1 = StringField()
def save(self, value):
#save the document in its normal collections
super(Original, self).save()
#switch collection and save a new event
with switch_collection(Original, 'log_events') as Events:
self.new_field = value
#let's give it a new id
self.identifier = 'something new'
super(Events, self).save()
Now everytime you save an instance of Original, you can pass in a value to be saved separately in the log_events collection. With a bit of creative work you can pass more than one value or set more than one attribute.