I'm running Django 1.5 with SQLite, and I have a model called Assignment. Whenever I create one, it gets created with the correct pk value. But, whenever I try to retrieve any Assignment from my database, it is always returned with a pk of 90. I've been fighting this for an hour, and I have to admit I'm completely confused.
Here's my code, if it's any use.
class Assignment(models.Model):
class Meta:
app_label = 'holiday'
unique_together = ('year', 'employee')
year = models.PositiveIntegerField(db_index=True)
employee = models.ForeignKey('bsc.Employee', db_index=True)
days = models.PositiveIntegerField(null=True)
This, and a bunch of methods that calculate some values based on models related to this one. Nothing fancy.
I've got to add that this model has had a somewhat rough past - with all my absent-mindedness, I had originally set year as the primary key, which quickly failed as soon as I added two Assignments to different employees. Maybe I should look at the DB schema and see if anything's wrong. Thankfully, the app hasn't made it to production yet, but hopefully this can be fixed without a full DB reset.
If you had created 90 previous records then deleted the rows from your database; your database key index will still be set to what would have been the next primary key number in your database.
The way to resolve this would be to as described in this other stackoverflow post:
SQLite Reset Primary Key Field
Related
I have a strange situation where Django seems to be giving me records that do not actually exist in the database when queried via a related_name on a foreign key. Here's a simplified example:
Let's say I have a Person model and a Pet model, where each Pet has an owner, which is a foreign key on Person:
class Pet(models.Model):
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
name =models.CharField(max_length=50, null=False, db_index=True)
owner = models.ForeignKey("Person", null=False, related_name="pets", on_delete=models.CASCADE)
relationship = models.IntegerField(null=False, choices=PetRelationship.choices(), db_index=True)
Now, I have the below function that retrieves a person's pets:
def pet_tester(person):
for pet in person.pets.filter(relationship=PetRelationship.FRIENDLY):
pet_id = pet.id
LOGGER.info(f"*************** pet.id = {pet_id}")
LOGGER.info(f"******* Pet exists? = {Pet.objects.filter(id=pet_id).exists()}")
...
Note that this is NOT a "minimal reproducible example". I can only reproduce it in my much larger application.
For some reason (in the real app), the "exists" query is coming back as False. The output is like this:
*************** pet.id = 123e4567-e89b-12d3-a456-426614174000
******* Pet exists? = False
If I query the actual database (Postgresql) directly (outside of Django), that pet ID sure enough does NOT exist. However, the person.pets.filter query is returning it just the same.
I do not understand how this is even possible. It JUST retrieved the pet from the database (or so it would seem - it even has a UUID), but it's not really there after all when I immediately try to query it back.
It seems like the Django "filter" results are somehow inconsistent with the underlying database, but I don't know why. This is very reproducible in my app (it happens every time, consistently). It doesn't appear to be a corrupt database, as it happens consistently even on brand new machines, with the app and database freshly installed.
I know you don't know what else my app is doing. I obviously can't post all the source code, but I can assure you that the beginning of the actual function is pretty much verbatim of the pet_tester function above. I only changed the class/field names. There are no extra lines in the actual app between the "for" line and the 3 lines below it.
I'm mostly looking for ideas about how this can possibly happen, and what to look for in the app's other code.
It turns out that there is another service that is deleting the "pets" from the database. There must be some kind of caching in Django, and since the deletion happened outside of Django, Django didn't invalidate its "person.pets" cache.
I'll simplify my problem:
In my project I have a Person model and that person has an age. I changed the default manager applied to Person using base_manager_name to my version of objects.
Person` has this:
objects = MySpecialAgeFilterManager()
objects_all = models.Manager()
So, when I ask for Person.objects.all() I will get only people between the ages of 20 and 30. Don't ask me why-- it's just for the sake of this question..
I have a certain Person instance with the age of 10. That person will not show up on Person.objects.all() -- however it will be included in Person.objects_all.all()
My problem is this.
p = Person.objects_all.filter(age=10).first() # works, I get my person.
p.age = 20 # trying to bring him into the light....
p.save() # CRASH!!!
The crash claims the pk for p creates a duplicate violation on the database. It's as if it's trying to save it with the objects scope in mind, and not the objects_all scope I retrieved that person with. In the objects context it doesn't exist so it tries to save... but the database has that PK already in use. The database doesn't care about Model Manager scopes....
How do I resolve this?
What worked:
Person.objects_all.filter(pk=p.id).update(age=20)
to sum it up, when you have a base_manager_name applied that's EXCLUSIVE. and you use it to get model rows in order to update it. Update via .update and not save because save doesn't work in the context of the special model manager you've obtained the object through.
I'm trying to track Foreign Keys using django-field-history, but when I add it, it does additional queries on every page using the Model
For example
from field_history.tracker import FieldHistoryTracker
class Author(models.Model):
user = models.ForeignKey('auth.user)
field_history = FieldHistoryTracker(['user'])
will always give more queries on pages using Author, like so
SELECT ••• FROM "auth_user" WHERE "auth_user"."id" = '2'
1239 similar queries. Duplicated 1235 times.
I've tried using user_id instead of user in Field History Tracker, but it will always return None. Using user.id or anything like it just returns an error.
I really need to keep that history data, but not at the cost of thousands of additional queries.
Also, would really enjoy keeping django-field-history as my whole DB is using it, but I'm aware I might have to switch package, and if so, which one would you advise ?
As far as my understanding goes, you are trying to log which user has updated, for this you should use _field_history_user as described in the documentation.
For example:
class Pizza(models.Model):
name = models.CharField(max_length=255)
updated_by = models.ForeignKey('auth.User')
field_history = FieldHistoryTracker(['name'])
#property
def _field_history_user(self):
return self.updated_by
It would always update which user has updated the row for this table.
We have 2 identical Django instances and one is trial the other is for production. Recently one client on trial purchased full product and I need to move ONLY their data from trial to production. I don't know if there's a convenient way to do that, since:
If I use Django fixtures then it might overwrite the existing data in production system because of the default id that Django assigned to each entry(I might be wrong but I think fixtures are only good for initialization).
Using sql to dump the DB might not help either because of the similar problem with the first approach, and it's also complex because there are other customers in trial but I only need to move this client's data.
Please give me some advice if you have similar experience.
The issue is you cannot transfer the primary keys (IDs) from the trial to the production DB, isn't it ? So 2 solutions:
1) A tank to kill an ant
You do a SQL export of your trial database, and you increase every primary key and foreign key by a number (for ex: 10000). This number needs to high enough to avoid unicity constraint violation when you will import it in the DB
2) The smart solution
If, and only if, your model is well designed: for every model you can find a set of its columns that could make a substitute primary key to the ID. If you have attributes with unique = True, or models with unique_together = (...) it's perfect: you can use natural keys !
In every model of your source code, you add the method get_by_natural_key:
class Person(models.Model):
firstname = models.CharField...
last_name = models.CharField...
class Meta:
unique_together = ("first_name", "last_name")
def get_by_natural_key(self, first_name, last_name):
return self.get(first_name=first_name, last_name=last_name)
Then you can use Django dumpdata command to export the trial database with the IDs replaced by the natural keys ! Then with the same code, you use the loaddata command to import these data files.
I have a model (lets call it Entity) that has an attribute (Attribute) that changes over time, but I want to keep a history of how that attribute changes in the database. I need to be able to filter my Entities by the current value of Attribute in its manager. But because Django (as far as I can tell) won't let me do this in one query natively, I have created a database view that produces the latest value of Attribute for every Entity. So my model structure looks something like this:
class Entity(models.Model):
def set_attribute(self, value):
self.attribute_history.create(value=value)
def is_attribute_positive(self, value):
return self.attribute.value > 0
class AttributeEntry(models.Model):
entity = models.ForeignKey(Entity, related_name='attribute_history')
value = models.IntegerField()
time = models.DateTimeField(auto_now_add=True)
class AttributeView(models.Model)
id = models.IntegerField(primary_key=True, db_column='id',
on_delete=models.DO_NOTHING)
entity = models.OneToOneField(Entity, related_name='attribute')
value = models.IntegerField()
time = models.DateTimeField()
class Meta:
managed = False
My database has the view that produces the current attribute, created with SQL like this:
CREATE VIEW myapp_attributeview AS
SELECT h1.*
FROM myapp_attributehistory h1
LEFT OUTER JOIN myapp_attributehistory h2
ON h1.entity_id = h2.entity_id
AND (h1.time < h2.time
OR h1.time = h2.time
AND h1.id < h2.id)
WHERE h2.id IS NULL;
My problem is that if I set the attribute on a model object using set_attribute() checking it with is_attribute_positive() doesn't always work, because Django may be caching that the related AttributeView object. How I can I make Django update its model, at the very least by requerying the view? Can I mark the attribute property as dirty somehow?
PS: the whole reason I'm doing this is so I can do things like Entity.objects.filter(attribute__value__exact=...).filter(...), so if someone knows an easier way to get that functionality, such an answer will be accepted, too!
I understand that the attribute value is modified by another process (maybe not even Django) accessing the same database. If this is not the case you should take a look at django-reversion.
On the other hand if that is the case, you should take a look at second answer of this. It says that commiting transaction invalidate query cache and offer this snippet.
>>> from django.db import transaction
>>> transaction.enter_transaction_management()
>>> transaction.commit() # Whenever you want to see new data
I never directly solved the problem, but I was able to sidestep it by changing is_attribute_positiive() to directly query the database table, instead of the view.
def is_attribute_positive(self, value):
return self.attribute_history.latest().value > 0
So while the view gives me the flexibility of being able to filter queries on Entity, it seems the best thing to do once the object is received is to operate directly on the table-backed model.