I've done a facebook like model, but I want the Personne to have more than one link with another Personne.
I have an intermediary table PersonneRelation with a custom save method. The idea is: when I add a relation to a person, I want to create another relation the other way. The problem is that if I try to save in the save method it's a recursive call. So my idea was to create a variable of the class and set it to True only when I want to avoid recursion.
Here's how I did:
class Personne(models.Model):
user = models.OneToOneField(User)
relations = models.ManyToManyField('self', through='PersonneRelation',
symmetrical=False)
class PersonneRelation(models.Model):
is_saving = False
# TAB_TYPES omitted for brevity
type_relation = models.CharField(max_length=1,
choices=[(a, b) for a, b in
list(TAB_TYPES.items())],
default=TYPE_FRIEND)
src = models.ForeignKey('Personne', related_name='src')
dst = models.ForeignKey('Personne', related_name='dst')
opposite = models.ForeignKey('PersonneRelation',
null=True, blank=True, default=None)
def save(self, *args, **kwargs):
if self.is_saving:
return super(PersonneRelation, self).save(args, kwargs)
old = None
if self.pk and self.opposite:
old = self.type_relation
retour = super(PersonneRelation, self).save(args, kwargs)
if old:
PersonneRelation.objects.filter(
src=self.dst, dst=self.src, opposite=self, type_relation=old
).update(type_relation=self.type_relation)
if self.opposite is None:
self.opposite = PersonneRelation(
src=self.dst, dst=self.src, opposite=self,
type_relation=self.type_relation, is_reverse=True)
self.opposite.save()
self.is_saving = True
self.save()
self.is_saving = False
return retour
My question is: is it safe to do so (using a class variable is_saving) (I dont know how Python deals with such variables)? If not, why? I feel like it's not ok, so what are the other possibilities to implement multiple many to many relationship that should behave like that?
Unfortunately, it's not safe, because it's not thread-safe. When two simultaneous Django threads will try to save your model, the behaviour can be unpredictable.
If you want to have more reliable locking, take a look, for example, at the Redis locking.
But to be honest, I'd try to implement it using plain reverse relations, maybe incapsulating the complexity into the ModelManager.
Here's how I modified it: I totally removed the save method and used the post_save message to check:
if it was created without opposite side, I create here with opposite side as the one created (and I can do it here without any problem!) then I update the one created with the "opposite"
if it wasn't created, this is an update, so just make sure the opposite side is changed as well.
I did this because I'll almost never have to change relationships between people, and when I'll create new ones there wont be any possible race conditions, because of the context where I will create new relationships
#receiver(post_save, sender=PersonneRelation)
def signal_receiver(sender, **kwargs):
created = kwargs['created']
obj = kwargs['instance']
if created and not obj.opposite:
opposite = PersonneRelation(
src=obj.dst, dst=obj.src, opposite=obj,
type_relation=obj.type_relation, is_reverse=True)
opposite.save()
obj.opposite = opposite
obj.save()
elif not created and obj.type_relation != obj.opposite.type_relation:
obj.opposite.type_relation = obj.type_relation
obj.opposite.save()
If I get the idea behind your code, then:
Django automatically makes relation available on both ends so you can get from src Personne to dst Personne via PersonneRelation and reverse dst -> src in your code. Therefore no need for additional opposite field in PersonneRelation.
If you need to have both symmetrical and asymmetrical realtions, i.e. src -> dst, but not dst -> src for particaular record, then I would suggest to add boolean field:
class PersonneRelation(models.Model):
symmetrical = models.BooleanField(default=False)
this way you can check if symmetrical is True when accessing relation in your code to identify if it's scr -> dst only or both src -> dst and dst -> src. In facebook terms: if symmetrical is False you get src is subscriber of dst, if it's True you get mutual friendship between src and dst. You might want to define custom manager to incapsulate this behavior, though it's more advanced topic.
If you need to check if the model instance is being saved or updated, there's no need in is_saving boolean field. Since you're using automatic primary key field, you can just check if pk on model instance is None. In Django before the model instance is first time saved to DB ('created') pk is None, when the instance is 'updated' (it has been read from DB before and is being saved now with some field values changed) it's pk is set to pk value from DB. This is the way Django ORM decides if it should update or create new record.
In general when redefining Save method on a model, or when using signals like pre_save/post_save take into consideration, that those functions you define on them might not be called by Django in some circumstances, i.e. when the model is updated in bulk. See Django docs for more info.
Related
Using Django 3.2 -- I will simplify the problem as much as I can.
I have three model classes:
# abstract base class
MyAbstractModel(models.Model)
# derived model classes
Person(MyAbstractModel)
LogoImage(MyAbstractModel)
Each Person has:
image = ForeignKey(LogoImage, db_index=True, related_name="person", null=True,
on_delete=models.PROTECT)
The MyAbstractModel defines a few model managers:
objects = CustomModelManager()
objects_all_states = models.Manager()
as well as a state field, that can be either active or inactive
CustomModelManager is defined as something that'll bring only records that have state == 'active':
class CustomModelManager(models.Manager):
def get_queryset(self):
return super().get_query().filter(self.model, using=self._db).filter(state='active')
In my database I have two objects in two tables:
Person ID 1 state = 'active'
Image ID 1 state = 'inactive'
Person ID 1 has a foreign key connection to Image ID 1 via the Person.image field.
------ NOW for the issue ----------------
# CORRECT: gives me the person object
person = Person.objects.get(id=1)
# INCORRECT: I get the image, but it should not work...
image = person.image
Why is that incorrect? because I queried for the person object using the objects model manager which is supposed to bring only those items with active status. It brought the Person which is fine, because Person (ID=1) is state==active -- but the object under person.image is state==inactive. Why am I getting it?
WORKAROND ATTEMPT:
added base_manager_name = "objects" to the MyAbstractModel class Meta: section
ATTEMPTING AGAIN:
# CORRECT: gives me the person object
person = Person.objects.get(id=1)
# CORRECT: gives me a "Does not Exist" exception.
image = person.image
However..... Now I try this:
# CORRECT: getting the person
person.objects_all_states.get(id=1)
# INCORRECT: throws a DoesNotExist, as it's trying to use the `objects` model manager I hard coded in the `MyAbstractModel` class meta.
image = person.image
Since I got the Person under the objects_all_states which does not care about state==active -- I expect I would also get the person.image in a similar way. But that doesn't work as expected.
THE ROOT ISSUE
How do I force the same model manager used to fetch the parent object (Person) -- in the fetching of every single ForeignKey object a Person has? I can't find the answer. I've been going in circles for days. There is simply no clear answer anywhere. Either I am missing something very fundamental, or Django has a design flaw (which of course I don't really believe) -- so, what am I missing here?
Why they don't play well together
Foreign key classes use separate instances of managers, so there's no shared state.
There's no information about the manager used on the parent instance either.
As per django.db.models.Model._base_manager, Django simply uses _base_manager:
return self.field.remote_field.model._base_manager.db_manager(hints=hints).all()
...where hints would be {'instance': <Person: Person object (1)>}.
Since we have a reference to the parent, in some scenarios, we could support this inference.
Fair warning
Django specifically mentions not to do this.
From django.db.models.Model._base_manager:
Don’t filter away any results in this type of manager subclass
This manager is used to access objects that are related to from some other model. In those situations, Django has to be able to see all the objects for the model it is fetching, so that anything which is referred to can be retrieved.
Therefore, you should not override get_queryset() to filter out any rows. If you do so, Django will return incomplete results.
1. How you could implement this inference
You could:
override get() to actively store some information on the instance (that will be passed as hint) about whether an instance of CustomModelManager was used to get it, and then
in get_queryset, check that and try to fallback on objects_all_states.
class CustomModelManager(models.Manager):
def get(self, *args, **kwargs):
instance = super().get(*args, **kwargs)
instance.hint_manager = self
return instance
def get_queryset(self):
hint = self._hints.get('instance')
if hint and isinstance(hint.__class__.objects, self.__class__):
hint_manager = getattr(hint, 'hint_manager', None)
if not hint_manager or not isinstance(hint_manager, self.__class__):
manager = getattr(self.model, 'objects_all_states', None)
if manager:
return manager.db_manager(hints=self._hints).get_queryset()
return super().get_queryset().filter(state='active')
Limitations
One of possibly many edge cases where this wouldn't work is if you queried person via Person.objects.filter(id=1).first().
2. Using explicit instance context
Usage:
person = Person.objects_all_states.get(id=1)
# image = person.image
with CustomModelManager.disable_for_instance(person):
image = person.image
Implementation:
class CustomModelManager(models.Manager):
_disabled_for_instances = set()
#classmethod
#contextmanager
def disable_for_instance(cls, instance):
is_already_in = instance in cls._disabled_for_instances
if not is_already_in:
cls._disabled_for_instances.add(instance)
yield
if not is_already_in:
cls._disabled_for_instances.remove(instance)
def get_queryset(self):
if self._hints.get('instance') in self._disabled_for_instances:
return super().get_queryset()
return super().get_queryset().filter(state='active')
3. Using explicit thread-local context
Usage:
# person = Person.objects_all_states.get(id=1)
# image = person.image
with CustomModelManager.disable():
person = Person.objects.get(id=1)
image = person.image
Implementation:
import threading
from contextlib import contextmanager
from django.db import models
from django.utils.functional import classproperty
class CustomModelManager(models.Manager):
_data = threading.local()
#classmethod
#contextmanager
def disable(cls):
is_disabled = cls._is_disabled
cls._data.is_disabled = True
yield
cls._data.is_disabled = is_disabled
#classproperty
def _is_disabled(cls):
return getattr(cls._data, 'is_disabled', None)
def get_queryset(self):
if self._is_disabled:
return super().get_queryset()
return super().get_queryset().filter(state='active')
Well, i must point out a few design flaws in your approach.
First - you should not override get_queryset method for manager. Instead, make a separate method to filter specific cases. Even better if you make a custom QuerySet class with those methods, since then you will able to chain them
class ActiveQuerySet(QuerySet):
def active(self):
return self.filter(state="active")
# in your model
objects = ActiveQueryset.as_manager()
Also, you should not place field state in every model and expect, that Django will handle this for you. It will be much easier to handle for you if you decide from domain perspective, which model is your root model and have state there. For example, if Person can be inactive, then probably all of his images are also inactive, so you may safely assume, that Persons status is shared by all related models.
I would specifically look for a way to avoid such issue from design perspective, instead of trying to brutforce Django to process such filtration cases
class PurchaseOrder(models.Model):
purchase_order_id = models.AutoField(primary_key=True)
purchase_order_number = models.CharField(unique=True)
vendor = models.ForeignKey(Vendor)
i am creating Purchase Order(po) table. when po created i have to update purchase_order_number as "PO0"+purchase_order_id ex PO0123 (123 is Primary key). so i am using def save in models to accomplish this
def save(self):
if self.purchase_order_id is not None:
self.purchase_order_number = "PO"+str(self.purchase_order_id)
return super(PurchaseOrder, self).save()
It is working fine with single creation but when i try to create bulk of data using locust(Testing tool) its giving an error duplicate entry for PurchseOrdernumber Can we modify field value in models itself some thing like this
purchase_order_number = models.CharField(unique=True,default=("PO"+self.purchase_order_id )
To be honest, I don't think it should work when you create multiple instances. Because as I can see from the code:
if self.purchase_order_id is not None:
self.purchase_order_number = "PO"+str(self.purchase_order_id)
Here purchase_order_id will be None when you are creating new instance. Also, until you call super(PurchaseOrder, self).save(), it will not generate purchase_order_id, meaning purchase_order_number will be empty.
So, what I would recommend is to not store this information in DB. Its basically the same as purchase_order_id with PO in front of it. Instead you can use a property method to get the same value. Like this:
class PurchaseOrder(models.Model):
purchase_order_id = models.AutoField(primary_key=True)
# need to remove `purchase_order_number = models.CharField(unique=True)`
...
#property
def purchase_order_number(self):
return "PO{}".format(self.purchase_order_id)
So, you can also see the purchase_order_number like this:
p = PurchaseOrder.objects.first()
p.purchase_order_number
Downside of this solution is that, you can't make any query on the property field. But I don't think it would be necessary anyway, because you can do the same query for the purchase_order_id, ie PurchaseOrder.objects.filter(purchase_order_id=1).
I have a database containing a list of ingredients. I'd like to avoid duplicate entries in this table. I don't want to use the unique keyword for 2 reasons :
My uniqueness constraints are a bit more sophisticated than a mere =
I don't want to raise an exception when a pre-existing ingredient model is created, instead I just want to return that model, so that I can write Ingredient(ingredient_name='tomato') and just go on with my day rather than encapsulating all of that in a try clause. This will allow me to easily add ingredients to my recipe table on the fly.
One solution is simply to have a wrapper function like create_ingredient, but I don't find that to be particularly elegant and more specifically it's not robust to some other developer down the line simply forgetting to use the wrapper. So instead, I'm playing around with the pre_init and post_init signals.
Here's what I have so far :
class Ingredient(models.Model):
ingredient_name = models.CharField(max_length=200)
recipes = models.ManyToManyField(Recipe,related_name='ingredients')
def __str__(self):
return self.ingredient_name
class Name(models.Model):
main_name = models.CharField(max_length=200, default=None)
equivalent_name = models.CharField(max_length=200, primary_key=True, default=None)
def _add_ingredient(sender, args, **kwargs):
if 'ingredient_name' not in kwargs['kwargs'] :
return
kwargs['kwargs']['ingredient_name'] = kwargs['kwargs']['ingredient_name'].lower()
# check if equivalent name exists, make this one the main one otherwise
try:
kwargs['kwargs']['ingredient_name'] = Name.objects.filter(
equivalent_name=kwargs['kwargs']['ingredient_name']
)[0].main_name
except IndexError:
name = Name(main_name=kwargs['kwargs']['ingredient_name'],
equivalent_name=kwargs['kwargs']['ingredient_name'])
name.save()
pre_init.connect(_add_ingredient, Ingredient)
So far so good. This actually works and will replace ingredient_name when needed before the model is initialized. Now what I'd like is to check if the ingredient in question already exists and have the initializer return it if it does. I think I need to play around with post_init to do this but I don't know how to modify the particular instance that's being created. Here's what I mean by that :
def _finalize_ingredient(sender, instance, **kwargs):
try:
# doesn't work because of python's "pass arguments in python's super unique way of doing things" thing
instance = Ingredient.objects.filter(ingredient_name=instance.ingredient_name)[0]
except IndexError:
pass
post_init.connect(_finalize_ingredient, Ingredient)
As I've commented, I don't expect this to work because instance = ... doesn't actually modify instance, it just reassigns the variable name (incidentally if you try to run this all sorts of terrible things happen which I don't care to understand because I know this is flat out wrong). So how do I actually do this ? I really hope wrapper functions aren't the cleanest option here. I'm a big fan of OOP and gosh darn it I want an OOP solution to this (which, as I've said, I think in the long run would be much more robust and safer than wrappers).
I realize of course that I can add an add_ingredient method to Recipe which will do all of this for me, but I really like the idea of containing all of this in my Ingredient class as it will guarantee the proper database behavior under any circumstance. I'm also curious as to know if/how the post_init method can be used to completely override the created object for a given circumstance.
By the way, some of you may be wondering why I don't have a ForeignKey entry in my Name class that would connect the Name table to the Ingredient table. After all, isn't this what my check is essentially accomplishing in my _add_ingredient method ? One of the reasons is that if I do this then I end up with the same problem I'm trying to solve here : If I want to create an ingredient on the fly to add it to my recipe, I could simply create a Name object when creating an Ingredient object, but that would raise an exception if it corresponds to a main_name that is already in use (rather than simply returning the object I need).
I believe you are looking for get_or_create(), which is already a built-in in Django.
You mention:
One solution is simply to have a wrapper function like create_ingredient, but I don't find that to be particularly elegant and more specifically it's not robust to some other developer down the line simply forgetting to use the wrapper.
Well, look at it the other way around. What if you actually need to create a "duplicate" ingredient? Then it is nice to have the possibility.
I've come up with something that is as elegant and robust as I think it's possible to be given what I'm after. I've still had to define an add_ingredient method, but I still have the robustness that I need. I've made it so that it can be generalized to any class with a primary key, and the Name table will contain the info that will define the name uniqueness of any table :
class Name(models.Model):
main_name = models.CharField(max_length=200, default=None)
equivalent_name = models.CharField(max_length=200, primary_key=True, default=None)
def _pre_init_unique_fetcher(sender, args, **kwargs):
pk_name = sender._meta.pk.name
if pk_name not in kwargs['kwargs'] :
return
kwargs['kwargs'][pk_name] = kwargs['kwargs'][pk_name].lower()
# check if equivalent name exists, make this one the main one otherwise
try:
kwargs['kwargs'][pk_name] = Name.objects.filter(
equivalent_name=kwargs['kwargs'][pk_name]
)[0].main_name
except IndexError:
name = Name(main_name=kwargs['kwargs'][pk_name],
equivalent_name=kwargs['kwargs'][pk_name])
name.save()
sender._input_dict = kwargs['kwargs']
def _post_init_unique_fetcher(sender, instance, **kwargs):
pk_name = sender._meta.pk.name
pk_instance = instance.__dict__[pk_name]
filter_dict = {}
filter_dict[pk_name] = pk_instance
try:
post_init.disconnect(_post_init_unique_fetcher,sender)
instance.__dict__ = sender.objects.filter(**filter_dict)[0].__dict__
post_init.connect(_post_init_unique_fetcher, sender)
for key in sender._input_dict:
instance.__dict__[key] = sender._input_dict[key]
del sender._input_dict
except IndexError:
post_init.connect(_post_init_unique_fetcher, sender)
except:
post_init.connect(_post_init_unique_fetcher, sender)
raise
unique_fetch_models = [Ingredient, Recipe, WeekPlan]
for unique_fetch_model in unique_fetch_models :
pre_init.connect(_pre_init_unique_fetcher, unique_fetch_model)
post_init.connect(_post_init_unique_fetcher, unique_fetch_model)
Now what this will do is load up any new model with the pre-existing data of the previous model (rather than the default values) if one with the same name exists. The reason I still need an add_ingredient method in my Recipe class is because I can't call Ingredient.objects.create() for a pre-existing ingredient without raising an exception despite the fact that I can create the model and immediately save it. This has to do with how Django handles the primary_key designation : if you create the model then save it, it assumes you're just updating the entry if it already exists with that key, and yet if you create it, it tries to add another entry and that conflicts with the primary_key designation. So now I can do things like recipe.add_ingredient(Ingredient(ingredient_name='tomato', vegetarian=True)).
I have 3 models.
class Picture(models.Model)
name = models.CharField(max_length=255)
image_field = models.ImageField(upload_to="foo/")
slug = models.SlugField()
[...]
class Size(models.Model):
name = models.CharField(max_length=255)
width = models.IntegerField()
height = models.IntegerField()
crop = models.BooleanField(default=True)
upscale = models.BooleanField(default=False)
def __unicode__(self):
return self.name
class Cache(models.Model):
size = models.ForeignKey('Size')
picture = models.ForeignKey('Picture')
image_field = models.ImageField(upload_to="picture/resize/")
I want to use them as follows: First generate Picture objects. Then create Size objects. For every Size and Picture a Cache object should be generated when needed.
My problem is that I don't know where to put the code. It should be something like (pseudocode):
def get_cached_picture(Picture,Size):
try:
cacheObj = Cache.objects.get(picture=Picture, size=Size):
[.. use cacheObj ..]
except Cache.DoesNotExist:
[.. resize Picture according to Size, insert into cache, use it ..]
So where can I plug this code ? I know I could do this within a view, but is there a way to embed it into the models ? Cache should never be filled in the admin, instead it should be generated whenever a certain combination between Cache and Picture is needed.
It is probably easy to do, but I'm lacking of the right keyword for google.
Assumptions:
You want to be able to load a "Representation" of an image at a certain size on the fly.
Size model will store a predefined set of sizes that a Representation can have.
I am going to refer to your Cache model as Representation because I think it makes more sense in context here.
The entry-point of this process would obviously be a view. A request comes in and you determine you need to search "Image1.jpg" # 800x600. The simplest approach is just to put your queries right into the view. But for reusability, it might be better to do any of the following:
A utility function which is then called from the view to retrieve a Representation object
A classmethod of your Representation model, since you are already referencing Picture and Size as foreign keys.
Using a custom manager on the Representation model to do something similar to the above
Any of these will make it reusable as an app if someone else were to have their own views and needed to simply retrieve a Rep instance directly.
While your pseudocode refers to this process being on the Picture mode, I maintain that it should probably be on the Representation model, since it contains both foreign keys and you can easily see if you have a proper rep. If not, create it.
The view should only have to call something simple as opposed to having all of the logic:
# as classmethod
rep = Representation.get_cached_picture(picture, size)
# or with a manager
rep = Representation.objects.get_cached_picture(picture, size)
I'm trying to write an internal API in my application without necessarily coupling it with the database.
class Product(models.Model):
name=models.CharField(max_length=4000)
price=models.IntegerField(default=-1)
currency=models.CharField(max_length=3, default='INR')
class Image(models.Model):
# NOTE -- Have changed the table name to products_images
width=models.IntegerField(default=-1)
height=models.IntegerField(default=-1)
url=models.URLField(max_length=1000, verify_exists=False)
product=models.ForeignKey(Product)
def create_product:
p=Product()
i=Image(height=100, widght=100, url='http://something/something')
p.image_set.add(i)
return p
Now, when I call create_product() Django throws up an error:
IntegrityError: products_images.product_id may not be NULL
However, if I call p.save() & i.save() before calling p.image_set.add(i) it works. Is there any way that I can add objects to a related object set without saving both to the DB first?
def create_product():
product_obj = Product.objects.create(name='Foobar')
image_obj = Image.objects.create(height=100, widght=100, url='http://something/something', product=product_obj)
return product_obj
Explanation:
Product object has to be created first and then assign it to the Image object because id and name here is required field.
I am wondering why wouldn't you not require to make a product entry in DB in first case? If there is any specific reason then i may suggest you some work around?
EDIT: Okay! i think i got you, you don't want to assign a product to an image object initially. How about creating a product field as null is equal to true.
product = models.ForeignKey(Product, null=True)
Now, your function becomes something like this:
def create_product():
image_obj = Image.objects.create(height=100, widght=100, url='http://something/something')
return image_obj
Hope it helps you?
I got same issue with #Saurabh Nanda
I am using Django 1.4.2. When I read in django, i see that
# file django/db/models/fields/related.py
def get_query_set(self):
try:
return self.instance._prefetched_objects_cache[rel_field.related_query_name()]
except (AttributeError, KeyError):
db = self._db or router.db_for_read(self.model, instance=self.instance)
return super(RelatedManager,self).get_query_set().using(db).filter(**self.core_filters)
# file django/db/models/query.py
qs = getattr(obj, attname).all()
qs._result_cache = vals
# We don't want the individual qs doing prefetch_related now, since we
# have merged this into the current work.
qs._prefetch_done = True
obj._prefetched_objects_cache[cache_name] = qs
That 's make sese, we only need to set property _prefetched_objects_cache for the object.
p = Product()
image_cached = []
for i in xrange(100):
image=Image(height=100, widght=100, url='http://something/something')
image_cached.append(image)
qs = p.images.all()
qs._result_cache = image_cached
qs._prefetch_done = True
p._prefetched_objects_cache = {'images': qs}
Your problem is that the id isn't set by django, but by the database (it's represented in the database by an auto-incremented field), so until it's saved there's no id. More about this in the documentation.
I can think of three possible solutions:
Set a different field of your Image model as the primary key (documented here).
Set a different field of your Production model as the foreign key (documented here).
Use django's database transactions API (documented here).