Django: making relationships in memory without saving to DB - django

I have some models with relationships like this:
class Item(model.Model):
name = models.CharField()
class Group(models.Model):
item = models.ManyToManyField(Item)
class Serie(models.Model):
name = models.CharField()
chart = models.ForeignKey(Chart)
group = models.ForeignKey(Group)
class Chart(models.Model):
name = models.CharField()
I need to create a Chart object on the fly, without saving to the DB. But I can't do it because Django tries to use the objects primary keys when assigning the relationships.
I just want Group.add(Item()) to work without having to save the objects to the DB.
Is there any simple way around this?

Reviving here for the sake of future readers:
I've gotten around this use case by defining a private attribute that represents the relationship inside the classes and a property to inspect wether the object can be retrieved from the DB or resides in memory.
Here is a simple example:
class Parent(models.Model):
_children = []
name = models.CharField(max_length=100)
#property
def children(self):
if _children:
return self._children
else:
return self.children_set.all()
def set_virtual_children(self, value): # could use a setter for children
self._children = value # Expose _children to modification
def some_on_the_fly_operation(self):
print(','.join([c.name for c in self.children]))
class Children(models.Model):
parent = models.ForeignKey(Parent)
name = models.CharField(max_length=100)
This way, I can set the "virtual children" and use all the defined methods "on the fly"

EDIT: It seems that approach described here isn't enough for django to allow adding to the ManyToMany relationship.
Have you tried to add primary_key=True and unique=True to the name attribute of the Item model. Then doing Group.add(Item("item_name_here")) should work if you have the possibility to create the name on the fly.
I didn't test it, but I think your way failed because add() wants to use the primary-key which by default is the autoincrementing id that is assigned when it is saved to the database.

Related

ForeignKey/OneToOneField referencing a set of unique_together fields?

I would like to add some metadata to a certain object in my Django models. Normally this would be a good situation for a OneToOneField or even Multi-table inheritance, but in my case there's an additional special requirement:
Due to the way data is synchronized in the overall system, the database storage for the actual "target" objects occasionally get re-created. That is, from time to time all the object rows get deleted and new ones are created in their place. But I need the metadata to persist.
Besides its implicit primary key, each object also has a textual slug field that uniquely identifies it — but only within a single container. This slug is stable across the data being re-created, as is the container itself. So I'd like to use a combination of an object's slug and its container to associate my metadata, rather than the database primary key for the object.
I found the to_field option on the ForeignKey model field, but if I set for example to_field='id_in_container' it complains since the id_in_container field is not unique on its own, only together with the container field.
Is there a way to do something like this?
class Container(models.Model):
name = models.CharField(max_length=255)
class Object(models.Model):
container = models.ForeignKey(Container)
id_in_container = models.SlugField()
name = models.CharField(max_length=255)
class Meta:
unique_together = ('container', 'id_in_container')
# BUG: this won't actually work, `to_field` can't take a tuple!
DESIRED_COMBO = ('container','id_in_container')
class ObjectMetadata(models.model):
object = models. OneToOneField(Object, to_field=DESIRED_COMBO)
# … additional fields …
That is, have a OneToOneField or ForeignKey that targets multiple columns of a target model, when those columns are unique_together?
I suppose I can implement some manual helpers, perhaps like this:
class ObjectMetadata(models.model):
object_container = models.ForeignKey(Container)
object_slug = models.SlugField()
# … additional fields …
#classmethod
def get_for_object(cls, obj):
return cls.objects.get(object_container=obj.container, object_slug=obj.id_in_container)
…but I'm wondering if there's a more appropriate way to model this.

Can I define fields on an implicit many-to-many relationship?

Here's a version of my models.py file. I've removed irrelevant fields and the model names are made up, for security reasons:
class FilmStudio(models.Model):
name = models.CharField(max_length=200, unique=True)
class ProductionCompany(models.Model):
name = models.CharField(max_length=200)
film_studio = models.ForeignKey(FilmStudio)
class Meta:
# Ensure that a given combination of ProductionCompany name and FilmStudio object is unique
unique_together = ('name', 'film_studio')
class Film(models.Model):
title = models.CharField(max_length=200)
production_company = models.ForeignKey(ProductionCompany)
class Actor(models.Model):
name = models.CharField(max_length=200)
films = models.ManyToManyField(Film, blank=True)
Although it is not explicitly defined, there is a many-to-many relationship between an Actor and a FilmStudio. This is evidenced by the following call to the Python API:
FilmStudio.objects.filter(productioncompany__film__actor__name='Samuel L. Jackson').distinct()
This returns all of the FilmStudio objects which Samuel L. Jackson is related to, and each one only once. What I'd like is to define extra fields on the relationship between an Actor and a FilmStudio (it doesn't work too well in this example, I know, but it makes sense for my scenario).
Following what is described in Extra fields on many-to-many relationships, I could use an intermediate model to define extra fields on the relationship between a Film and an Actor, for instance.
But this doesn't seem to help me with my problem. I don't want to define the Actor to FilmStudio relationship explicitly, since it's an existing relationship based on other relationships.
Is it possible to define fields on the relationship that I'm describing?
As far as I know, you are not able to do that.
The reason for that is that it is nowhere to store the extra fields of that relationship. If I understand you correctly, these "extra fields" are not implicit in the actor-film or productionstudio-film relationships, so even though you say they are implicit, the extra fields themselves are explicit.
You could try to emulate it by creating an explicit direct relationship whenever it is needed. Then you could simulate the extra fields using the model as an abstraction, but I am not sure if this is what you want. If you opt for this kind of solution you can use default values (in your abstraction) for filling in relationships that don't have a instance yet.
Does that explanation make sense to you?
EDIT:
(I have not double checked that the code works, so be vary)
OK, so you have the original models:
class FilmStudio(models.Model):
name = models.CharField(max_length=200, unique=True)
class ProductionCompany(models.Model):
name = models.CharField(max_length=200)
film_studio = models.ForeignKey(FilmStudio)
class Meta:
# Ensure that a given combination of ProductionCompany name and FilmStudio object is unique
unique_together = ('name', 'film_studio')
class Film(models.Model):
title = models.CharField(max_length=200)
production_company = models.ForeignKey(ProductionCompany)
class Actor(models.Model):
name = models.CharField(max_length=200)
films = models.ManyToManyField(Film, blank=True)
# The "solution" would be:
class ActorProductionComapny(models.Model):
production_company = models.ForeignKey(ProductionCompany, related_name='actors')
actor = models.ForeignKey(Actor, related_name='companies')
# your extra fields here
someproperty = models.CharField(max_length=200)
class Meta:
# let's say one per actor
unique_together = ('production_company', 'actor')
This is going to get messy really quickly
We use a F object like this:
FilmStudio.objects.filter(productioncompany__film__actor__name='Samuel L. Jackson',
productioncompany__film__actor=F('actors__actor'),
actors__someproperty="Plays poker with CEO").distinct()
The tricky part is going to be handling default values (i.e. when there is no value) This would have to be implemented using a custom Manager, but then I am out of my depth.
I will try to explain as well as I can, but it's going to be tricky.
If you want to make a filter on the relationship you may have to do something like this:
def filter_prod(pq_query, someproperty, actor_name):
if someproperty == "Default":
# Great, this means we can ignore the parallel relationship:
return pq_query.filter(productioncompany__film__actor__name=actor_name)
else:
# Here comes the hard part
FilmStudio.objects.filter(productioncompany__film__actor__name=actor_name,
productioncompany__film__actor=F('actors__actor'),
actors__someproperty=someproperty).distinct()
The thing I am trying to illustrate here is that there are two kinds of actor-productioncompany relationships, those with custom field values (non-default), and those without.
Now, you can make a custom getter that looks something like this:
class ProductionCompany(models.Model):
name = models.CharField(max_length=200)
film_studio = models.ForeignKey(FilmStudio)
def get_actors(self):
# This one is not lazy, so be aware
actors = list(self.actors)
# Get a list of actor IDs
actor_ids = [a.actor_id for a in actors]
for actor in Actor.objects.filter(films__production_company_id=self.id):
if actor.id not in actor_ids:
actors.append(ActorProductionComapny(actor=actor, production_company=self)
actor_ids.append(actor.id)
return actors
class Meta:
# Ensure that a given combination of ProductionCompany name and FilmStudio object is unique
unique_together = ('name', 'film_studio')
This should not save the relationship to the database until you call .save() on an instance. You can also add a custom save method that ignores/aports .save() calls where all the values are default. Just remember to check if it is a new instance or not, because you don't want it to cancel a "set back to default" call. You could also make it delete on a "set back to default", but check if you are allowed to do that within .save().
For even more complex queries (mix of default and non-default) you have Q-objects (further down on the page from F objects)
In short, you need to create an extra model to store this extra relational data between Actor and FilmStudio.
class Actor(models.Model):
name = models.CharField(max_length=200)
films = models.ManyToManyField(Film, blank=True)
film_studios = models.ManyToMany(FilmStudio, through='ActorFilmStudio')
class ActorFilmStudio(models.Model):
actor = models.ForeignKey(Actor)
film_studio = models.ForeignKey(FilmStudio)
# define extra data fields here
data1 = models.TextField()
data2 = models.IntegerField()
One way to think about this: the data you're trying to store belongs to an Actor-FilmStudio relation, and is not related in anyway to Film or ProductionCompany.
Your existing ability to retrieve the a set of Actors for a given FilmStudio (or vice-versa) does not necessarily imply you can store relational data belonging to these two models using the models defined in your example.
Keep in mind that each of the models you defined in your example are backed by a table in your database. In the case of Actor.films field, Django creates an extra table to store the many-to-many relationship data.
Since you're looking to store relational data between Actor and FilmStudio, you need to consider where the data will be stored in your database. Can you store the data in the Film model? or the ProductionCompany model?

django model field like view

I'd like to have a model field that will return the latest related of another model.
An example-
class Thing(models.Model):
name = models.CharField(max_length=200)
state = models.ForeignKey(State, query=latest) #pure fantasy
class State(models.Model):
change = models.DateTimeField(auto_now_add=True)
thing = models.ForeignKey(Thing)
Assume that the model class State has a many to one relationship to Thing.
given- some_thing = Thing.object.all()[0]
I want some_thing.state to be the instance of State that has the most recent State.change value for the instance Thing that I have in hand.
I'd like to specify at the Django Model level an instance variable that does what I've described above. I can think of two ways to do this, but neither is at this level:
at the actual DB level -- create a view and turn off django db
syncing
(http://stackoverflow.com/questions/507795/can-i-use-a-database-view-as-a-model-in-django)
the view level -- when I create new State row manually set that row to the related Thing.state instance.
You cannot have a database level dynamic foreign key. You can, however, have a property or a method that captures your item for you:
class Thing(models.Model):
name = models.CharField(max_length=200)
#property
def state(self):
return State.objects.latest('change')
class State(models.Model):
change = models.DateTimeField(auto_now_add=True)
thing = models.ForeignKey(Thing)

Copying a Django Field description from an existing Model to a new one

I'm trying to dynamically generate a new Model, based on fields from an existing Model. Both are defined in /apps/main/models.py. The existing model looks something like this:
from django.db import models
class People(models.Model):
name = models.CharField(max_length=32)
age = models.IntegerField()
height = models.IntegerField()
I have a list containing the names of fields that I would like to copy:
target_fields = ["name", "age"]
I want to generate a new model the has all of the Fields named in target_fields, but in this case they should be indexed (db_index = True).
I originally hoped that I would just be able to iterate over the class properties of People and use copy.copy to copy the field descriptions that are defined on it. Like this:
from copy import copy
d = {}
for field_name in target_fields:
old_field = getattr(People, field_name) # alas, AttributeError
new_field = copy(old_field)
new_field.db_index = True
d[field_name] = new_field
IndexedPeople = type("IndexedPeople", (models.Model,), d)
I wasn't sure if copy.copy()ing Fields would work, but I didn't get far enough to find out: the fields listed in the class definition don't aren't actually included as properties on the class object. I assume they're used for some metaclass shenanigans instead.
After poking around in the debugger, I found some type of Field objects listed in People._meta.local_fields. However, these aren't just simple description that can be copy.copy()ed and used to describe another model. For example, they include a .model property referring to People.
How can I create a field description for a new model based on a field of an existing model?
From poking around in the debugger and the source: all Django models use the ModelBase metaclass defined in /db/models/base.py. For each field in a model's class definition, ModelBase's .add_to_class method will call the field's .contribute_to_class method.
Field.contribute_to_class is defined in /db/models/fields/__init__.py and it is what's responsible for associating a field definition with a particular model. The field is modified by adding the .model property and by calling the .set_attributes_from_name method with the name used in the model's class definition. This in turn adds adds the .attname and .column properties and sets .name and .verbose_name if necessary.
When I inspect the __dict__ property of a newly-defined CharField and compare it with that of a CharField that was already associated with a model, I also see that these are the only differences:
The .creation_counter property is unique for each instance.
The .attrname, .column and .model properties do not exist on the new instance.
The .name and .verbose_name properties is None on the new instance.
It doesn't seem possible to distinguish between .name/.verbose_name properties that were manually specified to the constructor and ones that were automatically generated. You'll need to chose either to always reset them, ignoring any manually-specified values, or never clear them, which would cause them to always ignore any new name they were given in the new model. I want to use the same name as the original fields, so I am not going to touch them.
Knowing what differences exist, I am using copy.copy() to clone the existing instance, then apply these changes to make it behave like a new instance.
import copy
from django.db import models
def copy_field(f):
fp = copy.copy(f)
fp.creation_counter = models.Field.creation_counter
models.Field.creation_counter += 1
if hasattr(f, "model"):
del fp.attname
del fp.column
del fp.model
# you may set .name and .verbose_name to None here
return fp
Given this function, I create the new Model with the following:
target_field_name = "name"
target_field = People._meta.get_field_by_name(target_field_name)[0]
model_fields = {}
model_fields["value"] = copy_field(target_field)
model_fields["value"].db_index = True
model_fields["__module__"] = People.__module__
NewModel = type("People_index_" + field_name, (models.Model,), model_fields)
It works!
Solution
There is build in way for fields copying Field.clone() - method which deconstructs field removing any model dependent references:
def clone(self):
"""
Uses deconstruct() to clone a new copy of this Field.
Will not preserve any class attachments/attribute names.
"""
name, path, args, kwargs = self.deconstruct()
return self.__class__(*args, **kwargs)
So you can use following util to copy fields ensuring that you'll not accidentally affect source fields of model you're copying from:
def get_field(model, name, **kwargs):
field = model._meta.get_field(name)
field_copy = field.clone()
field_copy.__dict__.update(kwargs)
return field_copy
Also can pass some regular kwargs like verbose_name and etc:
def get_field_as_nullable(*args, **kwargs):
return get_field(*args, null=True, blank=True, **kwargs)
Does not work for m2m fields inside of model definition. (m2m.clone() on model definition raises AppRegistryNotReady: Models aren't loaded yet)
Why this instead of abstract models?
Well, depends on case. Some times you don't need inheristance but actuall fields copying. When? For example:
I have a User model and model which represents an application (document for user data update request) for user data update:
class User(models.Model):
first_name = ...
last_name = ...
email = ...
phone_number = ...
birth_address = ...
sex = ...
age = ...
representative = ...
identity_document = ...
class UserDataUpdateApplication(models.Model):
# This application must ONLY update these fields.
# These fiends must be absolute copies from User model fields.
user_first_name = ...
user_last_name = ...
user_email = ...
user_phone_number = ...
So, i shouldn't carry out duplicated fields from my User model to abstract class due to the fact that some other non-user-logic-extending model wants to have exact same fields. Why? Because it's not directly related to User model - User model shouldn't care what depends on it (excluding cases when you want to extend User model), so it shouldn't be separated due to fact that some other model with it's own non User related logic want's to have exact same fields.
Instead you can do this:
class UserDataUpdateApplication(models.Model):
# This application must ONLY update these fields.
user_first_name = get_field(User, 'first_name')
user_last_name = get_field(User, 'last_name')
user_email = get_field(User, 'user_email')
user_phone_number = get_field(User, 'phone_number')
You also would make som util which would generate some abc class "on fly" to avoid code duplication:
class UserDataUpdateApplication(
generate_abc_for_model(
User,
fields=['first_name', 'last_name', 'email', 'phone_number'],
prefix_fields_with='user_'),
models.Model,
):
pass

Filtering reverse forein key set in django

class item(models.Model):
name = models.CharField()
class itemTxns(models.Model):
item = models.ForeignKey(item)
txnDate = models.DateField()
txn = models.CharField()
Given an "item" object, how do I find the most recent itemTxn associated with that item?
I know I have access to item.itemTxns_set, but is it possible to query that without explicitly calling another get on the itemTxns class?
item.itemtxns_set.latest('txnDate')
I'm not sure how you'll get around calling it.
You could make it a property of the item class.
class item(models.Model):
#....
#property
def latest_itemtxns(self):
return self.itemtxns_set.latest('txndate')
By the way I recommend you capitalize your classes to differentiate between instances, variables, and classes.