I'm trying to provide for users possibility to store in SomeModel a list of ExtraParameters of any number or kind (it may be something small like IntegerField, BooleanField or quite large like TextField).
I tried to implement ExtraParameter abstract model class, that will keep ForeignKey to SomeModel, and also its child classes with only one parameter like:
class ExtraParameter(models.Model):
somemodel = models.ForeignKey(SomeModel, ...)
class Meta:
abstract = True
class IntegerExtraParameter(ExtraParameter):
value = models.IntegerField()
I believe it takes multiple small classes like this one so it could be migrated to multiple database tables of different fields.
Am I right? Please provide better solution. Maybe other way to decorate ExtraParameter is possible?
The problem with this approach is while implementing template it is not so easy to get all the stored parameters of all kind by doing somemodel.extraparameters.all(), rather I need to call every child class explicitly and build set from it. But also I've seen a solution with finding all subclasses of any class inside app's config, so it would help.
Jakub
I think the answer you are looking for is covered in great depth in here:
https://stackoverflow.com/a/7934577/
Personally I like the way Django and JsonField works together.
https://docs.djangoproject.com/en/3.1/ref/models/fields/#django.db.models.JSONField
JsonField would give you vast flexibility with dynamic metadata.
In this example I'm simply attaching a JsonField to those models that are in need of dynamic meta data. extra_information could then hold data about what kind of values is in relation with the parent model.
class ExtraInformationModel(models.Model):
# Abstract model that adds json field for additional meta data
extra_information = JSONField()
class Meta:
abstract = True
class SomeModel(ExtraInformationModel):
# extra_information field comes from the abstract model
...usual model stuff..
Making queries with JsonFields is as easy as using Django orm usually is.
https://docs.djangoproject.com/en/3.1/topics/db/queries/#querying-jsonfield
(example from Django documentation)
Dog.objects.create(name='Rufus', data={
'breed': 'labrador',
'owner': {
'name': 'Bob',
'other_pets': [{
'name': 'Fishy',
}],
},
})
Dog.objects.create(name='Meg', data={'breed': 'collie', 'owner': None})
Dog.objects.filter(data__breed='collie')
Related
I'm exposing an REST api for legacy application.
I have a Company model class that defines the following fields:
address_street (required)
address_street_number (required)
shipping_address_street (optional)
shipping_address_street_number (optional)
billing_address_street (optional)
... you got the point
I would like to group all address fields into an Adress serializer in order to have a cleaner structure.
Some thing like:
{
"adress": {"street": "foo", "street_number": "bar"},
"shipping_address": {"street": "ham", "street_number": "spam"},
"billing_address": null,
}
So far, I can create a CompanySerializer from a rest_framework.serializers.Serializer and manually build my Company objects from this.
It is tedious, but it will work.
But how can I build a rest_framework.serializers.ModelSerializer for my Company model, changing the way fields are structured to have my model fields automatically populated by rest framework ?
DRF nested model serializers seems to only work for relations, not groups of fields.
Am I to build my models instances by hands or is there a way to decouple the representation of a model serializer to my object ?
From ModelSerializer documentation
The process of automatically determining a set of serializer fields
based on the model fields is reasonably complex
You probably should stick to the "tedious" method you mention (you will have to put in some effort if serializer representation and model fields have different structures altogether).
ModelSerializer is tightly linked to the model in question, so overriding that behaviour seems to be for little benefit when you can do the same thing using a plain Serializer and put object creation under save.
Maybe you need to override the data property/method on the Serializer subclass so that you get a dict that is fit for consumption directly by the model, that might make it less tedious
You can build custom serialiser fields with SerializerMethodField:
from rest_framework.fields import SerializerMethodField
class AdressSerializer(ModelSerializer):
adress = SerializerMethodField()
shipping_address = SerializerMethodField()
def get_adress(self, instance):
return {
"street": instance.address_street,
"street_number": instance.address_street_number
}
def get_shipping_address(self, instance):
// same approach as above
If needed to populate the model from the same data representation, the best approach is to override serialiser's save method. I don't think there is an "automatic" way of doing it.
I would like to get a variable in my meta class
class Meta:
abstract = True
db_table = 'bikes_table'
managed = False
I want to get a variable from self like above :
class Meta:
abstract = True
db_table = 'bikes_table' if self.customers=True else 'cars_table'
managed = False
Is it possible de get variable ?
Thank you
As stated in the comments, the surface problem is that "self" won't be available at class definition time, of course.
Some of the fields in the Metadata could have values that could change at runtime, for each instance, those will accept callback functions instead of static values (but looking at the documentation, there seems to be none)
Now, the table corresponding to a certain Model is used by the ORM and will be bound at the class level. Doubly so because a lot of the queries dealing with a class have to issue database statements without having an instance, as is the case for any query. I mean, in your example, a vehicle.objects.all() would have to query both database tables and bring the results together in the same response.
It could be done by subclassing Django's internal Queryset databases, and carefully rewriting it to look at the "table" model meta data in a custom way, but it would be prone to error.
So, you'd rather rethink your design. It is easier to simply have a Vehicle base class, just use "Vehicle" on your forms and relations, and derive the Bicycle and Car classes from that - either a car or a bicycle could be used wherever a vehicle can. Django provides support for the relationship itself with Generic Relations
Say you're a publisher and have multiple types of article.
Standard article, sponsored article, and review
And sponsored article has some sponsored-only fields, say sponsor and promotion_end_date
And review has some other review only fields, like product return address
How would you design this?
I've come across this problem a few times, and I always end up doing it in one model with all the fields available but not required.
But it just feels bad because it leaves room for mistakes with the administrators. What if they fill in product return address but its not a review? etc.
And most of the time you want it in one model, to query the very similar objects together since 90% of the fieldset is the same between them. ANd its very helpful for things like finding most popular
EDIT:
These models will almost always be queried together. So it makes no sense to put them in different tables and then work around that.
They should be in the same table and indexed.
Otherwise every single request would be doing 3x queries, not one. And then having to merge and sort the queries computationally afterwards.
You can always have one base model which can be inherited by other subsequent models that you named. In your case.
class BaseArticle(models.Model):
class Meta:
abstract = True
fields that are common to the models that will be inheriting this
class StandardArticle(BaseArticle):
fields specific to StandardArticle
You can have your other models in same fashion as StandardArticle
Use abstract base classes:
class ArticleBase(models.Model):
# fields
class Meta:
abstract = True
class Article(ArticleBase):
pass
class SponsoredArticle(ArticleBase):
# additional fields
class Review(ArticleBase):
# additional fields
Or multi-table inheritance:
class Article(models.Model):
# fields
class SponsoredArticle(Article):
# additional fields
class Review(Article):
# additional fields
In the latter case you can query all articles with Article.objects.select_related('sponsoredarticle', 'review').all() (you may create custom manager to avoid typing select_related(...) each time). select_related() is necessary to avoid DB query when accessing child class, e. g. aricle.review.
I've searched around stack overflow for an answer to this (probably simple) question, but most of the solutions I see seem overly complicated and hard to understand.
I have a model "Post" which is an abstract base class. Models "Announcement" and "Event" inherit from Post.
Right now I'm keeping related lists of Events and Announcements in other models. For instance, I have "removed_events" and "removed_announcements" fields in another model.
However, in my project, "removed_events" and "removed_announcements" are treated exactly the same way. There is no need to disambiguate between a "removed event" and a "removed announcement." In other words, a field keeping track of "removed_posts" would be sufficient.
I don't know how to (or perhaps can't) create a field "removed_posts," since Post is abstract. However, right now I feel like I'm repeating myself in the code (and having to do a lot of clutter-some checks to figure out whether the post I'm looking at is an event or an announcement and add it to the appropriate removed field).
What is the best option here? I could make Posts non-abstract, but Post objects themselves should never be created, and I don't think I can enforce this on a non-abstract object.
My understanding of databases is weak, but I'm under the impression that making Post non-abstract would complicate the database due to joins. Is this a big deal?
Finally, there are other fields in other models where I'd like to condense things that amount to an event_list and an announcement_list into a post_list, but those fields do need to be disambiguated. I could filter the post_list based on post type, but the call to filter() would be slower than being able to directly access the event and announcement lists separately, wouldn't it? Any suggestions here?
Thanks a ton for reading through this.
There are two kinds of model subclassing in Django - Abstract Base Classes; and Multi-Table inheritance.
Abstract Base Classes aren't ever used by themselves, and do not have a database table or any form of identification. They are simply a way of shortening code, by grouping sets of common fields in code, not in the database.
For example:
class Address(models.Model):
street = ...
city = ...
class Meta:
abstract = True
class Employee(Address):
name = ...
class Employer(Address):
employees = ...
company_name = ...
This is a contrived example, but as you can see, an Employee isn't an Address, and neither is an Employer. They just both contain fields relating to an address. There are only two tables in this example; Employee, and Employer - and both of them contain all the fields of Address. An employer address can not be compared to an employee address at the database level - an address doesn't have a key of its own.
Now, with multi-table inheritance, (remove the abstract=True from Address), Address does have a table all to itself. This will result in 3 distinct tables; Address, Employer, and Employee. Both Employer and Employee will have a unique foreign key (OneToOneField) back to Address.
You can now refer to an Address without worrying about what type of address it is.
for address in Address.objects.all():
try:
print address.employer
except Employer.DoesNotExist: # must have been an employee
print address.employee
Each address will have its own primary key, which means it can be saved in a fourth table on its own:
class FakeAddresses(models.Model):
address = models.ForeignKey(Address)
note = ...
Multi-table Inheritance is what you're after, if you need to work with objects of type Post without worrying about what type of Post it is. There will be an overhead of a join if accessing any of the Post fields from the subclass; but the overhead will be minimal. It is a unique index join, which should be incredibly quick.
Just make sure, that if you need access to the Post, that you use select_related on the queryset.
Events.objects.select_related(depth=1)
That will avoid additional queries to fetch the parent data, but will result in the join occurring. So only use select related if you need the Post.
Two final notes; if a Post can be both an Announcement AND an Event, then you need to do the traditional thing, and link to Post via a ForeignKey. No subclassing will work in this case.
The last thing is that if the joins are performance critical between the parent and the children, you should use abstract inheritance; and use Generic Relations to refer to the abstract Posts from a table that is much less performance critical.
Generic Relations essentially store data like this:
class GenericRelation(models.Model):
model = ...
model_key = ...
DeletedPosts(models.Model):
post = models.ForeignKey(GenericRelation)
That will be a lot more complicated to join in SQL (django helps you with that), but it will also be less performant than a simple OneToOne join. You should only need to go down this route if the OneToOne joins are severely harming performance of your application which is probably unlikely.
Generic relationships and foreign keys are your friend in your path to succeed. Define an intermediate model where one side is generic, then the other side will get a related list of polymorphic models. It's just a little more complicated than a standard m2m join model, in that the generic side has two columns, one to ContentType (actually a FK) and the other to the PK of the actual linked model instance. You can also restrict the models to be linked with using standard FK parameters.
You'll get used with it quickly.
(now that I get an actual keyboard to write with, here there is the example:)
class Post(models.Model):
class Meta: abstract = True
CONCRETE_CLASSES = ('announcement', 'event',)
removed_from = generic.GenericRelation('OwnerRemovedPost',
content_type_field='content_type',
object_id_field='post_id',
)
class Announcement(Post): pass
class Event(Post): pass
class Owner(models.Model):
# non-polymorphic m2m
added_events = models.ManyToManyField(Event, null=True)
# polymorphic m2m-like property
def removed_posts(self):
# can't use ManyToManyField with through.
# can't return a QuerySet b/c it would be a union.
return [i.post for i in self.removed_post_items.all()]
def removed_events(self):
# using Post's GenericRelation
return Event.objects.filter(removed_from__owner=self)
class OwnerRemovedPost(models.Model):
content_type = models.ForeignKey(ContentType,
limit_choices_to={'name__in': Post.CONCRETE_CLASSES},
)
post_id = models.PositiveIntegerField()
post = generic.GenericForeignKey('content_type', 'post_id')
owner = models.ForeignKey(Owner, related_name='removed_post_items')
class Meta:
unique_together = (('content_type', 'post_id'),) # to fake FK constraint
You can't filter into the related collection like a classic many-to-many, but with the proper methods in Owner, and using the concrete classes' managers smartly, you get everywhere you want.
This is a problem concerning django.
I have a model say "Automobiles". This will have some basic fields like "Color","Vehicle Owner Name", "Vehicle Cost".
I want to provide a form where the user can add extra fields depending on the automobile that he is adding. For example, if the user is adding a "Car", he will extra fields in the form, dynamically at run time, like "Car Milage", "Cal Manufacturer".
Suppose if the user wants to add a "Truck", he will add "Load that can be carried", "Permit" etc.
How do I achieve this in django?
There are two questions here:
How to provide a form where the user can add new fields at run time?
How to add the fields to the database so that it can be retrieved/queried later?
There are a few approaches:
key/value model (easy, well supported)
JSON data in a TextField (easy, flexible, can't search/index easily)
Dynamic model definition (not so easy, many hidden problems)
It sounds like you want the last one, but I'm not sure it's the best for you. Django is very easy to change/update, if system admins want extra fields, just add them for them and use south to migrate. I don't like generic key/value database schemas, the whole point of a powerful framework like Django is that you can easily write and rewrite custom schemas without resorting to generic approaches.
If you must allow site users/administrators to directly define their data, I'm sure others will show you how to do the first two approaches above. The third approach is what you were asking for, and a bit more crazy, I'll show you how to do. I don't recommend using it in almost all cases, but sometimes it's appropriate.
Dynamic models
Once you know what to do, this is relatively straightforward. You'll need:
1 or 2 models to store the names and types of the fields
(optional) An abstract model to define common functionality for your (subclassed) dynamic models
A function to build (or rebuild) the dynamic model when needed
Code to build or update the database tables when fields are added/removed/renamed
1. Storing the model definition
This is up to you. I imagine you'll have a model CustomCarModel and CustomField to let the user/admin define and store the names and types of the fields you want. You don't have to mirror Django fields directly, you can make your own types that the user may understand better.
Use a forms.ModelForm with inline formsets to let the user build their custom class.
2. Abstract model
Again, this is straightforward, just create a base model with the common fields/methods for all your dynamic models. Make this model abstract.
3. Build a dynamic model
Define a function that takes the required information (maybe an instance of your class from #1) and produces a model class. This is a basic example:
from django.db.models.loading import cache
from django.db import models
def get_custom_car_model(car_model_definition):
""" Create a custom (dynamic) model class based on the given definition.
"""
# What's the name of your app?
_app_label = 'myapp'
# you need to come up with a unique table name
_db_table = 'dynamic_car_%d' % car_model_definition.pk
# you need to come up with a unique model name (used in model caching)
_model_name = "DynamicCar%d" % car_model_definition.pk
# Remove any exist model definition from Django's cache
try:
del cache.app_models[_app_label][_model_name.lower()]
except KeyError:
pass
# We'll build the class attributes here
attrs = {}
# Store a link to the definition for convenience
attrs['car_model_definition'] = car_model_definition
# Create the relevant meta information
class Meta:
app_label = _app_label
db_table = _db_table
managed = False
verbose_name = 'Dynamic Car %s' % car_model_definition
verbose_name_plural = 'Dynamic Cars for %s' % car_model_definition
ordering = ('my_field',)
attrs['__module__'] = 'path.to.your.apps.module'
attrs['Meta'] = Meta
# All of that was just getting the class ready, here is the magic
# Build your model by adding django database Field subclasses to the attrs dict
# What this looks like depends on how you store the users's definitions
# For now, I'll just make them all CharFields
for field in car_model_definition.fields.all():
attrs[field.name] = models.CharField(max_length=50, db_index=True)
# Create the new model class
model_class = type(_model_name, (CustomCarModelBase,), attrs)
return model_class
4. Code to update the database tables
The code above will generate a dynamic model for you, but won't create the database tables. I recommend using South for table manipulation. Here are a couple of functions, which you can connect to pre/post-save signals:
import logging
from south.db import db
from django.db import connection
def create_db_table(model_class):
""" Takes a Django model class and create a database table, if necessary.
"""
table_name = model_class._meta.db_table
if (connection.introspection.table_name_converter(table_name)
not in connection.introspection.table_names()):
fields = [(f.name, f) for f in model_class._meta.fields]
db.create_table(table_name, fields)
logging.debug("Creating table '%s'" % table_name)
def add_necessary_db_columns(model_class):
""" Creates new table or relevant columns as necessary based on the model_class.
No columns or data are renamed or removed.
XXX: May need tweaking if db_column != field.name
"""
# Create table if missing
create_db_table(model_class)
# Add field columns if missing
table_name = model_class._meta.db_table
fields = [(f.column, f) for f in model_class._meta.fields]
db_column_names = [row[0] for row in connection.introspection.get_table_description(connection.cursor(), table_name)]
for column_name, field in fields:
if column_name not in db_column_names:
logging.debug("Adding field '%s' to table '%s'" % (column_name, table_name))
db.add_column(table_name, column_name, field)
And there you have it! You can call get_custom_car_model() to deliver a django model, which you can use to do normal django queries:
CarModel = get_custom_car_model(my_definition)
CarModel.objects.all()
Problems
Your models are hidden from Django until the code creating them is run. You can however run get_custom_car_model for every instance of your definitions in the class_prepared signal for your definition model.
ForeignKeys/ManyToManyFields may not work (I haven't tried)
You will want to use Django's model cache so you don't have to run queries and create the model every time you want to use this. I've left this out above for simplicity
You can get your dynamic models into the admin, but you'll need to dynamically create the admin class as well, and register/reregister/unregister appropriately using signals.
Overview
If you're fine with the added complication and problems, enjoy! One it's running, it works exactly as expected thanks to Django and Python's flexibility. You can feed your model into Django's ModelForm to let the user edit their instances, and perform queries using the database's fields directly. If there is anything you don't understand in the above, you're probably best off not taking this approach (I've intentionally not explained what some of the concepts are for beginners). Keep it Simple!
I really don't think many people need this, but I have used it myself, where we had lots of data in the tables and really, really needed to let the users customise the columns, which changed rarely.
Database
Consider your database design once more.
You should think in terms of how those objects that you want to represent relate to each other in the real world and then try to generalize those relations as much as you can, (so instead of saying each truck has a permit, you say each vehicle has an attribute which can be either a permit, load amount or whatever).
So lets try it:
If you say you have a vehicle and each vehicle can have many user specified attributes consider the following models:
class Attribute(models.Model):
type = models.CharField()
value = models.CharField()
class Vehicle(models.Model):
attribute = models.ManyToMany(Attribute)
As noted before, this is a general idea which enables you to add as much attributes to each vehicle as you want.
If you want specific set of attributes to be available to the user you can use choices in the Attribute.type field.
ATTRIBUTE_CHOICES = (
(1, 'Permit'),
(2, 'Manufacturer'),
)
class Attribute(models.Model):
type = models.CharField(max_length=1, choices=ATTRIBUTE_CHOICES)
value = models.CharField()
Now, perhaps you would want each vehicle sort to have it's own set of available attributes. This can be done by adding yet another model and set foreign key relations from both Vehicle and Attribute models to it.
class VehicleType(models.Model):
name = models.CharField()
class Attribute(models.Model):
vehicle_type = models.ForeigngKey(VehicleType)
type = models.CharField()
value = models.CharField()
class Vehicle(models.Model):
vehicle_type = models.ForeigngKey(VehicleType)
attribute = models.ManyToMany(Attribute)
This way you have a clear picture of how each attribute relates to some vehicle.
Forms
Basically, with this database design, you would require two forms for adding objects into the database. Specifically a model form for a vehicle and a model formset for attributes. You could use jQuery to dynamically add more items on the Attribute formset.
Note
You could also separate Attribute class to AttributeType and AttributeValue so you don't have redundant attribute types stored in your database or if you want to limit the attribute choices for the user but keep the ability to add more types with Django admin site.
To be totally cool, you could use autocomplete on your form to suggest existing attribute types to the user.
Hint: learn more about database normalization.
Other solutions
As suggested in the previous answer by Stuart Marsh
On the other hand you could hard code your models for each vehicle type so that each vehicle type is represented by the subclass of the base vehicle and each subclass can have its own specific attributes but that solutions is not very flexible (if you require flexibility).
You could also keep JSON representation of additional object attributes in one database field but I am not sure this would be helpfull when querying attributes.
Here is my simple test in django shell- I just typed in and it seems work fine-
In [25]: attributes = {
"__module__": "lekhoni.models",
"name": models.CharField(max_length=100),
"address": models.CharField(max_length=100),
}
In [26]: Person = type('Person', (models.Model,), attributes)
In [27]: Person
Out[27]: class 'lekhoni.models.Person'
In [28]: p1= Person()
In [29]: p1.name= 'manir'
In [30]: p1.save()
In [31]: Person.objects.a
Person.objects.aggregate Person.objects.all Person.objects.annotate
In [32]: Person.objects.all()
Out[33]: [Person: Person object]
It seems very simple- not sure why it should not be a considered an option- Reflection is very common is other languages like C# or Java- Anyway I am very new to django things-
Are you talking about in a front end interface, or in the Django admin?
You can't create real fields on the fly like that without a lot of work under the hood. Each model and field in Django has an associated table and column in the database. To add new fields usually requires either raw sql, or migrations using South.
From a front end interface, you could create pseudo fields, and store them in a json format in a single model field.
For example, create an other_data text field in the model. Then allow users to create fields, and store them like {'userfield':'userdata','mileage':54}
But I think if you're using a finite class like vehicles, you would create a base model with the basic vehicle characteristics, and then create models that inherits from the base model for each of the vehicle types.
class base_vehicle(models.Model):
color = models.CharField()
owner_name = models.CharField()
cost = models.DecimalField()
class car(base_vehicle):
mileage = models.IntegerField(default=0)
etc