ManyToManyField and South migration - django

I have user profile model with M2M field
class Account(models.Model):
...
friends = models.ManyToManyField('self', symmetrical=True, blank=True)
...
Now I need to know HOW and WHEN add each other as a FRIEND
And I created a model for that
class Account(models.Model):
...
friends = models.ManyToManyField('self', symmetrical=False, blank=True, through="Relationship")
...
class Relationship(models.Model):
""" Friends """
from_account = models.ForeignKey(Account, related_name="relationship_set_from_account")
to_account = models.ForeignKey(Account, related_name="relationship_set_to_account")
# ... some special fields for friends relationship
class Meta:
db_table = "accounts_account_friends"
unique_together = ('from_account','to_account')
Should I create any migration for this changes or not ?
If you have any suggestions you are feel free write their here.
Thanks
PS: accounts_account table already contain records

First off, I'd avoid using the db_table alias if you can. This makes it harder to understand the table structure, as it is no longer in sync with the models.
Secondly, the South API offers functions like db.rename_table(), which can be used by manually editing the migration file. You can rename the accounts_account_friends table to accounts_relation (as Django would name it by default), and add the additional columns.
This combined gives you the following migration:
def forwards(self, orm):
# the Account.friends field is a many-to-many field which got a through= option now.
# Instead of dropping+creating the table (or aliasing in Django),
# rename it, and add the required columns.
# Rename table
db.delete_unique('accounts_account_friends', ['from_account', 'to_account'])
db.rename_table('accounts_account_friends', 'accounts_relationship')
# Add extra fields
db.add_column('accounts_relationship', 'some_field', ...)
# Restore unique constraint
db.create_unique('accounts_relationship', ['from_account', 'to_account'])
def backwards(self, orm):
# Delete columns
db.delete_column('accounts_relationship', 'some_field')
db.delete_unique('accounts_relationship', ['from_account', 'to_account'])
# Rename table
db.rename_table('accounts_relationship', 'accounts_account_friends')
db.create_unique('accounts_account_friends', ['from_account', 'to_account'])
models = {
# Copy this from the final-migration.py file, see below
}
The unique relation is removed, and recreated so the constraint has the proper name.
The add column statements are easily generated with the following trick:
Add the Relationship model in models.py with foreign key fields only, and no changes to the M2M field yet.
Migrate to it
Add the fields to the Relationship model.
Do a ./manage.py schemamigration app --auto --stdout | tee final-migration.py | grep column
Revert the first migration.
Then you have everything you need to construct the migration file.

The way you've got it coded there, you're manually defining a model which does the same job as the m2m join table that Django will have automatically created for you. The thing is, the automatically created table will be called accounts_relationship_friend.
So, what you're doing there will create a model that tries to duplicate what the ORM has done under the surface, but it's pointing at the wrong table.
If you don't need an explicit join model, I would leave remove it from your codebase and not create a migration to add it, and instead use the M2M to find relationships between friends. (I'm not thinking about this too deeply, but it should work).
If, however, you want to do something special with the Relationship model you have (eg store attributes about the type of relationship, etc), I would declare the Relationship model to be the through model you use in your Friend.friends m2m definition. See the docs here.

Related

How django creates internal tables dynamically for many-to-many fields

My model :
class Image(models.Model):
name=models.CharField(max_length=40,unique=True,help_text="name of the image")
tags = models.ManyToManyField(Tag)
class Tag(models.Model):
tag = models.CharField(max_length=100,unique=True)
here when I do makemigrations and migrate it is creating 3 tables inside my database 1.image 2.tag 3.image_tags table
so, my question is i am not specifying image_tags table in my models.py file ,from where django is creating image_tags table and what is the flow ??
I have checked in migrations file but I didnot get any clarity regarding this
An intermediary table is required for a Many-To-Many relationship in a database, and because most of the time you don't need to store extra data on the relationship, Django just silently creates this table for you. In your case it will create a table with 3 fields: id, image_id, tag_id.
If you want to specify your own intermediary table, for example if you want to store extra data, you can create a model with ForeignKey's to your related tables and then define your ManyToManyField with a "through" argument like so:
class ImageTag(models.Model):
image = models.ForeignKey('Image')
tag = models.ForeignKey('Tag')
extra_data = models.CharField()
class Tag(models.Model):
name = models.CharField()
class Image(models.Model):
name = models.CharField()
tags = models.ManyToManyField(Tag, through=ImageTag)
Actually, that is not Django Functionality. It's SQL functionality. SQL is creating the internal table. Because SQL can't create reference, it's not Foreign key. That's why this Bridging table concept is emerged. It will resolve the problem as well as it will hold data(ID) of both tables and few custom fields, depends on requirements.
Updated with new req:
Refer this:
https://gist.github.com/jacobian/827937
Django 1.8 - Intermediary Many-to-Many-Through Relationship - What is the consequence of where 'ManytoManyField' is used?
From my point of view, It's not good practice to customize the bridging table. You can specify the extra fields to any of two tables orelse create a new table make it as foreign key

Django m2m field. Through model with multiple foreign keys (fk) to same model type

I wanted all instances in all tables to have an object instance. A one to one primary key field looked like a good way to do this. Like a small example below.
from util.fields import BigAutoField,BigForeignKey
from django.db import models
class Common_document(models.Model):
pk = models.OneToOneField("Type_object", primary_key=True, related_name="common_document_content_pk")
author = models.ManyToManyField("Type_object", related_name = 'common_document_author',
limit_choices_to = {"Common_document_author_author":"Type_object"} ,null = True,
through = 'Common_document_author', blank = True)
#....
class Common_document_author(models.Model):
pk = models.OneToOneField("Type_object", primary_key=True, related_name="common_document_author_pk")
document = BigForeignKey("Common_document", related_name='Common_document_author_document')
author = BigForeignKey("Type_object", related_name='Common_document_author_author')
class Type_object(models.Model):
id = BigAutoField(primary_key=True)
#....
# Some of the fields are m2m
However this gave the following error:
django.core.management.base.CommandError: One or more models did not validate:
schema.common_document: Intermediary model Common_document_author has more than
one foreign key to Type_object, which is ambiguous and is not permitted.
This error is removed if I comment out the pk field in the document_author table. I guess the error comes because django is not sure witch object FK to use. How do i fix this? Is there a way to tell django which field in the m2m table to be used in the m2m field?
I am probably not going to do it like this. m2m tables are probably not going to need to have an object instance, but I would still like to know how to do this.
I guess I don't understand you motivation. Why do you want to use a foreign key as your primary index? Sure, index it, but primary?. You might also try changing its name from 'pk', I am sure Django makes assumptions about the field called 'pk'.

django admin many-to-many intermediary models using through= and filter_horizontal

This is how my models look:
class QuestionTagM2M(models.Model):
tag = models.ForeignKey('Tag')
question = models.ForeignKey('Question')
date_added = models.DateTimeField(auto_now_add=True)
class Tag(models.Model):
description = models.CharField(max_length=100, unique=True)
class Question(models.Model):
tags = models.ManyToManyField(Tag, through=QuestionTagM2M, related_name='questions')
All I really wanted to do was add a timestamp when a given manytomany relationship was created. It makes sense, but it also adds a bit of complexity. Apart from removing the .add() functionality [despite the fact that the only field I'm really adding is auto-created so it technically shouldn't interfere with this anymore]. But I can live with that, as I don't mind doing the extra QuestionTagM2M.objects.create(question=,tag=) instead if it means gaining the additional timestamp functionality.
My issue is I really would love to be able to preserve my filter_horizontal javascript widget in the admin. I know the docs say I can use an inline instead, but this is just too unwieldy because there are no additional fields that would actually be in the inline apart from the foreign key to the Tag anyway.
Also, in the larger scheme of my database schema, my Question objects are already displayed as an inline on my admin page, and since Django doesn't support nested inlines in the admin [yet], I have no way of selecting tags for a given question.
Is there any way to override formfield_for_manytomany(self, db_field, request=None, **kwargs) or something similar to allow for my usage of the nifty filter_horizontal widget and the auto creation of the date_added column to the database?
This seems like something that django should be able to do natively as long as you specify that all columns in the intermediate are automatically created (other than the foreign keys) perhaps with auto_created=True? or something of the like
There are ways to do this
As provided by #obsoleter in the comment below : set QuestionTagM2M._meta.auto_created = True and deal w/ syncdb matters.
Dynamically add date_added field to the M2M model of Question model in models.py
class Question(models.Model):
# use auto-created M2M model
tags = models.ManyToMany(Tag, related_name='questions')
# add date_added field to the M2M model
models.DateTimeField(auto_now_add=True).contribute_to_class(
Question.tags.through, 'date_added')
Then you could use it in admin as normal ManyToManyField.
In Python shell, use Question.tags.through to refer the M2M model.
Note, If you don't use South, then syncdb is enough; If you do, South does not like
this way and will not freeze date_added field, you need to manually write migration to add/remove the corresponding column.
Customize ModelAdmin:
Don't define fields inside customized ModelAdmin, only define filter_horizontal. This will bypass the field validation mentioned in Irfan's answer.
Customize formfield_for_dbfield() or formfield_for_manytomany() to make Django admin to use widgets.FilteredSelectMultiple for the tags field.
Customize save_related() method inside your ModelAdmin class, like
def save_related(self, request, form, *args, **kwargs):
tags = form.cleaned_data.pop('tags', ())
question = form.instance
for tag in tags:
QuestionTagM2M.objects.create(tag=tag, question=question)
super(QuestionAdmin, self).save_related(request, form, *args, **kwargs)
Also, you could patch __set__() of the ReverseManyRelatedObjectsDescriptor field descriptor of ManyToManyField for date_added to save M2M instance w/o raise exception.
From https://docs.djangoproject.com/en/dev/ref/contrib/admin/#working-with-many-to-many-intermediary-models
When you specify an intermediary model using the through argument to a ManyToManyField, the admin will not display a widget by default. This is because each instance of that intermediary model requires more information than could be displayed in a single widget, and the layout required for multiple widgets will vary depending on the intermediate model.
However, you can try including the tags field explicitly by using fields = ('tags',) in admin. This will cause this validation exception
'QuestionAdmin.fields' can't include the ManyToManyField field 'tags' because 'tags' manually specifies a 'through' model.
This validation is implemented in https://github.com/django/django/blob/master/django/contrib/admin/validation.py#L256
if isinstance(f, models.ManyToManyField) and not f.rel.through._meta.auto_created:
raise ImproperlyConfigured("'%s.%s' "
"can't include the ManyToManyField field '%s' because "
"'%s' manually specifies a 'through' model." % (
cls.__name__, label, field, field))
I don't think that you can bypass this validation unless you implement your own custom field to be used as ManyToManyField.
The docs may have changed since the previous answers were posted. I took a look at the django docs link that #Irfan mentioned and it seems to be a more straight forward then it used to be.
Add an inline class to your admin.py and set the model to your M2M model
class QuestionTagM2MInline(admin.TabularInline):
model = QuestionTagM2M
extra = 1
set inlines in your admin class to contain the Inline you just defined
class QuestionAdmin(admin.ModelAdmin):
#...other stuff here
inlines = (QuestionTagM2MInline,)
Don't forget to register this admin class
admin.site.register(Question, QuestionAdmin)
After doing the above when I click on a question I have the form to do all the normal edits on it and below that are a list of the elements in my m2m relationship where I can add entries or edit existing ones.

Creating Custom User Backends in Django

I have a Shops model and would like each shop to be able to login to my application. Following as best I can the guide at http://scottbarnham.com/blog/2008/08/21/extending-the-django-user-model-with-inheritance/ and various other googlings, I've got part of the way there, but I've run into a problem. When I try to login as a shop, I get the following error:
OperationalError at /login/
(1054, "Unknown column 'shops.user_ptr_id' in 'field list'")
Shops model:
class Shops(User):
shop_id = models.AutoField(primary_key=True)
shop_code = models.CharField(unique=True, max_length=5)
shop_type_fk = models.ForeignKey(ShopTypes,
null=True,
db_column='shop_type_id',
blank=True)
address_fk = models.ForeignKey(Addresses, db_column='address_id')
phone_number = models.CharField(max_length=30)
#email = models.EmailField(max_length=255, blank=True)
description = models.TextField(blank=True)
does_gift_aid = models.NullBooleanField(null=True, blank=True)
objects = UserManager()
class Meta:
db_table = u'shops'
I've sync'd the database, so surely it should have made the column user_ptr_id. Does anyone know where I'm going wrong?
"I've sync'd the database, so surely it should have made the column user_ptr_id."
What makes you think that? Especially in light of this clear statement in the docs for syncdb:
Syncdb will not alter existing tables
syncdb will only create tables for
models which have not yet been
installed. It will never issue ALTER
TABLE statements to match changes made
to a model class after installation.
Changes to model classes and database
schemas often involve some form of
ambiguity and, in those cases, Django
would have to guess at the correct
changes to make. There is a risk that
critical data would be lost in the
process.
If you have made changes to a model
and wish to alter the database tables
to match, use the sql command to
display the new SQL structure and
compare that to your existing table
schema to work out the changes.
It does sound like you had an existing shops table before changing it to inherit from User (as Daniel notes), and syncdb does not update the schema for existing tables.
You need to drop the table and then run syncdb, if possible. Otherwise you need to go into your database and add the user_ptr_id field manually, if you know how to do that. The definition should look something like this:
"user_ptr_id" integer NOT NULL UNIQUE REFERENCES "auth_user" ("id")

Creation of dynamic model fields in django

This is a problem concerning django.
I have a model say "Automobiles". This will have some basic fields like "Color","Vehicle Owner Name", "Vehicle Cost".
I want to provide a form where the user can add extra fields depending on the automobile that he is adding. For example, if the user is adding a "Car", he will extra fields in the form, dynamically at run time, like "Car Milage", "Cal Manufacturer".
Suppose if the user wants to add a "Truck", he will add "Load that can be carried", "Permit" etc.
How do I achieve this in django?
There are two questions here:
How to provide a form where the user can add new fields at run time?
How to add the fields to the database so that it can be retrieved/queried later?
There are a few approaches:
key/value model (easy, well supported)
JSON data in a TextField (easy, flexible, can't search/index easily)
Dynamic model definition (not so easy, many hidden problems)
It sounds like you want the last one, but I'm not sure it's the best for you. Django is very easy to change/update, if system admins want extra fields, just add them for them and use south to migrate. I don't like generic key/value database schemas, the whole point of a powerful framework like Django is that you can easily write and rewrite custom schemas without resorting to generic approaches.
If you must allow site users/administrators to directly define their data, I'm sure others will show you how to do the first two approaches above. The third approach is what you were asking for, and a bit more crazy, I'll show you how to do. I don't recommend using it in almost all cases, but sometimes it's appropriate.
Dynamic models
Once you know what to do, this is relatively straightforward. You'll need:
1 or 2 models to store the names and types of the fields
(optional) An abstract model to define common functionality for your (subclassed) dynamic models
A function to build (or rebuild) the dynamic model when needed
Code to build or update the database tables when fields are added/removed/renamed
1. Storing the model definition
This is up to you. I imagine you'll have a model CustomCarModel and CustomField to let the user/admin define and store the names and types of the fields you want. You don't have to mirror Django fields directly, you can make your own types that the user may understand better.
Use a forms.ModelForm with inline formsets to let the user build their custom class.
2. Abstract model
Again, this is straightforward, just create a base model with the common fields/methods for all your dynamic models. Make this model abstract.
3. Build a dynamic model
Define a function that takes the required information (maybe an instance of your class from #1) and produces a model class. This is a basic example:
from django.db.models.loading import cache
from django.db import models
def get_custom_car_model(car_model_definition):
""" Create a custom (dynamic) model class based on the given definition.
"""
# What's the name of your app?
_app_label = 'myapp'
# you need to come up with a unique table name
_db_table = 'dynamic_car_%d' % car_model_definition.pk
# you need to come up with a unique model name (used in model caching)
_model_name = "DynamicCar%d" % car_model_definition.pk
# Remove any exist model definition from Django's cache
try:
del cache.app_models[_app_label][_model_name.lower()]
except KeyError:
pass
# We'll build the class attributes here
attrs = {}
# Store a link to the definition for convenience
attrs['car_model_definition'] = car_model_definition
# Create the relevant meta information
class Meta:
app_label = _app_label
db_table = _db_table
managed = False
verbose_name = 'Dynamic Car %s' % car_model_definition
verbose_name_plural = 'Dynamic Cars for %s' % car_model_definition
ordering = ('my_field',)
attrs['__module__'] = 'path.to.your.apps.module'
attrs['Meta'] = Meta
# All of that was just getting the class ready, here is the magic
# Build your model by adding django database Field subclasses to the attrs dict
# What this looks like depends on how you store the users's definitions
# For now, I'll just make them all CharFields
for field in car_model_definition.fields.all():
attrs[field.name] = models.CharField(max_length=50, db_index=True)
# Create the new model class
model_class = type(_model_name, (CustomCarModelBase,), attrs)
return model_class
4. Code to update the database tables
The code above will generate a dynamic model for you, but won't create the database tables. I recommend using South for table manipulation. Here are a couple of functions, which you can connect to pre/post-save signals:
import logging
from south.db import db
from django.db import connection
def create_db_table(model_class):
""" Takes a Django model class and create a database table, if necessary.
"""
table_name = model_class._meta.db_table
if (connection.introspection.table_name_converter(table_name)
not in connection.introspection.table_names()):
fields = [(f.name, f) for f in model_class._meta.fields]
db.create_table(table_name, fields)
logging.debug("Creating table '%s'" % table_name)
def add_necessary_db_columns(model_class):
""" Creates new table or relevant columns as necessary based on the model_class.
No columns or data are renamed or removed.
XXX: May need tweaking if db_column != field.name
"""
# Create table if missing
create_db_table(model_class)
# Add field columns if missing
table_name = model_class._meta.db_table
fields = [(f.column, f) for f in model_class._meta.fields]
db_column_names = [row[0] for row in connection.introspection.get_table_description(connection.cursor(), table_name)]
for column_name, field in fields:
if column_name not in db_column_names:
logging.debug("Adding field '%s' to table '%s'" % (column_name, table_name))
db.add_column(table_name, column_name, field)
And there you have it! You can call get_custom_car_model() to deliver a django model, which you can use to do normal django queries:
CarModel = get_custom_car_model(my_definition)
CarModel.objects.all()
Problems
Your models are hidden from Django until the code creating them is run. You can however run get_custom_car_model for every instance of your definitions in the class_prepared signal for your definition model.
ForeignKeys/ManyToManyFields may not work (I haven't tried)
You will want to use Django's model cache so you don't have to run queries and create the model every time you want to use this. I've left this out above for simplicity
You can get your dynamic models into the admin, but you'll need to dynamically create the admin class as well, and register/reregister/unregister appropriately using signals.
Overview
If you're fine with the added complication and problems, enjoy! One it's running, it works exactly as expected thanks to Django and Python's flexibility. You can feed your model into Django's ModelForm to let the user edit their instances, and perform queries using the database's fields directly. If there is anything you don't understand in the above, you're probably best off not taking this approach (I've intentionally not explained what some of the concepts are for beginners). Keep it Simple!
I really don't think many people need this, but I have used it myself, where we had lots of data in the tables and really, really needed to let the users customise the columns, which changed rarely.
Database
Consider your database design once more.
You should think in terms of how those objects that you want to represent relate to each other in the real world and then try to generalize those relations as much as you can, (so instead of saying each truck has a permit, you say each vehicle has an attribute which can be either a permit, load amount or whatever).
So lets try it:
If you say you have a vehicle and each vehicle can have many user specified attributes consider the following models:
class Attribute(models.Model):
type = models.CharField()
value = models.CharField()
class Vehicle(models.Model):
attribute = models.ManyToMany(Attribute)
As noted before, this is a general idea which enables you to add as much attributes to each vehicle as you want.
If you want specific set of attributes to be available to the user you can use choices in the Attribute.type field.
ATTRIBUTE_CHOICES = (
(1, 'Permit'),
(2, 'Manufacturer'),
)
class Attribute(models.Model):
type = models.CharField(max_length=1, choices=ATTRIBUTE_CHOICES)
value = models.CharField()
Now, perhaps you would want each vehicle sort to have it's own set of available attributes. This can be done by adding yet another model and set foreign key relations from both Vehicle and Attribute models to it.
class VehicleType(models.Model):
name = models.CharField()
class Attribute(models.Model):
vehicle_type = models.ForeigngKey(VehicleType)
type = models.CharField()
value = models.CharField()
class Vehicle(models.Model):
vehicle_type = models.ForeigngKey(VehicleType)
attribute = models.ManyToMany(Attribute)
This way you have a clear picture of how each attribute relates to some vehicle.
Forms
Basically, with this database design, you would require two forms for adding objects into the database. Specifically a model form for a vehicle and a model formset for attributes. You could use jQuery to dynamically add more items on the Attribute formset.
Note
You could also separate Attribute class to AttributeType and AttributeValue so you don't have redundant attribute types stored in your database or if you want to limit the attribute choices for the user but keep the ability to add more types with Django admin site.
To be totally cool, you could use autocomplete on your form to suggest existing attribute types to the user.
Hint: learn more about database normalization.
Other solutions
As suggested in the previous answer by Stuart Marsh
On the other hand you could hard code your models for each vehicle type so that each vehicle type is represented by the subclass of the base vehicle and each subclass can have its own specific attributes but that solutions is not very flexible (if you require flexibility).
You could also keep JSON representation of additional object attributes in one database field but I am not sure this would be helpfull when querying attributes.
Here is my simple test in django shell- I just typed in and it seems work fine-
In [25]: attributes = {
"__module__": "lekhoni.models",
"name": models.CharField(max_length=100),
"address": models.CharField(max_length=100),
}
In [26]: Person = type('Person', (models.Model,), attributes)
In [27]: Person
Out[27]: class 'lekhoni.models.Person'
In [28]: p1= Person()
In [29]: p1.name= 'manir'
In [30]: p1.save()
In [31]: Person.objects.a
Person.objects.aggregate Person.objects.all Person.objects.annotate
In [32]: Person.objects.all()
Out[33]: [Person: Person object]
It seems very simple- not sure why it should not be a considered an option- Reflection is very common is other languages like C# or Java- Anyway I am very new to django things-
Are you talking about in a front end interface, or in the Django admin?
You can't create real fields on the fly like that without a lot of work under the hood. Each model and field in Django has an associated table and column in the database. To add new fields usually requires either raw sql, or migrations using South.
From a front end interface, you could create pseudo fields, and store them in a json format in a single model field.
For example, create an other_data text field in the model. Then allow users to create fields, and store them like {'userfield':'userdata','mileage':54}
But I think if you're using a finite class like vehicles, you would create a base model with the basic vehicle characteristics, and then create models that inherits from the base model for each of the vehicle types.
class base_vehicle(models.Model):
color = models.CharField()
owner_name = models.CharField()
cost = models.DecimalField()
class car(base_vehicle):
mileage = models.IntegerField(default=0)
etc