I have two models, Image and Dataset that are connected through a many-to-many relationship:
class Image(models.Model):
id = models.CharField(max_length=255, primary_key=True)
datasets = models.ManyToManyField(Dataset, blank=True)
class Dataset(models.Model):
...
I wanted to use a sequential integer as id instead of the VARCHAR. So I renamed the id field to id_old, which made Django create an integer auto-field as an id column with the following migrations like I wanted, to which I added to delete the id_old field.
operations = [
migrations.RenameField(
model_name='image',
old_name='id',
new_name='id_old',
),
migrations.AddField(
model_name='image',
name='id',
field=models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
),
But the problem is that now the through table in Postgres backend has the image_id column with type VARCHAR, hence I get the following error if I want to add a dataset to an image:
operator does not exist: character varying = integer
LINE 1: ...tasets" WHERE ("app_image_datasets"."image_id" = 12764 AN...
^
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
Since the image_id field didn't get altered during these operations, I also lose the relationships between datasets and images.
So how can I alter a primary key field in a model with M2M relationships?
Related
In case I am asking the wrong question, let me first state the end goal: I need to allow users to filter a ListView by a field that's not in the primary model (Salesleadquote), but instead the field comes from a model (Salesleadbusinessgroup) with a FK to a related model (Saleslead).
The way I am trying to approach this is by annotating a field on Salesleadquote.
The models:
class Salesleadquote(models.Model):
salesleadquoteid = models.AutoField(db_column='SalesLeadQuoteId', primary_key=True)
salesleadid = models.ForeignKey(Saleslead, models.DO_NOTHING, db_column='SalesLeadId')
...
class Saleslead(models.Model):
salesleadid = models.AutoField(db_column='SalesLeadId', primary_key=True)
...
class Salesleadbusinessgroup(models.Model):
salesleadbusinessgroupid = models.AutoField(db_column='SalesLeadBusinessGroupId', primary_key=True)
salesleadid = models.ForeignKey(Saleslead, models.DO_NOTHING, db_column='SalesLeadId')
businessgroupid = models.ForeignKey(Businessgroup, models.DO_NOTHING, db_column='BusinessGroupId')
The desired result (queryset), in SQL:
SELECT slq.*, slbg.BusinessGroupId FROM crm.SalesLeadQuote slq
LEFT JOIN
(SELECT SalesLeadId, BusinessGroupId
FROM crm.SalesLeadBusinessGroup ) slbg
ON slbg.SalesLeadId = slq.SalesLeadId
WHERE slbg.BusinessGroupId IN (5,21)
I know I can get a RawQuerySet by doing something like
Salesleadquote.objects.raw("SELECT salesleadquote.*, \
salesleadbusinessgroup.businessgroupid \
FROM salesleadquote \
LEFT JOIN salesleadbusinessgroup \
ON salesleadquote.salesleadid = salesleadbusinessgroup.salesleadid \
WHERE salesleadbusinessgroup.businessgroupid IN (5,21)")
But I need the functionality of a QuerySet, so my idea was to annotate the desired field (businessgroupid) in Salesleadquote, but I've been struggling with how to accomplish this.
I implemented a work-around that doesn't address my original question but works for my use case. I created a view at the database level (called SalesLeadQuoteBG) using the SQL I had posted and then tied that to a model to use with Django's ORM.
class Salesleadquotebg(models.Model):
"""
This model represents a database view that extends the Salesleadquote table with a business group id column.
There can be multiple business groups per quote, resulting in duplicate quotes, but this is handled at the view and template layer
via filtering (users are required to select a business group).
"""
salesleadquoteid = models.IntegerField(db_column='SalesLeadQuoteId', primary_key=True) # Field name made lowercase.
salesleadid = models.ForeignKey(Saleslead, models.DO_NOTHING, db_column='SalesLeadId') # Field name made lowercase.
...
businessgroupid = models.ForeignKey(Businessgroup, models.DO_NOTHING, db_column='BusinessGroupId')
I am using django-filters for the filtering.
filters.py:
BG_CHOICES = (
(5, 'Machine Vision'),
(21, 'Process Systems'),
)
class BGFilter(django_filters.FilterSet):
businessgroupid = django_filters.ChoiceFilter(choices=BG_CHOICES)
class Meta:
model = Salesleadquotebg
fields = ['businessgroupid', ]
This can be done with pure Django ORM. Relationships can be followed backwards (filtering on salesleadbusinessgroup) and the double underscore syntax can be used to query attributes of the related model and also follow more relationships
Salesleadquote.objects.filter(
salesleadbusinessgroup__businessgroupid__in=(5,21)
)
I'm trying to migrate a column from a Char field to a Many-to-Many field running Django 1.8.2. I'm doing a custom Data Migration, to move the data properly. When I try to migrate, I get a database error, can't insert null into the many to many table id column.
My models, simplified:
class LicenseArea(models.Model):
#appraisal_account = models.CharField(max_length=17, null=True, db_index=True)
appraisal_account = models.ManyToManyField(TaxAccount, db_table='LicAreaTaxAccount', related_name='accounts_for_license_area', related_query_name='license_area_for_account', null=True)
class TaxAccount(models.Model):
account = models.CharField(max_length=17, db_index=True)
So I first create TaxAccount objects in a RunPython block, then remove the old field and add the new one, like so:
migrations.RunPython(create_tax_account_objects),
migrations.RemoveField(
model_name='licensearea',
name='appraisal_account',
),
migrations.AddField(
model_name='licensearea',
name='appraisal_account',
field=models.ManyToManyField(related_query_name='license_area_for_account', related_name='accounts_for_license_area', db_table='licenses_LicAreaTaxAccount', to='licenses.TaxAccount'),
),
All that works. My issue comes when I try to migrate the data, relating the LicenseArea object with its corresponding TaxAccount object. In another RunPython block, I try the code shown below (I've tried in both directions; acct.licensearea_set.add indicates that the TaxAccount model has no licensearea_set attribute, while the second option shown below gives me the IntegrityError (ORA-01400) that I can't insert null into the ID column :
for la in LicenseArea.objects.all():
acct = TaxAccount.objects.get(account=la.appraisal_account_temp)
#acct.licensearea_set.add(la)
#la.appraisal_account.add(acct)
How do I solve this? Thanks in advance.
I am trying to add an order to a ManyToMany field that I created a while ago. I basically want to order pictures in collections of pictures. I am running on Django 1.7, so no more South migrations (I was trying to follow this tutorial: http://mounirmesselmeni.github.io/2013/07/28/migrate-django-manytomany-field-to-manytomany-through-with-south/)
Here's the "through" relationship that I have:
class CollectionPictures(models.Model):
picture = models.ForeignKey(
Picture,
verbose_name=u'Picture',
help_text=u'Picture is included in this collection.',
)
collection = models.ForeignKey(
Collection,
verbose_name=u'Collection',
help_text=u'Picture is included in this collection',
)
order = models.IntegerField(
verbose_name=u'Order',
help_text=u'What order to display this picture within the collection.',
max_length=255
)
class Meta:
verbose_name = u"Collection Picture"
verbose_name_plural = u"Collection Pictures"
ordering = ['order', ]
def __unicode__(self):
return self.picture.name + " is displayed in " + self.collection.name + (
" in position %d" % self.order)
class Collection(models.Model):
pictures = models.ManyToManyField(Picture, through='CollectionPictures', null=True)
[... Bunch of irrelevant stuff after]
So this should work if I didn't have to migrate my old data (the only difference in the model is that it didn't have the through='CollectionPictures'
Here's my migration :
class Migration(migrations.Migration):
dependencies = [
('artist', '0002_auto_20141013_1451'),
('business', '0001_initial'),
]
operations = [
migrations.CreateModel(
name='CollectionPictures',
fields=[
('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
('order', models.IntegerField(help_text='What order to display this picture within the collection.', max_length=255, verbose_name='Order')),
('collection', models.ForeignKey(verbose_name='Collection', to='business.Collection', help_text='Picture is included in this collection')),
('picture', models.ForeignKey(verbose_name='Picture', to='artist.Picture', help_text='Picture is included in this collection.')),
],
options={
'ordering': ['order'],
'verbose_name': 'Collection Picture',
'verbose_name_plural': 'Collection Pictures',
},
bases=(models.Model,),
),
migrations.AlterField(
model_name='collection',
name='pictures',
field=models.ManyToManyField(to=b'artist.Picture', null=True, through='business.CollectionPictures'),
),
]
This throws an error when migrating:
ValueError: Cannot alter field business.Collection.pictures into
business.Collection.pictures - they are not compatible types (you
cannot alter to or from M2M fields, or add or remove through= on M2M
fields)
Has anybody already tried that kind of manipulation with the new 1.7 migrations?
Thanks !
The safest approach would be to create a new field and copy the data over.
Leave pictures alone and add pictures2 with your through field. Run makemigrations.
Edit the generated migration file and add a RunPython command where you copy data from the old table to the new table. Perhaps you can programmatically choose a good value for the new order column as well.
Delete the old pictures field. Run makemgirations.
Rename pictures2 to pictures. Run makemigrations.
This approach should leave you in the state you want with your data intact.
If copying over the data is a big problem you could try something else, like adding the order column in SQL, using the db_table option on CollectionPictures to make it point to the existing table, and then wiping out migrations and redoing with --fake. But that seems riskier than the approach above.
Old question, but I had this problem too, and I've found a way with Django 1.11 that works, and should work with older versions too. The needed class exists back to 1.7 and still exists in 2.0
The fix involves manually changing the migration to do what we want, using the SeparateDatabaseAndState migration class. This class lets Django update the state, but gives us control over what operations to perform. In this case we just want to rename the model table, everything else is already set up right.
The steps:
Create your new ManyToMany Through model, but specify a custom table name, and no extra fields:
class CollectionPictures(models.Model):
collection = ...
picture = ...
class Meta:
# Change myapp to match.
db_table = "myapp_collection_pictures"
unique_together = (("collection", "picture"))
Taking the existing migration, and take the operations it generates and wrap it all in a single new SeparateDatabaseAndState:
class Migration(migrations.Migration):
dependencies = [
('artist', '0002_auto_20141013_1451'),
('business', '0001_initial'),
]
operations = [
migrations.SeparateDatabaseAndState(
database_operations=[
],
state_operations=[
migrations.CreateModel(
name='CollectionPictures',
fields=[
('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
('order', models.IntegerField(help_text='What order to display this picture within the collection.', max_length=255, verbose_name='Order')),
('collection', models.ForeignKey(verbose_name='Collection', to='business.Collection', help_text='Picture is included in this collection')),
('picture', models.ForeignKey(verbose_name='Picture', to='artist.Picture', help_text='Picture is included in this collection.')),
],
options={
'ordering': ['order'],
'verbose_name': 'Collection Picture',
'verbose_name_plural': 'Collection Pictures',
},
bases=(models.Model,),
),
migrations.AlterField(
model_name='collection',
name='pictures',
field=models.ManyToManyField(to=b'artist.Picture', null=True, through='business.CollectionPictures'),
),
]
)
Remove the db_table from the class Meta, and add this operation after the SeparateDatabaseAndState, (not into the database_operations.):
migrations.AlterModelTable(
name='collectionpicture',
table=None,
),
Now if you run `./mange.py sqlmigrate myapp 0003 (pick the right number prefix!) you should with any luck see something like this as output:
BEGIN;
--
-- Custom state/database change combination
--
--
-- Rename table for collection[Pictures to None
--
ALTER TABLE "myapp_collection_pictures" RENAME TO "myapp_collectionpictures";
COMMIT;
Add your new columns ("order" in this case) and create a new migration. It's probably possible to do this at the same time, but I decided it was easier to do in two migrations.
(Step 3 isn't strictly required if you are happy keeping the custom table name there.)
And double check with ./manage.py makemigrations --check -- it should print "No changes detected".
I'm new to Django and I have some issues with a ManyToMany relationship.
I work on a blastn automatisation and here are my classes:
class Annotation(models.Model):
sequence = models.IntegerField()
annotation = models.TextField()
start = models.IntegerField()
end = models.IntegerField()
class Blast(models.Model):
sequence = models.ManyToManyField(Annotation, through="AnnotBlast")
expectValue = models.IntegerField()
class AnnotBlast(models.Model):
id_blast = models.ForeignKey(Blast, to_field="id")
id_annot = models.ForeignKey(Annotation, to_field="id")
class Hit(models.Model):
id_hit = models.ForeignKey(Blast, to_field="id")
length = models.IntegerField()
evalue = models.IntegerField()
start_seq = models.IntegerField()
end_seq = models.IntegerField()
In a view, I want to access to Annotation's data from the rest of the model via this many to many field and then apply filters based on a form. But when I do a syncdb , the "sequence" field of the Blast class disappear :
In Sqlite3 :
.schema myApp_blast
CREATE TABLE "myApp_blast" (
"id" integer not null primary key,
"expectValue" integer not null
);
So I can't load data in this table as I want. I don't understand why this field disappear during the syncdb. How can I do to link the first class to the others (and then be able to merge data in a template) ?
A ManyToManyField isn't itself a column in the database. It's represented only by an element in the joining table, which you have here defined explicitly as AnnotBlast (note that since you're not defining any extra fields on the relationship, you didn't actually need to define a through table - Django would have done it automatically if you hadn't).
So to add data to your models, you add data to the AnnotBlast table pointing at the relevant Blast and Annotation rows.
For a many-to-many relationship an intermediate join table is created, see documentation here: https://docs.djangoproject.com/en/1.2/ref/models/fields/#id1
I've got a tiny little problem that, unfortunately, is taking all my time.
It is really simple, I already have my database and I created then modified models.py, and admin.py. Some staff users, who will need to enter values in my database, need the simpliest form to do so.
Here is my database :
-- Table NGSdb.line
CREATE TABLE IF NOT EXISTS `NGSdb`.`line` (
`id` INT NOT NULL AUTO_INCREMENT ,
`value` INT NOT NULL ,
PRIMARY KEY (`id`) )
ENGINE = InnoDB;
CREATE UNIQUE INDEX `value_UNIQUE` ON `NGSdb`.`line` (`value` ASC) ;
-- Table NGSdb.run_has_sample_lines
CREATE TABLE IF NOT EXISTS `NGSdb`.`run_has_sample_lines` (
`line_id` INT NOT NULL ,
`runhassample_id` INT NOT NULL ,
PRIMARY KEY (`line_id`, `runhassample_id`) ,
CONSTRAINT `fk_sample_has_line_line1`
FOREIGN KEY (`line_id` )
REFERENCES `NGSdb`.`line` (`id` )
ON DELETE NO ACTION
ON UPDATE NO ACTION,
CONSTRAINT `fk_sample_has_line_run_has_sample1`
FOREIGN KEY (`runhassample_id` )
REFERENCES `NGSdb`.`run_has_sample` (`id` )
ON DELETE NO ACTION
ON UPDATE NO ACTION)
-- Table NGSdb.run_has_sample
CREATE TABLE IF NOT EXISTS `NGSdb`.`run_has_sample` (
`id` INT NOT NULL AUTO_INCREMENT ,
`run_id` INT NOT NULL ,
`sample_id` INT NOT NULL ,
`dna_quantification_ng_per_ul` FLOAT NULL ,
PRIMARY KEY (`id`, `run_id`, `sample_id`) ,
CONSTRAINT `fk_run_has_sample_run1`
FOREIGN KEY (`run_id` )
REFERENCES `NGSdb`.`run` (`id` )
ON DELETE NO ACTION
ON UPDATE NO ACTION,
CONSTRAINT `fk_run_has_sample_sample1`
FOREIGN KEY (`sample_id` )
REFERENCES `NGSdb`.`sample` (`id` )
ON DELETE NO ACTION
ON UPDATE NO ACTION)
Here is my models.py :
class Run(models.Model):
id = models.AutoField(primary_key=True)
start_date = models.DateField(null=True, blank=True, verbose_name='start date')
end_date = models.DateField(null=True, blank=True, verbose_name='end date')
project = models.ForeignKey(Project)
sequencing_type = models.ForeignKey(SequencingType)
def __unicode__(self):
return u"run started %s from the project %s" % (self.start_date,self.project)
class Line(models.Model):
id = models.AutoField(primary_key=True)
value = models.IntegerField()
def __unicode__(self):
return u"%s" % str(self.value)
class RunHasSample(models.Model):
id = models.AutoField(primary_key=True)
run = models.ForeignKey(Run)
sample = models.ForeignKey(Sample)
dna_quantification_ng_per_ul = models.FloatField(null=True, blank=True)
lines = models.ManyToManyField(Line)
def __unicode__(self):
return u"Sample %s from run %s" % (self.sample, self.run)
And here is my admin.py :
class RunHasSamplesInLine(admin.TabularInline):
model = RunHasSample
fields = ['sample', 'dna_quantification_ng_per_ul', 'lines']
extra = 6
class RunAdmin(admin.ModelAdmin):
fields = ['project', 'start_date', 'end_date', 'sequencing_type']
inlines = [RunHasSamplesInLine]
list_display = ('project', 'start_date', 'end_date', 'sequencing_type')
As you can see, my samples are displayed in lines in the run form so that the staff can easily fullfill the database.
When I try to fill the database I have this error :
(1054, "Unknown column 'run_has_sample_lines.id' in 'field list'")
Of course, there are no field "lines" in my database ! It is a many to many field so I already created my intermediate table !
Okay okay ! So I tried to create the model for the intermediate table (run_has_sample_lines) and add a "through" to the ManyToManyField in the RunHasSample model. But, as I add manually the "through", I cannot use the ManyToMany field. The only way to add lines to the admin view is to stack them in lines... As you can see the samples are already in lines, it is impossible to put a new "inlines" in the already in lines samples...
Finally, I just tried to see what django had created with the manage.py sqlall.
I see that :
CREATE TABLE `run_has_sample_lines` (
`id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY,
`runhassample_id` integer NOT NULL,
`line_id` integer NOT NULL,
UNIQUE (`runhassample_id`, `line_id`)
)
;
ALTER TABLE `run_has_sample_lines` ADD CONSTRAINT `line_id_refs_id_4f0766aa` FOREIGN KEY (`line_id`) REFERENCES `line` (`id`);
It seems that there are no foreign key on the run_has_sample table whereas I created it in the database in the first place. I guess that the problem is coming from here but I cannot resolve it and I really hope that you can...
Thank you very much !
you may wish to try a 'through' attribute on the many-to-many relationship and declare your intermediate table in Django.
I found where the problem is...
It is not a problem in the ManyToManyField but in the intermediate table. Django refused that my intermediate table doesn't have an unique id !
So, in the sql which created django, it created automatically an unique id named "id", but in my database I didn't create one (because the couple of two foreign key is usually enough).
Next time, I'll be more carefull.