Migrate data from one Model to another with Django South - django

I currently have a structure that needs to be rewritten in order to cope with Django-CMS
Currently the setup is as follows
class Video(models.Model):
#embed_code_or_url = models.CharField(max_length=2000)
permalink = models.URLField(verify_exists=True, unique=True, max_length=255, default="http://", validators=[validate_youtube_address])
thumbnail = models.CharField(max_length=500, blank=True, null=True)
# Data
title = models.CharField(max_length=255, blank=True)
...
class VideoPlugin(CMSPlugin):
video = models.ForeignKey(Video)
when I now transfer all my fields from Video to VideoPlugin, run my schemamigration, I'd also like to transfer ALL the info from Video to VideoPlugin when I run the migration.
Does anyone have an example on how this can be achieved?
Here is the beginnig of the migration to be run
class Migration(SchemaMigration):
def forwards(self, orm):
# Adding field 'VideoPlugin.permalink'
db.add_column('cmsplugin_videoplugin', 'permalink', self.gf('django.db.models.fields.URLField')(default='http://', unique=True, max_length=255), keep_default=False)
# Adding field 'VideoPlugin.thumbnail'
db.add_column('cmsplugin_videoplugin', 'thumbnail', self.gf('django.db.models.fields.CharField')(max_length=500, null=True, blank=True), keep_default=False)
# Adding field 'VideoPlugin.title'
db.add_column('cmsplugin_videoplugin', 'title', self.gf('django.db.models.fields.CharField')(default='', max_length=255, blank=True), keep_default=False)
...
Your help is much appreciated

You create a datamigration:
$ python manage.py datamigration yourapp name_of_this_migration
This freezes the models in your app. If another app(s) is/are involved in the migration, you'll need to add --freeze app1 --freeze app2, etc., to that line to include those in your migration as well.
This sets up the basic migration file structure for you, but the forwards and backwards migrations are empty. It's up to your to determine the logic that will migrate data from one to the other. But this works like anything else in Django, except you use the South ORM. For any model in your app in which this migration resides, you use orm.MyModel.objects for any other app that your added with the --freeze parameters, you use orm['someapp.SomeModel'].objects.
Other than that, you just get/filter/create, etc., the objects as normal moving the data from one to the other. Obviously, your forwards migration needs the logic that moves the data where you want it now, and your backwards migration should have the logic required to restore the data to where it was originally.
You can then migrate forwards and backwards in your dev environment to ensure it works properly. One important note: this is only for moving data around. DO NOT alter or delete any table structures in your datamigration. If you need to delete tables after the data has been moved. Create a schemamigration after the datamigration.

Related

Django change model, migrate, and update the existing records

I had a Django model defined as:
from users.models import User
class Record(models.Model):
user = models.ForeignKey(
User,
on_delete=SET_NULL,
null=True)
...
I realized that it would be better to rewrite the above model as:
from userprofile.models import UserProfile
class Record(models.Model):
user = models.ForeignKey(
UserProfile,
on_delete=SET_NULL,
null=True)
....
How can I migrate (so the new definition of the model Record) is used, without having to lose the old Record instances in the databases? What is the general process of such migrations? Because if I migrate with the new definition of Record, I lose access to the old Record objects so I can't update them. However, if I don't migrate, Django won't let me use UserProfile as the user field.
There is nothing to migrate in this way, as user is a Python attribute and there is no backing db column since it is not defined as ForeignKey.
In that case Django will handle this automatically, as it will discover the change of name for the model and it will alter the table name on DB side, then it will update the referencing table for Record table.
you can run python manage.py makemigrations and see the generated migrations.

Restoring Django Model Field after accidental deletion

I have an existing events model in my django app that lists events. The events has a poster field that was using the models.ImageField for storing the related poster. The uploading of events was controlled from the admin only so no public uploading. The database had about 50 entries of events and things were working fine.
Since the images was getting heavier, I tried adding the sorl.thumnail to django. In my models.py file, I imported the ImageField from sorl.thumbnails and added it to the poster field.
When I ran makemigrations, it reported 'alter field poster' Then I ran migrate and on reloading, I do not see the Poster field in the admin.
Even the existing events have lost the poster field and the images cannot be shown in the templates too.
I have reverted the changes and run migrations again, but the poster field is still not visible. Is there any way to restore the model, and reconnect it with existing poster images?
I am using python 3.5 and django 1.11 Everything else works fine, this seems like a stupid mistake for the sake of getting a little more speed.
Will be grateful for any suggestions.
Here is what my Events model looks likeclass Events(models.Model):
title = models.CharField(max_length=200)
note = RichTextUploadingField()
category = models.CharField(max_length=1, choices=EVENT_TYPES, verbose_name=_('Event Category'))
date_from = models.DateTimeField(auto_now=False, blank=False)
date_to = models.DateTimeField(auto_now=False, blank=False)
poster = models.ImageField(upload_to='Programs/', blank=True, max_length=100, width_field=None, height_field=None)
video = models.URLField(max_length=500, blank=True)
capacity = models.SmallIntegerField(blank=True)
venue = GeopositionField(blank=True)
Admin.py is simply a register function
class EventAdmin(admin.ModelAdmin):
list_display = ('title', 'date', 'venue', 'category')
admin.site.register(Event, EventAdmin)
So, it doesn't seem likely to recover a lost field with the data intact. I am using MySQL for the project. So here is what I did after making all possible efforts to salvage it.
First I deleted all migrations in the applications/migrations folder. Then I changed the name of the field in the model. Then when I ran makemigrations, it showed me the the message 'Alter Field in Events' I proceeded with migrate after that.
It is good to tell you that I am on the Amazon EC2 instance running Ubuntu Server. So I had to reboot the instance and then reload the admin page. But the changes in the model admin were still not visible.
Then I had to stop the server, and start it after 5 minutes. This changed the public IP and on reloading the site, I saw the ImageUpload field in admin. The rest of the data was intact. Now I just have to re-upload the images in the respective events. This is not the proper solution that I was expecting, but this seems to be the best that could be done.

Migrations in Django 1.7

I am currently involved in a project where I am using Django 1.7 development version.I want to propogate changes that I make in my models (adding a field, deleting a model, etc.) into the database schema using "makemigrations" and "migrate" commmands.I added a "age" field to one of the models in my application.
country = models.CharField(max_length=50, blank=True)
address = models.CharField(max_length=100, blank=True)
postal_code = models.IntegerField(max_length=50, blank=True)
city = models.CharField(max_length=50, blank=True)
phone_no = models.CharField(max_length=25, blank=True)
skype_name = models.CharField('Skype Username',max_length=50, blank=True)
age=models.IntegerField(max_length=25,blank=True)
When I use "makemigrations" command ,the output is like---"No changes detected".I guess that "makemigrations" is not able to figure out the changes made to the schema.Any suggestions how can I make it work??
If you are adding initial migrations to an app, you must include the app name when using the makemigrations command.
python manage.py makemigrations your_app_label
If it is the first time you are migrating that app you have to use:
manage.py makemigrations myappname
Once you do that you can do:
manage.py migrate
If you had your app in database, modified its model and its not updating the changes on makemigrations you probably havent migrated it yet.
Change your model back to its original form, run the first command (with the app name) and migrate...it will fake it. Once you do that put back the changes on your model, run makemigrations and migrate again and it should work.
I have sometimes the same problem.
I manage to populate the change in the database by following :
rm -rf your_app/migrations/*
python manage.py migrate
if it doesn't work, consider a manual drop table before, if you don't have data in it.
it worked for me with django 1.7c1

Django ManyToMany Through not syncing or migrating

I have this model (truncated here for brevity):
class Meal(models.Model):
host = models.ForeignKey(User, related_name="cooking")
cost = models.IntegerField(default=1)
summary = models.CharField(max_length=1024, default="A good dinner")
diners = models.ManyToManyField(User, through='Attendance',
related_name="diners", blank=True)
When I sync it, the diners Field is completely ignored. It doesn't appear in the database and there is no error when running syncdb. It's as if it's not there. The User and Attendance tables are all fine.
I discovered this problem when trying to add this field with South, so I've tried that as an alternative too.
Any ideas?
Thanks
Did you already run syncdb fyrir Meal before you added the diners field?
Because syncdb will not alter existing tables as you can read here:
Django docs
Side note - I have not used south personally but I have used Django evolution while developing.
Edit:
After reading your comment I think I know what the problem is.
When using through with ManyToManyField Django doesn't add a field to that table, all the necessary information is in the attendance table.

Adding a "through" table to django field and migrating with South?

Seems like this should be "easy" or at least documented somewhere, I just cant find it.
Lets say I have a model:
class A(models.Model):
users = models.ManyToMany('auth.User', blank=True)
Now I want to migrate to have a through table to add fields to the ManyToMany relation...
class AUsers(models.Model):
user = models.ForeignKey('auth.User')
a = models.ForeignKey('A')
new_field = models.BooleanField()
class A(models.Model):
users = models.ManyToMany('auth.User', blank=True, through='AUsers')
Then I do:
% ./manage.py schemamigration app --auto
Not totally surprising, it tells me it is going to drop the original auto-created through table and create a new one for AUsers. What's the best practice at this point? Is there a decent way to migrate to the new through table? Do I use db_table in Meta? Do I just not use the through=... right away... then do a schemamigration --auto, then a datamigration to copy the current table (somehow, not sure...) and then add the through relation and let it kill the table?
What's the trick here? Is this really that hard?
You should be able to do this pretty easily.
First of all, make sure that the manual through table that you are creating has the same table name in the database as the one Django originally created automatically.
So, first, let's consider a manual through model before your change:
class AUsers(models.Model):
user = models.ForeignKey('auth.User')
a = models.ForeignKey('A')
class Meta:
db_table = 'appname_a_user'
That should be functionally (almost) identical to the ManyToManyField you used to have. Actually, you could make an empty migration and apply it, and then use --auto for your changes (but don't).
Now, add your field like you did in your sample code above, and then run ./manage.py schemamigration appname manual_through_table --empty. That will give you an empty migration named ####_manual_through_table.py.
In the migration itself, there will be a forwards and backwards method. Each one needs to be one line each:
def forwards(self, orm):
db.add_column('appname_a_user', 'new_field', self.gf('django.db.models.fields.BooleanField')(default=False))
def backwards(self, orm):
db.delete_column('appname_a_user', 'new_field')
That should get you what you are after.
If anyone comes across this question when trying to do the same thing with the moderns migration framework, here are the steps:
Create a new model class that exactly matches the built-in through table
Use the Meta class to set the table name to match the existing table
Generate a migration, which will create the new table and set it as the through for the field.
Without running that migration, edit it to wrap it in a migrations. SeparateDatabaseAndState migration, where the auto-generated steps are in the state_operations field and the database operations are empty.
Modify your through table, as required, making sure to generate new migrations as normal.
As mentioned in a comment, the first step may be simplified using db.rename_table as described here, which gives this through model:
class AUsers(models.Model):
user = models.ForeignKey('auth.User')
a = models.ForeignKey('A')
class Meta:
unique_together = (('user', 'a'),)
Then, create a migration with --auto (this way you'll have the names of the DB tables visible), and replace the content with:
class Migration(SchemaMigration):
def forwards(self, orm):
db.rename_table('appname_a_user', 'appname_auser')
def backwards(self, orm):
db.rename_table('appname_auser','appname_a_user')
I just applied it in my project without issues.