Test a data migration ManyToMany in Django - django

I tried to add a field to my ManyToMany relationship models in Django.
So step by step, I created the new model and apply makemigrations and migrate.
I checked I have the new table in my postgresql database.
Now before I will add the through keyword in the ManyToMany field I want to write a function in the migration file that will copy the old data of the previous ManyToMany table to the new one with the additional field.
I followed a solution explained here:
Django migration error :you cannot alter to or from M2M fields, or add or remove through= on M2M fields
I want to test the function that will migrate the data in a test function but I don't understand what to do.
here my code:
survey/models:
class Survey(BaseModel):
name = models.CharField(max_length=256, help_text='Survey name')
user = models.ManyToManyField(User, blank=True, help_text='patient')
survey/models:
class SurveyStatus(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE)
survey = models.ForeignKey(Survey, on_delete=models.CASCADE)
survey_status = models.CharField(max_length=10,
blank=True,
null=True,
choices=STATUS_SURVEY_CHOICES,
)
The function I wrote that need to copy the data from the previous M2M to the new one is the following one:
def create_through_relations(apps, schema_editor):
Survey = apps.get_model('survey', 'Survey')
SurveyStatus = apps.get_model('survey', 'SurveyStatus')
for survey in Survey.objects.all():
for user in survey.user.all():
SurveyStatus(
user=user,
survey=survey,
survey_status='active'
).save()
I don't understand what is apps? because it is not recognized by python
I don't understand why i need schema_editor because it's not used
it doesn't recognized my Survey or SurveyStatus models too
when i tried to run this script with
if __name__ == "__main__":
create_through_relations(survey)
I've got this error
NameError: name 'survey' is not defined
and if i tried this function
from django.apps import apps
def create_through_relations():
Survey = apps.get_model('survey', 'Survey')
SurveyStatus = apps.get_model('survey', 'SurveyStatus')
for survey in Survey.objects.all():
for user in survey.user.all():
SurveyStatus(
user=user,
survey=survey,
survey_status='active'
).save()
when i tried to run this script with
if __name__ == "__main__":
create_through_relations()
I've got this error
django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.
If someone can help and explain me how to solve.thanks

1: Apps represent the different parts of your project (Django Apps)
2: You don't need it at this point. In general, it translates the models into SQL syntax.
3: python manage.py <...> does load the models for execution. Your file is trying to access data that isn't available that way.
4: The variable survey can't be found in python's main function, since you never declared it there. You need to trigger it inside your project.
5: You can test things by creating a test.py (Django Tests)
6: You don't need to transfer the data to a whole new table after changing a model, just extend the existing one and migrate the changes:
class BaseModel(models.Model):
created = models.DateTimeField('created', default=timezone.now)
changed = models.DateTimeField('changed', default=timezone.now, blank=True, null=True)
class Survey(BaseModel):
uuid = models.UUIDField(primary_key=False, default=uuid.uuid4, editable=False)
name = models.CharField(max_length=256, help_text='Survey name')
description = models.TextField('description', blank=True)
status = models.BooleanField(default=False) # paused/ active
class SurveyQuestion(BaseModel):
survey = models.ForeignKey(Survey, related_name='survey', on_delete=models.CASCADE)
text = models.CharField(max_length=256)
# 1 -> Text, # Integer, # ChoiceField, etc.
requested_result = models.IntegerField(default=0)
class QuestionResult(BaseModel):
user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
question = models.ForeignKey(SurveyQuestion, related_name='survey_question', on_delete=models.CASCADE)
answer = models.CharField(default='', max_length=256)

Related

Django ID of foreign key doesn't exist after migrating

I'm new to Django, and I'm trying to create a "game" model with two attributes:
A many-to-one field where multiple instances of the game model are associated with an instance of a custom user model.
A many-to-many field where instances of the game model are connected with multiple instances of words, and instances of the word model are connected with multiple instances of the game model
Top of my models.py model:
from django.db import models
from users.models import CustomUser
from django.contrib.postgres.fields import ArrayField
Game model:
class SortingGame(models.Model):
user_current_player = models.ForeignKey(CustomUser, on_delete=models.CASCADE, null=True, blank=True)
field_words = models.ManyToManyField(Word, related_name="field_sorting_games")
Word model:
class Word(models.Model):
str_word = models.CharField(max_length=50,null=True)
int_grade_level = models.IntegerField()
arrint_phonemes = ArrayField(models.CharField(max_length=50),null=True)
arrstr_graphemes = ArrayField(models.CharField(max_length=50),null=True)
int_num_syllables = models.IntegerField()
arrstr_syllables = ArrayField(models.CharField(max_length=50),null=True)
User model:
class CustomUser(AbstractBaseUser):
# must have the following fields for django
email = models.EmailField(verbose_name="email",max_length = 100,unique=True)
username = models.CharField(max_length = 30, unique = True)
date_joined = models.DateTimeField(verbose_name = "date_joined",auto_now_add=True)
last_login = models.DateTimeField(verbose_name = "last_login",auto_now = True)
is_admin = models.BooleanField(default=False)
is_superuser = models.BooleanField(default = False)
is_staff = models.BooleanField(default = False)
is_active = models.BooleanField(default = True)
first_name = models.CharField(max_length=15, blank=True)
last_name = models.CharField(max_length=30, blank=True)
spelling_level = models.IntegerField(default=1, unique=False)
time_played = models.IntegerField(default=0, unique=False)
percent_correct = models.IntegerField(default=0, unique=False)
admin.py:
from django.contrib import admin
from .models import Word, SortingGame
admin.site.register(SortingGame)
When I run python3 manage.py makemigrations and python3 manage.py migrate, it doesn't complain, but when I go to the admin page of my django site it says psycopg2.errors.UndefinedColumn: column "user_current_player_id" of relation "game_sortinggame" does not exist.
This makes me think the issue is with user_current_player in SortingGame (it worked fine before I added that attribute), but I've looked around on different forums to see what might be going wrong and I can't seem to figure it out. I tried starting from scratch with a new database, and it's still throwing the same exception. Any ideas would be appreciated—thanks!
Nathan!
First thing would be make sure that you have the app where CustomUser model is created in your settings.py file, at INSTALLED_APPS.
If so, please have a look at this folder (app) where you have CustomUser defined to verify if there is in deed a migrations folder there.
I suspect that Django in not aware of this app (not under INSTALLED_APPS) and therefore did not migrated it. So, your database is not finding the User Model connection.
That said, I would suggested you to keep your account model as defined by Django User and create another model with a direct relationship to it to deal with profile/game fields such as spelling level, percentage_correct and so on.
This would keep your Model "concerns" more organized later on.
if you did make a migrations before try to use (python manage.py makemigrations -appname)
Also after That you need to Add the module in your admin.py
from django.contrib import admin
from .models import *
admin.site.register(SortingGame)
... all other modules

Django custom migration RunPython cannot access ForeignKey field in another database

I'm writing a django migration operation to change some data in the 'default' database. My app has access to a 'services' database, which contains data I cannot change.
The two relevant fields in the default.table are:
data_sets_to_remove = models.CharField(blank=True, null=False, max_length=100, default="")
data_sets_new = ArrayField(models.IntegerField(null=True, blank=True), null=True, blank=True)
i.e., i'm migrating data from data_sets_to_remove into a new format and adding to data_sets_new. To do this, I need to access data from the 'services' database in the migration operation.
def forwards_func(apps, schema_editor):
DataRelease = apps.get_model('registry', "datarelease")
Survey = apps.get_model('registry', "survey")
data_release_queryset = DataRelease.objects.using('services').all().values('id', 'name', 'survey')
But for some reason, the foreign key field 'survey' on the DataRelease model is not available in this context.
django.core.exceptions.FieldError: Cannot resolve keyword 'survey'
into field. Choices are: id, name
Can anyone shed any light on this? I assume the issue is around accessing data from a secondary database within a migration operation, but if i run the same code in the console, it works fine...
The relevant Survey and DataRelease models in the services database:
class Survey(models.Model):
name = models.CharField(blank=False, null=False, max_length=100, unique=True)
class DataRelease(models.Model):
name = models.CharField(blank=False, null=False, max_length=100)
survey = models.ForeignKey(Survey, related_name='data_releases', on_delete=models.CASCADE)
:facepalm:
The answer was staring me in the face. Switched the import of the associated models from:
DataRelease = apps.get_model('registry', "datarelease")
Survey = apps.get_model('registry', "survey")
to:
from services.registry.models import DataRelease, Survey
Now I can access the related fields in the migration operation. Hopefully this helps someone else in the future!

makemigrations does not create the trough model

I have made changes to one of my models in my project and migrate, makemigrations does not work as expected. Rebuilding the database creates only 2 out of 3 tables from my models.py and i cannot figure out the problem.
There are two different apps; "blog" and "users". both are registered in the setting.py.
I completely removed the database and deleted the migrations folders.
then i tried the following stuff:
django makemigrations blog
django migrate blog
doing a global django makemigrations does not have any effect, no changes are detected.
here is the relevant models.py of "blog":
class Room(models.Model):
roomname = models.CharField(max_length=6, unique=True)
roomeditors=models.ManyToManyField(User,related_name='rooms_user_can_edit', blank=True)
displayadmin=models.ForeignKey(User,
related_name='room_user_is_displayadmin',null=True, on_delete=models.SET_NULL)
def __str__(self):
return self.roomname
class Post(models.Model):
title = models.CharField(max_length=40)
content = models.TextField(max_length=300)
date_posted = models.DateTimeField(default=timezone.now)
author = models.ForeignKey(User, on_delete=models.CASCADE)
rooms = models.ManyToManyField(Room, related_name='roomposts', through='Display')
def __str__(self):
return self.title
def get_absolute_url(self):
return "/post/{}/".format(self.pk)
class Display(models.Model):
class Meta:
auto_created = True
post = models.ForeignKey(Post, on_delete=models.CASCADE)
room = models.ForeignKey(Room, on_delete=models.CASCADE)
isdisplayed = models.BooleanField(default=0)
def __str__(self):
return str(self.isdisplayed)
every table gets created except from display. the output is:
Migrations for 'blog':
blog\migrations\0001_initial.py
- Create model Room
- Create model Post
You are giving auto_created = True in your model's Meta class, which is not recommended neither its documented. Here is the list of all possible meta options you can give inside your model.
Official documentation says:
auto_created: Boolean flag that indicates if the field was automatically created, such as the OneToOneField used by model inheritance.
Giving this in Meta refrains Django to create this model itself.

How to populate a django manytomany database from excel file

I'm trying to transfer data from a excel file into a manytomany table in my sqlite3 database.
model.py
from django.db import models
class Major(models.Model):
name = models.CharField(max_length=30, db_index=True)
class School(models.Model):
name = models.CharField(max_length=50, db_index=True)
majors = models.ManyToManyField(Major)
class professor(models.Model):
ProfessorIDS = models.IntegerField()
ProfessorName = models.CharField(max_length=100)
ProfessorRating = models.DecimalField(decimal_places=2,max_digits=4)
NumberofRatings = models.CharField(max_length=50)
#delete major from the model
school = models.ForeignKey(School , on_delete=models.CASCADE)
major = models.ForeignKey(Major , on_delete=models.CASCADE)
def __str__(self):
return self.ProfessorName
Populating Script
# populate.py, school_major_link
import os
import django
from django_xlspopulator.populator import Populator
os.environ.setdefault('DJANGO_SETTINGS_MODULE','blog_project.settings')
django.setup()
from locate.models import Major
pop = Populator('C:/Users/David/Desktop/db_setup/School_Majors.xlsx', school_majors)
pop.populate()
Error message when attempting to run script
Traceback (most recent call last):
File "populate_school_major.py", line 9, in <module>
pop = Populator('C:/Users/David/Desktop/db_setup/School_Majors.xlsx', school_majors)
NameError: name 'school_majors' is not defined
But it makes sense since this script looks for the class name in the models section verses the name of the table, so I'm not too sure how I would be able to populate the correct table, Note that I already have a table named majors which is already populated using this script, but since django makes manytomany relationships via a variable verses a seperate class i'm stuck.
I tried using the populating script above, but noticed that this wouldn't work since it's locating the class, verses what the database table is saved as. (In sqlite3, the tables name for the majors manytomanyfield is called school_majors).
If anyone has any recommendations on how I can populate the db that would be great.
Picture below of the database table name.
Excel file below
When you define a ManyToManyField in Django, it creates a model behind-the-scenes to store the mapping.
You can access this model using the through attribute of the ManyToManyField. In your case, this would be:
locate.models.School.majors.through
Your current script is failing because school_majors is not a defined name at all - try replacing it with a reference to the model you wish to populate:
pop = Populator('C:/Users/David/Desktop/db_setup/School_Majors.xlsx', locate.models.School.majors.through)
pop.populate()
If this does not work, you might wish to consider defining an explicit through model on the ManyToManyField (as explained in the Django docs) and then specify that in your Populator instantiation.
Good luck!
As lukewarm stated, setting the through allowed me to create a class, which I was able to reference with my script which strictly would be able to populate by locating a class in the model.py
from django.db import models
class Major(models.Model):
name = models.CharField(max_length=30, db_index=True)
class School(models.Model):
name = models.CharField(max_length=50, db_index=True)
school_Major_merge = models.ManyToManyField(Major, through='School_Major')
class School_Major(models.Model):
major = models.ForeignKey(Major, on_delete=models.CASCADE)
school = models.ForeignKey(School, on_delete=models.CASCADE)
class professor(models.Model):
ProfessorIDS = models.IntegerField()
ProfessorName = models.CharField(max_length=100)
ProfessorRating = models.DecimalField(decimal_places=2,max_digits=4)
NumberofRatings = models.CharField(max_length=50)
#delete major from the model
school = models.ForeignKey(School , on_delete=models.CASCADE)
major = models.ForeignKey(Major , on_delete=models.CASCADE)
def __str__(self):
return self.ProfessorName

How to save a table entry in django

I am working on a django admin based project now i am stuck with a big thing.i want to add a field named "item_issued" in the user_profile model.
in the "item issued" field there is a table which consist of 3 column "item_name","quantity" and "price".I am unable to apply this.Can u guys please help me in this?
Thanks in advance
If I understand you correctly you want to add a ForeignKey to your user_profile pointing to item_issued. You can accomplish that by creating a new model ItemIssued with the fields you mentioned:
class ItemIssued(models.Model):
item_name = models.CharField(max_length=100)
quantity = models.IntegerField()
price = models.FloatField()
Now, when you're having ItemIssued model you can add a ForeignKey to user_profile (I assume the model is called UserProfile):
class UserProfile(models.Model):
... # your existing fields
item_issued = models.ForeignKey(ItemIssued)
After that, don't forget to run
python manage.py makemigrations app
python manage.py migrate
Here is a starting point:
models.py:
class ItemIssued(models.Model):
item_name = models.CharField(max_length=50)
quantity = models.IntegerField()
models.DecimalField(max_digits=6, decimal_places=2) #use decimal field for price values.
class UserProfile(models.Model):
# some other fields..
issued_items = models.ManyToManyField("ItemIssued", related_name="+issued_items", null=True, blank=True)
And if you need to use this field outside of Django Admin, views.py:
user = UserProfile.objects.get(username="ali")
new_issued_item = ItemIssued.objects.get(item_name="test_item")
user.issued_items.add(new_issued_item) #add
user.issued_items.delete(new_issued_item) #delete
items = user.issued_items.all() # get all issued items of user
i didn't test the code. But they should work.