I'm writing a django migration operation to change some data in the 'default' database. My app has access to a 'services' database, which contains data I cannot change.
The two relevant fields in the default.table are:
data_sets_to_remove = models.CharField(blank=True, null=False, max_length=100, default="")
data_sets_new = ArrayField(models.IntegerField(null=True, blank=True), null=True, blank=True)
i.e., i'm migrating data from data_sets_to_remove into a new format and adding to data_sets_new. To do this, I need to access data from the 'services' database in the migration operation.
def forwards_func(apps, schema_editor):
DataRelease = apps.get_model('registry', "datarelease")
Survey = apps.get_model('registry', "survey")
data_release_queryset = DataRelease.objects.using('services').all().values('id', 'name', 'survey')
But for some reason, the foreign key field 'survey' on the DataRelease model is not available in this context.
django.core.exceptions.FieldError: Cannot resolve keyword 'survey'
into field. Choices are: id, name
Can anyone shed any light on this? I assume the issue is around accessing data from a secondary database within a migration operation, but if i run the same code in the console, it works fine...
The relevant Survey and DataRelease models in the services database:
class Survey(models.Model):
name = models.CharField(blank=False, null=False, max_length=100, unique=True)
class DataRelease(models.Model):
name = models.CharField(blank=False, null=False, max_length=100)
survey = models.ForeignKey(Survey, related_name='data_releases', on_delete=models.CASCADE)
:facepalm:
The answer was staring me in the face. Switched the import of the associated models from:
DataRelease = apps.get_model('registry', "datarelease")
Survey = apps.get_model('registry', "survey")
to:
from services.registry.models import DataRelease, Survey
Now I can access the related fields in the migration operation. Hopefully this helps someone else in the future!
Related
I am using Django 4.1, Oracle 19 with the python package oracledb.
I am logged in as user djangousr and the schema I am trying to get the data from is "user123"
I am able to retrieve data in a file outside of Django with the same connection information.
But in Django, I keep getting the same error.
I hope you have a solution for me as I was not able to find anything elsewhere.
Thank you.
ORA-00942: table or view does not exist
Below is the SQL from the debug screen that I am able to run fine in SQL developer.
('SELECT "USER"."USER_ID" FROM "user123.USER_TABLE"')
I will also provide the model and the view:
class SecurityUserData(models.Model):
user_id = models.IntegerField(primary_key=True)
user_name = models.CharField(max_length=100)
user_password = models.CharField(max_length=100)
user_first_name = models.CharField(max_length=30, blank=True, null=True)
user_last_name = models.CharField(max_length=30, blank=True, null=True)
class Meta:
managed = False
db_table = 'SEC_USER'
And the view:
def display_user_data(request):
user_data = SecurityUserData.user123.all()
return render(request, 'all.html', {'user_data': user_data})
I tried to add a field to my ManyToMany relationship models in Django.
So step by step, I created the new model and apply makemigrations and migrate.
I checked I have the new table in my postgresql database.
Now before I will add the through keyword in the ManyToMany field I want to write a function in the migration file that will copy the old data of the previous ManyToMany table to the new one with the additional field.
I followed a solution explained here:
Django migration error :you cannot alter to or from M2M fields, or add or remove through= on M2M fields
I want to test the function that will migrate the data in a test function but I don't understand what to do.
here my code:
survey/models:
class Survey(BaseModel):
name = models.CharField(max_length=256, help_text='Survey name')
user = models.ManyToManyField(User, blank=True, help_text='patient')
survey/models:
class SurveyStatus(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE)
survey = models.ForeignKey(Survey, on_delete=models.CASCADE)
survey_status = models.CharField(max_length=10,
blank=True,
null=True,
choices=STATUS_SURVEY_CHOICES,
)
The function I wrote that need to copy the data from the previous M2M to the new one is the following one:
def create_through_relations(apps, schema_editor):
Survey = apps.get_model('survey', 'Survey')
SurveyStatus = apps.get_model('survey', 'SurveyStatus')
for survey in Survey.objects.all():
for user in survey.user.all():
SurveyStatus(
user=user,
survey=survey,
survey_status='active'
).save()
I don't understand what is apps? because it is not recognized by python
I don't understand why i need schema_editor because it's not used
it doesn't recognized my Survey or SurveyStatus models too
when i tried to run this script with
if __name__ == "__main__":
create_through_relations(survey)
I've got this error
NameError: name 'survey' is not defined
and if i tried this function
from django.apps import apps
def create_through_relations():
Survey = apps.get_model('survey', 'Survey')
SurveyStatus = apps.get_model('survey', 'SurveyStatus')
for survey in Survey.objects.all():
for user in survey.user.all():
SurveyStatus(
user=user,
survey=survey,
survey_status='active'
).save()
when i tried to run this script with
if __name__ == "__main__":
create_through_relations()
I've got this error
django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.
If someone can help and explain me how to solve.thanks
1: Apps represent the different parts of your project (Django Apps)
2: You don't need it at this point. In general, it translates the models into SQL syntax.
3: python manage.py <...> does load the models for execution. Your file is trying to access data that isn't available that way.
4: The variable survey can't be found in python's main function, since you never declared it there. You need to trigger it inside your project.
5: You can test things by creating a test.py (Django Tests)
6: You don't need to transfer the data to a whole new table after changing a model, just extend the existing one and migrate the changes:
class BaseModel(models.Model):
created = models.DateTimeField('created', default=timezone.now)
changed = models.DateTimeField('changed', default=timezone.now, blank=True, null=True)
class Survey(BaseModel):
uuid = models.UUIDField(primary_key=False, default=uuid.uuid4, editable=False)
name = models.CharField(max_length=256, help_text='Survey name')
description = models.TextField('description', blank=True)
status = models.BooleanField(default=False) # paused/ active
class SurveyQuestion(BaseModel):
survey = models.ForeignKey(Survey, related_name='survey', on_delete=models.CASCADE)
text = models.CharField(max_length=256)
# 1 -> Text, # Integer, # ChoiceField, etc.
requested_result = models.IntegerField(default=0)
class QuestionResult(BaseModel):
user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
question = models.ForeignKey(SurveyQuestion, related_name='survey_question', on_delete=models.CASCADE)
answer = models.CharField(default='', max_length=256)
Scenario:
I am trying to create/insert data to Django model (POSTGRES database) employee and profile.
When I insert duplicate records (i.e. with duplicate work_email and duplicate employee_id) I am expecting a database error in an atomic transaction.
However, the below code doesn't throw any error, but when I see the database table no duplicate is created.
Dropped a database and created a new DB and new migrations to make sure there is no problem with the sync. However, the result is the same with no errors.
Any help is appreciated. Thanks
class Employee(models.Model):
"""
Employee table containing all the employee information.
"""
profile = models.OneToOneField(Profile, on_delete=models.CASCADE)
id = models.CharField(max_length=50, unique=True, blank=False, default=uuid.uuid4, editable=False, db_index=True)
employee_id = models.CharField(
max_length=100,
blank=False,
unique=True,
primary_key=True,
error_messages={'employee_id': "A user with employee id already exists."}
)
class Profile(AbstractUser):
"""
Enhancing user model with additional fields. This is in relation with a table ProfileExtras.
Extras can be utilised to add any fields to further enhance Profile with key value pair.
"""
email = None
date_of_birth = models.DateField(blank=False)
work_email = models.EmailField(
max_length=50,
blank=False,
unique=True,
db_index=True,
error_messages={'work_email': "A user with work email already exists."}
)
def create_employee(app_context, data_dict):
try:
with transaction.atomic():
employee_model = models.Employee()
profile_model = models.Profile()
# Data insertion logic
# e.g. setattr(employee_model, "first_name", "xxxxxx")
employee_model.profile = profile_model
profile_model.save()
employee_model.save()
except Exception as e:
log.error(e)
raise e
Found the solution. I was testing with Django test. It is found that data is not retained between each test, hence the duplicate error is not raised.
My Model:
class Wishlist(models.Model):
home = models.ForeignKey(Home, on_delete=models.CASCADE, null=False, blank=False)
user = models.ForeignKey(User, on_delete=models.CASCADE, null=False, blank=False)
def __str__(self):
return "{} - {}".format(self.user.username, self.home.address)
class Meta:
ordering = ('user',)
My serializer
class WishlistSerializer(serializers.ModelSerializer):
home = serializers.RelatedField(required=True, queryset=home_models.Home.objects.all())
user = serializers.RelatedField(required=True, queryset=User.objects.all())
class Meta:
model = Wishlist
fields = ('id', 'user', 'home',)
My View
class WishlistAdd(CreateAPIView):
"""
Add a new Home in User wishlist
"""
serializer_class = serializers.UserWishlistSerializer
queryset = Wishlist.objects.all()
When I try to do a POST request to create the new record I receive the following error: IntegrityError at /user/wishlist/ NOT NULL constraint failed: user_wishlist.home_id
All of these happens after a git merge, but i don't notice any differences between the branches
My guess is that your sqlite database is tracked by git. This means that whenever you switch to the broken branch, lack of data integrity causes this error – even if the application code is perfectly the same. I recommend adding the database file to your .gitignore, and looking into Django's fixtures instead.
That error means you did not provide a home id in your post payload, though this error should be caught on the serializer level because you specified required there as well. The reason it is required is because because you specified in your model field as a non-nullable field
home = models.ForeignKey(Home, on_delete=models.CASCADE, null=False, blank=False)
if you dont wish for it to be required specify null=True, blank=True. Also you need to ensure that home exists. good luck.
This my model of first Database DB1:
from django.db import models
class Company(models.Model):
name = models.CharField(max_length=100, null=True)
address = models.TextField(max_length=200, null=True)
website = models.CharField(max_length=200, null=True)
conatct_no = models.CharField(max_length=20, null=True)
email = models.EmailField(max_length=20, null=True)
logo = models.FileField(upload_to='logo/', blank=True, null=True)
created = models.DateTimeField('company created', auto_now_add=True)
updated = models.DateTimeField('company updated', auto_now=True, null=True)
def __unicode__(self): # Python 3: def __str__(self):
return self.name
Model of 2nd Database Db2:
from django.db import models
from leavebuddymaster.models import Company
class Department(models.Model):
company = models.ForeignKey(Company)
department_name = models.CharField(max_length=50, null=True)
created = models.DateTimeField('department created', auto_now_add=True)
updated = models.DateTimeField('department updated', auto_now=True, null=True)
def __unicode__(self): # Python 3: def __str__(self):
return self.department_name
Now when i open the Department table it gives me a error as:
ProgrammingError at /admin/leavebuddyapp/department/
(1146, "Table 'leavebuddy_master.leavebuddyapp_department' doesn't exist")
I have done all the settings in settings.py correctly for the two databases. Can you please guide me in the right direction. Thanx in advance.
You're correct, Django does not currently support foreign key relationships spanning multiple databases.
From Cross-database relations [Edit 2020: Django version bump]:
If you have used a router to partition models to different databases, any foreign key and many-to-many relationships defined by those models must be internal to a single database.
This is because of referential integrity. In order to maintain a relationship between two objects, Django needs to know that the primary key of the related object is valid. If the primary key is stored on a separate database, it’s not possible to easily evaluate the validity of a primary key.
A solution I thought up that you could try (though it may present other problems):
from leavebuddymaster.models import Company
class Department(models.Model):
company_id = models.IntegerField()
#property
def company(self):
return Company.objects.get(pk=self.company_id)
This allows you to refer to Department.company like you normally would in your example. Setting it would just be a matter of Department.company_id = Company.pk. Hope it helps, or at least inspires a better solution!