Django, getting data from another user (schema) in an Oracle 19 DB - django

I am using Django 4.1, Oracle 19 with the python package oracledb.
I am logged in as user djangousr and the schema I am trying to get the data from is "user123"
I am able to retrieve data in a file outside of Django with the same connection information.
But in Django, I keep getting the same error.
I hope you have a solution for me as I was not able to find anything elsewhere.
Thank you.
ORA-00942: table or view does not exist
Below is the SQL from the debug screen that I am able to run fine in SQL developer.
('SELECT "USER"."USER_ID" FROM "user123.USER_TABLE"')
I will also provide the model and the view:
class SecurityUserData(models.Model):
user_id = models.IntegerField(primary_key=True)
user_name = models.CharField(max_length=100)
user_password = models.CharField(max_length=100)
user_first_name = models.CharField(max_length=30, blank=True, null=True)
user_last_name = models.CharField(max_length=30, blank=True, null=True)
class Meta:
managed = False
db_table = 'SEC_USER'
And the view:
def display_user_data(request):
user_data = SecurityUserData.user123.all()
return render(request, 'all.html', {'user_data': user_data})

Related

Test a data migration ManyToMany in Django

I tried to add a field to my ManyToMany relationship models in Django.
So step by step, I created the new model and apply makemigrations and migrate.
I checked I have the new table in my postgresql database.
Now before I will add the through keyword in the ManyToMany field I want to write a function in the migration file that will copy the old data of the previous ManyToMany table to the new one with the additional field.
I followed a solution explained here:
Django migration error :you cannot alter to or from M2M fields, or add or remove through= on M2M fields
I want to test the function that will migrate the data in a test function but I don't understand what to do.
here my code:
survey/models:
class Survey(BaseModel):
name = models.CharField(max_length=256, help_text='Survey name')
user = models.ManyToManyField(User, blank=True, help_text='patient')
survey/models:
class SurveyStatus(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE)
survey = models.ForeignKey(Survey, on_delete=models.CASCADE)
survey_status = models.CharField(max_length=10,
blank=True,
null=True,
choices=STATUS_SURVEY_CHOICES,
)
The function I wrote that need to copy the data from the previous M2M to the new one is the following one:
def create_through_relations(apps, schema_editor):
Survey = apps.get_model('survey', 'Survey')
SurveyStatus = apps.get_model('survey', 'SurveyStatus')
for survey in Survey.objects.all():
for user in survey.user.all():
SurveyStatus(
user=user,
survey=survey,
survey_status='active'
).save()
I don't understand what is apps? because it is not recognized by python
I don't understand why i need schema_editor because it's not used
it doesn't recognized my Survey or SurveyStatus models too
when i tried to run this script with
if __name__ == "__main__":
create_through_relations(survey)
I've got this error
NameError: name 'survey' is not defined
and if i tried this function
from django.apps import apps
def create_through_relations():
Survey = apps.get_model('survey', 'Survey')
SurveyStatus = apps.get_model('survey', 'SurveyStatus')
for survey in Survey.objects.all():
for user in survey.user.all():
SurveyStatus(
user=user,
survey=survey,
survey_status='active'
).save()
when i tried to run this script with
if __name__ == "__main__":
create_through_relations()
I've got this error
django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.
If someone can help and explain me how to solve.thanks
1: Apps represent the different parts of your project (Django Apps)
2: You don't need it at this point. In general, it translates the models into SQL syntax.
3: python manage.py <...> does load the models for execution. Your file is trying to access data that isn't available that way.
4: The variable survey can't be found in python's main function, since you never declared it there. You need to trigger it inside your project.
5: You can test things by creating a test.py (Django Tests)
6: You don't need to transfer the data to a whole new table after changing a model, just extend the existing one and migrate the changes:
class BaseModel(models.Model):
created = models.DateTimeField('created', default=timezone.now)
changed = models.DateTimeField('changed', default=timezone.now, blank=True, null=True)
class Survey(BaseModel):
uuid = models.UUIDField(primary_key=False, default=uuid.uuid4, editable=False)
name = models.CharField(max_length=256, help_text='Survey name')
description = models.TextField('description', blank=True)
status = models.BooleanField(default=False) # paused/ active
class SurveyQuestion(BaseModel):
survey = models.ForeignKey(Survey, related_name='survey', on_delete=models.CASCADE)
text = models.CharField(max_length=256)
# 1 -> Text, # Integer, # ChoiceField, etc.
requested_result = models.IntegerField(default=0)
class QuestionResult(BaseModel):
user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
question = models.ForeignKey(SurveyQuestion, related_name='survey_question', on_delete=models.CASCADE)
answer = models.CharField(default='', max_length=256)

Django custom migration RunPython cannot access ForeignKey field in another database

I'm writing a django migration operation to change some data in the 'default' database. My app has access to a 'services' database, which contains data I cannot change.
The two relevant fields in the default.table are:
data_sets_to_remove = models.CharField(blank=True, null=False, max_length=100, default="")
data_sets_new = ArrayField(models.IntegerField(null=True, blank=True), null=True, blank=True)
i.e., i'm migrating data from data_sets_to_remove into a new format and adding to data_sets_new. To do this, I need to access data from the 'services' database in the migration operation.
def forwards_func(apps, schema_editor):
DataRelease = apps.get_model('registry', "datarelease")
Survey = apps.get_model('registry', "survey")
data_release_queryset = DataRelease.objects.using('services').all().values('id', 'name', 'survey')
But for some reason, the foreign key field 'survey' on the DataRelease model is not available in this context.
django.core.exceptions.FieldError: Cannot resolve keyword 'survey'
into field. Choices are: id, name
Can anyone shed any light on this? I assume the issue is around accessing data from a secondary database within a migration operation, but if i run the same code in the console, it works fine...
The relevant Survey and DataRelease models in the services database:
class Survey(models.Model):
name = models.CharField(blank=False, null=False, max_length=100, unique=True)
class DataRelease(models.Model):
name = models.CharField(blank=False, null=False, max_length=100)
survey = models.ForeignKey(Survey, related_name='data_releases', on_delete=models.CASCADE)
:facepalm:
The answer was staring me in the face. Switched the import of the associated models from:
DataRelease = apps.get_model('registry', "datarelease")
Survey = apps.get_model('registry', "survey")
to:
from services.registry.models import DataRelease, Survey
Now I can access the related fields in the migration operation. Hopefully this helps someone else in the future!

How to save unicode string in admin site django?

I want to change data in admin site of django but when I save it, it will turn to something like 'C?a 4'. '?' is the unicode word. I have change data in database to utf8mb4 and if I change data directly in mysql, it will display properly in admin site. However, it can not display like i want if I change data in admin site of django. I'm using Python 3. This is my model
class Camera(models.Model):
camera_id = models.AutoField(primary_key=True)
camera_name = models.CharField(max_length=45, blank=True, null=True)
stream_url = models.CharField(max_length=2083, blank=True, null=True)
class Meta:
managed = True
db_table = 'camera'
def __str__(self):
return self.camera_name

Django cms - plugin manyToMany variable always empty

I wrote a simple plugin which display contact people, but I need to exclude some contact on certain pages. So I added a related model to my plugin which use an "structure_to_exclude" ManyToMany relationship. My issue, when I get this variable it's allways empty.
The cms_plugins.py
class VMContactContactPlugin(CMSPluginBase):
module = 'VM Contact Plugin'
render_template = 'vm_contact/calendars/contacts_list.html'
model = VMContactCalendarPluginModel
name = _('VM Contact plugin')
def render(self, context, instance, placeholder):
print 'Instance : {0}'.format(instance)
inst = instance.structure_to_exclude.all()
print 'Instance.all() result : {0}'.format(inst)
structures = Structure.objects.exclude(contact=None).exclude(pk__in=instance.structure_to_exclude.all().values_list('id',flat=True))
context.update({
'structures': structures,
})
return context
plugin_pool.register_plugin(VMContactContactPlugin)
The related model
class VMContactCalendarPluginModel(CMSPlugin):
structure_to_exclude = models.ManyToManyField(
Structure,
verbose_name=_(u'Structures à exclure'),
)
The Structure Models (Polymorphic !!)
class Structure(PolymorphicModel):
contact = models.ForeignKey(Contact, blank=True, null=True)
members = models.ManyToManyField(Contact, blank=True, null=True, related_name='%(class)s_members')
title = models.CharField(max_length=50, default='Castor')
description = models.CharField(max_length=254, blank=True)
categories = CategoryManyToManyField('aldryn_categories.Category',
verbose_name=_('categories'),
blank=True)
calendars = models.ManyToManyField(Calendar, blank=True)
has_pages = models.BooleanField(default=True)
avatar = FilerFileField(null=True, blank=True,
on_delete=models.SET_NULL)
classcss = models.CharField(max_length=1, choices=CSS_CLASS, default='5')
order = models.PositiveSmallIntegerField(default=0)
class Meta:
ordering = ['order']
Print results :
Instance : 93
Instance.all() result : []
Any idea ? I tried to retrieve the plugin instance with the ID (93) to be sure that was not an issue with instance var but it doesn't change anything...
Regards, robin
For every plugin you create, there's two versions once you publish.
The public and the draft versions. So its perfectly fine for the ids to change.
Because relationships vary from project to project, anytime your plugin has relationships, you need to explicitly tell the cms how to "copy over" these relationships when publishing the page.
Please adapt your plugin model to have the following method:
def copy_relations(self, oldinstance):
self.structure_to_exclude = oldinstance.structure_to_exclude.all()
You can read more about plugin relations in our docs.

Why is a Django model taking so long to load in admin?

I have a fairly simple Django set up for a forum, and one of the most basic models is this, for each thread:
class Post(models.Model):
created = models.DateTimeField(auto_now_add=True)
last_reply = models.DateTimeField(auto_now_add=True, blank=True, null=True)
username = models.ForeignKey(User, related_name="forumuser")
fixed = models.BooleanField(_("Sticky"), default=False)
closed = models.BooleanField(default=False)
markdown_enabled = models.BooleanField(default=False)
reply_count = models.IntegerField(default=0)
title = models.CharField(_("Title Post"), max_length=255)
content = models.TextField(_("Content"), blank=False)
rating = models.IntegerField(default=0)
followers = models.IntegerField(default=0)
ip_address = models.CharField(max_length=255)
def __unicode__(self):
return self.title
def get_absolute_url(self):
return "/post/%s/" % self.id
Then we have some replies:
class PostReply(models.Model):
user = models.ForeignKey(User, related_name='replyuser')
post = models.ForeignKey(Post, related_name='replypost')
created = models.DateTimeField(auto_now_add=True)
content = models.TextField()
ip_address = models.CharField(max_length=255)
quoted_post = models.ForeignKey('self', related_name='quotedreply', blank=True, null=True)
rating = models.IntegerField(default=0)
reply_order = models.IntegerField(default=1)
Now, currently there just over 1600 users, 6000 Posts, and 330,000 PostReply objects in the db for this setup. When I run this SQL query:
SELECT * FROM `forum_post` LIMIT 10000
I see that Query took 0.0241 sec which is fine. When I browse to the Django admin section of my site, pulling up an individual Post is rapid, as is the paginated list of Posts.
However, if I try and pull up an individual PostReply, it takes around 2-3 minutes to load.
Obviously each PostReply admin page will have a dropdown list of all the Posts in it, but can anyone tell me why this or anything else would cause such a dramatically slow query? It's worth noting that the forum itself is pretty fast.
Also, if it is something to do with that dropdown list, has anyone got any suggestions for making that more usable?
Try to add all foreign keys in raw_id_fields in admin
class PostReplyAdmin(ModelAdmin):
raw_id_fields = ['user', 'post', 'quoted_post']
This will decrease page's load time in change view. The problem is that django loads ForeignModel.objects.all() for each foreign key's dropdowns.
Another way is to add foreign keys in autocomplete_fields (docs) in admin
class PostReplyAdmin(ModelAdmin):
autocomplete_fields = ['user', 'post', 'quoted_post']
As pointed by #Andrey Nelubin the problem for me was indeed in the page loading all related models for each foreign key's dropdown. However, with autocomplete_fields selects are turned into autocomplete inputs (see figure below), which load options asynchronously.