How can I dynamically create multiple tables for one specific model?
I have a model (e.g. "Data") and want each user to get a separate database table for this model.
There will be between 30 and 40 users, each with about 20 million entries in "Data". So that the database table does not get 800 million rows, I would like to split it.
Of course, it is also possible that there will be more in the future
Is this still possible with Django ORM?
I'm on PostgreSQL
Thanks :)
For a strict requirement, I had to go through this. I will just leave my solution here for future reference.
After User creation, let's say we want to create a "user_data_100" table with model "UserData" with pgsql database.
Note: here "100" is the "User" model(table entries) primary key value.
in app/model.py create the user model.
from django.db import models
from django.db.models.signals import pre_save, post_save
from django.dispatch import receiver
from django.db import connection
from django.contrib.auth.models import User
class UserData(models.Model):
class Meta:
pass
dt_column_one = models.CharField(max_length=10,)
dt_column_two = models.CharField(max_length=5,)
def create_model_by_prototype(model_name, schema_name, tbl_name, prototype):
import copy
class Meta:
pass
setattr(Meta, "db_table", schema_name + "\".\"" + table_name)
fields = {}
for field in prototype._meta.fields:
fields[field.name] = copy.deepcopy(field)
attrs = {'__module__': "app.models.models", 'Meta': Meta}
attrs.update(fields)
model = type(model_name, (models.Model,), attrs)
return model
Note: function "create_model_by_prototype" will generate a model with given table name.
Now let's use the function. I assume the user will get created from the admin site. Let's create a post_save function to create UserData table.
#receiver(post_save, sender=User)
def create_user_data_table(sender, instance, *args, **kwargs):
print('User post save')
if kwargs['created']:
user_id = instance.id
dy_table_name = "user_data_" + str(user_id)
model= create_model_by_prototype("MyUserData", "public", dy_table_name, UserData)
with connection.schema_editor() as schema_editor: schema_editor.create_model(model)
else:
print('updated')
hope this works for you. Thanks.
Related
Suppose we have two models that have signal to the User model:
from django.db import models
from django.contrib.auth.models import User
from django.db.models import signals
class Company(User):
name = models.CharField(null=True, blank=True, max_length=30)
if created:
Company.objects.create(
user_ptr_id=instance.id,
username=instance.username,
password=instance.password,
email=instance.email,
first_name=instance.first_name,
last_name=instance.last_name,
is_active=instance.is_active,
is_superuser=instance.is_superuser,
is_staff=instance.is_staff,
date_joined=instance.date_joined,
)
signals.post_save.connect(
create_company, sender=User, weak=False, dispatch_uid="create_companies"
)
class Individual(User):
name = models.CharField(null=True, blank=True, max_length=30)
def create_job_seeker(sender, instance, created, **kwargs):
"""
:param instance: Current context User instance
:param created: Boolean value for User creation
:param kwargs: Any
:return: New Seeker instance
"""
if created:
'''
we should add a condition on whether the Company uses the same username
if true, then, we must not create a JobSeeker and we would disable the account using
Firebase Admin
'''
JobSeeker.objects.create(
user_ptr_id=instance.id,
username=instance.username,
password=instance.password,
email=instance.email,
first_name=instance.first_name,
last_name=instance.last_name,
is_active=instance.is_active,
is_superuser=instance.is_superuser,
is_staff=instance.is_staff,
date_joined=instance.date_joined,
)
signals.post_save.connect(
create_job_seeker, sender=User, weak=False, dispatch_uid="create_job_seekers"
)
Now, each time a User is created we should be allowed to extend it through both Individual and Company models. But, I want to prohibit the usage of both objects. User can either have a Company or an Individual object to be edited not both. Should I override the save method such as this:
def save(self, *args, **kwargs):
if not Company.objects.filter(username=self.username).exists():
super(Model, self).save(*args, **kwargs)
else:
raise 'Some error'
Or should I add a condition on the created method such as this:
...
if created and Company.objects.filter(username, self.username).exists() == False:
Company.objects.create(
...
Which approach is better? And is there another approach that you might suggest?
Signals, for most cases, I believe are the best way to handle sharing data between models assuming the CRUD for each related model isn't done together. So post_save, pre_save, post_delete, pre_delete and so are typically the best way to go about handling data that any given model instance relies on. This can be true about manipulating model data after a save. Signals were designed specifically for this reason. The other great thing about signals is you can connect them throughout your project and not necessarily just where the Model is defined. Just import the model and the signal you want to connect to it and bam!
How to use signals? follow django's documentation here. it's very simple
https://docs.djangoproject.com/en/4.0/topics/signals/
In one celery task I need to create temporary table in database. In this article Daniel Roseman explained how to create one. But this solution does not work in Django 1.9. I tried to look into Django docs and Google but I was unable to find anything useful.
Code from mentioned article which worked in Django 1.8:
from django.db import models, cursor
from django.contrib.contenttypes.management import update_contenttypes
from django.core.management import call_command
class TempCustomerAddress(models.Model):
address = models.ForeignKey('accounts.Address')
legacy_id = models.CharField(max_length=12, unique=True)
class Meta:
app_label = 'utils'
class Command(NoArgsCommand):
def handle_noargs(self, **options):
models.register_models('utils', TempCustomerAddress)
models.signals.post_syncdb.disconnect(update_contenttypes)
call_command('syncdb')
# ... do importing and stuff referring to TempCustomerAddress ...
cursor = connection.cursor()
cursor.execute('DROP TABLE `utils_tempcustomeraddress`')
In django 1.9 you actually don't need to register anything. You just create model the same way as in models.py and that's it. You only need to make sure that it is not in models.py file because than it will be permanent model.
This example assumes that you already ran all migrations.
from django.db import models, cursor
from django.contrib.contenttypes.management import update_contenttypes
from django.core.management import call_command
class TempCustomerAddress(models.Model):
address = models.ForeignKey('accounts.Address')
legacy_id = models.CharField(max_length=12, unique=True)
class Meta:
app_label = 'utils'
class Command(NoArgsCommand):
def handle_noargs(self, **options):
with connection.cursor() as cursor:
cursor.execute('DROP TABLE IF EXISTS utils_tempcustomeraddress')
cursor.execute('''
CREATE TABLE utils_tempcustomeraddress (
id INTEGER PRIMARY KEY NOT NULL,
address_id REFERENCES accounts_address (id),
legacy_id VARCHAR(12) UNIQUE
);
'''
# ... do importing and stuff referring to TempCustomerAddress ...
cursor.execute('DROP TABLE `utils_tempcustomeraddress`')
I needed to create a temporary model derived from a "permanent" model, and use temporary table storage to avoid polluting the tables of the permanent one. After a lot of poking around including an article relating to Django 0.96, some newer material it points to for Django 1.2 and some old material based on migration tech integrated into Django I finally came up with a recipe that works with Django 2.0.
First, I needed to explicitly specify the database table name using the Meta:
model_name = re.sub('[#.]', '_', 'some_string')
class Meta:
app_label = original_model._meta.app_label
#
# Use the explicit name for the database table.
#
db_table = '"' + model_name.lower() + '"'
Then I created the Model class by copying what I needed from the original:
attr = {'__module__': __name__, 'Meta': Meta}
local_fields = [field.name for field in original_model._meta.local_fields]
for field in original_model._meta.fields:
#
# Clone only the fields which we need, not forgetting that we only
# want one primary key.
#
clone = field.clone()
if field.name in local_fields:
local_fields.remove(field.name)
else:
clone.primary_key = False
if not isinstance(field, (db_models.AutoField, db_models.OneToOneField, db_models.ManyToManyField)):
attr[field.name] = clone
new_model = type(model_name, (db_models.Model,), attr)
The hard part was tracking down how to create the new table for the model. Once found, the answer was simple:
from django.db import connection
with connection.schema_editor() as schema_editor:
schema_editor.create_model(new_model)
I'm pretty sure my question isn't clear but I didn't know how to put it.
The thing is : I have a Task model, and a TaskHistory model. When I create a Task model, I want automatically to create an associated TaskHistory object.
For example, I want to create a Task at datetime.now(), with the value "example", and when I do this, a TaskHistory object is created, with
created_on = datetime.now()
last_modification = datetime.now()
old_value = "example"
new_value = "example".
Can I do that in the model field of my Task model ?
I'm pretty new to Django.
Edit : the two models are linked to each other, my TaskHistory object would have task = Task if it's created like that.
If all the fields you've specified is available on the TaskHistory model you could override the create() method.
class Task(models.Model):
#other fields snipped for brevity
history = models.ForeignKey(TaskHistory)
#classmethod
def create(cls, *args, **kwargs):
task = cls(**kwargs)
history_kwargs = {'created_on':datetime.now(), 'last_modification':datetime.now(), 'old_value':'None', 'new_value': 'wee'}
history = TaskHistory(**history_kwargs)
history.save()
task.history = history
return task
and in your view or what have you call it like this
task = Task.create(insert_values_here_for_Task)
task.save()
Two ways to do this:
Override the save method on your Task model.
Use the post_save signal.
This is how you do it by using signals:
from django.db.models.signals import post_save
from django.dispatch import receiver
from myapp.models import Task
#receiver(post_save, sender=Task)
def my_handler(sender, **kwargs):
task=sender
history_kwargs = {'created_on':datetime.now(), 'last_modification':datetime.now(), 'old_value':'None', 'new_value': 'wee'}
history = TaskHistory(**history_kwargs)
history.save()
task.history = history
task.save
I am developing an app which extends the auth_user table, whenever any user register to the app, its one instance is created in the extended table,It works very well from simple url(manually), but when I login from admin side and add user it will also create the instance in extended table but does not show or take the user name, it just shows the blank row.
model.py code is as:
from django.db import models
from django.contrib.auth.models import User
from django.db.models.signals import post_save
class Drinker(models.Model):
user =models.OneToOneField(User)
birthday =models.DateField()
name =models.CharField(max_length=100)
def __unicode__(self):
return self.name
#create our user object to attach to our drinker object
def create_drinker_user_callback(sender, instance, **kwargs):
drinker, new=Drinker.objects.get_or_create(user=instance)
post_save.connect(create_drinker_user_callback, User)
the last 3 lines three create the instance in extended table drinker but does not shows any value just take the user_id from auth_user table. it does not take name and birthday
You're not populating the Drinker.name field.
The row is blank because the unicode() method is returning nothing.
One way to solve this would be to set it explicitly in the signal handler:
def create_drinker_user_callback(sender, instance, **kwargs):
drinker, new=Drinker.objects.get_or_create(user=instance, name=instance.get_full_name())
post_save.connect(create_drinker_user_callback, User)
or ditch the (possibly redundant) name field altogether and normalise it a bit:
class Drinker(models.Model):
user = models.OneToOneField(User)
birthday = models.DateField()
def __unicode__(self):
return self.user.get_full_name()
http://docs.djangoproject.com/en/dev/ref/models/fields/#choices
i've read through the documentation and this implies using a database table for dynamic data, however it states
choices is meant for static data that doesn't change much, if ever.
so what if i want to use choices, but have it select multiple because the data i'm using is quite static, e.g days of the week.
is there anyway to achieve this without a database table?
ChoiceField is not really suitable for multiple choices, instead I would use a ManyToManyField. Ignore the fact that Choices can be used instead of ForeignKey for static data for now. If it turns out to be a performance issue, there are ways to represent this differently (one being a binary mask approach), but they require way more work.
This worked for me:
1) create a Form class and set an attribute to provide your static choices to a MultipleChoiceField
from django import forms
from myapp.models import MyModel, MYCHOICES
class MyForm(forms.ModelForm):
myfield = forms.MultipleChoiceField(choices=MYCHOICES, widget=forms.SelectMultiple)
class Meta:
model = MyModel
2) then, if you're using the admin interface, set the form attribute in your admin class so tit will use your customized form
from myapp.models import MyModel
from myapp.forms import MyForm
from django.contrib import admin
class MyAdmin(admin.ModelAdmin):
form = MyForm
admin.site.register(MyModel, MyAdmin)
Try following configuration. In models.py
class MyModel(models.Model):
my_choices = models.TextField(help_text="It's a good manners to write it")
in forms.py
CHOICES = ((1,1), (2,2))
class MyForm(forms.ModelForm):
my_choices = forms.MultipleChoiceField(choices=CHOICES)
def __init__(self, *args, **kwargs):
super(MyForm, self).__init__(*args, **kwargs)
# maybe you can set initial with self.fields['my_choices'].initial = initial
# but it doesn't work wity dynamic choices
obj = kwargs.get('instance')
if obj:
initial = [i for i in obj.my_choices.split(',')]
self.initial['my_choices'] = initial
def clean_lead_fields(self):
return ','.join(self.cleaned_data.get('my_choices', []))
in admin.py
#admin.register(MyModel)
class MyModelAdmin(admin.ModelAdmin):
form = MyModelForm