I want to develop a server side of an app that holds users.
Of course I need a table in database holding the user information.
At first I may write
class User(models.Model): # using django models
userid = ...
password = ...
which gives me a database table containing userid and password.
However, I might want to add some attributes (maybe Credit, Birthday...so on) to each user in the future. I just can't think up all of them right now. And I can't know what attributes I would really need in the future.
How can I deal with it?
There's already a user table in Django. This table is automatically create when you first apply the migration with 'manage.py migrate' command.
In database schema, this table is listed as auth_user and you can import it into Django with the following command
from django.contrib.auth.models import User
Django provides a default model for User. you can use it like this.
from django.contrib.auth.models import User
and as per your second query. you can do so by creating another model and adding a ForeignKey or OneToOneField of User model to link it with each user.
class Customuserprofile(models.Model):
user=models.OneToOneField(settings.AUTH_USER_MODEL)
credit=models.CharField()
birthday=models.DateTimeField()
well, if you want to add some field to the User you can use AbstractUser or AbstractBaseUser model read this that explain the differences and give a example https://simpleisbetterthancomplex.com/tutorial/2016/07/22/how-to-extend-django-user-model.html#abstractbaseuser
suddenly, my page got so many users in db that the a filter for email over the auth_user table almost failing because of the extremely big number of users.
Since the table comes built-in, I need to add db_index=True to columns in this table, any idea how to do this?
One quick and easy way would be to manually add the index using RunSQL in a migration.
operations = [
migrations.RunSQL("CREATE INDEX..."),
]
It's not very elegant. For one thing, the migration will be for a different app (since you don't control the auth migrations). For another, the schema will technically be out of sync with the database. However, I don't think there are any negative consequences in this case, since Django doesn't do anything with db_index other than create the index.
One possibility is to substitute the user model with a custom one, which will have proper indices and any other field that you require. There is extensive documentation on Django docs: Substituting a custom User model on how to achieve this. This is how I did it on a particular case with a similar issue.
Another possibility is to extend the user model, which could have a particular field repeated from the original model, on which there is an index. Disclaimer: I am genuinely against that for obvious reasons, but I have seen this happening, as this approach is easier to code than the first. This would be very bad though if there are many fields.
This is a good question imo. I would love to know if there is another possibility which I miss.
I had the same problem, but with an additional twist--- I already had an index created by South in my table. So if I added a RunSQL("Create index") to my migration, it would add a second index to my database. But at the same time, if I don't include some create index action in the migrations, then I won't be able to properly spin up new databases.
Here's my solution, some python code to check for the existence of an index using some of the private-ish methods in schema_editor
project/appname/migrations/0002.py:
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
from django.db.migrations import RunPython
def forward__auth_user__email__index(apps, schema_editor):
auth_user = apps.get_model("auth", "User")
target_column_to_index = 'email'
se = schema_editor
# if there aren't any indexes already, create one.
index_names = se._constraint_names(auth_user, [target_column_to_index], index=True)
if len(index_names) == 0:
se.execute(se._create_index_sql(auth_user, [auth_user._meta.get_field('email')]))
def reverse__auth_user__email__index(apps, schema_editor):
auth_user = apps.get_model("auth", "User")
target_column_to_index = 'email'
se = schema_editor
# delete any indexes for this table / column.
index_names = se._constraint_names(model, [target_column_to_index], index=True)
for index_name in index_names:
se.execute(se._delete_constraint_sql(se.sql_delete_index, auth_user, index_name))
class Migration(migrations.Migration):
dependencies = [
('frontend', '0001_initial'),
]
operations = [
RunPython(forward__auth_user__email__index, reverse_code=reverse__auth_user__email__index)
]
I use Django and South for my database. Now I want to add a new Model and a field in an existing model, referencing the new model. For example:
class NewModel(models.Model):
# a new model
# ...
class ExistingModel(models.Model):
# ... existing fields
new_field = models.ForeignKey(NewModel) # adding this now
Now South obviously complains that I added a non-null field and asks me to enter a one-off value. But what I really want is to create a new NewModel instance for every existing ExistingModel instance, thus fulfilling the database requirements. Is that possible somehow?
The easiest way to do this is to write a schema migration that makes the column change, and then write a datamigration to correctly fill in the value. Depending on the database you're using you'll have to do this in slightly different ways.
Sqlite
For Sqlite, you can add a sentinel value for the relation and use a datamigration to fill it in without any issue:
0001_schema_migration_add_foreign_key_to_new_model_from_existing_model.py
from south.db import db
from south.v2 import SchemaMigration
from django.db import models
class Migration(SchemaMigration):
def forwards(self, orm):
db.add_column('existing_model_table', 'new_model',
self.gf('django.db.models.fields.related.ForeignKey')(default=0, to=orm['appname.new_model']), keep_default=False)
0002_data_migration_for_new_model.py:
class Migration(DataMigration):
def forwards(self, orm):
for m in orm['appname.existing_model'].objects.all():
m.new_model = #custom criteria here
m.save()
This will work just fine, with no issues.
Postgres and MySQL
With MySql, you have to give it a valid default. If 0 isn't actually a valid Foreignkey, you'll get errors telling you so.
You could default to 1, but there are instances where that isn't a valid foreign key (happened to me because we have different environments, and some environments publish to other databases, so the IDs rarely match up (we use UUIDs for cross-database identification, as God intended).
The second issue you get is that South and MySQL don't play well together. Partially because MySQL doesn't have the concept of DDL transactions.
In order to get around some issues you will inevitably face (including the error I mentioned above and from South asking you to mark orm items in a SchemaMigration as no-dry-run), you need to change the above 0001 script to do the following:
0001_schema_migration_add_foreign_key_to_new_model_from_existing_model.py
from south.db import db
from south.v2 import SchemaMigration
from django.db import models
class Migration(SchemaMigration):
def forwards(self, orm):
id = 0
if not db.dry_run:
new_model = orm['appname.new_model'].objects.all()[0]
if new_model:
id = new_model.id
db.add_column('existing_model_table', 'new_model',
self.gf('django.db.models.fields.related.ForeignKey')(default=id, to=orm['appname.new_model']), keep_default=False)
And then you can run the 0002_data_migration_for_new_model.py file as normal.
I advise using the same example above for Postgres and for MySql. I don't remember any issues offhand with Postgres with the first example, but I'm certain the second example works for both (tested).
You want a data migration to supplement your schema migration in this scenario.
South has a nice step by step tutorial on how to achieve this in the docs, here.
It's not uncommon in South to have the desired outcome spread over two or three schema/data migrations as its not always possible to do it in one big hit (sometimes depends on the underlying db if it will tolerate adding a non null column with no default). So in this case you might add a schema migration that has a default, then a data migration with your object manipulation then a final schema migration.
Is there anything wrong with running alter table on auth_user to make username be varchar(75) so it can fit an email? What does that break if anything?
If you were to change auth_user.username to be varchar(75) where would you need to modify django? Is it simply a matter of changing 30 to 75 in the source code?
username = models.CharField(_('username'), max_length=30, unique=True, help_text=_("Required. 30 characters or fewer. Letters, numbers and #/./+/-/_ characters"))
Or is there other validation on this field that would have to be changed or any other repercussions to doing so?
See comment discussion with bartek below regarding the reason for doing it.
Edit: Looking back on this after many months. For anyone who doesn't know the premise: Some apps don't have a requirement or desire to use a username, they use only email for registration & auth. Unfortunately in django auth.contrib, username is required. You could start putting emails in the username field, but the field is only 30 char and emails may be long in the real world. Potentially even longer than the 75 char suggested here, but 75 char accommodates most sane email addresses. The question is aimed at this situation, as encountered by email-auth-based applications.
There's a way to achieve that without touching the core model, and without inheritance, but it's definitely hackish and I would use it with extra care.
If you look at Django's doc on signals, you'll see there's one called class_prepared, which is basically sent once any actual model class has been created by the metaclass. That moment is your last chance of modifying any model before any magic takes place (ie: ModelForm, ModelAdmin, syncdb, etc...).
So the plan is simple, you just register that signal with a handler that will detect when it is called for the User model, and then change the max_length property of the username field.
Now the question is, where should this code lives? It has to be executed before the User model is loaded, so that often means very early. Unfortunately, you can't (django 1.1.1, haven't check with another version) put that in settings because importing signals there will break things.
A better choice would be to put it in a dummy app's models module, and to put that app on top of the INSTALLED_APPS list/tuple (so it gets imported before anything else). Here is an example of what you can have in myhackishfix_app/models.py :
from django.db.models.signals import class_prepared
def longer_username(sender, *args, **kwargs):
# You can't just do `if sender == django.contrib.auth.models.User`
# because you would have to import the model
# You have to test using __name__ and __module__
if sender.__name__ == "User" and sender.__module__ == "django.contrib.auth.models":
sender._meta.get_field("username").max_length = 75
class_prepared.connect(longer_username)
That will do the trick.
A few notes though:
You might want to change also the help_text of the field, to reflect the new maximum length
If you want to use the automatic admin, you will have to subclass UserChangeForm, UserCreationForm and AuthenticationForm as the maximum length is not deduced from the model field, but directly in the form field declaration.
If you're using South, you can create the following migration to change the column in the underlying database:
import datetime
from south.db import db
from south.v2 import SchemaMigration
from django.db import models
class Migration(SchemaMigration):
def forwards(self, orm):
# Changing field 'User.username'
db.alter_column('auth_user', 'username', models.CharField(max_length=75))
def backwards(self, orm):
# Changing field 'User.username'
db.alter_column('auth_user', 'username', models.CharField(max_length=35))
models = {
# ... Copy the remainder of the file from the previous migration, being sure
# to change the value for auth.user / usename / maxlength
Based on Clément and Matt Miller's great combined answer above, I've pulled together a quick app that implements it. Pip install, migrate, and go. Would put this as a comment, but don't have the cred yet!
https://github.com/GoodCloud/django-longer-username
EDIT 2014-12-08
The above module is now deprecated in favor of https://github.com/madssj/django-longer-username-and-email
Updated solution for the Django 1.3 version (without modifying manage.py):
Create new django-app:
monkey_patch/
__init__.py
models.py
Install it as first: (settings.py)
INSTALLED_APPS = (
'monkey_patch',
#...
)
Here is models.py:
from django.contrib.auth.models import User
from django.core.validators import MaxLengthValidator
NEW_USERNAME_LENGTH = 300
def monkey_patch_username():
username = User._meta.get_field("username")
username.max_length = NEW_USERNAME_LENGTH
for v in username.validators:
if isinstance(v, MaxLengthValidator):
v.limit_value = NEW_USERNAME_LENGTH
monkey_patch_username()
The solutions above do seem to update the model length. However, to reflect your custom length in admin, you also need to override the admin forms (frustratingly, they don't simply inherit the length from the model).
from django.contrib.auth.forms import UserChangeForm, UserCreationForm
UserChangeForm.base_fields['username'].max_length = NEW_USERNAME_LENGTH
UserChangeForm.base_fields['username'].widget.attrs['maxlength'] = NEW_USERNAME_LENGTH
UserChangeForm.base_fields['username'].validators[0].limit_value = NEW_USERNAME_LENGTH
UserChangeForm.base_fields['username'].help_text = UserChangeForm.base_fields['username'].help_text.replace('30', str(NEW_USERNAME_LENGTH))
UserCreationForm.base_fields['username'].max_length = NEW_USERNAME_LENGTH
UserCreationForm.base_fields['username'].widget.attrs['maxlength'] = NEW_USERNAME_LENGTH
UserCreationForm.base_fields['username'].validators[0].limit_value = NEW_USERNAME_LENGTH
UserCreationForm.base_fields['username'].help_text = UserChangeForm.base_fields['username'].help_text.replace('30', str(NEW_USERNAME_LENGTH))
As far as I know one can override user model since Django 1.5 which will solve a problem. Simple example here
If you simply modify the database table, you'll still have to deal with Django's validation, so it won't let you make one over 30 characters anyways. Additionally, the username validates so that it can't have special characters like # so simply modifying the length of the field wouldn't work anyways. My bad, looks like it handles that. Here's the username field from models.py in django.contrib.auth:
username = models.CharField(_('username'), max_length=30, unique=True, help_text=_("Required. 30 characters or fewer. Letters, numbers and #/./+/-/_ characters"))
Creating email auth is not hard. Here's a super simple email auth backend you can use. Al l you need to do after that is add some validation to ensure the email address is unique and you're done. That easy.
Yes, it can be done. At least I think this should work; I wound up replacing the whole auth model, so am ready to be corrected if this doesn't work out...
If you have no user records you care about:
drop the auth_user table
change username to max_length=75 in the model
syncdb
If you have user records you need to retain then it's more complicated as you need to migrate them somehow. Easiest is backup and restore of the data from old to new table, something like:
backup the user table data
drop the table
syncdb
reimport user data to the new table; taking care to restore the original id values
Alternatively, using your mad python-django skillz, copy the user model instances from old to new and replace:
create your custom model and temporarily stand it alongside the default model
write a script which copies the instances from the default model to the new model
replace the default model with your custom one
The latter is not as hard as it sounds, but obviously involves a bit more work.
Fundamentally, the problem is that some people want to use an email address as the unique identifier, while the user authentication system in Django requires a unique username of at most 30 characters. Perhaps that will change in the future, but that's the case with Django 1.3 as I'm writing.
We know that 30 characters is too short for many email addresses; even 75 characters is not enough to represent some email addresses, as explained in What is the optimal length for an email address in a database?.
I like simple solutions, so I recommend hashing the email address into a username that fits the restrictions for usernames in Django. According to User authentication in Django, a username must be at most 30 characters, consisting of alphanumeric characters and _, #, +, . and -. Thus, if we use base-64 encoding with careful substitution of the special characters, we have up to 180 bits. So we can use a 160-bit hash function like SHA-1 as follows:
import hashlib
import base64
def hash_user(email_address):
"""Create a username from an email address"""
hash = hashlib.sha1(email_address).digest()
return base64.b64encode(hash, '_.').replace('=', '')
In short, this function associates a username for any email address. I'm aware that there is a tiny probability of a collision in the hash function, but this should not be an issue in most applications.
Just adding the below code at the bottom of settings.py
from django.contrib.auth.models import User
User._meta.get_field("username").max_length = 75
C:...\venv\Lib\site-packages\django\contrib\auth\models.py
first_name = models.CharField(_('first name'), max_length=30, blank=True)
change to
first_name = models.CharField(_('first name'), max_length=75, blank=True)
save
and change in the database
I am using django 1.4.3 which makes it pretty easy and I did not have to change anything else in my code after realising I wanted to use long email addresses as usernames.
If you have direct access to the database, change it there to the amount of characters you would like to, in my case 100 characters.
In your app model (myapp/models.py) add the following
from django.contrib.auth.models import User
class UserProfile(models.Model):
# This field is required.
User._meta.get_field("username").max_length = 100
user = models.OneToOneField(User)
Then in your settings.py you specify the model:
AUTH_USER_MODEL = 'myapp.UserProfile'
If you are using venv (virtual environment), the simplest solution probably is just update the core code directly, i.e. opening the following two files:
- - venv/lib/python2.7/sites-packages/django/contrib/auth/model.py
- venv/lib/python2.7/sites-packages/django/contrib/auth/forms.py
Search for all username field and change max_length from 30 to 100. It is safe since you are already using venv so it won't affect any other Django project.
The best solution is to use email field for email and the username for username.
In the input login form validation, find whether the data is username or the email and if email, query the email field.
This only requires monkey patching the contrib.auth.forms.login_form which is a few lines in the corresponding view.
And it is far better than trying to modify the models and the database tables.
Django now provides a BigIntegerField for use in django models (available in svn trunk and also 1.2 alpha-1 release).
I need my django.contrib.auth.user model to have a BigIntegerField as its auto-incrementing primary key, whereas it currently uses an auto-incrementing IntegerField as its primary key. Also, wherever contrib.auth.user is used as a ForeginKey, it would need to be BigIntegerField as well.
What is the best and safest way to go about achieving this?
While I'm not sure why you need a BigIntegerField on User (you must have a whole lotta users) its pretty easy to implement. First you'll need to get a database migration system like South. We'll use this to do a handful of migrations of your current data. If you don't have anything in your database then just ignore this part and skip to the end.
I would start by making a custom user class which inherits from the contrib.auth version like so:
from django.contrib.auth.models import User, UserManager
from django.db import models
class BigUser(User):
id = models.BigIntegerField(pk = True)
objects = UserManager()
#this lets you transperantly use any
#query methods that you could on User
Then use South's data-migration capability to make a copy of all of you User.objects.all() into your new BigUser model.
Then go through and ADD a foriegnkey in each model where its needed. DO NOT delete the original FK yet, otherwise you're links will be lost. After adding the new keys do another schema migration.
Then make another data migration which copies the FK's from the old User model to the new BigUser model. Migrate that data.
Then its safe to delete the old FK to the User model.
If you want to avoid changing the rest of your code to use the new field-name for the BigUser you can use the South rename-field utility (South can't auto-detect field renames so make sure to read the docs).
If you don't have any data in the database then you can simply implement the class above and drop it into your current models.
If you need help writing data-migrations you'll have to post a model or two.
Since you need something that's a "drop-in" replacement for User you'll need two more steps:
First we need to create a custom authentication back-end, this makes sure that any authentication requests go to your new model and that request.user returns BigUser and not User. Just cut and paste this snippet into a file called auth_backend.py in the same directory as settings.py:
from django.conf import settings
from django.contrib.auth.backends import ModelBackend
from django.core.exceptions import ImproperlyConfigured
from django.db.models import get_model
class CustomUserModelBackend(ModelBackend):
def authenticate(self, username=None, password=None):
try:
user = self.user_class.objects.get(username=username)
if user.check_password(password):
return user
except self.user_class.DoesNotExist:
return None
def get_user(self, user_id):
try:
return self.user_class.objects.get(pk=user_id)
except self.user_class.DoesNotExist:
return None
#property
def user_class(self):
if not hasattr(self, '_user_class'):
self._user_class = get_model(*settings.CUSTOM_USER_MODEL.split('.', 2))
if not self._user_class:
raise ImproperlyConfigured('Could not get custom user model')
return self._user_class
Then in your settings.py file you need to add this back-end and set the custom user model setting ... like so:
AUTHENTICATION_BACKENDS = (
'auth_backends.CustomUserModelBackend',
)
...
CUSTOM_USER_MODEL = 'your-app-name.BigUser'
This last section of code comes from another website describing subclassing the User model.
Now all you need to do to "drop-in" in the rest of your code is to replace all of the from django.contrib.auth.models import User with from your-app-name import BigUser as User. By doing this you wont have to update any references of User with BigUser
I am weighing the option of changing the code of django.contrib.auth.models.user to include an id field as BigIntegerField primary key.
Seems to me to be the best way to go.
(I am ready to migrate the data manually via sql)