Django South Postgres Datamigration Issue - django

I'm adding a new model, running a schemamigration. Then adding data to that table with a datamigration. I'm running this on a Postgres db. I've run it successfully on a SQLite db locally, so I'm guessing it's a db-specific issue. The error that comes up in the datamigration is:
Error in migration: app:0109_add_reservation_rates
DatabaseError: column "rate_currency" specified more than once
LINE 1: ...rate_currency", "rate", "reservation_id", "date", "rate_curr...
^
There is no better error that South gives me and I haven't found a way to produce the SQL that South is supposed to be running. I've checked the schemamigration, datamigration and there is no repeated field name...
Here are the actual migrations:
0108_etc...py
# -*- coding: utf-8 -*-
import datetime
from south.db import db
from south.v2 import SchemaMigration
from django.db import models
class Migration(SchemaMigration):
def forwards(self, orm):
# Adding model 'ReservationRate'
db.create_table(u'app_reservationrate', (
(u'id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
('created_on', self.gf('django.db.models.fields.DateTimeField')(auto_now_add=True, blank=True)),
('updated_on', self.gf('django.db.models.fields.DateTimeField')(auto_now=True, blank=True)),
('reservation', self.gf('django.db.models.fields.related.ForeignKey')(related_name='reservation_rates', to=orm['app.Reservation'])),
('room', self.gf('django.db.models.fields.related.ForeignKey')(related_name='reservation_rates', to=orm['app.Room'])),
('date', self.gf('django.db.models.fields.DateField')(db_index=True)),
('rate_currency', self.gf('djmoney.models.fields.CurrencyField')(default='USD')),
('rate', self.gf('djmoney.models.fields.MoneyField')(max_digits=10, decimal_places=2, default_currency='USD')),
))
db.send_create_signal(u'app', ['ReservationRate'])
# Adding unique constraint on 'ReservationRate', fields ['reservation', 'room', 'date']
db.create_unique(u'app_reservationrate', ['reservation_id', 'room_id', 'date'])
0109_etc...py
coding: utf-8 -*-
import datetime
from south.db import db
from south.v2 import DataMigration
from django.db import models
class Migration(DataMigration):
def forwards(self, orm):
for reservation in orm.Reservation.objects.all():
# Loop through some code, get rates, and dates for creating ReservationRate objects...
orm.ReservationRate.objects.create(reservation=reservation, room=stay.room, date=date, rate=rate)
Anybody have a clue how to fix this?

Not exactly sure why, but running ./manage.py migrate -v2 app made everything work correctly. I believe that option is supposed to run the migration and output more info on what it's running.

Related

Can't create Django superuser [duplicate]

I am trying to use Django's default Auth to handle register and log in.
setting.py:
INSTALLED_APPS = (
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'books',
)
MIDDLEWARE_CLASSES = (
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
)
AUTH_USER_MODEL = 'books.User'
books.models.py:
class User(AbstractUser):
account_balance = models.DecimalField(max_digits=5, decimal_places=2, default=0)
views.py:
from django.contrib.auth.forms import UserCreationForm
def register(request):
if request.method == 'POST':
form = UserCreationForm(request.POST)
if form.is_valid():
new_user = form.save()
return HttpResponseRedirect("/accounts/profile/")
else:
form = UserCreationForm()
return render(request, "registration/register.html", {'form': form,})
urls.py:
urlpatterns = patterns('',
(r'^accounts/login/$', login),
(r'^accounts/logout/$', logout),
(r'^accounts/profile/$', profile),
(r'^accounts/register/$', register),
)
I tried deleting the db.sqlite3 file and re-ran python manage.py syncdb but I still get this error message:
OperationalError at /accounts/register/
no such table: auth_user
Request Method: POST
Request URL: http://127.0.0.1:8000/accounts/register/
Django Version: 1.7b4
Exception Type: OperationalError
Exception Value:
no such table: auth_user
./manage.py migrate
If you've just enabled all the middlewares etc this will run each migration and add the missing tables.
Only thing you need to do is :
python manage.py migrate
and after that:
python manage.py createsuperuser
after that you can select username and password.
here is the sample output:
Username (leave blank to use 'hp'): admin
Email address: xyz#gmail.com
Password:
Password (again):
Superuser created successfully.
Update
You are probably getting this error because you are using UserCreationForm modelform, in which in META it contains User(django.contrib.auth.models > User) as model.
class Meta:
model = User
fields = ("username",)
And here you are using your own custom auth model, so tables related to User has not been created. So here you have to use your own custom modelform. where in Meta class, model should be your User(books.User) model
This will work for django version <1.7:
Initialize the tables with the command
manage.py syncdb
This allows you to nominate a "super user" as well as initializing any tables.
it is need to make migration before create superuser.
python manage.py makemigrations
python manage.py migrate
python manage.py createsuperuser
Username : admin
Password : 12345678
python manage.py runserver
Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
try running
python manage.py migrate
then run
python manage.py createsuperuser
For custom forms( if you have made your own forms) use this command to migrate
python manage.py migrate --run-syncdb
If using a custom auth model, in your UserCreationForm subclass, you'll have to override both the metaclass and clean_username method as it references a hardcoded User class (the latter just until django 1.8).
class Meta(UserCreationForm.Meta):
model = get_user_model()
def clean_username(self):
username = self.cleaned_data['username']
try:
self.Meta.model.objects.get(username=username)
except self.Meta.model.DoesNotExist:
return username
raise forms.ValidationError(
self.error_messages['duplicate_username'],
code='duplicate_username',
)
Before creating a custom user model, a first migration must be performed. Then install the application of your user model and add the AUTH_USER_MODEL.
As well:
class UserForm(UserCreationForm):
class Meta:
model = User
fields = ("username",)
and
python manage.py migrate auth
python manage.py migrate
On Django 1.11 I had to do this after following instructions in docs https://docs.djangoproject.com/en/1.11/topics/auth/customizing/#substituting-a-custom-user-model
# create default database:
./manage.py migrate
# create my custom model migration:
# running `./manage.py makemigrations` was not enough
./manage.py makemigrations books
# specify one-off defaults
# create table with users:
./manage.py migrate
Just do the following flow
$ django-admin createproject <your project name>
under <your project dict> type django-admin createapp <app name>
under <app name>/admin.py
from django.contrib import admin
from .models import Post
admin.site.register(Post)
Go to the root project. Then $python manage.py migrate
Then it asks for username and password
Just perform migrations before registering the user.
theres four steps for adding a custom user model to django
Create a CustomUser model
update project/settings.py AUTH_USER_MODEL
customize UserCreationForm & UserChangeForm
add the custom user model to admin.py
you missed customize forms , add the CustomUser and CustomUserAdmin to admin.site.register() , then makemigrations nd migrate .
#proj_app/forms.py
from django.contrib.auth import get_user_model
from django.contrib.auth.forms import UserCreationForm, UserChangeForm
class CustomUserCreationForm(UserCreationForm):
class Meta:
model = get_user_model()
fields = ('email','username',)
class CustomUserChangeForm(UserChangeForm):
class Meta:
model = get_user_model()
fields = ('email', 'username',)
#proj_app/admin.py
from django.contrib import admin
from django.contrib.auth import get_user_model
from django.contrib.auth.admin import UserAdmin
from .forms import CustomUserCreationForm , CustomUserChangeForm
CustomUser = get_user_model()
class CustomUserAdmin(UserAdmin):
add_form = CustomUserCreationForm
form = CustomUserChangeForm
model = CustomUser
list_display = ['email','username',]
admin.site.register(CustomUser, CustomUserAdmin)
here we extend the existing UserAdmin into CustomUserAdmin and tell django to use our new forms, custom user model, and list only the email and username of a user also we could add more of existing User fields to list_display
I have no idea what I did wrong but got to the point where I decided to clear the whole database. So I ran the command:
python manage.py flush
After that my database was clear then I ran the commands;
python manage.py makemigrations
python manage.py migrate
then:
python manage.py createsuperuser
That worked for me.
I have also faced the same problem "no such table: auth_user" when I was trying to deploy one of my Django website in a virtual environment.
Here is my solution which worked in my case:
In your settings.py file where you defined your database setting like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(os.getcwd(), 'db.sqlite3'),
}
}
just locate your db.sqlite3 database or any other database that you are using and write down a full path of your database , so the database setting will now look something like this ;
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': '/home/django/django_project/db.sqlite3',
}
}
I hope that your problem will resolve now.
python manage.py makemigrations then → python manage.py migrate fixes it.
Assuming Apps defined/installed in settings.py exist in the project directory.
Please check how many python instances are running in background like in windows go--->task manager and check python instances and kill or end task i.e kill all python instances. run again using "py manage.py runserver" command.
i hope it will be work fine....
If You did any changes in project/app then execute:
python manage.py migrate
python manage.py makemigrations
python manage.py createsuperuser
call these command
python manage.py makemigrations
python manage.py migrate

How to generate a function-based index in my Python model migration?

I'm using Django, Python 3.7 and PostGres 9.5. To speed along a particular query, I want to create this functional index, which I'd normally do in PostGres like so ...
CREATE INDEX my_article_idx ON article (regexp_replace(url, '\?.*$', ''))
However in the world of Django and auto-generated migrations, I'm not sure how to annotate my class in my models.py file so that this function-based index would be auto-generated. The field in question in my model looks like this ...
class Article(models.Model):
...
url = models.TextField(default='', null=False)
You have to create a data migration. Read more about them in docs
Step 1 - create an empty data migration file
python manage.py makemigrations --empty yourappname
Step 2 - Add custom sql to the migration:
# -*- coding: utf-8 -*-
# Generated by Django 1.11.14 on 2018-09-20 08:01
from __future__ import unicode_literals
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('yourapp', '0001_name_of_depending_migration'),
]
operations = [
migrations.RunSQL(
sql="CREATE INDEX my_article_idx ON article (regexp_replace(url, '\?.*$', ''))",
reverse_sql='DROP INDEX my_article_idx ON article'
)
]

right way to create a django data migration that creates a group?

I would like to create data migrations that create Permissions and Groups, so that my other developers can just run the migrations and get everything set up. I was able to create the migrations and run them just fine, but now I'm getting an error when running my tests.
But if I do this:
from django.contrib.auth.models import Group
def add_operations_group(apps, schema_editor):
Group.objects.get_or_create(name='operations')
I get:
django.db.utils.OperationalError: no such table: auth_group
If I do this:
def add_operations_group(apps, schema_editor):
Group = apps.get_model("django.contrib.auth", "group")
Group.objects.get_or_create(name='operations')
I get:
LookupError: No installed app with label 'django.contrib.auth'
Is there a way to do this? Or is there a "Django Way" to make sure things like permissions and groups are created?
This is how I do it:
from django.db import models, migrations
def apply_migration(apps, schema_editor):
Group = apps.get_model('auth', 'Group')
Group.objects.bulk_create([
Group(name=u'group1'),
Group(name=u'group2'),
Group(name=u'group3'),
])
def revert_migration(apps, schema_editor):
Group = apps.get_model('auth', 'Group')
Group.objects.filter(
name__in=[
u'group1',
u'group2',
u'group3',
]
).delete()
class Migration(migrations.Migration):
dependencies = [
('someapp', 'XXXX_some_migration'),
]
operations = [
migrations.RunPython(apply_migration, revert_migration)
]
Although, there must be a more Djangonic way.
Answer from César is correct. To make it more Django create the migration file automatically by going to your django app root folder and entering:
python manage.py makemigrations <yourappname> --empty
Note: You may need python3 instead of python depending on your system configuration.
This creates an empty migration file in a sub directory of your app called 0001_initial.py
You can then alter it as per César instructions. Which worked correctly with Django 2.2

django: data migrate permissions

I have a bunch of new permissions which I need to migrate. I tried doing it through data migration but complains about ContentType not being available.
Doing quick research I found out that ContentType table is populated after all the migrations applied.
I even tried using update_all_contenttypes() from from django.contrib.contenttypes.management import update_all_contenttypes
which causes migration to load data which is not consistent to the fixture.
What is the best way to migrate permission data in Django?
Here is a quick and dirty way to ensure all permissions for all apps have been created:
def add_all_permissions(apps=None, schema_editor=None):
from django.contrib.auth.management import create_permissions
if apps is None:
from django.apps import apps
for app_config in apps.get_app_configs():
app_config.models_module = True
create_permissions(app_config, verbosity=0)
app_config.models_module = None
class Migration(migrations.Migration):
dependencies = [('myapp', '0123_do_the_thing')]
operations = [
migrations.RunPython(add_all_permissions,
reverse_code=migrations.RunPython.noop)
# ...
]
NOTE: edited to include ruohola's excellent suggestion
There are 2 ways to solve this:
1) The ugly way:
Run manage.py migrate auth before your wanted migration
2) Recommended way:
from django.contrib.auth.management import create_permissions
def add_permissions(apps, schema_editor):
apps.models_module = True
create_permissions(apps, verbosity=0)
apps.models_module = None
# rest of code here....
Here are steps for adding custom permissions to the User model:
First create a migration file, for example under your authentication application,
Here i named it 0002_permission_fixtures.py:
account (your authentication application)
|_migrations
|__ 0001_initial.py
|__ 0002_permission_fixtures.py
|__ __init__.py
Then adding your permission objects, as follow:
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations
def forwards_func(apps, schema_editor):
# Get models that we needs them
user = apps.get_model("auth", "User")
permission = apps.get_model("auth", "Permission")
content_type = apps.get_model("contenttypes", "ContentType")
# Get user content type object
uct = content_type.objects.get_for_model(user)
db_alias = schema_editor.connection.alias
# Adding your custom permissions to User model:
permission.objects.using(db_alias).bulk_create([
permission(codename='add_sample', name='Can add sample', content_type=uct),
permission(codename='change_sample', name='Can change sample', content_type=uct),
permission(codename='delete_sample', name='Can delete sample', content_type=uct),
])
class Migration(migrations.Migration):
dependencies = [
('contenttypes', '__latest__'),
]
operations = [
migrations.RunPython(
forwards_func,
),
]
To run this migration, first migrate contenttype model, and then migrate your application (here is account).
$ python manage.py migrate contenttypes
$ python manage.py migrate account

Django OperationalError: missing table; migration does not recognize missing table

I'm having trouble in Django 1.7, I am trying to save a user to a table, but I'm getting an error that the table does not exist.
Here is the code I'm executing:
from django.conf import settings
from django.contrib.auth import BACKEND_SESSION_KEY, SESSION_KEY, get_user_model
User = get_user_model()
from django.contrib.sessions.backends.db import SessionStore
from django.core.management.base import BaseCommand
class Command(BaseCommand):
def handle(self, email, *_, **__):
session_key = create_pre_authenticated_session(email)
self.stdout.write(session_key)
def create_pre_authenticated_session(email):
user = User.objects.create(email=email)
session = SessionStore()
session[SESSION_KEY] = user.pk
session[BACKEND_SESSION_KEY] = settings.AUTHENTICATION_BACKENDS[0]
session.save()
return session.session_key
However, at
user = User.objects.create(email=email)
I get an Error message :
django.db.utils.OperationalError: no such table: accounts_user
Here is the user model at accounts/models.py that I'm trying to use to build the table:
from django.db import models
from django.utils import timezone
class User(models.Model):
email = models.EmailField(primary_key=True)
last_login = models.DateTimeField(default=timezone.now)
REQUIRED_FIELDS = ()
USERNAME_FIELD = 'email'
def is_authenticated(self):
return True
I've run sqlmigrate against this migration with 'manage.py accounts 0001.initial' and I have gotten the correct create table SQL back, but running 'manage.py migrate' gives me the following :
Operations to perform:
Apply all migrations: sessions, admin, lists, contenttypes, accounts, auth
Running migrations:
No migrations to apply.
The migration is just the result of running 'makemigration' from the shell, no custom code. I do see accounts listed in the included applications, but the migration isn't being ran, so my site is in an odd spot where Django says the table is missing when I try to use it, but Django says it exists when I try to run the migration to create it. Why does Django erroneously think that the table already exists when I can look at the database and see that it doesn't?
#user856358 Your comment about the other sqlite file seems like the root cause. I encountered the same error, and it was resolved by removing that file and running another migration. In my case, the file was located as specified in settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, '../database/db.sqlite3'),
}
}
By removing the .sqlite3 file there, I was able to successfully run the migration and resolve the no-such-table error...
django.db.utils.OperationalError: no such table: accounts_user
$ rm ../database/db.sqlite3
$ python3 manage.py migrate