I've set up a new Django project (based on the template django cookie cutter) but get a postgres error when trying to migrate the database for the first time
psycopg2.ProgrammingError: relation "django_site_domain_a2e37b91_uniq" already exists
I'm not clear what the issue is or how best to diagnose?
What is SITE_ID?
Django was created from a set of scripts developed at a newspaper to
publish content on multiple domains; using one single content base.
This is where the "sites" module comes in. Its purpose is to mark
content to be displayed for different domains.
In previous versions of django, the startproject script automatically
added the django.contrib.sites application to INSTALLED_APPS, and when
you did syncdb, a default site with the URL example.com was added to
your database, and since this was the first site, its ID was 1 and
that's where the setting comes from.
Keep in mind that starting from 1.6, this framework is not enabled by
default. So if you need it, you must enable it
The SITE_ID setting sets the default site for your project. So, if you
don't specify a site, this is the one it will use.
Ways to fix this:
Increment SITE_ID:
1. Increment `SITE_ID` variable in settings.py
2. python manage.py makemigrations
3. python manage.py migrate --run-syncdb
Cons: Had to increment SITE_ID without good reason
Attempt to migrate without --run-syncdb:
1. python manage.py makemigrations
2. python manage.py migrate
Note: May have to try multiple times before it works. Unsure why, possibly because I was in the process of deleting the pvc
Related
On our staging server we observed a runtime error stating a field is missing from a database,
column our_table.our_field does not exist
LINE 1: ...d"."type", "our_table"...
The field was added during a recent update with a complicated migration squashing process. It's possible that some errors were made during this process, but "manage.py showmigrations" command shows that the migration has been applied and "manage.py makemigrations" does not create any new migration. As we do not run tests on our staging or production databases, we are trying to figure out the most effective method for identifying such errors.
In short, how can we identify mismatches between the database and Django models caused by an incorrect migration like the following?
python manage.py migrate our_app --fake
I suppose I am looking for something like
python manage.py check_database
Edit: Many thanks for the suggestions. However, this is more of a deployment than a development question because the problem likely occurred when our devops tried to apply the squashed migrations while retaining data on the staging server (which will be case on production). It was scary to learn that such inconsistency can occur when makemigrations and showmigrations do not show any problem and can therefore also happen on production.
The bottom line is that we need some way to ensure our database matches our models after deployment.
I want to remove all data from all models defined in a single Django app. The number of models might change in the app, hence I don't want to hard code model names. It would be great if it might be done using manage.py command, however I was not able to find one.
The manage.py flush command is not suitable, because it only allows to clear data from all apps.
If you are using django version greater than 1.7, which you are. You can simply use migrate zero command to drop from specific app. Like:
py manage.py migrate APPNAME zero
Here, APPNAME is name of the app from where you want to flush data.
Refs
I have updated a Django blog to use slug urls <slug:slug>/ instead of id <int:pk>/. I also added a slugField to the Article model, and converted all hrefs to article.slug. Ran migrations and everything worked fine locally. However, when I pushed to Heroku I got an error.
This is what happens when I attempt to save a new article.
ProgrammingError at /admin/articles/article/
column articles_article.slug does not exist
LINE 1: ...ticles_article"."id", "articles_article"."title", "articles_...
^
Request Method: GET
Request URL: https://***********.herokuapp.com/admin/articles/article/
Django Version: 2.1.4
Exception Type: ProgrammingError
Exception Value:
column articles_article.slug does not exist
LINE 1: ...ticles_article"."id", "articles_article"."title", "articles_...
^
I checked my Heroku Postgress database and I found that the new slug column hadn't been added even though I did migrations. I'm not exactly sure what to do next. I'm currently searching for ways to manually update the heroku postgress, but if there's a less invasive way to solve this problem I'm all ears.
first verify that the migration was executed (heroku)
verify that the last migration of the article model is in the django_migrations table
In case you can't find it, make sure you have updated migrations (local)
python manage.py makemigrations
run the migration in db production (heroku)
heroku run python manage.py migrate
Did you happen to make migrations first then migrate? It's noted backward in your comment above. Also, are you sure you applied the migrations to your production database? It requires a settings flag added to the migrate command to point to your production settings/db location.
I have a django 1.8 app working with a db.
I'm trying to change the schema of a table using the built-in migration.
Here are the steps I did:
In my dev invironment, I grabbed the app source and ran
python manage.py sycdb
then I ran
python manage.py loaddata ~/my_data.json
then I modified modes.py. Added a field and renamed a field...all from the same table 'TABLE1' which had no data.
then
python manage.py makemigrations myapp
python manage.py migrate
Error: django.db.utils.OperationalError: table "myapp_someother_table" already exists
then ran
python manage.py migrate --fake-initial
worked!
but when I browsed to the admin page for TABLE1, I get this error:
OperationalError: no such column: myapp_table1.my_new_field_id
I checked the db and yes, there is no such column.
How can I procceed from here? I prefer to fix this via django.
If I fix it straight in the db, then the migration goes out of sync.
Migrations do not automagically see that you have made changes. Migrations detect changes by comparing the current model with the historical model saved in the migration files.
In this case, you didn't have any historical models, since you didn't have any migrations. Django was not able to detect any changes in your models, even though they were different from your database.
The correct way to make changes to your model is to first run manage.py makemigration <my_app>, and then make the changes to your model, followed by another manage.py makemigrations.
You might not be able to do it via pure django and keep your data. I don't have personal experience with south but there are a lot of mentions if this tool. Just in case if nothing else works for you...
Here is what I did to make things work, but there must be a better way so please add more answers/comments...
I deleted the sqlite db and the migration folder
I made the desired changes to model.py
ran syncdb
ran loaddata to load the json data dump that I had saved previously.
just started the dev server
What is the procedure for completely uninstalling a Django app, complete with database removal?
Django < 1.7 has a handy management command that will give you the necessary SQL to drop all the tables for an app. See the sqlclear docs for more information. Basically, running ./manage.py sqlclear my_app_name gets you get the SQL statements that should be executed to get rid of all traces of the app in your DB. You still need to copy and paste (or pipe) those statements into your SQL client. For Django 1.7 and up, use ./manage.py migrate my_app_name zero (see the migrate docs), which runs the database cleaning automatically.
To remove the app from your project, all you need to do is remove it from INSTALLED_APPS in your project's settings.py. Django will no longer load the app.
If you no longer want the app's files hanging around, delete the app directory from your project directory or other location on your PYTHONPATH where it resides.
(optional) If the app stored media files, cache files, or other temporary files somewhere, you may want to delete those as well. Also be wary of lingering session data that might be leftover from the app.
(optional) I would also remove any stale content types.
Like so.
from django.contrib.contenttypes.models import ContentType
for c in ContentType.objects.all():
if not c.model_class():
print "deleting %s"%c # print(f"deleting {c}") # for Python 3.6+
c.delete()
In my context the projects exists several times: I have a development system, some team mates have a development system, there is a staging system for the customer and a production system. This means I don't want to execute sql commands by hand. I want it to be automated.
Goal: Remove the app and all database tables.
Step 1: empty the app, but leave it installed
remove all files from the app, except the folder "migrations"
Execute this command:
python manage.py makemigrations -n drop_all_tables my_app_to_remove
The directory looks now like this:
my_app_to_remove/
my_app_to_remove/__init__.py
my_app_to_remove/migrations
my_app_to_remove/migrations/0001_initial.py
my_app_to_remove/migrations/....
my_app_to_remove/migrations/0030_drop_all_tables.py
my_app_to_remove/migrations/__init__.py
Leave my_app_to_remove in the file "settings.py".
Step 2: Deploy the changes
Update all projects. Tell the team mates to update their project and to run the migrations.
Step 3: remove "my_app_to_remove" from settings.py
Now remove "my_app_to_remove" from settings.py and deploy again.
comment out on settings.py in INSTALLED_APPS unnecessary app's line
delete all folder __pycache__ and migrate at your project
delete unnecessary model in models.py
delete all import link in views.py, admin.py end etc.
delete all link's in urls.py on your unnecessary app's
in database delete unnecessary tables wich associated with the app (I do it with help program "Valentina Studio")
delete app's folder
in command line do it: python manage.py migrate and python manage.py syncdb
django app is a "set" of *.py files and a directory with a django-app-name. So you can simply delete the whole folder with all *.py files
To "remove" tables from DB you should use DELETE FROM <app-name_table-names>
Furthermore, you have to delete lines witgh app-name from setting.py in a root directory
As a first step, prevent any usage of the app models and deploy. This is essential for rolling deployment. Check notes below.
Then you have two options.
Option 1
(Manual) Run python manage.py migrate <app_name> zero. This will revert all the migrations for the app and clean the django_migrations table
Remove app (code, INSTALLED_APPS, urls.py, etc.)
Deploy (python manage.py migrate)
Option 2
Remove all models in the app
Deploy
Remove the app (code, INSTALLED_APPS, urls.py, etc.)
Deploy
(Manual) Clean django_migrations table
This answer is thinking in an automated rolling deployment context (e.g. Heroku).
Manual means it normally cannot be done in automated deployment. Deploy basically refers to python manage.py migrate which is normally done in automated deployment.
Notes
Make sure to remove code importing this app from other apps. Also any references in settings.py (but keep it in INSTALLED_APPS so we can run migrations), urls.py, etc.
migrate <app_name> zero will revert all migrations depending on those migrations. So be careful if your other apps migrations depend on these migrations.
Also be aware of cascade delete if you have not just schema migrations but also data migrations.
Signal receivers receiving signals defined in other apps might be an issue in rolling deployment.
References
https://docs.djangoproject.com/en/stable/ref/django-admin/#migrate
Safely Remove a Django app by Jordan Haines
To completely remove a Django app (with models), follow the steps below. Let’s pretend we’re removing an app called note_app:
Search through your other apps for any note_app imports. The easiest way to do this is a project-wide search for “from note_app”. You may want to exclude the note_app directory from your search.
Comment out all models in note_app.models. You may need to also remove entries from note_app.admin file if you registered your note_app models with Django admin.
Look for and resolve errors when trying to run your Django app. P.S. You may have missed some model imports in step 1.
Note that depending on how you define ForeignKey, OneToOne and ManyToMany fields, you may have missed some keys to your note_app models in step 1. It’s important that any fields that point to note_app models from models in other apps need to be deleted before you continue. Take a minute to make sure none of these fields were missed; if they were, then delete those fields that remain and create migrations that will remove the fields from your database.
Run makemigrations. This should create a note_app migration that deletes all of the models you just commended out. This migration should also remove fields referring to note_app models from models in other apps.
Run your migrations. You must run your migrations in all environments (including production) BEFORE you delete the app directory. The migrations that remove your app’s models from the database will be — surprise — in your note_app directory. If you delete the app directory prematurely, you will delete these migrations before they have a chance to clean up your database.
You may get a notice that the content types for your deleted models are stale. When asked whether or not you want to delete these content types, reply “yes”
Git Tip: Commit your migrations, take note of the commit, and then create a separate commit that deletes the note_app directory. When you are ready to apply your changes in a staging or production environment, checkout the commit you noted, run the migration, and then checkout the latest commit to delete the app directory. Your first commit should still have note_app in INSTALLED_APPS.
Delete the directory that contains the note_app.
Remove note_app from your INSTALLED_APPS setting.
And that’s really about it…super easy :)