My project is in early development. I frequently delete the database and run manage.py syncdb to set up my app from scratch.
Unfortunately, this always pops up:
You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no):
Then you have supply a username, valid email adress and password. This is tedious. I'm getting tired of typing test\nx#x.com\ntest\ntest\n.
How can I automatically skip this step and create a user programatically when running manage.py syncdb ?
I know the question has been answered already but ...
A Much simpler approach is to dump the auth module data into a json file once the superuser has been created:
./manage.py dumpdata --indent=2 auth > initial_data.json
You can also dump the sessions data:
./manage.py dumpdata --indent=2 sessions
You can then append the session info to the auth module dump (and probably increase the expire_date so it does not expire... ever ;-).
From then, you can use
/manage.py syncdb --noinput
to load the superuser and his session when creating the db with no interactive prompt asking you about a superuser.
Instead of deleting your entire database, just delete the tables of your app before running the syncdb
This will accomplish it for you in a single line (per app):
python manage.py sqlclear appname | python manage.py dbshell
The first command will look at your app and generate the required SQL to drop the tables. This output is then piped to the dbshell to execute it.
After its done, run your syncdb to recreate the tables:
python manage.py syncdb
The key is to use --noinput at the time of syncdb & then use this one liner to create superuser
echo "from django.contrib.auth.models import User; User.objects.create_superuser('myadmin', 'myemail#example.com', 'hunter2')" | python manage.py shell
Credit : http://source.mihelac.org/2009/10/23/django-avoiding-typing-password-for-superuser/
If you want the ability — as I do — to really start with a fresh database without getting asked that superuser question, then you can just de-register the signal handler that asks that question. Check out the very bottom of the file:
django/contrib/auth/management/__init__.py
to see how the registration of the superuser function gets performed. I found that I could reverse this registration, and never get asked the question during "syncdb", if I placed this code in my "models.py":
from django.db.models import signals
from django.contrib.auth.management import create_superuser
from django.contrib.auth import models as auth_app
# Prevent interactive question about wanting a superuser created. (This
# code has to go in this otherwise empty "models" module so that it gets
# processed by the "syncdb" command during database creation.)
signals.post_syncdb.disconnect(
create_superuser,
sender=auth_app,
dispatch_uid = "django.contrib.auth.management.create_superuser")
I am not sure how to guarantee that this code gets run after the Django code that does the registration. I had thought that it would depend on whether your app or the django.contrib.auth app gets mentioned first in INSTALLED_APPS, but it seems to work for me regardless of the order I put them in. Maybe they are done alphabetically and I'm lucky that my app's name starts with a letter later than "d"? Or is Django just smart enough to do its own stuff first, then mine in case I want to muck with their settings? Let me know if you find out. :-)
I've overcome this feature using south
Its a must have for any django developer.
South is a tool designed to help migrate changes over to the live site without destroying information or database structure. The resulting changes can be tracked by south and using the generated python files - can perform the same actions on an alternative database.
During development, I use this tool to git track my database changes - and to make a change to the database without the need to destroy it first.
easy_install south
Add 'south' to your installed apps
Proposing first time run of south on an app.
$ python manage.py schemamigration appname --init
This will initiate schema detection on that app.
$ python manage.py migrate appname
This will apply the model changes
The database will have the new models.
Changing a model after the first run
$ python manage.py schemamigration appname --auto
$ python manage.py migrate appname
Models will have changed - data is not destroyed.
Plus south does much more...
Note: since version 1.7 syncdb command is deprecated. Use migrate instead.
Also Django 1.7 introduced AppConfig as means of customizing applications' initialization process.
Thus since Django 1.7 the simplest way to achieve what you want is to employ an AppConfig's subclass.
Let say, you happen to have your own example_app that is added to your INSTALLED_APPS and you want to create and admin user with admin password whenever you run ./manage.py migrate from scratch. I also assume that automatic admin user creation is required only in dev environment - not in production.
Add the following code to example_app/apps.py
# example_app/apps.py
from django.apps import AppConfig
from django.conf import settings
from django.db.models.signals import post_migrate
from django.contrib.auth.apps import AuthConfig
USERNAME = "admin"
PASSWORD = "admin"
def create_test_user(sender, **kwargs):
if not settings.DEBUG:
return
if not isinstance(sender, AuthConfig):
return
from django.contrib.auth.models import User
manager = User.objects
try:
manager.get(username=USERNAME)
except User.DoesNotExist:
manager.create_superuser(USERNAME, 'x#x.com', PASSWORD)
class ExampleAppConfig(AppConfig):
name = __package__
def ready(self):
post_migrate.connect(create_test_user)
Also add the following reference to the app configuration inside apps example_app/__init__.py:
# example_app/__init__.py
default_app_config = 'example_app.apps.ExampleAppConfig'
Where the default_app_config is a string Python path to the AppConfig subclass as mentioned here.
The manage.py reset command will reset your database without destroying your created super user. Data does however need to be re-imported.
You can use django-finalware to do this for you. Just add finalware to your INSTALLED_APPS and include the following in your settings.py:
SITE_SUPERUSER_USERNAME = 'myadmin'
SITE_SUPERUSER_EMAIL = 'myadmin#example.com'
SITE_SUPERUSER_PASSWORD = 'mypass' # this can be set from a secret file.
# optional object id. Ensures that the superuser id is not set to `1`.
# you can use this as a simple security feature
SITE_SUPERUSER_ID = '343'
Then just run ./manage.py syncdb (Django <1.7) or ./manage.py migrate (Django >= 1.7), and it will automatically create a superuser or update the existing one for you.
You are never prompted to created a superuser anymore.
Since Django 1.7 the suggested way of populating the database is through data migrations. To create a data migration for creating the admin you should first create an empty migration:
./manage.py makemigrations --empty myapp --name create-superuser
This will create an empty migration in myapp/migrations/000x__create-superuser.py. Edit the file to make it look like this:
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations, models
from django.contrib.auth.models import User
def create_superuser(apps, schema_editor):
User.objects.create_superuser(username='myadmin', password='mypassword', email='myemail#gmail.com')
class Migration(migrations.Migration):
dependencies = [('myapp', '000y_my-previous-migration-file'),]
operations = [migrations.RunPython(create_superuser)]
I have resolved creating a python script like this one to reset all my stuff [updated version][1.8 too]:
import os
import sys
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "main.settings.dev")
from django.conf import settings
from django.core import management
from django import get_version
PROJECT_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir))
if PROJECT_ROOT not in sys.path:
sys.path.append(PROJECT_ROOT)
yn = raw_input('Are you sure you want to reset everything? (y/n) ')
if yn == 'y':
# Drops the db / creates the db
if settings.DATABASES['default']['ENGINE'].find('mysql') != -1:
os.system('mysqladmin -uroot -pIronlord0 -f drop db')
os.system('mysqladmin -uroot -pIronlord0 -f create db')
elif settings.DATABASES['default']['ENGINE'].find('psycopg2') != -1:
os.system('psql -U postgres -c "DROP DATABASE db"')
os.system('psql -U postgres -c "CREATE DATABASE db WITH OWNER = admin"')
elif settings.DATABASES['default']['ENGINE'].find('sqlite3') != -1:
try:
os.remove(os.path.join(PROJECT_ROOT, 'data.db'))
except:
pass
# Getting application handle here otherwise db gets allocated and it can not be destroyed.
if get_version() > '1.6.10':
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
management.call_command('syncdb', interactive=False)
# Creates admin/password
from django.contrib.auth.management.commands import changepassword
management.call_command('createsuperuser', interactive=False, username="admin", email="xxx#example.com")
command = changepassword.Command()
command._get_pass = lambda *args: 'password'
if get_version() >= '1.8':
command.execute(username="admin")
else:
command.execute("admin")
# Creates the default site entry
from django.contrib.sites.models import Site
site = Site.objects.get_current()
site.domain = 'www.example.com'
site.name = ' xxx '
site.save()
it works like a charm!
P.S.: Be sure to stop your (testing) server where above db is in charge before running this script!
Take a look at the dumpdata management command. For instance:
python manage.py dumpdata > initial_data.json
If this file, called a fixture, is named initial_data (.xml or .json), then the syncdb command will pick it up and populate your tables accordingly. It will still ask you if you want to create a user, but I believe you may safely answer "no", after which point it will populate the database based on your fixture.
More info on this can be found in the docs.
Developing with sqlite.
Clear database by deleting file.
Load admin from fixtures.
change manage.py (django 1.4):
# hack to prevent admin promt
if len(sys.argv) == 2 and sys.argv[1] == 'syncdb':
sys.argv.append('--noinput')
My solution to this was to just not delete that auth tables when wiping out my database.
If you prefer to type initializing code direct into python source file, this code modified manage.py might help (and thanks for Cjkjvfnby's little code!):
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
# set your django setting module here
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "app.settings")
from django.core.management import execute_from_command_line
# hack to prevent admin prompt
if len(sys.argv) == 2 and sys.argv[1] == 'syncdb':
sys.argv.append('--noinput')
execute_from_command_line(sys.argv)
# additional process for creation additional user, misc data, and anything
for arg in sys.argv:
# if syncdb occurs and users don't exist, create them
if arg.lower() == 'syncdb':
print 'syncdb post process...'
from django.contrib.auth.models import User
admin_id = 'admin'
admin_email = 'superuser#mail.com'
admin_password = 'superuser_password'
additional_users = [
['tempuser', 'user_email#mail.com', 'tempuser_password']
]
# admin exists?
user_list = User.objects.filter(username=admin_id)
if len(user_list) == 0:
print 'create superuser: ' + admin_id
new_admin = User.objects.create_superuser(admin_id, admin_email, admin_password)
# additional user exists?
for additional_user in additional_users:
user_list = User.objects.filter(username=additional_user[0])
if len(user_list) == 0:
print 'create additional user: ' + additional_user[0]
new_admin = User.objects.create_user(additional_user[0], additional_user[1], additional_user[2])
# any other data
I'm just showing the user creation code here, but you can enhance this code more as you want.
I'm using sqlite as a dev database. After changing model classes, just drop the corresponding tables with sqlite manager (a firefox plugin, open to inspect the data anyways) and run manage.py syncdb to recreate what's missing.
Related
Currently, I run
$ flask db init
$ flask db migrate -m "initialization"
$ flask db upgrade
if the database does not exist. I would like to run this within Python, e.g. something like
app.create_db()
so that I don't have to care about setting the database up. Is that possible?
I use the flask-sqlalchemy and flask-migrations plugins
You can use SQLAlchemy-Utils for this.
from sqlalchemy import create_engine
from sqlalchemy_utils import database_exits,create_database
def validate_database():
engine = create_engine('postgres://postgres#localhost/name')
if not database_exists(engine.url): # Checks for the first time
create_database(engine.url) # Create new DB
print("New Database Created"+database_exists(engine.url)) # Verifies if database is there or not.
else:
print("Database Already Exists")
call this method in your __init__.py file so that it checks every time your server starts.
Obviously, you have installed flask-migrate, flask-sqlalchemy.
So, you can do like this:
from flask_sqlalchemy import SQLAlchemy
from flask import Flask
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
db.create_all()
API DOC: flask.ext.sqlalchemy.SQLAlchemy.create_all
but your Question has confused me. why restricted by SQLAlchemy and Alembic?
There's db.create_all() but I think that when you're using migrations you should stick to migration scripts.
Something to note is that if you have your migration files all set up (i.e migration folder) then all you need is flask db migrate
If you're running this locally, I would stick to doing this command manually.
If you're using this on a server, you should probably use a deployment script that does this for you. You can look at fabric (www.fabfile.org) for information on how to run terminal commands
I've got some Postgres stored procedures that my selenium tests will depend on. In development, I load them with a line in a script:
cat stored_procedures.sql | python manage.py dbshell
This doesn't work when unit testing, since a fresh database is created from scratch. How can I load stored procedures saved in a file into the test database before unit tests are run?
I think you have few ways to make this. In my opinion, the best solution - to add migration with your custom SQL. In future, you'll need that migration not only at development, but also at production stage. So, It would be not clear deploy procedure, if you'll store change to DB in few places.
Other way - just to add execution of your SQL to setUp method of testCase.
Additional migration
You should create a new empty migration ./manage.py makemigrations --empty myApp
Add your SQL code to operations list
operations = [
migrations.RunSQL('RAW SQL CODE')
]
Another solution to this is to create a management command that executes the required SQL Query and then just execute this command at the very beginning of your tests.
Below is my case:
Management command:
import os
from django.core.management import BaseCommand
from django.db import connection
from applications.cardo.utils import perform_query
from backend.settings import BASE_DIR
class Command(BaseCommand):
help = 'Loads all database scripts'
def handle(self, **options):
db_scripts_path = os.path.join(BASE_DIR, 'scripts', 'database_scripts')
utils_path = os.path.join(db_scripts_path, 'utils.sql')
with open(utils_path, mode='r') as f:
sql_query = f.read()
with connection.cursor() as cursor:
cursor.execute(sql_query)
Now you can just go to the terminal and type
python manage.py load_database_scripts
and the scripts will be loaded.
With a helper function like this one
def load():
with django_db_blocker.unblock():
call_command('load_database_scripts')
You just call this load function before the test suite runs.
How do I delete all the data in the database for on Django app? In previous version manage.py reset APPNAME did the job, but that's been deprecated.
What are we supposed to do now if we want to delete all the data from an app using the command line?
reset and sqlreset were both just wrappers around other management commands. sqlreset in particular can be duplicate by simply running:
python manage.py sqlclear myapp
python manage.py sqlall myapp
reset only served to automatically run the result of sqlreset on the database. Personally, I think removing that is a fantastic idea. Still, if you want similar functionality, you can just pipe the output to your database's shell commands.
For PostgreSQL, for example:
python manage.py sqlclear myapp | psql mydatabase
python manage.py sqlall myapp | psql mydatabase
If you want single command that should work with most database types you can pipe the drop table statements, that sqlclear generates, to dbshell
python manage.py sqlclear myapp | python manage.py dbshell
from django.contrib.contenttypes.models import ContentType
for ct in ContentType.objects.all()
ct.model_class().objects.all().delete()
Now that Django integrates migrations by default, you first need to make migrations for your app are first unapplied then deleted.
Here is the command line that works at least with Django 1.8 (replacing by the application you want to delete all associated data and:
# First, update the DB so it thinks no migrations were applied to the app
python manage.py migrate --fake <app_name> zero
# Erase all migrations in the app folder
rm -r "<app_name>/migrations/*"
# Erase the application tables
python manage.py sqlclear <app_name> | python manage.py dbshell
# Recreate the app tables, that will be empty
python manage.py makemigrations <app_name>
python manage.py migrate <app_name>
DIY
If you want to do that from the command line, create the following custom command:
from django.core.management.base import AppCommand, CommandError
from django.utils.six.moves import input
from django.db import DEFAULT_DB_ALIAS, connections
class Command(AppCommand):
help = (
'Removes ALL DATA related to the given app from the database '
'by calling model.objects.all().delete() for all app models. '
'This also removes related data in other apps via cascade.'
)
def add_arguments(self, parser):
super(Command, self).add_arguments(parser)
parser.add_argument(
'--noinput', '--no-input',
action='store_false', dest='interactive', default=True,
help='Tells Django to NOT prompt the user for input of any kind.',
)
parser.add_argument(
'--database', action='store', dest='database', default=DEFAULT_DB_ALIAS,
help='Nominates a database to reset. Defaults to the "default" database.',
)
def handle_app_config(self, app_config, **options):
app_label = app_config.label
database = options['database']
interactive = options['interactive']
db_name = connections[database].settings_dict['NAME']
confirm = (ask_confirmation(app_label, db_name)
if interactive else 'yes')
if confirm == 'yes':
for model in app_config.get_models():
model.objects.using(database).all().delete()
self.stdout.write('Reset done.\n')
else:
self.stdout.write("Reset cancelled.\n")
def ask_confirmation(app_label, db_name):
return input("""You have requested a reset of the application {app_label}.
This will IRREVERSIBLY DESTROY all data related to the app currently in
the {db_name} database, and return each table to empty state.
Are you sure you want to do this?
Type 'yes' to continue, or 'no' to cancel: """.format(**locals()))
Copy it to app/management/commands folder in any of your apps folders and run it with
./manage.py app_db_tables_reset any_installed_app_name
Ready-made package
The command is available in the django_commands package, you can install it with
pip install git+http://github.com/mrts/django-commands.git
and add it to INSTALLED_APPS to activate the command.
Tested with Django 1.9, it may or may not work with 1.8.
I am unable to use syncdb because my app uses some MySQL views. I have run manage.py sqlall <app>, but this does not output the SQL for django_content_type table or the auth_permission tables. I have also had a look into south and django evolution, but they both require syncdb, and I'm not sure they would help anyway.
I have manually added some models to the tables, but this is getting frustrating, and having installed the dbsettings app I am unsure of what I now need to enter.
Does anyone know of a way to get manage.py (or something else) to output the SQL for these tables and their contents?
Thanks.
Having done a bit more digging, I found these:
Fixing the auth_permission table after renaming a model in Django and manage.py sql command for django models - Django.
These output the tables, but not the data:
python manage.py sql auth
python manage.py sql admin
But this gets a lot closer. In the end I managed it with the following:
from django.contrib.auth.management import create_permissions
from django.db.models import get_apps
for app in get_apps():
create_permissions(app, None, 2)
from django.contrib.contenttypes.management import update_all_contenttypes
update_all_contenttypes(interactive=True)
This adds all the permissions and then all the content types which are needed. interactive=True means that it asks you if you want to remove stale content types.
#hajamie solution works for older supported version, taking a hint, below is what worked for me!
django = 1.9.7
from django.contrib.auth.management import create_permissions
from django.contrib.auth.models import Permission
from django.apps import apps
def fix_user_permission():
"""
run this method via shell whenever any amendments in any of the tables is made
"""
print "fixing user permissions"
# delete pre-existing user permission
Permission.objects.all().delete()
apps.models_module = True
create_permissions(apps, verbosity=0)
apps.models_module = None
print "process completed - fixed user permissions"
The easiest solution I found is to install Django Extensions, add it to settings.INSTALLED_APPS and run:
manage.py update_permissions
With what command can I delete tables from a django db of a specific app/ only one table?
I did find all sorts of things, but not how to get rid of a table.
Also while I am at this.
I create the models and hit syncdb and then I have the tables.
If I want to update/add to those tables, do I run into problems?
Your best bet would be to get django-south installed in your machine.
if you are using pip, do pip install django-south
This allows you too migrate data forward and backwards.
This is very handy especially if you need to update tables, and new tables etc.
check it out.
adding south to apps are as easy as python manage.py schemamigration appname --initial
Make your changes in a model and run the following python manage.py schemamigration appname --auto
Once your data migration file has been created it'll tell you data is now ready to migrate.
Simply use python manage.py migrate appname
http://south.aeracode.org/docs/about.html
Hope this helps
If you are deleting a table, this is done in the model file of the specific app that you are trying to delete, there is no command for this, you just go into the file and delete it and then re-run syncdb, for your other question, the answer is the same.. every app folder should have a file called "models.py" and here is where the models which are, in this case, equivalent to tables are specified, along with their fields, you simply edit this to make any changes.
def reset():
import install
from django.db.models import get_models
removed_tables = []
exceptions = []
for model in get_models():
if model.__name__ not in ('User','Session','Group','Permission'):
try:
model.objects.all().delete() # So we can remove the table without complaints from the database server.
CURSOR.execute('DROP TABLE %s' % model._meta.db_table)
print "Dropped table %s from model %s" % (model._meta.db_table, model.__name__)
except Exception as e:
exceptions.append([model._meta.db_table, str(e)])
continue
removed_tables.append(model._meta.db_table)
print "Removed %s tables" % len(removed_tables)
syncdb()
install.install() # A function that leads to the creation of my default data