I would like to add data to a database with Flask-SQLAlchemy without the Flask app running.
Is there a way to get db back from the app after the app and the database have been initialized.
My code looks like
db = SQLAlchemy()
def init_app():
app = Flask(__name__)
db = SQLAlchemy(app)
db.init_app(app)
return app
And what I would like to do is something like
from app import init_app
app = init_app() # initialized but not running
# db is used in model.py, but not initialized
# with Flask
# from db = SQLAlchemy()
from model import Machine # class Machine(db.Model)
p = Machine(name='something')
# now I need the initialized db from somewhere
db.session.add(p)
db.session.commit()
Basically I would like to do what's described here:
Another disadvantage is that Flask-SQLAlchemy makes using the database
outside of a Flask context difficult. This is because, with
FLask-SQLAlchemy, the database connection, models, and app are all
located within the app.py file. Having models within the app file, we
have limited ability to interact with the database outside of the app.
This makes loading data outside of your app difficult. Additionally,
this makes it hard to retrieve data outside of the Flask context.
Well, once you initialize the app, Flask spines a server (either development or production, whichever you set), so if you would like to add data to a database with Flask-SQLAlchemy without the Flask app running, you would better use the flask shell command which runs in the context of the current app, then you could add your data.
But first, it would be better is you set up your app as the following so we could directly import stuff like db, auth, etc:
...
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
def init_app():
app = Flask(__name__)
db.init_app(app)
return app
In the root of your project, type in the terminal the following command
flask shell
Now that you have a shell running in the context of the current app but not the server not running:
from app import db
from model import Machine # class Machine(db.Model)
p = Machine(name='something')
# now I need the initialized db from somewhere
db.session.add(p)
db.session.commit()
From the wonderful tutorial...
https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-iv-database
Define something like this...
from app import app, db
from app.models import User, Post
#app.shell_context_processor
def make_shell_context():
return {'db': db, 'User': User, 'Post': Post}
And then...
(venv) $ flask shell
>>> db
<SQLAlchemy engine=sqlite:////Users/migu7781/Documents/dev/flask/microblog2/app.db>
>>> User
<class 'app.models.User'>
>>> Post
<class 'app.models.Post'>
my question is more about setting up and adding data.
I am making the first project with Mongo DB on VS Code and can not find solution what to do next.
Google said Instal Cosmos DB, so I did but what next? Does it talk to Flask already? I am lost, need help from someone who works on Flask and MongoDB on VSCode!
Feel like an idiot now!
from flask import Flask
from flask_pymongo import PyMongo
app = Flask(__name__)
app.config["MONGO_URI"] = "mongodb://localhost:27017/"
mongo = PyMongo(app)
from cocktails.main.views import main
app.register_blueprint(main)
Does it talk to Flask already?
Hi,Patrycja. Quick answer is YES! I follow your description and do the steps in this document1 and document2 to create Flask project which accesses Cosmos DB Mongo API.
My app.py looks like below:
from flask import Flask
from flask_pymongo import PyMongo
app = Flask(__name__)
app.debug = True
app.config["MONGO_URI"] = "******"
mongo = PyMongo(app)
print(mongo.db)
#app.route("/")
def home_page():
items = mongo.db.test.find()
for item in items:
name = item["name"]
return name
I use Cosmos DB Mongo API, the sample data as below:
Run the command python -m flask run: ,get output:
The key point is "MONGO_URI", it is the connection string:
If you concern about the cost,you could use Cosmos DB Emulator for your test. Surely, as #Stennie mentioned in the comment,cosmos db mongo api only supports partial features of Mongo DB. But if your work is only limited to simple queries, you could use it.Otherwise,please consider using Mongo Atlas.
I've got some Postgres stored procedures that my selenium tests will depend on. In development, I load them with a line in a script:
cat stored_procedures.sql | python manage.py dbshell
This doesn't work when unit testing, since a fresh database is created from scratch. How can I load stored procedures saved in a file into the test database before unit tests are run?
I think you have few ways to make this. In my opinion, the best solution - to add migration with your custom SQL. In future, you'll need that migration not only at development, but also at production stage. So, It would be not clear deploy procedure, if you'll store change to DB in few places.
Other way - just to add execution of your SQL to setUp method of testCase.
Additional migration
You should create a new empty migration ./manage.py makemigrations --empty myApp
Add your SQL code to operations list
operations = [
migrations.RunSQL('RAW SQL CODE')
]
Another solution to this is to create a management command that executes the required SQL Query and then just execute this command at the very beginning of your tests.
Below is my case:
Management command:
import os
from django.core.management import BaseCommand
from django.db import connection
from applications.cardo.utils import perform_query
from backend.settings import BASE_DIR
class Command(BaseCommand):
help = 'Loads all database scripts'
def handle(self, **options):
db_scripts_path = os.path.join(BASE_DIR, 'scripts', 'database_scripts')
utils_path = os.path.join(db_scripts_path, 'utils.sql')
with open(utils_path, mode='r') as f:
sql_query = f.read()
with connection.cursor() as cursor:
cursor.execute(sql_query)
Now you can just go to the terminal and type
python manage.py load_database_scripts
and the scripts will be loaded.
With a helper function like this one
def load():
with django_db_blocker.unblock():
call_command('load_database_scripts')
You just call this load function before the test suite runs.
I'm building flask application and decided to try application factory approach this time, but got into trouble with flask-migrate and can't figure out simple solution.
Please note that I want to pass config location as an option to the script
manage.py:
manager = Manager(create_app)
manager.add_option("-c", "--config", dest="config_module", required=False)
then i need to create migrate instance and add command to the manager:
with manager.app.app_context():
migrate = Migrate(current_app, db)
manager.add_command('db', MigrateCommand)
but app instance is not created yet, so it fails
I know I can pass config in environment variable and create application before creating manager instance, but how to do it using manager options?
When you use a --config option you have to use an application factory function as you know, as this function gets the config as an argument and that allows you to create the app with the right configuration.
Since you have an app factory, you have to use the deferred initialization for all your extensions. You instantiate your extensions without passing any arguments, and then in your app factory function after you create the application, you call init_app on all your extensions, passing the app and db now that you have them.
The MigrateCommand is completely separate from this, you can add that to the Manager without having app and db instances created.
Example:
manage.py:
from app import create_app
from flask_migrate import MigrateCommand, Manager
manager = Manager(create_app)
manager.add_option("-c", "--config", dest="config_module", required=False)
manager.add_command('db', MigrateCommand)
app.py (or however your app factory module is called)
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
db = SQLAlchemy()
migrate = Migrate()
def create_app(config):
app = Flask(__name__)
# apply configuration
# ...
# initialize extensions
db.init_app(app)
migrate.init_app(app, db)
return app
My project is in early development. I frequently delete the database and run manage.py syncdb to set up my app from scratch.
Unfortunately, this always pops up:
You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no):
Then you have supply a username, valid email adress and password. This is tedious. I'm getting tired of typing test\nx#x.com\ntest\ntest\n.
How can I automatically skip this step and create a user programatically when running manage.py syncdb ?
I know the question has been answered already but ...
A Much simpler approach is to dump the auth module data into a json file once the superuser has been created:
./manage.py dumpdata --indent=2 auth > initial_data.json
You can also dump the sessions data:
./manage.py dumpdata --indent=2 sessions
You can then append the session info to the auth module dump (and probably increase the expire_date so it does not expire... ever ;-).
From then, you can use
/manage.py syncdb --noinput
to load the superuser and his session when creating the db with no interactive prompt asking you about a superuser.
Instead of deleting your entire database, just delete the tables of your app before running the syncdb
This will accomplish it for you in a single line (per app):
python manage.py sqlclear appname | python manage.py dbshell
The first command will look at your app and generate the required SQL to drop the tables. This output is then piped to the dbshell to execute it.
After its done, run your syncdb to recreate the tables:
python manage.py syncdb
The key is to use --noinput at the time of syncdb & then use this one liner to create superuser
echo "from django.contrib.auth.models import User; User.objects.create_superuser('myadmin', 'myemail#example.com', 'hunter2')" | python manage.py shell
Credit : http://source.mihelac.org/2009/10/23/django-avoiding-typing-password-for-superuser/
If you want the ability — as I do — to really start with a fresh database without getting asked that superuser question, then you can just de-register the signal handler that asks that question. Check out the very bottom of the file:
django/contrib/auth/management/__init__.py
to see how the registration of the superuser function gets performed. I found that I could reverse this registration, and never get asked the question during "syncdb", if I placed this code in my "models.py":
from django.db.models import signals
from django.contrib.auth.management import create_superuser
from django.contrib.auth import models as auth_app
# Prevent interactive question about wanting a superuser created. (This
# code has to go in this otherwise empty "models" module so that it gets
# processed by the "syncdb" command during database creation.)
signals.post_syncdb.disconnect(
create_superuser,
sender=auth_app,
dispatch_uid = "django.contrib.auth.management.create_superuser")
I am not sure how to guarantee that this code gets run after the Django code that does the registration. I had thought that it would depend on whether your app or the django.contrib.auth app gets mentioned first in INSTALLED_APPS, but it seems to work for me regardless of the order I put them in. Maybe they are done alphabetically and I'm lucky that my app's name starts with a letter later than "d"? Or is Django just smart enough to do its own stuff first, then mine in case I want to muck with their settings? Let me know if you find out. :-)
I've overcome this feature using south
Its a must have for any django developer.
South is a tool designed to help migrate changes over to the live site without destroying information or database structure. The resulting changes can be tracked by south and using the generated python files - can perform the same actions on an alternative database.
During development, I use this tool to git track my database changes - and to make a change to the database without the need to destroy it first.
easy_install south
Add 'south' to your installed apps
Proposing first time run of south on an app.
$ python manage.py schemamigration appname --init
This will initiate schema detection on that app.
$ python manage.py migrate appname
This will apply the model changes
The database will have the new models.
Changing a model after the first run
$ python manage.py schemamigration appname --auto
$ python manage.py migrate appname
Models will have changed - data is not destroyed.
Plus south does much more...
Note: since version 1.7 syncdb command is deprecated. Use migrate instead.
Also Django 1.7 introduced AppConfig as means of customizing applications' initialization process.
Thus since Django 1.7 the simplest way to achieve what you want is to employ an AppConfig's subclass.
Let say, you happen to have your own example_app that is added to your INSTALLED_APPS and you want to create and admin user with admin password whenever you run ./manage.py migrate from scratch. I also assume that automatic admin user creation is required only in dev environment - not in production.
Add the following code to example_app/apps.py
# example_app/apps.py
from django.apps import AppConfig
from django.conf import settings
from django.db.models.signals import post_migrate
from django.contrib.auth.apps import AuthConfig
USERNAME = "admin"
PASSWORD = "admin"
def create_test_user(sender, **kwargs):
if not settings.DEBUG:
return
if not isinstance(sender, AuthConfig):
return
from django.contrib.auth.models import User
manager = User.objects
try:
manager.get(username=USERNAME)
except User.DoesNotExist:
manager.create_superuser(USERNAME, 'x#x.com', PASSWORD)
class ExampleAppConfig(AppConfig):
name = __package__
def ready(self):
post_migrate.connect(create_test_user)
Also add the following reference to the app configuration inside apps example_app/__init__.py:
# example_app/__init__.py
default_app_config = 'example_app.apps.ExampleAppConfig'
Where the default_app_config is a string Python path to the AppConfig subclass as mentioned here.
The manage.py reset command will reset your database without destroying your created super user. Data does however need to be re-imported.
You can use django-finalware to do this for you. Just add finalware to your INSTALLED_APPS and include the following in your settings.py:
SITE_SUPERUSER_USERNAME = 'myadmin'
SITE_SUPERUSER_EMAIL = 'myadmin#example.com'
SITE_SUPERUSER_PASSWORD = 'mypass' # this can be set from a secret file.
# optional object id. Ensures that the superuser id is not set to `1`.
# you can use this as a simple security feature
SITE_SUPERUSER_ID = '343'
Then just run ./manage.py syncdb (Django <1.7) or ./manage.py migrate (Django >= 1.7), and it will automatically create a superuser or update the existing one for you.
You are never prompted to created a superuser anymore.
Since Django 1.7 the suggested way of populating the database is through data migrations. To create a data migration for creating the admin you should first create an empty migration:
./manage.py makemigrations --empty myapp --name create-superuser
This will create an empty migration in myapp/migrations/000x__create-superuser.py. Edit the file to make it look like this:
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations, models
from django.contrib.auth.models import User
def create_superuser(apps, schema_editor):
User.objects.create_superuser(username='myadmin', password='mypassword', email='myemail#gmail.com')
class Migration(migrations.Migration):
dependencies = [('myapp', '000y_my-previous-migration-file'),]
operations = [migrations.RunPython(create_superuser)]
I have resolved creating a python script like this one to reset all my stuff [updated version][1.8 too]:
import os
import sys
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "main.settings.dev")
from django.conf import settings
from django.core import management
from django import get_version
PROJECT_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir))
if PROJECT_ROOT not in sys.path:
sys.path.append(PROJECT_ROOT)
yn = raw_input('Are you sure you want to reset everything? (y/n) ')
if yn == 'y':
# Drops the db / creates the db
if settings.DATABASES['default']['ENGINE'].find('mysql') != -1:
os.system('mysqladmin -uroot -pIronlord0 -f drop db')
os.system('mysqladmin -uroot -pIronlord0 -f create db')
elif settings.DATABASES['default']['ENGINE'].find('psycopg2') != -1:
os.system('psql -U postgres -c "DROP DATABASE db"')
os.system('psql -U postgres -c "CREATE DATABASE db WITH OWNER = admin"')
elif settings.DATABASES['default']['ENGINE'].find('sqlite3') != -1:
try:
os.remove(os.path.join(PROJECT_ROOT, 'data.db'))
except:
pass
# Getting application handle here otherwise db gets allocated and it can not be destroyed.
if get_version() > '1.6.10':
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
management.call_command('syncdb', interactive=False)
# Creates admin/password
from django.contrib.auth.management.commands import changepassword
management.call_command('createsuperuser', interactive=False, username="admin", email="xxx#example.com")
command = changepassword.Command()
command._get_pass = lambda *args: 'password'
if get_version() >= '1.8':
command.execute(username="admin")
else:
command.execute("admin")
# Creates the default site entry
from django.contrib.sites.models import Site
site = Site.objects.get_current()
site.domain = 'www.example.com'
site.name = ' xxx '
site.save()
it works like a charm!
P.S.: Be sure to stop your (testing) server where above db is in charge before running this script!
Take a look at the dumpdata management command. For instance:
python manage.py dumpdata > initial_data.json
If this file, called a fixture, is named initial_data (.xml or .json), then the syncdb command will pick it up and populate your tables accordingly. It will still ask you if you want to create a user, but I believe you may safely answer "no", after which point it will populate the database based on your fixture.
More info on this can be found in the docs.
Developing with sqlite.
Clear database by deleting file.
Load admin from fixtures.
change manage.py (django 1.4):
# hack to prevent admin promt
if len(sys.argv) == 2 and sys.argv[1] == 'syncdb':
sys.argv.append('--noinput')
My solution to this was to just not delete that auth tables when wiping out my database.
If you prefer to type initializing code direct into python source file, this code modified manage.py might help (and thanks for Cjkjvfnby's little code!):
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
# set your django setting module here
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "app.settings")
from django.core.management import execute_from_command_line
# hack to prevent admin prompt
if len(sys.argv) == 2 and sys.argv[1] == 'syncdb':
sys.argv.append('--noinput')
execute_from_command_line(sys.argv)
# additional process for creation additional user, misc data, and anything
for arg in sys.argv:
# if syncdb occurs and users don't exist, create them
if arg.lower() == 'syncdb':
print 'syncdb post process...'
from django.contrib.auth.models import User
admin_id = 'admin'
admin_email = 'superuser#mail.com'
admin_password = 'superuser_password'
additional_users = [
['tempuser', 'user_email#mail.com', 'tempuser_password']
]
# admin exists?
user_list = User.objects.filter(username=admin_id)
if len(user_list) == 0:
print 'create superuser: ' + admin_id
new_admin = User.objects.create_superuser(admin_id, admin_email, admin_password)
# additional user exists?
for additional_user in additional_users:
user_list = User.objects.filter(username=additional_user[0])
if len(user_list) == 0:
print 'create additional user: ' + additional_user[0]
new_admin = User.objects.create_user(additional_user[0], additional_user[1], additional_user[2])
# any other data
I'm just showing the user creation code here, but you can enhance this code more as you want.
I'm using sqlite as a dev database. After changing model classes, just drop the corresponding tables with sqlite manager (a firefox plugin, open to inspect the data anyways) and run manage.py syncdb to recreate what's missing.