Flask-sqlalchemy 3.0.2 creating a new DB inside a Flask endpoint - flask

I'm currently migrating my Flask app from flask-sqlalchemy 2.5.1 to 3.0.2. The app includes an endpoint that lets the client create a new database file at a selected path. Before the migration, it was achieved by simply setting the path and creating the tables in the following way:
app.config['SQLALCHEMY_DATABASE_URI'] = filepath
db.create_all()
However, the file is no longer automatically created in 3.0.2.
Question
I have been digging through the flask-sqlalchemy and sqlalchemy documentations for the past two days yet I can't find anything that would mention the changed behaviour. Creating the file before setting the config doesn't work either, as the tables are not created and the file size is 0b after calling the endpoint.
How can I create a new sqlalchemy DB file inside of a Flask endpoint?
Below is a minimal reproducible example:
server.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
import os
TEST_DB_PATH = os.path.join(os.getcwd(), "test.db")
# create the extension
db = SQLAlchemy()
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
# create the app
app = Flask(__name__)
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = True
# configure the first SQLite database, in memory
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///"
# initialize the app with the extension
db.init_app(app)
with app.app_context():
db.create_all()
#app.route('/test')
def test():
# configure the second SQLite database, relative to the app instance folder
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///" + TEST_DB_PATH
db.create_all()
return "OK", 200
if __name__ == '__main__':
app.run()
test_server.py
import server
import os
import pytest
#pytest.fixture(scope="session")
def client():
app = server.app
return app.test_client()
def test_create_db_file(client):
# clean local test.db file if it exists
try:
os.remove(server.TEST_DB_PATH)
except OSError:
pass
ans = client.get("test")
assert ans.status_code == 200, ans.data
# this assertion fails
assert os.path.exists(
server.TEST_DB_PATH), f"DB file created at execution cannot be found at " + server.TEST_DB_PATH
assert os.path.getsize(
server.TEST_DB_PATH) > 0, "Created db file size is 0"
Running the test
py -m pytest .\test_server.py
The assertions fail with flask-sqlalchemy 3.0.2 and pass with 2.5.1.
Environment
Windows 11 x64
flask-sqlalchemy 2.2.2
sqlalchemy 1.4.46
pytest 7.2.0
python 3.9.13

Related

flask_socketio & flask_sqlalchemy - socketio.emit() is blocked when flask_sqlalchemy is querying / committing

Here's a test block of code that can be used to repeat the issue I'm experiencing
from gevent import monkey as curious_george
curious_george.patch_all(thread=True)
from flask_socketio import SocketIO
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = "my_custom_uri"
app.config['SQLALCHEMY_POOL_SIZE'] = 20
app.config['SQLALCHEMY_MAX_OVERFLOW'] = 50
db = SQLAlchemy(app)
socketio = SocketIO(app)
class test_table(db.Model):
_id = db.Column("id",db.Integer,primary_key=True)
flagged = db.Column("flagged",db.Boolean(), default=False)
def __init__(self,flagged):
self.flagged = flagged
#app.route('/test')
def test_route():
for x in range(100):
test_flag = db.session.query(test_table).filter_by(_id=1).first()
test_flag.flagged = not test_flag.flagged
db.session.commit()
# print(x)
socketio.emit('x_test',x, broadcast=True)
return("success",200)
if __name__ == '__main__':
socketio.run(app, host='0.0.0.0', port=8080)
The database commits happen first and once they finish all of the socketio emits dump (in order) at once.
Maybe I have something misconfigured? Does sqlalchemy need to be handled in a more concurrent method?
I just reinstalled my virtual environment for other reasons
Flask-Socketio 5.2.0
python-engineio 4.3.4
python-socketio 5.7.2
gevent 22.10.2
gevent-websocket 0.10.1
I don't have any related warnings on the flask server startup
I expected the socketio emits to happen in time with the database query commits
Turns out I needed to patch psycopg2 as well using the "psycogreen" library
I used this patch to fix it, and it works now.
from psycogreen import gevent
gevent.patch_psycopg()

How to get a handle to the initialized database from a Flask application with SQLAlchemy?

I would like to add data to a database with Flask-SQLAlchemy without the Flask app running.
Is there a way to get db back from the app after the app and the database have been initialized.
My code looks like
db = SQLAlchemy()
def init_app():
app = Flask(__name__)
db = SQLAlchemy(app)
db.init_app(app)
return app
And what I would like to do is something like
from app import init_app
app = init_app() # initialized but not running
# db is used in model.py, but not initialized
# with Flask
# from db = SQLAlchemy()
from model import Machine # class Machine(db.Model)
p = Machine(name='something')
# now I need the initialized db from somewhere
db.session.add(p)
db.session.commit()
Basically I would like to do what's described here:
Another disadvantage is that Flask-SQLAlchemy makes using the database
outside of a Flask context difficult. This is because, with
FLask-SQLAlchemy, the database connection, models, and app are all
located within the app.py file. Having models within the app file, we
have limited ability to interact with the database outside of the app.
This makes loading data outside of your app difficult. Additionally,
this makes it hard to retrieve data outside of the Flask context.
Well, once you initialize the app, Flask spines a server (either development or production, whichever you set), so if you would like to add data to a database with Flask-SQLAlchemy without the Flask app running, you would better use the flask shell command which runs in the context of the current app, then you could add your data.
But first, it would be better is you set up your app as the following so we could directly import stuff like db, auth, etc:
...
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
def init_app():
app = Flask(__name__)
db.init_app(app)
return app
In the root of your project, type in the terminal the following command
flask shell
Now that you have a shell running in the context of the current app but not the server not running:
from app import db
from model import Machine # class Machine(db.Model)
p = Machine(name='something')
# now I need the initialized db from somewhere
db.session.add(p)
db.session.commit()
From the wonderful tutorial...
https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-iv-database
Define something like this...
from app import app, db
from app.models import User, Post
#app.shell_context_processor
def make_shell_context():
return {'db': db, 'User': User, 'Post': Post}
And then...
(venv) $ flask shell
>>> db
<SQLAlchemy engine=sqlite:////Users/migu7781/Documents/dev/flask/microblog2/app.db>
>>> User
<class 'app.models.User'>
>>> Post
<class 'app.models.Post'>

command "python3 app.py db migrate" but server is running, not migrating

Experts!
I am a beginner in Flask.
I have any project and it includes some models(already defined).
This is one of my models and these are placed as other files, not in app.py.
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
from sqlalchemy import Column, Integer, SmallInteger, String, Text, Date, Boolean, Float
class CategoryTopicLink(Base):
__tablename__ = 'category_topic_link'
id = Column(Integer, primary_key=True)
category_id = Column(Integer)
topic_id = Column(Integer)
And I am going to migrate these by running app.py.
So I inputted some code in app.py and my app file look at following:
from flask import Flask, jsonify, request, make_response
...
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
app = Flask(__name__)
CORS(app)
api = Api(app)
db = SQLAlchemy(app)
migrate = Migrate(app, db)
from .models import CategoryTopicLink
........
if __name__=="__main__":
app.run(debug=True,host="127.0.0.1",port="5001")
And To migrate model I commanded
"python3 app.py db init" (but server run)
"python3 app.py db migrate" (also server run, not migrate)
"flask run db init" (error - flask has not db attribute)
"flask run db migrate" (error - flask has not db attribute)
......
Please let me know how to migrate models.
Any help will be appreciated.
Regards, from jis0324!!!
The commands are always in the form
flask db command
Also see the manual
https://flask-migrate.readthedocs.io/en/latest/

Define models in seperate file when using app factory flask

I am creating a flask application where I am using the app factory method. I have a file in the application folder __init__.py which has the create_app function with the following code
def create_app(test_config=None):
app = Flask(__name__,instance_relative_config=True)
app.config.from_mapping(
SECRET_KEY='dev',
)
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:#localhost/database'
db = SQLAlchemy(app)
if test_config == None:
app.config.from_pyfile('config.py',silent=True)
else:
app.config.form_mapping(test_config)
from flaskr import models
try:
os.makedirs(app.instance_path)
except OSError:
pass
class User(db.Model):
id = db.Column(db.Integer,primary_key=True)
uname = db.Column(db.String(50))
#app.route('/hello')
def hello():
return json.dumps({'message':'hello','status':True})
#app.route('/getusers')
def getusers():
u = User.query.get(1)
return json.dumps({'uname':u.uname})
return app
What I want is to define the models in a seperate file. How can I do this?
I have tried defining in a seperate file and importing it. But the problem is the model inherits the db.Model which is then not available in the imported file.
Leave the creation of db object outside create_app without passing any app instance and use the SQLAlchemy.init_app method to configure and init. your db object, this way you can import it from any file:
db = SQLAlchemy()
#...
def create_app(test_config=None):
app = Flask(__name__,instance_relative_config=True)
#...
db.init_app(app)
More on this topic can be found at flask's documentation
I've fought with this problem a few hours too. Couldn't fix it until I came to the realization, that I have to return the app inside the app.app_context()
model.py:
from flask_user import UserMixin
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class User(db.Model, UserMixin):
__tablename__ = 'users'
id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(255, collation='NOCASE'), nullable=False,
unique=True)
# ...
__init__.py:
from flask import Flask
from flask_migrate import Migrate
def create_app(test_config=None):
app = Flask(__name__, instance_relative_config=True)
app.config.from_pyfile('config.py', silent=True)
migrate = Migrate()
from appname.model import db
db.init_app(app)
migrate.init_app(app, db)
with app.app_context():
# Create all database tables
db.create_all()
import appname.routes
# Apply the blueprints to the app
from appname import bp1, bp2
appname.register_blueprint(bp1.bp)
appname.register_blueprint(bp2.bp)
return app
Run with:
cd ~/Folder/Webapp
. appname/bin/activate
export FLASK_APP=appname
export FLASK_ENV=development
flask run
(in Linux terminal)
or create this file and run it in python shell:
from appname import create_app
if __name__ == '__main__':
app = create_app()
app.run(debug=True)

flask test_client() unittest

I am new to programming in general and this is my first web application in python (flask, sqlalchemy, wtforms, etc). I have been using the realpython.com course 2 as my study material on this subject. I have gotten to the point where i am learning about unit testing and i having trouble getting it to work correctly. I have compared the course example to the examples i found online and i am not seeing the issue with my code.
The problem i am encountering is that the test.py script correctly creates my test.db database but when it attempts to insert a test customer and it puts it into my production db (madsenconcrete.db) instead of my test db (test.db). If i remove the production db from the script directory it will raise this error when it cant find the db because its looking for madsenconcrete.db not test.db.
OperationalError: (sqlite3.OperationalError) no such table: customer [SQL: u'INSERT INTO customer (name, email, telephone, created_date) VALUES (?, ?, ?, ?)'] [parameters: ('Acme Company', 'acme#domain.com', '6125551000', '2016-01-03')]
I am not sure how to troubleshoot this issue. I have doing a lot of stare and compares and i do not see the difference.
import os
import unittest
import datetime
import pytz
from views import app, db
from _config import basedir
from models import Customer
TEST_DB = 'test.db'
class AllTests(unittest.TestCase):
############################
#### setup and teardown ####
############################
# executed prior to each test
def setUp(self):
app.config['TESTING'] = True
app.config['WTF_CSRF_ENABLED'] = False
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + os.path.join(basedir, TEST_DB)
app.config['SQLALCHEMY_ECHO'] = True
self.app = app.test_client()
db.create_all()
# executed after each test
def tearDown(self):
db.session.remove()
db.drop_all()
# each test should start with 'test'
def test_customer_setup(self):
new_customer = Customer("Acme Company", "acme#domain.com", "6125551000",
datetime.datetime.now(pytz.timezone('US/Central')))
db.session.add(new_customer)
db.session.commit()
if __name__ == "__main__":
unittest.main()
There would be an extensive amount of code i would have to paste so show all the dependencies. You can find the source code here.
https://github.com/ande0581/madsenconcrete
Thanks
Ultimately, the problem is that you are creating your db object from an already configured app:
# config
app = Flask(__name__)
app.config.from_object('_config')
db = SQLAlchemy(app)
If you use the create_app pattern (documented in more detail in this answer) you will be able to alter the configuration you are loading for your test application.