Does anyone have an example of successfully using flask-migrate on pythonanywhere? Does anyone have a simple example of what an app.py using migration should look like? Something along the lines of:
from flask import Flask, request, render_template
from flask.ext.sqlalchemy import SQLAlchemy
from flask.ext.migrate import Migrate, MigrateCommand
from flask.ext.script import Manager
app = Flask(__name__ )
app.secret_key = 'This is really unique and secret'
app.config['SQLALCHEMY_DATABASE_URI'] = '<whatever>'
db = SQLAlchemy(app)
db.create_all()
manager = Manager( app )
migrate = Migrate( app, db )
manager.add_command('db', MigrateCommand )
I find that when I try to run
python app.py db init
it fails to generate the migration repository. Could this be a file permission issue on pythonanywhere?
Author of Flask-Migrate here.
The idea is that you generate your db repository in your development environment. You have to commit your repository along with your source files to source control.
Then when you install the application on your hosting service all you need to do is run the upgrade command to get your db created and migrated to the latest revision.
Update: base on your comments below, you want to develop an application from scratch. I just tested this myself, and was able to create a db repository, create a migration, and apply it. What I did is start the pythonanywhere bash console. Here is a copy of my complete session:
17:39 ~ $ mkdir dbtest
17:39 ~ $ cd dbtest
17:39 ~/dbtest $ virtualenv venv
New python executable in venv/bin/python2.7
Also creating executable in venv/bin/python
Installing setuptools............done.
Installing pip...............done.
17:39 ~/dbtest $ . venv/bin/activate
(venv)17:39 ~/dbtest $ pip install flask flask-migrate
...
(venv)17:42 ~/dbtest $ vi dbtest.py
... (entered flask-migrate example code, see below)
(venv)17:47 ~/dbtest $ python dbtest.py
usage: dbtest.py [-?] {shell,db,runserver} ...
positional arguments:
{shell,db,runserver}
shell Runs a Python shell inside Flask application context.
db Perform database migrations
runserver Runs the Flask development server i.e. app.run()
optional arguments:
-?, --help show this help message and exit
(venv)17:47 ~/dbtest $ python dbtest.py db init
Creating directory /home/miguelgrinberg/dbtest/migrations ... done
Creating directory /home/miguelgrinberg/dbtest/migrations/versions ... done
Generating /home/miguelgrinberg/dbtest/migrations/README ... done
Generating /home/miguelgrinberg/dbtest/migrations/alembic.ini ... done
Generating /home/miguelgrinberg/dbtest/migrations/env.py ... done
Generating /home/miguelgrinberg/dbtest/migrations/script.py.mako ... done
Generating /home/miguelgrinberg/dbtest/migrations/env.pyc ... done
Please edit configuration/connection/logging settings in '/home/miguelgrinberg/dbtest/migrations/alembic.ini' before proceeding.
(venv)17:54 ~/dbtest $ python dbtest.py db migrate
INFO [alembic.migration] Context impl SQLiteImpl.
INFO [alembic.migration] Will assume non-transactional DDL.
INFO [alembic.autogenerate.compare] Detected added table 'user'
Generating /home/miguelgrinberg/dbtest/migrations/versions/1c4aa671e23a_.py ... done
(venv)17:54 ~/dbtest $ python dbtest.py db upgrade
INFO [alembic.migration] Context impl SQLiteImpl.
INFO [alembic.migration] Will assume non-transactional DDL.
INFO [alembic.migration] Running upgrade None -> 1c4aa671e23a, empty message
(venv)17:55 ~/dbtest $ ls -l
total 8
-rw-r--r-- 1 miguelgrinberg registered_users 3072 Sep 28 2014 app.db
-rwxrwxr-x 1 miguelgrinberg registered_users 511 Sep 28 17:48 dbtest.py
drwxrwxr-x 3 miguelgrinberg registered_users 100 Sep 28 17:55 migrations
drwxrwxr-x 6 miguelgrinberg registered_users 52 Sep 28 17:41 venv
The example application that I used to test this is below:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
from flask.ext.script import Manager
from flask.ext.migrate import Migrate, MigrateCommand
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///app.db'
db = SQLAlchemy(app)
migrate = Migrate(app, db)
manager = Manager(app)
manager.add_command('db', MigrateCommand)
class User(db.Model):
id = db.Column(db.Integer, primary_key = True)
name = db.Column(db.String(128))
if __name__ == '__main__':
manager.run()
Any chance you already have a migration repository created? Or a database?
Related
I followed the tutorial on http://docs.celeryproject.org/en/latest/ and I am on virtualbox (Xubuntu 16.XX TLS), Django 1.11.3, Celery 4.1 . rabbitmq 3.6.14, Python 2.7 .
and when I started the daemonization with the init-script: celerybeat (with /etc/default/celeryd config file)
[2017-11-19 01:13:00,912: INFO/MainProcess] beat: Starting...
and nothing more after. Do you see what could I make wrong ?
My celery.py:
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'oscar.settings')
app = Celery('oscar')
# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
# should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')
# Broker settings
app.conf.broker_url = 'amqp://oscar:oscar#localhost:5672/oscarRabbit'
# Load task modules from all registered Django app configs.
app.autodiscover_tasks()
some_app/tasks.py:
from __future__ import absolute_import, unicode_literals
from oscar import celery_app
from celery.schedules import crontab
from .models import HelpRequest
from datetime import datetime, timedelta
import logging
""" CONSTANTS FOR THE TIMER """
# Can be changed (by default 1 week)
WEEKS_BEFORE_PENDING = 0
DAYS_BEFORE_PENDING = 0
HOURS_BEFORE_PENDING = 0
MINUTES_BEFORE_PENDING = 1
# http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html
# for schedule : http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html#crontab-schedules
#celery_app.on_after_configure.connect
def setup_periodic_tasks(sender, **kwargs):
sender.add_periodic_task(
crontab(minute=2),
set_open_help_request_to_pending
)
#celery_app.task(name="HR_OPEN_TO_PENDING")
def set_open_help_request_to_pending():
""" For timedelay idea : https://stackoverflow.com/a/27869101/6149867 """
logging.info("RUNNING CRON TASK FOR STUDENT COLLABORATION : set_open_help_request_to_pending")
request_list = HelpRequest.objects.filter(
state=HelpRequest.OPEN,
timestamp__gte=datetime.now() - timedelta(hours=HOURS_BEFORE_PENDING,
minutes=MINUTES_BEFORE_PENDING,
days=DAYS_BEFORE_PENDING,
weeks=WEEKS_BEFORE_PENDING)
)
if request_list:
logging.info("FOUND ", request_list.count(), " Help request(s) => PENDING")
for help_request in request_list.all():
help_request.change_state(HelpRequest.PENDING)
/etc/default/celeryd:
# Names of nodes to start
# most people will only start one node:
CELERYD_NODES="worker1"
# but you can also start multiple and configure settings
# for each in CELERYD_OPTS
#CELERYD_NODES="worker1 worker2 worker3"
# alternatively, you can specify the number of nodes to start:
#CELERYD_NODES=10
# Absolute or relative path to the 'celery' command:
CELERY_BIN="/home/jy95/Documents/oscareducation/ve/local/bin/celery"
# App instance to use
# comment out this line if you don't use an app
CELERY_APP="oscar"
# Where to chdir at start.
CELERYD_CHDIR="/home/jy95/Documents/oscareducation"
# Extra command-line arguments to the worker
# django_celery_beat for admin purpuse
CELERYD_OPTS="--scheduler django_celery_beat.schedulers:DatabaseScheduler -f /var/log/celery/celery_tasks.log"
# Set logging level to DEBUG
#CELERYD_LOG_LEVEL="DEBUG"
# %n will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"
# Workers should run as an unprivileged user.
# You need to create this user manually (or you can choose
# a user/group combination that already exists (e.g., nobody).
CELERYD_USER="celery"
CELERYD_GROUP="celery"
# If enabled pid and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1
My setup of rabbitmq :
$ sudo rabbitmqctl add_user oscar oscar
$ sudo rabbitmqctl add_vhost oscarRabbit
$ sudo rabbitmqctl set_user_tags oscar administrator
$ sudo rabbitmqctl set_permissions -p oscarRabbit oscar ".*" ".*" ".*"
The commands I run to start (and their messages) :
sudo rabbitmq-server -detached
sudo /etc/init.d/celerybeat start
Warning: PID file not written; -detached was passed.
/etc/init.d/celerybeat: lerybeat: not found
celery init v10.1.
Using configuration: /etc/default/celeryd
Starting celerybeat...
sudo /etc/init.d/celerybeat start
source ve/bin/activate
python manage.py runserver
Performing system checks...
System check identified no issues (0 silenced).
November 19, 2017 -01:49:22 Django version 1.11.3, using settings 'oscar.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
Thanks for your answer
It looks like you've started a celerybeat process and your server, but haven't started a celery worker process.
python celery -A proj worker -B
(where proj is the name of your project).
Note that you can start a celery worker with an embedded beat process rather than needing to run celerybeat separately.
Trying to read an .ini db file from python to test a connection to a PostGres database
Python Version 2.7.11
In Fedora, I installed with
sudo dnf install python-configparser
Install 1 Package
Total download size: 41 k
Installed size: 144 k
Is this ok [y/N]: y
Downloading Packages:
python-configparser-3.5.0b2-0.2.fc23.noarch.rpm 40 kB/s | 41 kB 00:01
Total 31 kB/s | 41 kB 00:01
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Installing : python-configparser-3.5.0b2-0.2.fc23.noarch 1/1
Verifying : python-configparser-3.5.0b2-0.2.fc23.noarch 1/1
Installed:
python-configparser.noarch 3.5.0b2-0.2.fc23
in my config.py I do a
import configparser
when I run my script I get 'module' object is not callable
db.ini
[test1]
host=IP address
database=db
port=5432
user=username
password=pswd
[test2]
host=localhost
database=postgres
port=7999
user=abc
password=abcd
config.py
#!/usr/bin/env python
#from configparser import ConfigParser
import configparser
def config(filename='database.ini', section='gr'):
# create a parser
parser = configparser()
# read config file
parser.read(filename)
# get section, default to gr
db = {}
if parser.has_section(section):
params = parser.items(section)
for param in params:
db[param[0]] = param[1]
else:
raise Exception('Section {0} not found in the {1} file'.format(section, filename))
return db
testing.py
#!/usr/bin/env python
import psycopg2
from config import config
def connect():
""" Connect to the PostgreSQL database server """
conn = None
try:
# read connection parameters
params = config()
# connect to the PostgreSQL server
print('Connecting to the PostgreSQL database...')
conn = psycopg2.connect(**params)
# create a cursor
cur = conn.cursor()
# execute a statement
print('PostgreSQL database version:')
cur.execute('SELECT version()')
# display the PostgreSQL database server version
db_version = cur.fetchone()
print(db_version)
# close the communication with the PostgreSQL
cur.close()
except (Exception, psycopg2.DatabaseError) as error:
print(error)
finally:
if conn is not None:
conn.close()
print('Database connection closed.')
if __name__ == '__main__':
connect()
[root#svr mytest]# ./testing.py
'module' object is not callable
any ideas ?
thank you.
You have an error in your parser creation, see the ConfigParser examples:
An example of reading the configuration file again:
import ConfigParser
config = ConfigParser.RawConfigParser()
config.read('example.cfg')
In Python, a module is basically just a file, so if you want to use anything from this module, you have to specify what from the module you want. In your case, change the lines to
# create a parser
parser = configparser.ConfigParser()
This way, you are using the class ConfigParser from the module.
You can also use the follow to import the parser:
from configparser import ConfigParser
instead of
import configparser
I had the same issue as you and this worked for me.
Python 2.7
Django 1.10
settings.ini file(located at "/opts/myproject/settings.ini"):
[settings]
DEBUG: True
SECRET_KEY: '5a88V*GuaQgAZa8W2XgvD%dDogQU9Gcc5juq%ax64kyqmzv2rG'
On my django settings file I have:
import os
from ConfigParser import RawConfigParser
config = RawConfigParser()
config.read('/opts/myproject/settings.ini')
SECRET_KEY = config.get('settings', 'SECRET_KEY')
DEBUG = config.get('settings', 'DEBUG')
The setup works fine locally, but when I deploy to my server I get the following error if I try run any django management commands:
ConfigParser.NoSectionError: No section: 'settings'
If I go into Python shell locally I type in the above imports and read the file I get back:
['/opts/myproject/settings.ini']
On server I get back:
[]
I have tried changing "confif.read()" to "config.readfp()" as suggested on here but it didn't work.
Any help or advice is appreciated.
I created several servers, without any issue, with the stack nginx - uwsgi - flask using virtualenv.
with the current one uwsgi is throwing the error cannot import name "appl"
here is the myapp directory structure:
/srv/www/myapp
+ run.py
+ venv/ # virtualenv
+ myapp/
+ init.py
+ other modules/
+ logs/
here is the /etc/uwsgi/apps-avaliable/myapp.ini
[uwsgi]
# Variables
base = /srv/www/myapp
app = run
# Generic Config
# plugins = http, python
# plugins = python
home = %(base)/venv
pythonpath = %(base)
socket = /tmp/%n.sock
module = %(app)
callable = appl
logto = %(base)/logs/uwsgi_%n.log
and this is run.py
#!/usr/bin/env python
from myapp import appl
if __name__ == '__main__':
DEBUG = True if appl.config['DEBUG'] else False
appl.run(debug=DEBUG)
appl is defined in myapp/ _ init _ .py as an instance of Flask()
(underscores spaced just to prevent SO to turn them into bold)
I accurately checked the python code and indeed if I activate manually the virtualenvironment and execute run.py manually everything works like a charm, but uwsgi keeps throwing the import error.
Any suggestion what should I search more ?
fixed it, it was just a read permissions issue. The whole python app was readable by my user but not by the group, therefore uwsgi could not find it.
This was a bit tricky because I deployed successfully many time with the same script and never had permissions issues
In my uwsgi config, I have these options:
[uwsgi]
chmod-socket = 777
socket = 127.0.0.1:9031
plugins = python
pythonpath = /adminserver/
callable = app
master = True
processes = 4
reload-mercy = 8
cpu-affinity = 1
max-requests = 2000
limit-as = 512
reload-on-as = 256
reload-on-rss = 192
no-orphans
vacuum
My app structure looks like this:
/adminserver
app.py
...
My app.py has these bits of code:
app = Flask(__name__)
...
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5003, debug=True)
The result is that when I try to curl my server, I get this error:
Wed Sep 11 23:28:56 2013 - added /adminserver/ to pythonpath.
Wed Sep 11 23:28:56 2013 - *** no app loaded. going in full dynamic mode ***
Wed Sep 11 23:28:56 2013 - *** uWSGI is running in multiple interpreter mode ***
What do the module and callable options do? The docs say:
module, wsgi Argument: string
Load a WSGI module as the application. The module (sans .py) must be
importable, ie. be in PYTHONPATH.
This option may be set with -w from the command line.
callable Argument: string Default: application
Set default WSGI callable name.
Module
A module in Python maps to a file on disk - when you have a directory like this:
/some-dir
module1.py
module2.py
If you start up a python interpreter while the current working directory is /some-dir you will be able to import each of the modules:
some-dir$ python
>>> import module1, module2
# Module1 and Module2 are now imported
Python searches sys.path (and a few other things, see the docs on import for more information) for a file that matches the name you are trying to import. uwsgi uses Python's import process under the covers to load the module that contains your WSGI application.
Callable
The WSGI PEPs (333 and 3333) specify that a WSGI application is a callable that takes two arguments and returns an iterable that yields bytestrings:
# simple_wsgi.py
# The simplest WSGI application
HELLO_WORLD = b"Hello world!\n"
def simple_app(environ, start_response):
"""Simplest possible application object"""
status = '200 OK'
response_headers = [('Content-type', 'text/plain')]
start_response(status, response_headers)
return [HELLO_WORLD]
uwsgi needs to know the name of a symbol inside of your module that maps to the WSGI application callable, so it can pass in the environment and the start_response callable - essentially, it needs to be able to do the following:
wsgi_app = getattr(simple_wsgi, 'simple_app')
TL;PC (Too Long; Prefer Code)
A simple parallel of what uwsgi is doing:
# Use `module` to know *what* to import
import simple_wsgi
# construct request environment from user input
# create a callable to pass for start_response
# and then ...
# use `callable` to know what to call
wsgi_app = getattr(simple_wsgi, 'simple_app')
# and then call it to respond to the user
response = wsgi_app(environ, start_response)
For anyone else having this problem, if you are sure your configuration is correct, you should check your uWSGI version.
Ubuntu 12.04 LTS provides 1.0.3. Removing that and using pip to install 2.0.4 resolved my issues.
First, check your configuration whether is correct.
my uwsgi.ini configuration:
[uwsgi]
chdir=/home/air/repo/Qiy
uid=nobody
gid=nobody
module=Qiy.wsgi:application
socket=/home/air/repo/Qiy/uwsgi.sock
master=true
workers=5
pidfile=/home/air/repo/Qiy/uwsgi.pid
vacuum=true
thunder-lock=true
enable-threads=true
harakiri=30
post-buffering=4096
daemonize=/home/air/repo/Qiy/uwsgi.log
then use uwsgi --ini uwsgi.ini to run uwsgi.
if not work, you rm -rf the venv directory, and re-initial the venv, and re-try my step.
I re-initial the venv solved my issue, seems the problem is when I pip3 install some packages of requirements.txt, and upgrade the pip, then install uwsgi package. so, I delete the venv, and re-initial my virtual environment.