I am trying to run a django command on my heroku production server, but get the following error:
Note: The same command works fine in my local dev environment.
I took the following steps:
ssh onto my django server:
heroku ps:exec -a library-backend
I run my custom command:
python manage.py test_command
Receive error above
My environment variables are set in my settings.py as follows:
import environ
# Setting environment variables
env = environ.Env(DEBUG=(bool, False))
environ.Env.read_env()
DEBUG = env('DEBUG')
SECRET_KEY = env('SECRET_KEY')
DATABASE_URL = env('DATABASE_URL')
My django app runs normally on the heroku server. I am only getting this error when I try to run a custom django management command.
Can anyone see where I'm going wrong?
For reference, the management command is specified in library/management/commands/test_command.py:
from django.core.management.base import BaseCommand
class Command(BaseCommand):
def handle(self, *args, **options):
print("Testing management command")
According to the documentation around heroku ps:exec:
The SSH session created by Heroku Exec will not have the config vars set as environment variables (i.e., env in a session will not list config vars set by heroku config:set).
Ok, so I figured this out at last with a hint in the right direction from the comment. When you run:
heroku ps:exec -a <myapp>
Heroku will give you an ssh session with access to the files and folders, but won't set any of your environment variables (like SECRET_KEY or DATABASE_URL)
To get an ssh session with environment variables set, instead used:
heroku run bash -a <myapp>
Now you can run your django command and you won't get any ImproperlyConfigured errors.
How are you meant to debug errors in Flask? Print to the console? Flash messages to the page? Or is there a more powerful option available to figure out what's happening when something goes wrong?
Running the app in debug mode will show an interactive traceback and console in the browser when there is an error. As of Flask 2.2, to run in debug mode, pass the --app and --debug options to the flask command.
$ flask --app example --debug run
Prior to Flask 2.2, this was controlled by the FLASK_ENV=development environment variable instead. You can still use FLASK_APP and FLASK_DEBUG=1 instead of the options above.
For Linux, Mac, Linux Subsystem for Windows, Git Bash on Windows, etc.:
$ export FLASK_APP=example
$ export FLASK_DEBUG=1
$ flask run
For Windows CMD, use set instead of export:
set FLASK_DEBUG=1
For PowerShell, use $env:
$env:FLASK_DEBUG = "1"
If you're using the app.run() method instead of the flask run command, pass debug=True to enable debug mode.
Tracebacks are also printed to the terminal running the server, regardless of development mode.
If you're using PyCharm, VS Code, etc., you can take advantage of its debugger to step through the code with breakpoints. The run configuration can point to a script calling app.run(debug=True, use_reloader=False), or point it at the venv/bin/flask script and use it as you would from the command line. You can leave the reloader disabled, but a reload will kill the debugging context and you will have to catch a breakpoint again.
You can also use pdb, pudb, or another terminal debugger by calling set_trace in the view where you want to start debugging.
Be sure not to use too-broad except blocks. Surrounding all your code with a catch-all try... except... will silence the error you want to debug. It's unnecessary in general, since Flask will already handle exceptions by showing the debugger or a 500 error and printing the traceback to the console.
You can use app.run(debug=True) for the Werkzeug Debugger edit as mentioned below, and I should have known.
From the 1.1.x documentation, you can enable debug mode by exporting an environment variable to your shell prompt:
export FLASK_APP=/daemon/api/views.py # path to app
export FLASK_DEBUG=1
python -m flask run --host=0.0.0.0
One can also use the Flask Debug Toolbar extension to get more detailed information embedded in rendered pages.
from flask import Flask
from flask_debugtoolbar import DebugToolbarExtension
import logging
app = Flask(__name__)
app.debug = True
app.secret_key = 'development key'
toolbar = DebugToolbarExtension(app)
#app.route('/')
def index():
logging.warning("See this message in Flask Debug Toolbar!")
return "<html><body></body></html>"
Start the application as follows:
FLASK_APP=main.py FLASK_DEBUG=1 flask run
If you're using Visual Studio Code, replace
app.run(debug=True)
with
app.run()
It appears when turning on the internal debugger disables the VS Code debugger.
If you want to debug your flask app then just go to the folder where flask app is. Don't forget to activate your virtual environment and paste the lines in the console change "mainfilename" to flask main file.
export FLASK_APP="mainfilename.py"
export FLASK_DEBUG=1
python -m flask run --host=0.0.0.0
After you enable your debugger for flask app almost every error will be printed on the console or on the browser window.
If you want to figure out what's happening, you can use simple print statements or you can also use console.log() for javascript code.
To activate debug mode in flask you simply type set FLASK_DEBUG=1 on your CMD for windows, or export FLASK_DEBUG=1 on Linux terminal then restart your app and you are good to go!!
Install python-dotenv in your virtual environment.
Create a .flaskenv in your project root. By project root, I mean the folder which has your app.py file
Inside this file write the following:
FLASK_APP=myapp
FLASK_ENV=development
Now issue the following command:
flask run
When running as python app.py instead of the flask command, you can pass debug=True to app.run.
if __name__ == "__main__":
app.run(debug=True)
$ python app.py
with virtual env activate
export FLASK_DEBUG=true
you can configure
export FLASK_APP=app.py # run.py
export FLASK_ENV = "development"
to start
flask run
the result
* Environment: development
* Debug mode: on
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: xxx-xxx-xxx
and if you change
export FLASK_DEBUG=false
* Environment: development
* Debug mode: off
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
For Windows users:
Open Powershell and cd into your project directory.
Use these commandos in Powershell, all the other stuff won't work in Powershell.
$env:FLASK_APP = "app"
$env:FLASK_ENV = "development"
If you have PyCharm Professional, you can create a Flask server run configuration and enable the FLASK_DEBUG checkbox. Go to Run > Edit Configurations, select or create a Flask server configuration, and enable the FLASK_DEBUG checkbox. Click OK, then click the run button.
You can install python-dotenv with
pip install python-dotenv then create a .flask_env or a .env file
The contents of the file can be:
FLASK_APP=myapp
FLASK_DEBUG=True
Use loggers and print statements in the Development Environment, you can go for sentry in case of production environments.
I want to start the flask server using anaconda Prompt.
while starting,it is showing the following error.How to resolve error.
I am getting the following error.
Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
Debug mode: off
Usage: flask run [OPTIONS]
Error: Could not locate a Flask application. You did not provide the "FLASK_APP" environment variable, and a "wsgi.py" or "app.py" module was not found in the current directory.
The problem is you need to set flask's debug mode to true.
set FLASK_APP = yourfilename.py
set FLASK_DEBUG = 1
and then use:
flask run
to run your server.
Note: If you are on mac/linux use export instead of set
Have you used:
$ export FLASK_APP=yourfilename.py
$ export FLASK_DEBUG=1
$ flask run
My institute has some parallel computers with CentOS release 6.4 and Python2.6 installed. I want to use python 2.7, then I installed python 2.7.11 locally (without root permission, because I am not a root). My problem is that when I use Portable Batch System (PBS) to run python scripts, some import statements raise errors while packages were definitely installed (locally). For more detail, my scripts are:
test.py:
import ssl
test.pbs:
#!/bin/csh
cd working_dir
python test.py
When I run python test.py, no errors displayed.
When I submit the PBS job qsub test.pbs, the job was started and ended with error:
Traceback (most recent call last):
File "test.py", line 1, in
import ssl
File "/home/s1310007/python/lib/python2.7/ssl.py", line 97, in
import _ssl # if we can't import it, let the error propagate
ImportError: /home/s1310007/python/lib/python2.7/lib-dynload/_ssl.so: symbol SSL_get0_next_proto_negotiated, version libssl.so.10 not defined in file libssl.so.10 with link time reference
I guess when I ran python test.py on the login node (I logged in using ssh), the environment paths were set correctly. When the PBS job was started, the different environment paths were set, then the dynamic symbol SSL_get0_next_proto_negotiated was not resolved. I tried to modify my .cshrc or .dtprofile, .modulesbeginenv; but nothing works. Any suggestions? Thanks!
I've deployed a Django application on Heroku. The application by itself works fine. I can run commands such as heroku run python project/manage.py syncdband heroku run python project/manage.py shell and this works well.
My Django project makes use of the Python web scraping library called Scrapy. Scrapy comes with a command called scrapy crawl abc which helps me scrape websites I have defined in the scrapy application. When I run a scrapy command such as scrapy crawl spidername on my local machine, the application is able to scrape date and copy it to my database. However when I run the same command on Heroku under a sub-directory of my project directory heroku run scrapy crawl spidername, nothing happens.
I don't see anything in the Heroku logs which can point to where I'm going wrong:
2012-01-26T15:45:38+00:00 heroku[run.1]: State changed from created to starting
2012-01-26T15:45:43+00:00 app[run.1]: Awaiting client
2012-01-26T15:45:43+00:00 app[run.1]: Starting process with command `project/spiderMainDir scrapy crawl spidername`
2012-01-26T15:45:44+00:00 heroku[run.1]: State changed from starting to up
2012-01-26T15:45:46+00:00 heroku[run.1]: State changed from up to complete
2012-01-26T15:45:46+00:00 heroku[run.1]: Process exited
Some additional information:
My scrapy app calls pipelines.py to save the scraped items to the database. In the pipelines.py file, this is what I've written to invoke the Django settings so that I can import my models and save data to the database from the scrapy application.
import os,sys
PROJECT_PATH = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
sys.path.append(PROJECT_PATH)
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
Any pointers on where exactly am I going wrong? How do I execute the scrapy command on Heroku such that my application can scrape an external website and save that data to the database. Isn't the way external commands are run in Heroku like - heroku run command?
I'm answering my own question because I discovered what the problem was. Heroku for some reason was not able to find scrapy when I executed the command from a sub-directory and not the top-level directory.
The command heroku run ... is generally run from the top-level directory. For my project which uses scrapy, I was required to go to a sub-directory and run the scrapy command from the sub-directory (this is how scrapy is designed). This wasn't working in Heroku. So I went to the Heroku bash by typing heroku run bash to see what was going on. When I ran the scrapy command from the top-level directory, Heroku recognized the command but when I went to a sub-directory, it failed to recognize the scrapy command. I suppose there is some problem related to the path. From the sub-directory, I had to specify the complete path to scrapy (~/bin/scrapy crawl spidername) to be able to execute it.
To run the scrapy command without going to the Heroku bash manually each time, my work around this problem was that I created a shell script containing the following code and put it under the bin directory of my top-level directory and pushed the changes to Heroku.
bin/scrapy.sh :
#!/usr/bin/env bash
cd ~/project/spiderSubDirectory
~/bin/scrapy $#
After this was done, I could execute $ heroku run scrapy.sh crawl spidername from my local bash. I suppose its not the best solution but this works.
Isn't the way external commands are run in Heroku like - heroku run
appdir command?
It's actually heroku run command. By including your appdir in there, it resulted in an invalid command. Heroku's output doesn't give useful error messages when these commands fail, and instead just tells you that the command finished which is what you're seeing. So for you, just change the command to something like:
heroku run scrapy crawl spidername