Fully empty logs using Gunicorn, Nginx and Django - django

I have the following Django API set up with Nginx and Gunicorn and all my logs except one are completely empty. I have never set up logs before. I was expecting to get the normal server output of any 500 errors and such like I get on local in my terminal:
nano /etc/nginx/sites-available/stocks_backend
server {
...
access_log /root/stocks_backend/log/nginx-access.log;
error_log /root/stocks_backend/log/nginx-error.log;
}
/etc/supervisor/conf.d/stocks
[program:stocks]
user=root
directory=/root/stocks_backend
command=/root/stocks_backend/stocks-env/bin/gunicorn --workers 3 --bind unix:st>
autostart=true
autorestart=true
stdout_logfile=/root/stocks_backend/log/gunicorn.log
stderr_logfile=/root/stocks_backend/log/gunicorn.err
redirect_stderr=true
/etc/systemd/system/gunicorn.service
[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target
[Service]
User=root
Group=root
WorkingDirectory=/root/stocks_backend
ExecStart=/root/stocks_backend/stocks-env/bin/gunicorn \
--access-logfile - \
--workers 3 \
--bind unix:/run/gunicorn.sock \
myproject.wsgi:application
[Install]
WantedBy=multi-user.target
3 of the 4 logs are empty:
root#-droplet:~/stocks_backend# cat log/nginx-error.log
root#-droplet:~/stocks_backend# cat log/nginx-access.log
root#-droplet:~/stocks_backend# cat log/gunicorn.err
[2022-08-25 21:27:39 +0000] [61321] [INFO] Starting gunicorn 20.1.0
[2022-08-25 21:27:39 +0000] [61321] [INFO] Listening at: unix:stocks_backend.sock (61321)
[2022-08-25 21:27:39 +0000] [61321] [INFO] Using worker: sync
....
[2022-08-27 07:27:06 +0000] [67233] [INFO] Shutting down: Master
[2022-08-27 07:27:08 +0000] [69840] [INFO] Starting gunicorn 20.1.0
[2022-08-27 07:27:08 +0000] [69840] [INFO] Listening at: unix:stocks_backend.sock (69840)
[2022-08-27 07:27:08 +0000] [69840] [INFO] Using worker: sync
[2022-08-27 07:27:08 +0000] [69842] [INFO] Booting worker with pid: 69842
[2022-08-27 07:27:08 +0000] [69843] [INFO] Booting worker with pid: 69843
[2022-08-27 07:27:08 +0000] [69844] [INFO] Booting worker with pid: 69844
root#-droplet:~/stocks_backend# cat log/gunicorn.log
root#-droplet:~/stocks_backend#
The only log which has anything is the gunicorn-err which shows Gunicorn starting and stopping but does not have any prints or server errors.
I have logger in use in the view with the errors:
import logging
logger = logging.getLogger(__name__)
I am getting a 500 error on every request but I cannot see the error anywhere to debug. How can I get logs to write with this setup?
I tried the Django docs example to write to a file but it is not working at all, the log files are empty:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'file': {
'level': 'INFO',
'class': 'logging.FileHandler',
'filename': '/root/stocks_backend/log/django.logs',
},
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'INFO',
'propagate': True,
},
},
}

Related

failed to find 'app' attribute while using gunicorn

I have two files app.py which is in root and the init.py inside a directory named website.
error log:
(vidscript) root#localhost:~# gunicorn --workers 3 --bind 0.0.0.0:8000 vidscript:app
[2023-02-06 17:01:56 +0000] [5272] [INFO] Starting gunicorn 20.1.0
[2023-02-06 17:01:56 +0000] [5272] [INFO] Listening at: http://0.0.0.0:8000 (5272)
[2023-02-06 17:01:56 +0000] [5272] [INFO] Using worker: sync
[2023-02-06 17:01:56 +0000] [5273] [INFO] Booting worker with pid: 5273
Failed to find attribute 'app' in 'vidscript'.
app.py:
from website import create_app
if __name__ == "__main__":
app = create_app()
app.run()
I expected to run the home page while it actually wokers in development server mode using. python app.py

gunicorn Failed to find attribute 'app' in 'app'

I am hoping someone can help. I have been trying to launch my flask app on ubuntu. The app runs on dev mode, but when I try to run it with gunicorn, I get the following error:
[2022-07-24 21:55:44 +0000] [3710] [INFO] Starting gunicorn 20.1.0
[2022-07-24 21:55:44 +0000] [3710] [INFO] Listening at: http://127.0.0.1:8000 (3710)
[2022-07-24 21:55:44 +0000] [3710] [INFO] Using worker: sync
[2022-07-24 21:55:44 +0000] [3711] [INFO] Booting worker with pid: 3711
Failed to find attribute 'app' in 'app'.
[2022-07-24 21:55:44 +0000] [3711] [INFO] Worker exiting (pid: 3711)
[2022-07-24 21:55:44 +0000] [3710] [INFO] Shutting down: Master
[2022-07-24 21:55:44 +0000] [3710] [INFO] Reason: App failed to load.
Here is my app.py file
from website import create_app
if __name__ == "__main__":
app = create_app()
app.run(debug=True)
Here the site structure.
Website
script
static
-templates
-apy.py
-auth.py
-db
-models
-views
-innit.py
app.py
Can someone help?
Thanks
S
I found the issue. Had to move the app=create_app above the 'if' statement.

Start CKAN 2.9.X with Gunicorn?

Is there a way to start CKAN 2.9 with gunicorn?
I can see from the documentation that gunicorn is mentioned:
Install Supervisor (a Process Control System) used to control
starting, stopping the uwsgi or gunicorn servers:
But not much information is in there.
Since 2.9 is flask I was looking at how to serve a flask application with Gunicorn an I came across this post: How To Serve Flask Applications with Gunicorn and Nginx on Ubuntu 18.04 and I tried by using this code (wsgi.py):
import os
from ckan.config.middleware import make_app
from ckan.cli import CKANConfigLoader
from logging.config import fileConfig as loggingFileConfig
app = None
if __name__ == "__main__":
config_filepath = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'ckan.ini')
abspath = os.path.join(os.path.dirname(os.path.abspath(__file__)))
loggingFileConfig(config_filepath)
config = CKANConfigLoader(config_filepath).get_config()
app = make_app(config)
Then running:
gunicorn --bind 0.0.0.0:5000 wsgi:app
But I get:
[2022-06-22 20:09:34 +0000] [365] [INFO] Starting gunicorn 20.1.0
[2022-06-22 20:09:34 +0000] [365] [INFO] Listening at: http://0.0.0.0:5000 (365)
[2022-06-22 20:09:34 +0000] [365] [INFO] Using worker: sync
[2022-06-22 20:09:34 +0000] [367] [INFO] Booting worker with pid: 367
Failed to find application object: 'app'
[2022-06-22 20:09:35 +0000] [367] [INFO] Worker exiting (pid: 367)
[2022-06-22 20:09:35 +0000] [365] [INFO] Shutting down: Master
[2022-06-22 20:09:35 +0000] [365] [INFO] Reason: App failed to load.
Any idea is appreciated

gunicorn with --access-logfile enabled throws "TypeError: format requires a mapping"

I am running a flask application on python 2.6, gunicorn 19.9.0. When access-logfile is enabled I get a logging error. Below is the full execution/stacktrace. I really would like to have my access logs :(.
/usr/bin/gunicorn --log-level=debug --access-logfile=- --error-logfile=- -w 4 -b 0.0.0.0:8000 api:application
[2018-07-10 22:23:56 +0000] [32082] [DEBUG] Current configuration:
...
accesslog: -
disable_redirect_access_to_syslog: False
statsd_host: None
...
errorlog: -
enable_stdio_inheritance: False
worker_class: sync
...
logconfig: None
logconfig_dict: {}
config: None
access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
....
loglevel: debug
syslog_addr: udp://localhost:514
syslog: False
syslog_prefix: None
.....
logger_class: gunicorn.glogging.Logger
[2018-07-10 22:23:56 +0000] [32082] [INFO] Starting gunicorn 19.9.0
[2018-07-10 22:23:56 +0000] [32082] [DEBUG] Arbiter booted
[2018-07-10 22:23:56 +0000] [32082] [INFO] Listening at: http://0.0.0.0:8000 (32082)
[2018-07-10 22:23:56 +0000] [32082] [INFO] Using worker: sync
[2018-07-10 22:23:56 +0000] [32087] [INFO] Booting worker with pid: 32087
[2018-07-10 22:23:56 +0000] [32092] [INFO] Booting worker with pid: 32092
[2018-07-10 22:23:56 +0000] [32097] [INFO] Booting worker with pid: 32097
[2018-07-10 22:23:56 +0000] [32102] [INFO] Booting worker with pid: 32102
[2018-07-10 22:23:56 +0000] [32082] [DEBUG] 4 workers
[2018-07-10 22:24:02 +0000] [32092] [DEBUG] GET /
Traceback (most recent call last):
File "/usr/lib64/python2.6/logging/__init__.py", line 784, in emit
msg = self.format(record)
File "/usr/lib64/python2.6/logging/__init__.py", line 662, in format
return fmt.format(record)
File "/usr/lib64/python2.6/logging/__init__.py", line 444, in format
record.message = record.getMessage()
File "/usr/lib64/python2.6/logging/__init__.py", line 314, in getMessage
msg = msg % self.args
TypeError: format requires a mapping
[2018-07-10 22:24:13 +0000] [32087] [DEBUG] Closing connection.

nginx+gunicorn+django+aws bad request

I was following a tutorial at https://ashokfernandez.wordpress.com/2014/03/11/deploying-a-django-app-to-amazon-aws-with-nginx-gunicorn-git/
on deploying a django application. This is my current situation:
fab spawn instance created the aws instance with nginx and gunicorn installed but when I tried accessing the site on that machine I got a 400 Bad Request. I checked the nginx-error log but that was empty and the nginx-access log showed that it had received the requests. The supervisor log had the following:
[2015-01-31 21:26:20 +0000] [15823] [INFO] Starting gunicorn 19.2.0
[2015-01-31 21:26:20 +0000] [15823] [INFO] Listening at: http://127.0.0.1:8002/ (15823)
[2015-01-31 21:26:20 +0000] [15823] [INFO] Using worker: sync
[2015-01-31 21:26:20 +0000] [15832] [INFO] Booting worker with pid: 15832
[2015-01-31 21:26:20 +0000] [15833] [INFO] Booting worker with pid: 15833
[2015-01-31 21:26:20 +0000] [15834] [INFO] Booting worker with pid: 15834
[2015-01-31 21:26:20 +0000] [15835] [INFO] Booting worker with pid: 15835
[2015-01-31 21:26:20 +0000] [15836] [INFO] Booting worker with pid: 15836
[2015-01-31 21:26:31 +0000] [15837] [INFO] Starting gunicorn 19.2.0
[2015-01-31 21:26:31 +0000] [15837] [ERROR] Connection in use: (‘127.0.0.1′, 8002)
[2015-01-31 21:26:31 +0000] [15837] [ERROR] Retrying in 1 second.
[2015-01-31 21:26:32 +0000] [15837] [ERROR] Connection in use: (‘127.0.0.1′, 8002)
[2015-01-31 21:26:32 +0000] [15837] [ERROR] Retrying in 1 second.
[2015-01-31 21:26:33 +0000] [15837] [ERROR] Connection in use: (‘127.0.0.1′, 8002)
[2015-01-31 21:26:33 +0000] [15837] [ERROR] Retrying in 1 second.
[2015-01-31 21:26:34 +0000] [15837] [ERROR] Connection in use: (‘127.0.0.1′, 8002)
[2015-01-31 21:26:34 +0000] [15837] [ERROR] Retrying in 1 second.
[2015-01-31 21:26:35 +0000] [15837] [ERROR] Connection in use: (‘127.0.0.1′, 8002)
[2015-01-31 21:26:35 +0000] [15837] [ERROR] Retrying in 1 second.
[2015-01-31 21:26:36 +0000] [15837] [ERROR] Can’t connect to (‘127.0.0.1′, 8002)
[2015-01-31 21:26:37 +0000] [15846] [INFO] Starting gunicorn 19.2.0
[2015-01-31 21:26:37 +0000] [15846] [INFO] Listening at: http://127.0.0.1:8002 (15846)
[2015-01-31 21:26:37 +0000] [15846] [INFO] Using worker: sync
[2015-01-31 21:26:37 +0000] [15855] [INFO] Booting worker with pid: 15855
[2015-01-31 21:26:37 +0000] [15856] [INFO] Booting worker with pid: 15856
[2015-01-31 21:26:37 +0000] [15857] [INFO] Booting worker with pid: 15857
[2015-01-31 21:26:38 +0000] [15858] [INFO] Booting worker with pid: 15858
[2015-01-31 21:26:38 +0000] [15859] [INFO] Booting worker with pid: 15859
I changed ALLOWED_HOSTS from [] to ["*"] and then '*'
When I changed it to the string, I got "The requested URL / was not found on this server." On other instances, I got 400 bad request.
This is the first time I am deploying a django app on nginx and I can’t figure out what the problem might be. Could you please help me debug this error?
Thanks in advance!!
PS: Please let me know if I need to post any config files. So far I have just followed the tutorial and I have not changed any configurations.
UPDATE:
common.py
import os
BASE_DIR = os.path.dirname(os.path.dirname(__file__))
SECRET_KEY = 'blah'
ALLOWED_HOSTS = []
# Application definition
INSTALLED_APPS = (
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
)
MIDDLEWARE_CLASSES = (
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
)
ROOT_URLCONF = 'mosaic.urls'
WSGI_APPLICATION = 'mosaic.wsgi.application'
# Internationalization
# https://docs.djangoproject.com/en/1.7/topics/i18n/
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
prod.py
from mosaic.settings.common import *
DEBUG = True #Change this after the project starts working
TEMPLATE_DEBUG = True
ALLOWED_HOSTS = []
# Database
# https://docs.djangoproject.com/en/1.6/ref/settings/#databases
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'mosaic',
'USER' : 'adminuser',
'PASSWORD' : '****',
'HOST' : 'abcd.cjcgmgnogsvc.us-west-1.rds.amazonaws.com',
'PORT' : '5432',
}
}
# Static files via Amazon S3 (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.6/howto/static-files/
INSTALLED_APPS += ('storages',)
AWS_STORAGE_BUCKET_NAME = "xyz"
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
S3_URL = 'http://%s.s3.amazonaws.com/' % AWS_STORAGE_BUCKET_NAME
STATIC_URL = S3_URL
Also in my project config files I have
fabconf['DOMAINS'] = ""
I thought this might be relevant as well
The allowed hosts has to match the host names you'll be using. So put the ALLOWED_HOSTS = [] in a settings file specific to your environment (Dev or production). On the dev side set it to ALLOWED_HOSTS = ['localhost', '127.0.0.1'] and on production set it go your domain & server IPs.
From the docs:
Values in this list can be fully qualified names (e.g. 'www.example.com'), in which case they will be matched against the request’s Host header exactly (case-insensitive, not including port). A value beginning with a period can be used as a subdomain wildcard: '.example.com' will match example.com, www.example.com, and any other subdomain of example.com. A value of '*' will match anything; in this case you are responsible to provide your own validation of the Host header (perhaps in a middleware; if so this middleware must be listed first in MIDDLEWARE_CLASSES).
The status 400 response you get is due to a SuspiciousOperation exception being raised when your host header doesn't match any values in that list.