While trying to host my Django application on Cloud foundry using Gunicorn, my application will be hosted correctly on the URL, but when I see the logs by doing cf logs <app> --recent I see some errors:
2019-10-18T17:06:36.85+0530 [APP/PROC/WEB/0] ERR [2019-10-18 11:36:36 +0000] [9] [INFO] Starting gunicorn 19.9.0
2019-10-18T17:06:36.86+0530 [APP/PROC/WEB/0] ERR [2019-10-18 11:36:36 +0000] [9] [INFO] Listening at: http://0.0.0.0:8080 (9)
2019-10-18T17:06:36.86+0530 [APP/PROC/WEB/0] ERR [2019-10-18 11:36:36 +0000] [9] [INFO] Using worker: sync
2019-10-18T17:06:36.86+0530 [APP/PROC/WEB/0] ERR [2019-10-18 11:36:36 +0000] [68] [INFO] Booting worker with pid: 68
I want to know what are these errors. And, after these errors also the application is hosted and is up.
None of these are error messages. They are just messages that your application wrote to STDERR. That's all that [APP/PROC/WEB/0] ERR is telling you. Sometimes apps write error messages to STDERR, sometimes they write other things to STDERR. It just depends on the app.
The log messages look normal and expected. gunicorn is starting, listening on a socket and starting workers. Nothing wrong with that.
Hope that helps!
Related
Ive deployed a few heroku webapps. But for some reason, this one is giving me pain.
with
git push heroku main
my app starts, and is deployed. No issues whatsoever. But when I click the link, I get a this site cant be reached. ____ took too long to respond
Upon checking the logs, there seems to be no errors:
2022-09-01T17:46:08.937248+00:00 heroku[web.1]: State changed from down to starting
2022-09-01T17:46:12.101512+00:00 heroku[web.1]: Starting process with command `gunicorn wsgi:app`
2022-09-01T17:46:13.204452+00:00 app[web.1]: [2022-09-01 17:46:13 +0000] [4] [INFO] Starting gunicorn 20.1.0
2022-09-01T17:46:13.204789+00:00 app[web.1]: [2022-09-01 17:46:13 +0000] [4] [INFO] Listening at: http://0.0.0.0:42097 (4)
2022-09-01T17:46:13.204842+00:00 app[web.1]: [2022-09-01 17:46:13 +0000] [4] [INFO] Using worker: sync
2022-09-01T17:46:13.207772+00:00 app[web.1]: [2022-09-01 17:46:13 +0000] [9] [INFO] Booting worker with pid: 9
2022-09-01T17:46:13.268559+00:00 app[web.1]: [2022-09-01 17:46:13 +0000] [10] [INFO] Booting worker with pid: 10
2022-09-01T17:46:13.737219+00:00 heroku[web.1]: State changed from starting to up
What is going on?
It runs completely fine when I run the wsgi.py file locally. Im so confused why its taking too long to respond because It's just rendering an HTML template.
We have a Google Cloud run application that runs at 1am every morning to ingest, process, and output some data files. Currently, we have max concurrency, as timing is important.
Over the past weeks we have noticed several errors generated by the platform.
The Cloud Run logs themselves show Shutting down: Master
At a similar timestamp, a logging sink picks up
The request failed because either the HTTP response was malformed or connection to the instance had an error.
We have modified the instances and concurrency, as well as memory and vCPUs available, and they just show the same error but generated over a longer period of time.
There are no other errors or logs generated to give us any further info.
Any ideas where we may be going wrong?
Edit:
The full trace from the Cloud Run log is below
[INFO] Starting gunicorn 20.0.4
[1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[1] [INFO] Using worker: sync
[4] [INFO] Booting worker with pid: 4
[5] [INFO] Booting worker with pid: 5
[6] [INFO] Booting worker with pid: 6
[1] [INFO] Handling signal: term
[6] [INFO] Worker exiting (pid: 6)
[5] [INFO] Worker exiting (pid: 5)
[4] [INFO] Worker exiting (pid: 4)
[1] [INFO] Shutting down: Master
My situation was the same.
I sloved it by stop the firewall.
I am new to server and apps.
Currently, I have created an app on my aws instance.
gunicorn --threads 4 -b 0.0.0.0:5000 --access-logfile server.log --timeout 60 server:app
But I want to keep it running after I ssh the instance, how could I achieve this?
[2018-09-24 17:45:28 +0000] [8318] [INFO] Starting gunicorn 19.9.0
[2018-09-24 17:45:28 +0000] [8318] [INFO] Listening at:
http://0.0.0.0:5000 (8318)
[2018-09-24 17:45:28 +0000] [8318] [INFO] Using worker: threads
[2018-09-24 17:45:28 +0000] [8321] [INFO] Booting worker with pid: 8321
I have to use control+c to exit too at the moment.
add --daemon to your command line or use screen (https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn-and-nginx-on-ubuntu-14-04)
I have working django REST API docker image with following dependencies:
python 3.5.2, django 1.10.6, djangorestframework 3.6.2, gevent 1.2.2
In my dockerfile, port 5000 is exposed.
docker command:
/usr/local/bin/gunicorn --log-level=DEBUG --worker-class gevent --timeout=300 config.wsgi -w 4 -b :5000
In the ECS task definition, 5000 container port is forwarded to port 80 of the host. The security group has an inbound rule allowing everyone at port 80.
When I ran the ECS task with this ECS task definition, following are the application logs, which seem fine.
[2017-09-13 16:45:34 +0000] [9] [INFO] Starting gunicorn 19.6.0
[2017-09-13 16:45:34 +0000] [9] [INFO] Listening at: http://0.0.0.0:5000 (9)
[2017-09-13 16:45:34 +0000] [9] [INFO] Using worker: gevent
[2017-09-13 16:45:34 +0000] [12] [INFO] Booting worker with pid: 12
[2017-09-13 16:45:34 +0000] [13] [INFO] Booting worker with pid: 13
[2017-09-13 16:45:35 +0000] [15] [INFO] Booting worker with pid: 15
[2017-09-13 16:45:35 +0000] [16] [INFO] Booting worker with pid: 16
But I am unable to access the service endpoints using the EC2 instance's public IP/Public DNS address.
I tried to get into the running container and curl the application url curl localhost:5000. Following are the logs that I see (the connections are closed)
[2017-09-13 17:42:42 +0000] [14] [DEBUG] GET /
[2017-09-13 17:42:42 +0000] [14] [DEBUG] Closing connection.
[2017-09-13 17:42:56 +0000] [12] [DEBUG] GET /
[2017-09-13 17:42:56 +0000] [12] [DEBUG] Closing connection.
[2017-09-13 17:53:20 +0000] [14] [DEBUG] GET /users/get_mfatype/
[2017-09-13 17:53:20 +0000] [14] [DEBUG] Closing connection.
The same docker image is working as expected when I run locally. I even tried running the same docker image inside EC2 instance, which is working fine.
I am not able to find the root cause why the application is not running as ECS task.
Am I missing anything?
I have Django app using StreamingHttpResponse that fails when gunicorn worker times out. Basically extending timeout is not an option as streaming can take longer as it depends on network speed. Web server won't time out as is actually doing something but gunicorn workers seems to not recognize it.
I am aware of a choice between sync and async workers supported by gunicorn and using for example gevent.
it starts:
gunicorn -D -p /path/to/django.pid --bind 127.0.0.1:8000 --workers 2 -k gevent --worker-connections 10 --max-requests 100 myapp.wsgi:application
gunicorn log:
[2016-01-21 15:12:34 +0000] [30333] [INFO] Listening at: ...
[2016-01-21 15:12:34 +0000] [30333] [INFO] Using worker: gevent
[2016-01-21 15:14:22 +0000] [30338] [DEBUG] GET /url/1/
[2016-01-21 15:14:22 +0000] [30338] [DEBUG] Closing connection.
[2016-01-21 15:14:24 +0000] [30343] [DEBUG] GET /download/1/
...
[2016-01-21 15:15:43 +0000] [30333] [CRITICAL] WORKER TIMEOUT (pid:30343)
[2016-01-21 15:15:43 +0000] [30343] [DEBUG] Closing connection.
[2016-01-21 15:15:43 +0000] [30343] [INFO] Worker exiting (pid: 30343)
[2016-01-21 15:15:43 +0000] [31203] [INFO] Booting worker with pid: 31203
nginx log
2016/01/21 15:15:43 [error] 23160#0: *10849 upstream prematurely closed connection while reading upstream...
Why deploying the same app using fastcgi and flup never exposed that problem? Anyone could advice?