Django Production Site keeps going down after 3 weeks - django

I am running a website using Django, Gunicorn and Nginx off a Digital Ocean droplet. The website runs fine for about three weeks and then it random stops connecting users to the webpage. When I visit the webpage in my browser it returns:
This site can’t be reached ERR_CONNECTION_RESET
Restarting the droplet fixes the issue, but it will likely happen again after three weeks.
When I check the Gunicorn worker it says its running fine and my Django logs are clean.
Nginx only reported this:
2019/08/31 23:11:56 [error] 28183#28183: *36352 open() "/home/projects/server/mysite/static/img/icon.jpg" failed (2: No such file or directory), client: 66.249.64.149, server: removedurl.com, request: "GET /static/img/icon.jpg HTTP/1.1", host: "removedurl.com.com"
It seems that restarting Nginx fixes the problem.
EDIT:
I just remembered that I have a crontab set up that renews the Letsencrypt SSL certificate, which could be the problem. Here is the crontab command:
0 0 1 * * /etc/init.d/nginx stop && /opt/letsencrypt/letsencrypt-auto renew && /etc/init.d/nginx restart
EDIT2:
The command above is not a great way to go about this, instead I deleted my original letencrypt certification and used the Nginx webserver plugin for Certbot that allowed me to renew the certification through my existing webserver instead of letsencrypt --standalone trying to start up a new webserver on port 80 (which it couldn't)

Related

Nginx (13: Permission denied) while connecting to upstream

I'm deploying my Djano application on a VPS and I'm following the steps in the below link to configure my app with Gunicorn and Nginx.
How To Set Up Django with Postgres, Nginx, and Gunicorn on Ubuntu 16.04
Everything went well with the tutorial (gunicorn and nginx are running) but the issue is that when Im' visiting the VPS through the static IP its showing a white screen that is always reloading.
After checking nginx log I found the following:
(13: Permission denied) while connecting to upstream, client: <client_ip>, server: <server_ip>, request: "GET / HTTP/1.1, upstream: "http://unix:/root/myproject/myproject.sock:/", host: "<server_ip>", referrer: "http://<server_ip>/"
After searching for roughly 7 hours, I was finally able to find a solution to this issue in the Nginx forum:
Nginx connet to .sock failed (13:Permission denied) - 502 bad gateway
What I simply did was changing the name of the user on the first line in /etc/nginx/nginx.conf file.
In my case the default user was www-data and I changed it to my root machine username.
In the top of nginx.conf file is a user name (user nginx;). just add this user in same group that your site or project is. www-data or any is yours. sorry for english.

502 Bad Gateway: nginx - Log: upstream prematurely closed connection while reading... [Django 2.1 | gunicorn 19.7.1 | Google Cloud Platform]

I'm new in a company that have this project on Google Cloud PLatform, that I've never used. In my first day I've got this 502 Bad Gateway error. Lookin the log on the Google Cloud Platform I've got the following:
[error] 33#33: *285 upstream prematurely closed connection while reading response header from upstream, client: 172.217.172.212, server: , request: "POST /product/fast_appraisal/result/ HTTP/1.1", upstream: "http://172.17.0.1:8080/product/fast_appraisal/result/", host: "avalieidjango.appspot.com", referrer: "https://avalieidjango.appspot.com/product/fast_appraisal/search/"
I've tried to edit the app.yaml adding --timeout and --graceful-timeout parameters to it like the following:
# [START runtime]
runtime: python
env: flex
entrypoint: gunicorn -b :$PORT mysite.wsgi --timeout=90 --graceful-timeout=10
beta_settings:
cloud_sql_instances: avalieidjango:southamerica-east1:avaliei
runtime_config:
python_version: 3
handlers:
- url: /.*
script: manage.py
secure: always
redirect_http_response_code: 301
# [END runtime]
In the settings.py file DEBUG variable is setted to False
Looking for answers on the internet I found a few cases, but no one looks like mine exactly.
Locally I'm running the project on Windows 7, so the error only occurs when I deploy it to GCP. I'm new on GCP and gunicorn.
Edit
After these days I've passed through a lot of forums, and added a few new configurations to my app.yaml trying to work with threads and workers to solve the question.
The entrypoint line looks like this:
entrypoint: gunicorn -b :$PORT --worker-class=gevent --worker-connections=1000 --workers=3 mysite.wsgi --timeout 90
This project consists in search a Postgres database on GCP gathering information about properties and running an AI to show some predictions about its values.
I've tried threads and process, but even with just my requests the application is too slow, even a simple page take some time to render.
Local tests run better, but in production it isn't working at all.
The AI wasn't developed for me and it uses a large joblib file.
The project doesn't use containers like Docker. Maybe it could help in some way if I "dockerize" the project?
I've stoped seeing this error changing CONN_MAX_AGE value to None, which keeps the database connetion time undefined. However this may cause some security issues that must be evaluated before deploy your application. If you change it, stay tuned on Google Cloud Logs looking for strange connection attempts.

*10 upstream timed out (110: Connection timed out) while reading response header from upstream with uwsgi

I currently have a server setup with nginx and uwsgi with django
This error doesn't happen until I try to change my rds instance
my fully error message is
*10 upstream timed out (110: Connection timed out) while reading response header from upstream, client: xxx.xxx.xxx.xxx, server: xxx.xxx.xxx.xxx, request: "GET /load/ HTTP/1.1", upstream: "uwsgi://unix:/tmp/load.sock", host: "example.com", referrer: "https://example.com/"
I was using aws rds (postgres) which works perfectly fine. The only change I made is changing from regular postgres service to aurora postgres I didn't upgrade the db, from regular to aurora. I created a new aurora postgres. I got everything setup...changed host and everything in my django db setting. runserver locally works fine. It does connect to db with read and write. Works perfectly. But when I deploy to server, open up my domain. Anything ui related looks fine but db related, NO. Took awhile then of course the 504 gateway timeout. I went to checkout the nginx error log. That's the error message I found. Googled, tried a few settings other stackoverflow posts suggested such as addingsingle-interpreter = true into uwsgi.ini file. No luck.
Can someone please give me an idea where I should look into for this problem?
Thanks in advance.
try going to your rds instance, check its' security group setting. Happened to me once, too me a while to find out that the security group setting is the problem. I didn't recall setting up the security group but it restricted with local IP

AWS hosted site not working

I have hosted my site using amazon web services, and the site was running fine for past 30 days. after 30days we can't able to get the site. Its went down, I don't what the major problem in that.
while loading in chrome its showing "This site can’t be reached" Error : ERR_CONNECTION_RESET
Please suggest me any solution regarding handling the aws regarding this problem, I'm new to aws.
Try restarting the Apache2 webserver:
For ubuntu
service apache2 restart
For linux
service httpd restart
If you receive a message that the restart has failed, identify which process is currently using your port:
netstat -tulpn | grep :80
This will output something like
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1607/apache2
The 'apache2' in this case is the name of the process using port 80.
'1607' here is the PID(process id), to shut it down execute
kill 1607
After this, try starting the service again
service httpd start #linux
service apache2 start #ubuntu
If this still doesn't work, try restarting the machine.

502 error after adding application to a Django project running on nginx and gunicorn

I am trying to add an application to an existing Django project, but once I have done it I get a 502 error.The server is running Ubuntu. I don't think it has to do with the applications code because I got it running on the django development server. It goes away when I take out the app's name from settings.py and restart gunicorn.
Here's a part of the log
2011/07/15 01:24:45 [error] 16136#0: *75593 connect() failed (111: Connection refused) while connecting to upstream, client: 24.17.8.152, server: staging.site.org, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8020/", host: "staging.site.org"
Here's the nginx config file.
Nginx Config File
I'm not sure what other information is needed. Not sure where the gunicorn logs are located. My server admin skills are kind of lacking.
Nginx isn't able to connect to your backend (gunicorn) or gunicorn is refusing the connection. You provided no details about the configuration so that's all the help you'll get. You are correct that the application code has nothing to do with it. It's a configuration error on your part.