Solution
Always make sure you reserve your IPs when using a Static IP
Versions
VirtualBox Version: 6.0.0 ( I think )
Vagrant Version: 2.2.3
CentosBox: "centos/7"
Nginx Version: 1.16.1
uWSGI Version: 2.0.18
Django Version: 2.2.1
Background
I have two vagrant boxes running, a test and a production. The only difference is IP and core count. I've set up both so I can ssh directly into the boxes, instead of having to ssh into the host machine and then run 'vagrant ssh'
General Issue
The production version will randomly boot me out of the ssh (Connection reset by IP port 22) and then i'll get Connection Refused. If I ssh into the Host machine and then 'vagrant ssh' I can still get in and everything seems to be fine, I can even still ping other computers on the network. But I can't access it from outside the host, this goes for the nginx server as well (IP refused to connect.) on chrome
The issue will occasionally fix itself in a couple minutes, but the majority of the time requires a 'vagrant destroy' and 'vagrant up --provision' / recreate the box. I also occasionally get booted out of the Host Machine and well as the test box, but both I can still access externally after (even the nginx server on test) I'm working over a VPN and I also occasionally get booted out of that as well, but i can reconnect when I notice
VagrantFile
# -*- mode: ruby -*-
# vi: set ft=ruby :
# Please don't change it unless you know what you're doing.
Vagrant.configure("2") do |config|
config.vm.box = "centos/7"
config.vm.hostname = "DjangoProduction"
# Disable automatic box update checking. If you disable this, then
# boxes will only be checked for updates when the user runs
# `vagrant box outdated`. This is not recommended.
# config.vm.box_check_update = false
# Create a public network, which generally matched to bridged network.
# Bridged networks make the machine appear as another physical device on
# your network.
config.vm.network "public_network", ip: "IP"
# Share an additional folder to the guest VM. The first argument is
# the path on the host to the actual folder. The second argument is
# the path on the guest to mount the folder. And the optional third
# argument is a set of non-required options.
config.vm.synced_folder "./", "D:/abcd", type: "sshfs", group:'vagrant', owner:'vagrant'
# Provider-specific configuration so you can fine-tune various
# backing providers for Vagrant. These expose provider-specific options.
# Example for VirtualBox:
#
config.vm.provider "virtualbox" do |v|
v.name = "DjangoProduction"
# test has these two commented out
v.memory = 6000
v.cpus = 4
end
#
# View the documentation for the provider you are using for more
# information on available options.
## Keys
### For SSH directly into the Box
# Work Laptop Key
config.vm.provision "file", source: ".provision/keys/work.pub", destination: "~/.ssh/work.pub"
config.vm.provision "shell", inline: "cat ~vagrant/.ssh/work.pub >> ~vagrant/.ssh/authorized_keys"
# Personal Laptop Key
config.vm.provision "file", source: ".provision/keys/msi.pub", destination: "~/.ssh/msi.pub"
config.vm.provision "shell", inline: "cat ~vagrant/.ssh/msi.pub >> ~vagrant/.ssh/authorized_keys"
##
required_plugins = %w( vagrant-sshfs )
required_plugins.each do |plugin|
exec "vagrant plugin install #{plugin};vagrant #{ARGV.join(" ")}" unless Vagrant.has_plugin? plugin || ARGV[0] == 'plugin'
end
# Enable provisioning with a shell script. Additional provisioners such as
# Puppet, Chef, Ansible, Salt, and Docker are also available. Please see the
# documentation for more information about their specific syntax and use.
config.vm.provision :shell, path: ".provision/boot.sh"
end
boot.sh
# networking
sudo yum -y install net-tools
ifconfig eth1 IP netmask 255.255.252.0
route add -net 10.1.0.0 netmask 255.255.252.0 dev eth1
route add default gw 10.1.0.1
# I manually set the gateway so It can be accessed through VPN
## install, reqs + drop things to places - gonna leave all that out
Error messages
Django
This issue starting popping up earlier this week with django sending me error emails saying. it's always random URLs there's no consistency
OperationalError at /
(2003, 'Can\'t connect to MySQL server on \'external-ip\' (110 "Connection timed out")')
I used to get this email once every other day and paid it no attention, but currently it's sending me at least 20 a day and the site is almost unusable- it's either really slow or I get chrome errors: 'ERR_CONNECTION_TIMED_OUT' or 'ERR_CONNECTION_REFUSED' or 'ERR_CONNECTION_RESET' .. it will be fine for an hour and then everything hits the fan
I originally thought it was an issue with the db or uwsgi or django, but working with it yesterday I realized there was a correlation with the timed out and getting kicked out of ssh.
Nginx Server Settings ( I have't changed nginx.conf )
upstream django {
server unix:///vagrant/abcd.sock;
}
server{
listen 8080;
return 301 https://$host$request_uri;
}
server{
charset utf-8;
listen 443 ssl;
ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt;
ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key;
location / {
uwsgi_pass django;
include /vagrant/project/uwsgi_params;
uwsgi_read_timeout 3600;
uwsgi_ignore_client_abort on;
}
location /static {
alias /vagrant/static;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /vagrant/templates/core;
}
}
UWSGI command used
uwsgi --socket abcd.sock --module project.wsgi --chmod-socket=664 --master --processes 8 --threads 4 --buffer-size=65535 --lazy
Nginx Error Logs
Nothing.
Messages file
only shows the '(110 "Connection timed out")' dump when it happens
Can you test the behaviour but commenting the line "config.vm.synced_folder..."?
Related
I and my team of developers have Vagrant machines running on our Windows 10 computers that have local file systems which contain our scripts to run a developer version of our web application.
We connect to an AWS RDS DB, but the connection is extremely slow to the database.
If we use our database IDE (dbforge) and run SQL queries, then they run fast.
Is there anything we can do to our vagrant boxes or setup that would speed this connection up?
Here are the contents of our Vagrantfile
# -*- mode: ruby -*-
# vi: set ft=ruby :
# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing. Vagrant.configure("2") do |config|
# The most common configuration options are documented and commented below.
# For acomplete reference, please see the online documentation at
# https://docs.vagrantup.com.
# Every Vagrant development environment requires a box. You can search for
# boxes at https://vagrantcloud.com/search.
config.vm.box = "hcp" config.vm.network "private_network", ip:
"10.23.45.30" config.vm.synced_folder "etc/", "/etc/hcp"
config.vm.synced_folder "html/", "/var/www/html"
config.vm.synced_folder "html_data/", "/var/www/html/_data"
# Disable automatic box update checking. If you disable this, then
# boxes will only be checked for updates when the user runs
# `vagrant box outdated`. This is not recommended.
#config.vm.box_check_update = false
# Create a forwarded port mapping which allows access to a specific port
# within the machine from a port on the host machine. In the example below,
# accessing "localhost:8080" will access port 80 on the guest machine.
# NOTE: This will enable public access to the opened port
# config.vm.network "forwarded_port", guest: 80, host:8080
# Create a forwarded port mapping which allows access to a specific port
# within the machine from a port on the host machine and only allow access
# via 127.0.0.1 to disable public access
# config.vm.network "forwarded_port", guest: 80, host: 8080, host_ip: "127.0.0.1"
# Create a private network, which allows host-only access to the machine
# using a specific IP.
# config.vm.network "private_network", ip: "192.168.33.10"
# Create a public network, which generally matched to bridged network.
# Bridged networks make the machine appear as another physical device on
# your network.
# config.vm.network "public_network"
# Share an additional folder to the guest VM. The first argument is
# the path on the host to the actual folder. The second argument is
# the path on the guest to mount the folder. And the optional third
# argument is a set of non-required options.
# config.vm.synced_folder "../data", "/vagrant_data"
# Provider-specific configuration so you can fine-tune various
# backing providers for Vagrant. These expose provider-specific options.
# Example for VirtualBox:
#
# config.vm.provider "virtualbox" do |vb|
#
# Display the VirtualBox GUI when booting the machine
# vb.gui = true
#
#
# Customize the amount of memory on the VM:
# vb.memory = "1024"
# end
#
# View the documentation for the provider you are using for more
# information on available options.
# Enable provisioning with a shell script. Additional provisioners such as
# Ansible, Chef, Docker, Puppet and Salt are also available. Please see the
# documentation for more information about their specific syntax and use.
# config.vm.provision "shell", inline:<<-SHELL
# apt-get update
# apt-get install -y apache2
#
SHELL end
trying to run django app "mysite" through uwsgi with nginx on Ubuntu 16.04, but when I start uwsgi and check in my browser, it just hangs.
i set django upstream socket to on port 8002 and nginx to listen on 8003. In the browser i visit 192.168.0.17:8003 prior to running uwsgi and it throws 502 which is expected, so I start uwsgi with
uwsgi --http :8002 --module mysite.wsgi --logto /tmp/uwsgi.log --master
and 8003 now hangs when I reload in the browser. I looked through /var/log/nginx/error.log but it's blank (so is access.log).
Here is nginx config, which is symlinked to /etc/nginx/sites-enabled:
sudo nano /etc/nginx/sites-available/mysite_nginx.conf
# mysite_nginx.conf
# the upstream component nginx needs to connect to
upstream django {
# server unix:///path/to/your/mysite/mysite.sock; # for a file socket
server 127.0.0.1:8002; # for a web port socket (we'll use this first)
}
# configuration of the server
server {
# the port your site will be served on
listen 8003;
# the domain name it will serve for
server_name 192.168.0.17; # substitute your machine's IP address or FQDN
charset utf-8;
# max upload size
client_max_body_size 75M; # adjust to taste
# Django media
location /media {
alias /home/myusername/uwsgi-tutorial/mysite/media; # your Django project's media files - amend as required
}
location /static {
alias /home/myusername/uwsgi-tutorial/mysite/static; # your Django project's static files - amend as required
}
# Finally, send all non-media requests to the Django server.
location / {
uwsgi_pass django;
include /home/myusername/uwsgi-tutorial/mysite/uwsgi_params; # the uwsgi_params file you installed
}
}
I know that Django is running because in my app's settings.py I have ALLOWED_HOSTS = ['192.168.0.17','localhost','127.0.0.1'] and when I visit port 8002 in the browser I get the django "Congratulations!" page. And when I remove 192.168.0.17 from ALLOWED_HOSTS, django still runs on that machine from localhost or 127.0.0.1, so this seems that it must be something to do with how ngnix and uwsgi are talking to each other.
Any ideas??
It turns out systemd does not like lines in config files to be too long. I removed a couple long comments in /etc/systemd/system/uwsgi.service, restarted uwgsi service and all is well.
I found this out by running sudo journalctl -u uwsgi and finding the following error:
[/etc/systemd/system/uwsgi.service:5] Unbalanced quoting, ignoring: "/bin/bash -c 'mkdir -p /run/uwsgi; chown myusername:myusern
In researching Unbalanced quoting, found this git issue regarding maximum file line length.
I'm trying to set up my Django app with uWSGI and nginx by following this guide. I'm able to run my app with Django's development server, as well as being served directly from uWSGI.
I'm running everything on a university managed Ubuntu 16.04 virtual machine, and my user has sudo access.
My problem:
When getting to this bit of the tutorial, and try to fetch an image, I get a 403 error from nginx.
The next section results in a 502.
/var/log/nginx/error.log shows
connect() to unix:///me/myproject/media/image.jpg failed (13: Permission denied) while connecting to upstream
connect() to unix:///me/myproject/project.sock failed (13: Permission denied) while connecting to upstream
for the 403 and 502, respectively.
I have read multiple questions and guides (one here, another here and yet another one, and this is not all of them), changed my permissions and even moved my .sock to another folder (one of the SO answers recommended that).
What else can I try?
Update:
I mentioned it in a comment, but I've gotten a bit further. A part of the problem was that, apparently, the /home directory on my VM is NFS, which messes up a good many permissions.
What I've done:
I've set up my project in /var/www/myproject/
Run chown -R me:www-data myproject
Run chmod -R 764 myproject
My new results:
Without nginx running:
uwsgi --http :8000 --module myproject.wsgi
works perfectly
With nginx running:
uwsgi --socket myproject.sock --module myproject.wsgi --chmod-socket=664
gives me a 502
uwsgi --ini myproject.ini
gives me a 502
So now it's not a general permission issue, it's definitely an issue with nginx...
Update #2:
For the moment, everything is working when other has read-write permissions on the socket, and read-execute permissions on the rest of the project.
So nginx is not recognized as it should... I've double-checked, and nginx is running as the www-data user, which is the group-owner of my entire project, and which has read-execute permissions, just as other now has.
Here's my (updated) nginx.conf
# myproject_nginx.conf
# the upstream component nginx needs to connect to
upstream django {
# server unix:///path/to/your/mysite/mysite.sock; # for a file socket
server unix:///var/www/myproject/myproject.sock;
# server 127.0.0.1:8001; # for a web port socket (we'll use this first)
}
# configuration of the server
server {
# the port your site will be served on
listen 8000;
# the domain name it will serve for
server_name my.ip.goes.here; # substitute your machine's IP address or FQDN
charset utf-8;
# max upload size
client_max_body_size 75M; # adjust to taste
# Django media
location /media {
alias /var/www/myproject/media; # your Django project's media files - amend as required
}
location /static {
alias /var/www/myproject/static; # your Django project's static files - amend as required
# Finally, send all non-media requests to the Django server.
location / {
uwsgi_pass django;
include /var/www/myproject/uwsgi_params; # the uwsgi_params file you installed
}
}
And here's my (updated) uwsgi.ini
# myproject_uwsgi.ini file
[uwsgi]
# Django-related settings
# the base directory (full path)
chdir = /var/www/myproject
# Django's wsgi file
module = myproject.wsgi
# the virtualenv (full path)
home = /var/www/myenv
# process-related settings
master = true
# maximum number of worker processes
processes = 10
# the socket (full path)
socket = /var/www/myproject/myproject.sock
# ... with appropriate permissions - may be needed
chmod-socket = 666
uid = me
gid = www-data
# clear environment on exit
vacuum = true
From my experience, most of the permission problems around web server are by accessing file which is owned by root, but Apache (nginx) is running under www-data user.
Try running sudo chown www-data -R /path/to/your/data/folder.
As the tutorial said:
You may also have to add your user to nginx’s group (which is probably
www-data), or vice-versa, so that nginx can read and write to your
socket properly.
Try that and see what happens.
As well I wouldn't recommend you doing things with sudo or as root, do it as a normal user and place the permission as it get necessary, otherwise you might end up in a situation that Nginx or uWSGI need to do something with the files and they are owned by root.
I am trying to configure private pub on my server on an ip, same configuration was working earlier on different server , but not working on current.
config/private_pub.yml
development:
server: "http://52.76.xx.xx:9292/faye"
#server: "http://faye.myserver.com/faye"
secret_token: "secret"
test:
server: "http://localhost:9292/faye"
secret_token: "secret"
production:
server: "http://example.com/faye"
private_pub.ru
# Run with: rackup private_pub.ru -s thin -E production
require "bundler/setup"
require "yaml"
require "faye"
require "private_pub"
Faye::WebSocket.load_adapter('thin')
PrivatePub.load_config(File.expand_path("../config/private_pub.yml", __FILE__), ENV["RAILS_ENV"] || "development")
run PrivatePub.faye_app
When I run rackup private_pub.ru -s thin -E production I get below
Thin web server (v1.6.4 codename Gob Bluth)
Maximum connections set to 1024
Listening on localhost:9292, CTRL+C to stop
But when I try to curl or call in browser, I get
Connection refused - connect(2) for "52.76.xx.xx" port 9292
I also tried with separate domain pointing to same IP for faye server and proxy it to localhost:9292 as below
server {
listen 80;
server_name faye.myserver.com www.faye.myserver.com;
# passenger_enabled on;
location / {
proxy_pass http://localhost:9292;
}
# passenger_app_env production;
}
But that also didn't work. Can you please tell me how to resolve the issue?
I'm a very beginner in python and django. However I'm trying to create a server to deploy my application. But when I want to access my app, I always get the default nginx page "Welcome to nginx".
This server is on Ubuntu 12.04 (precise)
I've installed nginx, python, django and uwsgi packages with apt.
Next I've created a django project to /var/www/djangoApps and a django app to /var/www/djangoApps/testApp
This is my /etc/nginx/sites-available/djangoApps :
server {
listen 80
server_name django.pommesky.com;
rewrite ^(.*) http://www.django.pommesky.com/$1 permanent;
}
server {
listen 80;
server_name www.django.pommesky.com;
access_log /var/log/nginx/djangoApps_access.log;
error_log /var/log/nginx/djangoApps_error.log;
location /media {
alias /var/www/djangoApps/media/;
}
location /static {
alias /var/www/djangoApps/static/;
}
location / {
uwsgi_pass unix:///run/uwsgi/app/djangoApps/socket;
include uwsgi_params;
}
}
And this is my /etc/uwsgi/apps-available/djangoApps.ini :
env = DJANGO_SETTINGS_MODULE=djangoApps.settings
module = django.core.handlers.wsgi:WSGIHandler()
chdir = /var/www/djangoApps
socket = /run/uwsgi/djangoApps/socket
logto = /var/log/uwsgi/djangoApps.log
The uwsgi log doesn't show anything, everything seems to run well, it finishes by spawned uWSGI worker ...
But /var/log/nginx/djangoApps_access.log; and /var/log/nginx/djangoApps_error.log; don't exist, which is very strange.
I can't figure out what's wrong with my configuration.
Please help me ...
The domain django.pommesky.com doesn't look like it's alive, so it's possible that Nginx is receiving requests with wrong Host: field in the HTTP request header. (sect. 14.23) So Nginx serves a default catch-all page.
You can disable the default Nginx site by removing the /etc/nginx/sites-enabled/default link, and then restarting the daemon.
sudo rm -v /etc/nginx/sites-enabled/default
sudo service nginx restart
You can reenable by recreating the link:
sudo ln -sf /etc/nginx/sites-available/default /etc/nginx/sites-enabled/default
sudo service nginx restart
The other thing you can try is to setup Bind or another DNS daemon to serve a fake pommesky.com zone with all the subdomains you want and use that DNS while you're developing your site.
Of course you can also register that domain with a hosting provider, and then use the DNS zone editor in its control panel to setup your subdomains and all the PTRs you want to whatever public IP addresses you need.