Nginx configuration for jenkins on ec2 - amazon-web-services

Im learning nginx and jenkins by setting up a build server on ec2. Setting up jenkins was easy and I was able to even create a test job. I now want to move on to nginx config and mighty confused as to how to set it up. I have hosted zone with my domain, lets call it domain.com . I created an A record for jenkins.domain.com and in the value box put it the IP of the ec2 instance.
Then added this to the /etc/nginx/site-enabled/default
server {
listen 80;
server_name jenkins.domain.com;
return 301 https://$host$request_uri;
}
server {
listen 80;
server_name jenkins.domain.com;
location / {
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Fix the "It appears that your reverse proxy set up is broken" error.
proxy_pass http://127.0.0.1:8080;
proxy_read_timeout 90;
proxy_redirect http://127.0.0.1:8080 https://jenkins.domain.com;
# Required for new HTTP-based CLI
proxy_http_version 1.1;
proxy_request_buffering off;
# workaround for https://issues.jenkins-ci.org/browse/JENKINS-45651
add_header 'X-SSH-Endpoint' 'jenkins.domain.com:50022' always;
}
}
However when I go to jenkins.domain.com:80 I get the site cannot be reached page...

proxy_redirect is not needed here. You can use below site configuration. You should create file jenkins in /etc/nginx/site-available(for ubuntu) or /etc/nginx/conf.d/ (centos or rhel) & copy the configuration in that file. You have to create the soft link on Ubuntu in site-enabled.
ln -s /etc/nginx/site-available/jenkins /etc/nginx/site-enabled/jenkins
Jenkins conf file
server {
listen 80;
server_name jenkins.domain.com;
access_log /var/log/nginx-access.log;
error_log /var/log/nginx-error.log;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_read_timeout 150s;
proxy_next_upstream error;
proxy_pass http://127.0.0.1:8080;
# Add HTTP Strict Transport Security for good measure.
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains;";
}
}

Related

Nginx proxy server to another proxy server Gateway timeout

It's my first time deploying an application on a deployment environment so I am a complete beginner at this, I have an nginx proxy server (call it server1) on an instance with an exposed IP to the internet & it routes requests to another server on a different instance (call it server2) that hosts my Django application, the conf file for server1 goes like this :
`server{
server_name _;
location / {
proxy_pass_header Authorization;
proxy_pass http://10.156.0.4:80;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Connection “”;
proxy_buffering off;
client_max_body_size 0;
proxy_read_timeout 36000s;
proxy_redirect off;
}
listen 443 ssl;
listen [::]:443 ssl;
include snippets/self-signed.conf;
include snippets/ssl-params.conf; }
server{
listen 80;
listen [::]:80;
server_name _;
return 302 https://35.246.244.220;}
and the second server:
server{
listen 80;
listen [::]:80;
server_name _;
location / {
proxy_pass_header Authorization;
proxy_pass http://10.156.0.4:8880;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Connection “”;
proxy_buffering off;
client_max_body_size 0;
proxy_read_timeout 36000s;
proxy_redirect off;
}
location /static/ {
alias /opt/app/mydjangoapp/staticfiles/;
autoindex off; }
}
I am running my django application using this command python manage.py runserver 0.0.0.0:8880 & I also did collectstatic before running the application.
Everything works fine when i edit proxy_pass in server1 to http://10.156.0.4:8880 directly but i needed the second nginx server so i can serve the static files for my application.
I feel like I am doing something obviously wrong here, but for the life of me i can't figure it out.
I figured out the problem, there was a firewall rule between those two instances that didn't allow ingress or egress from port 80, only port 8880. Didn't think of this at all!

Getting rid of port in URL for django installation in production

I'm trying for the first time to deploy my Django application on a server but so far I wasn't able to get rid of port in my URL. Right now I'm using Gunicorn with Nginx with the following configuration.
Nginx /etc/nginx/sites-enabled/site.conf
server {
listen 8000;
server_name example.com;
location = /favicon.ico {access_log off;log_not_found off;}
location /static/ {
root /home/webapp/um;
}
location /media/ {
root /home/webapp/um;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/webapp/um/um.sock;
}
}
/etc/nginx/proxy_params
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
Gunicorn /etc/systemd/system/gunicorn.service
Description=gunicorn service
After=network.target
[Service]
User=root
Group=www-data
WorkingDirectory=/home/webapp/um/
ExecStart=/root/um/bin/gunicorn --access-logfile - --workers 3 --bind unix:/home/webapp/um/um.sock um.wsgi:application
[Install]
WantedBy=multi-user.target
Gunicorn binding
gunicorn --bind 0.0.0.0:8000 um.wsgi:application
Changing port 8000 with port 80 in /etc/nginx/sites-enabled/site.conf gives me a 404 on nginx. Using port 8000 I'm able to see the site using http://example.com:8000/myapp but I'm aiming at using http://example.com/myapp as my address.
As a side note, the VPS I'm installing the app on came with Plesk already installed with which I'm also not familiar with. I don't know if Plesk might be interferring in catching traffic from port 80.
Thanks in advance
You just need to listen this server on port 80 instead of 8000
save gunicorn as described
server {
listen 80;
server_name 52.14.64.58 example.com www.example.com;
location = /favicon.ico {access_log off;log_not_found off;}
location /static/ {
root /home/webapp/um;
}
location /media/ {
root /home/webapp/um;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/webapp/um/um.sock;
}
}
52.14.64.58 => is ipv4 of your virtual machine, it could be anything in your case.
Now time to make changes in our django settings
ALLOWED_HOSTS = ['IP_ADDRESS', 'example.com', 'www.example.com']
Now check nginx status then restart gunicorn and nginx . I hope it would work for you.
sudo nginx -t
sudo systemctl restart nginx
sudo systemctl restart gunicorn
Now setup your domain by it's dns settings.
After a bit of struggling, I found the solution. Turns out my config was correct, but there was an nginx config file automatically written by plesk that was catching requests on port 80. The content of such file is
server {
listen 111.111.111.111:80;
location ^~ /plesk-site-preview/ {
proxy_pass http://127.0.0.1:8880;
proxy_set_header Host plesk-site-preview.local;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_cookie_domain plesk-site-preview.local $host;
access_log off;
}
location / {
proxy_pass http://111.111.111.111:7080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen 111.111.111.111:443 ssl;
ssl_certificate /opt/psa/var/certificates/certWpPLaPv;
ssl_certificate_key /opt/psa/var/certificates/certWpPLaPv;
location ^~ /plesk-site-preview/ {
proxy_pass http://127.0.0.1:8880;
proxy_set_header Host plesk-site-preview.local;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_cookie_domain plesk-site-preview.local $host;
access_log off;
}
location / {
proxy_pass https://111.111.111.111:7081;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
Once I removed that file from inclusion in nginx.conf everything started working. As a suggestion for whoever is facing a similar situation, I would recommend to check what Nginx is processing using the following command
nginx -T

Nginx reverse proxy configuration for AWS web console

Is there a way to access AWS web console via nginx reverse proxy through my subdomain?
Here is the nginx configuration is have been using :
server {
listen localhost:443 ssl;
server_name aws1.subdomain.com;
include snippets/proxy_ssl.conf;
location / {
proxy_pass https://console.aws.amazon.com/;
proxy_set_header Host $host;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_read_timeout 86400;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
proxy_hide_header X-Frame-Options;
}
}
The above configuration throws:
NetworkError: 400 Bad Request
And shows amazon's default 400 bad request page when i try to access https://aws1.subdomain.com in my browser.
I have this working using the following lines in nginx.conf. You can also add lines for http auth as required depending on your config.
location = / { rewrite ^ /_plugin/kibana/ redirect; }
location / {
proxy_pass https://<es-domain-url>.es.amazonaws.com;
proxy_http_version 1.1;
proxy_set_header Authorization "";
proxy_hide_header Authorization;
proxy_set_header X-Forwarded-Proto $scheme;
}

Nginx Share Cookies Between Subdomains without Access to Backend

TLDR: How to share cookies between subdomains for a backend application sever that I cannot "configure" using nginx (1.8.x) as a proxy - some magical combination of proxy_*?
A tornado web server is running on "127.0.0.1:9999/ipython" that I cannot configure (it's running as part of an ipython notebook server). I'm using nginx to proxy from "www.mysite.com" to 127.0.0.1:9999 successfully (http traffic at least).
However, part of the backend application requires Websockets. Because I am using CloudFlare, I have to use a separate domain for Websockets ("Websockets are currently only available for Enterprise customers ... All other customers ... should create a subdomain for Websockets in their CloudFlare DNS and disable the CloudFlare proxy"). I'm using "ws.mysite.com".
When a user logs in at "https :// www.mysite.com", a cookie is set by the tornado web server for "www.mysite.com" (I can't seem to configure it, otherwise I would just set it to ".mysite.com"). When the websocket part of the application kicks in, it sends a request to "wss :// ws.mysite.com", but fails to authenticate because the cookie is set for a different domain("www.mysite.com").
Is it possible for nginx to "spoof" the domain so the tornado webserver registers it for ".mysite.com"? proxy_cookie_domain doesn't seem to work as I'd expect... Should I hard code "proxy_set_header Host"?
I was thinking a nginx conf similar to....
upstream ipython_server {
server 127.0.0.1:8888;
}
server {
listen 443;
server_name www.mysite.com;
ssl_certificate cert.crt;
ssl_certificate_key cert.key;
ssl on;
# **** THIS DOESN'T WORK ??? ****
proxy_cookie_domain www.mysite.com .mysite.com;
location /ipython/static {
proxy_pass https://ipython_server$request_uri;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /ipython/api/sessions {
proxy_pass https://ipython_server$request_uri;
proxy_set_header Host $host;
proxy_set_header Origin "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /ipython {
proxy_pass https://ipython_server$request_uri;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location / {
try_files $uri $uri/ =404;
}
}
server {
listen 443;
server_name ws.azampagl.com;
ssl_certificate cert.crt;
ssl_certificate_key cert.key;
ssl on;
# **** THIS DOESN'T WORK ??? ****
proxy_cookie_domain ws.mysite.com .mysite.com;
# This is the websocket location
location /ipython/api/kernels/ {
proxy_pass https://ipython_server$request_uri;
proxy_redirect off;
proxy_http_version 1.1;
chunked_transfer_encoding off;
proxy_buffering off;
proxy_read_timeout 86400;
proxy_set_header Host $host;
proxy_set_header Origin "";
proxy_set_header Upgrade websocket;
proxy_set_header Connection "upgrade";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
I've been looking in the nginx lua module? It looks like you can set cookie domains, but it looks hackish...
Thanks greatly in advance for your assistance!
(Side note: I do technically have access to the tornado configuration, but there is zero documentation on how to set the "cookie domain" for the server. i.e.
c.NotebookApp.tornado_settings = {'cookie_domain????':'.mysite.com'}
)

How can I run multiple Ring apps on the same server?

I’m new to Ring (and Clojure server-side programming in general). I have a Ring-based app that works well in “development mode”, i.e. it can listen on localhost:3000 and it responds appropriately. As part of deploying this app I’d like to change the base URL for the app to something like myserver.com/analytics/v1, so that for example a request that previously went to localhost:3000/foo should now go to myserver.com/analytics/v1/foo.
I guess I have two closely-related questions here: How can I tell Ring/Jetty to listen only at a certain URL that is not the root URL of the server? And how can I set this up so that I could add another app (for example, myserver.com/analytics/v2) without downtime for the first app? Do I need to write another Ring app that will listen on myserver.com/ and route the requests to my other apps as appropriate?
The way I'm currently handling this is let each Ring app run in it's own embedded Jetty instance, each listens on their own port, like for example: 8080 en 8085.
On the server I block these ports externally, so only localhost can access them.
Then I setup Nginx to select the right app based on the subdomain:
http://twitter.michielborkent.nl
http://tictactoe.michielborkent.nl
There are more advanced setups possible, but for me this is the one with least configuration.
Here is my nginx.conf. If you want to have more configuration details, just let me know.
server { listen 80;
server_name twitter.michielborkent.nl;
access_log /var/log/twitter-service.log;
location / {
proxy_pass http://localhost:8080;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
}
}
server { listen 80;
server_name tictactoe.michielborkent.nl;
access_log /var/log/tictactoe.log;
location / {
proxy_pass http://localhost:8085;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
}
}
Here’s how I adapted #Michiel Borkent’s nginx.conf to fit my needs:
server {
listen 80;
server_name www.myserver.com;
location /analytics/v1/ {
proxy_pass http://localhost:3001/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
}
location /trac/ {
proxy_pass http://localhost:3002/trac/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
}
}
With this situation I can just set my Ring app to serve on port 3001; I have Trac serving on port 3002, or I could have another Ring app or whatever. Both of these applications are accessible from www.myserver.com (port 80), just under different paths.