I have a website I'm running on cloudflare that is proxied and I want to get the visitors ip. I need to get this header request.headers.get('cf-connecting-ip') and replace the ip in gunicorn with that header. How would I do this? My current gunicorn access logs look like this:
172.70.114.46 - - [27/Sep/2022:19:37:12 -0400] "GET / HTTP/1.1" 200 2332 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:104.0) Gecko/20100101 Firefox/104.0"
Related
I have an issue with login into django admin site which is almost the same question
five years ago. Unfortunately, there is no specific answer until now. Here is the brief introduction for the question.
My nginx serves the 80 port and it will proxy all the URL starts with prefix to 8000 port which Django is listening.
location /prefix/ {
proxy_pass http://0.0.0.0:8000/;
}
access /prefix/admin/, it gives me a 302 and redirect to /admin/login/?next=/admin/. However, if we access /prefix/admin/login, it works and we have the Django Administration login page as below.
However, if we are trying to login(url is /admin/login/) with username and password, it gives me a 404.
Let me make a summary, here we have two issues in total.
prefix/admin not working, prefix/admin/login works.
Login into the admin site(admin/login) not working.
The first issue has been solved by
location /prefix/admin/ {
proxy_pass http://0.0.0.0:8000/admin/login/;
}
The second issue, however, not working by the following.
location = /admin/login {
proxy_pass http://0.0.0.0:8000/admin/;
}
It told me that I have too many redirects. How can I fix this? Thanks in advance.
Edit:
I have compared my local login and remote login. Here is the local.
[16/Sep/2022 13:58:55] "POST /admin/login/?next=/admin/ HTTP/1.1" 302 0
[16/Sep/2022 13:58:55] "GET /admin/ HTTP/1.1" 200 6211
And here is the remote.
192.168.12.33 - - [16/Sep/2022:05:59:36 +0000] "POST /admin/login/?next=/admin/ HTTP/1.1" 302 0 "http://192.168.6.32/admin/login/?next=/admin/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"
192.168.12.33 - - [16/Sep/2022:05:59:36 +0000] "GET /admin/ HTTP/1.1" 302 0 "http://192.168.6.32/admin/login/?next=/admin/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"
In the remote, the second GET request returns 302.
I keep getting bad requests (400) when trying to open a websocket, and I'm trying to figure out why. The back-end is built on flask with flask-socketio. Here is my Docker container for the back-end:
FROM alpine:edge
RUN apk update
RUN apk add python3 py3-cffi py3-bcrypt libc-dev py3-psycopg2 py3-gevent
RUN pip3 install --upgrade pip
RUN pip3 install flask flask-restful flask-jwt-extended gunicorn requests flask-sqlalchemy flask-socketio
ADD ./rest-api /root/rest-api
ADD ./ui/dist/ui /root/ui
CMD ["gunicorn", "-k", "gevent", "-w", "1", "--bind", "0.0.0.0:3001", "--access-logfile", "-", "--chdir", "/root/rest-api/", "app:app"]
Here are my yaml files:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: my-global-ip
networking.gke.io/managed-certificates: my-certificate
nginx.ingress.kubernetes.io/add-base-url: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/websocket-services: "my-service-web"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
spec:
rules:
- http:
paths:
- path: /grafana/*
backend:
serviceName: my-service-web
servicePort: grafana-port
- path: /*
backend:
serviceName: my-service-web
servicePort: web-app-port
apiVersion: v1
kind: Service
metadata:
name: my-service-web
spec:
ports:
- port: 3000
name: grafana-port
targetPort: grafana-port
protocol: TCP
- port: 3001
name: web-app-port
targetPort: web-app-port
protocol: TCP
selector:
app: my-cloud
type: NodePort
I can see from the logs that the requests reach the back-end container:
10.166.0.42 - - [26/Nov/2019:08:52:22 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:23 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:25 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:28 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:34 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.166.0.41 - - [26/Nov/2019:08:52:39 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.166.0.41 - - [26/Nov/2019:08:52:44 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.166.0.42 - - [26/Nov/2019:08:52:49 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:54 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
What I can't figure out is how to configure kubernetes ingress to follow flask-socketio configuration for nginx:
location /socket.io {
include proxy_params;
proxy_http_version 1.1;
proxy_buffering off;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_pass http://127.0.0.1:5000/socket.io;
}
How do I do the connection upgrade to websocket in kubernetes ingress?
Update:
I instantiated SocketIO with logs enabled and got the following:
6f3a03945f174b039b033e887079b97d: Sending packet OPEN data {'sid': '6f3a03945f174b039b033e887079b97d', 'upgrades': [], 'pingTimeout': 60000, 'pingInterval': 25000}
6f3a03945f174b039b033e887079b97d: Sending packet MESSAGE data 0
6f3a03945f174b039b033e887079b97d: Received request to upgrade to websocket
10.166.0.43 - - [26/Nov/2019:10:19:45 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0
I almost had it right from the beginning. I changed the CMD in my Dockerfile to (as per Flask-socketio docs):
gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 module:app
Here's the complete Dockerfile:
FROM alpine:edge
RUN echo "http://dl-cdn.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories
RUN apk update && apk upgrade
RUN apk add python3 py3-cffi py3-bcrypt libc-dev py3-psycopg2 py3-gevent-websocket
RUN pip3 install --upgrade pip
RUN pip3 install flask flask-restful flask-jwt-extended gunicorn requests flask-sqlalchemy flask-socketio
ADD ./rest-api /root/rest-api
ADD ./ui/dist/ui /root/ui
CMD ["gunicorn", "-k", "geventwebsocket.gunicorn.workers.GeventWebSocketWorker", "-w", "1", "--bind", "0.0.0.0:3001", "--timeout", "180", "--access-logfile", "-", "--chdir", "/root/rest-api/", "app:app"]
I am setting up Fail2ban on my server, recently got a lots bad bots is crawling my site cause my SQL server down
From my Apache2 logs
51.255.65.13 - - [10/Dec/2017:12:03:19 +0800] "GET /crew/nm0935095-gary-winick HTTP/1.0" 200 17985 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
51.255.65.30 - - [10/Dec/2017:12:03:31 +0800] "GET /movie/tt0498567-summer-time-machine-blues HTTP/1.0" 200 17658 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
217.182.132.190 - - [10/Dec/2017:12:03:36 +0800] "GET /movie/tt1705064-genji-monogatari:-sennen-no-nazo/ HTTP/1.0" 200 17344 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
how to create a failregex for "ahrefs.com" ?
Many Thanks
In order to catch anything containing "ahrefs.com", your failregex would look as follows:
failregex = ^<HOST>.*ahrefs\.com.*
Where the <HOST> tag is built-in Fail2ban as an alias for (?:::f{4,6}:)?(?P<host>\S+):
https://www.fail2ban.org/wiki/index.php/Apache
I have successfully got the credentials on my mail id for enrolling my device through WSO2 EMM. But when i am signing in using the mail credentials its giving an error which "Enrollment failed -Please contact administrator "
The logs were:
Chrome/44.0.2403.133 Mobile Safari/537.36"
172.17.29.121 - - [16/Jun/2016:12:34:53 +0530] "GET /emm-web-agent/public/asset-download-agent-android/asset/android-agent.apk HTTP/1.1" 200 2896941 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1)"
172.17.29.121 - - [16/Jun/2016:12:35:32 +0530] "GET /emm-web-agent/public/asset-download-agent-android/asset/android-agent.apk HTTP/1.1" 200 590411 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1)"
172.17.29.121 - - [16/Jun/2016:12:40:13 +0530] "GET /emm-web-agent/public/asset-download-agent-android/asset/android-agent.apk HTTP/1.1" 200 590411 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1)
172.17.29.121 - - [16/Jun/2016:12:42:45 +0530] "POST /dynamic-client-web/register HTTP/1.1" 201 148 "-" "Mozilla/5.0 ( compatible ), Android"
172.17.29.121 - - [16/Jun/2016:12:42:45 +0530] "POST /oauth2/token HTTP/1.1" 200 160 "-" "Apache-HttpClient/UNAVAILABLE (java 1.4)"
172.17.29.121 - - [16/Jun/2016:12:42:45 +0530] "GET /mdm-android-agent/device/license HTTP/1.1" 401 23 "-" "Mozilla/5.0 ( compatible ), Android"
According to the conversation the problem is with the permission you have provided to the given user role.
You can use an existing role with device management permission as sashika has suggested.
There is a login permission as the very last permission entry in the permission management UI, please include that permission to the related role in order to overcome the situation.
You need to add permissions to the role - specifically the "enroll" role.
I have followed this tutorial: http://blog.wercker.com/2013/11/25/django-16-part3.html and I am just trying to make it work locally with Vagrant for now. I am not trying to use Wercker.
After everything is installed, I try to access the website but I get a Bad Request (400) error every time. I do not know if that is due to a problem in nginx or in gunicorn.
They both have a log entry so at least I know that the request goes all the way through gunicorn and is not stopped at the nginx level.
Where is the problem located? Gunicorn? nginx?
Here are the logs of gunicorn and nginx.
I see that the favicon is missing but that only should not stop the page from being displayed right?
Gunicorn:
>>> cat /var/local/sites/hellocities/run/gunicorn.error.log
10.0.0.1 - - [28/Jan/2014:07:05:16] "GET / HTTP/1.0" 400 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
10.0.0.1 - - [28/Jan/2014:07:09:43] "GET / HTTP/1.0" 400 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
Nginx:
>>> cat /var/log/nginx/hellocities-access.log
10.0.0.1 - - [28/Jan/2014:07:05:16 +0000] "GET / HTTP/1.1" 400 37 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
10.0.0.1 - - [28/Jan/2014:07:05:20 +0000] "GET /favicon.ico HTTP/1.1" 404 200 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
10.0.0.1 - - [28/Jan/2014:07:09:43 +0000] "GET / HTTP/1.1" 400 37 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
10.0.0.1 - - [28/Jan/2014:07:09:44 +0000] "GET /favicon.ico HTTP/1.1" 404 200 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
>>> cat /var/log/nginx/hellocities-error.log
2014/01/28 07:05:20 [error] 13886#0: *1 open() "/var/local/sites/hellocities/static/favicon.ico" failed (2: No such file or directory), client: 10.0.0.1, server: _, request: "GET /favicon.ico HTTP/1.1", host: "10.0.0.200"
2014/01/28 07:09:44 [error] 13886#0: *3 open() "/var/local/sites/hellocities/static/favicon.ico" failed (2: No such file or directory), client: 10.0.0.1, server: _, request: "GET /favicon.ico HTTP/1.1", host: "10.0.0.200"
I had the same problem and adding ALLOWED_HOSTS = ("yourdomain.com",) to settings fixed it.
UPDATE: there few other possibilities:
Nginx (or whatever web server you use) doesn't pass the $host variable to the app
Host contains underscores
See details: https://blog.anvileight.com/posts/how-to-fix-bad-request-400-in-django/
As I was having the same issue (400 error code when trying to share with vagrant share), I stumble upon this question. The answer and comments are right, as the obvious solution is to set ALLOWED_HOSTS list, but I was already setting it correctly (I thought).
I can't speak for nginx as I'm running this on apache2, but here's what solved the issue:
Take a look at the ALLOWED_HOSTS doc to find what's best for your case.
With vagrant, you might find it useful to accept all the vagrantshare.com subdomain, so just add '.vagrantshare.com' (notice the dot) to the ALLOWED_HOSTS list.
Not sure if it is really necessary, but I changed the modified date of the wsgi.py file
touch wsgi.py
As I'm using apache2, I needed to restart the service.
sudo service apache2 restart
And then it worked.
I ran into this issue. It was because I forgot to add the proxy_set_header settings in the nginx config:
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
So Django didn't see the original hostname that was requested, so it didn't match with what was in ALLOWED_HOSTS. Then it gave back the 400 response.
After adding this to my nginx config (at the spot where you do the proxy_pass to Gunicorn) and then restarting nginx, it worked.
More info: https://docs.gunicorn.org/en/stable/deploy.html#nginx-configuration