kubernetes ingress websockets (socket.io) - flask

Question

I keep getting bad requests (400) when trying to open a websocket, and I'm trying to figure out why. The back-end is built on flask with flask-socketio. Here is my Docker container for the back-end:
FROM alpine:edge
RUN apk update
RUN apk add python3 py3-cffi py3-bcrypt libc-dev py3-psycopg2 py3-gevent
RUN pip3 install --upgrade pip
RUN pip3 install flask flask-restful flask-jwt-extended gunicorn requests flask-sqlalchemy flask-socketio
ADD ./rest-api /root/rest-api
ADD ./ui/dist/ui /root/ui
CMD ["gunicorn", "-k", "gevent", "-w", "1", "--bind", "0.0.0.0:3001", "--access-logfile", "-", "--chdir", "/root/rest-api/", "app:app"]
Here are my yaml files:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: my-global-ip
networking.gke.io/managed-certificates: my-certificate
nginx.ingress.kubernetes.io/add-base-url: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/websocket-services: "my-service-web"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
spec:
rules:
- http:
paths:
- path: /grafana/*
backend:
serviceName: my-service-web
servicePort: grafana-port
- path: /*
backend:
serviceName: my-service-web
servicePort: web-app-port
apiVersion: v1
kind: Service
metadata:
name: my-service-web
spec:
ports:
- port: 3000
name: grafana-port
targetPort: grafana-port
protocol: TCP
- port: 3001
name: web-app-port
targetPort: web-app-port
protocol: TCP
selector:
app: my-cloud
type: NodePort
I can see from the logs that the requests reach the back-end container:
10.166.0.42 - - [26/Nov/2019:08:52:22 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:23 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:25 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:28 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:34 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.166.0.41 - - [26/Nov/2019:08:52:39 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.166.0.41 - - [26/Nov/2019:08:52:44 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.166.0.42 - - [26/Nov/2019:08:52:49 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:54 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
What I can't figure out is how to configure kubernetes ingress to follow flask-socketio configuration for nginx:
location /socket.io {
include proxy_params;
proxy_http_version 1.1;
proxy_buffering off;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_pass http://127.0.0.1:5000/socket.io;
}
How do I do the connection upgrade to websocket in kubernetes ingress?
Update:
I instantiated SocketIO with logs enabled and got the following:
6f3a03945f174b039b033e887079b97d: Sending packet OPEN data {'sid': '6f3a03945f174b039b033e887079b97d', 'upgrades': [], 'pingTimeout': 60000, 'pingInterval': 25000}
6f3a03945f174b039b033e887079b97d: Sending packet MESSAGE data 0
6f3a03945f174b039b033e887079b97d: Received request to upgrade to websocket
10.166.0.43 - - [26/Nov/2019:10:19:45 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0

Answer 1

I almost had it right from the beginning. I changed the CMD in my Dockerfile to (as per Flask-socketio docs):
gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 module:app
Here's the complete Dockerfile:
FROM alpine:edge
RUN echo "http://dl-cdn.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories
RUN apk update && apk upgrade
RUN apk add python3 py3-cffi py3-bcrypt libc-dev py3-psycopg2 py3-gevent-websocket
RUN pip3 install --upgrade pip
RUN pip3 install flask flask-restful flask-jwt-extended gunicorn requests flask-sqlalchemy flask-socketio
ADD ./rest-api /root/rest-api
ADD ./ui/dist/ui /root/ui
CMD ["gunicorn", "-k", "geventwebsocket.gunicorn.workers.GeventWebSocketWorker", "-w", "1", "--bind", "0.0.0.0:3001", "--timeout", "180", "--access-logfile", "-", "--chdir", "/root/rest-api/", "app:app"]