How to deploy older ingress-nginx-controller or specify version with minikube? - kubectl

Question

I am trying to deploy a specific version of ingress-controller with minikube and kubernetesv1.13, but from what I see it is only possible to have latest version of ingress-nginx-controller deployed.
I expect the ingress-nginx-controller-#####-#### pod to come back online and run with the nginx-ingress image version I point to in the deployments details.
After editing the ingress-nginx-controller deployment via kubectl edit and changing the image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller property from 0.32.0 to 0.24.1, the pod restarts and goes into CrashLoopBackOff state.
By hitting describe, the pod seems complaining about the node not having free ports:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 5m8s (x2 over 5m8s) default-scheduler 0/1 nodes are available: 1 node(s) didn't have free ports for the requested pod ports.
Normal Scheduled 4m54s default-scheduler Successfully assigned kube-system/ingress-nginx-controller-6c4b64d58c-s5ddz to minikube
After searching for a similar case I tried the following:
I check ss but see no port 80 or 443 being busy on the host:
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 32 192.168.122.1:53 0.0.0.0:*
LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:*
LISTEN 0 5 127.0.0.1:631 0.0.0.0:*
LISTEN 0 5 [::1]:631 [::]:*
No pods seems to be in terminating status:
NAME READY STATUS RESTARTS AGE
coredns-86c58d9df4-7s55r 1/1 Running 1 3h14m
coredns-86c58d9df4-rtssn 1/1 Running 1 3h14m
etcd-minikube 1/1 Running 1 3h13m
ingress-nginx-admission-create-gpfml 0/1 Completed 0 47m
ingress-nginx-admission-patch-z96hd 0/1 Completed 0 47m
ingress-nginx-controller-6c4b64d58c-s5ddz 0/1 CrashLoopBackOff 9 24m
kube-apiserver-minikube 1/1 Running 0 145m
kube-controller-manager-minikube 1/1 Running 0 145m
kube-proxy-pmwxr 1/1 Running 0 144m
kube-scheduler-minikube 1/1 Running 0 145m
storage-provisioner 1/1 Running 2 3h14m
I did not create any yml file or custom deployment, just installed minikube and enabled the ingress addon.
How to use a different nginx-ingress-controller version ?

Answer 1

The Nginx Version is tied to minikube version.
First I tried previous versions. Unfortunatelly, the available Minikube v1.3 uses nginx 0.25.0 and Minikube v1.2 uses nginx 0.23.0
So the only way I found to run nginx 0.24.0 in Minikube was building the binary myself using minikube v1.4, here is the step-by-step:
Download the minikube 1.4 repository and extract it:
$ wget https://github.com/kubernetes/minikube/archive/v1.4.0.tar.gz
$ tar -xvzf v1.4.0.tar.gz
Then, cd into the newly created minikube-1.4.0 folder and edit the file deploy/addons/ingress/ingress-dp.yaml.tmpl changing the image version to 0.24.1 as below:
spec:
serviceAccountName: ingress-nginx
containers:
- name: controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.24.1
In order to build you have to download a Go distribution from original repo: https://golang.org/dl/
then follow the steps in https://golang.org/doc/install to install it. If you are running linux 64 bits, you can use the bellow comments:
$ wget https://dl.google.com/go/go1.14.4.linux-amd64.tar.gz
$ sudo tar -C /usr/local -xzf go1.14.4.linux-amd64.tar.gz
$ export PATH=$PATH:/usr/local/go/bin
Then from the Minikube 1.4.0 folder, run make:
/minikube-1.4.0$ ls
CHANGELOG.md CONTRIBUTING.md go.mod images Makefile OWNERS SECURITY_CONTACTS test.sh
cmd deploy go.sum installers netlify.toml pkg site third_party
code-of-conduct.md docs hack LICENSE README.md test translations
/minikube-1.4.0$ make
It may take a few minutes to download all dependencies, then let's copy the freshly build binary to /usr/local/bin and deploy minikube:
/minikube-1.4.0$ cd out/
/minikube-1.4.0/out$ ls
minikube minikube-linux-amd64
$ sudo cp minikube-linux-amd64 /usr/local/bin/minikube
$ minikube version
minikube version: v1.4.0
$ minikube start --vm-driver=kvm2 --kubernetes-version 1.13.12
NOTE: if you get an error about kvm2 driver when starting minikube, run the following command:
$ curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-kvm2 && sudo install docker-machine-driver-kvm2 /usr/local/bin/
This version comes with ingress enabled by default, let's check the deployment status:
$ minikube addons list | grep ingress
- ingress: enabled
$ kubectl describe deploy nginx-ingress-controller -n kube-system |
grep Image:
Image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.24.1
$ kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-54ff9cd656-d95w5 1/1 Running 0 2m14s
coredns-54ff9cd656-tnvnw 1/1 Running 0 2m14s
etcd-minikube 1/1 Running 0 78s
kube-addon-manager-minikube 1/1 Running 0 71s
kube-apiserver-minikube 1/1 Running 0 71s
kube-controller-manager-minikube 1/1 Running 0 78s
kube-proxy-wj2d6 1/1 Running 0 2m14s
kube-scheduler-minikube 1/1 Running 0 87s
nginx-ingress-controller-f98c6df-5h2l7 1/1 Running 0 2m9s
storage-provisioner 1/1 Running 0 2m8s
As you can see, the pod nginx-ingress-controller-f98c6df-5h2l7 is in running state.
If you have any question let me know in the comments.