container engine ingress not working - google-cloud-platform

Question

I have a simple container in google container registry which basically does a few things and executes a binary which is a go based server, here are the contents of the DockerFile:
FROM debian:stable
WORKDIR /workspace/
COPY key.json .
COPY bin/user-creds.
EXPOSE 1108
ENV GOOGLE_APPLICATION_CREDENTIALS /workspace/key.json
RUN apt-get update \
&& apt-get install -y ca-certificates \
&& chmod +x user-creds
CMD ["./user-creds"]
this container has been tested locally and works perfectly. So using the google cloud shell I ran this container:
kubectl run user-creds --image=eu.gcr.io/GCLOUD_PROJECT/user-creds:COMMIT_SHA --port=1108
Then like it says on the doc, i exposed it on a nodeport
kubectl expose deployment user-creds --target-port=1108 --type=NodePort
Then I created an ingress with a path to the sevice:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: INGRESS_NAME
annotations:
kubernetes.io/ingress.global-static-ip-name: IP_NAME
spec:
rules:
- http:
paths:
- path: /user/creds/*
backend:
serviceName: user-creds
servicePort: 1108
then i created the ingress:
kubectl create -f INGRESS_NAME.yaml
the ingress was created and i waited some time, here is the details of the ingress:
NAME HOSTS ADDRESS PORTS AGE
INGRESS_NAME * IP_ADDRESS 80 38m
but when i go the the actual url with the path I get a 502 error:
When I go to any other path I get the default backend 404 error but when i visit the specific /user/creds/ path i get the 502 error.
To check if it is something wrong with the cluster or my specific container, port or something else, I tried exposing the container as a LoadBalancer and it works perfectly, the Command:
kubectl expose deployment user-creds --target-port=1108 --port=80 --type=LoadBalancer
service details:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP INT_IP_ADDRESS <none> 443/TCP 1h
user-creds LoadBalancer INT_IP_ADDRESS IP_ADDRESS 80:31618/TCP 1m
result: 200 with the correst response body.
Been stuck on this for time now, tried the ingress with no paths just the user-creds as the backend but still has the same error.
Any help or suggestion would be appreciated, thanks :)

Answer 1

Finally figured it out, it was to do with the health check. The health check visits / and expects a 200, if it doesn't get it then it marks the backend as unhealthy and returns 502 for every requests sent to it. My problem was that I was using the / endpoint which would've normally returned a 400 if its being called with no specific request parameters.
It was really a human error on my side, it even specifically said that in the docs: https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer#remarks
Another thing to consider is that the ingress returns all the the paths before the route so the the server needs to literally listen for /user/creds/ in my case.