Error running Canary Deployment in Spinnaker - amazon-web-services

Question

I am trying to enable the canary deployment for the AWS eks but my kayenta pod is not starting. When I describe the pod I see this error. Can anyone help?
Warning Unhealthy 12m (x2 over 12m) kubelet Readiness probe failed: wget: can't connect to remote host (127.0.0.1): Connection refused
Warning Unhealthy 2m56s (x59 over 12m) kubelet Readiness probe failed: wget: server returned error: HTTP/1.1 503
This is the status of pod:
NAME READY STATUS RESTARTS AGE
spin-clouddriver-d796bdc59-tpznw 1/1 Running 0 3h40m
spin-deck-77cc75b57d-w7rfp 1/1 Running 0 3h40m
spin-echo-db954bb9-phfd5 1/1 Running 0 3h40m
spin-front50-7c5684cf9-t7vl8 1/1 Running 0 3h40m
spin-gate-78d6779854-7xqz4 1/1 Running 0 3h40m
spin-kayenta-6d7b5fdfc6-p5tcp 0/1 Running 0 21m
spin-kayenta-869c46bfcf-8t5fh 0/1 Running 0 28m
spin-orca-7ddd66758d-mpnkg 1/1 Running 0 3h40m
spin-redis-5975cfcdc8-rnm9g 1/1 Running 0 45h
spin-rosco-b7dbb577-z4szz 1/1 Running 0 3h40m

Answer 1

I will try to address your issue from the Kubernetes perspective.
The errors you were experiencing:
Warning Unhealthy 12m (x2 over 12m) kubelet Readiness probe failed: wget: can't connect to remote host (127.0.0.1): Connection refused
Warning Unhealthy 2m56s (x59 over 12m) kubelet Readiness probe failed: wget: server returned error: HTTP/1.1 503
indicates that there was a problem with your ReadinessProbe configuration. Removing the ReadinessProbe from the deployment "fixed" the error but can cause some more issues in the future. To avoid that I recommend adding it back with a proper configuration:
Probes have a number of fields that you can use to more precisely
control the behavior of liveness and readiness checks:
initialDelaySeconds: Number of seconds after the container has started before liveness or readiness probes are initiated. Defaults to
0 seconds. Minimum value is 0.
periodSeconds: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.
timeoutSeconds: Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1.
successThreshold: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1
for liveness. Minimum value is 1.
failureThreshold: When a probe fails, Kubernetes will try failureThreshold times before giving up. Giving up in case of liveness
probe means restarting the container. In case of readiness probe the
Pod will be marked Unready. Defaults to 3. Minimum value is 1.
You'll need to adjust the Probe's configuration based on your apps behavior (usually by trial and error). The two resources I would recommend that will help you with that are:
Configure Liveness, Readiness and Startup Probes
Kubernetes best practices: Setting up health checks with readiness and liveness probes