How to start airflow-scheduler pod for Google Cloud Composer? - google-cloud-platform

Question

A Composer cluster went down because its airflow-worker pods needed a Docker image that was not accessible.
Now access to the Docker image was restore, but the airflow-scheduler pod has disappeared.
I tried updating the Composer Environment by setting a new Environment Variable, with the following error :
UPDATE operation on this environment failed X minutes ago with the following error message: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({
"Date": "recently",
"Audit-Id": "my-own-audit-id",
"Content-Length": "236",
"Content-Type": "application/json",
"Cache-Control": "no-cache, private"
})
HTTP response body: {
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "deployments.apps \"airflow-scheduler\" not found",
"reason": "NotFound",
"details": {
"name": "airflow-scheduler",
"group": "apps",
"kind": "deployments"
},
"code": 404
}
Error in Composer Agent
How can I launch a airflow-scheduler pod on my Composer cluster ?
What is the .yaml configuration file I need to apply ?
I tried launching the scheduler from inside another pod with airflow scheduler, and while it effectively starts a scheduler, it's not a Kubernetes pod and will not integrate well with the managed airflow cluster.

Answer 1

To restart the airflow-scheduler, run the following
# Fetch the old deployment, and pipe it into the replace command.
COMPOSER_WORKSPACE=$(kubectl get namespace | egrep -i 'composer|airflow' | awk '{ print $1 }')
kubectl get deployment airflow-scheduler --output yaml \
--namespace=${COMPOSER_WORKSPACE}| kubectl replace --force -f -