SSHOperator with ComputeEngineSSHHook - google-cloud-platform

Question

I am trying to run a command using ssh in a GCP VM in airflow via the SSHOperator as described here:
ssh_to_vm_task = SSHOperator(
task_id="ssh_to_vm_task",
ssh_hook=ComputeEngineSSHHook(
instance_name=<MYINSTANCE>,
project_id=<MYPROJECT>,
zone=<MYZONE>,
use_oslogin=False,
use_iap_tunnel=True,
use_internal_ip=False
),
command="echo test_message",
dag=dag
)
However, I get a airflow.exceptions.AirflowException: SSH operator error: [Errno 2] No such file or directory: 'gcloud' error.
Docker is installed via docker-compose following these instructions.
Other Airflow GCP operators (such as BigQueryCheckOperator) work correctly. So at first sight it does not seem like a configuration problem.
Could you please help me? Is this a bug?

Answer 1

It seems the issue is that gcloud was not installed in the docker container by default. This has been solved by following instructions in here: it is necessary to add
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - && apt-get update -y && apt-get install google-cloud-sdk -y
to the dockerfile that is used to install airflow / install dependencies.

Answer 2

Check if the TCP port 22 is allowed through the firewall on your GCP VM instance, and make sure that the VM instance also allows SSH access and is properly configured in that VM instance. Furthermore, be sure that the IP from which you are trying to SSH at the VM instance is whitelisted through the firewall.
You can use the following command in GCP to check the ingress firewall rule for the network that contains the destination VM instance. Additionally, you can consult this [link]for more information.
This is an example of what you have to do.
´´´
gcloud compute firewall-rules list --filter network=[NETWORK-NAME] \
--filter INGRESS \
--sort-by priority \
--format="table(
name,
network,
direction,
priority,
sourceRanges.list():label=SRC_RANGES,
destinationRanges.list():label=DEST_RANGES,
allowed[].map().firewall_rule().list():label=ALLOW,
denied[].map().firewall_rule().list():label=DENY,
sourceTags.list():label=SRC_TAGS,
sourceServiceAccounts.list():label=SRC_SVC_ACCT,
targetTags.list():label=TARGET_TAGS,
targetServiceAccounts.list():label=TARGET_SVC_ACCT
)"
´´´