Hostname not resolving for CE in same network - google-cloud-platform

Question

I'm doing a deployment of 4 CE in 2 different zones (bastion in europe-west1-c and the other ones in europe-west2-c). I can ssh from cassandra-node-1 to cassandra-node-2 just using the hostname:
pedro_gordo_gmail_com#cassandra-node-1:~$ ssh cassandra-node-2
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0-1049-gcp x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
0 packages can be updated.
0 updates are security updates.
New release '18.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
Last login: Sun Dec 1 13:48:17 2019 from 10.154.0.14
groups: cannot find name for group ID 926993188
But I can't do the same from the bastion CE:
pedro_gordo_gmail_com#bastion:~$ ssh cassandra-node-1
ssh: Could not resolve hostname cassandra-node-1: Name or service not known
But I can ssh using the internal/external IP:
pedro_gordo_gmail_com#bastion:~$ ssh 10.154.0.14
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0-1049-gcp x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
* Overheard at KubeCon: "microk8s.status just blew my mind".
https://microk8s.io/docs/commands#microk8s.status
0 packages can be updated.
0 updates are security updates.
New release '18.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
Last login: Sun Dec 1 13:48:10 2019 from 173.194.92.32
groups: cannot find name for group ID 926993188
According to this GCP documentation, if I choose a custom name for my CE, then I need to edit the DNS. But on the other hand, if I don't provide a name: in my deployment-manager config, then I get the following error when I try to deploy:
gcloud deployment-manager deployments create cluster --config create-vms.yaml
ERROR: (gcloud.deployment-manager.deployments.create) ResponseError: code=412, message=Missing resource name in resource "type: compute.v1.instance
This is my deployment-manager configuration. How can I change this so that I can ssh from bastion to cassandra-node-1/2/3 just using the hostname?
# Copyright 2016 Google Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Import all templates used in deployment
# Declare all resources. In this case, one highly available service
# as defined in the ha-service.py template.
resources:
- type: compute.v1.instance
name: bastion
properties:
zone: europe-west1-c
machineType: https://www.googleapis.com/compute/v1/projects/affable-seat-213016/zones/europe-west1-c/machineTypes/n1-standard-1
disks:
- deviceName: boot
boot: true
autoDelete: true
initializeParams:
sourceImage: https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/ubuntu-1604-xenial-v20190514
networkInterfaces:
- accessConfigs:
- name: External NAT
type: ONE_TO_ONE_NAT
metadata:
items:
- key: startup-script
value: |
#!/bin/bash
sudo apt-add-repository -y ppa:ansible/ansible
sudo apt-get update
sudo apt-get install -y ansible
- type: compute.v1.instance
name: cassandra-node-1
properties:
zone: europe-west2-c
machineType: https://www.googleapis.com/compute/v1/projects/affable-seat-213016/zones/europe-west2-c/machineTypes/n1-standard-1
disks:
- deviceName: boot
boot: true
autoDelete: true
initializeParams:
sourceImage: https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/ubuntu-1604-xenial-v20190514
- deviceName: data
boot: false
autoDelete: true
initializeParams:
diskSizeGb: 1
diskType: zones/europe-west2-c/diskTypes/pd-ssd
networkInterfaces:
- accessConfigs:
- name: External NAT
type: ONE_TO_ONE_NAT
- type: compute.v1.instance
name: cassandra-node-2
properties:
zone: europe-west2-c
machineType: projects/affable-seat-213016/zones/europe-west2-c/machineTypes/n1-standard-1
disks:
- deviceName: boot
boot: true
autoDelete: true
initializeParams:
sourceImage: https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/ubuntu-1604-xenial-v20190514
- deviceName: data
boot: false
autoDelete: true
initializeParams:
diskSizeGb: 1
diskType: zones/europe-west2-c/diskTypes/pd-ssd
networkInterfaces:
- accessConfigs:
- name: External NAT
type: ONE_TO_ONE_NAT
- type: compute.v1.instance
name: cassandra-node-3
properties:
zone: europe-west2-c
machineType: https://www.googleapis.com/compute/v1/projects/affable-seat-213016/zones/europe-west2-c/machineTypes/n1-standard-1
disks:
- deviceName: boot
boot: true
autoDelete: true
initializeParams:
sourceImage: https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/ubuntu-1604-xenial-v20190514
- deviceName: data
boot: false
autoDelete: true
initializeParams:
diskSizeGb: 1
diskType: zones/europe-west2-c/diskTypes/pd-ssd
networkInterfaces:
- accessConfigs:
- name: External NAT
type: ONE_TO_ONE_NAT

Answer 1

You have two solutions:
Use Google Cloud DNS and set up a private zone to resolve hostnames for your VPC.
Use the Compute Engine internal DNS name.
However, for method #2, I do not remember if hostname resolution for internal names is resolved across zones as the Compute Engine internal DNS is used for name resolution. Method #1 will always work provided that DNS is set up correctly.