I am trying to create an instance-template, where a instance create with this template automatically gets an public ipv4 asigned.
Currently I am using something like following gcloud command:
gcloud compute instance-templates create TEMPLATENAME \
--project=PROJECT \
--machine-type=e2-small \
--network-interface=network=default,network-tier=PREMIUM \
--maintenance-policy=MIGRATE --provisioning-model=STANDARD \
--service-account=SERVICE_ACCOUNT \
--scopes=https://www.googleapis.com/auth/cloud-platform \
--tags=http-server,https-server \
--create-disk=CREATE_DISK \
--no-shielded-secure-boot \
--shielded-vtpm \
--shielded-integrity-monitoring \
--reservation-affinity=any
This command is generated by the Google Cloud Console, but I have to use gcloud since I have to use a image-family to create the disk (which is to my knowledge not supported using gui).
If running this command I get the following result:
The result I want to get is:
What am I missing?
In order to get an ephemeral IP adress has to be set as empty string in the network interface flag.
--network-interface=network=default,network-tier=PREMIUM,adress=''
see https://cloud.google.com/sdk/gcloud/reference/compute/instance-templates/create?hl=de#--network-interface
Related
Is there a way to pass user-data flag as a remote script? Similar to startup-script-url ?
I have configured a yaml file using cloud-config and use --metadata-from-file user-data=config-basic.yaml flag to create a new VM with CoS.
I want to create VMs programmatically and a local file may not be accessible nor passing whole script content as user-data metadata property is feasible.
Option 1) Write cloud-config script as a shell script?
Option 2) Find logic that invokes cloud-config and populate it with metadata and insert contents there.
Option 3) A better option???
https://cloud.google.com/compute/docs/instances/startup-scripts/linux
gcloud compute instances create cos-vertex-gpu \
--image cos-101-17162-40-34 \
--image-project cos-cloud \
--boot-disk-size 100 \
--machine-type n1-standard-4 \
--zone us-west1-a \
--metadata="google-logging-enabled=true,google-monitoring-enabled=true" \
--metadata-from-file user-data=config-basic.yaml \
--maintenance-policy=TERMINATE \
--accelerator=type=nvidia-tesla-t4,count=1
I was not able to find a way to pass a remote: cloud-config programmatically.
I ended up re-writing my cloud-config as a shell script and pass it using startup-script-url
I want to create a new instance template based on existing one, with just a new container (docker).
The docs say to use,
gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME
--source-instance=SOURCE_INSTANCE
--source-instance-zone=SOURCE_INSTANCE_ZONE
But my template is a global resource not in a zone, and cannot be found.
The "create similar" in the instance template console gives an equivalent gcloud command but that fails with an error (13 Sept 2022).
(gcloud.compute.instance-templates.create-with-container) argument --create-disk: valid keys are [architecture, auto-delete, description, device-name, disk-resource-policy, image, image-family, image-project, mode, name, provisioned-iops, size, type]; received: boot
According to gcloud it is up to date.
How can I automate instance template generation with a new docker image?
Console creation/"create similar" works, but not cli.
Note: Creation with equivalent cli command given by the console fails with the same error as above.
I eventually used a "version" of the equivalent cli. Removing the --create-disk option and replacing that with some boot disk options. The cmd also needed the addition of ',address=""' to the network interface option to get an external ip address.
Note your --tags may differ based on your firewall tags. The --boot-disk-device-name can be any name, as far as I can see!
gcloud compute instance-templates create-with-container $myTemplateName \
--container-image=$myrepo/$myimage \
--boot-disk-size 10GB --boot-disk-device-name exchange-template \
--boot-disk-type=pd-balanced --project=$myproject \
--machine-type=e2-medium \
--network-interface=network=default,network-tier=PREMIUM,address="" \
--maintenance-policy=MIGRATE \
--service-account=$myServiceAccount --scopes=https://www.googleapis.com/auth/devstorage.read_only,https://www.googleapis.com/auth/logging.write,https://www.googleapis.com/auth/monitoring.write,https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/trace.append \
--tags=http-server,https-server,allowport8080 \
--container-restart-policy=always \
--no-shielded-secure-boot --shielded-vtpm --shielded-integrity-monitoring
I know how to do it when I create an instance:
gcloud compute instances create ${INSTANCE_NAME} \
--machine-type=n1-standard-8 \
--scopes=https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/userinfo.email \
--min-cpu-platform="Intel Skylake" \
${IMAGE} \
--image-project=deeplearning-platform-release \
--boot-disk-size=100GB \
--boot-disk-type=pd-ssd \
--accelerator=type=nvidia-tesla-p100,count=1 \
--boot-disk-device-name=${INSTANCE_NAME} \
--maintenance-policy=TERMINATE --restart-on-failure \
--metadata="proxy-user-mail=${GCP_LOGIN_NAME},install-nvidia-driver=True,startup-script=${STARTUP_SCRIPT}"
but what if I already have an instance, how do I update/create the startup script?
To add or update the metadata, you can use the endpoint "add-metadata" like this
gcloud compute instances add-metadata ${INSTANCE_NAME} \
--metadata startup-script=${NEW_STARTUP_SCRIPT}
The other metadatas are kept.
So I am using this gcloud console command to create an instance from container image
gcloud compute instances create-with-container test-instance \
--zone us-xx \
--container-image asia.gcr.io/my-project/my-docker-image \
--container-privileged \
--network my-network \
--subnet my-net-sub \
--create-disk name=test-data,device-name=test-data,auto-delete=yes,size=200GB,type=pd-ssd \
--container-mount-disk name=test-data,mount-path=/mnt/disks/data \
--service-account me#myproject.iam.gserviceaccount.com
which works fine and creates the instance, but it does not mount the data-disk. why?
more precisely, to add the data disk I need to
create a single partition of the whole disk and create ext4 file system
and mount the disk on given path
How can I specify the create partition with ext4 and then mount the partition part?
You can't mount the host's disk to the container (use the same disk in both). You can however mount a directory or another disk. Either way you will be able to store data on it and both OS'es (host & container) will be able to read/write from it.
Let's say you want to store all data in the host OS disk in /datadir/ and you want it to be mounted inside the container under /mnt/disks/data. Below you will find a complete (and tested) example to use:
gcloud compute instances create-with-container mytestvm1 \
--zone=europe-west3-c \
--container-image=gcr.io/google-containers/mycontainer \
--container-privileged \
--network default \
--subnet default \
--create-disk name=test-data,device-name=test-data,auto-delete=yes,size=20GB,type=pd-ssd \
--container-mount-host-path=mount-path=/mnt/disks/data,host-path=/home/myhomedir/,mode=rw \
--service-account=my_service_account#developer.gserviceaccount.com
If you need another disk mounted then just change the line:
--container-mount-host-path=mount-path=/mnt/disks/data,host-path=/home/myhomedir/,mode=rw \
to
--container-mount-disk=mount-path=/mnt/disks/data,name=data1,mode=rw \
I can't figure out how to specify preemptible GPU Deep Learning VM on GCP
This what I used:
export IMAGE_FAMILY="tf-latest-gpu"
export ZONE="europe-west4-a "
export INSTANCE_NAME="deeplearning"
gcloud compute instances create $INSTANCE_NAME \
--zone=$ZONE \
--image-family=$IMAGE_FAMILY \
--image-project=deeplearning-platform-release \
--maintenance-policy=TERMINATE \
--accelerator='type=nvidia-tesla-v100,count=2' \
--metadata='install-nvidia-driver=True'
Thank you!
You can create a preemptible Compute Engine instance with GPU by adding the --preemptible gcloud command option. As per your example, that would be:
export IMAGE_FAMILY="tf-latest-gpu"
export ZONE="europe-west4-a "
export INSTANCE_NAME="deeplearning"
gcloud compute instances create $INSTANCE_NAME \
--zone=$ZONE \
--image-family=$IMAGE_FAMILY \
--image-project=deeplearning-platform-release \
--maintenance-policy=TERMINATE \
--accelerator type=nvidia-tesla-v100,count=2 \
--metadata='install-nvidia-driver=True'
--preemptible
See documentation here and here for more details on available options.