I have a private image called 'test-1' and I can use it to create an instance
$ gcloud compute instances create demo --image test-1
However I cannot use a public image:
$ gcloud compute instances create demo --image ubuntu-1804-bionic-v20190204
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
- Invalid value for field 'resource.disks[0].initializeParams.sourceImage': 'https://www.googleapis.com/compute/v1/projects/szabgab-149/global/images/ubuntu-1804-bionic-v20190204'. The referenced image resource cannot be found.
Nor a public image family:
$ gcloud compute instances create demo --image-family ubuntu-1804-lts
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
- The resource 'projects/szabgab-149/global/images/family/ubuntu-1804-lts' was not found
What am I doing wrong here?
I've noticed that in the generated error message, that you are generating the location of the image as if it is located in your project, which I understand it is named szabgab-149:
The resource
'projects/szabgab-149/global/images/family/ubuntu-1804-lts'
The thing is that public images are located in its own projects, see the output of the command gcloud compute images list.
For example, the ubuntu-1804-bionic-v20190204 is under the project ubuntu-os-cloud and the image family ubuntu-1804-lts.
To solve this, you can simply add the --image-project flag, from the information of the previous gcloud command, you execute this command, to use this specific image:
gcloud compute instances create demo \
--image ubuntu-1804-bionic-v20190204 \
--image-project ubuntu-os-cloud
Or this one, to use the default image under the project ubuntu-os-cloud and family ubuntu-1804-lts:
gcloud compute instances create demo \
--image-family ubuntu-1804-lts \
--image-project ubuntu-os-cloud
https://github.com/khushbuparakh/gcp/blob/master/instance.py
You can use this. It creates a bucket. Upload your private file to the bucket. Create an image using that file and spin up an instance
Related
When I do something in GCP console (by clicking in GUI), I imagine some gcloud command is executed underneath. Is it possible to view this command?
(I created a notebooks instance on Vertex AI and wanted to know what exactly I should put after gcloud notebooks instances create... to get the same result)
I think it's not possible to view a gcloud command from GUI.
You should test your gcloud command to create another instance alongside the current with all the needed parameters.
When the 2 instances are the same, you know that your gcloud command is ready.
The documentation seems to be clear and complete for this :
https://cloud.google.com/vertex-ai/docs/workbench/user-managed/create-new#gcloud
If it's possible for you, you can also think about Terraform to automate this creation for you with a state management.
Try this for a Python based User Managed Notebook (GUI version of Python instance is using the base image as boot disk, which does not containg Pythong.
The Python suite is installed explicitly via Metadata parameters):
export NETWORK_URI="NETWORK URI"
export SUBNET_URI="SUBNET URI"
export INSTANCE_NAME="instance-name-of-your-liking"
export VM_IMAGE_PROJECT="deeplearning-platform-release"
export VM_IMAGE_FAMILY="common-cpu-notebooks-debian-10"
export MACHINE_TYPE="n1-standard-4"
export LOCATION="europe-west3-b"
gcloud notebooks instances create $INSTANCE_NAME \
--no-public-ip \
--vm-image-project=$VM_IMAGE_PROJECT \
--vm-image-family=$VM_IMAGE_FAMILY \
--machine-type=$MACHINE_TYPE \
--location=$LOCATION \
--network=$NETWORK_URI \
--subnet=$SUBNET_URI \
--metadata=framework=NumPy/SciPy/scikit-learn,report-system-health=true,proxy-mode=service_account,shutdown-script=/opt/deeplearning/bin/shutdown_script.sh,notebooks-api=PROD,enable-guest-attributes=TRUE
To get a list of Network URIs in your project:
gcloud compute networks list --uri
To get a list of Subnet URIs in your project:
gcloud compute networks subnets list --uri
Put the corresponding URIs in between the quotation marks in the first two variables:
export NETWORK_URI="NETWORK URI"
export SUBNET_URI="SUBNET URI"
Name the instance (keep the quotation marks):
export INSTANCE_NAME="instance-name-of-your-liking"
When done copy paste the complete block in your Google Cloud Shell (assuming you are in a correct project).
To additionally enable secure boot (which is a thick box in the GUI setup):
gcloud compute instances stop $INSTANCE_NAME
gcloud compute instances update $INSTANCE_NAME --shielded-secure-boot
Hope it works for you, as it does for me.
I'm currently using windows VM instance in GCP(project1), i want to create same VM with data in my another project(project2)(project in another gmail account). The project2 have access to my project1 which i had setup IAM role.
So what the next step to copy the instance between the project1 to project2 (without loosing the data)?
You can copy a VM instance from one project to another either using a custom image or using disk.
Method 1: Using Custom image
You have to use the following steps to create a mirror VM in another project:
In your source project, create a snapshot of the VM's boot disk, using one of the below command:
$ gcloud compute snapshots create <SNAPSHOT_NAME> --source-disk <SOURCE_DISK> --source-disk-zone <SOURCE_DISK_ZONE>
Create a custom image from the snapshot using the following command:
$ gcloud compute images create <IMAGE_NAME> --source-snapshot=<SOURCE_SNAPSHOT> [--storage-location=<LOCATION>]
In your destination project, create a VM from the custom image using the following command:
$ gcloud compute instances create <VM_NAME> --image-project <IMAGE_PROJECT> [--image <IMAGE> | --image-family <IMAGE_FAMILY>]
Method 2: Using disk
In your source project, create a snapshot of the VM's boot disk, using one of the below command:
$ gcloud compute snapshots create <SNAPSHOT_NAME> --source-disk <SOURCE_DISK> --source-disk-zone <SOURCE_DISK_ZONE>
Then create a disk in the destination project with --source-snapshot:
$ gcloud compute disks create <DISK_NAME> --source-snapshot <SOURCE_SNAPSHOT> --project <destination-project>
Create a VM based on the new disk from step 2
$ gcloud compute instances create <VM_NAME> --project <destination-project> --disk name=<DISK_NAME>,boot=yes
Refer Copying VMs between projects for information.
The following works great - creating VM from source image and additional persistent disk(s).
gcloud compute instances create ${INSTANCE_NAME} \
--image-project ${PROJECT_NAME} \
--image ${BASE_IMAGE_NAME} \
--zone=${ZONE_NAME} \
--create-disk=size=128GB,type=pd-balanced,name=${INSTANCE_NAME}-home,device-name=homedisk
The following, however, creates a VM BUT no additional disk(s) are created.
gcloud beta compute instances create ${INSTANCE_NAME} \
--source-machine-image ${BASE_IMAGE_NAME} \
--zone=${ZONE_NAME} \
--create-disk=size=128GB,type=pd-balanced,name=${INSTANCE_NAME}-homedisk,device-name=homedisk
The documentation for the command does not suggest that --source-machine-image and --create-disk cannot work in tandem. The property overrides when creating a VM from machine image suggests that any of the properties can be overridden.
Any insights as to what might be going on?
the problem here is with the --source-machine-image ${BASE_IMAGE_NAME} flag because your BASE_IMAGE_NAME must already have the desired additional disk, that is why it is not being created, because it is creating everything from the BASE_IMAGE_NAME which does not have an additional disk, try it by creating a new Machine image with the desired additional disk attached and then run your gcloud beta compute instances create again (the second command you have) and confirm that it creates the instance based on that Machine image including the additional disk.
If you need to create a new instance with 1 additional disk you should use (your first command) --image ${NAME} --image-project ${PROJECT}
So --source-machine-image and --image ... --image-project are very different.
Here is the documentation for Machine images which may explain this better.
https://cloud.google.com/compute/docs/machine-images
I'm working my way through GCP ACE material, and when creating a compute instance you can do the following:
gcloud compute instance create --machine-type=f1-micro mylovelyVM
This is all good, but I want to know how to override region and zone in the same command, I know I can just do config set the region and zone but would be nice to know if it's possible in the one line command.
I have tried the following to no avail :
gcloud compute instances create --machine-type=f1-micro mylovelyVM --region us-west2 --zone us-west2-b
gcloud compute instances create --region us-west2 --zone us-west2-b --machine-type=f1-micro mylovelyVM
I have tried variations on this.
I have been using the -h and --help commands to try and work out at what point to specify the region, but so far no luck.
Error message is --region (did you mean '--reservation'?) for every iteration, which leads me to think its not expecting --region flag at that point.
Documentation states the flow should be gcloud <global flags> <service/product> <group/area> <command> <flags> <parameters>
In which case the command should be
gcloud compute --region us-west2 --zone us-west2-b instances create --machine-type=f1-micro mylovelyVM
Is there a limit on chaining overrides or am I doing this wrong?
You create instances in zones not regions. Regions comprise multiple zones and, while some other services are regional, (compute engine) instances must be placed in zones.
So, use only --zone=.. and not --region=...
The documentation is good but gcloud compute instances create has many flags and so it can be confusing:
https://cloud.google.com/sdk/gcloud/reference/compute/instances/create
Using Cloud Console, you can try permutations of the command and, at the bottom, there's an option to have Console show you the equivalent REST or Cloud SDK (gcloud) command. This is helpful.
IMO, it is good practice to specify e.g. --zone, --project and other flags on the command-line rather than use config. It's more typing but it's more explicit and can avoid errors that result from implicit (assumed) values from config.
Best wishes for your learning!
Update
gcloud compute instances create mylovelyvm \
--zone=us-west2-b \
--machine-type=f1-micro \
--project=${PROJECT}
Created [https://www.googleapis.com/compute/v1/projects/${PROJECT}/zones/us-west2-b/instances/mylovelyvm].
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
mylovelyvm us-west2-b f1-micro 10.168.0.2 35.236.23.189 RUNNING
Marking Daz's answer as correct, though I did find a way without specifying project :
gcloud compute instances create my-lovelyVM --zone=us-west2-b --machine-type=f1-micro
In AWS I will create an AMI, copy to other regions and make them all public. My customers can then choose from Community AMIs.
I have been trying to replicate this workflow in GCP and I found that GCP does not have an option of community images. And you cannot make it 'public' either. But you can use gcloud compute images export command to export an image to an external file and upload it to a bucket.
But how to use this to create an instance? I checked console to 'create VM Instance' but it does not have an option to upload or choose from drive. Only public and custom images already in your account.
To share custom image publicly.
Make custom image public using below command
gcloud compute images add-iam-policy-binding [image-name] \
--member='allAuthenticatedUsers' \
--role='roles/compute.imageUser'
Get public URL of custom image
gcloud compute images list --uri | grep [image-name]
It will be in https://www.googleapis.com/compute/v1/projects/[project-name]/global/images/[image-name] format
Create VM using public image URL
gcloud beta compute instances create instance-1 --zone=us-central1-a \
--machine-type=n1-standard-1 --subnet=default \
--image=https://www.googleapis.com/compute/v1/projects/[project-name]/global/images/[image-name] \
--boot-disk-size=10GB --boot-disk-device-name=instance-1
For details, gcp manage images here