When trying to create a commitment
gcloud beta compute commitments create commitment-hyper-gpus-2 --region=europe-west1 --resources=vcpu=16,memory=104 --resources-accelerator=type=nvidia-tesla-p100,count=1 --plan 12-month --reservation=hyper-gpus-p100-2 --reservation-zone=europe-west1-d --machine-type=n1-highmem-16 --accelerator=type=nvidia-tesla-p100,count=1 --vm-count=1
Its giving me the below error.
ERROR: (gcloud.beta.compute.commitments.create) Some requests did not succeed:
- Quota 'COMMITTED_NVIDIA_P100_GPUS' exceeded. Limit: 0.0 in region europe-west1
But, there is no quota with name COMMITTED_NVIDIA_P100_GPUS
You will need to request a quote increase for all GPUs (global).
Make sure that you have setup billing and a payment method
Go to the Google Cloud Console Quotas page: https://console.cloud.google.com/iam-admin/quotas
Under "Metric" select GPUs (all regions). This will limit the number of services displayed.
Select (don't click on) Compute Engine APIs.
Click on "EDIT QUOTAS".
Fill out the form.
Wait to either be contacted or approved or declined.
Related
Using a GCP account that started as free, but does have billing enabled, I can't create a managed notebook and get the following popup error:
Quota exceeded for quota metric 'Create Runtime API requests' and limit 'Create Runtime API requests per minute' of service 'notebooks.googleapis.com' for consumer 'project_number:....'
Navigating to Quotas --> Notebook API --> Create Runtime API requests per minute
Edit Quota: Create Runtime API requests per minute
Current limit: 0
Enter a new quota limit between 0 and 0.
0 doesn't work..
Is there something that I can do, or should have done already to increase this quota?
TIA for any help.
Managed notebooks is still pre-GA and is currently unavailable to the projects with insufficient service usage history.
You can wait for the GA of the service or use a project with more service usage.
Trying to get an alert when the GCE VM is in down state by creating Alerting Policy.
Metric: compute.googleapis.com/instance/uptime
Resource : VM instance
And made the configuration that in order to trigger an alert when this condition is absent for 3 minutes.
To simulate this above behavior , I have stopped the VM but it is not triggering an alert , meanwhile data is not visible in graph of the alerting policy
Have attached trigger configuration
None of the metrics are giving reliable alerts when the VM is in stopped state,which are compute.googleapis.com/instance/uptime or uptime of the monitoring agent or cpu utilization metrics until you create alerting poilicy with MQL - Monitoring Query language.
"metrics associated with TERMINATED or DELETED Google Cloud resources are not considered for metric-absence policies. This means you can't use metric-absence policies to test for TERMINATED or DELETED Google Cloud VMs."
https://cloud.google.com/monitoring/alerts/types-of-conditions#metric-absence
So as per the above statement we cannot use metic absence policy for stopped vm - As It goes to terminated state after it stopped for sometime.The reason is , it calculates the instance stop time only when it becomes running state again.
But when you configure the same condition with MQL with the same set of metrics , Metric-absence policies works without any issues.
Sample:
Instead of configuring the condition by selecting resource & metric , go to Query Editor and type the below query for getting the alert when the Development environment VM is not in running state for 3 minutes.
fetch gce_instance
| metric 'compute.googleapis.com/instance/uptime'
| filter (metadata.user_labels.env == 'dev')
| group_by 1m, [value_uptime_aggregate: aggregate(value.uptime)]
| every 1m
| absent_for 180s
Not sure this is the bug or not , but this is limitation when we configure the alerting condition in a traditional way and we can resolve this by leveraging MQL.
Behavior you're describing is unusual.
I reproduced your case and created the exact alerting policy using the same metric compute.googleapis.com/instance/uptime with the same settings. I forwarded all alerts to my e-mail.
Unfortunatelly I wasn't able to reproduce this behavior. After playing with various settings (agregation, absence time) and I was getting alerting emails.
Try maybe setting the alerting policy again. If your goal is just to monitor the state of the VM (responding or no) then you can use any other metrics such as cpu usage which will be absent when the VM is off (or unresponsive).
Finally you can try installing monitoring agent on your VM which will give you more metrics available thus more information on the machine.
Have a look at how to manage alerting policies documentation which may be usefull to you. Additionally this documentation describes alerting policies types and how to choose apropriate one for you use case.
Ultimately try creating another VM and set up alerting policy for it. If that doesn't work your best shot is to go to Google IssueTracker and file a new bug report and it will be handled by the product team - however this may take some (or a lot) of time depending on the issue.
I have been trying to increase Quota for Google Cloud Platform(GCP) Compute Engine API for a Location and it is not allowing me to Edit or Even select the location.
I have tried the same thing before few months back and it was properly working then. I just created a new project and tried the same thing.
I do have the Owner Permission assigned to me.
After concluding that you are in Free Tier, that is part of constraints.
Your free trial credit applies to all Google Cloud resources, with the following exceptions:
You can't have more than 8 cores (or virtual CPUs) running at the same time.
You can't add GPUs to your VM instances.
You can't request a quota increase. For an overview of Compute Engine quotas, see Resource quotas.
You can't create VM instances that are based on Windows Server images.
You must upgrade your account to perform any of the actions in the preceding list.
Upgrading to a paid account:
https://cloud.google.com/free/docs/gcp-free-tier#how-to-upgrade
Free Tier conditions:
https://cloud.google.com/free/docs/gcp-free-tier
Update: To be able to increase Quotas or Submit Quota Increase, you need to:
For New Project need to wait for 48hrs
You need to have Billing Enabled (Enable it by going into top-left gift icon and following along to Enable Billing in GCP)
I would like to try PEGASUS to summarize article.
https://github.com/google-research/pegasus
I followed this instruction.
https://github.com/google-research/pegasus/tree/f76b63c2886748f7f5c6c9fb547456d8c6002562#setup
I checked the region which I can use NVIDIA Tesla V100 and I decided to use us-central1-a
https://cloud.google.com/compute/docs/gpus
I used this command.
gcloud compute instances create pegasustest --zone=us-central1-a
--machine-type=n1-highmem-8 --accelerator type=nvidia-tesla-v100,count=1
--boot-disk-size=500GB --image-project=ml-images --image-family=tf-1-15
--maintenance-policy TERMINATE --restart-on-failure
I got this error message.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
- The zone 'projects/covid19agent/zones/us-central1-a' does not have enough
resources available to fulfill the request.
Try a different zone, or try again later.
I took 3 hours and tried again, but I got the same result.
So, I changed the region from us-central1-a to asia-east1-c.
I used this command.
gcloud compute instances create pegasustest --zone=asia-east1-c
--machine-type=n1-highmem-8 --accelerator type=nvidia-tesla-v100,count=1
--boot-disk-size=500GB --image-project=ml-images --image-family=tf-1-15
--maintenance-policy TERMINATE --restart-on-failure
Then I got this error message.
WARNING: Some requests generated warnings:
- Disk size: '500 GB' is larger than image size: '10 GB'.
You might need to resize the root repartition manually
if the operating system does not support automatic resizing.
See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd
for details.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
- Quota 'GPUS_ALL_REGIONS' exceeded. Limit: 0.0 globally.
Is it impossible for me to try PEGASUS? And, does it cost too much to try PEGASUS?
Let's start with the first issue. Have a look again at the error message:
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
- The zone 'projects/covid19agent/zones/us-central1-a' does not have enough resources available to fulfill the request. Try a different
zone, or try again later.
When you start an instance it requests resources like vCPU, memory, GPU and if there's not enough resources available in the zone you'll get such message, more information available in the documentation:
If you receive a resource error (such as ZONE_RESOURCE_POOL_EXHAUSTED
or ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS) when requesting new
resources, it means that the zone cannot currently accommodate your
request. This error is due to Compute Engine resource obtainability,
and is not due to your Compute Engine quota.
Resource availability are depending from users requests and therefore are dynamic.
There are a few ways to solve this issue:
Wait for a while and try to start your VM instance again (as you tried, but fruitless this time).
Move your instance to another zone (as you did).
Reserve resources for your VM by following documentation to avoid such issue in future:
Create reservations for Virtual Machine (VM) instances in a specific
zone, using custom or predefined machine types, with or without
additional GPUs or local SSDs, to ensure resources are available for
your workloads when you need them. After you create a reservation, you
begin paying for the reserved resources immediately, and they remain
available for your project to use indefinitely, until the reservation
is deleted.
Now, let's have a look at the second issue. Have a look again at this error message:
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
- Quota 'GPUS_ALL_REGIONS' exceeded. Limit: 0.0 globally.
More information about quotas you can find in the documentation.
To solve this issue you should follow steps below:
Ensure that billing is enabled for your project.
Request an increase in quota:
Go to the Quotas page.
In the Quotas page, select the quotas you want to change.
Click the Edit Quotas button on the top of the page.
Check the box of the service you want to edit.
Fill out your name, email, and phone number, and click Next.
Enter your request to increase your quota, and click Next.
Submit your request.
A request to decrease quota is rejected by default. If you must reduce your quota, reply to the support email with an explanation of
your requirements. A support representative from the Compute Engine
team will respond to your request within 24 to 48 hours.
You're not able to request an increase in quota if you use 12-month, $300 free trial because of the limitations:
Your free trial credit applies to all Google Cloud resources, with the
following exceptions:
You can't have more than 8 cores (or virtual CPUs) running at the same time.
You can't add GPUs to your VM instances.
You can't request a quota increase. For an overview of Compute Engine quotas, see Resource quotas.
You can't create VM instances that are based on Windows Server images.
You must upgrade your account to perform any of the actions in
the preceding list.
You can estimate cost of usage with Google Cloud Pricing Calculator.
I tried to purchase additional committed use discount in GCP but I got this error.
Quota 'COMMITTED_CPUS' exceeded. Limit: 75.0 in region asia-southeast1.
Previously, I already purchase some CPU and memory successfully. I tried to search in Quota page but I can't find anything with COMMITTED_CPUS or 75 limits. Digging to commitments metric only purchase quota not cpu quota.
I also try this command but still can't find it.
gcloud compute project-info describe --project "project_name"
I can't purchase it both in console and using this command.
gcloud compute commitments create asia-southeast-1y-3 --plan 12-month --region asia-southeast1 --resources=MEMORY=106GB,VCPU=36
I've confront this issue before. I can provide you some SOP for solving this:
File a new case to increase your COMMITMENTS quota in the same region you’re seeing the problem in order to address the problem in an automated way. Each COMMITMENTS increase also increases COMMITTED_CPUS to the current quota of CPUS in the region.
To be very specific, if you need an increase for COMMITTED_CPUS, the only way is to request an increase for COMMITMENTS. To request quota for Commitments, follow these instructions:
Go to the Quota Increase page in the Google Cloud Platform Console
Expand the Quota type dropdown menu and select All quotas.
Expand the Metric dropdown menu.
Click on None to hide all quotas and then type Commitments in the search box to search for Commitments quota.
Select Commitments from the list.
Select the checkbox next to Commitments for the desired region and click the Edit Quotas button.
If prompted, fill in your first and last name, and phone number. Click Next.
Fill out the request form and click Next.
Click Submit Request to submit your request.
Once your request is received, it takes about 2 business days. Hope this helps!