It doesn't seem possible to create an API Gateway config for a gateway i've created using:
gcloud api-gateway apis create test-api --project=acme-prd
Then the following command fails
gcloud api-gateway api-configs create 01 \
--api=test-api --openapi-spec=./acme-web-gateway-v2.yaml \
--project=acme-prd --backend-auth-service-account=svc-owner#acme-prd.iam.gserviceaccount.com
With the error:
ERROR: (gcloud.api-gateway.api-configs.create) FAILED_PRECONDITION: API Gateway Management Service Agent does not have permission to create Service Configs for Service "test-api-3qz6mxqfw7klr.apigateway.acme-prd.cloud.goog", or the Service does not exist.
Noting the service account svc-owner#acme-prd.iam.gserviceaccount.com has Owner privileges on the project.
Is there something I am missing? This is preventing a Terraform deployment. I've used gcloud commands to demonstrate the issue.
Also of note, this does not work in the GCP UI either. :(
Permissions granted to the account being used:
Cheers
KH
To resolve this, you will need to ensure that the Service Agent account has the necessary permissions for the specified service. Check API Gateway Service Account and verify if it has “Service Account User '' role associated with it.The apigateway.apis.create should have owner/editor permissions.
Check the Google Cloud Console or by using command gcloud services list to see if the Gateway API, Service Management API, Service Control API are enabled because these api are prerequisites.You will need to enable it if it is not already enabled.you enable by using below commands:
gcloud services enable apigateway.googleapis.com
gcloud services enable servicemanagement.googleapis.com
gcloud services enable servicecontrol.googleapis.com
Attaching documents for creating an api, Gateway API access , Troubleshooting for your reference.
Edit-1:
I have tried to create an API Gateway config for a gateway using below steps and successfully created an api config
Create an api gateway using below command
gcloud api-gateway apis create test-api
Creating an API config using the below command.
gcloud api-gateway api-configs create 01 --api=test-api --openapi-spec=openapi2-functions.yaml --project=project-id
Output is
waiting for API Config [01] to be created for API [test-api]...done.
I have taken openapi2-functions. Yaml file for this doc. Can you check if your yaml files has any mistakes.
The image below has the api config that i have created.
I have followed this guide, can try to create an api gateway using this and let me know if you have an issues.
Related
I have created an API Gateway and created an API config as well associated with it which lists a set of APIs. Now I want to add/modify a set of APIs. How do I update the same API config of this gateway?
I did it using gcloud CLI.
First create a new config using your updated open api spec file:
gcloud api-gateway api-configs create NEW_CONFIG_ID --api=MY_API --openapi-spec=openapi2-functions.yaml
Then update your api gateway with new config:
gcloud api-gateway gateways update MY-GATEWAY --api=MY-API --api-config=NEW_CONFIG_ID --location=YOUR_LOCATION
Referencing the official documentation, it states that:
NOTE: Only the name and labels may be updated on an API config.
Therefore, you would need to create a new API config if you want to add/modify a set of APIs.
The GCP documentation mentions you cannot update an existing API config, only its display name and labels, and you cannot delete an existing API config that is in use by a gateway; meaning you need to create a new API config altogether if your config file has changed.
A workaround I found for this was to:
Create new API config, point to gateway:
gcloud api-gateway api-configs create NEW_CONFIG_ID --api=MY_API --openapi-spec=openapi2-functions.yaml
gcloud api-gateway gateways update MY-GATEWAY --api=MY-API --api-config=NEW_CONFIG_ID --location=YOUR_LOCATION
Delete original config:
gcloud api-gateway api-configs delete OLD_CONFIG_ID --api=MY_API
Redeploy new API config with name of original config, using your updated config file, point to gateway:
gcloud api-gateway api-configs create OLD_CONFIG_ID --api=MY_API --openapi-spec=openapi2-functions.yaml
gcloud api-gateway gateways update MY-GATEWAY --api=MY-API --api-config=OLD_CONFIG_ID --location=YOUR_LOCATION
Delete new config:
gcloud api-gateway api-configs delete NEW_CONFIG_ID --api=MY_API
An awkward workaround but it allows for an automated redeployment of a similarly named API config file in a gateway.
I am trying to deploy some GCP resources using terraform.
Executed gcloud auth login (authenticated with my gcp
account, assigned gcp project).
Executed gcloud auth application-default login.
Assigned roles to my useraccount (user99#gmail.com) at project level and the Terraform Service Account at organisation level.
Now, when I run terraform scripts from my CLI on my local machine, I get the "Error 403: The caller does not have permissions" error.
My question is:
When running terraform commands from my local machines CLI, which account is terraform using to deploy resources (user99#gmail.com or Terraform Service Account)?
Is Terraform complaining about missing permissions for my user99#gmail.com or the Terraform Service Account?
Is there a way to check which account is being used to deploy resources on GCP?
Without changing the project on gcloud auth login, can we deploy resources in other GCP projects?
If you're running on a (e.g. local) host (i.e. that's not on GCP):
with gcloud and you've gcloud auth application-default-login then (!) Terraform should be using that user's credentials (gcloud config get-value account).
and the environment exports GOOGLE_APPLICATION_CREDENTIALS (and this correctly points to a service account's key), then the Service Account will be used.
If you're running Terraform on GCP (e.g. on Compute Engine) then the Compute Engine's service account will be automatically determined by ADCs (see below).
Google Provider Configuration Authentication
Application Default Credentials (ADCs) Finding Credentials Automatically
Is there a way to check which account is being used to deploy resources on GCP?
This simple code will show you which account is used to terraform resources:
data "google_client_openid_userinfo" "me" {
}
output "my-email" {
value = data.google_client_openid_userinfo.me.email
}
Without changing the project on gcloud auth login, can we deploy resources in other GCP projects?
Actually, you can rely on self_links that contain full path with the project itself, so you can specify different projects. However, the proper way would be to use different provider aliases for each of the project.
Reference: https://www.terraform.io/language/providers/configuration#alias-multiple-provider-configurations
I ignorantly deleted the service account to my GCP project rather than the service account to Google Calendar API and Dialogflow service account.
I'm now having issues trying to deploy my dialogflow agent through the inline code editor to Cloud Functions.
When I check the logs, I get this message:
2020-07-30 15:48:40.350 WAT
Dialogflow API
CreateCloudFunction
us-central1
bashorun.emma#gmail.com
userFacingMessage:
Default service account 'northern-timer-231210#appspot.gserviceaccount.com' doesn't exist.
Please recreate this account (for example by disabling and enabling the Cloud Functions API),
or specify a different account.;
com.google.cloud.eventprocessing.manager.api.error.DefaultServiceAccountDoesNotExistException: userFacingMessage:
Default service account 'northern-timer-231210#appspot.gserviceaccount.com' doesn't exist. Please recreate this account (for example by disabling and enabling the Cloud Functions API), or specify a different account.; Code: FAILED_PRECONDITION com.google.apps.framework.request.StatusException: <eye3 title='FAILED_PRECONDITION'/> generic::FAILED_PRECONDITION: userFacingMessage:
Default service account 'northern-timer-231210#appspot.gserviceaccount.com' doesn't exist.
Please recreate this account (for example by disabling and enabling the Cloud Functions API), or specify a different account.; com.google.cloud.eventprocessing.manager.api.error.DefaultServiceAccountDoesNotExistException: userFacingMessage:
Default service account 'northern-timer-231210#appspot.gserviceaccount.com' doesn't exist. Please recreate this account (for example by disabling and enabling the Cloud Functions API), or specify a different account.; Code: FAILED_PRECONDITION
Is it possible to retrieve back the service account or am I getting these errors as a result of a different problem?
After a service account is deleted, you can recover it between 30 days after its deletion.
To do it, you can run the following command from cloud shell:
gcloud beta iam service-accounts undelete ACCOUNT_ID
The account ID can be taken from stackdriver logging with the following filter
resource.type="service_account"
resource.labels.email_id="service-account-name"
"DeleteServiceAccount"
Hope this helps to recover your service account.
Recover App Engine or any deleted service account
You can undelete service accounts. You will need the service account's unique ID. If you don't have it, you can find it on Google Cloud Logging.
You can find Logging service here on the side menu:
Then you will need to filter by date and type service account to find the exact moment the service was deleted.
Then you can either
Option 1: Use Google Cloud Command Line
You can run the command line by installing it on your computer (https://cloud.google.com/sdk/docs/install). Or you can run it online using the Active Shell offered by Google Cloud Platform.
The command you want to run is the following.
gcloud beta iam service-accounts undelete 12345678901234567890
Option 2: Use Google Cloud API
Using curl, call the API with the following command.
You will need to change API_KEY, PROJECT_ID and SERVICE_ACCOUNT_UID for real values.
curl -X POST \
-H "Authorization: Bearer API_KEY \
-H "Content-Type: application/json; charset=utf-8" \
-d "" \
"https://iam.googleapis.com/v1/projects/PROJECT_ID/serviceAccounts/SERVICE_ACCOUNT_UID:undelete"
You can get the API_KEY from Google Cloud Command Line:
gcloud auth application-default print-access-token
Again you can either have gcloud installed on your local machine or you can use it online with the Active Shell.
I want to compare Google Cloud Run to both Google App Engine and Google Cloud Functions. The Cloud Run Quickstart: Build and Deploy seems like a good starting point.
My Application Default Credentials are too broad to use during development. I'd like to use a service account, but I struggle to configure one that can complete the quickstart without error.
The question:
What is the least privileged set of predefined roles I can assign to a service account that must execute these commands without errors:
gcloud builds submit --tag gcr.io/{PROJECT-ID}/helloworld
gcloud beta run deploy --image gcr.io/{PROJECT-ID}/helloworld
The first command fails with a (seemingly spurious) error when run via a service account with two roles: Cloud Build Service Account and Cloud Run Admin. I haven't run the second command.
Edit: the error is not spurious. The command builds the image and copies it to the project's container registry, then fails to print the build log to the console (insufficient permissions).
Edit: I ran the second command. It fails with Permission 'iam.serviceaccounts.actAs' denied on {service-account}. I could resolve this by assigning the Service Account User role. But that allows the deploy command to act as the project's runtime service account, which has the Editor role by default. Creating a service account with (effectively) both Viewer and Editor roles isn't much better than using my Application Default Credentials.
So I should change the runtime service account permissions. The Cloud Run Service Identity docs have this to say about least privileged access configuration:
This changes the permissions for all services in a project, as well
as Compute Engine and Google Kubernetes Engine instances. Therefore,
the minimum set of permissions must contain the permissions required
for Cloud Run, Compute Engine, and Google Kubernetes Engine in a
project.
Unfortunately, the docs don't say what those permissions are or which set of predefined roles covers them.
What I've done so far:
Use the dev console to create a new GCP project
Use the dev console to create a new service account with the Cloud Run Admin role
Use the dev console to create (and download) a key for the service account
Create (and activate) a gcloud configuration for the project
$ gcloud config list
[core]
account = {service-account-name}#{project-id}.iam.gserviceaccount.com
disable_usage_reporting = True
project = {project-id}
[run]
region = us-central1
Activate the service account using the downloaded key
Use the dev console to enable the Cloud Run API
Use the dev console to enable Container Registry→Settings→Container Analysis API
Create a sample application and Dockerfile as instructed by the quickstart documentation
Run gcloud builds submit --tag gcr.io/[PROJECT-ID]/helloworld
...fails due to missing cloud build permissions
Add the Cloud Build Editor role to service account and resubmit build
...fails due to missing storage permissions. I didn't pay careful attention to what was missing.
Add the Storage Object Admin role to service account and resubmit build
...fails due to missing storage bucket permissions
Replace service account's Storage Object Admin role with the Storage Admin role and resubmit build
...fails with
Error: (gcloud.builds.submit) HTTPError 403:
<?xml version='1.0' encoding='UTF-8'?>
<Error>
<Code>AccessDenied</Code>
<Message>Access denied.</Message>
<Details>
{service-account-name} does not have storage.objects.get access to
{number}.cloudbuild-logs.googleusercontent.com/log-{uuid}.txt.</Details>
</Error>
Examine the set of available roles and the project's automatically created service accounts. Realize that the Cloud Build Service Account role has many more permissions that the Cloud Build Editor. This surprised me; the legacy Editor role has "Edit access to all resources".
Remove the Cloud Build Editor and Storage Admin roles from service account
Add the Cloud Build Service Account role to service account and resubmit build
...fails with the same HTTP 403 error (missing get access for a log file)
Check Cloud Build→History in the dev console; find successful builds!
Check Container Registry→Images in the dev console; find images!
At this point I think I could finish Google Cloud Run Quickstart: Build and Deploy. But I don't want to proceed with (seemingly spurious) error messages in my build process.
Cloud Run PM here:
We can break this down into the two sets of permissions needed:
# build a container image
gcloud builds submit --tag gcr.io/{PROJECT_ID}/helloworld
You'll need:
Cloud Build Editor and Cloud Build Viewer (as per #wlhee)
# deploy a container image
gcloud beta run deploy --image gcr.io/{PROJECT_ID}/helloworld
You need to do two things:
Grant your service account the Cloud Run Deployer role (if you want to change the IAM policy, say to deploy the service publicly, you'll need Cloud Run Admin).
Follow the Additional Deployment Instructions to grant that service account the ability to deploy your service account
#1
gcloud projects add-iam-policy-binding PROJECT_ID \
--member="serviceAccount:{service-account-name}#{project-id}.iam.gserviceaccount.com" \
--role="roles/run.developer"
#2
gcloud iam service-accounts add-iam-policy-binding \
PROJECT_NUMBER-compute#developer.gserviceaccount.com \
--member="serviceAccount:{service-account-name}#{project-id}.iam.gserviceaccount.com" \
--role="roles/iam.serviceAccountUser"
EDIT: As noted, the latter grants your service account the ability to actAs the runtime service account. What role this service account has is dependent on what it needs to access: if the only thing Run/GKE/GCE accesses is GCS, then give it something like Storage Object Viewer instead of Editor. We are also working on per-service identities, so you can create a service account and "override" the default with something that has least-privilege.
According to https://cloud.google.com/cloud-build/docs/securing-builds/set-service-account-permissions
"Cloud Build Service Account" - Cloud Build executes your builds using a service account, a special Google account that executes builds on your behalf.
In order to call
gcloud builds submit --tag gcr.io/path
Edit:
Please "Cloud Build Editor" and "Viewer" your service account that starts the build, it's due to the current Cloud Build authorization model.
Sorry for the inconvenience.
I am the owner of my newly created organization, I created a project under this organization and linked it to the organization billing account where I have 1000$ in credits. Through the web UI, I am able to spin up clusters, VMs, networks... But when I want to do so through gcloud, I am getting permissions denied. E.g.:
$ gcloud compute networks list
API [compute.googleapis.com] not enabled on project [XXX].
Would you like to enable and retry (this will take a few minutes)?
(y/N)? y
ERROR: (gcloud.compute.networks.create) PERMISSION_DENIED: The caller does not have permission
but I can see in the web UI GCP that the API is clearly enabled (and can be used), it's just the gcloud not letting me work with them. The account under gcloud is exactly the same I am using in the web console - validated by gcloud auth list and:
$ gcloud config configurations describe myproject
is_active: true
name: myproject
properties:
compute:
region: europe-west1
zone: europe-west1-b
core:
account: <my-email>
project: <the-project-I-want>
or
$ gcloud services list
ERROR: (gcloud.services.list) User [<myusername>] does not have permission to access project [myproject] (or it may not exist): The caller does not have permission
It works totally fine with a different account (and different organization/projects), but I didn't set up that one in the past. What should I do? Thanks a lot!
UPDATE:
After gcloud init, at least the gcloud services list started to work. But the rest did not:
$ gcloud services list
NAME TITLE
bigquery-json.googleapis.com BigQuery API
cloudapis.googleapis.com Google Cloud APIs
clouddebugger.googleapis.com Stackdriver Debugger API
cloudtrace.googleapis.com Stackdriver Trace API
compute.googleapis.com Compute Engine API
container.googleapis.com Kubernetes Engine API
containerregistry.googleapis.com Container Registry API
datastore.googleapis.com Cloud Datastore API
logging.googleapis.com Stackdriver Logging API
monitoring.googleapis.com Stackdriver Monitoring API
oslogin.googleapis.com Cloud OS Login API
pubsub.googleapis.com Cloud Pub/Sub API
servicemanagement.googleapis.com Service Management API
serviceusage.googleapis.com Service Usage API
sql-component.googleapis.com Cloud SQL
storage-api.googleapis.com Google Cloud Storage JSON API
storage-component.googleapis.com Google Cloud Storage
$ gcloud compute networks create testing-net --subnet-mode=custom '--description=Network to host testing kubernetes cluster'
API [compute.googleapis.com] not enabled on project [{PROJECT_ID}].
Would you like to enable and retry (this will take a few minutes)?
(y/N)? y
ERROR: (gcloud.compute.networks.create) PERMISSION_DENIED: The caller does not have permission
^ the PROJECT_ID above shows my organization's ID, not the actual project under this org.
So the problem was that I used the wrong project_id when gcloud config set project and gcloud defaulted to organization for some reason.
So I had to find correct project id using gcloud projects list and then use gcloud config set project {PROJECT-ID} (not the project name!)
gcloud init - if you wanted to switch gcloud to work between projects which will configure its settings to point to the right project.