gcloud - ERROR: (gcloud.app.deploy) Permissions error fetching application - google-cloud-platform

I am trying to deploy node js app on google cloud but getting following error -
Step #1: ERROR: (gcloud.app.deploy) Permissions error fetching application [apps
/mytest-240512]. Please make sure you are using the correct project ID and that
you have permission to view applications on the project.
I am running following command -
gcloud builds submit . --config cloudbuild.yaml
My cloudbuild.yaml file looks like -
steps:
#install
- name: 'gcr.io/cloud-builders/npm'
args: ['install']
#deploy
- name: 'gcr.io/cloud-builders/gcloud'
args: ['app', 'deploy']

The default Cloud Build service account does not allow access to deploy App Engine. You need to enable the Cloud Build service account to perform actions such as deploy.
The Cloud Build service account is formatted like this:
[PROJECT_NUMBER]#cloudbuild.gserviceaccount.com
Go to the Google Cloud Console -> IAM & admin -> IAM.
Locate the service account and click the pencil icon.
Add the role "App Engine Deployer" to the service account.
Wait a couple of minutes for the service account to update globally and then try again.

I had this same error today and the way I resolve it was by running: $ gcloud auth login on the console.
This will open a new browser tab for you to login with the credentials that has access to the project you're trying to deploy.
I was able to deploy to gcloud after that.
ps.: I'm not sure this is the best approach, but I'm leaving this as a possible solution as this is how I usually go around this problem. Worst case, I'll stand corrected and learn something new.

The most common way to deploy an app to App Engine is to use gcloud app deploy ....
When you use gcloud app deploy against App Engine Flex, the service uses Cloud Build.
It's entirely possible|reasonable to use Cloud Build to do your deployments too, it's just more involved.
I've not tried this but I think that, if you wish to use Cloud Build to perform the deployment, you will need to ensure that the Cloud Build service account has permissions to deploy to App Engine.
Here's an example of what you would need to do, specifically granting Cloud Build's service account the correct role.

Two commands can handle the perms needed (run in your terminal if you have gcloud sdk installed and authenticated or run in cloud shell for your project):
export PROJECT_ID=[[put your project id here]]
export PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format="value(projectNumber)")
gcloud iam service-accounts add-iam-policy-binding ${PROJECT_ID}#appspot.gserviceaccount.com \
--member=serviceAccount:${PROJECT_NUMBER}#cloudbuild.gserviceaccount.com \
--role=roles/iam.serviceAccountUser \
--project=${PROJECT_ID}
```
gcloud projects add-iam-policy-binding ${PROJECT_ID} \
--member=serviceAccount:${PROJECT_NUMBER}#cloudbuild.gserviceaccount.com \
--role=roles/appengine.appAdmin

Related

App Engine – Restrict deploy to specific service

We have two services running on Google App Engine.
We would like to restrict deployment to only specific users to the default (prod) target, but allow any devs to deploy to dev target.
Can't figure out the IAM conditions for it.
App engine doesn't seem to be an official resource type here https://cloud.google.com/iam/docs/conditions-resource-attributes#resource-name
and it's consistent with the service filter dropdown
I've tried using the name i get from gcloud app services describe dev:
resource.name == 'apps/my-project/services/dev'
Bt that doesn't seem to work either, it just gives access denied so guessing that's not the right resource name filter.
Is there a way to limit this as above?
App Engine permissions are granted at the project level and cannot be filtered for each different service of the application.
There is an open feature request https://issuetracker.google.com/115904598 to allow specific deployments of versions that I recommend you to star and follow.
Separating your prod and dev environments (I understand that this can be inconvenient sometimes) in different GCP projects could be the only viable alternative for the time being.
AFAIK, you can't restrict permissions to deploy to specific service because users can create custom services per GCP account.
Two options that I can suggest:
Create a different GCP project for prod. If you're using CLI, the prod devs can simply change the GCP project and deploy.
Use CICD with Cloud Build, and only grant merge access to prod branch to prod devs. No dev in this case would need access to your GCP projects.
For anyone still looking for options, it is possible to accomplish deployment isolation via gcloud config. It allows the creation of named configuration for each environment, for example.
Configuration governs the behavior of gcloud CLI.
Usage example:
Deploy to DEV (default) environment:
deploy:
image: google/cloud-sdk:alpine
stage: deploy
environment: Development
script:
- cp $GAE_ENV_VARIABLES ./env_variables.yaml
- echo $GAE_SERVICE_ACCOUNT > /tmp/$CI_PIPELINE_ID.json
- gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json
- gcloud app deploy app.yaml --project $GCP_PROJECT_ID --version $CI_COMMIT_SHORT_SHA --image-url=us.gcr.io/$GCP_PROJECT_ID/$IMAGE_NAME:$CI_COMMIT_REF_NAME-$CI_COMMIT_SHORT_SHA
Deploy to TEST environment :
you can make it a requirement to have a specific service account to create/activate this environment and even a dedicated app.yaml like app_test.yaml with different service name.
deploy_test:
image: google/cloud-sdk:alpine
stage: deploy
environment: Test
only:
- master
script:
- cp $GAE_ENV_VARIABLES ./env_variables.yaml
- echo $GAE_SERVICE_ACCOUNT > /tmp/$CI_PIPELINE_ID.json
- gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json
- gcloud config configurations create test --quiet --project $GCP_PROJECT_ID
- gcloud config configurations activate test --quiet --project $GCP_PROJECT_ID
- gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json #activate service account under the new configuration
- gcloud app deploy app_test.yaml --project $GCP_PROJECT_ID --version $CI_COMMIT_SHORT_SHA --promote --image-url=us.gcr.io/$GCP_PROJECT_ID/$IMAGE_NAME:$CI_COMMIT_REF_NAME-$CI_COMMIT_SHORT_SHA

What IAM permissions do I need for a service key that can run "gcloud builds submit" and "gcloud run deploy"?

I'm trying to figure out the absolute minimum set of IAM permissions I need to assign to a service key that will be used to run the following commands:
gcloud builds submit --tag gcr.io/MYPROJECT/MYNAME
gcloud run deploy --allow-unauthenticated --platform=managed --image gcr.io/MYPROJECT/MYNAME ...
I've had a lot of trouble figuring out IAM, so the more detailed instructions anyone can give me the better!
Here's what I've figured out so far (I ended up going with way more open permissions than I wanted): https://simonwillison.net/2020/Jan/21/github-actions-cloud-run/#google-cloud-service-key
I'm actually running these commands inside a Python script - relevant code is here: https://github.com/simonw/datasette/blob/07e208cc6d9e901b87552c1be2854c220b3f9b6d/datasette/publish/cloudrun.py#L134-L141
I understand you are running these commands with a service account, and your goal is to determine the minimal set of IAM permissions to assign to this service account so that it can build and deploy. I am going to list a set of minimal IAM roles (not IAM permissions)
To run gcloud builds submit --tag gcr.io/MYPROJECT/MYNAME, you need:
roles/cloudbuild.builds.editor to trigger the build
roles/storage.admin to push te image
To run gcloud run deploy --allow-unauthenticated --platform=managed --image gcr.io/MYPROJECT/MYNAME ... you need:
roles/run.admin (to deploy and allow allUsers to access the service)
roles/iam.serviceAccountUser (because the code will then run under a servie account, so the service account used to deploy needs to also be able to "act as" the runtime service account)

Permission error when trying to deploy to Google Cloud Run

I'm getting a permission error when trying to deploy an image to cloud run:
gcloud beta run deploy endpoints_proxy \
--image="gcr.io/endpoints-release/endpoints-runtime-serverless:1.30.0" \
--allow-unauthenticated
This is the error:
ERROR: (gcloud.beta.run.deploy) User [email-goes-here] does not have permission to access namespace [project-id-goes-here] (or it may not exist): Cloud Run does not have permission to get access tokens for the default compute service account, 1088973916567-compute#developer.gserviceaccount.com. Please give Google Cloud Run Service Agent the permission iam.serviceAccounts.getAccessToken on the default compute service account.
My account has owner and editor permissions, I even tried attaching the Cloud Run Service Agent role.
I also tried adding these roles to the "default compute service account" listed in the error, didn't work.
The error message is very misleading, the error occurs because the Cloud Run Service Agent was missing.
After creating the following service account:
Name: service-<account-id>#serverless-robot-prod.iam.gserviceaccount.com
Role: Google Cloud Run Service Agent
The problem got solved. Looks like Cloud Run needs this service account to work, so don't ever delete it :)
You should enable permissions to the Cloud Build service account.
It won't work unless you do so.
Google Cloud Console > Cloud Build > Cloud Build Settings > enable access to the GCP services that you are using.
(see the images attached)
Cloud Build Panel
Cloud Build Panel - enable services that you want
Also, just make sure billing hasn't expired. When mine did, I got these opaque errors (such as this one) that didn't reference to billing in any way.
If the service-<project-id>#serverless-robot-prod.iam.gserviceaccount.com is not present in GCP console, just run this command:
gcloud projects add-iam-policy-binding --member=serviceAccount:service-<project-id>#serverless-robot-prod.iam.gserviceaccount.com --role=roles/run.serviceAgent <project-id>

(gcloud.projects.list) PERMISSION_DENIED

I have a machine that needs to run the following gcloud command:
gcloud projects list --format=json
The output error that gives me:
ERROR: (gcloud.projects.list) PERMISSION_DENIED: Request had insufficient authentication scopes.
Is pretty obvious that the current configuration and account set for the machine do not have the permissions.
Funny that when I use gcloud compute instances list --project=<project_ID> --format=json
It gives me a list of the machines listed in the project I specify.
I enabled the Google Resource Manager API
I even created some service account credentials and activated them in the machine. Still the same error.
In the SDK documentation there is no reference on how to enable credentials to see other projects
Anyone had this issue before? I saw outdated questions whose solutions didn't work out for me.
Edit
I should mention that the machine in question is a GCE instance and there is no way (unless I install manually the SDK, which is a mess I am not going to get into) to update the SDK.
Cloud API access scopes are set manually and there is no mention of the "Resource Manager" and i can't seem to add or remove any new API Accesses
According to this document, gcloud projects list shows all the active projects were the account has the Owner, Editor or Viewer project level role. As long as the service account you activated in your instance has one of those roles in a gcp project, you should be able to run the command.
For example, from your cloud shell grant the viewer role to your service account:
gcloud projects add-iam-policy-binding <your_project_id> \
--member serviceAccount:<your_service_account> --role roles/viewer
Activate the service account in your instance using the json key file:
gcloud auth activate-service-account --key-file=/path/key.json
Run the projects list command:
gcloud projects list --format=json

Using Google Cloud Source Repositories with service account

Is it possible to access a Google Cloud Source Repository in an automated way, i.e. from a GCE instance using a service account?
The only authentication method I am seeing in the docs is to use the gcloud auth login command, which will authenticate my personal user to access the repo, not the machine I am running commands from.
If you want to clone with git rather than running through gcloud, you can run:
git config --global credential.helper gcloud.sh
...and then this will work:
git clone https://source.developers.google.com/p/$PROJECT/r/$REPO
On GCE vms running
gcloud source repos clone default ~/my_repo
should work automatically without extra step of authentication, as it will use VMs service account.
If you running on some other machine you can download from https://console.cloud.google.com service account .json key file and activate it with
gcloud auth activate-service-account --key-file KEY_FILE
and then run the above clone command.
In case somebody like me was trying to do this as part of Dockerfile, after struggling for a while I've only managed to get it to work like this:
RUN gcloud auth activate-service-account --key-file KEY_FILE ; \
gcloud source repos clone default ~/my_repo
As you can see, having it to be part of the same RUN command was the key, otherwise it kept failing with
ERROR: (gcloud.source.repos.clone) You do not currently have an active account selected.
Enable access to the "Cloud Source Repositories" Cloud API for the instance. You should do this while creating or editing the instance in the Admin console
From a shell inside the instance, execute gcloud source repos clone <repo_name_in_cloud_source> <target_path_to_clone_into>
If you are running on GCE, take advantage of the new authentication method that needs fewer lines of code.
When creating your VM instance, under "Access & Security," set "Cloud Platform" to "Enabled."
Then the authentication code is this simple:
from oauth2client.client import GoogleCredentials
credentials = GoogleCredentials.get_application_default()
http = credentials.authorize(httplib2.Http())
See
https://developers.google.com/identity/protocols/application-default-credentials