So I'm trying to run a training job on google cloud's AI-platform for an image classifier written in tensorflow by the command line:
gcloud ai-platform jobs submit training my_job \
--module-name trainer.final_task \
--staging-bucket gs://project_bucket \
--package-path trainer/ \
but I keep getting the ERROR: (gcloud.ai-platform.jobs.submit.training) User [myemail#gmail.com] does not have permission to access project [my_project] (or it may not exist): Permission denied on 'locations/value' (or it may not exist).
I don't get how this is possible as I own the project on gcloud (with that e-mail address) and am even expressly linked to it on the IAM policy bindings. Has anyone experienced this before?
EXTRA INFO:
I am using gcloud as an individual, there are no organisations involved. Hence the only members linked in IAM policy bindings are me and gcloud service accounts.
The code works perfectly when trained locally (using gcloud ai-platform local train) with the same parameters.
I encountered the same problem, having an owner account have permissions denied for training a job. I had accidentally added "central1" as the server when it had to be "us-central1". Hopefully this helps!
I need little more information to be sure, but such error appears when you have different project set in Gcloud SDK. Please verify if project in gcloud config list project is the same as the project you want to use. If not please submit gcloud config set project [YOUR PROJECT]. You can verify the changes with list command again.
The issue with me was that my notebook location was in a different region and I was trying to deploy in a different region. After I changed the location to my notebook location, it worked.
Related
Due to corporate restrictions, I'm supposed to host everything on GCP in Europe. The organisation I work for, has set a restriction policy to enforce this.
When I deploy a cloud run instance from source with gcloud beta run deploy --source . --region europe-west1 it seems the command tries to store the temporary files in a storage bucket in the us, which is not allowed. The command then throws a 412 error.
➜ gcloud beta run deploy cloudrun-service-name --source . --platform managed --region=europe-west1 --allow-unauthenticated
This command is equivalent to running `gcloud builds submit --tag [IMAGE] .` and `gcloud run deploy cloudrun-service-name --image [IMAGE]`
Building using Dockerfile and deploying container to Cloud Run service [cloudrun-service-name] in project [PROJECT_ID] region [europe-west1]
X Building and deploying new service... Uploading sources.
- Uploading sources...
. Building Container...
. Creating Revision...
. Routing traffic...
. Setting IAM Policy...
Deployment failed
ERROR: (gcloud.beta.run.deploy) HTTPError 412: 'us' violates constraint 'constraints/gcp.resourceLocations'
I see the Artifact Registry Repository being created in the correct region, but not the storage bucket.
To bypass this I have to create a storage bucket first in the correct region with the name PROJECT_ID_cloudbuild. Is there any other way to fix this?
Looking at the error message indicates that the bucket is forced to be created in the US regardless of the Organisation policy set in Europe. As per this public issue tracker comment,
“Cloud build submit creates a [PROJECT_ID]_cloudbuild bucket in the
US. This will of course not work when resource restrictions apply.
What you can do as a workaround is to create that bucket yourself in
another location. You should do this before your first cloud build
submit.”
This has been a known issue and I found two workarounds that can help you achieve what you want.
The first workaround is by using “gcloud builds submit” with additional flags:
Create a new bucket with the name [PROJECT_ID]_cloudbuild in the
preferred location.
Specify non-buckets using --gcs-source-staging-dir and
--gcs-log-dir 2 ===> this flag is required as if it is not set
it will create a bucket in the US.
The second workaround is by using a cloudbuild.yaml and the “--gcs-source-staging-dir” flag:
Create a bucket in the region, dual-region or multi-region you may
want
Create a cloudbuild.yaml for storing a build artifacts
You can find an example of the YAML file in the following external
documentation, please note that I cannot vouch for its accuracy
since it is not from GCP.
Run the command :
gcloud builds submit
--gcs-source-staging-dir="gs://example-bucket/cloudbuild-custom" --config cloudbuild.yaml
Please try these workarounds and let me know if it worked for you.
I'm trying to figure out the absolute minimum set of IAM permissions I need to assign to a service key that will be used to run the following commands:
gcloud builds submit --tag gcr.io/MYPROJECT/MYNAME
gcloud run deploy --allow-unauthenticated --platform=managed --image gcr.io/MYPROJECT/MYNAME ...
I've had a lot of trouble figuring out IAM, so the more detailed instructions anyone can give me the better!
Here's what I've figured out so far (I ended up going with way more open permissions than I wanted): https://simonwillison.net/2020/Jan/21/github-actions-cloud-run/#google-cloud-service-key
I'm actually running these commands inside a Python script - relevant code is here: https://github.com/simonw/datasette/blob/07e208cc6d9e901b87552c1be2854c220b3f9b6d/datasette/publish/cloudrun.py#L134-L141
I understand you are running these commands with a service account, and your goal is to determine the minimal set of IAM permissions to assign to this service account so that it can build and deploy. I am going to list a set of minimal IAM roles (not IAM permissions)
To run gcloud builds submit --tag gcr.io/MYPROJECT/MYNAME, you need:
roles/cloudbuild.builds.editor to trigger the build
roles/storage.admin to push te image
To run gcloud run deploy --allow-unauthenticated --platform=managed --image gcr.io/MYPROJECT/MYNAME ... you need:
roles/run.admin (to deploy and allow allUsers to access the service)
roles/iam.serviceAccountUser (because the code will then run under a servie account, so the service account used to deploy needs to also be able to "act as" the runtime service account)
I'm trying to setup credentials for kubernetes on my local.
gcloud container clusters get-credentials ***** --zone **** --project elo-project-267109
This query works fine when I tried it from cloud shell, but I got this error when I tried run it from my terminal:
ERROR: (gcloud.container.clusters.get-credentials) get-credentials requires edit permission on elo-project-267109
I've tried this query from admin account as well as default service account also from new service account by assigning editor role and it still doesn't seem to work for me.
i am using macOs Mojave(10.14.6) and gcloud SDK version installed in my system is 274.0.1
i was able to resolve this issue on my local but i was actually trying to build a CI/CD from gitlab and the issue persists there, i have tried using gcloud(279.0.0) image version.
i am new to both gitlab and gcloud. i am trying to build CI/CD pipeline for the first time.
Do gcloud auth list to see which account are you logged into.
You need to login with the account which has the correct credentials to access the action that you're trying to perform.
To set the gcloud account: gcloud config set account <ACCOUNT>
It's turned out to be the image version mismatch issue on GitLab.
I am trying to deploy node js app on google cloud but getting following error -
Step #1: ERROR: (gcloud.app.deploy) Permissions error fetching application [apps
/mytest-240512]. Please make sure you are using the correct project ID and that
you have permission to view applications on the project.
I am running following command -
gcloud builds submit . --config cloudbuild.yaml
My cloudbuild.yaml file looks like -
steps:
#install
- name: 'gcr.io/cloud-builders/npm'
args: ['install']
#deploy
- name: 'gcr.io/cloud-builders/gcloud'
args: ['app', 'deploy']
The default Cloud Build service account does not allow access to deploy App Engine. You need to enable the Cloud Build service account to perform actions such as deploy.
The Cloud Build service account is formatted like this:
[PROJECT_NUMBER]#cloudbuild.gserviceaccount.com
Go to the Google Cloud Console -> IAM & admin -> IAM.
Locate the service account and click the pencil icon.
Add the role "App Engine Deployer" to the service account.
Wait a couple of minutes for the service account to update globally and then try again.
I had this same error today and the way I resolve it was by running: $ gcloud auth login on the console.
This will open a new browser tab for you to login with the credentials that has access to the project you're trying to deploy.
I was able to deploy to gcloud after that.
ps.: I'm not sure this is the best approach, but I'm leaving this as a possible solution as this is how I usually go around this problem. Worst case, I'll stand corrected and learn something new.
The most common way to deploy an app to App Engine is to use gcloud app deploy ....
When you use gcloud app deploy against App Engine Flex, the service uses Cloud Build.
It's entirely possible|reasonable to use Cloud Build to do your deployments too, it's just more involved.
I've not tried this but I think that, if you wish to use Cloud Build to perform the deployment, you will need to ensure that the Cloud Build service account has permissions to deploy to App Engine.
Here's an example of what you would need to do, specifically granting Cloud Build's service account the correct role.
Two commands can handle the perms needed (run in your terminal if you have gcloud sdk installed and authenticated or run in cloud shell for your project):
export PROJECT_ID=[[put your project id here]]
export PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format="value(projectNumber)")
gcloud iam service-accounts add-iam-policy-binding ${PROJECT_ID}#appspot.gserviceaccount.com \
--member=serviceAccount:${PROJECT_NUMBER}#cloudbuild.gserviceaccount.com \
--role=roles/iam.serviceAccountUser \
--project=${PROJECT_ID}
```
gcloud projects add-iam-policy-binding ${PROJECT_ID} \
--member=serviceAccount:${PROJECT_NUMBER}#cloudbuild.gserviceaccount.com \
--role=roles/appengine.appAdmin
I have a machine that needs to run the following gcloud command:
gcloud projects list --format=json
The output error that gives me:
ERROR: (gcloud.projects.list) PERMISSION_DENIED: Request had insufficient authentication scopes.
Is pretty obvious that the current configuration and account set for the machine do not have the permissions.
Funny that when I use gcloud compute instances list --project=<project_ID> --format=json
It gives me a list of the machines listed in the project I specify.
I enabled the Google Resource Manager API
I even created some service account credentials and activated them in the machine. Still the same error.
In the SDK documentation there is no reference on how to enable credentials to see other projects
Anyone had this issue before? I saw outdated questions whose solutions didn't work out for me.
Edit
I should mention that the machine in question is a GCE instance and there is no way (unless I install manually the SDK, which is a mess I am not going to get into) to update the SDK.
Cloud API access scopes are set manually and there is no mention of the "Resource Manager" and i can't seem to add or remove any new API Accesses
According to this document, gcloud projects list shows all the active projects were the account has the Owner, Editor or Viewer project level role. As long as the service account you activated in your instance has one of those roles in a gcp project, you should be able to run the command.
For example, from your cloud shell grant the viewer role to your service account:
gcloud projects add-iam-policy-binding <your_project_id> \
--member serviceAccount:<your_service_account> --role roles/viewer
Activate the service account in your instance using the json key file:
gcloud auth activate-service-account --key-file=/path/key.json
Run the projects list command:
gcloud projects list --format=json