When attempting to deploy a simple function, I am running into a 403 error. Here is my full yaml:
steps:
# Install Dependencies
- name: 'python'
id: Pip install
args: ['pip3', 'install', '-r', 'requirements.txt', '--user']
# Run Tests
- name: 'python'
args: ['python3', '-m', 'pytest', 'functions/folder_a/test/']
# Deploy Cloud Functions
- name: 'gcr.io/cloud-builders/gcloud'
id: 'helloFunction'
args: ['functions', 'deploy', 'hello', '--source' , 'functions/folder_a/main.py',
'--runtime' , 'python37' ,'--trigger-http']
Step one finishes fine but step two errors here.
$ gcloud info
Account: [projectname#appspot.gserviceaccount.com]
Project: [projectname]
Current Properties:
[core]
project: [projectname]
account: [projectname#appspot.gserviceaccount.com]
disable_usage_reporting: [True]
I tried adding the following permissions to the above account in IAM:
Cloud Build Service Agent
Cloud Functions Developer
Cloud Functions Service Agent
Editor
When Google Build is running, which service account is it using? How can I change the permissions to allow automatic deploying of functions?
Using Cloud Build to deploy Cloud Functions is documented in https://cloud.google.com/functions/docs/bestpractices/testing#continuous_testing_and_deployment
From that page the guidance is:
To deploy Cloud Functions, you might want to assign the Cloud Functions Developer role to the Cloud Build service account (PROJECT_NUMBER#cloudbuild.gserviceaccount.com).
If you use the Cloud Functions Developer role, you also need to grant the Cloud Functions Runtime service account (PROJECT_ID#appspot.gserviceaccount.com) the IAM Service Account User role.
The Cloud Build service account is PROJECT_NUMBER#cloudbuild.gserviceaccount.com, if you look for it in the console there's only going to be one service account with *#cloudbuild*
Related
I was trying to deploy an http triggered Cloud Function with Cloud Build using this configuration.
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args:
- beta
- functions
- deploy
- myfunction
- --source=start_shopify_installation
- --trigger-http
- --region=europe-west1
- --runtime=nodejs14
- --allow-unauthenticated
- --ingress-settings=all
- --security-level=secure-always
- --set-secrets=env_1=secret_1:latest
When I got an error saying Cloud Build could't set an IAM policy.
WARNING: Setting IAM policy failed, try:
gcloud alpha functions add-iam-policy-binding myfunction\
--region=europe-west1 \
--member=allUsers \
--role=roles/cloudfunctions.invoker
The function gets deployed and when I check in the GCP console it looks like the allUsers member has the role Cloud Functions Invoker, but it doesn't have allow unauthenticated in the Authentication column. When I go to invoke the function I get a 'missing permissions' error.
When I execute the suggested command from my Cloud Shell it works just fine. However if I fill it in as an extra step in my deployment configuration that step fails.
I think that my Cloud Build service account must be missing a role in order to make the function accessible without authetication? Currently it has these roles: Cloud Build Service Account, Cloud Functions Developer and Service Account User.
EDIT
I added the Project IAM Admin role to the Cloud Build service account and tried again.
Unfortunately, it didn't change anything.
I reproduced your error (warning) on my side and fixed it: I can see allUsers having Cloud Functions Invoker role in the function's PERMISSIONS tab.
In fact your cloud build service account needs the cloudfunctions.functions.setIamPolicy permission. So the solution is replace Cloud Functions Developer role with Cloud Functions Admin role.
Use of the --allow-unauthenticated flag modifies IAM permissions. To ensure that unauthorized developers cannot modify function permissions, the user or service that is deploying the function must have the cloudfunctions.functions.setIamPolicy permission. This permission is included in both the Owner and Cloud Functions Admin roles.
Ref: https://cloud.google.com/functions/docs/securing/managing-access-iam#at_deployment
We have separate GCP projects for "build" and "prod" environments. I would like to use Cloud Build for the "build" project to deploy a Cloud Function in the "prod" project. When following the documentation.
Notably, I added the "Cloud Functions Developer" role to the build service account in the build project and the "IAM Service Account User" role to the PROJECT_ID#appspot.gserviceaccount.com account as noted in the docs and in this question, when running a build, I get:
ERROR: (gcloud.functions.deploy) ResponseError: status=[403], code=[Forbidden], message=[The caller does not have permission]
A simplified example of my cloudbuild.yaml is
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['source', 'repos', 'clone', 'a_mirrored_repo', '--project=build-xxxx']
- name: 'gcr.io/cloud-builders/gcloud'
args: ['functions', 'deploy', 'some_function', '--trigger-http', '--runtime', 'python37', '--project', 'prod-yyyy']
I am able to deploy my fucntion to prod using the gsutil command line utility from my laptop, and I am able to use my cloudbuild.yaml to deploy this function to the build project. But I'm unsure what roles I need to assign to what accounts to enable the build project to deploy the cloudfunction to the prod project.
If you want to deploy the function in the prod project then the Cloud Build service account of the build project must be added as a "Cloud Functions Developer" in the prod project. If I understand your description correctly, then you've set it in the build project.
Specifically: in the GCP console, in the prod project, under IAM > members add a new member named [build-project-id]#cloudbuild.gserviceaccount.com and add the role "Cloud Functions Developer" to this member. At this point attempting to deploy may result in an error message:
ERROR: (gcloud.functions.deploy) ResponseError: status=[403],
code=[Forbidden], message=[Missing necessary permission
iam.serviceAccounts.actAs for prod-xxxx#appspot.gserviceaccount.com
on project prod-xxxx.
Please grant
prod-xxxx#appspot.gserviceaccount.com the
roles/iam.serviceAccountUser role.
You can do that by running
'gcloud projects add-iam-policy-binding prod-xxxx
--member=prod-xxxx#appspot.gserviceaccount.com --role=roles/iam.serviceAccountUser'
The command suggested in the error message is slightly incorrect. The correct syntax is:
gcloud projects add-iam-policy-binding prod-xxxx --member='serviceAccount:[build-project-number]#cloudbuild.gserviceaccount.com' --role='roles/iam.serviceAccountUser'
I am trying to deploy the new changes to kubernetes cluster using the Google Cloud Provider CloudBuild. Whenever I make some changes the trigger is working fine and its starting a new build but here is the issue I am getting with this cloudbuild.yaml.
cloudbuild.yaml
steps:
#step1
- name: 'gcr.io/cloud-builders/docker'
args: [ 'build', '-t', 'gcr.io/$PROJECT_ID/cloudbuildtest-image', '.' ]
#step 2
- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/$PROJECT_ID/cloudbuildtest-image']
#step 3 for testing
name: 'gcr.io/cloud-builders/kubectl'
args: ['get', 'pods']
env:
- 'CLOUDSDK_COMPUTE_ZONE=us-central1-a'
- 'CLOUDSDK_CONTAINER_CLUSTER=cloudbuild-test'
#STEP-4
images:
- 'gcr.io/$PROJECT_ID/cloudbuildtest-image'
Step 1 and 2 are working fine but the issue is with the step3 where for testing purpose I simply ran the get pods command to test if it will work or not. Here is the issue I am getting in the logs.
Running: gcloud container clusters get-credentials --project="journeyfoods-io" --zone="us-central1-a" "cloudbuild-test"
Fetching cluster endpoint and auth data.
ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Required "container.clusters.get" permission(s) for "projects/XXXX/zones/us-central1-a/clusters/cloudbuild-test".
What permissions is it looking for? Do I need to do some
authentication before running the steps or What exactly am I missing?
The steps of a Cloud Build build are executed using with the [PROJECT_NUMBER]#cloudbuild.gserviceaccount.com service account. From the Cloud Build documentation page about this topic:
When you enable the Cloud Build API, the service account is
automatically created and granted the Cloud Build Service Account role
for your project. This role is sufficient for several tasks,
including:
Fetching code from your project's Cloud Source Repository
Downloading files from any Cloud Storage bucket owned by your project
Saving build logs in Cloud Logging
Pushing Docker images to Container Registry
Pulling base images from Container Registry
But this service account does not have permissions for certain actions by default ( in particular the container.clusters.get permission is not grated by default). So you need to grant it with a proper IAM role. In your case the Kubernetes Engine Developer role contains the container.clusters.get permission as you can see in this page.
I am the owner of my newly created organization, I created a project under this organization and linked it to the organization billing account where I have 1000$ in credits. Through the web UI, I am able to spin up clusters, VMs, networks... But when I want to do so through gcloud, I am getting permissions denied. E.g.:
$ gcloud compute networks list
API [compute.googleapis.com] not enabled on project [XXX].
Would you like to enable and retry (this will take a few minutes)?
(y/N)? y
ERROR: (gcloud.compute.networks.create) PERMISSION_DENIED: The caller does not have permission
but I can see in the web UI GCP that the API is clearly enabled (and can be used), it's just the gcloud not letting me work with them. The account under gcloud is exactly the same I am using in the web console - validated by gcloud auth list and:
$ gcloud config configurations describe myproject
is_active: true
name: myproject
properties:
compute:
region: europe-west1
zone: europe-west1-b
core:
account: <my-email>
project: <the-project-I-want>
or
$ gcloud services list
ERROR: (gcloud.services.list) User [<myusername>] does not have permission to access project [myproject] (or it may not exist): The caller does not have permission
It works totally fine with a different account (and different organization/projects), but I didn't set up that one in the past. What should I do? Thanks a lot!
UPDATE:
After gcloud init, at least the gcloud services list started to work. But the rest did not:
$ gcloud services list
NAME TITLE
bigquery-json.googleapis.com BigQuery API
cloudapis.googleapis.com Google Cloud APIs
clouddebugger.googleapis.com Stackdriver Debugger API
cloudtrace.googleapis.com Stackdriver Trace API
compute.googleapis.com Compute Engine API
container.googleapis.com Kubernetes Engine API
containerregistry.googleapis.com Container Registry API
datastore.googleapis.com Cloud Datastore API
logging.googleapis.com Stackdriver Logging API
monitoring.googleapis.com Stackdriver Monitoring API
oslogin.googleapis.com Cloud OS Login API
pubsub.googleapis.com Cloud Pub/Sub API
servicemanagement.googleapis.com Service Management API
serviceusage.googleapis.com Service Usage API
sql-component.googleapis.com Cloud SQL
storage-api.googleapis.com Google Cloud Storage JSON API
storage-component.googleapis.com Google Cloud Storage
$ gcloud compute networks create testing-net --subnet-mode=custom '--description=Network to host testing kubernetes cluster'
API [compute.googleapis.com] not enabled on project [{PROJECT_ID}].
Would you like to enable and retry (this will take a few minutes)?
(y/N)? y
ERROR: (gcloud.compute.networks.create) PERMISSION_DENIED: The caller does not have permission
^ the PROJECT_ID above shows my organization's ID, not the actual project under this org.
So the problem was that I used the wrong project_id when gcloud config set project and gcloud defaulted to organization for some reason.
So I had to find correct project id using gcloud projects list and then use gcloud config set project {PROJECT-ID} (not the project name!)
gcloud init - if you wanted to switch gcloud to work between projects which will configure its settings to point to the right project.
I am trying to deploy code from this repo:
https://github.com/anishkny/puppeteer-on-cloud-functions
in Google Cloud Build. My cloudbuild.yaml file contents are:
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['beta', 'functions', 'deploy', 'screenshot', '--trigger-http', '--runtime', 'nodejs8', '--memory', '1024MB']
I have given the following roles to my Cloud Build Service account (****#cloudbuild.gserviceaccount.com):
Cloud Build Service Account
Cloud Functions Developer
Yet, in my Cloud Build log I see the following error:
starting build "1f04522c-fe60-4a25-a4a8-d70e496e2821"
FETCHSOURCE
Fetching storage object: gs://628906418368.cloudbuild-source.googleusercontent.com/94762cc396ed1bb46e8c5dbfa3fa42550140c2eb-b3cfa476-cb21-45ba-849c-c28423982a0f.tar.gz#1534532794239047
Copying gs://628906418368.cloudbuild-source.googleusercontent.com/94762cc396ed1bb46e8c5dbfa3fa42550140c2eb-b3cfa476-cb21-45ba-849c-c28423982a0f.tar.gz#1534532794239047...
/ [0 files][ 0.0 B/ 835.0 B]
/ [1 files][ 835.0 B/ 835.0 B]
Operation completed over 1 objects/835.0 B.
tar: Substituting `.' for empty member name
BUILD
Already have image (with digest): gcr.io/cloud-builders/gcloud
ERROR: (gcloud.beta.functions.deploy) ResponseError: status=[403], code=[Forbidden], message=[The caller does not have permission]
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/gcloud" failed: exit status 1
What am I missing?
It would appear that the permissions changed when (perhaps) Cloud Functions went GA. Another customer raised this issue today and I recalled your question.
The Cloud Build robot (${NUM}#cloudbuild.gserviceaccount.com) additionally needs to be a serviceAccountUser of the ${PROJECT-ID}#appspot.gserviceaccount.com account:
NB While the Cloud Build robot local part is the project number (${NUM}), the appspot robot local part is the project ID (${PROJECT})
Please try:
PROJECT=[[YOUR-PROJECT-ID]]
NUM=$(gcloud projects describe $PROJECT --format='value(projectNumber)')
gcloud iam service-accounts add-iam-policy-binding \
${PROJECT}#appspot.gserviceaccount.com \
--member=serviceAccount:${NUM}#cloudbuild.gserviceaccount.com \
--role=roles/iam.serviceAccountUser \
--project=${PROJECT}
Let me know!
I struggled with this too after reading quite a bit of documentation. A combination of the above answers got me on the right track. Basically, something like the following is needed:
PROJECT=[PROJECT-NAME]
NUM=$(gcloud projects describe $PROJECT --format='value(projectNumber)')
gcloud iam service-accounts add-iam-policy-binding \
${PROJECT}#appspot.gserviceaccount.com \
--member=serviceAccount:${NUM}#cloudbuild.gserviceaccount.com \
--role=roles/iam.serviceAccountUser \
--project=${PROJECT}
gcloud iam service-accounts add-iam-policy-binding \
${PROJECT}#[INSERT_YOUR_IAM_OWNER_SERVICE_ACCOUNT_NAME].iam.gserviceaccount.com \
--member='serviceAccount:service-${NUM}#gcf-admin-robot.iam.gserviceaccount.com' \
--role='roles/iam.serviceAccountUser'
Also, I added the "Cloud Functions Developer" role to my #cloudbuild.gserviceaccount.com account via the IAM Console.
According to Cloud Build documentation, for Cloud Functions you have to grant the "Project Editor" role to your service account.
But, Cloud Functions documentation states that alternatively to using the Project Editor role, you can use "the Cloud Functions Developer role [but you have to] ensure that you have granted the Service Account User role". Regarding Service Accounts, it indicates to have "the CloudFunctions.ServiceAgent role on your project" and to "have permissions for trigger sources, such as Pub/Sub or the Cloud Storage bucket triggering your function".
Due to those considerations, my understanding is that the documentation omitted to specify all the roles your service account would need and went directly to indicate to grant the Project Editor role.
You have to update Service Account permissions on Cloud Build settings page.
Here is instructions https://cloud.google.com/cloud-build/docs/deploying-builds/deploy-cloud-run#fully-managed
You just have to set the status of the Cloud Run Admin role to ENABLED on that page:
start your cloud build with auth
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['auth', 'activate-service-account', 'xoxox#xoxo-dev.iam.gserviceaccount.com', '--key-file=account.json', '--project=rabbito-dev']
and then simply your code deployment on cloud function
- name: 'gcr.io/cloud-builders/gcloud'
args: ['beta', 'functions', 'deploy', 'screenshot', '--trigger-http', '--runtime', 'nodejs8', '--memory', '1024MB']
Please add 'Cloud Functions Service Agent' role to your service account alongside 'Cloud Functions Developer'.