Cloudbuild having issue when running the kubelet cloudbuilder - google-cloud-platform

I am trying to deploy the new changes to kubernetes cluster using the Google Cloud Provider CloudBuild. Whenever I make some changes the trigger is working fine and its starting a new build but here is the issue I am getting with this cloudbuild.yaml.
cloudbuild.yaml
steps:
#step1
- name: 'gcr.io/cloud-builders/docker'
args: [ 'build', '-t', 'gcr.io/$PROJECT_ID/cloudbuildtest-image', '.' ]
#step 2
- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/$PROJECT_ID/cloudbuildtest-image']
#step 3 for testing
name: 'gcr.io/cloud-builders/kubectl'
args: ['get', 'pods']
env:
- 'CLOUDSDK_COMPUTE_ZONE=us-central1-a'
- 'CLOUDSDK_CONTAINER_CLUSTER=cloudbuild-test'
#STEP-4
images:
- 'gcr.io/$PROJECT_ID/cloudbuildtest-image'
Step 1 and 2 are working fine but the issue is with the step3 where for testing purpose I simply ran the get pods command to test if it will work or not. Here is the issue I am getting in the logs.
Running: gcloud container clusters get-credentials --project="journeyfoods-io" --zone="us-central1-a" "cloudbuild-test"
Fetching cluster endpoint and auth data.
ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Required "container.clusters.get" permission(s) for "projects/XXXX/zones/us-central1-a/clusters/cloudbuild-test".
What permissions is it looking for? Do I need to do some
authentication before running the steps or What exactly am I missing?

The steps of a Cloud Build build are executed using with the [PROJECT_NUMBER]#cloudbuild.gserviceaccount.com service account. From the Cloud Build documentation page about this topic:
When you enable the Cloud Build API, the service account is
automatically created and granted the Cloud Build Service Account role
for your project. This role is sufficient for several tasks,
including:
Fetching code from your project's Cloud Source Repository
Downloading files from any Cloud Storage bucket owned by your project
Saving build logs in Cloud Logging
Pushing Docker images to Container Registry
Pulling base images from Container Registry
But this service account does not have permissions for certain actions by default ( in particular the container.clusters.get permission is not grated by default). So you need to grant it with a proper IAM role. In your case the Kubernetes Engine Developer role contains the container.clusters.get permission as you can see in this page.

Related

How do I deploy a Google Cloud Function (2nd generation)?

I've previously only used Cloud Functions of gen. 1 but now plan to move to 2nd generation and is just trying to deploy/test a first basic function. I'm just taking the Google sample for a storage triggered function and try to deploy it, but it keeps failing.
This is what it looks like:
> gcloud functions deploy nodejs-finalize-function --gen2 --runtime=nodejs16 --project myproject --region=europe-west3 --source=. --entry-point=handleImage --trigger-event-filters='type=google.cloud.storage.object.v1.finalized' --trigger-event-filters='bucket=se_my_images'
Preparing function...done.
X Deploying function...
✓ [Build] Logs are available at [https://console.cloud.google.com/cloud-build/builds;region=europe-west3/a8355043-adf0-4485-a510-1d54b7e11111?project=123445666123]
✓ [Service]
✓ [Trigger]
- [ArtifactRegistry] Deleting function artifacts in Artifact Registry...
. [Healthcheck]
. [Triggercheck]
Failed.
ERROR: (gcloud.functions.deploy) OperationError: code=7, message=Creating trigger failed for projects/myproject/locations/europe-west3/triggers/nodejs-finalize-function-898863: The Cloud Storage service account for your bucket is unable to publish to Cloud Pub/Sub topics in the specified project.
To use GCS CloudEvent triggers, the GCS service account requires the Pub/Sub Publisher (roles/pubsub.publisher) IAM role in the specified project. (See https://cloud.google.com/eventarc/docs/run/quickstart-storage#before-you-begin)
The error looks easy to understand, but I have aded the Pub/Sub Publisher role to all my service accounts now (the ones listed below) and I still keep getting the same error.
>gcloud iam service-accounts list --project myproject
DISPLAY NAME EMAIL DISABLED
firebase-adminsdk firebase-adminsdk-u2x33#myproject.iam.gserviceaccount.com False
Default compute service account 930445666575-compute#developer.gserviceaccount.com False
backend-dev backend-dev#myproject.iam.gserviceaccount.com False
App Engine default service account myproject#appspot.gserviceaccount.com False
I don't know how to move forward from here so I hope someone can help.
*** EDIT ***.
I added the role to the listed service accounts in the GCP console, IAM > Permissions > View By Principal page/view where I used the Edit Principal button to assign an additional role (Pub/Sub Publisher) to the service accounts (note that I added the role to all my listed service accounts since I'm not 100% sure which one is used by GCP for cloud deployment).
since you are already using gcloud cli, i suggest you follow the step 2 which says:
PROJECT_ID=$(gcloud config get-value project)
PROJECT_NUMBER=$(gcloud projects list --filter="project_id:$PROJECT_ID" --format='value(project_number)')
SERVICE_ACCOUNT=$(gsutil kms serviceaccount -p $PROJECT_NUMBER)
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member serviceAccount:$SERVICE_ACCOUNT \
--role roles/pubsub.publisher
After these 4 cmd's, you should have no problems.. I don't use gcp interface for iam purposes, since all my iam policies are uploaded by terraform/terragrunt.

Gitlab GKE runner push to GCR using workload identity

I am attempting to use a gitlab runner to build an image in GKE and then push that image to GCR. This always fails and outputs an AWS error at first.
My understanding is that the building pod is created within the same namespace as my GCR appropriate service account should push. The pod seems to not be using workload identity or failing for some other reason.
Gitlab Yaml
buildmygcr:
stage: build
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
tags:
- cluster
- kubernetes
script:
- /kaniko/executor --verbosity debug --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination gcr.io/myproject/test:test --cache=false
Build Error
DEBU[0000] Copying file /builds/user/test/Dockerfile to /kaniko/Dockerfile
E0403 20:18:16.646782 21 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "gcr.io/myproject/test:test": creating push check transport for gcr.io failed: GET https://gcr.io/v2/token?scope=repository%myproject%2Ftest%3Apush%2Cpull&service=gcr.io: UNAUTHORIZED: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication
Cleaning up file based variables
00:00
ERROR: Job failed: command terminated with exit code 1
Workload Identity Test shows the proper account for the namespace
root#workload-identity-test:/# gcloud auth list
Credentialed Accounts
ACTIVE ACCOUNT
* mysvcaccount#myproject.iam.gserviceaccount.com
To set the active account, run:
$ gcloud config set account `ACCOUNT`

Correct permissions for Google Cloud Build to deploy a Cloudfunction in a separate project

We have separate GCP projects for "build" and "prod" environments. I would like to use Cloud Build for the "build" project to deploy a Cloud Function in the "prod" project. When following the documentation.
Notably, I added the "Cloud Functions Developer" role to the build service account in the build project and the "IAM Service Account User" role to the PROJECT_ID#appspot.gserviceaccount.com account as noted in the docs and in this question, when running a build, I get:
ERROR: (gcloud.functions.deploy) ResponseError: status=[403], code=[Forbidden], message=[The caller does not have permission]
A simplified example of my cloudbuild.yaml is
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['source', 'repos', 'clone', 'a_mirrored_repo', '--project=build-xxxx']
- name: 'gcr.io/cloud-builders/gcloud'
args: ['functions', 'deploy', 'some_function', '--trigger-http', '--runtime', 'python37', '--project', 'prod-yyyy']
I am able to deploy my fucntion to prod using the gsutil command line utility from my laptop, and I am able to use my cloudbuild.yaml to deploy this function to the build project. But I'm unsure what roles I need to assign to what accounts to enable the build project to deploy the cloudfunction to the prod project.
If you want to deploy the function in the prod project then the Cloud Build service account of the build project must be added as a "Cloud Functions Developer" in the prod project. If I understand your description correctly, then you've set it in the build project.
Specifically: in the GCP console, in the prod project, under IAM > members add a new member named [build-project-id]#cloudbuild.gserviceaccount.com and add the role "Cloud Functions Developer" to this member. At this point attempting to deploy may result in an error message:
ERROR: (gcloud.functions.deploy) ResponseError: status=[403],
code=[Forbidden], message=[Missing necessary permission
iam.serviceAccounts.actAs for prod-xxxx#appspot.gserviceaccount.com
on project prod-xxxx.
Please grant
prod-xxxx#appspot.gserviceaccount.com the
roles/iam.serviceAccountUser role.
You can do that by running
'gcloud projects add-iam-policy-binding prod-xxxx
--member=prod-xxxx#appspot.gserviceaccount.com --role=roles/iam.serviceAccountUser'
The command suggested in the error message is slightly incorrect. The correct syntax is:
gcloud projects add-iam-policy-binding prod-xxxx --member='serviceAccount:[build-project-number]#cloudbuild.gserviceaccount.com' --role='roles/iam.serviceAccountUser'

Deploying Google Cloud function from a Google Build

When attempting to deploy a simple function, I am running into a 403 error. Here is my full yaml:
steps:
# Install Dependencies
- name: 'python'
id: Pip install
args: ['pip3', 'install', '-r', 'requirements.txt', '--user']
# Run Tests
- name: 'python'
args: ['python3', '-m', 'pytest', 'functions/folder_a/test/']
# Deploy Cloud Functions
- name: 'gcr.io/cloud-builders/gcloud'
id: 'helloFunction'
args: ['functions', 'deploy', 'hello', '--source' , 'functions/folder_a/main.py',
'--runtime' , 'python37' ,'--trigger-http']
Step one finishes fine but step two errors here.
$ gcloud info
Account: [projectname#appspot.gserviceaccount.com]
Project: [projectname]
Current Properties:
[core]
project: [projectname]
account: [projectname#appspot.gserviceaccount.com]
disable_usage_reporting: [True]
I tried adding the following permissions to the above account in IAM:
Cloud Build Service Agent
Cloud Functions Developer
Cloud Functions Service Agent
Editor
When Google Build is running, which service account is it using? How can I change the permissions to allow automatic deploying of functions?
Using Cloud Build to deploy Cloud Functions is documented in https://cloud.google.com/functions/docs/bestpractices/testing#continuous_testing_and_deployment
From that page the guidance is:
To deploy Cloud Functions, you might want to assign the Cloud Functions Developer role to the Cloud Build service account (PROJECT_NUMBER#cloudbuild.gserviceaccount.com).
If you use the Cloud Functions Developer role, you also need to grant the Cloud Functions Runtime service account (PROJECT_ID#appspot.gserviceaccount.com) the IAM Service Account User role.
The Cloud Build service account is PROJECT_NUMBER#cloudbuild.gserviceaccount.com, if you look for it in the console there's only going to be one service account with *#cloudbuild*

gcloud - no permissions for any API even though I am owner and works fine through web UI

I am the owner of my newly created organization, I created a project under this organization and linked it to the organization billing account where I have 1000$ in credits. Through the web UI, I am able to spin up clusters, VMs, networks... But when I want to do so through gcloud, I am getting permissions denied. E.g.:
$ gcloud compute networks list
API [compute.googleapis.com] not enabled on project [XXX].
Would you like to enable and retry (this will take a few minutes)?
(y/N)? y
ERROR: (gcloud.compute.networks.create) PERMISSION_DENIED: The caller does not have permission
but I can see in the web UI GCP that the API is clearly enabled (and can be used), it's just the gcloud not letting me work with them. The account under gcloud is exactly the same I am using in the web console - validated by gcloud auth list and:
$ gcloud config configurations describe myproject
is_active: true
name: myproject
properties:
compute:
region: europe-west1
zone: europe-west1-b
core:
account: <my-email>
project: <the-project-I-want>
or
$ gcloud services list
ERROR: (gcloud.services.list) User [<myusername>] does not have permission to access project [myproject] (or it may not exist): The caller does not have permission
It works totally fine with a different account (and different organization/projects), but I didn't set up that one in the past. What should I do? Thanks a lot!
UPDATE:
After gcloud init, at least the gcloud services list started to work. But the rest did not:
$ gcloud services list
NAME TITLE
bigquery-json.googleapis.com BigQuery API
cloudapis.googleapis.com Google Cloud APIs
clouddebugger.googleapis.com Stackdriver Debugger API
cloudtrace.googleapis.com Stackdriver Trace API
compute.googleapis.com Compute Engine API
container.googleapis.com Kubernetes Engine API
containerregistry.googleapis.com Container Registry API
datastore.googleapis.com Cloud Datastore API
logging.googleapis.com Stackdriver Logging API
monitoring.googleapis.com Stackdriver Monitoring API
oslogin.googleapis.com Cloud OS Login API
pubsub.googleapis.com Cloud Pub/Sub API
servicemanagement.googleapis.com Service Management API
serviceusage.googleapis.com Service Usage API
sql-component.googleapis.com Cloud SQL
storage-api.googleapis.com Google Cloud Storage JSON API
storage-component.googleapis.com Google Cloud Storage
$ gcloud compute networks create testing-net --subnet-mode=custom '--description=Network to host testing kubernetes cluster'
API [compute.googleapis.com] not enabled on project [{PROJECT_ID}].
Would you like to enable and retry (this will take a few minutes)?
(y/N)? y
ERROR: (gcloud.compute.networks.create) PERMISSION_DENIED: The caller does not have permission
^ the PROJECT_ID above shows my organization's ID, not the actual project under this org.
So the problem was that I used the wrong project_id when gcloud config set project and gcloud defaulted to organization for some reason.
So I had to find correct project id using gcloud projects list and then use gcloud config set project {PROJECT-ID} (not the project name!)
gcloud init - if you wanted to switch gcloud to work between projects which will configure its settings to point to the right project.