Using GCP secrets as part of cloud build - google-cloud-platform

I'm trying to pass in confidential info into a bash script that is part of our cloud build process
I've followed the CloudBuild docs to try set it all up.
After many many different tires copied the docker example as a test even that is not working.
Here is what I have at the moment
*gcloud secrets list*
NAME CREATED REPLICATION_POLICY LOCATIONS
SECRET1 2021-08-18T04:37:47 automatic -
SECRET2 2021-08-18T04:38:11 automatic -
*gcloud secrets versions access latest --secret="SECRET1"*
Secret2Value
*gcloud secrets versions access latest --secret="SECRET2"*
Secret2Value
**cloudbuild.yaml**
steps:
- name: 'gcr.io/cloud-builders/docker'
entrypoint: 'bash'
args: ['-c', 'docker login --username=$$USERNAME --password=$$PASSWORD']
secretEnv: ['USERNAME', 'PASSWORD']
availableSecrets:
secretManager:
- versionName: projects/$PROJECT_ID/secrets/SECRET1/versions/1
env: 'PASSWORD'
- versionName: projects/$PROJECT_ID/secrets/SECRET2/versions/latest
env: 'USERNAME'
My understanding was that it would substituite the value of SECRET1 and SECRET2 into the USERNAME and PASSWORD envs but I'm getting $USERNAME and $PASSWORD
Arguments
bash -c docker login --username=$USERNAME --password=$PASSWORD
Feels like I've missed something simple yet fundamental
== Update ==
Here is the full build log
FETCHSOURCE
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /workspace/.git/
From https://source.developers.google.com/p/amiti-plex-can- dev/r/bitbucket_noldortech_amiti-payments
* branch a72363459d9ff5bed31411e960cc3e021febc322 -> FETCH_HEAD
HEAD is now at a723634 Secrets test 1
BUILD
Already have image (with digest): gcr.io/cloud-builders/docker
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get "https://registry-1.docker.io/v2/": unauthorized: incorrect username or password
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1
== Update 2==
After testing the echo commands below, I switched over to using a script, which is what I actually need.
**Yaml File:**
steps:
- name: 'gcr.io/cloud-builders/docker'
entrypoint: 'bash'
args: ['-c', 'echo Username=$$USERNAME && echo Password=$$PASSWORD']
secretEnv: ['USERNAME', 'PASSWORD']
- name: 'gcr.io/cloud-builders/docker'
entrypoint: 'bash'
args: ['cloudbuilds/script.sh', '$$USERNAME', '$$PASSWORD']
secretEnv: ['USERNAME', 'PASSWORD']
availableSecrets:
secretManager:
- versionName: projects/$PROJECT_ID/secrets/SECRET1/versions/1
env: 'PASSWORD'
- versionName: projects/$PROJECT_ID/secrets/SECRET2/versions/latest
env: 'USERNAME'
**Script:**
#/bin/bash
secretVar1="$1"
secretVar2="$2"
printf "\n\nVARIABLES\nSecret1: $secretVar1\nSecret2: $secretVar2\n\n"
**Build Log:**
BUILD
Starting Step #0
Step #0: Already have image (with digest): gcr.io/cloud-builders/docker
Step #0: Username=Secret2Value
Step #0: Password=Secret1Value
Finished Step #0
Starting Step #1
Step #1: Already have image (with digest): gcr.io/cloud-builders/docker
Step #1:
Step #1:
Step #1: VARIABLES
Step #1: Secret1: $USERNAME
Step #1: Secret2: $PASSWORD
Step #1:
Finished Step #1
PUSH
DONE

Finally got the correct yaml syntax to execute my script as I need
steps:
- name: 'gcr.io/cloud-builders/docker'
entrypoint: 'bash'
args: ['-c', ". ./cloudbuilds/script.sh $$SOMEVAR1 $$SOMEVAR2"]
secretEnv: ['SOMEVAR1', 'SOMEVAR2']
availableSecrets:
secretManager:
- versionName: projects/$PROJECT_ID/secrets/SECRET1/versions/1
env: 'SOMEVAR2'
- versionName: projects/$PROJECT_ID/secrets/SECRET2/versions/latest
env: 'SOMEVAR1'
BUILD
Already have image (with digest): gcr.io/cloud-builders/docker
VARIABLES
Secret1: Secret2Value
Secret2: Secret1Value
PUSH
DONE

USERNAME & PASSWORD are environment variables, for them to be expanded on the command line, you need to run the command via the shell. (Note that doing so is considered bad practice from a security perspective.)
This is why only the first step works in your second update.

Related

Access GCP Secret during Cloud Build build step

Lets say I have a cloudbuild.yaml file that looks like this:
steps:
- name: 'gcr.io/cloud-builders/docker'
id: build
args: ['build', '-t', 'us.gcr.io/${PROJECT_ID}/image_name', '--build-arg', 'secret=$$SECRET', '.']
secretEnv: ['SECRET']
images:
- 'us.gcr.io/${PROJECT_ID}/image_name'
availableSecrets:
secretManager:
- versionName: projects/project/secrets/my_secret/versions/latest
env: 'SECRET'
Right now, the --build-arg is assigning to the Docker secret arg the value $SECRET instead of the value actually stored in the secret. How can I access the secret value during this step? All of the examples I can find online say to add a bash entrypoint however only for steps that aren't actually doing the build call.
It's a usual issue with Cloud Build and Secret Manager integration. You can access to the secret only in a script, not in entry-point and arguments (your case)
Try that
steps:
- name: 'gcr.io/cloud-builders/docker'
id: build
entrypoint: 'bash'
args:
- -c
- |
docker build -t us.gcr.io/${PROJECT_ID}/image_name --build-arg secret=$$SECRET .
secretEnv: ['SECRET']
The syntax for assigning secrets to docker args seems to be slightly different to that for normal environment variables. The following snippet is taken from a working project of my own and correctly accesses the secret, and you can see the difference compared to the normal environment variables:
...
env:
- PROJECT_ID=$PROJECT_ID
- NO_DEPLOY=$_NO_DEPLOY
- NO_E2E=$_NO_E2E
secretEnv:
- "EXAMPLE_API_KEY"
args:
- --destination=gcr.io/$PROJECT_ID/api
- --cache=true
- --build-arg=PROJECT_ID
- --build-arg=EXAMPLE_API_KEY
- --build-arg=NO_DEPLOY
- --build-arg=NO_E2E
availableSecrets:
secretManager:
- versionName: projects/$PROJECT_ID/secrets/example-api-key/versions/latest
env: "EXAMPLE_API_KEY"
...

Google cloud - cloubduild and app variabile substitution in trigger

I'm using cloudbuild to deploy new version of my app when a new commit appears in github.
Everything is working good.
Now I'm trying to setup a variable substitution in the trigger configuration, because I want to put my version number in the trigger once, so that I can find the deployed correct version without modifying cloudbuild configuration file.
Variabile substitution works great in my cloudbuild file, for example:
(cloudbuild.yaml)
# TEST: PRINT VARIABLE IN LOG
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args: ['-c', 'echo', '${_VERSION}']
# DEPLOY APP
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy", "-v", "${_VERSION}", "app.yaml"]
dir: 'frontend'
timeout: "20m"
${_VERSION} is correctly replaced with the string I put into my trigger.
Now I want to obtain the same result in app.yaml file, substituting an env variabile, something like:
(app.yaml)
runtime: nodejs
env: flex
service: backend
env_variables:
VERSION: "${_VERSION}"
TEST_ENV: "read from google"
When I read TEST_ENV from my app, it works, but _VERSION is not replaced.
Any suggestion?
When you perform this step
# DEPLOY APP
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy", "-v", "${_VERSION}", "app.yaml"]
dir: 'frontend'
timeout: "20m"
The app.yaml is provided as-is to the gcloud command, and it's not evaluated. You have to update it manually. Something like this
# REPLACE: PUT THE CORRECT VALUE IN APP.YAML FILE
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args: ['-c', 'sed', "-i", "sed -i "s/\$${_VERSION}/${_VERSION}/g", 'app.yaml']
Of course if you let the
env_variables:
VERSION: "${_VERSION}"
as-is in your app.yaml file. You can change this replacement string
I want to add this solution in case someone has problems with the one proposed by giullade (in my case, cloudbuild gave me an error in executing the sed command).
I also changed my replacement string to one more readable and to avoid escaping the $ sign.
# Step 0: REPLACE variables in app.yaml file
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
dir: 'backend'
args:
- '-c'
- |
sed -i "s/__VERSION/${_VERSION}/g" app-staging.yaml
and in my app.yaml:
env_variables:
VERSION_ENV: "__VERSION"

How to access GSM secrets through Cloud Build and pass to Cloud Function

How does one pass a secret from Google Secrets Manager (GSM) to a Cloud Function when using Cloud Build? The below cloudbuild.yaml has three steps. Further, I'm using volumes to create permanent storage between build steps. I can confirm GSM retrieval by Cloud Build. However, when I attempt to pass a secret in yaml format using --env-vars-file I encounter the following error ...
Already have image (with digest): gcr.io/cloud-builders/gcloud
ERROR: gcloud crashed (AttributeError): 'str' object has no attribute 'items'
cloudbuild.yaml:
steps:
- name: 'gcr.io/cloud-builders/gcloud'
volumes:
- name: 'secrets'
path: '/secrets'
entrypoint: "bash"
args:
- "-c"
- |
echo -n 'gsm_secret:' > /secrets/my-secret-file.txt
- name: 'gcr.io/cloud-builders/gcloud'
volumes:
- name: 'secrets'
path: '/secrets'
entrypoint: "bash"
args:
- "-c"
- |
gcloud components update
gcloud beta secrets versions access --secret=MySecret latest >> /secrets/my-secret-file.txt
cat /secrets/my-secret-file.txt
- name: 'gcr.io/cloud-builders/gcloud'
volumes:
- name: 'secrets'
path: '/secrets'
args: [
'functions', 'deploy', 'gsm-foobar',
'--project=[...]',
'--trigger-http',
'--runtime=go111',
'--region=us-central1',
'--memory=256MB',
'--timeout=540',
'--entry-point=GSM',
'--allow-unauthenticated',
'--source=https://source.developers.google.com/[...]',
'--service-account', '[...]#appspot.gserviceaccount.com',
'--env-vars-file', '/secrets/my-secret-file.txt'
]
Update:
Usage of volumes is not required as /workspace is permanent storage between steps in Cloud Build. Also, gcloud components update is no longer necessary as the default Cloud SDK version, as of today, is 279.0.0
A Solution:
steps:
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: "bash"
args:
- "-c"
- |
echo "gsm_secret: $(gcloud beta secrets versions access --secret=MySecret latest)" > /workspace/my-secret-file.txt
cat /workspace/my-secret-file.txt
- name: 'gcr.io/cloud-builders/gcloud'
args: [
'functions', 'deploy', 'gsm-foobar',
[...]
'--entry-point=GSM',
'--allow-unauthenticated',
'--source=https://source.developers.google.com/[...]',
'--service-account', '[...]#appspot.gserviceaccount.com',
'--env-vars-file=/workspace/my-secret-file.txt'
]
On second read, I realize your 2nd step puts the secret value in the file. I think you're missing the newline.
NB I've not tried this for myself!
Ensure you have a newline at the end of your secrets file.
See: https://cloud.google.com/functions/docs/env-var
Update: tried it ;-)
I think your issue was the final newline.
Using the following in a step prior to the deployment, works:
echo "gsm_secret: $(gcloud beta secrets versions access --secret=MySecret latest)" > /secrets/my-secret-file.txt
Or, more simply, perhaps:
steps:
- name: "gcr.io/cloud-builders/gcloud"
entrypoint: /bin/bash
args:
- "-c"
- |
gcloud functions deploy ... \
--set-env-vars=NAME=$(gcloud beta secrets versions access --secret=name latest)
Also, see secretEnv. This is a more elegant mechanism..This functionality should perhaps be augmented by Google to support secret manager (in addition to KMS).
As of 2021 February 10, you can access Secret Manager secrets directly from Cloud Build using the availableSecrets field:
steps:
- id: 'deploy'
name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args:
- '-c'
- 'gcloud functions deploy --set-env-vars=SECRET=$$MY_SECRET'
secretEnv: ['MY_SECRET']
availableSecrets:
secretManager:
- versionName: 'projects/my-project/secrets/my-secret/versions/latest'
env: 'MY_SECRET'
Documentation

How do I get Google Cloud Build to properly substitute values when it responds to a GitHub trigger?

The Problem
A GitHub trigger set up in Google Cloud Build doesn't actually substitute the configured values while running the Build (cloudbuild.yaml)
This is the Google Cloud Build config
cloudbuild.yaml
steps:
- name: 'gcr.io/cloud-builders/gcloud'
id: 'decrypt .npmrc'
args:
- kms
- decrypt
- --ciphertext-file=npmrc.enc
- --plaintext-file=/root/.npmrc
- --location=global
- --keyring=my-keyring
- --key=npm-key
- --project=${_CLOUD_KMS_PROJECT}
volumes:
- name: 'home'
path: /root/
- name: 'gcr.io/cloud-builders/npm'
id: 'install'
args: ['install']
env:
- HOME=/root/
volumes:
- name: 'home'
path: /root/
- name: 'gcr.io/cloud-builders/npm'
id: 'test'
args: ['run', 'test']
- name: gcr.io/$PROJECT_ID/skaffold:alpha
id: 'deploy'
args: ['run', '-f=${_SKAFFOLD_FILE}']
env:
- CLOUDSDK_COMPUTE_ZONE=${_CLOUDSDK_COMPUTE_ZONE}
- CLOUDSDK_CONTAINER_CLUSTER=${_CLOUDSDK_CONTAINER_CLUSTER}
substitutions:
_SKAFFOLD_FILE: dummy.yaml
_CLOUDSDK_COMPUTE_ZONE: us-west1-a
_CLOUDSDK_CONTAINER_CLUSTER: dummy
timeout: 1000s
Curiously, when triggering the build via a gcloud SDK call it works ✅ ex:
gcloud builds submit --config=cloudbuild.yaml --substitutions=_SKAFFOLD_FILE=skaffold.yaml,_CLOUDSDK_COMPUTE_ZONE=us-west1-a,_CLOUDSDK_CONTAINER_CLUSTER=skaffold .
Some more context
Cloud Build Trigger dashboard (img)
Trigger configuration with substitutions (img)
Missing substitutions in Build history from triggered build (img)
In contrast: Correctly substituted values when running gcloud builds submit (img)

Why is my Container Builder build failing with "failed to find one or more images after execution of build steps"

I don't understand what this error message means. It happens at the end of my build, when the build is complete and the image is being tagged. Here's the tail end of the log:
Step 17/18 : WORKDIR /var/www
---> 0cb8de2acd8f
Removing intermediate container 7e7838eac6fb
Step 18/18 : CMD bundle exec puma -C config/puma.rb
---> Running in 9089eb79192b
---> 890a53af5964
Removing intermediate container 9089eb79192b
Successfully built 890a53af5964
Successfully tagged us.gcr.io/foo-staging/foobar:latest
ERROR
ERROR: failed to find one or more images after execution of build steps: ["us.gcr.io/foo-staging/foobar:a2122696c92f430529197dea8213c96b3eee8ee4"]
Here's my cloudbuild.yaml:
steps:
- name: 'gcr.io/cloud-builders/docker'
args: [ 'build', '-t', 'us.gcr.io/$PROJECT_ID/foobar', '.' ]
images:
- 'us.gcr.io/$PROJECT_ID/foobar:$COMMIT_SHA'
- 'us.gcr.io/$PROJECT_ID/foobar:latest'
timeout: 3600s
I thought maybe it was a transient failure, but I retried the build and it happened again.
Ah I needed to tag the build in the build step:
steps:
- name: 'gcr.io/cloud-builders/docker'
args: [ 'build', '-t', 'us.gcr.io/$PROJECT_ID/foobar:$COMMIT_SHA', '-t', 'us.gcr.io/$PROJECT_ID/foobar:latest', '.' ]
images:
- 'us.gcr.io/$PROJECT_ID/foobar:$COMMIT_SHA'
- 'us.gcr.io/$PROJECT_ID/foobar:latest'
timeout: 3600s