Cloud Build Building Python applications example has the lines below which has _REPO_NAME variable specified.
# [START cloudbuild_python_image_yaml]
# Docker Build
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t',
'us-central1-docker.pkg.dev/${PROJECT_ID}/${_REPO_NAME}/myimage:${SHORT_SHA}', '.']
The Substituting variable values documentation has $REPO_NAME: the name of your repository but does not have _REPO_NAME.
Please help understand where it is defined and what it is.
With Cloud Build, all the user managed substitution variables start with _. If you variable doesn't have a _, like $REPO_NAME, it's a auto generated environment variable.
Therefore in your example you have to provide the $_REPO_NAME and $_BUCKET_NAME if you want to start your Cloud Build process. Else it will fail because it doesn't know that variable.
Why using $_REPO_NAME instead of $REPO_NAME? IMO, it's a mistake. A developer trick is to replace the auto-generated variable by a user managed variable during the tests. Like that, you don't have to push new code to git to test your build pipeline, you simply have to set that variable manually (with gcloud command).
And it might have been forgotten when that code example has been released. Just an assumption.
Related
In the app.js, I want to require a different "config" file depending on the stage/account.
for example:
dev account: const config = require("config-dev.json")
prod account: const config = require("config-prod.json")
At first I tried passing it using build --container-env-var-file but after getting undefined when using process.env.myVar, I think that env file is used at the build stage and has nothing to do with my function, but I could use it in the template creation stage..
So I'm looking now at deploy and there are a few different things that seem relevant, but it's quite confusing to chose which one is relevant for my use case.
There is the config file, in which case, I have no idea how to configure it since I'm in a pipeline context, so where would I instruct my process to use the correct json?
There is also parameters, and mapping.
My json is not just a few vars. its a bit of a complex object. nothing crazy not simple enough to pass the vars 1 by 1.
So I thought a single one containing the filename that I want to use could do the job
But I have no idea how to tell which stage of deployment I currently am in, or how to pass that value to access it from the lambda function.
I also faced this issue while exectuing aws lambda function locally.By this command my issue was solved.
try to configure your file using the sam build command
I'd like to add a transformer to my kustomize setup, one that includes a dynamic value for the case where the local tag needs to override the production tag. If there's a simpler and better way to do this that would be great. I've looked through the list of transformers and generators to see if there was a way to provide this value at runtime (though I think specifically kustomize is designed to never use runtime values).
I can specify something like this:
images:
- name: my-image
newTag: my-sha1
The problem is to change the my-sha1 value after each new local build to then pick that image when I go to apply the local deployment.
How can I set newTag after I run a build locally to match a tag for the image I made locally? I can easily obtain the latest build tag and provide with to kubectl apply -f, but I'm not seeing a flag or environment variable, or something to do so.
I'm deploying my Node apps to Google Cloud Run using Cloud Build and I want to run some tests during the build. My tests require some environment variables, so I have been following this guide to achieve this.
The guide makes the following note:
Note: To use the secret in an environment variable, you need to prefix
the variable name with an underscore "_" and escape the value using
'('. For example: _VARIABLE_NAME=$(cat password.txt) && echo -n
\)_VARIABLE_NAME.
However, I am not quite sure how to implement this.
I have attempted the following in my cloudbuild.yaml.
- id: Execute tests
name: node
args: ['_VAR_ONE=$(cat var-one.txt)', '_VAR_TWO=$(cat var-two.txt)', 'jest -V']
Which returns the following: Error: Cannot find module '/workspace/_VAR_ONE=$(cat var-one.txt)'.
I also tried a few variations of the escape that the above note mentions, but they result in the same error.
What's the best way to get the secrets into my code as environment variables?
Also, if I need to use multiple environment variables, is it better to use Cloud KMS with an .env file?
Thanks!
It looks like you are incorrectly using the entrypoint provided by the node image. You are effectively running the command:
node _VAR_ONE=$(cat var-one.txt) _VAR_TWO=$(cat var-two.txt) jest -V
I want to digress for a moment and say this pattern does not work in Node, you need to specify the environment variables first before calling node, for example VAR_ONE=$(cat foo.txt) VAR_TWO=bar node run test
Anyway, I think what you want to run is:
_VAR_ONE=$(cat var-one.txt) _VAR_TWO=$(cat var-two.txt) jest -V
This is how we will do that - Assuming you have a previous step where you write out the contents of the secret into the files var-one.txt and var-two.txt in a previous step - here is how you would use it in the node step, it's just the standard way you use environment variables when running a command from the command line:
- id: Execute tests
name: node
entrypoint: '/bin/bash'
args:
'-c',
'_VAR_ONE=$(cat var-one.txt) _VAR_TWO=$(cat var-two.txt) jest -V'
]
You need to ensure in the node environment you are using the variables as specified (ie. process.env._VAR_ONE or process.env._VAR_TWO). I don't think you need to have the _ character prefixed here but I haven't tested it to confirm that. You can try the above and it should get you much further I think.
Trying to understand how the docker-compose file was created as I want to replicate this into a kubernetes deployment yaml file.
In reference to a cookiecutter-django's docker-compose production.yaml file:
...
services:
django: &django
...
By docker-compose design, the name of service here is already defined as django but then I noticed this extra bit &django. This made me wonder why its here. Further down, I noticed the following:
...
celeryworker:
<<: *django
...
I don't understand how that works. The docker-compose docs have no reference or mention for using << let alone, making a reference to a named service like *django.
Can anyone explain how the above work and how do I replicate it to a kubernetes deployment or services yaml file (or both?) if possible?
Edit:
The question that #jonsharpe shared was similar but the answer wasn't clear to me on how its used.
There are three different things happening, and none of them are specifically compose syntax, rather they are yaml syntax.
First is defining an anchor with the & followed by a name. That's similar to defining a variable to use later in the yaml, with the value matching the value of the yaml object where it appears.
Next is the alias, specified with * and the same name as the anchor. That uses the anchor in the second location in the yaml file.
Last is a mapping merge using the << syntax, which merges all of the mapped values in the alias with the rest of the values in the current map, allowing you to override values in the saved anchor with values specific to that section of the compose file.
To dig more into this, try searching on "yaml anchors and aliases". The first hit for me is this blog post: https://medium.com/#kinghuang/docker-compose-anchors-aliases-extensions-a1e4105d70bd
I need to make a request from a pipeline, which is optional on windows- and linux-based agents. The request may fail for whatever reason, but the pipeline should continue anyway - always.
All CICD-Systems I've worked with so far have the possibility to mark a task as optional or - more specifically - to ignore errors:
Example: https://docs.gitlab.com/ee/ci/yaml/#allow_failure
job1:
stage: test
script:
- execute_script_that_will_fail
allow_failure: true
I have not found a similar concept in GoCD - is there one I am not aware of?
You can use the run_if attribute of the task to achieve that. It takes passed, failed or any. Passed is the default value, that is only if the current task is passed it moves on to the next one. Failed is the exact opposite of this. I generally use it for setting up assertions within the job. Any is basically you don't care the outcome of that task. GoCD will not fail irrespective of the outcome of that task within the job.
exec:
run_if: any
working_directory: dir
command: make
arguments:
- -j3
- docs
- install
References
From GoCD's documentation - https://docs.gocd.org/current/advanced_usage/dev_conditional_task_execution.html
If you're using the YAML to define your pipelines - https://github.com/tomzo/gocd-yaml-config-plugin#tasks