Cloud Build Building Python applications example has the lines below which has _REPO_NAME variable specified.
# [START cloudbuild_python_image_yaml]
# Docker Build
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t',
'us-central1-docker.pkg.dev/${PROJECT_ID}/${_REPO_NAME}/myimage:${SHORT_SHA}', '.']
The Substituting variable values documentation has $REPO_NAME: the name of your repository but does not have _REPO_NAME.
Please help understand where it is defined and what it is.
With Cloud Build, all the user managed substitution variables start with _. If you variable doesn't have a _, like $REPO_NAME, it's a auto generated environment variable.
Therefore in your example you have to provide the $_REPO_NAME and $_BUCKET_NAME if you want to start your Cloud Build process. Else it will fail because it doesn't know that variable.
Why using $_REPO_NAME instead of $REPO_NAME? IMO, it's a mistake. A developer trick is to replace the auto-generated variable by a user managed variable during the tests. Like that, you don't have to push new code to git to test your build pipeline, you simply have to set that variable manually (with gcloud command).
And it might have been forgotten when that code example has been released. Just an assumption.
I am having a syntax error when I test my gitlab-ci.yml in CI Lint. Can someone suggest a solution to this problem?
build-production:
stage: build
only:
- master
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
script:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE:$CI_COMMIT_TAG
rules:
- if: $CI_COMMIT_TAG
Status: syntax is incorrect
jobs:build-production config key may not be used with `rules`: only
Documentation is pretty clear :
rules replaces only/except and they can’t be used together in the same job. If you configure one job to use both keywords, the linter returns a key may not be used with rules error.
I suggest to use rules: for both of your conditions :
rules:
- if: '$CI_COMMIT_REF_NAME == "master" && $CI_COMMIT_TAG'
This is not correct, unless you would create a tag master.
See: https://gitlab.sron.nl/help/ci/variables/predefined_variables.md
CI_COMMIT_REF_NAME
The branch or tag name for which project is built.
There is a workaround described here: Gitlab CI: Run Pipeline job only for tagged commits that exist on protected branches
I'm deploying my Node apps to Google Cloud Run using Cloud Build and I want to run some tests during the build. My tests require some environment variables, so I have been following this guide to achieve this.
The guide makes the following note:
Note: To use the secret in an environment variable, you need to prefix
the variable name with an underscore "_" and escape the value using
'('. For example: _VARIABLE_NAME=$(cat password.txt) && echo -n
\)_VARIABLE_NAME.
However, I am not quite sure how to implement this.
I have attempted the following in my cloudbuild.yaml.
- id: Execute tests
name: node
args: ['_VAR_ONE=$(cat var-one.txt)', '_VAR_TWO=$(cat var-two.txt)', 'jest -V']
Which returns the following: Error: Cannot find module '/workspace/_VAR_ONE=$(cat var-one.txt)'.
I also tried a few variations of the escape that the above note mentions, but they result in the same error.
What's the best way to get the secrets into my code as environment variables?
Also, if I need to use multiple environment variables, is it better to use Cloud KMS with an .env file?
Thanks!
It looks like you are incorrectly using the entrypoint provided by the node image. You are effectively running the command:
node _VAR_ONE=$(cat var-one.txt) _VAR_TWO=$(cat var-two.txt) jest -V
I want to digress for a moment and say this pattern does not work in Node, you need to specify the environment variables first before calling node, for example VAR_ONE=$(cat foo.txt) VAR_TWO=bar node run test
Anyway, I think what you want to run is:
_VAR_ONE=$(cat var-one.txt) _VAR_TWO=$(cat var-two.txt) jest -V
This is how we will do that - Assuming you have a previous step where you write out the contents of the secret into the files var-one.txt and var-two.txt in a previous step - here is how you would use it in the node step, it's just the standard way you use environment variables when running a command from the command line:
- id: Execute tests
name: node
entrypoint: '/bin/bash'
args:
'-c',
'_VAR_ONE=$(cat var-one.txt) _VAR_TWO=$(cat var-two.txt) jest -V'
]
You need to ensure in the node environment you are using the variables as specified (ie. process.env._VAR_ONE or process.env._VAR_TWO). I don't think you need to have the _ character prefixed here but I haven't tested it to confirm that. You can try the above and it should get you much further I think.
I am learning Kubernetes recently, and I am not very clear about the difference between "kubectl apply" and "kubectl replace". Is there any situation that we can only use one of them?
I have written up a thorough explanation of the differences between apply, replace, and patch: Kubernetes Apply vs. Replace vs. Patch. It includes an explanation that the current top-ranked answer to this question is wrong.
Briefly, kubectl apply uses the provided spec to create a resource if it does not exist and update, i.e., patch, it if it does. The spec provided to apply need only contain the required parts of a spec, when creating a resource the API will use defaults for the rest and when updating a resource it will use its current values.
The kubectl replace completely replaces the existing resource with the one defined by the provided spec. replace wants a complete spec as input, including read-only properties supplied by the API like .metadata.resourceVersion, .spec.nodeName for pods, .spec.clusterIP for services, and .secrets for service accounts. kubectl has some internal tricks to help you get that right, but typically the use case for replace is getting a resource spec, changing a property, and then using that changed, complete spec to replace the existing resource.
The kubectl replace command has a --force option which actually does not use the replace, i.e., PUT, API endpoint. It forcibly deletes (DELETE) and then recreates, (POST) the resource using the provided spec.
Updated Answer
My original was rather controversial and I would even say now, in hindsight, half incorrect. So here is an updated answer which I hope will be more helpful:
commands like kubectl patch, replace, delete, create, even edit are all imperative: they tell kubectl exactly what to do
the kubectl apply command is OTOH "declarative" in that it tells kubernetes, here is a desired state (the yaml from the file provided to the apply command), now figure out how to get there: create, patch, replace the object, etc whatever it takes... you get the idea.
So the 2 commands are hugely different.
EG with apply you can give it just the changes you want: it will figure out what properties of the object need to be changed, and leave the other ones alone; if those properties are "immutable" (eg, the nodeName of a pod), it will complain, and if you then repeat the command with --force, it is smart enough to know to do the equivalent of a replace --force.
In general, you should favor apply (with --force when necessary), and only use the imperative commands when the declarative approach does not give the expected result (although I would love to see examples of this -- I'm guessing this would happen only when you would need several steps because of interdependencies that will have negative consequences if done with apply).
The difference between apply and replace is similar to the difference between apply and create.
create / replace uses the imperative approach, while apply uses the declarative approach.
If you used create to create the resource, then use replace to update it. If you used apply to create the resource, then use apply to update it.
Note that both replace and apply require a complete spec, and both create the new resources first before deleting the old ones (unless --force is specified).
you can add option -v=8 when use kubectl, and you will find the log like this
apply --force
patch 422
delete 200
get 200
get 200
get 404
post 201
replace --force
get 200
delete 200
get 404
post 201
kubectl apply .. will use various heuristics to selectively update the values specified within the resource.
kubectl replace ... will replace / overwrite the entire object with the values specified. This should be preferred as you're avoiding the complexity of the selective heuristic update. However some resources like ingresses/load balancers can't really be replaced as they're immutable.
Example of the heuristic update leading to non obvious operation: https://github.com/kubernetes/kubernetes/issues/67135
From: https://github.com/kubernetes/website/blob/master/content/en/docs/concepts/cluster-administration/manage-deployment.md
Disruptive updates
In some cases, you may need to update resource fields that cannot be
updated once initialized, or you may just want to make a recursive
change immediately, such as to fix broken pods created by a
Deployment. To change such fields, use replace --force, which deletes
and re-creates the resource.
I am using the flexible publish plugin for jenkins and found it very useful. My goal is to pass a set of predefined parameters from a job to trigger another downstream job. The downstream job performs an action based on the string match evaluation. But if i dont pass the parameters the job would fail saying
Exception caught evaluating condition: [org.jenkinsci.plugins.tokenmacro.MacroEvaluationException: Unrecognized macro 'multinodeind' in '${multinodeind}'], action = [Fail the build]
I want to do an if else flow. If the parameter is present evaluate it and proceed. Otherwise perform another action. Any clue?
You know you can add parameters with default values under Meta Data → ☑ This build is parameterized → Add parameter, do you? The default values are supposed to be taken if a value for a parameter isn't passed, IIRC.
However, you can use the Conditional BuildStep Plugin in the project configuration's Build section:
A buildstep wrapping any number of other buildsteps, controlling their execution based on a defined condition.