Cloud Build Building Python applications example has the lines below which has _REPO_NAME variable specified.
# [START cloudbuild_python_image_yaml]
# Docker Build
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t',
'us-central1-docker.pkg.dev/${PROJECT_ID}/${_REPO_NAME}/myimage:${SHORT_SHA}', '.']
The Substituting variable values documentation has $REPO_NAME: the name of your repository but does not have _REPO_NAME.
Please help understand where it is defined and what it is.
With Cloud Build, all the user managed substitution variables start with _. If you variable doesn't have a _, like $REPO_NAME, it's a auto generated environment variable.
Therefore in your example you have to provide the $_REPO_NAME and $_BUCKET_NAME if you want to start your Cloud Build process. Else it will fail because it doesn't know that variable.
Why using $_REPO_NAME instead of $REPO_NAME? IMO, it's a mistake. A developer trick is to replace the auto-generated variable by a user managed variable during the tests. Like that, you don't have to push new code to git to test your build pipeline, you simply have to set that variable manually (with gcloud command).
And it might have been forgotten when that code example has been released. Just an assumption.
I'd like to add a transformer to my kustomize setup, one that includes a dynamic value for the case where the local tag needs to override the production tag. If there's a simpler and better way to do this that would be great. I've looked through the list of transformers and generators to see if there was a way to provide this value at runtime (though I think specifically kustomize is designed to never use runtime values).
I can specify something like this:
images:
- name: my-image
newTag: my-sha1
The problem is to change the my-sha1 value after each new local build to then pick that image when I go to apply the local deployment.
How can I set newTag after I run a build locally to match a tag for the image I made locally? I can easily obtain the latest build tag and provide with to kubectl apply -f, but I'm not seeing a flag or environment variable, or something to do so.
I'm deploying my Node apps to Google Cloud Run using Cloud Build and I want to run some tests during the build. My tests require some environment variables, so I have been following this guide to achieve this.
The guide makes the following note:
Note: To use the secret in an environment variable, you need to prefix
the variable name with an underscore "_" and escape the value using
'('. For example: _VARIABLE_NAME=$(cat password.txt) && echo -n
\)_VARIABLE_NAME.
However, I am not quite sure how to implement this.
I have attempted the following in my cloudbuild.yaml.
- id: Execute tests
name: node
args: ['_VAR_ONE=$(cat var-one.txt)', '_VAR_TWO=$(cat var-two.txt)', 'jest -V']
Which returns the following: Error: Cannot find module '/workspace/_VAR_ONE=$(cat var-one.txt)'.
I also tried a few variations of the escape that the above note mentions, but they result in the same error.
What's the best way to get the secrets into my code as environment variables?
Also, if I need to use multiple environment variables, is it better to use Cloud KMS with an .env file?
Thanks!
It looks like you are incorrectly using the entrypoint provided by the node image. You are effectively running the command:
node _VAR_ONE=$(cat var-one.txt) _VAR_TWO=$(cat var-two.txt) jest -V
I want to digress for a moment and say this pattern does not work in Node, you need to specify the environment variables first before calling node, for example VAR_ONE=$(cat foo.txt) VAR_TWO=bar node run test
Anyway, I think what you want to run is:
_VAR_ONE=$(cat var-one.txt) _VAR_TWO=$(cat var-two.txt) jest -V
This is how we will do that - Assuming you have a previous step where you write out the contents of the secret into the files var-one.txt and var-two.txt in a previous step - here is how you would use it in the node step, it's just the standard way you use environment variables when running a command from the command line:
- id: Execute tests
name: node
entrypoint: '/bin/bash'
args:
'-c',
'_VAR_ONE=$(cat var-one.txt) _VAR_TWO=$(cat var-two.txt) jest -V'
]
You need to ensure in the node environment you are using the variables as specified (ie. process.env._VAR_ONE or process.env._VAR_TWO). I don't think you need to have the _ character prefixed here but I haven't tested it to confirm that. You can try the above and it should get you much further I think.
I am learning Kubernetes recently, and I am not very clear about the difference between "kubectl apply" and "kubectl replace". Is there any situation that we can only use one of them?
I have written up a thorough explanation of the differences between apply, replace, and patch: Kubernetes Apply vs. Replace vs. Patch. It includes an explanation that the current top-ranked answer to this question is wrong.
Briefly, kubectl apply uses the provided spec to create a resource if it does not exist and update, i.e., patch, it if it does. The spec provided to apply need only contain the required parts of a spec, when creating a resource the API will use defaults for the rest and when updating a resource it will use its current values.
The kubectl replace completely replaces the existing resource with the one defined by the provided spec. replace wants a complete spec as input, including read-only properties supplied by the API like .metadata.resourceVersion, .spec.nodeName for pods, .spec.clusterIP for services, and .secrets for service accounts. kubectl has some internal tricks to help you get that right, but typically the use case for replace is getting a resource spec, changing a property, and then using that changed, complete spec to replace the existing resource.
The kubectl replace command has a --force option which actually does not use the replace, i.e., PUT, API endpoint. It forcibly deletes (DELETE) and then recreates, (POST) the resource using the provided spec.
Updated Answer
My original was rather controversial and I would even say now, in hindsight, half incorrect. So here is an updated answer which I hope will be more helpful:
commands like kubectl patch, replace, delete, create, even edit are all imperative: they tell kubectl exactly what to do
the kubectl apply command is OTOH "declarative" in that it tells kubernetes, here is a desired state (the yaml from the file provided to the apply command), now figure out how to get there: create, patch, replace the object, etc whatever it takes... you get the idea.
So the 2 commands are hugely different.
EG with apply you can give it just the changes you want: it will figure out what properties of the object need to be changed, and leave the other ones alone; if those properties are "immutable" (eg, the nodeName of a pod), it will complain, and if you then repeat the command with --force, it is smart enough to know to do the equivalent of a replace --force.
In general, you should favor apply (with --force when necessary), and only use the imperative commands when the declarative approach does not give the expected result (although I would love to see examples of this -- I'm guessing this would happen only when you would need several steps because of interdependencies that will have negative consequences if done with apply).
The difference between apply and replace is similar to the difference between apply and create.
create / replace uses the imperative approach, while apply uses the declarative approach.
If you used create to create the resource, then use replace to update it. If you used apply to create the resource, then use apply to update it.
Note that both replace and apply require a complete spec, and both create the new resources first before deleting the old ones (unless --force is specified).
you can add option -v=8 when use kubectl, and you will find the log like this
apply --force
patch 422
delete 200
get 200
get 200
get 404
post 201
replace --force
get 200
delete 200
get 404
post 201
kubectl apply .. will use various heuristics to selectively update the values specified within the resource.
kubectl replace ... will replace / overwrite the entire object with the values specified. This should be preferred as you're avoiding the complexity of the selective heuristic update. However some resources like ingresses/load balancers can't really be replaced as they're immutable.
Example of the heuristic update leading to non obvious operation: https://github.com/kubernetes/kubernetes/issues/67135
From: https://github.com/kubernetes/website/blob/master/content/en/docs/concepts/cluster-administration/manage-deployment.md
Disruptive updates
In some cases, you may need to update resource fields that cannot be
updated once initialized, or you may just want to make a recursive
change immediately, such as to fix broken pods created by a
Deployment. To change such fields, use replace --force, which deletes
and re-creates the resource.
I would like to define default host(s) for a specific task in case no roles or hosts were explicitly provided from the command line.
I can use env.all_hosts to check the lists of current hosts is empty, but I don't know how to set it to some host.
I also considered using #hosts(), but it seems to override the input from the command line, which is undesirable.
You can override the #hosts decorator from the command line, using the command-line host lists, as described in the first point of Fabric's Order of Precedence:
Order of precedence
We’ve been pointing out which methods of setting host lists trump the others, as we’ve gone along. However, to make things clearer, here’s a quick breakdown:
Per-task, command-line host lists (fab mytask:host=host1) override absolutely everything else.
Per-task, decorator-specified host lists (#hosts('host1')) override the env variables.
Globally specified host lists set in the fabfile (env.hosts = ['host1']) can override such lists set on the command-line, but only if you’re not careful (or want them to.)
Globally specified host lists set on the command-line (--hosts=host1) will initialize the env variables, but that’s it.
Give that first option a try, and let us know if you have any other questions.