I am having a syntax error when I test my gitlab-ci.yml in CI Lint. Can someone suggest a solution to this problem?
build-production:
stage: build
only:
- master
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
script:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE:$CI_COMMIT_TAG
rules:
- if: $CI_COMMIT_TAG
Status: syntax is incorrect
jobs:build-production config key may not be used with `rules`: only
Documentation is pretty clear :
rules replaces only/except and they can’t be used together in the same job. If you configure one job to use both keywords, the linter returns a key may not be used with rules error.
I suggest to use rules: for both of your conditions :
rules:
- if: '$CI_COMMIT_REF_NAME == "master" && $CI_COMMIT_TAG'
This is not correct, unless you would create a tag master.
See: https://gitlab.sron.nl/help/ci/variables/predefined_variables.md
CI_COMMIT_REF_NAME
The branch or tag name for which project is built.
There is a workaround described here: Gitlab CI: Run Pipeline job only for tagged commits that exist on protected branches
Related
Cloud Build Building Python applications example has the lines below which has _REPO_NAME variable specified.
# [START cloudbuild_python_image_yaml]
# Docker Build
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t',
'us-central1-docker.pkg.dev/${PROJECT_ID}/${_REPO_NAME}/myimage:${SHORT_SHA}', '.']
The Substituting variable values documentation has $REPO_NAME: the name of your repository but does not have _REPO_NAME.
Please help understand where it is defined and what it is.
With Cloud Build, all the user managed substitution variables start with _. If you variable doesn't have a _, like $REPO_NAME, it's a auto generated environment variable.
Therefore in your example you have to provide the $_REPO_NAME and $_BUCKET_NAME if you want to start your Cloud Build process. Else it will fail because it doesn't know that variable.
Why using $_REPO_NAME instead of $REPO_NAME? IMO, it's a mistake. A developer trick is to replace the auto-generated variable by a user managed variable during the tests. Like that, you don't have to push new code to git to test your build pipeline, you simply have to set that variable manually (with gcloud command).
And it might have been forgotten when that code example has been released. Just an assumption.
I'm deploying my Node apps to Google Cloud Run using Cloud Build and I want to run some tests during the build. My tests require some environment variables, so I have been following this guide to achieve this.
The guide makes the following note:
Note: To use the secret in an environment variable, you need to prefix
the variable name with an underscore "_" and escape the value using
'('. For example: _VARIABLE_NAME=$(cat password.txt) && echo -n
\)_VARIABLE_NAME.
However, I am not quite sure how to implement this.
I have attempted the following in my cloudbuild.yaml.
- id: Execute tests
name: node
args: ['_VAR_ONE=$(cat var-one.txt)', '_VAR_TWO=$(cat var-two.txt)', 'jest -V']
Which returns the following: Error: Cannot find module '/workspace/_VAR_ONE=$(cat var-one.txt)'.
I also tried a few variations of the escape that the above note mentions, but they result in the same error.
What's the best way to get the secrets into my code as environment variables?
Also, if I need to use multiple environment variables, is it better to use Cloud KMS with an .env file?
Thanks!
It looks like you are incorrectly using the entrypoint provided by the node image. You are effectively running the command:
node _VAR_ONE=$(cat var-one.txt) _VAR_TWO=$(cat var-two.txt) jest -V
I want to digress for a moment and say this pattern does not work in Node, you need to specify the environment variables first before calling node, for example VAR_ONE=$(cat foo.txt) VAR_TWO=bar node run test
Anyway, I think what you want to run is:
_VAR_ONE=$(cat var-one.txt) _VAR_TWO=$(cat var-two.txt) jest -V
This is how we will do that - Assuming you have a previous step where you write out the contents of the secret into the files var-one.txt and var-two.txt in a previous step - here is how you would use it in the node step, it's just the standard way you use environment variables when running a command from the command line:
- id: Execute tests
name: node
entrypoint: '/bin/bash'
args:
'-c',
'_VAR_ONE=$(cat var-one.txt) _VAR_TWO=$(cat var-two.txt) jest -V'
]
You need to ensure in the node environment you are using the variables as specified (ie. process.env._VAR_ONE or process.env._VAR_TWO). I don't think you need to have the _ character prefixed here but I haven't tested it to confirm that. You can try the above and it should get you much further I think.
Trying to understand how the docker-compose file was created as I want to replicate this into a kubernetes deployment yaml file.
In reference to a cookiecutter-django's docker-compose production.yaml file:
...
services:
django: &django
...
By docker-compose design, the name of service here is already defined as django but then I noticed this extra bit &django. This made me wonder why its here. Further down, I noticed the following:
...
celeryworker:
<<: *django
...
I don't understand how that works. The docker-compose docs have no reference or mention for using << let alone, making a reference to a named service like *django.
Can anyone explain how the above work and how do I replicate it to a kubernetes deployment or services yaml file (or both?) if possible?
Edit:
The question that #jonsharpe shared was similar but the answer wasn't clear to me on how its used.
There are three different things happening, and none of them are specifically compose syntax, rather they are yaml syntax.
First is defining an anchor with the & followed by a name. That's similar to defining a variable to use later in the yaml, with the value matching the value of the yaml object where it appears.
Next is the alias, specified with * and the same name as the anchor. That uses the anchor in the second location in the yaml file.
Last is a mapping merge using the << syntax, which merges all of the mapped values in the alias with the rest of the values in the current map, allowing you to override values in the saved anchor with values specific to that section of the compose file.
To dig more into this, try searching on "yaml anchors and aliases". The first hit for me is this blog post: https://medium.com/#kinghuang/docker-compose-anchors-aliases-extensions-a1e4105d70bd
I need to make a request from a pipeline, which is optional on windows- and linux-based agents. The request may fail for whatever reason, but the pipeline should continue anyway - always.
All CICD-Systems I've worked with so far have the possibility to mark a task as optional or - more specifically - to ignore errors:
Example: https://docs.gitlab.com/ee/ci/yaml/#allow_failure
job1:
stage: test
script:
- execute_script_that_will_fail
allow_failure: true
I have not found a similar concept in GoCD - is there one I am not aware of?
You can use the run_if attribute of the task to achieve that. It takes passed, failed or any. Passed is the default value, that is only if the current task is passed it moves on to the next one. Failed is the exact opposite of this. I generally use it for setting up assertions within the job. Any is basically you don't care the outcome of that task. GoCD will not fail irrespective of the outcome of that task within the job.
exec:
run_if: any
working_directory: dir
command: make
arguments:
- -j3
- docs
- install
References
From GoCD's documentation - https://docs.gocd.org/current/advanced_usage/dev_conditional_task_execution.html
If you're using the YAML to define your pipelines - https://github.com/tomzo/gocd-yaml-config-plugin#tasks
This is a self-answered question.
Please provide edits, additional points of view and input if needed.
What is the best practice for conditionally enforcing states (depending on other command outputs)?
Here's my case:
# vim: set syntax=yaml:
# Ensures that outbound connections are allowed for httpd
httpd:selinux:
cmd.run:
- name: /usr/sbin/setsebool -P httpd_can_network_connect 1
Now, I only want to run this if SELinux is enabled (enforced).
1)
One way of solving it is by using the onlyif parameter of salt.states.cmd.run:
# Ensures that outbound connections are allowed for httpd
httpd:selinux:
cmd.run:
- name: /usr/sbin/setsebool -P httpd_can_network_connect 1
- onlyif:
- 'if [[ $(getenforce) == "Disabled" ]]; then exit 1; else exit 0; fi' # if SELinux is disabled, then don't enforce this state
This is a very quick and easy way of solving this problem and as we're dealing with Shell commands in this example it is probably the most preferable way.
Please note that onlyif is dependent on the return status code of the command that is being tested, so if more flexibility is needed then this might not be what you're looking for.
2)
Another way is to access execution modules with jinja templating, wrap your state in this conditional:
{% if salt.selinux.getenforce() == "Disabled" %}
httpd:selinux:
...
{% endif %}
This is a more flexible solution, but also takes up a bit more space.
More info on: salt.modules.selinux.getenforce()