I have below conditional logic in the Drone YAML, but I saw that the control is not going inside that, even though drone branch is "develop". How to fix this, did I do anything wrong?
commands:
- "./gradlew clean build"
- echo "${DRONE_BRANCH}"
- echo "${DRONE_BRANCH}" = "develop"
- >
if [ "${DRONE_BRANCH}" = "develop" ]; then
export CLOUD_USER_KEY=$STAGE_CLOUD_USER_KEY
export HOST_NAME="11.22.111.111"
fi
- >
if [ "${DRONE_BRANCH}" = "master" ]; then
export CLOUD_USER_KEY=$PROD_CLOUD_USER_KEY
export HOST_NAME="11.22.111.112"
fi
- echo "CLOUD_USER_KEY "${CLOUD_USER_KEY}
- echo "HOST NAME "${HOST_NAME}
> is the YAML indicator for a folded block scalar. Line folding in YAML means that a line break between two non-empty consecutive lines is changed into a space. This is not what you want when writing bash commands into YAML!
For example, this simple folded block scalar (using only parts of your original YAML file):
- >
export CLOUD_USER_KEY=$STAGE_CLOUD_USER_KEY
export HOST_NAME="11.22.111.111"
will be parsed as:
- "export CLOUD_USER_KEY=$STAGE_CLOUD_USER_KEY export HOST_NAME=\"11.22.111.111\"\n"
Which won't work. Use a literal block scalar instead which preserves line endings:
- |
export CLOUD_USER_KEY=$STAGE_CLOUD_USER_KEY
export HOST_NAME="11.22.111.111"
Related
I would like to automatically format the code when I do commit using rustfmt the same way as I did it before for clang-format -i. I.e. format only the lines of code which has been updated in the commit without touching other code. How to do it?
It might be done using git pre-commit hook in the following way:
Add file pre-commit to the folder .githooks in your repo with the following text:
#!/bin/bash
exe=$(which rustfmt)
if [ -n "$exe" ]
then
# field separator to the new line
IFS=$'\n'
for line in $(git status -s)
do
# if added or modified
if [[ $line == A* || $line == M* ]]
then
# check file extension
if [[ $line == *.rs ]]
then
# format file
rustfmt $(pwd)/${line:3}
# add changes
git add $(pwd)/${line:3}
fi
fi
done
else
echo "rustfmt was not found"
fi
Run in your repo folder:
chmod +x .githooks/pre-commit
git config core.hooksPath .githooks
To make it work for clang-format you need to replace rustfmt with clang-format -i and do corresponding modifications in the check for file extension (cpp\h\hpp\etc).
I am trying to create a variable in gitlab-ci.yaml based on the name of the branch.
Suppose I am pushing to a branch named 3.2.7
Here is the situation:
include:
- template: "Workflows/Branch-Pipelines.gitlab-ci.yml"
variables:
PRODUCTION_BRANCH: "master"
STAGING_BRANCH: (\d)\.(\d)\.(\d)
.deploy_rules:
rules:
- if: '$CI_COMMIT_BRANCH =~ /$STAGING_BRANCH/'
variables:
SERVER_PORT: 3007 # TODO: should be 300d ; d is the second digit
I want to generate 3002 inline using regex matching.
How can I do this?
I have done some research and seems I have to use sed but I am not sure if it is the best way to do it and how to do it.
TO MAKE THE PROBLEM SIMPLER
include:
- template: "Workflows/Branch-Pipelines.gitlab-ci.yml"
variables:
TEST_VAR: sed -E 's/(\d)\.(\d)\.(\d)/300\2/gm;t;d' <<< $CI_COMMIT_BRANCH
stages:
- temp
temp:
stage: temp
script:
- echo $TEST_VAR
Should be echoing 3002 but it is echoing sed -E 's/(\d)\.(\d)\.(\d)/300\2/gm;t;d' <<< 3.2.7
You can't use variables in the regex pattern. You just have to write the regex verbatim, it cannot be directly parameterized. You also cannot use sed or other Linux utilities in variables: or other parts of your yaml. You're bound to the limitations of YAML specification and features provided by GitLab.
However, there is an option available to you that will fit your stated use case.
Dynamic variables
TEST_VAR: sed -E 's/(\d).(\d).(\d)/300\2/gm;t;d' <<< $CI_COMMIT_BRANCH
While you can't use sed or other utilities directly in variables: declarations, you can use dotenv artifacts via artifacts:reports:dotenv to set variables dynamically.
For example, a job can use sed or whatever other utilities you like to create variables which will be used by the rest of the pipeline.
stages:
- temp
create_variables:
stage: .pre
script:
- TEST_VAR="$(sed -E 's/(\d)\.(\d)\.(\d)/300\2/gm;t;d' <<< ${CI_COMMIT_BRANCH})"
- echo "TEST_VAR=${TEST_VAR}" >> dotenv.txt
artifacts:
reports:
dotenv: dotenv.txt
temp:
stage: temp
script:
- echo $TEST_VAR
Here, the .pre stage is used, which is a special stage that is always ordered before every other stage. The dotenv artifact from the create_variables job will dynamically create variables for the jobs in subsequent stages that receive the artifact.
How to use if else condition inside the gitlab-CI.
I have below code:
deploy-dev:
image: testimage
environment: dev
tags:
- kubectl
script:
- kubectl apply -f demo1 --record=true
- kubectl apply -f demo2 --record=true
Now I want to add a condition something like this
script:
- (if [ "$flag" == "true" ]; then kubectl apply -f demo1 --record=true; else kubectl apply -f demo2 --record=true);
Could someone provide the correct syntax for the same? Is there any documentation for the conditions (if-else, for loop) in gitlabci?
Hereunder three syntax options for that kind of statement. From gitlab-ci documentation :
Using shell variable
deploy-dev:
image: testimage
environment: dev
tags:
- kubectl
script:
- if [ "$flag" == "true" ]; then MODULE="demo1"; else MODULE="demo2"; fi
- kubectl apply -f ${MODULE} --record=true
Using shell variable with yaml multiline block
deploy-dev:
image: testimage
environment: dev
tags:
- kubectl
script:
- >
if [ "$flag" == "true" ]; then
kubectl apply -f demo1 --record=true
else
kubectl apply -f demo2 --record=true
fi
Using gitlab rules
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "schedule"'
when: never
- if: '$CI_PIPELINE_SOURCE == "push"'
when: never
- when: always
Using gitlab templates and variables
demo1-deploy-dev:
extends: .deploy-dev
only:
variables: [ $flag == "true" ]
variables:
MODULE: demo1
demo2-deploy-dev:
extends: .deploy-dev
only:
variables: [ $flag == "false" ]
variables:
MODULE: demo2
.deploy-dev:
image: testimage
environment: dev
tags:
- kubectl
script:
- kubectl apply -f ${MODULE} --record=true
Note that with GitLab 13.3 (August 2020), there is an improvement to the if-else rule syntax:
CI/CD rules:if support logical expressions with parentheses
If you use the rules keyword with if clauses, it’s now even more powerful, with support for bracketed expressions evaluated by the pipeline processor.
You can use more complex and efficient AND (&&) / OR (||) expressions, making your pipelines rules more logical, powerful, and easier to manage.
See Documentation and Issue.
And, with GitLab 13.8 (January 2021)
Support variables for pipeline rules
Previously, the rules keyword was limited in scope and only determined if a job should be included or excluded from pipelines. In this release, you can now decide if certain conditions are met and subsequently override variables in jobs, providing you with more flexibility when configuring your pipelines.
See Documentation and Issue.
With GitLab 13.12 (May 2021):
Support variables in CI/CD pipeline 'workflow:rules'
Previously, the rules keyword was limited in scope and only determined if a job should be included or excluded from pipelines. In 13.8, we added the ability to use the variables keyword with rules to set variable values in a job based on which rule matched.
In this release we’ve extended this ability to workflow: rules, so you can set variable values for the whole pipeline if certain conditions match.
This helps you make your pipelines even more flexible.
See Documentation and Issue.
I think you need to just add a semicolon and closing "fi" at the end.
I couldn't find a link to documentation.
script:
- (if [ "$flag" == "true" ]; then kubectl apply -f demo1 --record=true; else kubectl apply -f demo2 --record=true; fi);
In addition, in the case of a multiline block if you want or need to preserve line breaks you can use the pipe character:
script: |
if [ "$flag" == "true" ]; then
kubectl apply -f demo1 --record=true
else
kubectl apply -f demo2 --record=true
fi
To go deeper, visit https://yaml-multiline.info/
You may consider checking rules
It allows for a list of individual rule objects to be evaluated in order, until one matches and dynamically provides attributes to the job.
Available rule clauses include:
if (similar to only:variables)
changes (same as only:changes)
exists
Example:
job:
script: "echo Hello, Rules!"
rules:
- if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"'
when: always
- if: '$VAR =~ /pattern/'
when: manual
- when: on_success
This worked for me when using powershell based gitlab scripts:
script:
- 'if ($flag -eq "true") { kubectl apply -f demo1 --record=true; } else { kubectl apply -f demo2 --record=true; }'
I have branch folder "feature-set" under this folder there's multibranch
I need to run the below script in my Jenkinsfile with a condition if this build runs from any branches under the "feature-set" folder like "feature-set/" then run the script
the script is:
sh """
if [ ${env.BRANCH_NAME} = "feature-set*" ]
then
echo ${env.BRANCH_NAME}
branchName='${env.BRANCH_NAME}' | cut -d'\\/' -f 2
echo \$branchName
npm install
ng build --aot --output-hashing none --sourcemap=false
fi
"""
the current output doesn't get the condition:
[ feature-set/swat5 = feature-set* ]
any help?
I would re-write this to be primarily Jenkins/Groovy syntax and only go to shell when required.
Based on the info you provided I assume your env.BRANCH_NAME always looks like `feature-set/
// Echo first so we can see value if condition fails
echo(env.BRANCH_NAME)
// startsWith better than contains() based on current usecase
if ( (env.BRANCH_NAME).startsWith('feature-set') ) {
// Split branch string into list based on delimiter
List<String> parts = (env.BRANCH_NAME).tokenize('/')
/**
* Grab everything minus the first part
* This handles branches that include additional '/' characters
* e.g. 'feature-set/feat/my-feat'
*/
branchName = parts[1..-1].join('/')
echo(branchName)
sh('npm install && ng build --aot --output-hashing none --sourcemap=false')
}
This seems to be more on shell side. Since you are planning to use shell if condition the below worked for me.
Administrator1#XXXXXXXX:
$ if [[ ${BRANCH_NAME} = feature-set* ]]; then echo "Success"; fi
Success
Remove the quotes and add an additional "[]" at the start and end respectively.
The additional "[]" works as regex
I am currently using a template_file to provision user_data into an aws_launch_configuration, like so:
resource "aws_launch_configuration" "launch_config" {
...
user_data = "${data.template_file.init.rendered}"
}
data "template_file" "init" {
template = "${file("router-init.sh.tpl")}"
vars {
hub_ip_addresses = "${join(",", aws_instance.gridHub.*.private_ip)}"
}
}
I am feeding in a variable (i.e. hub_ip_addresses) into the router-init.sh.tpl file, and in this file I am making use of the argument like so:
`#!/bin/sh
...
IFS=',' read -r -a array <<< "$hub_ip_addresses"
for element in "${array[#]}"
do
#do stuff with $element
done
Basically, I am splitting the string based on a delimiter, and then looping through each ip address in the array.
This bash script works fine when I run it on my local machine -- however, when terraform executes it, it throws a error:
* data.template_file.init: data.template_file.init: failed to render : parse error at 13:25: expected expression but found invalid sequence "#"
I'm supposing the '#' symbol is causing an issue. Is there a reason why this is so? Do I need to escape it with a '\' ?
EDIT: Not sure if related to this issue, but in the preceeding line in the bash script, IFS=',' read -r -a array <<< "$hub_ip_addresses", the <<< seems to be causing everything else that follows to look as if they are inside a comment (i.e. greyed out as if it was within a quotation mark ').)
You need to escape the $ characters in your template by doubling them up or Terraform will attempt to interpolate them as the input variables to the template.
The template docs cover this briefly although the example given is for inline templates rather than for all templates, including those that are loaded with the file() function.
So something like:
#!/bin/sh
...
IFS=',' read -r -a array <<< "$hub_ip_addresses"
for element in "$${array[#]}"
do
#do stuff with $$element
done