https://github.com/jenkinsci/job-dsl-plugin/wiki/JCasC provides the following example of a separate groovy file referenced in the Jenkins Configuration as Code (CasC) syntax.
jobs:
- providedEnv:
SUPERHERO: 'Midnighter'
- file: ./jobdsl/job.groovy
//job.groovy
job('awesome-job') {
description("favorite job of ${SUPERHERO}")
}
I'm looking for a way of defining a few very similar jobs which only differ by a value or two. In the example above, the SUPERHERO variable looks to be a global, but I need a way to reuse the same groovy include with per-include variables.
Pseudo-code example:
jobs:
- file: ./jobdsl/job.groovy
providedEnv:
SUPERHERO: 'Superman'
- file: ./jobdsl/job.groovy
providedEnv:
SUPERHERO: 'Batman'
Does such a construct exist?
This appears to work:
jobs:
- providedEnv:
SUPERHERO: 'Batman'
- file: ./jobdsl/job.groovy
- providedEnv:
SUPERHERO: 'Superman'
- file: ./jobdsl/job.groovy
But I don't know if I'm exploiting a fragile, undocumented behavior or not.
I would prefer to see an explicit association of the environment variables with the file reference; something like this:
jobs:
- file: ./jobdsl/job.groovy
providedEnv:
SUPERHERO: 'Batman'
- file: ./jobdsl/job.groovy
providedEnv:
SUPERHERO: 'Superman'
...but alas this (guess at a) syntax doesn't work. (Jenkins starts but the jobs are absent.)
How about just doing this?
jobs:
- providedEnv:
JOB_NAMES: 'Batman,Superman'
- file: ./jobdsl/jobs.groovy
You then have a loop inside jobs.groovy to split JOB_NAMES into a list.
Related
All afternoon I have been trying to get my head around concatenating a parameter in an ADO template. The parameter is a source path and in the template a next folder level needs to be added. I would like to achieve this with a "simple" concatenation.
The simplified template takes the parameter and uses it to form the inputPath for a PowerShell script, like this:
parameters:
sourcePath: ''
steps:
- task: PowerShell#2
inputs:
filePath: 'PSRepo/Scripts/MyPsScript.ps1'
arguments: '-inputPath ''$(sourcePath)/NextFolder''
I have tried various ways to achieve this concatenation:
'$(sourcePath)/NextFolder'
see above
'$(variables.sourcePath)/NextFolder'
I know sourcePath is not a variable, but tried based on the fact that using a parameter in a task condition it apparently only works when referencing through variables
'${{ parameters.sourcePath }}/NextFolder'
And some other variations, all to no avail.
I also tried to introduce a variables section in the template, but that is not possible.
I have searched the internet for examples/documentation, but no direct answers and other issues seemed to hint to some solution, but were not working.
I will surely be very pleased if someone could help me out.
Thanx in advance.
We can add the variables in our temp yaml file and pass the sourcePath to the variable, then we can use it. Here is my demo script:
Main.yaml
resources:
repositories:
- repository: templates
type: git
name: Tech-Talk/template
trigger: none
variables:
- name: Test
value: TestGroup
pool:
# vmImage: windows-latest
vmImage: ubuntu-20.04
extends:
template: temp.yaml#templates
parameters:
agent_pool_name: ''
db_resource_path: $(System.DefaultWorkingDirectory)
# variable_group: ${{variables.Test}}
temp.yaml
parameters:
- name: db_resource_path
default: ""
# - name: 'variable_group'
# type: string
# default: 'default_variable_group'
- name: agent_pool_name
default: ""
stages:
- stage:
jobs:
- job: READ
displayName: Reading Parameters
variables:
- name: sourcePath
value: ${{parameters.db_resource_path}}
# - group: ${{parameters.variable_group}}
steps:
- script: |
echo sourcePath: ${{variables.sourcePath}}
- powershell: echo "$(sourcePath)"
Here, I just use the workingDirectory to as the test path. You can use the variables also.
Attach my build result:
Thanx, Yujun. In meantime did get it working. Apparently there must have been some typo that did block the script from executing right as the se solution looks like one of the options mentioned above.
parameters:
sourcePath: ''
steps:
- task: PowerShell#2
inputs:
filePath: 'PSRepo/Scripts/MyPsScript.ps1'
arguments: '-inputPath ''$(sourcePath)/NextFolder''
I am new to YAML and build pipelines. I am receiving the following error, can anyone advice what's wrong with the target folder.
Unhandled: Input required: TargetFolder
[warning]Directory 'D:\a\1\a' is empty. Nothing will be added to build
artifact 'drop'.
Below is my YAML file:
# Build app using Azure Pipelines
pool:
vmImage: 'vs2017-win2016'
steps:
- script: echo hello world
- task: NodeTool#0
inputs:
versionSpec: '8.x'
- task: CopyFiles#1
displayName: 'Copy Files to: $(Build.ArtifactStagingDirectory)'
inputs:
SourceFolder: '$(build.sourcesdirectory)'
Contents:
\C:\VSCodeGit\CollMod.Web\Web.config\
TartgetFolder: '$(Build.ArtifactStagingDirectory)'
condition: succeededOrFailed()
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: drop'
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
condition: succeededOrFailed()
I think it's the contents field that looks to be invalid here.
The docs at https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/copy-files?view=vsts&tabs=yaml and further documentation on https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/file-matching-patterns?view=vsts which both give some great examples.
If you're unsure set the contents to **/* which will copy absolutely everything in the $(build.sourcesdirectory), but it will give you a feel for the shape of the directory structure so that you can change **/* into something more selective and scoped for the file(s) you want to copy.
The Source folder should be : Build.SourcesDirectory instead of '$(build.sourcesdirectory)'
This is from : https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#build-variables
I have three stages in my CI file, they all have only/except like this:
test:
only:
- tags
except:
- branches
script:
- npm run test
Seems redundant to have the only/except in three places. Is there a way to set this at the top level of the script config? Don't see anything like that in the docs.
You can use the map merging feature: https://docs.gitlab.com/ee/ci/yaml/#special-yaml-features
.job_template: &job_definition
only:
- tags
except:
- branches
test1:
<<: *job_definition
script:
- npm run test
test2:
<<: *job_definition
script:
- # ...
I am using assemble module to make one file from multiple files. I have a list of files and from that list I want only .pub files to be assembled, but I am not sure how to use this.
list of files
root_rsa_comiskey-v01
root_rsa_comiskey-v01.pub
root_rsa_comiskey-v02
root_rsa_comiskey-v02.pub
root_rsa_comiskey-v03
root_rsa_comiskey-v03.pub
root_rsa_comiskey-v05
root_rsa_comiskey-v05.pub
Playbook file:
---
- hosts: 10.1.31.81
become_user: yes
tasks:
- name: list files
shell: ls -1 /tmp/root_ssh_key*
register: dumpfiles
- name: fetch files
assemble: src=/tmp/root_ssh_key/ dest=/tmp/root_ssh_key/id_rsa regexp='(*.pub)'
register: test
- debug: var=test
Regexp parameter is a Python regular expression – not shell glob.
If you want to join all files ending with pub use:
assemble: src=/tmp/root_ssh_key/ dest=/tmp/root_ssh_key/id_rsa regexp='pub$'
Here's my target file:
Sonatype Nexus
# ==============
# This is the most basic configuration of Nexus.
# Jetty section
application-port=8081
application-host=0.0.0.0
nexus-webapp=${bundleBasedir}/nexus
nexus-webapp-context-path=/nexus
# Nexus section
nexus-work=/opt/nexuswork
runtime=${bundleBasedir}/nexus/WEB-INF
I know there's an easy way to do this with regex or a simple sed script:
sed -i 's/${bundleBasedir}\/..\/my\/second\/path\/002\/\/nexus/\/myfirstdir001\/g'
However, I would, ideally, prefer the saltstack way.
I would like it to look something like this:
Sonatype Nexus
# ==============
# This is the most basic configuration of Nexus.
# Jetty section
application-port=8081
application-host=0.0.0.0
nexus-webapp=/my/second/path/002/nexus # changed
nexus-webapp-context-path=/nexus
# Nexus section
nexus-work=/opt/nexuswork
runtime=/myfirstdir001/nexus/WEB-INF # changed
I haven't yet made sense of the saltstack documentation on this.
Saltstack's documentation for salt.states.file.replace seems fairly straightforward:
http://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html#salt.states.file.replace
Here's what I tried:
/opt/nexus-2.8.0/conf/nexus.properties
file: # state
- replace
- pattern: '\$\{bundleBasedir\}' # without escapes: '${bundleBasedir}/nexus'
- repl: '/my/second/path/002/nexus'
# - name: /opt/nexus-2.8.0/conf/nexus.properties
# - count=0
# - append_if_not_found=False
# - prepend_if_not_found=False
# - not_found_content=None
# - backup='.bak'
# - show_changes=True
- pattern: '\$\{bundleBasedir\}\/WEB-INF' # without escapes: ${bundleBasedir}/WEB-INF
- repl: '/myfirstdir001/'
I could maybe try multiple state IDs, but that seems inelegant.
If there's anything else I'm fuffing up, please advise!
I'd shore love to find a solution to this.
Also, if there's any demand for people improving the salt documentation, I think my team could be convinced to pitch in some.
Here's the closest thing I've found to someone else asking this question:
http://comments.gmane.org/gmane.comp.sysutils.salt.user/15138
For such a small file I would probably go with a template as ahus1 suggested.
If the file was bigger and/or we didn't want to control other lines just ensure that those two are correct, I think multiple state IDs (as mentioned by OP) is a good way to go. Something like:
/opt/nexus-2.8.0/conf/nexus.properties-jetty:
file:
- replace
- name: /opt/nexus-2.8.0/conf/nexus.properties
- pattern: '\$\{bundleBasedir\}' # without escapes: '${bundleBasedir}/nexus'
- repl: '/my/second/path/002/nexus'
/opt/nexus-2.8.0/conf/nexus.properties-nexus:
file:
- replace:
- name: /opt/nexus-2.8.0/conf/nexus.properties
- pattern: '\$\{bundleBasedir\}\/WEB-INF' # without escapes: ${bundleBasedir}/WEB-INF
- repl: '/myfirstdir001/'
I have a similar setup in my configuration but I use salt.states.file.line to replace some lines with my values. In addition I used salt.states.file.managed with a template and replace: False to initialize the file if it's missing but once it exists, only the line states are doing changes.
The salt way to do this as I understand it: Place a template file for nexus.properties inside salt and use file.managed like shown in the docs http://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html
You will end up with something like:
/opt/nexus-2.8.0/conf/nexus.properties:
file.managed:
- source: salt://nexus/nexus.properties.jinja
- template: jinja
- defaults:
bundleBasedir: "..."
You'll then use Jinja templating in your file:
# Jetty section
application-port=8081
application-host=0.0.0.0
nexus-webapp={{ bundleBasedir }}/nexus
nexus-webapp-context-path=/nexus
See here for Jinja templating: http://docs.saltstack.com/en/latest/ref/renderers/all/salt.renderers.jinja.html
I hope it helps.