Ansible jinja2 templating - templates

In Ansible, I currently have vars set out like this
container_a_version: 1
container_b_version: 6
container_c_version: 3
container_d_version: 2
...
containers:
- container_a
- container_b
- container_c
- container_d
...
Each container has its own template file which has a line like this
image: registry.sportpesa.com/sportpesa/global/container-a:{{ container_a_version }}
My playbook
- name: Copy templates
template:
src: templates/docker-compose-{{ item }}.yml
dest: /srv/docker-compose-{{ item }}.yml
loop: "{{ containers }}"
If I want to deploy only container a and c. I change my variable to this
containers:
- container_a
# - container_b
- container_c
# - container_d
What I want to do condense my variables to just have 1 var like this
containers:
- { name: a, version: 1 }
- { name: b, version: 6 }
- { name: c, version: 3 }
- { name: d, version: 2 }
However I'm not sure how I can call the container version in my template. Is this possible?

For example, the playbook
shell> cat pb.yml
- hosts: localhost
vars:
containers:
- {name: a, version: 1}
- {name: b, version: 2}
tasks:
- template:
src: containers.txt.j2
dest: containers.txt
and the template
shell> cat containers.txt.j2
{% for item in containers %}
name: {{ item.name }} version {{ item.version }}
{% endfor %}
give
shell> cat containers.txt
name: a version 1
name: b version 2
It's possible to iterate the list
- debug:
msg: "image: registry.example.com/container-{{ item.name }}:container_{{ item.version }}_version"
loop: "{{ containers }}"
gives
msg: 'image: registry.example.com/container-a:container_1_version'
msg: 'image: registry.example.com/container-b:container_2_version'

Since you want two different templates, each with it's respective container name and version without looping the containers variable. You can use a playbook that renders two templates. In my example I will call the templates:
container-a
container-b
The template container-a.j2:
image: registry.example.com/container-{{ containers[0]['name'] }}:{{ containers[0]['version'] }}
The template container-b.j2:
image: registry.example.com/container-{{ containers[1]['name'] }}:{{ containers[1]['version'] }}
Then my playbook will have below tasks:
vars:
containers:
- { name: a, version: 1 }
- { name: b, version: 2 }
tasks:
- template:
src: "container-{{ containers[0]['name'] }}.j2"
dest: "/tmp/container-{{ containers[0]['name'] }}"
- template:
src: "container-{{ containers[1]['name'] }}.j2"
dest: "/tmp/container-{{ containers[1]['name'] }}"
When this playbook is run, it renders /tmp/container-a:
image: registry.example.com/container-a:1
and /tmp/container-b:
image: registry.example.com/container-b:2

Related

There's a way to template a variable in ansible

applications:
appA:
someDB:
enable: true
datadir: '/var/lib/postgresql/11/appA'
appB:
someDB:
enable: true
datadir: '/var/lib/postgresql/11/appB'
- ansible.builtin.set_fact:
clusters: "{{ clusters + [ item ] }}"
when: applications.{{ item }}.someDB.enable
loop:
- appA
- appB
- ansible.builtin.shell: |
pg_createcluster \
-d {{ aplications.item.someDB.datadir }}
when: item == 'appA'
loop: "{{ clusters }}"
Is there a simple way to make ansible do substitutions inside a var ? Like some precedence operand.
In the loop: item == appA
so, {{ aplications.item.someDB.datadir }} is like this {{ aplications.appA.someDB.datadir }}, and autmatocally is its content is: '/var/lib/postgresql/11/appA' and so on.
Probably I'm using the wrong approach, but seems reasonable to me do that.
Thank's for anyu help.
The applications variable can be flattened, so it will be easier to set the conditions for the task
applications:
- name: appA
someDB:
enable: true
datadir: '/var/lib/postgresql/11/appA'
- name: appB
someDB:
enable: true
datadir: '/var/lib/postgresql/11/appB'
- name: Create PG cluster
ansible.builtin.shell: >
pg_createcluster
-d {{ item.someDB.datadir }}
when:
- item.name == 'appA'
- item.someDB.enable
loop: "{{ applications }}"
Some comments:
Adding a unique name to the task will make it easier to debug your playbooks, specially when you have a long playbook or role
using > in shell will take care to collapse the instructions as a single line, so you won't need the slash
Usually to create a postgress database, you would prefer to use the postgresql_db Ansible module; the -d <directory> is preferred to be set in the configuration file, so you wouldn't need it as an argument.
Create the list of enabled clusters
clusters: "{{ applications|dict2items|
json_query('[?value.someDB.enable].key') }}"
gives
clusters:
- appA
- appB
Then, the task below creates the run-strings you want
- debug:
msg: "pg_createcluster -d {{ applications[item].someDB.datadir }}"
loop: "{{ clusters }}"
gives (abridged)
msg: pg_createcluster -d /var/lib/postgresql/11/appA
msg: pg_createcluster -d /var/lib/postgresql/11/appB
Example of a complete playbook for testing
- hosts: localhost
vars:
applications:
appA:
someDB:
enable: true
datadir: '/var/lib/postgresql/11/appA'
appB:
someDB:
enable: true
datadir: '/var/lib/postgresql/11/appB'
clusters: "{{ applications|dict2items|
json_query('[?value.someDB.enable].key') }}"
tasks:
- debug:
var: clusters
- debug:
msg: "pg_createcluster -d {{ applications[item].someDB.datadir }}"
loop: "{{ clusters }}"
Notes:
You can get both the key and the value by substitution
- debug:
msg: "{{ _key }}: {{ _val }}"
loop:
- appA
- appB
when: applications[item].someDB.enable
vars:
_key: "applications.{{ item }}.someDB.datadir"
_val: "{{ applications[item].someDB.datadir }}"
gives (abridged)
msg: 'applications.appA.someDB.datadir: /var/lib/postgresql/11/appA'
msg: 'applications.appB.someDB.datadir: /var/lib/postgresql/11/appB'
Create a dictionary to simplify the searching
app_datadir: "{{ dict(applications|dict2items|
json_query('[].[key, value.someDB.datadir]')) }}"
gives
app_datadir:
appA: /var/lib/postgresql/11/appA
appB: /var/lib/postgresql/11/appB
Use this dictionary "to make Ansible do substitutions". For example,
- debug:
msg: "applications.{{ item }}.someDB.datadir: {{ app_datadir[item] }}"
loop:
- appA
- appB
gives (abridged)
msg: 'applications.appA.someDB.datadir: /var/lib/postgresql/11/appA'
msg: 'applications.appB.someDB.datadir: /var/lib/postgresql/11/appB'
Example of a complete playbook for testing
- hosts: localhost
vars:
applications:
appA:
someDB:
enable: true
datadir: '/var/lib/postgresql/11/appA'
appB:
someDB:
enable: true
datadir: '/var/lib/postgresql/11/appB'
app_datadir: "{{ dict(applications|dict2items|
json_query('[].[key, value.someDB.datadir]')) }}"
tasks:
- debug:
var: app_datadir
- debug:
msg: "applications.{{ item }}.someDB.datadir: {{ app_datadir[item] }}"
loop:
- appA
- appB
You can also create the dictionary app_enable
app_enable: "{{ dict(applications|dict2items|
json_query('[].[key, value.someDB.enable]')) }}"
gives
app_enable:
appA: true
appB: true
and use it in the condition. For example,
- debug:
msg: "applications.{{ item }}.someDB.datadir: {{ app_datadir[item] }}"
loop:
- appA
- appB
when: app_enable[item]

Access yaml list of dictionaries file with ansible

So I am trying take values from file, let's call it "test.yaml"
file looks like this (sorry for long output, but it is the shortest cut containing all patterns and structure):
---
results:
- failed: false
item: XXX.XX.XX.XX
invocation:
module_args:
validate_certs: false
vm_type: vm
show_tag: false
username: DOMAIN\domain-user
proxy_host:
proxy_port:
show_attribute: false
password: VALUE_SPECIFIED_IN_NO_LOG_PARAMETER
port: XXX
folder:
hostname: XXX.XX.XX.XX
changed: false
virtual_machines:
- ip_address: XXX.XX.XX.XX
mac_address:
- XX:XX:XX:aa:XX:XX
uuid: XXXX-XX-XX-XXXX-XXXXX
guest_fullname: Red Hat Enterprise Linux X (XX-bit)
moid: vm-XXX
folder: "/DOMAIN-INTERXION/vm"
cluster:
attributes: {}
power_state: poweredOn
esxi_hostname: esx.hostname
tags: []
guest_name: VMnameXX
vm_network:
XX:XX:XX:aa:XX:XX:
ipv6:
- XX::XXX:XX:XXXX
ipv4:
- XXX.XX.XX.XX
I would like, for example to have something like:
results.invocation.virtual_machines.ip_address
results.invocation.module_args.user_name
I tried all kind of stuff but it doesn't work :)
last attempt is this:
---
- name: demo how register works
hosts: localhost
tasks:
- name: Include all .json and .jsn files in vars/all and all nested directories (2.3)
include_vars:
file: test.yml
name: vm
- name: debug
debug:
msg: "{{ item.0.item }}"
with_subelements:
- "{{ vm.results }}"
- virtual_machines
register: subelement
following your structure and after fixing some errors:
results.invocation.virtual_machines.ip_address is results[0].virtual_machines[0].ip_address
and
results.invocation.module_args.user_name is results[0].invocation.module_args.username
(results and virtual_machines are arrays, write results[0] or results.0 is same)
so a sample of playbook doing job:
- name: vartest
hosts: localhost
tasks:
- name: Include all .json and .jsn files in vars/all and all nested directories (2.3)
include_vars:
file: test.yml
name: vm
- name: ip
set_fact:
ipadress: "{{ vm.results[0].virtual_machines[0].ip_address }}"
- name: username
set_fact:
username: "{{ vm.results[0].invocation.module_args.username }}"
- name: display
debug:
msg: "ip: {{ ipadress }} and username: {{ username }}"
result:
ok: [localhost] =>
msg: 'ip: XXX.XX.XX.XX and username: DOMAIN\domain-user'

Is there a way to deploy a file using Ansible copy and replace values from parameter store?

I would like to know if I can use Ansible template/copy and replace multiple values in a file(Ex: .properties,.xml, etc..) while deploying the file to destination with values from Parameter store?
Ex-file: app.properties
app.timeout.value=APP-TIMEOUT-VALUE
app.zone.value=APP-ZONE-VALUE
Ex-playbook: app.yaml
- name: Deploy app properties
template:
src: path/app_name.properties
dest: /../app_name.properties
notify:
- restart app
In the above example I want to replace values like(APP-TIMEOUT-VALUE, APP-ZONE-VALUE, etc..) with actual values stored in Parameter Store with Keys as(APP-TIMEOUT-VALUE, APP-ZONE-VALUE, etc..).
Someone please suggest me if there is a straight forward way for it without additional scripts.
Many Thanks
Given the file
shell> cat app.properties
app.timeout.value=APP_TIMEOUT_VALUE
app.zone.value=APP_ZONE_VALUE
Create the template
shell> cat app.properties.j2
app.timeout.value={{ APP_TIMEOUT_VALUE }}
app.zone.value={{ APP_ZONE_VALUE }}
If you can't create the template manually the block below would do it for you
- block:
- copy:
force: false
src: app.properties
dest: app.properties.j2
- lineinfile:
backrefs: true
path: app.properties.j2
regex: '^(.*)=\s*{{ item }}\s*$'
line: '\1={{ eval_open }}{{ item }}{{ eval_close }}'
loop:
- APP_TIMEOUT_VALUE
- APP_ZONE_VALUE
delegate_to: localhost
run_once: true
vars:
eval_open: "{{ '{{ ' }}"
eval_close: "{{ ' }}' }}"
Customize the paths and loop to your needs
Then, use the template. For example
- template:
src: app.properties.j2
dest: app_name.properties
vars:
APP_TIMEOUT_VALUE: 10
APP_ZONE_VALUE: 1
gives
shell> cat app_name.properties
app.timeout.value=10
app.zone.value=1
If you can't rename the names of the variables put them into a dictionary. There is no restriction for a key of a dictionary except for being unique. For example
shell> cat app.properties.j2
app.timeout.value={{ app_conf["APP-TIMEOUT-VALUE"] }}
app.zone.value={{ app_conf["APP-ZONE-VALUE"] }}
If you can't create the template manually the block below would do it for you
- block:
- copy:
force: false
src: app.properties
dest: app.properties.j2
- lineinfile:
backrefs: true
path: app.properties.j2
regex: '^(.*)=\s*{{ item }}\s*$'
line: '\1={{ eval_open }}app_conf["{{ item }}"]{{ eval_close }}'
loop:
- APP-TIMEOUT-VALUE
- APP-ZONE-VALUE
delegate_to: localhost
run_once: true
vars:
eval_open: "{{ '{{ ' }}"
eval_close: "{{ ' }}' }}"
Then, the template below will give the same result
- template:
src: app.properties.j2
dest: app_name.properties
vars:
app_conf:
APP-TIMEOUT-VALUE: 10
APP-ZONE-VALUE: 1

ansible list files in a directory

Can somone explain to me why this doesn't work? I want to get a list of files within a directory and use it as an input for the loop.
---
tasks:
- set_fact:
capabilities: []
- name: find CE_Base capabilities
find:
paths: /opt/netsec/ansible/orchestration/capabilities/CE_BASE
patterns: '*.yml'
register: CE_BASE_capabilities
- name: debug_files
debug:
msg: "{{ item.path }}"
with_items: "{{ CE_BASE_capabilities.files }}"
- set_fact:
thispath: "{{ item.path }}"
capabilities: "{{ capabilities + [ thispath ] }}"
with_items: "{{ CE_BASE_capabilities.files }}"
- name: Include CE_BASE
include_tasks: /opt/netsec/ansible/orchestration/process_capabilities_CE_BASE.yml
loop: "{{ capabilities }}"
Edit:
This code is attempting to create a list called capabilties, which contatins a list of files in a particular directory.
When i ran this code without trying to get the files automatically, it looked like this.
- hosts: localhost
vars:
CE_BASE_capabilities:
- '/opt/netsec/ansible/orchestration/capabilities/CE_BASE/CE_BASE_1.yml'
- '/opt/netsec/ansible/orchestration/capabilities/CE_BASE/CE_BASE_2.yml'
- name: Include CE_BASE
include_tasks: /opt/netsec/ansible/orchestration/process_capabilities_CE_BASE.yml
loop: "{{ CE_BASE_capabilities }}"
Don't define thispath as a fact but as a local vars in the set_fact task. Beside that, you don't need to init capabilities if you use the default filter.
- vars:
thispath: "{{ item.path }}"
set_fact:
capabilities: "{{ capabilities | default([]) + [ thispath ] }}"
with_items: "{{ CE_BASE_capabilities.files }}"
Moreover, you don't even need to loop. You can extract the info directly from the existing result:
- set_fact:
capabilities: "{{ CE_BASE_capabilities.files | map(attribute='path') | list }}"

set_facts with dict as argument of a loop

I'd like to obtain the list of bridged interfaces grouped by master like this:
brv100:
- vnet0
- eth0
brv101:
- vnet1
- eth1
I want to use native json output from the shell commands.
The only thing I managed to do is to get a predefined number of interfaces like this:
- hosts: localhost
gather_facts: no
tasks:
- shell:
cmd: ip -details -pretty -json link show type bridge
register: list_bridges
- set_fact:
bridges: "{{ list_bridges.stdout }}"
- debug:
msg: "{{ bridges | map(attribute='ifname') | list}}"
- name: get json
shell:
cmd: ip -details -pretty -json link show master "{{ifname}}"
with_items: "{{bridges | map(attribute='ifname') | list}}"
loop_control:
loop_var: ifname
register: list_interfaces
- set_fact:
interfaces: "{{ list_interfaces.results | map(attribute='stdout') | list }}"
- set_fact:
toto: "{{interfaces.1}} + {{interfaces.2}}"
- debug:
msg: "{{toto | map(attribute='ifname')|list}}"
Now if I want to do the same with a loop :
- set_fact:
toto: " {{item|default([])}}+ {{ item |default([])}}.{{idx}} "
loop: "{{interfaces}}"
loop_control:
label: "{{item}}"
index_var: idx
- debug: var=toto
The result doesn't seem to be a list of list, but a list of strings and I can't extract the 'ifname' values with a simple debug
- debug:
msg: "{{toto | map(attribute='ifname')|list}}"
What am I supposed to do so as to get benefit of the json native output and get simple list of bridged interfaces (like brctl show was used to do)?
The lists of bridged interfaces grouped by the master are available in ansible_facts. The task below sets the dictionary of the bridges and bridged interfaces
- set_fact:
bridges: "{{ dict(ansible_facts|
dict2items|
json_query('[?value.type == `bridge`].[key, value.interfaces]')) }}"
Q: "Manage to get the same result manipulating JSON data."
A: The output of the ip -json ... command is JSON formated string which must be converted to JSON dictionary in Ansible by the from_yaml filter (JSON is a subset of YAML). For example, the tasks below give the same result
vars:
my_debug: false
tasks:
- name: Get bridges names
command: "ip -details -json link show type bridge"
register: list_bridges
- set_fact:
bridges: "{{ list_bridges.stdout|
from_yaml|
map(attribute='ifname')|
list }}"
- debug:
var: bridges
when: my_debug
- name: Get bridges interfaces
command: "ip -details -json link show master {{ item }}"
loop: "{{ bridges }}"
register: list_interfaces
- set_fact:
bridges_interfaces: "{{ list_interfaces.results|
json_query('[].stdout')|
map('from_yaml')|
list }}"
- debug:
msg: "{{ msg.split('\n') }}"
vars:
msg: "{{ item|to_nice_yaml }}"
loop: "{{ bridges_interfaces }}"
loop_control:
label: "{{ item|json_query('[].ifname') }}"
when: my_debug
- name: Set dictionary of bridges
set_fact:
bridges_dict: "{{ bridges_dict|
default({})|
combine({item.0: item.1|json_query('[].ifname')}) }}"
loop: "{{ bridges|zip(bridges_interfaces)|list }}"
loop_control:
label: "{{ item.1|json_query('[].ifname') }}"
- debug:
var: bridges_dict
Template to write the bridges to a file
{% for k,v in bridges_dict.items() %}
{{ k }}:
{% if v is iterable %}
{% for i in v %}
- {{ i }}
{% endfor %}
{% endif %}
{% endfor %}
- name: Write the bridges to file
template:
src: bridges.txt.j2
dest: bridges.txt
The file bridges.txt will be created in the remote host running the task.