Ansible merge list with dict - list

I have below playbook in which I search for all vars starting with static_routes__ and then merge them.
---
- hosts: localhost
gather_facts: no
vars:
static_routes__host:
management:
- address: '0.0.0.0/0'
next_hop: '192.168.0.1'
static_routes__lab:
management:
- address: '1.1.1.1/32'
next_hop: '192.168.0.1'
static_routes__test:
test:
- address: '8.8.8.8/32'
next_hop: '192.168.2.1'
tasks:
- set_fact:
static_routes: "{{ static_routes | default({}) | combine(lookup('vars', item, default={}), recursive=True, list_merge='append') }}"
loop: "{{ query('varnames', 'static_routes__') }}"
- name: Output static_routes
debug:
var: static_routes
The above will result in:
TASK [Output static_routes] ***************************************************************************************************************************************************************
ok: [localhost] => {
"static_routes": {
"management": [
{
"address": "0.0.0.0/0",
"next_hop": "192.168.0.1"
},
{
"address": "1.1.1.1/32",
"next_hop": "192.168.0.1"
}
],
"test": [
{
"address": "8.8.8.8/32",
"next_hop": "192.168.2.1"
}
]
}
}
However the merge_list_ is only available in Ansible version > 2.9, which is currently not available to me due to company policies. I'm looking for a way to replicate above output in Ansible version =< 2.9.
With the below task I'm able to sort of reproduce it but it only allows one list item.
- set_fact:
static_routes: "{{ static_routes | default({}) | combine({vrf: route | default([]) }) }}"
loop: "{{ query('varnames', 'static_routes__') }}"
vars:
vrf: "{{ lookup('dict', lookup('vars', item)).key }}"
route: "{{ lookup('dict', lookup('vars', item)).value }}"
subnet: "{{ lookup('dict', lookup('vars', item)).value.0.address }}"
next_hop: "{{ lookup('dict', lookup('vars', item)).value.0.next_hop }}"
- name: Output static_routes
debug:
var: static_routes

Found the solution:
- set_fact:
static_routes_list: "{{ static_routes_list | default({}) | combine({item: lookup('vars', item)}) }}"
loop: "{{ query('varnames', 'static_routes__') }}"
- set_fact:
static_routes: "{{ static_routes|
default({})|
combine({item.0: item.1|json_query('[].value')|flatten | unique})
}}"
loop: "{{ static_routes_list|
dict2items|
json_query('[*].value')|
map('dict2items')|list|flatten|
groupby('key')
}}"

This one seems simpler
- set_fact:
_list: "{{ _list|default([]) + [lookup('vars', item)] }}"
loop: "{{ query('varnames', 'static_routes__') }}"
- set_fact:
static_routes: "{{ static_routes|default({})|
combine({item.0: item.1|json_query('[].value')|flatten}) }}"
loop: "{{ _list|map('dict2items')|map('first')|groupby('key') }}"
gives
static_routes:
management:
- address: 0.0.0.0/0
next_hop: 192.168.0.1
- address: 1.1.1.1/32
next_hop: 192.168.0.1
test:
- address: 8.8.8.8/32
next_hop: 192.168.2.1

Related

Is it possible to loop into two different lists in the same playbook (Ansible)?

I'm writing a Playbook Ansible and I want to loop into two different lists.
I now that I can use with_items to loop in a list, but can I use with_items twice in the same playbook?
Here is what I want to do:
- name: Deploy the network in fabric 1 and fabric 2
tags: [merged]
role_network:
config:
- net_name: "{{ networkName }}"
vrf_name: "{{ vrf }}"
net_id: 30010
net_template: "{{ networkTemplate }}"
net_extension_template: "{{ networkExtensionTemplate }}"
vlan_id: "{{ vlan }}"
gw_ip_subnet: "{{ gw }}"
attach: "{{ item }}"
deploy: false
fabric: "{{ item }}"
state: merged
with_items:
- "{{ attachs }}"
"{{ fabric }}"
register: networks
So for the first call, I want to use the playbook with fabric[0] and attachs[0].
For the second call, I want to use the playbook with fabric[1] and attachs[1].
And so on...
Can someone help me please?
What you are looking to achieve is what was with_together and that is, now, recommanded to achieve with the zip filter.
So: loop: "{{ attachs | zip(fabric) | list }}".
Where the element of the first list (attachs) would be item.0 and the element of the second list (fabric) would be item.1.
- name: Deploy the network in fabric 1 and fabric 2
tags: [merged]
role_network:
config:
- net_name: "{{ networkName }}"
vrf_name: "{{ vrf }}"
net_id: 30010
net_template: "{{ networkTemplate }}"
net_extension_template: "{{ networkExtensionTemplate }}"
vlan_id: "{{ vlan }}"
gw_ip_subnet: "{{ gw }}"
attach: "{{ item.0 }}"
deploy: false
fabric: "{{ item.1 }}"
state: merged
loop: "{{ attachs | zip(fabric) | list }}"
register: networks

ansible list files in a directory

Can somone explain to me why this doesn't work? I want to get a list of files within a directory and use it as an input for the loop.
---
tasks:
- set_fact:
capabilities: []
- name: find CE_Base capabilities
find:
paths: /opt/netsec/ansible/orchestration/capabilities/CE_BASE
patterns: '*.yml'
register: CE_BASE_capabilities
- name: debug_files
debug:
msg: "{{ item.path }}"
with_items: "{{ CE_BASE_capabilities.files }}"
- set_fact:
thispath: "{{ item.path }}"
capabilities: "{{ capabilities + [ thispath ] }}"
with_items: "{{ CE_BASE_capabilities.files }}"
- name: Include CE_BASE
include_tasks: /opt/netsec/ansible/orchestration/process_capabilities_CE_BASE.yml
loop: "{{ capabilities }}"
Edit:
This code is attempting to create a list called capabilties, which contatins a list of files in a particular directory.
When i ran this code without trying to get the files automatically, it looked like this.
- hosts: localhost
vars:
CE_BASE_capabilities:
- '/opt/netsec/ansible/orchestration/capabilities/CE_BASE/CE_BASE_1.yml'
- '/opt/netsec/ansible/orchestration/capabilities/CE_BASE/CE_BASE_2.yml'
- name: Include CE_BASE
include_tasks: /opt/netsec/ansible/orchestration/process_capabilities_CE_BASE.yml
loop: "{{ CE_BASE_capabilities }}"
Don't define thispath as a fact but as a local vars in the set_fact task. Beside that, you don't need to init capabilities if you use the default filter.
- vars:
thispath: "{{ item.path }}"
set_fact:
capabilities: "{{ capabilities | default([]) + [ thispath ] }}"
with_items: "{{ CE_BASE_capabilities.files }}"
Moreover, you don't even need to loop. You can extract the info directly from the existing result:
- set_fact:
capabilities: "{{ CE_BASE_capabilities.files | map(attribute='path') | list }}"

Add one element to a list of dictionary ansible / jinja

I'm trying to add a element to a list of dictionary.
_gitlab_runner_config:
server:
url: "https://gitlab.mydomain.com"
api_token: "XXXXXXXXXXXXXXXXXX"
registration_token: "YYYYYYYYYYYYYYYY"
global:
listen_address: ":9200"
concurent: 5
check_interval: 15
session_server:
listen_address: "0.0.0.0:8093"
advertise_address: "{{ ansible_fqdn }}:8093"
session_timeout: 600
runners:
- description: "Test runner 1"
token: ""
tags:
- test1
locked: False
active: False
run_untagged: False
access_level: "not_protected"
maximum_timeout: "3600"
executor: "docker"
executor_config:
tls_verify: false
image: "test-image"
pull_policy: "always"
volumes:
cpus:
In another task, i register the token value. Si, i want to set the value of gitlab_runner_config.runners.LIST_INDEX.token
I have try:
- name: "Save runner token"
set_fact:
_gitlab_runner_config: "{{ _gitlab_runner_config|combine({'runners': {runner_index: {'token': _gitlab_server_registered.runner.token}}} ) }}"
but it override the list.
_gitlab_runner_config.runners is a list. This implicates there might be more items on the list. If all items in the list shall be updated with the same token, e.g. mytoken, the play below does the job
vars:
mytoken: token000
tasks:
- set_fact:
config_updated: "{{ {'runners': _gitlab_runner_config.runners|
map('combine', {'token': mytoken})|
list} }}"
- set_fact:
_gitlab_runner_config: "{{ _gitlab_runner_config|
combine(config_updated) }}"
If there might be different tokens for each item of the list the list shall be updated in a loop. For example, given the list of tokens mytokens, the play below
vars:
mytokens:
- {'token': 'token000'}
- {'token': 'token001'}
- {'token': 'token002'}
tasks:
- set_fact:
runners: "{{ runners|default([]) +
[item|combine(mytokens[ansible_loop.index0])] }}"
loop: "{{ _gitlab_runner_config.runners }}"
loop_control:
extended: yes
- set_fact:
config_updated: "{{ {'runners': runners} }}"
- set_fact:
_gitlab_runner_config: "{{ _gitlab_runner_config|
combine(config_updated) }}"
- debug:
var: _gitlab_runner_config
gives
"_gitlab_runner_config": {
"global": {
"check_interval": 15,
"concurent": 5,
"listen_address": ":9200"
},
"runners": [
{
"access_level": "not_protected",
"active": false,
"description": "Test runner 1",
"executor": "docker",
"executor_config": {
"cpus": "",
"image": "test-image",
"pull_policy": "always",
"tls_verify": false,
"volumes": ""
},
"locked": false,
"maximum_timeout": "3600",
"run_untagged": false,
"tags": [
"test1"
],
"token": "token000"
}
],
"server": {
"api_token": "XXXXXXXXXXXXXXXXXX",
"registration_token": "YYYYYYYYYYYYYYYY",
"url": "https://gitlab.mydomain.com"
},
"session_server": {
"advertise_address": "srv.example.com:8093",
"listen_address": "0.0.0.0:8093",
"session_timeout": 600
}
}
Thanks for your help and answer #vladimir-botka.
But my problem is more complex but, my fault, i didn't give all the detail in my last post.
I have a dict gitlab_runner_config which contain list of runners _gitlab_runner_config.runners. I already loop in this list to register each runner, and i get a token in response (each runner will have a different token). I want to insert this token into teh field token. The whole dict gitlab_runner_config will be used for templating a config file.
The dict:
_gitlab_runner_config:
server:
url: "https://gitlab.mydomain.com"
api_token: "XXXXXXXXXXXXXXXXXX"
registration_token: "YYYYYYYYYYYYYYYY"
global:
listen_address: ":9200"
concurent: 5
check_interval: 15
session_server:
listen_address: "0.0.0.0:8093"
advertise_address: "{{ ansible_fqdn }}:8093"
session_timeout: 600
runners:
- description: "Test runner 1"
token: ""
tags:
- test1
locked: False
active: False
run_untagged: False
access_level: "not_protected"
maximum_timeout: "3600"
executor: "docker"
executor_config:
tls_verify: false
image: "test-image"
pull_policy: "always"
volumes:
cpus:
- description: "Test runner 2"
token: ""
tags:
- test2
locked: False
active: False
run_untagged: False
access_level: "not_protected"
maximum_timeout: "3600"
executor: "docker"
executor_config:
tls_verify: false
image: "test-image"
pull_policy: "always"
volumes:
cpus:
The tasks which register each runenr with a loop:
- name: "Registered runners"
include_tasks: register.yml
loop: "{{ _gitlab_runner_config.runners }}"
loop_control:
index_var: runner_index
register.yml:
- name: "Register runner on gitlab server"
gitlab_runner:
api_url: "{{ _gitlab_runner_config.server.url }}"
api_token: "{{ _gitlab_runner_config.server.api_token }}"
registration_token: "{{ _gitlab_runner_config.server.registration_token }}"
description: "[{{ ansible_fqdn }}] {{ item.description }}"
state: "present"
active: " {{ item.active }}"
tag_list: "{{ item.tags }}"
run_untagged: "{{ item.run_untagged }}"
maximum_timeout: "{{ item.maximum_timeout }}"
access_level: "{{ item.access_level }}"
locked: "{{ item.locked }}"
validate_certs: "no"
register: _gitlab_server_registered
- name: Debug
debug:
msg: "Token to merge for runner id: {{ runner_index }} : {{ gitlab_server_registered.runner.token }}"

set_facts with dict as argument of a loop

I'd like to obtain the list of bridged interfaces grouped by master like this:
brv100:
- vnet0
- eth0
brv101:
- vnet1
- eth1
I want to use native json output from the shell commands.
The only thing I managed to do is to get a predefined number of interfaces like this:
- hosts: localhost
gather_facts: no
tasks:
- shell:
cmd: ip -details -pretty -json link show type bridge
register: list_bridges
- set_fact:
bridges: "{{ list_bridges.stdout }}"
- debug:
msg: "{{ bridges | map(attribute='ifname') | list}}"
- name: get json
shell:
cmd: ip -details -pretty -json link show master "{{ifname}}"
with_items: "{{bridges | map(attribute='ifname') | list}}"
loop_control:
loop_var: ifname
register: list_interfaces
- set_fact:
interfaces: "{{ list_interfaces.results | map(attribute='stdout') | list }}"
- set_fact:
toto: "{{interfaces.1}} + {{interfaces.2}}"
- debug:
msg: "{{toto | map(attribute='ifname')|list}}"
Now if I want to do the same with a loop :
- set_fact:
toto: " {{item|default([])}}+ {{ item |default([])}}.{{idx}} "
loop: "{{interfaces}}"
loop_control:
label: "{{item}}"
index_var: idx
- debug: var=toto
The result doesn't seem to be a list of list, but a list of strings and I can't extract the 'ifname' values with a simple debug
- debug:
msg: "{{toto | map(attribute='ifname')|list}}"
What am I supposed to do so as to get benefit of the json native output and get simple list of bridged interfaces (like brctl show was used to do)?
The lists of bridged interfaces grouped by the master are available in ansible_facts. The task below sets the dictionary of the bridges and bridged interfaces
- set_fact:
bridges: "{{ dict(ansible_facts|
dict2items|
json_query('[?value.type == `bridge`].[key, value.interfaces]')) }}"
Q: "Manage to get the same result manipulating JSON data."
A: The output of the ip -json ... command is JSON formated string which must be converted to JSON dictionary in Ansible by the from_yaml filter (JSON is a subset of YAML). For example, the tasks below give the same result
vars:
my_debug: false
tasks:
- name: Get bridges names
command: "ip -details -json link show type bridge"
register: list_bridges
- set_fact:
bridges: "{{ list_bridges.stdout|
from_yaml|
map(attribute='ifname')|
list }}"
- debug:
var: bridges
when: my_debug
- name: Get bridges interfaces
command: "ip -details -json link show master {{ item }}"
loop: "{{ bridges }}"
register: list_interfaces
- set_fact:
bridges_interfaces: "{{ list_interfaces.results|
json_query('[].stdout')|
map('from_yaml')|
list }}"
- debug:
msg: "{{ msg.split('\n') }}"
vars:
msg: "{{ item|to_nice_yaml }}"
loop: "{{ bridges_interfaces }}"
loop_control:
label: "{{ item|json_query('[].ifname') }}"
when: my_debug
- name: Set dictionary of bridges
set_fact:
bridges_dict: "{{ bridges_dict|
default({})|
combine({item.0: item.1|json_query('[].ifname')}) }}"
loop: "{{ bridges|zip(bridges_interfaces)|list }}"
loop_control:
label: "{{ item.1|json_query('[].ifname') }}"
- debug:
var: bridges_dict
Template to write the bridges to a file
{% for k,v in bridges_dict.items() %}
{{ k }}:
{% if v is iterable %}
{% for i in v %}
- {{ i }}
{% endfor %}
{% endif %}
{% endfor %}
- name: Write the bridges to file
template:
src: bridges.txt.j2
dest: bridges.txt
The file bridges.txt will be created in the remote host running the task.

Ansible - force a variable/fact to be undefined

I'm trying to run a playbook multiple times in a loop which creates AWS route53 records.
My task to create the route53 record looks like this:
- name: Create Public DNS record
route53:
profile: "{{ route53_profile_id }}"
command: "{{ dns_command }}"
zone: "{{ dns_zone }}"
record: "{{ dns_record_name }}.{{ dns_zone }}"
type: "{{ dns_type }}"
value: "{{ dns_value }}"
overwrite: "{{ dns_overwrite }}"
ttl: "{{ dns_ttl }}"
health_check: "{{ healthcheck.health_check.id | default(omit) }}"
failover: "{{ dns_setting.failover | default(omit) }}"
weight: "{{ dns_setting.weight | default(omit) }}"
region: "{{ region | default(omit) }}"
identifier: "{{ identifier | default(omit) }}"
My problem is that the health check isn't always defined every time.
Creation of the health check looks like this:
- name: Create healthcheck with IP address for EC2 instance
route53_health_check:
state: "{{ healthcheck.state | default( healthcheck_defaults.state ) }}"
profile: "{{ route53_profile_id }}"
region: "{{ vpc.region }}"
ip_address: "{{ dns_value }}"
type: "{{ healthcheck.type | default( healthcheck_defaults.type ) }}"
resource_path: "{{ healthcheck.resource_path | default( omit ) }}"
port: "{{ healthcheck.port | default( omit ) }}"
security_token: "{{ healthcheck.security_token | default( omit ) }}"
validate_certs: "{{ healthcheck.validate_certs | default( omit ) }}"
string_match: "{{ healthcheck.string_match | default( omit ) }}"
request_interval: "{{ healthcheck.request_interval | default( healthcheck_defaults.request_interval ) }}"
failure_threshold: "{{ healthcheck.failure_threshold | default( healthcheck_defaults.failure_threshold ) }}"
register: healthcheck
when:
- dns_type == "A"
- dns_setting.healthcheck is defined
If the loop runs 5 times, it may only be defined in one iteration. If the health check runs then the 'healthcheck' variable contains the details of the health check, e.g. the ID. If it does not run on a given loop, the 'healthcheck' variable contains the following:
"healthcheck": {
"changed": false,
"skip_reason": "Conditional check failed",
"skipped": true
}
In my route53 creation, the health check is omitted if the 'healthcheck' variable is undefined. However if it is defined, ansible attempts to dereference the id parameter of the health_check parameter of healthcheck, which doesn't exist.
If I try to set health check to a default value when not in use, e.g. {} then it is still defined, and my route53 creation fails.
Is there a way to force a variable or fact to be undefined? Something like:
- name: Undefine health check
set_fact:
healthcheck: undef
Try something like this:
- name: Create Public DNS record
route53:
... cut ...
health_check: "{{ healthcheck | skipped | ternary( omit, (healthcheck.health_check | default({})).id ) }}"
... cut ...
This will pass omit if healthcheck was skipped and healthcheck.health_check.id otherwise.
From my experience, default is not working properly with nested dicts of 2+ levels (i.e. works with mydict.myitem | default('ok') but fails with mydict.mysubdict.myitem | default('ok'), so I used the hack to default subdict to {} first to safely access id: (healthcheck.health_check | default({})).id