I'm trying to add list element index as hash, preferably without using with_indexed_items because result is to be filtered later.
I'd like to convert this:
vmware_disks:
- datastore: "datastores0"
size_gb: 24
- datastore: "datastores0"
size_gb: 397
storage_policy: "vSAN standard cluster RAID-1"
to something like this:
vmware_disks:
- datastore: "datastores0"
size_gb: 24
nr: 0
- datastore: "datastores0"
size_gb: 397
storage_policy: "vSAN standard cluster RAID-1"
nr: 1
So far I was able to:
using "{{ vmware_disks | zip(range(0,2)) }}"
[
{
"datastore": "datastores0",
"size_gb": 24
},
0
],
[
{
"datastore": "datastores0",
"size_gb": 397,
"storage_policy": "vSAN standard cluster RAID-1"
},
1
]
"{{ vmware_disks | map('combine', {'nr':0}) }}"
[
{
"datastore": "datastores0",
"nr": 0,
"size_gb": 24
},
{
"datastore": "datastores0",
"nr": 0,
"size_gb": 397,
"storage_policy": "vSAN standard cluster RAID-1"
}
]
Please help me to combine both to get desired result.
You can use a set_fact to create another dictionary and add the index in:
- set_fact:
indexed_vmware_disks: >-
{{
indexed_vmware_disks | default([])
+ [ item | combine( { 'nr': index } ) ]
}}
loop: "{{ vmware_disks }}"
loop_control:
index_var: index
Then if you really need the dictionary to have the exact name it had before, you can just reassign it in another set_fact.
So an example playbook:
- hosts: all
gather_facts: no
vars:
vmware_disks:
- datastore: "datastores0"
size_gb: 24
- datastore: "datastores0"
size_gb: 397
storage_policy: "vSAN standard cluster RAID-1"
tasks:
- set_fact:
indexed_vmware_disks: >-
{{
indexed_vmware_disks | default([])
+ [ item | combine( { 'nr': index } ) ]
}}
loop: "{{ vmware_disks }}"
loop_control:
index_var: index
- set_fact:
vmware_disks: "{{ indexed_vmware_disks }}"
- debug:
var: vmware_disks
Would yield the recap:
PLAY [all] *******************************************************************************************************
TASK [set_fact] **************************************************************************************************
ok: [localhost] => (item={'datastore': 'datastores0', 'size_gb': 24})
ok: [localhost] => (item={'datastore': 'datastores0', 'size_gb': 397, 'storage_policy': 'vSAN standard cluster RAID-1'})
TASK [set_fact] **************************************************************************************************
ok: [localhost]
TASK [debug] *****************************************************************************************************
ok: [localhost] =>
vmware_disks:
- datastore: datastores0
nr: 0
size_gb: 24
- datastore: datastores0
nr: 1
size_gb: 397
storage_policy: vSAN standard cluster RAID-1
PLAY RECAP *******************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Related
I have the below lists stored in variable results_one.msg
[
{
"IP": [
"192.168.1.100",
"192.168.1.101"
],
"Pool": "lan_pool_sftp",
"Members": [
"sftpnode01:5425",
"sftpnode02:5425"
]
},
{
"IP": [
"192.168.1.103",
"192.168.1.104"
],
"Pool": "icmp-net-pool",
"Members": [
"icmpnet01:8443",
"icmpnet02:8443"
]
}
]
I have another variable node_name
I would like to get the Pool and Members information from above output, by querying one of the members name.
For example, if I assign the variable node_name: icmpnet02
I want to get the output stored as in respective variable names as below.
pool_name: icmp-net-pool
pool_members: [ icmpnet01:8443,icmpnet02:8443 ]
I tried as below and I'm unable to get it
- set_fact:
pool_name: "{{ item.Pool }}"
pool_members: "{{ item.Members }}"
with_items: "{{results_one.msg }}"
when: 'item.Members.0 is defined and "node_name:*" in item.Members'
Create a list of lists of members with their names only:
_members_hostnames: "{{ results_one.msg | map(attribute='Members')
| map('map', 'regex_replace', '^(.*):.*$', '\\1') }}"
Gives:
"_members_hostnames": [
[
"sftpnode01",
"sftpnode02"
],
[
"icmpnet01",
"icmpnet02"
]
]
select the matching entry from your relevant variable, i.e.
create a list of tuples associating each original element with its counterpart calculated members hostnames
retain only element where hostname is present in the list
keep only the first element of tupple (i.e. the orginal entry)
keep only the first element from list
_matching_entry: "{{ results_one.msg | zip(_members_hostnames)
| selectattr(1, 'contains', node_name) | map(attribute=0) | first }}"
gives
"_matching_entry": {
"IP": [
"192.168.1.103",
"192.168.1.104"
],
"Members": [
"icmpnet01:8443",
"icmpnet02:8443"
],
"Pool": "icmp-net-pool"
}
use the matching entry to extract whatever variable you need:
pool_name: "{{ _matching_entry.Pool }}"
pool_members: "{{ _matching_entry.Members }}"
Putting it all together in a test playbook:
---
- hosts: localhost
gather_facts: false
vars:
# Your orig data on a single line for legibility
results_one: {"msg":[{"IP":["192.168.1.100","192.168.1.101"],"Pool":"lan_pool_sftp","Members":["sftpnode01:5425","sftpnode02:5425"]},{"IP":["192.168.1.103","192.168.1.104"],"Pool":"icmp-net-pool","Members":["icmpnet01:8443","icmpnet02:8443"]}]}
node_name: icmpnet02
_members_hostnames: "{{ results_one.msg | map(attribute='Members')
| map('map', 'regex_replace', '^(.*):.*$', '\\1') }}"
_matching_entry: "{{ results_one.msg | zip(_members_hostnames)
| selectattr(1, 'contains', node_name) | map(attribute=0) | first }}"
pool_name: "{{ _matching_entry.Pool }}"
pool_members: "{{ _matching_entry.Members }}"
tasks:
- debug:
msg:
- Pool name is {{ pool_name }}
- Pool members are {{ pool_members }}
Which gives:
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [debug] ***************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"Pool name is icmp-net-pool",
"Pool members are ['icmpnet01:8443', 'icmpnet02:8443']"
]
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
I'm unable to filter the following output with json_query, even though it works on a jmespath validator website.
Let me explain in detail with the code.
type_debug returns AnsibleUnicode for members value, so when I tried to use json_query to filter output it returned null. I need to get value from key ip, but I don't know how to do it in a good way.
---
- hosts: localhost
gather_facts: false
vars:
- RPN:
stdout: >-
"members": [
{
"id": 0,
"ip": "x.x.x.x",
"owner": "buzut",
"private_ip": "10.91.154.39",
"speed": 100,
"status": "active"
}
]
tasks:
- debug:
msg: "{{ RPN.stdout | type_debug }}"
From your question I understand that you like to gather all member with a json_query. To do so I've corrected the JSON slightly.
---
- hosts: localhost
become: false
gather_facts: false
vars:
RPN:
stdout: >-
{
"members": [{
"id": 0,
"ip": "x.x.x.x",
"owner": "buzut",
"private_ip": "10.91.154.39",
"speed": 100,
"status": "active"
}]
}
tasks:
- name: Show type(s)
debug:
msg:
- "{{ RPN.stdout | type_debug }}"
- "{{ RPN.stdout | from_json | type_debug }}"
- "{{ RPN.stdout | to_json | type_debug }}"
- name: Show members
debug:
msg: "{{ RPN.stdout | from_json | json_query('members') }}"
resulting into an output of
TASK [Show type(s)] *********
ok: [localhost] =>
msg:
- AnsibleUnicode
- dict
- str
TASK [Show members] *********
ok: [localhost] =>
msg:
- id: 0
ip: x.x.x.x
owner: buzut
private_ip: 10.91.154.39
speed: 100
status: active
I need to get value from key ip
- name: Show value of key 'ip'
debug:
msg: "{{ RPN.stdout | from_json | json_query('members[*].ip') }}"
resulting into an output of
TASK [Show value of key 'ip'] *********
ok: [localhost] =>
msg:
- x.x.x.x
Further Documentation
Selecting JSON data: JSON queries
I have below playbook in which I search for all vars starting with static_routes__ and then merge them.
---
- hosts: localhost
gather_facts: no
vars:
static_routes__host:
management:
- address: '0.0.0.0/0'
next_hop: '192.168.0.1'
static_routes__lab:
management:
- address: '1.1.1.1/32'
next_hop: '192.168.0.1'
static_routes__test:
test:
- address: '8.8.8.8/32'
next_hop: '192.168.2.1'
tasks:
- set_fact:
static_routes: "{{ static_routes | default({}) | combine(lookup('vars', item, default={}), recursive=True, list_merge='append') }}"
loop: "{{ query('varnames', 'static_routes__') }}"
- name: Output static_routes
debug:
var: static_routes
The above will result in:
TASK [Output static_routes] ***************************************************************************************************************************************************************
ok: [localhost] => {
"static_routes": {
"management": [
{
"address": "0.0.0.0/0",
"next_hop": "192.168.0.1"
},
{
"address": "1.1.1.1/32",
"next_hop": "192.168.0.1"
}
],
"test": [
{
"address": "8.8.8.8/32",
"next_hop": "192.168.2.1"
}
]
}
}
However the merge_list_ is only available in Ansible version > 2.9, which is currently not available to me due to company policies. I'm looking for a way to replicate above output in Ansible version =< 2.9.
With the below task I'm able to sort of reproduce it but it only allows one list item.
- set_fact:
static_routes: "{{ static_routes | default({}) | combine({vrf: route | default([]) }) }}"
loop: "{{ query('varnames', 'static_routes__') }}"
vars:
vrf: "{{ lookup('dict', lookup('vars', item)).key }}"
route: "{{ lookup('dict', lookup('vars', item)).value }}"
subnet: "{{ lookup('dict', lookup('vars', item)).value.0.address }}"
next_hop: "{{ lookup('dict', lookup('vars', item)).value.0.next_hop }}"
- name: Output static_routes
debug:
var: static_routes
Found the solution:
- set_fact:
static_routes_list: "{{ static_routes_list | default({}) | combine({item: lookup('vars', item)}) }}"
loop: "{{ query('varnames', 'static_routes__') }}"
- set_fact:
static_routes: "{{ static_routes|
default({})|
combine({item.0: item.1|json_query('[].value')|flatten | unique})
}}"
loop: "{{ static_routes_list|
dict2items|
json_query('[*].value')|
map('dict2items')|list|flatten|
groupby('key')
}}"
This one seems simpler
- set_fact:
_list: "{{ _list|default([]) + [lookup('vars', item)] }}"
loop: "{{ query('varnames', 'static_routes__') }}"
- set_fact:
static_routes: "{{ static_routes|default({})|
combine({item.0: item.1|json_query('[].value')|flatten}) }}"
loop: "{{ _list|map('dict2items')|map('first')|groupby('key') }}"
gives
static_routes:
management:
- address: 0.0.0.0/0
next_hop: 192.168.0.1
- address: 1.1.1.1/32
next_hop: 192.168.0.1
test:
- address: 8.8.8.8/32
next_hop: 192.168.2.1
The below code is attempting to find all the keys associated with the value HWIC-8A. I have tried a few different variations & I can't get the key to print, without doing something really long winded. As i'll be repeating this code with different values, i don't want to search each key/value pair individually within that list.
MODULES:
Slot_0_SubSlot_0: HWIC-8A
Slot_0_SubSlot_1: EHWIC-VA-DSL-M
Slot_0_SubSlot_3: HWIC-8A
- name: Apply HWIC-8A Build
debug:
msg: "{{ item.key }}"
with_items: "{{ MODULES }}"
when: "{{ item.value }} == HWIC-8A"
Maybe that's something for you:
---
- hosts: localhost
vars:
MODULES:
Slot_0_SubSlot_0: HWIC-8A
Slot_0_SubSlot_1: EHWIC-VA-DSL-M
Slot_0_SubSlot_3: HWIC-8A
tasks:
- debug: var=MODULES
- debug: msg="{{ MODULES | dict2items }}"
- debug: msg="{{ MODULES | dict2items | selectattr('value','match','HWIC-8A') | map(attribute='key')| list }}"
Then if you would like to have multiple matches, you could solve it with an MATCH list:
---
- hosts: localhost
vars:
MODULES:
Slot_0_SubSlot_0: HWIC-8A
Slot_0_SubSlot_1: EHWIC-VA-DSL-M
Slot_0_SubSlot_3: HWIC-8A
Slot_1_SubSlot_3: HWIC-8C
Slot_1_SubSlot_2: HWIC-8C
MATCH:
- HWIC-8A
- HWIC-8C
tasks:
- debug:
msg: "{{ MODULES | dict2items | selectattr('value','match',item) | map(attribute='key')| list }}"
with_items: "{{ MATCH }}"
Output:
TASK [debug] ***********************************************************************************************************************************************************************************
Thursday 27 August 2020 15:08:10 +0200 (0:00:00.042) 0:00:02.037 *******
ok: [localhost] => (item=HWIC-8A) => {
"msg": [
"Slot_0_SubSlot_0",
"Slot_0_SubSlot_3"
]
}
ok: [localhost] => (item=HWIC-8C) => {
"msg": [
"Slot_1_SubSlot_3",
"Slot_1_SubSlot_2"
]
}
I would use use jinja templates to do it. Something like this:
- name: Apply HWIC-8A Build
debug:
msg: '{% for m in MODULES %}{% if MODULES[m] == "HWIC-8A" %}{{ m }} {% endif %}{% endfor %}'
Which will give you this:
ok: [localhost] => {
"msg": "Slot_0_SubSlot_0 Slot_0_SubSlot_3 "
}
There is probably a fancy way using filters as well.
I'm trying to add a element to a list of dictionary.
_gitlab_runner_config:
server:
url: "https://gitlab.mydomain.com"
api_token: "XXXXXXXXXXXXXXXXXX"
registration_token: "YYYYYYYYYYYYYYYY"
global:
listen_address: ":9200"
concurent: 5
check_interval: 15
session_server:
listen_address: "0.0.0.0:8093"
advertise_address: "{{ ansible_fqdn }}:8093"
session_timeout: 600
runners:
- description: "Test runner 1"
token: ""
tags:
- test1
locked: False
active: False
run_untagged: False
access_level: "not_protected"
maximum_timeout: "3600"
executor: "docker"
executor_config:
tls_verify: false
image: "test-image"
pull_policy: "always"
volumes:
cpus:
In another task, i register the token value. Si, i want to set the value of gitlab_runner_config.runners.LIST_INDEX.token
I have try:
- name: "Save runner token"
set_fact:
_gitlab_runner_config: "{{ _gitlab_runner_config|combine({'runners': {runner_index: {'token': _gitlab_server_registered.runner.token}}} ) }}"
but it override the list.
_gitlab_runner_config.runners is a list. This implicates there might be more items on the list. If all items in the list shall be updated with the same token, e.g. mytoken, the play below does the job
vars:
mytoken: token000
tasks:
- set_fact:
config_updated: "{{ {'runners': _gitlab_runner_config.runners|
map('combine', {'token': mytoken})|
list} }}"
- set_fact:
_gitlab_runner_config: "{{ _gitlab_runner_config|
combine(config_updated) }}"
If there might be different tokens for each item of the list the list shall be updated in a loop. For example, given the list of tokens mytokens, the play below
vars:
mytokens:
- {'token': 'token000'}
- {'token': 'token001'}
- {'token': 'token002'}
tasks:
- set_fact:
runners: "{{ runners|default([]) +
[item|combine(mytokens[ansible_loop.index0])] }}"
loop: "{{ _gitlab_runner_config.runners }}"
loop_control:
extended: yes
- set_fact:
config_updated: "{{ {'runners': runners} }}"
- set_fact:
_gitlab_runner_config: "{{ _gitlab_runner_config|
combine(config_updated) }}"
- debug:
var: _gitlab_runner_config
gives
"_gitlab_runner_config": {
"global": {
"check_interval": 15,
"concurent": 5,
"listen_address": ":9200"
},
"runners": [
{
"access_level": "not_protected",
"active": false,
"description": "Test runner 1",
"executor": "docker",
"executor_config": {
"cpus": "",
"image": "test-image",
"pull_policy": "always",
"tls_verify": false,
"volumes": ""
},
"locked": false,
"maximum_timeout": "3600",
"run_untagged": false,
"tags": [
"test1"
],
"token": "token000"
}
],
"server": {
"api_token": "XXXXXXXXXXXXXXXXXX",
"registration_token": "YYYYYYYYYYYYYYYY",
"url": "https://gitlab.mydomain.com"
},
"session_server": {
"advertise_address": "srv.example.com:8093",
"listen_address": "0.0.0.0:8093",
"session_timeout": 600
}
}
Thanks for your help and answer #vladimir-botka.
But my problem is more complex but, my fault, i didn't give all the detail in my last post.
I have a dict gitlab_runner_config which contain list of runners _gitlab_runner_config.runners. I already loop in this list to register each runner, and i get a token in response (each runner will have a different token). I want to insert this token into teh field token. The whole dict gitlab_runner_config will be used for templating a config file.
The dict:
_gitlab_runner_config:
server:
url: "https://gitlab.mydomain.com"
api_token: "XXXXXXXXXXXXXXXXXX"
registration_token: "YYYYYYYYYYYYYYYY"
global:
listen_address: ":9200"
concurent: 5
check_interval: 15
session_server:
listen_address: "0.0.0.0:8093"
advertise_address: "{{ ansible_fqdn }}:8093"
session_timeout: 600
runners:
- description: "Test runner 1"
token: ""
tags:
- test1
locked: False
active: False
run_untagged: False
access_level: "not_protected"
maximum_timeout: "3600"
executor: "docker"
executor_config:
tls_verify: false
image: "test-image"
pull_policy: "always"
volumes:
cpus:
- description: "Test runner 2"
token: ""
tags:
- test2
locked: False
active: False
run_untagged: False
access_level: "not_protected"
maximum_timeout: "3600"
executor: "docker"
executor_config:
tls_verify: false
image: "test-image"
pull_policy: "always"
volumes:
cpus:
The tasks which register each runenr with a loop:
- name: "Registered runners"
include_tasks: register.yml
loop: "{{ _gitlab_runner_config.runners }}"
loop_control:
index_var: runner_index
register.yml:
- name: "Register runner on gitlab server"
gitlab_runner:
api_url: "{{ _gitlab_runner_config.server.url }}"
api_token: "{{ _gitlab_runner_config.server.api_token }}"
registration_token: "{{ _gitlab_runner_config.server.registration_token }}"
description: "[{{ ansible_fqdn }}] {{ item.description }}"
state: "present"
active: " {{ item.active }}"
tag_list: "{{ item.tags }}"
run_untagged: "{{ item.run_untagged }}"
maximum_timeout: "{{ item.maximum_timeout }}"
access_level: "{{ item.access_level }}"
locked: "{{ item.locked }}"
validate_certs: "no"
register: _gitlab_server_registered
- name: Debug
debug:
msg: "Token to merge for runner id: {{ runner_index }} : {{ gitlab_server_registered.runner.token }}"