Ansible - remove empty elements from the list - list

I have an ansible output from which I want to remove all empty (none) elements for a specific attribute. Here is the example list:
"resources": [
{
"id": "9c40900909",
"name": "some_name1"
},
{
"id": "pc4b09090"
},
{
"id": "8lknkkn45"
},
{
"id": "9df40900909",
"name": "some_name2"
}
]
Here is how I reduced the list to be just "resources" with attribute "name":
- set_fact:
resources_names: "{{ output.resources | map(attribute='name') }}"
Problem with that is that I get the elements that does not have any value for the name. These are "AnsibleUndefined" in the following output:
- debug:
msg: resources_names list is "{{ resources_names }}"
ok: [localhost] => {
"msg": "resources_names list is \"['some_name1', AnsibleUndefined, AnsibleUndefined, 'some_name2']\""
}
I tried to remove it with reject and regexp but that's not working.
- set_fact:
list2: "{{ resources_names | reject('match', '^$') | list }}"
Same with this one:
- set_fact:
resources_names: "{{ output.resources | map(attribute='name') | rejectattr('name', 'none') }}"
Any idea?
Thanks.

In a nutshell:
---
- hosts: localhost
gather_facts: false
vars:
resources: [
{
"id": "9c40900909",
"name": "some_name1"
},
{
"id": "pc4b09090"
},
{
"id": "8lknkkn45"
},
{
"id": "9df40900909",
"name": "some_name2"
}
]
tasks:
- debug:
msg: "{{ resources | selectattr('name', 'defined') | map(attribute='name') }}"
which gives:
$ ansible-playbook /tmp/play.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
PLAY [localhost] *******************************************************************************************************************************************************************************
TASK [debug] ***********************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"some_name1",
"some_name2"
]
}
PLAY RECAP *************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Note for your next question: please make sure you double check your examples and paste valid json/yaml data in your question. Thanks

Use the filter json_query. This filter ignores missing attributes
resources_names: "{{ output.resources|json_query('[].name') }}"
gives
resources_names:
- some_name1
- some_name2

Related

Ansible filter item in list using variables+wildcard

I have the below lists stored in variable results_one.msg
[
{
"IP": [
"192.168.1.100",
"192.168.1.101"
],
"Pool": "lan_pool_sftp",
"Members": [
"sftpnode01:5425",
"sftpnode02:5425"
]
},
{
"IP": [
"192.168.1.103",
"192.168.1.104"
],
"Pool": "icmp-net-pool",
"Members": [
"icmpnet01:8443",
"icmpnet02:8443"
]
}
]
I have another variable node_name
I would like to get the Pool and Members information from above output, by querying one of the members name.
For example, if I assign the variable node_name: icmpnet02
I want to get the output stored as in respective variable names as below.
pool_name: icmp-net-pool
pool_members: [ icmpnet01:8443,icmpnet02:8443 ]
I tried as below and I'm unable to get it
- set_fact:
pool_name: "{{ item.Pool }}"
pool_members: "{{ item.Members }}"
with_items: "{{results_one.msg }}"
when: 'item.Members.0 is defined and "node_name:*" in item.Members'
Create a list of lists of members with their names only:
_members_hostnames: "{{ results_one.msg | map(attribute='Members')
| map('map', 'regex_replace', '^(.*):.*$', '\\1') }}"
Gives:
"_members_hostnames": [
[
"sftpnode01",
"sftpnode02"
],
[
"icmpnet01",
"icmpnet02"
]
]
select the matching entry from your relevant variable, i.e.
create a list of tuples associating each original element with its counterpart calculated members hostnames
retain only element where hostname is present in the list
keep only the first element of tupple (i.e. the orginal entry)
keep only the first element from list
_matching_entry: "{{ results_one.msg | zip(_members_hostnames)
| selectattr(1, 'contains', node_name) | map(attribute=0) | first }}"
gives
"_matching_entry": {
"IP": [
"192.168.1.103",
"192.168.1.104"
],
"Members": [
"icmpnet01:8443",
"icmpnet02:8443"
],
"Pool": "icmp-net-pool"
}
use the matching entry to extract whatever variable you need:
pool_name: "{{ _matching_entry.Pool }}"
pool_members: "{{ _matching_entry.Members }}"
Putting it all together in a test playbook:
---
- hosts: localhost
gather_facts: false
vars:
# Your orig data on a single line for legibility
results_one: {"msg":[{"IP":["192.168.1.100","192.168.1.101"],"Pool":"lan_pool_sftp","Members":["sftpnode01:5425","sftpnode02:5425"]},{"IP":["192.168.1.103","192.168.1.104"],"Pool":"icmp-net-pool","Members":["icmpnet01:8443","icmpnet02:8443"]}]}
node_name: icmpnet02
_members_hostnames: "{{ results_one.msg | map(attribute='Members')
| map('map', 'regex_replace', '^(.*):.*$', '\\1') }}"
_matching_entry: "{{ results_one.msg | zip(_members_hostnames)
| selectattr(1, 'contains', node_name) | map(attribute=0) | first }}"
pool_name: "{{ _matching_entry.Pool }}"
pool_members: "{{ _matching_entry.Members }}"
tasks:
- debug:
msg:
- Pool name is {{ pool_name }}
- Pool members are {{ pool_members }}
Which gives:
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [debug] ***************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"Pool name is icmp-net-pool",
"Pool members are ['icmpnet01:8443', 'icmpnet02:8443']"
]
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

How to combine/match list with specific key/values as hostvar

I have a question about dicts and lists.
What I want to achieve is, have the key and value from a seperate list saved as fact/hostvar for each matching host.
I'm getting a list from a Confluence API that looks like this (abbreviated):
[
{
"title": "MACHINE1",
"_links": {
"tinyui": "/x/1234"
}
},
{
"title": "MACHINE2",
"_links": {
"tinyui": "/x/5678"
}
},
{
"title": "MACHINE3",
"_links": {
"tinyui": "/x/9876"
}
}
]
What worked to get each individual item (just to debug, and show that the loop itself works) is:
- name: DEBUG specific item in list of get_children.json.results
debug:
msg: "{{ item.title }} {{ item._links.tinyui }}"
loop:
"{{ get_children.json.results }}"
delegate_to: 127.0.0.1
Ansible Output (here: output for only one machine):
"msg": "MACHINE1 /x/1234"
Machine Hostnames:
Yes, they are lowercase in my inventory, and in the above list output they are uppercase. But I guess a simple item.title|lower would do fine.
machine1
machine2
machine3
How can I now match the item.title with ansible_hostname and save above API Output as a fact for each machine?
And for clarification: item.title|lower == ansible_hostname
I hope it gets clear to what I want to achieve and thanks to everyone in advance :)
EDIT: Thanks to both answers I managed to get it to work. Using '(?i)^'+VAR+'$' and some other conditional checks you guys posted definitely helped. :)
In a nutshell, given the inventories/tinyui/main.yml inventory:
---
all:
hosts:
machine1:
machine2:
machine3:
i.do.not.exist:
The folowing tinyui.yml playbook:
---
- hosts: all
gather_facts: false
vars:
# In real life, this is returned by your API call
get_children:
json:
results: [
{
"title": "MACHINE1",
"_links": {
"tinyui": "/x/1234"
}
},
{
"title": "MACHINE2",
"_links": {
"tinyui": "/x/5678"
}
},
{
"title": "MACHINE3",
"_links": {
"tinyui": "/x/9876"
}
}
]
# This won't be defined before you call the API which
# returns and registers the correct result. If there is
# no match for host in the returned json, '!no uri!' will
# be returned below. Adapt with a default uri if needed
tinyui: "{{ get_children.json.results
| selectattr('title', '==', inventory_hostname | upper)
| map(attribute='_links.tinyui')
| default(['!no uri!'], true) | first }}"
tasks:
# In real life, you would have called your API
# and registered the result in `get_children` e.g.
# - name: get info from confluence
# uri:
# url: <confluence api endpoint url>
# <more parameters here>
# run_once: true
# delegate_to: localhost
# register: get_children
- name: Display tinyui for host
debug:
msg: "tinyui for host {{ inventory_hostname }} is {{ tinyui }}"
Gives:
$ ansible-playbook -i inventories/tinyui/ tinyui.yml
PLAY [all] ***********************************************************************************************************************
TASK [Display tinyui for host] ***************************************************************************************************
ok: [machine1] => {
"msg": "tinyui for host machine1 is /x/1234"
}
ok: [machine2] => {
"msg": "tinyui for host machine2 is /x/5678"
}
ok: [machine3] => {
"msg": "tinyui for host machine3 is /x/9876"
}
ok: [i.do.not.exist] => {
"msg": "tinyui for host i.do.not.exist is !no uri!"
}
PLAY RECAP ***********************************************************************************************************************
i.do.not.exist : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
machine1 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
machine2 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
machine3 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
You can filter the list of dictionaries by title, taking out the right dict.
You can do this with the following line:
"{{ hostsdata | selectattr('title', 'match', '(?i)^'+host+'$') | first }}"
With selectattr you filter your list of dicts by the title, where this must match '(?i)^'+host+'$'.
The (?i) is the ignore case inline flag for pattern matching, concatenated with the hostname (case does not matter because of ignore case flag). ^...$ specifies that the whole string must match, from start to end.
Since selectattr returns a list as result, you can use first to take out the first element of the list.
Instead of using (?i) you can also set the ignorecase parameter, which will look like this:
"{{ hostsdata | selectattr('title', 'match', '^'+host+'$', 'ignorecase=true') | first }}"
Both variants work equivalently.
Entire playbook:
---
- hosts: localhost
gather_facts: no
vars:
hostsdata:
- {
"title": "MACHINE1",
"_links": {
"tinyui": "/x/1234"
}
}
- {
"title": "MACHINE2",
"_links": {
"tinyui": "/x/5678"
}
}
- {
"title": "MACHINE3",
"_links": {
"tinyui": "/x/9876"
}
}
tasks:
- debug:
var: hostsdata
- name: Pick out host specific dict
set_fact:
machine_data: "{{ hostsdata | selectattr('title', 'match', '(?i)'+host) | first }}"
vars:
host: machine3
- debug:
var: machine_data
- debug:
msg: "{{ machine_data.title }} {{ machine_data._links.tinyui }}"
Resulting output:
TASK [debug] ***********************************************************************************************************
ok: [localhost] => {
"hostsdata": [
{
"_links": {
"tinyui": "/x/1234"
},
"title": "MACHINE1"
},
{
"_links": {
"tinyui": "/x/5678"
},
"title": "MACHINE2"
},
{
"_links": {
"tinyui": "/x/9876"
},
"title": "MACHINE3"
}
]
}
TASK [Pick out host specific dict] *************************************************************************************
ok: [localhost]
TASK [debug] ***********************************************************************************************************
ok: [localhost] => {
"machine_data": {
"_links": {
"tinyui": "/x/9876"
},
"title": "MACHINE3"
}
}
TASK [debug] ***********************************************************************************************************
ok: [localhost] => {
"msg": "MACHINE3 /x/9876"
}
To filter multiple machines, here is another example:
- debug:
msg: "{{ md.title }} {{ md._links.tinyui }}"
when: md | length
vars:
md: "{{ hostsdata | selectattr('title', 'match', '(?i)^'+item+'$') | first | default('') }}"
with_items:
- MachINE1
- MACHINE2
- machine3
- unknown
Add a default('') and a when: to skip a non-existent hostname.
Output:
TASK [debug] ***********************************************************************************************************
ok: [localhost] => (item=MachINE1) => {
"msg": "MACHINE1 /x/1234"
}
ok: [localhost] => (item=MACHINE2) => {
"msg": "MACHINE2 /x/5678"
}
ok: [localhost] => (item=machine3) => {
"msg": "MACHINE3 /x/9876"
}
skipping: [localhost] => (item=unknown)
Convert the titles to lowercase
titles: "{{ get_children.json.results|
map(attribute='title')|
map('lower')|
map('community.general.dict_kv', 'title')|
list }}"
gives
titles:
- title: machine1
- title: machine2
- title: machine3
Replace the lowercase titles and create a dictionary
title_links: "{{ get_children.json.results|
zip(titles)|
map('combine')|
items2dict(key_name='title', value_name='_links') }}"
gives
title_links:
machine1:
tinyui: /x/1234
machine2:
tinyui: /x/5678
machine3:
tinyui: /x/9876
Put these declarations, for example, into the group_vars
shell> cat group_vars/all/title_links.yml
titles: "{{ get_children.json.results|
map(attribute='title')|
map('lower')|
map('community.general.dict_kv', 'title')|
list }}"
title_links: "{{ get_children.json.results|
zip(titles)|
map('combine')|
items2dict(key_name='title', value_name='_links') }}"
Now, you can use the dictionary. For example, given the inventory
shell> cat hosts
10.1.0.11 ansible_hostname=machine1
10.1.0.12 ansible_hostname=machine2
10.1.0.13 ansible_hostname=machine3
the playbook
- hosts: all
gather_facts: false
vars:
get_children:
json:
results:
- _links: {tinyui: /x/1234}
title: MACHINE1
- _links: {tinyui: /x/5678}
title: MACHINE2
- _links: {tinyui: /x/9876}
title: MACHINE3
tasks:
- debug:
msg: "My links: {{ title_links[ansible_hostname] }}"
gives (abridged)
TASK [debug] *********************************************************
ok: [10.1.0.11] =>
msg: 'My links: {''tinyui'': ''/x/1234''}'
ok: [10.1.0.12] =>
msg: 'My links: {''tinyui'': ''/x/5678''}'
ok: [10.1.0.13] =>
msg: 'My links: {''tinyui'': ''/x/9876''}'

Ansible returns 'AnibleUnicode' instead of JSON, unable to filter

I'm unable to filter the following output with json_query, even though it works on a jmespath validator website.
Let me explain in detail with the code.
type_debug returns AnsibleUnicode for members value, so when I tried to use json_query to filter output it returned null. I need to get value from key ip, but I don't know how to do it in a good way.
---
- hosts: localhost
gather_facts: false
vars:
- RPN:
stdout: >-
"members": [
{
"id": 0,
"ip": "x.x.x.x",
"owner": "buzut",
"private_ip": "10.91.154.39",
"speed": 100,
"status": "active"
}
]
tasks:
- debug:
msg: "{{ RPN.stdout | type_debug }}"
From your question I understand that you like to gather all member with a json_query. To do so I've corrected the JSON slightly.
---
- hosts: localhost
become: false
gather_facts: false
vars:
RPN:
stdout: >-
{
"members": [{
"id": 0,
"ip": "x.x.x.x",
"owner": "buzut",
"private_ip": "10.91.154.39",
"speed": 100,
"status": "active"
}]
}
tasks:
- name: Show type(s)
debug:
msg:
- "{{ RPN.stdout | type_debug }}"
- "{{ RPN.stdout | from_json | type_debug }}"
- "{{ RPN.stdout | to_json | type_debug }}"
- name: Show members
debug:
msg: "{{ RPN.stdout | from_json | json_query('members') }}"
resulting into an output of
TASK [Show type(s)] *********
ok: [localhost] =>
msg:
- AnsibleUnicode
- dict
- str
TASK [Show members] *********
ok: [localhost] =>
msg:
- id: 0
ip: x.x.x.x
owner: buzut
private_ip: 10.91.154.39
speed: 100
status: active
I need to get value from key ip
- name: Show value of key 'ip'
debug:
msg: "{{ RPN.stdout | from_json | json_query('members[*].ip') }}"
resulting into an output of
TASK [Show value of key 'ip'] *********
ok: [localhost] =>
msg:
- x.x.x.x
Further Documentation
Selecting JSON data: JSON queries

ansible: how to split string with multiple delimiters

Here is my output.
"result.containers": [
{
"Image": "cna.docker.dev/webproxy:1.0.0",
},
{
"Image": "cna-docker-dev-local.docker.dev/lega-customer:1.0.1",
}
]
Here is my code
- name: adding it to groups using images
add_host:
name: "{{ inventory_hostname }}"
groups: '{{ (item.Image.split("/")[1:] | regex_replace("^/", "")).split(":")[0] }}'
with_items: "{{ result.containers }}"
I'm getting the below output:
"add_host": {
"groups": [
"['webproxy"
],
"host_name": "vm4.nodekite.com",
"host_vars": {
}
I would like to filter webproxy from cna.docker.dev/webproxy:1.0.0.
But I'm getting "['webproxy" instead of "webproxy.
Could someone please help!! I even tried this code which fails
(item.Image.split("/")[1:]).split(":")[0]
- debug:
msg: "{{ item.Image | regex_replace('.*?/(.*?):.*', '\\1') }}"
loop: "{{ result.containers | flatten(1) }}"
Example regex is non greedy, adjust as needed.
Gives
ok: [localhost] => (item={'Image': 'cna.docker.dev/webproxy:1.0.0'}) => {
"msg": "webproxy"
}
ok: [localhost] => (item={'Image': 'cna-docker-dev-local.docker.dev/lega-customer:1.0.1'}) => {
"msg": "lega-customer"
}

Searching Ansible debug messages for a string

I'm trying to find all the up interfaces on a switch, by looking at the results of some output from nxos_facts:
- name: get nxos facts via nxapi
nxos_facts:
provider: "{{ provider['nxapi'] }}"
gather_subset:
- "interfaces"
register: nxfacts_nxapi
- debug:
msg: "{{ nxfacts_nxapi.ansible_facts.ansible_net_interfaces | to_nice_json}} "
And I can successfully print out the debug to show the structure of the dictionary:
"ansible_net_interfaces": {
"Ethernet1/1": {
"bandwidth": 1000000,
"duplex": "full",
"ipv4": {
"address": "10.0.1.2",
"masklen": 24
},
"macaddress": "0800.276d.ee15",
"mtu": "1500",
"speed": "1000 Mb/s",
"state": "up",
"type": "100/1000/10000 Ethernet"
},
"Ethernet1/10": {
"bandwidth": 10000000,
"duplex": "auto",
"macaddress": "0800.276c.eecc",
"mode": "access",
"mtu": "1500",
"speed": "auto-speed",
"state": "down",
"type": "100/1000/10000 Ethernet"
},
But I'm struggling with the syntax to dereference the dictionary to only print when the "state" is "up"?
I'm running with the following version:
ansible 2.3.1.0
Any help is much appreciated.
You can iterate over the dictionary of interfaces and print only those elements for which the condition is true. Example:
- name: mytask
debug:
msg: "{{ item }}"
when: "item.value.state == 'up'"
with_dict: "{{ nxfacts_nxapi.ansible_facts.ansible_net_interfaces }}"