List creation addding 'undefined' entries when false in Ansible - list

I have the following playbook which is essentially working -
vars:
ansible_network_os: ios
IOSserials: []
tasks:
- name: Get all facts from ios devices
register: all_facts
ios_facts:
gather_subset: hardware
- name: Create list Serials
set_fact:
IOSserials: "{{IOSserials|default([]) + [{ 'name': all_facts.ansible_facts.ansible_net_hostname, 'IOS_serial': all_facts.ansible_facts.ansible_net_serialnum }] }}"
when: hostvars[inventory_hostname].serial != all_facts.ansible_facts.ansible_net_serialnum
- name: Display list
debug:
msg: "These switches have a difference in serial number {{ ansible_play_hosts_all|map('extract', hostvars, 'IOSserials')|list }}"
run_once: true
With the following result (I have one 'not equal' scenario in the switches):
TASK [Create list Serials] *****************************************************
skipping: [lab3650s1] => {"changed": false, "skip_reason": "Conditional result was False"}
skipping: [lab4500s1] => {"changed": false, "skip_reason": "Conditional result was False"}
ok: [lab3650s2] =>
{"ansible_facts": {"IOSserials": [{"IOS_serial": "FDO201XXXXD", "name": "lab3650s2"}]}, "changed": false}
TASK [Display list] ************************************************************
ok: [lab3650s1] => {
"msg": "These switches have a difference in serial number [Undefined, [{'name': 'lab3650s2', 'IOS_serial': 'FDO201XXXXD'}], Undefined]"
}
PLAY RECAP *********************************************************************
lab3650s1 : ok=2 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
lab3650s2 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
lab4500s1 : ok=1 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
I do not want the 'undefined' entries in the output list, I'd also like to understand why Ansible is inserting this in the list when its is skipping the set_fact for these entries.

That map pipeline is missing the select that would filter out the ones where extract did not produce a meaningful value; you can see it trivially reproducible:
- set_fact:
thingy:
one:
apple: is red
two:
banana: is yellow
three:
apple: is green
- debug:
msg: >
{{ ["one", "two", "three"] | map("extract", thingy, "banana") | list }}
- debug:
msg: >
{{ ["one", "two", "three"]
| map("extract", thingy, "banana")
| select
| list }}

Related

How to combine/match list with specific key/values as hostvar

I have a question about dicts and lists.
What I want to achieve is, have the key and value from a seperate list saved as fact/hostvar for each matching host.
I'm getting a list from a Confluence API that looks like this (abbreviated):
[
{
"title": "MACHINE1",
"_links": {
"tinyui": "/x/1234"
}
},
{
"title": "MACHINE2",
"_links": {
"tinyui": "/x/5678"
}
},
{
"title": "MACHINE3",
"_links": {
"tinyui": "/x/9876"
}
}
]
What worked to get each individual item (just to debug, and show that the loop itself works) is:
- name: DEBUG specific item in list of get_children.json.results
debug:
msg: "{{ item.title }} {{ item._links.tinyui }}"
loop:
"{{ get_children.json.results }}"
delegate_to: 127.0.0.1
Ansible Output (here: output for only one machine):
"msg": "MACHINE1 /x/1234"
Machine Hostnames:
Yes, they are lowercase in my inventory, and in the above list output they are uppercase. But I guess a simple item.title|lower would do fine.
machine1
machine2
machine3
How can I now match the item.title with ansible_hostname and save above API Output as a fact for each machine?
And for clarification: item.title|lower == ansible_hostname
I hope it gets clear to what I want to achieve and thanks to everyone in advance :)
EDIT: Thanks to both answers I managed to get it to work. Using '(?i)^'+VAR+'$' and some other conditional checks you guys posted definitely helped. :)
In a nutshell, given the inventories/tinyui/main.yml inventory:
---
all:
hosts:
machine1:
machine2:
machine3:
i.do.not.exist:
The folowing tinyui.yml playbook:
---
- hosts: all
gather_facts: false
vars:
# In real life, this is returned by your API call
get_children:
json:
results: [
{
"title": "MACHINE1",
"_links": {
"tinyui": "/x/1234"
}
},
{
"title": "MACHINE2",
"_links": {
"tinyui": "/x/5678"
}
},
{
"title": "MACHINE3",
"_links": {
"tinyui": "/x/9876"
}
}
]
# This won't be defined before you call the API which
# returns and registers the correct result. If there is
# no match for host in the returned json, '!no uri!' will
# be returned below. Adapt with a default uri if needed
tinyui: "{{ get_children.json.results
| selectattr('title', '==', inventory_hostname | upper)
| map(attribute='_links.tinyui')
| default(['!no uri!'], true) | first }}"
tasks:
# In real life, you would have called your API
# and registered the result in `get_children` e.g.
# - name: get info from confluence
# uri:
# url: <confluence api endpoint url>
# <more parameters here>
# run_once: true
# delegate_to: localhost
# register: get_children
- name: Display tinyui for host
debug:
msg: "tinyui for host {{ inventory_hostname }} is {{ tinyui }}"
Gives:
$ ansible-playbook -i inventories/tinyui/ tinyui.yml
PLAY [all] ***********************************************************************************************************************
TASK [Display tinyui for host] ***************************************************************************************************
ok: [machine1] => {
"msg": "tinyui for host machine1 is /x/1234"
}
ok: [machine2] => {
"msg": "tinyui for host machine2 is /x/5678"
}
ok: [machine3] => {
"msg": "tinyui for host machine3 is /x/9876"
}
ok: [i.do.not.exist] => {
"msg": "tinyui for host i.do.not.exist is !no uri!"
}
PLAY RECAP ***********************************************************************************************************************
i.do.not.exist : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
machine1 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
machine2 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
machine3 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
You can filter the list of dictionaries by title, taking out the right dict.
You can do this with the following line:
"{{ hostsdata | selectattr('title', 'match', '(?i)^'+host+'$') | first }}"
With selectattr you filter your list of dicts by the title, where this must match '(?i)^'+host+'$'.
The (?i) is the ignore case inline flag for pattern matching, concatenated with the hostname (case does not matter because of ignore case flag). ^...$ specifies that the whole string must match, from start to end.
Since selectattr returns a list as result, you can use first to take out the first element of the list.
Instead of using (?i) you can also set the ignorecase parameter, which will look like this:
"{{ hostsdata | selectattr('title', 'match', '^'+host+'$', 'ignorecase=true') | first }}"
Both variants work equivalently.
Entire playbook:
---
- hosts: localhost
gather_facts: no
vars:
hostsdata:
- {
"title": "MACHINE1",
"_links": {
"tinyui": "/x/1234"
}
}
- {
"title": "MACHINE2",
"_links": {
"tinyui": "/x/5678"
}
}
- {
"title": "MACHINE3",
"_links": {
"tinyui": "/x/9876"
}
}
tasks:
- debug:
var: hostsdata
- name: Pick out host specific dict
set_fact:
machine_data: "{{ hostsdata | selectattr('title', 'match', '(?i)'+host) | first }}"
vars:
host: machine3
- debug:
var: machine_data
- debug:
msg: "{{ machine_data.title }} {{ machine_data._links.tinyui }}"
Resulting output:
TASK [debug] ***********************************************************************************************************
ok: [localhost] => {
"hostsdata": [
{
"_links": {
"tinyui": "/x/1234"
},
"title": "MACHINE1"
},
{
"_links": {
"tinyui": "/x/5678"
},
"title": "MACHINE2"
},
{
"_links": {
"tinyui": "/x/9876"
},
"title": "MACHINE3"
}
]
}
TASK [Pick out host specific dict] *************************************************************************************
ok: [localhost]
TASK [debug] ***********************************************************************************************************
ok: [localhost] => {
"machine_data": {
"_links": {
"tinyui": "/x/9876"
},
"title": "MACHINE3"
}
}
TASK [debug] ***********************************************************************************************************
ok: [localhost] => {
"msg": "MACHINE3 /x/9876"
}
To filter multiple machines, here is another example:
- debug:
msg: "{{ md.title }} {{ md._links.tinyui }}"
when: md | length
vars:
md: "{{ hostsdata | selectattr('title', 'match', '(?i)^'+item+'$') | first | default('') }}"
with_items:
- MachINE1
- MACHINE2
- machine3
- unknown
Add a default('') and a when: to skip a non-existent hostname.
Output:
TASK [debug] ***********************************************************************************************************
ok: [localhost] => (item=MachINE1) => {
"msg": "MACHINE1 /x/1234"
}
ok: [localhost] => (item=MACHINE2) => {
"msg": "MACHINE2 /x/5678"
}
ok: [localhost] => (item=machine3) => {
"msg": "MACHINE3 /x/9876"
}
skipping: [localhost] => (item=unknown)
Convert the titles to lowercase
titles: "{{ get_children.json.results|
map(attribute='title')|
map('lower')|
map('community.general.dict_kv', 'title')|
list }}"
gives
titles:
- title: machine1
- title: machine2
- title: machine3
Replace the lowercase titles and create a dictionary
title_links: "{{ get_children.json.results|
zip(titles)|
map('combine')|
items2dict(key_name='title', value_name='_links') }}"
gives
title_links:
machine1:
tinyui: /x/1234
machine2:
tinyui: /x/5678
machine3:
tinyui: /x/9876
Put these declarations, for example, into the group_vars
shell> cat group_vars/all/title_links.yml
titles: "{{ get_children.json.results|
map(attribute='title')|
map('lower')|
map('community.general.dict_kv', 'title')|
list }}"
title_links: "{{ get_children.json.results|
zip(titles)|
map('combine')|
items2dict(key_name='title', value_name='_links') }}"
Now, you can use the dictionary. For example, given the inventory
shell> cat hosts
10.1.0.11 ansible_hostname=machine1
10.1.0.12 ansible_hostname=machine2
10.1.0.13 ansible_hostname=machine3
the playbook
- hosts: all
gather_facts: false
vars:
get_children:
json:
results:
- _links: {tinyui: /x/1234}
title: MACHINE1
- _links: {tinyui: /x/5678}
title: MACHINE2
- _links: {tinyui: /x/9876}
title: MACHINE3
tasks:
- debug:
msg: "My links: {{ title_links[ansible_hostname] }}"
gives (abridged)
TASK [debug] *********************************************************
ok: [10.1.0.11] =>
msg: 'My links: {''tinyui'': ''/x/1234''}'
ok: [10.1.0.12] =>
msg: 'My links: {''tinyui'': ''/x/5678''}'
ok: [10.1.0.13] =>
msg: 'My links: {''tinyui'': ''/x/9876''}'

Ansible - remove empty elements from the list

I have an ansible output from which I want to remove all empty (none) elements for a specific attribute. Here is the example list:
"resources": [
{
"id": "9c40900909",
"name": "some_name1"
},
{
"id": "pc4b09090"
},
{
"id": "8lknkkn45"
},
{
"id": "9df40900909",
"name": "some_name2"
}
]
Here is how I reduced the list to be just "resources" with attribute "name":
- set_fact:
resources_names: "{{ output.resources | map(attribute='name') }}"
Problem with that is that I get the elements that does not have any value for the name. These are "AnsibleUndefined" in the following output:
- debug:
msg: resources_names list is "{{ resources_names }}"
ok: [localhost] => {
"msg": "resources_names list is \"['some_name1', AnsibleUndefined, AnsibleUndefined, 'some_name2']\""
}
I tried to remove it with reject and regexp but that's not working.
- set_fact:
list2: "{{ resources_names | reject('match', '^$') | list }}"
Same with this one:
- set_fact:
resources_names: "{{ output.resources | map(attribute='name') | rejectattr('name', 'none') }}"
Any idea?
Thanks.
In a nutshell:
---
- hosts: localhost
gather_facts: false
vars:
resources: [
{
"id": "9c40900909",
"name": "some_name1"
},
{
"id": "pc4b09090"
},
{
"id": "8lknkkn45"
},
{
"id": "9df40900909",
"name": "some_name2"
}
]
tasks:
- debug:
msg: "{{ resources | selectattr('name', 'defined') | map(attribute='name') }}"
which gives:
$ ansible-playbook /tmp/play.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
PLAY [localhost] *******************************************************************************************************************************************************************************
TASK [debug] ***********************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"some_name1",
"some_name2"
]
}
PLAY RECAP *************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Note for your next question: please make sure you double check your examples and paste valid json/yaml data in your question. Thanks
Use the filter json_query. This filter ignores missing attributes
resources_names: "{{ output.resources|json_query('[].name') }}"
gives
resources_names:
- some_name1
- some_name2

Ansible regex replace String

I am trying to replace the unnecessary words from the below string using regex_replace but unable to do to so for the last part of it.
String: Test_[u'Net::Route Domain: RD2001']
Regex being used: regex_replace('.*: (.*)$', '\\1')
Output printing: Test_RD2001']
Expected Output: Test_RD2001
Could someone suggest how can I get rid of the last characters after RDXXXX.
Also this RDXXXX number is dynamic.
As dicussed in comments, you should try to fix the data source. However, here is the regex to print the expected result:
---
- name: Sample playbook
connection: local
#gather_facts: false
hosts: localhost
vars:
String: "Test_[u'Net::Route Domain: RD2001']"
tasks:
- debug:
msg: "{{ String| regex_replace('^(\\w+)\\[.*:\\s+([\\w]+).*', '\\1\\2') }}"
Results:
PLAY [Sample playbook] ***************************************************************************************************************
TASK [Gathering Facts] ***************************************************************************************************************
ok: [localhost]
TASK [debug] *************************************************************************************************************************
ok: [localhost] => {
"msg": "Test_RD2001"
}
PLAY RECAP ***************************************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Match a key to value from a dict and replace a value matching another dictionary using ansible

I've a dictionary containing the below items.
PID:
Cisco:
- A9K-MOD80-TR
- A9K-RSP440-TR
- ASR9001-LC
- SFP-10G-SR
The above dictionary I want to match with the list below with the following logic, if "SFP-10G-SR" (or any of the other values exist) exist in any of the part_id below, then replace 'Manufacturer' with Cisco (Cisco in this case is the key in the dict PID).
ok: [localhost] => (item={'hostname': '-', 'device_id': 287, 'serial': '-', 'Name': '-', 'Manufacturer': '', 'part_id': 'ASR9001-LC'})
ok: [localhost] => (item={'hostname': '-', 'device_id': 287, 'serial': '-', 'Name': '-', 'Manufacturer': '', 'part_id': 'SFP-10G-SR'})
I've somehow managed to get it to work but it's ugly and it's in a weird format with the below code.
- name: test
set_fact:
new_merged_list: "{{new_merged_list}} {{ item|combine({'Manufacturer': PID|dict2items|json_query(query)}) }}"
loop: "{{ merged_list | flatten(levels=1) }}"
vars:
query: "[?contains(value, '{{item.part_id }}')].key"
- debug:
msg: "{{ new_merged_list }}"
It gives me the below output
{
"msg": "
{'hostname': '-', 'device_id': 287, 'serial': '-', 'Name': '-', 'Manufacturer': ['Cisco'], 'part_id': 'ASR9001-LC'}
{'hostname': '-', 'device_id': 287, 'serial': '-', 'Name': '-', 'Manufacturer': ['Cisco'], 'part_id': 'SFP-10G-SR'}
}
If I run a debug on the "new_merged_list" I'm getting the below message
"msg": "AnsibleUnsafeText"
What I want is to keep the above as a dictionary, could anyone assist with a better solution or maybe assist in converting this back to a dictionary? In it's current state I'm having issues working with the "new_merged_list".
Your approach is correct despite the fact that it might seem ugly and weird. Let's simplify the data, e.g.
PID:
Cisco: [A, B]
HP: [C, D]
merged_list:
- {Manufacturer: '', part_id: A}
- {Manufacturer: '', part_id: D}
Your code with only a few modifications
- set_fact:
new_merged_list: "{{ new_merged_list|default([]) +
[item|combine({'Manufacturer': manufacturer})] }}"
loop: "{{ merged_list }}"
vars:
_dict: "{{ PID|dict2items }}"
query: "[?contains(value, '{{ item.part_id }}')].key"
manufacturer: "{{ _dict|json_query(query)|first }}"
gives the new list
new_merged_list:
- Manufacturer: Cisco
part_id: A
- Manufacturer: HP
part_id: D
In a nutshell, the folowing playbook:
---
- hosts: localhost
gather_facts: false
vars:
PID:
Cisco:
- A9K-MOD80-TR
- A9K-RSP440-TR
- ASR9001-LC
- SFP-10G-SR
toto:
- bla
- ble
- bli
model_brand: >-
{{
PID
| dict2items(key_name='brand', value_name='model_list')
| selectattr('model_list', 'contains', model | default('none'))
| map(attribute='brand')
| default(['Unkown brand'], true)
| first
}}
tasks:
- name: Show brand for known models (pass as extra param -e model='test')
debug:
msg: "Model {{ model | default('none')}} is of brand {{ model_brand }}"
Gives
$ ansible-playbook playbook.yml
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [Show brand for known models (pass as extra param -e model='test')] ***************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "Model none is of brand Unkown brand"
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
$ ansible-playbook playbook.yml -e brand='ASR9001-LC'
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [Show brand for known models (pass as extra param -e model='test')] ***************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "Model none is of brand Unkown brand"
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
$ ansible-playbook playbook.yml -e model='ASR9001-LC'
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [Show brand for known models (pass as extra param -e model='test')] ***************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "Model ASR9001-LC is of brand Cisco"
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
$ ansible-playbook playbook.yml -e model='ble'
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [Show brand for known models (pass as extra param -e model='test')] ***************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "Model ble is of brand toto"
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

'ansible_date_time' is undefined

Trying to register an ec2 instance in AWS with Ansible's ec2_ami module, and using current date/time as version (we'll end up making a lot of AMIs in the future).
This is what I have:
- name: Create new AMI
hosts: localhost
connection: local
gather_facts: false
vars:
tasks:
- include_vars: ami_vars.yml
- debug: var=ansible_date_time
- name: Register ec2 instance as AMI
ec2_ami: aws_access_key={{ ec2_access_key }}
aws_secret_key={{ ec2_secret_key }}
instance_id={{ temp_instance.instance_ids[0] }}
region={{ region }}
wait=yes
name={{ ami_name }}
with_items: temp_instance
register: new_ami
From ami_vars.yml:
ami_version: "{{ ansible_date_time.iso8601 }}"
ami_name: ami_test_{{ ami_version }}
When I run the full playbook, I get this error message:
fatal: [localhost]: FAILED! => {"failed": true, "msg": "ERROR! ERROR! ERROR! 'ansible_date_time' is undefined"}
However, when run the debug command separately, from a separate playbook, it works fine:
- name: Test date-time lookup
hosts: localhost
connection: local
tasks:
- include_vars: ami_vars.yml
- debug: msg="ami version is {{ ami_version }}"
- debug: msg="ami name is {{ ami_name }}"
Result:
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "ami version is 2016-02-05T19:32:24Z"
}
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "ami name is ami_test_2016-02-05T19:32:24Z"
}
Any idea what's going on?
Remove this:
gather_facts: false
ansible_date_time is part of the facts and you are not gathering it.
> cat tasks/test.yml
---
- hosts: node
gather_facts: no
tasks:
- setup:
gather_subset:
- min
- name: q
debug: var=ansible_date_time.epoch
and run
> ansible-playbook -i conf/share_var.conf tasks/test.yml --private-key=/root/.ssh/id_rsa -u ${USER} -b --become-method=sudo
PLAY [node] ********************************************************************************************************************************************************************************************************************************
TASK [setup] *******************************************************************************************************************************************************************************************************************************
ok: [xxxx.94.182]
ok: [xxxx.94.183]
ok: [xxxx.94.181]
TASK [q] ***********************************************************************************************************************************************************************************************************************************
ok: [xxxx.94.181] => {
"ansible_date_time.epoch": "1636712223"
}
ok: [xxxx.94.182] => {
"ansible_date_time.epoch": "1636712223"
}
ok: [xxxx.94.183] => {
"ansible_date_time.epoch": "1636712223"
}
PLAY RECAP *********************************************************************************************************************************************************************************************************************************
xxxx.94.181 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
xxxx.94.182 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
xxxx.94.183 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0