How to strip quotes from a variable in Ansible? - regex

I'm trying to get the value of a variable in a file into an Ansible variable so I can use it.
Here's what I've got:
- name: extract Unique Key
shell: "grep UNIQUE_KEY ../config.py | cut -d' ' -f 3"
register: command_output
- set_fact:
unique_key: x{{ command_output.stdout | regex_replace("^'", '') | regex_replace('^"', '') | regex_replace("'$", '') | regex_replace('"$', '') }}
- set_fact:
unique_key: "{{ unique_key | regex_replace('^x', '') }}"
- debug: var=unique_key
This works, but feels kludgy and looks ugly.
I've already tried to add sed to my original shell module, but I couldn't figure out how to get the quotes escaped correctly. I also couldn't figure out how to escape the regex_replace to get it to work in a single variable assignment.
Is there a simpler way to go from this:
"TEST"
or
'TEST'
to this:
TEST
in Ansible? (I'm also really new to Ansible so that's not helping either)
EDIT: After the answer by #Vladimir-Botka which I initially accepted, I found this issue:
If I don't strip the quotes and embed the variable in another variable, it keeps the quotes:
I need to use this value to construct a path:
vars:
service_location: "/opt/{{ unique_key }}-scheduler-service"
If I don't remove the quotes using my method above, The variable will contain the quotes as in this output of a debug statement:
ok: [fedorasvr1] => {
"service_location": "/opt/'TEST'-scheduler-service"
}

The short answer is "omit the first and the last character" if the quotes are part of the string
- set_fact:
unique_key: command_output.stdout[1:-1]
Internal interpretation is all the same. The quotes control the expansion of the variables. See 7.3.1. Double-Quoted Style and 7.3.2. Single-Quoted Style.
As an example. The play below
- hosts: localhost
vars:
var1: TEST
var2: 'TEST'
var3: "TEST"
tasks:
- template:
src: test.j2
dest: test
and the template
shell> cat test.j2
{{ var1 }}
{{ var2 }}
{{ var3 }}
give
shell> cat test
TEST
TEST
TEST
The quotes, if part of the string, can be removed. As an example the play below
- hosts: localhost
vars:
regex: "[`'\"]"
replace: ""
service_location: "/opt/{{ item|regex_replace(regex, replace)
}}-scheduler-service"
tasks:
- debug:
var: service_location
loop:
- '`TEST`'
- '"TEST"'
- '''TEST'''
- "'TEST'"
gives
ok: [localhost] => (item=`TEST`) =>
item: '`TEST`'
service_location: /opt/TEST-scheduler-service
ok: [localhost] => (item="TEST") =>
item: '"TEST"'
service_location: /opt/TEST-scheduler-service
ok: [localhost] => (item='TEST') =>
item: '''TEST'''
service_location: /opt/TEST-scheduler-service
ok: [localhost] => (item='TEST') =>
item: '''TEST'''
service_location: /opt/TEST-scheduler-service
It is also possible to use custom filter_plugins/string_filters.py which might be more convenient than complex escape constructs.
As an example. The play below
- hosts: localhost
vars:
replace: ""
service_location: "/opt/{{ item.0|string_replace(item.1, replace)
}}-scheduler-service"
tasks:
- debug:
var: service_location
with_together:
- - '`TEST`'
- '"TEST"'
- "'TEST'"
- - '`'
- '"'
- "'"
gives
ok: [localhost] => (item=[u'`TEST`', u'`']) =>
item:
- '`TEST`'
- '`'
service_location: /opt/TEST-scheduler-service
ok: [localhost] => (item=[u'"TEST"', u'"']) =>
item:
- '"TEST"'
- '"'
service_location: /opt/TEST-scheduler-service
ok: [localhost] => (item=[u"'TEST'", u"'"]) =>
item:
- '''TEST'''
- ''''
service_location: /opt/TEST-scheduler-service
FWIW, see other examples of filter_plugins.

Slightly different, but maybe related: I was getting unwanted quotes when using a lookup to initialize a variable which were then used in a "blockinfile" task. But it turns out the quotes were caused by the "blockinfile" and not the lookup, ie:
- name: set some variables
set_fact:
my_var: "{{ lookup('ini', 'my_var section=variables file=~/myconf.ini') }}"
- name: update myconf.cfg
blockinfile:
dest: myconf.cfg
state: present
create: true
owner: "{{user}}"
group: "{{user}}"
block: |
[Credentials]
access_key_id = blabla
secret_access_key = "{{my_var}}"
become: true
removing the quotes from this line worked:
secret_access_key = "{{my_var}}"

Related

ansible: stdout to list and dictionary

below is output of the "RESULT":
stdout:
- |-
ospf T1 VRF vrf1
ospf T2 VRF vrf2
ospf T3 VRF vrf3
stdout_lines:
- ospf T1 VRF vrf1
- ospf T2 VRF vrf2
- ospf T3 VRF vrf3
I want output in list and in dictionary:
1st output will be list. list will have following:
- T1
- T2
- T3
2nd output will be list like below:
ospf_vrf:
- vrf: vrf1
process: T1
- vrf: vrf2
process: T2
- vrf: vrf3
process: T3
3rd output will be dictionary.
how to do that?
I would start by parsing the data from stdout via regex. Assuming the components in your output are always: ospf <process_value> VRF <vrf_falue>, the regex looks like this:
{{ stdout | regex_findall('^ospf (.+) VRF (.+)$', multiline=True) }}
Based on this, you can then put the values into different forms and store them in variables:
- set_fact:
ospf: "{{ extracted | map('first') }}"
ospf_vrf: "{{ extracted | map('zip', ['process', 'vrf']) |
map('map', 'reverse') | map('community.general.dict') }}"
ospf_dict: "{{ dict(extracted) }}"
vrf_dict: "{{ dict(extracted | map('reverse')) }}"
vars:
extracted: "{{ stdout | regex_findall('^ospf (.+) VRF (.+)$', multiline=True) }}"
extracted is a helper with the regex values and the base for all other values
ospf is a list of the process_values
In each case the first element of the regex is taken.
ospf_vrf is a list of dicts
the keys process and vrf are assigned to the regex values
by reverse the keys are brought to the front
by community.general.dict a dict is created from the key-value pairs
ospf_dict: a dict based on process
vrf_dict: a dict based on vrf
Note: Both ospf_dict and vrf_dict will lose values if there are duplicates for the keys.
The output of the 4 variables looks like this:
TASK [debug] ***************************
ok: [localhost] => {
"ospf": [
"T1",
"T2",
"T3"
]
}
TASK [debug] ***************************
ok: [localhost] => {
"ospf_vrf": [
{
"process": "T1",
"vrf": "vrf1"
},
{
"process": "T2",
"vrf": "vrf2"
},
{
"process": "T3",
"vrf": "vrf3"
}
]
}
TASK [debug] ***************************
ok: [localhost] => {
"ospf_dict": {
"T1": "vrf1",
"T2": "vrf2",
"T3": "vrf3"
}
}
TASK [debug] ***************************
ok: [localhost] => {
"vrf_dict": {
"vrf1": "T1",
"vrf2": "T2",
"vrf3": "T3"
}
}
I have created this sample playbook for your requirements. I am not clear on third expected output. please clarify on that
- hosts: localhost
gather_facts: no
vars:
stdout_lines:
- ospf T1 VRF vrf1
- ospf T2 VRF vrf2
- ospf T3 VRF vrf3
tasks:
- set_fact:
first_output: "{{ first_output | default([]) + [ (item | split())[1] ] }}"
with_items:
"{{ stdout_lines }}"
- debug:
var: first_output
- set_fact:
second_output: "{{ second_output | default([]) + [ { 'vrf': (item | split())[3] , 'process': (item | split())[1] } ] }}"
with_items:
"{{ stdout_lines }}"
- debug:
var: second_output

Inconsistent results while formatting a list with Ansible

I'm using Ansible to build inventories dynamically according to several parameters.
I get raw info from our CMDB's API, then use set_fact to format a list of FQDNs.
Those FQDNs always follow the same formula, so it looks easy, but Ansible only seems to apply the templating to the first item.
The code:
- name: populate list of all hosts
set_fact:
all_hosts: '{{ all_hosts + [ "{{item.value.fields.friendlyname | lower}}" "." "{{item.value.fields.friendlyname[:3]|lower}}" ".sncoia.lan" ] }}'
loop: "{{ hostname.json.objects | dict2items }}"
when:
- item.value.fields.friendlyname | length == 12
- debug: var=all_hosts
The result (abridged for brevity):
TASK [debug] **********************************************************************************************************************************************************************************
ok: [localhost] => {
"all_hosts": [
"devlappaps03.dev.sncoia.lan",
"{{item.value.fields.friendlyname | lower}}.{{item.value.fields.friendlyname[:3]|lower}}.sncoia.lan",
"{{item.value.fields.friendlyname | lower}}.{{item.value.fields.friendlyname[:3]|lower}}.sncoia.lan",
Given the data for testing
hostname:
json:
objects:
foo:
fields:
friendlyname: devlappaps03
bar:
fields:
friendlyname: devlappaps02
baz:
fields:
friendlyname: devlappaps01
Properly concatenate the FQDN. For example
- set_fact:
all_hosts: "{{ all_hosts|d([]) + [_fqdn] }}"
loop: "{{ hostname.json.objects|dict2items }}"
when: item.value.fields.friendlyname|length == 12
vars:
_fqdn_list:
- "{{ item.value.fields.friendlyname|lower }}"
- "{{ item.value.fields.friendlyname[:3]|lower }}"
- "sncoia.lan"
_fqdn: "{{ _fqdn_list|join('.') }}"
gives
all_hosts:
- devlappaps03.dev.sncoia.lan
- devlappaps02.dev.sncoia.lan
- devlappaps01.dev.sncoia.lan

Ansible search a list and if a match is found append value as a new value to the dict

I've a dictionary that I'm gathering through an API that looks like below,
ok: [localhost] => {
"filtered_inventory": [
{
"Manufacturer": "",
"Name": "daughtercard 0/0/CPU0",
"Serial_number": "serial",
"device_id": 287,
"part_id": "ASR9001-LC"
},
{
"Manufacturer": "",
"Name": "module mau 0/0/2/3",
"Serial_number": "serial",
"device_id": 287,
"part_id": "SFP-10G-SR"
},
I'm also using the below API to get the below values but it's stored as a list
ok: [localhost] => {
"devices": [
{
"device_id": 287,
"hostname": "example.test.com"
},
{
"device_id": 350,
"hostname": "example.test.se"
},
What I want to achive is, if device_id is the same in both the list and the dict then append the hostname to the dictionary. I've solved it with the below code but as I'm working with a big dictionary it's highly inefficiant, running it with the below loop takes around 1 hour and eats a ton of memory.
- name: Merge hostname with inventory
#no_log: True
set_fact:
merged_list: "{{ merged_list|default([]) +
[{'hostname': item[0].hostname|upper,
'device_id': item[0].device_id,
'serial': item[1].Serial_number,
'Name': item[1].Name,
'Manufacturer': item[1].Manufacturer,
'part_id': item[1].part_id}] }}"
when: "item[0].device_id == item[1].device_id"
loop: "{{ query('nested', devices, filtered_inventory) }}"
I've so far been thinking that this probably could be solved by only looping through the dictionary and then match it to the list but haven't figured out how to do that, any ideas?
I'm thinking the below code could be a start but I've tried a few different approaches but failed to get it to work.
- name: Merge hostname with inventory
#no_log: True
set_fact:
merged_list: "{{ merged_list|default([]) + [item|combine
({'device_id': item.device_id,
'serial': item.Serial_number,
'Name': item.Name,
'Manufacturer': item.Manufacturer,
'part_id': item.part_id,
'hostname': '' })] }}"
loop: "{{ filtered_inventory }}"
when:
- "item.device_id is in devices|json_query(query)"
vars:
query: "'[*].device_id'"
Create a dictionary of the hostnames, e.g.
- set_fact:
devices_dict: "{{ devices|items2dict(key_name='device_id', value_name='hostname') }}"
gives
devices_dict:
287: example.test.com
350: example.test.se
Then, use this dictionary to combine items of a modified list, e.g.
- set_fact:
fi2: "{{ fi2|default([]) +
[item|combine({'hostname': devices_dict[item.device_id]})] }}"
loop: "{{ filtered_inventory }}"
gives
fi2:
- Manufacturer: ''
Name: daughtercard 0/0/CPU0
Serial_number: serial
device_id: 287
hostname: example.test.com
part_id: ASR9001-LC
- Manufacturer: ''
Name: module mau 0/0/2/3
Serial_number: serial
device_id: 287
hostname: example.test.com
part_id: SFP-10G-SR
Q: "It takes around 6 minutes to run ..."
A: This is too much. Try the filter below
shell> cat filter_plugins/combine_attr.py
def combine_attr(l, d, k, v):
x = []
for i in l:
i[k] = d[i[v]]
x.append(i)
return x
class FilterModule(object):
''' Ansible filter. Add an attribute to the dictionaries in the list.'''
def filters(self):
return {
'combine_attr': combine_attr,
}
For example
- set_fact:
fi2: "{{ filtered_inventory|
combine_attr(devices_dict, 'hostname', 'device_id') }}"

Build a list from multiple hosts in Ansible

I am trying to build a list of switch serial numbers using Ansible. The serial number is retrieved using IOS Facts and is easy but I cannot get the list no to be incremental, it just overwrites the list with the latest serial.
Here's the partial code I have:
tasks:
- name: Get all facts from ios devices
register: all_facts
ios_facts:
gather_subset: hardware
- name: Create list Serials
set_fact:
IOSserials: "{{ IOSserials + [ all_facts.ansible_facts.ansible_net_serialnum ] }}"
- name: Display list
debug:
msg: "The list is: {{ IOSserials }}"
And here's the result I get:
TASK [Create list Serials] *****************************************************
ok: [lab3650s2] => {"ansible_facts": {"IOSserials": ["FDO201XXXXD"]}, "changed": false}
ok: [lab3650s1] => {"ansible_facts": {"IOSserials": ["FDO192XXXXV"]}, "changed": false}
ok: [lab4500s1] => {"ansible_facts": {"IOSserials": ["FOX141XXXXV"]}, "changed": false}
TASK [Display list] ************************************************************
ok: [lab4500s1] => {
"msg": "The list is: ['FOX141XXXXV']"
}
ok: [lab3650s2] => {
"msg": "The list is: ['FDO201XXXXD']"
}
ok: [dev-lab3650s1] => {
"msg": "The list is: ['FDO192XXXXV']"
I'm trying to get a result like:
['FOX141XXXXV','FDO201XXXXD','FDO192XXXXV']"
It's possible to map extract. For example
- debug:
msg: "{{ ansible_play_hosts_all|map('extract', hostvars, 'IOSserials')|list }}"
run_once: true
gives
"msg": [
"FOX141XXXXV",
"FDO201XXXXD",
"FDO192XXXXV"
]

Ansible: How to filter down strings stored in a varriable based on keywords

I am trying to look up the un used disks using ansible. So far I am able to save the unused disk paths in a variable. But I am getting a lot of other useless information such as plus signs and brackets. I was wondering how can I filter the list down and get the strings that include /dev.
This is the code I have so far:
- set_fact:
list_of_disks: []
- name: getting list of disks
set_fact:
list_of_disks: "[{{list_of_disks}} + '/dev/{{item.key}}']"
when:
- not item.value.partitions
- not item.value.holders
- not item.value.links.ids
- item.key | search ("sd")
with_dict: "{{ ansible_devices }}"
- name: display unused disks
debug:
var: list_of_disks.split("'")
And this is the output I am getting
"list_of_disks.split(\"'\")": [
"[[[] + ",
"/dev/sde",
"] + ",
"/dev/sdc",
"]"
]
}
I would like the output to look like
"/dev/sde",
"/dev/sdc",
Note: the disk names would change from one server to another. So I don't want to look for sde and sdc only. It should be able to filer down all disks that come up in the list.
Well, might not be exactly the format you want, but:
- debug:
msg: "{{ item.device }}"
with_items: "{{ ansible_facts.mounts }}"
when: "'/dev/' in item.device"
Should do the trick.
On my local, I got:
TASK [debug] **********************************************************************************************************************************
ok: [localhost] => (item=None) => {
"msg": "/dev/mapper/fedora-root"
}
ok: [localhost] => (item=None) => {
"msg": "/dev/nvme0n1p1"
}
ok: [localhost] => (item=None) => {
"msg": "/dev/mapper/fedora-home"
}
If you need them all on one line, then create a variable and append to it:
- set_fact:
list_of_disks=""
- set_fact:
list_of_disks: "{{ list_of_disks | regex_replace('^, ','') }}, {{ item.device }}"
with_items: "{{ ansible_facts.mounts }}"
when: "'/dev/' in item.device"
- debug:
msg: "{{ list_of_disks }}"
For this, I got:
TASK [debug] **********************************************************************************************************************************
ok: [localhost] => {
"msg": "/dev/mapper/fedora-root, /dev/nvme0n1p1, /dev/mapper/fedora-home"
}