I'm trying to add an existing list to an existing dict in Ansible.
I have a dict "jbossvars" containing the following (ansible debug)
"jbossvars": {
"environments": {
"TEST_ENV": {
"key1": "value1",
"key2": "value2"
},
"TEST_ENV2": {
"key1": "value1",
"key2": "value2"
}
}
}
and a list "env_homes" containing the following (ansible debug)
"env_homes": [
"/opt/redhat/jboss-7.2.0/TEST_ENV",
"/opt/redhat/jboss-7.2.0/TEST_ENV2"
]
which I want to combine to a new dictionary "new_dict"
"jbossvars": {
"environments": {
"TEST_ENV": {
"key1": "value1",
"key2": "value2",
"key3": "/opt/redhat/jboss-7.2.0/TEST_ENV"
},
"TEST_ENV2": {
"key1": "value1",
"key2": "value2",
"key3": "/opt/redhat/jboss-7.2.0/TEST_ENV2"
}
}
}
The following play does not give me the desired situation:
- name: Create dict to append
set_fact:
env_homes: "{{ {'TEST_ENV': [ jbossvars.environments.TEST_ENV ] + env_homes} }}"
- name: Insert created dict into existing dict and save it into a new variable newdict
set_fact:
newdict: "{{ jbossvars.environments|combine(env_homes) }}"
- debug: var: newdict
To get the result
TEST_ENV: { "a": 1, "b": [ 2, "x1", "x2" ] }
The play below
vars:
TEST_ENV:
a: 1
b: 2
add_this:
c: [ x1, x2 ]
tasks:
- set_fact:
add_this: "{{ {'b': [ TEST_ENV.b ] + add_this.c} }}"
- set_fact:
TEST_ENV: "{{ TEST_ENV|combine(add_this) }}"
- debug:
var: TEST_ENV
gives
"TEST_ENV": {
"a": 1,
"b": [
2,
"x1",
"x2"
]
}
Related
I want to extract values from a nested dictionary. The dictionary is
"customers": {
"customer_1": [
{
"c1_cluster_1": [
{
"primary": [
"c1_server_1"
]
},
{
"secondaries": [
"c1_server_2",
"c1_server_3"
]
},
{
"tiebreakers": [
"c1_server_4",
"c1_server_5"
]
}
]
},
{
"c1_cluster_2": [
{
"primary": [
"c1_server_1"
]
},
{
"secondaries": [
"c1_server_2",
"c1_server_3"
]
},
{
"tiebreakers": [
"c1_server_4",
"c1_server_5"
]
}
]
},
{
"c1_cluster_3": [
{
"primary": [
"c1_server_1"
]
},
{
"secondaries": [
"c1_server_2",
"c1_server_3"
]
},
{
"tiebreakers": [
"c1_server_4",
"c1_server_5"
]
}
]
}
],
"customer_2": [
{
"c2_cluster_1": [
{
"primary": [
"c2_server_1"
]
},
{
"secondaries": [
"c2_server_2",
"c2_server_3"
]
},
{
"tiebreakers": [
"c2_server_4",
"c2_server_5"
]
}
]
},
{
"c2_cluster_2": [
{
"primary": [
"c2_server_1"
]
},
{
"secondaries": [
"c2_server_2",
"c2_server_3"
]
},
{
"tiebreakers": [
"c2_server_4",
"c2_server_5"
]
}
]
}
]
}
I need to have a list of unique server names per customer, the Cluster number or server role are not needed. End result would be
customer_servers: [
customer_1: [
"c1_server_1",
"c1_server_2",
"c1_server_3",
"c1_server_4",
"c1_server_5"
],
customer_2: [
"c2_server_1",
"c2_server_2",
"c2_server_3",
"c2_server_4",
"c2_server_5"
]
]
I've tried the following
- name: use Jinja to extract only the customer names and server names from customers
ansible.builtin.set_fact:
cust_servers: |
{% for cust in customers %}
{{ cust.key }}:
{% for serv in cust.value %}
{% for k, v in serv.items() %}
- {{v}}
{% endfor %}
{% endfor %}
{% endfor %}
- name: Convert cust_servers to a dictionary
ansible.builtin.set_fact:
cust_servers_dict: "{{cust_servers|from_yaml}}"
I've also attempted to break down the components of customers using debug but, have failed to get to the level that I wanted
- name: print the customer dictionary component parts
debug:
msg:
- "ClusterName is {{customers[0].key}} and it's value is"
- "{{customers[0].value}}
- " ----------------------------------------------------- "
- "{{customers[0].value[0].values()}}
My intention with this second attempt was to loop through the customer list but it became apparent that i might need multiple loops with this approach and i was probably over complicating it.
For example,
customer_servers_keys: "{{ customers.keys()|list }}"
customer_servers_vals: "{{ customers.values()|
map('ansible.utils.to_paths')|
map('dict2items')|
map('map', attribute='value')|
map('unique')|list }}"
customer_servers: "{{ dict(customer_servers_keys|
zip(customer_servers_vals)) }}"
gives the expected result
customer_servers:
customer_1: [c1_server_1, c1_server_2, c1_server_3, c1_server_4, c1_server_5]
customer_2: [c2_server_1, c2_server_2, c2_server_3, c2_server_4, c2_server_5]
Example of a complete playbook for testing
- hosts: localhost
vars:
customers:
customer_1:
- c1_cluster_1:
- primary: [c1_server_1]
- secondaries: [c1_server_2, c1_server_3]
- tiebreakers: [c1_server_4, c1_server_5]
- c1_cluster_2:
- primary: [c1_server_1]
- secondaries: [c1_server_2, c1_server_3]
- tiebreakers: [c1_server_4, c1_server_5]
- c1_cluster_3:
- primary: [c1_server_1]
- secondaries: [c1_server_2, c1_server_3]
- tiebreakers: [c1_server_4, c1_server_5]
customer_2:
- c2_cluster_1:
- primary: [c2_server_1]
- secondaries: [c2_server_2, c2_server_3]
- tiebreakers: [c2_server_4, c2_server_5]
- c2_cluster_2:
- primary: [c2_server_1]
- secondaries: [c2_server_2, c2_server_3]
- tiebreakers: [c2_server_4, c2_server_5]
# Expected result
# customer_servers:
# - customer_1: [c1_server_1, c1_server_2, c1_server_3, c1_server_4, c1_server_5]
# - customer_2: [c2_server_1, c2_server_2, c2_server_3, c2_server_4, c2_server_5]
customer_servers_keys: "{{ customers.keys()|list }}"
customer_servers_vals: "{{ customers.values()|
map('ansible.utils.to_paths')|
map('dict2items')|
map('map', attribute='value')|
map('unique')|list }}"
customer_servers: "{{ dict(customer_servers_keys|
zip(customer_servers_vals)) }}"
tasks:
- debug:
var: customer_servers|to_yaml
I have json file (I can't change what is in json file)
{
"application": {
...,
"VS": [
{
"LTM": "server1",
"ltm_vs": "VS_1",
"address": "IP",
"port": "80",
"link": "/Common/link"
},
{
"LTM": "server1",
"ltm_vs": "VS_2",
"address": "IP",
"port": "8081",
"link": "/Common/link"
},
{
"LTM": "server1",
"ltm_vs": "VS_3",
"address": "IP",
"port": "443",
"link": "/Common/link"
}
]
}
}
I need to take that file to create list with only some information.
So presently I import the json file like that:
...
tasks:
- name: Include application information
include_vars:
file: path/file.json
...
and was create a variables list application.VS I need to map attribute LTM to server and ltm_vs to virtual_server and create list:
VS_lst:
- { server: "server1", virtual_server: "VS_1" }
- { server: "server1", virtual_server: "VS_2" }
- { server: "server1", virtual_server: "VS_3" }
I have try with that:
- name: Add variables
set_fact:
VS_lst: >-
{{ VS_lst +[{
'server': "{{ item.LTM }}",
'virtual_server': "{{ item.ltm_vs }}"
}]
}}
with_items: "{{ application.VS }}"
- debug:
msg: "{{VS_lst}}"
the output:
TASK [Add variables] *******************************************************************************
ok: [GTM_Server] => (item={'LTM': 'server1', 'ltm_vs': 'VS_1', 'address': 'IP', 'port': '80', 'link': '/Common/link'})
ok: [GTM_Server] => (item={'LTM': 'server1', 'ltm_vs': 'VS_2', 'address': 'IP', 'port': '8081', 'link': '/Common/link'})
ok: [GTM_Server] => (item={'LTM': 'server1', 'ltm_vs': 'VS_3', 'address': 'IP', 'port': '443', 'link': '/Common/link'})
TASK [debug] ***************************************************************************************
ok: [GTM_Server] => {
"msg": [
{
"server": "server1",
"virtual_server": "VS_2"
},
{
"server": "{{ item.LTM }}",
"virtual_server": "{{ item.ltm_vs }}"
},
{
"server": "{{ item.LTM }}",
"virtual_server": "{{ item.ltm_vs }}"
}
]
}
This is the fixed version
- name: Add variables
set_fact:
VS_lst: "{{ VS_lst|default([]) +
[{'server': item.LTM,
'virtual_server': item.ltm_vs}] }}"
loop: "{{ application.VS }}"
There are also other options. For example, it's possible to use json_query
- set_fact:
VS_lst: "{{ application.VS|
json_query('[].{server: LTM,
virtual_server: ltm_vs}') }}"
- debug:
var: VS_lst
give
"VS_lst": [
{
"server": "server1",
"virtual_server": "VS_1"
},
{
"server": "server1",
"virtual_server": "VS_2"
},
{
"server": "server1",
"virtual_server": "VS_3"
}
]
Without json_query, this task "creates a list of items base on another list of items" and gives the same result
- set_fact:
VS_lst: "{{ VS_lst|default([]) +
[dict(dictionary_keys|
zip(selected_params|
map('extract', item)))] }}"
loop: "{{ application.VS }}"
vars:
selected_params: ['LTM','ltm_vs']
dictionary_keys: ['server', 'virtual_server']
I've got a list of dict like:
- { "a": "zzz", "b": [1, 2] }
- { "a": "yyy", "b": [7, 9] }
I need with ansible to loop over it so item would be successively:
- { "a": "zzz", "b": 1 }
- { "a": "zzz", "b": 2 }
- { "a": "yyy", "b": 7 }
- { "a": "yyy", "b": 9 }
How could I do that?
Perhaps it can be done in 1 task, but here is how i would do it:
You need 1 loop for the "master" list and then 1 dynamic loop for the list of the 'b' key. We will use an include_tasks task in order to process each dict of the "master" list, then in the included tasks file just a set_fact to loop the "b" key, and populate our variable.
CODE:
main.yml:
---
- hosts: localhost
gather_facts: false
vars:
source_var:
- { "a": "zzz", "b": [1, 2] }
- { "a": "yyy", "b": [7, 9] }
tasks:
- name: print var
include_tasks: "set_fact.yml"
with_items: "{{ source_var }}"
loop_control:
loop_var: itemoflist
- name: print var
debug:
var: target_var
included PB, called set_fact.yml:
- name: populate_var
set_fact:
target_var: "{{ target_var | default([]) + [{ 'a': itemoflist.a, 'b': item }] }}"
with_items:
- "{{ itemoflist['b'] }}"
variable created:
TASK [print var] *******************************************************************************************************************************************************************************************************
ok: [localhost] => {
"target_var": [
{
"a": "zzz",
"b": 1
},
{
"a": "zzz",
"b": 2
},
{
"a": "yyy",
"b": 7
},
{
"a": "yyy",
"b": 9
}
]
}
hope it helps.
I have a dictionary which is loaded by the following play:
- name: Get variables from ../main.yml and save them into dict
include_vars:
file: "../main.yml"
name: dict
"dict" contains the following:
"dict": {
"environments": {
"MYENV": {
"key1": "value1",
"key2": "value2"
},
"MYENV2": {
"key1": "value1",
"key2": "value2"
},
"MYENV3": {
"key1": "value1",
"key2": "value2"
}
}}
Question: How can I loop through this dictionary in Ansible and add a 3rd key "key3" with accompanied value to each entry in "environments"?
The desired situation would be a new_dict which contains the following:
"new_dict": {
"environments": {
"MYENV": {
"key1": "value1",
"key2": "value2",
"key3": "value3"
},
"MYENV2": {
"key1": "value1",
"key2": "value2",
"key3": "value3"
},
"MYENV3": {
"key1": "value1",
"key2": "value2",
"key3": "value3"
}
}}
With "value3" being a string built like "MYENV" + "value1" + "value2".
The decomposition of the directory is needed. Then {key3: value3} is added and the directory is combined again. The tasks below
vars:
add_this:
key3: value3
tasks:
- set_fact: # collect dictionary keys
keys: "{{ dict.environments|
dict2items|
json_query('[].key') }}"
- set_fact: # collect dictionary values and add item
values: "{{ dict.environments|
dict2items|
json_query('[].value')|
map('combine', add_this)|list }}"
- set_fact: # create dict environments
environments: "{{ environments|
default({})|
combine({item.0: item.1}) }}"
loop: "{{ keys|zip(values)|list }}"
- set_fact: # create dictionary new_dict
new_dict: "{{ new_dict|
default({})|
combine({'environments': environments}) }}"
- debug:
var: new_dict
give
"new_dict": {
"environments": {
"MYENV": {
"key1": "value1",
"key2": "value2",
"key3": "value3"
},
"MYENV2": {
"key1": "value1",
"key2": "value2",
"key3": "value3"
},
"MYENV3": {
"key1": "value1",
"key2": "value2",
"key3": "value3"
}
}
}
Given the fact that all values are different, how can I dynamically create "value3" for each of the environments by e.g. appending the values of "key1" and "key2"?
With a couple of filters
$ cat filter_plugins/filter1.py
def custom_1(h):
return {'key3': h.values()}
def dict_merge(x, y, recursive=False):
if recursive:
z = dict(list(x.items()) + list(y.items()))
else:
z = x.copy()
z.update(y)
return z
def dict_keys(d):
return list(d)
class FilterModule(object):
def filters(self):
return {
'custom_1' : custom_1,
'dict_keys' : dict_keys,
'dict_merge' : dict_merge
}
the tasks below
tasks:
- set_fact:
env: "{{ env|default({})|
combine({item: dict.environments[item]|
dict_merge((dict.environments[item]|custom_1), True)
}) }}"
loop: "{{ dict.environments|dict_keys }}"
- set_fact:
new_dict: "{{ {}|combine({'environments': env}) }}"
- debug:
var: new_dict
give
"new_dict": {
"environments": {
"MYENV": {
"key1": "value1",
"key2": "value2",
"key3": [
"value2",
"value1"
]
},
"MYENV2": {
"key1": "value1",
"key2": "value2",
"key3": [
"value2",
"value1"
]
},
"MYENV3": {
"key1": "value1",
"key2": "value2",
"key3": [
"value2",
"value1"
]
}
}
}
Fit the custom_1 filter to your needs.
How to check if an object of a list of dictionaries is present in another list of dictionaries (with Ansible?)
You could try jinja filters selectattr - I had an issue with using it, so I did revert to a simplified but ugly solution - build a filtered list and compare filtered attributes only (list-to-list).
I do not like it, but it works.
Let me know if you know other way.
playbook:
- name: find existing system_crontabs #would generate a list of dict
find:
path: /var/spool/cron/crontabs/
register: system_side_crontabs
become: True
- name: create lists of system_cron_names and repo_cron_names
set_fact:
system_cron_names: "[]"
repo_cron_names: "[]"
- name: build list of system_cron_names
set_fact:
system_cron_names: "{{ system_cron_names }} + [ '{{ item.path |basename }}' ]"
with_items: "{{ system_side_crontabs.files }}"
- name: build lists of repo_cron_names
set_fact:
repo_cron_names: "{{ repo_cron_names }} + [ '{{ item.user }}' ]"
with_items: "{{ crontabs }}"
- name: assert check if an object of system_crontab is defined in repo_crontab
assert:
that: "{{ [item] |intersect(repo_cron_names) | length }} == 1"
with_items: "{{ system_cron_names }}"
hosts_vars/prd-inner-mgmt202 #a list of dictionaries
crontabs:
- user: root
crontab_rules: |
11 1 * * * find /home/ansible/.ansible/tmp/ -atime +10 -delete
a result of find
ok: [prd-inner-mgmt202] => {
"changed": false,
"examined": 1,
"files": [ ### List of dictionary
{
"path": "/var/spool/cron/crontabs/root",
},
{
"path": "/var/spool/cron/crontabs/another_file",
}
],
"invocation": {
"module_args": {
"age": null,
"age_stamp": "mtime",
"contains": null,
"file_type": "file",
"follow": false,
"get_checksum": false,
"hidden": false,
"path": "/var/spool/cron/crontabs/",
"paths": [
"/var/spool/cron/crontabs/"
],
"patterns": [
"*"
],
"recurse": false,
"size": null,
"use_regex": false
},
"module_name": "find"
},
"matched": 1,
"msg": ""
}
generate a list of strings that are easy to compare
TASK [mid_crontab : build list of system_cron_names] **************************
"ansible_facts": {
"system_cron_names": [
"root",
]
},
"changed": false,
"invocation": {
"module_args": {
"system_cron_names": [
"root"
]
},
"module_name": "set_fact"
},
generate another list of strings
TASK [mid_crontab : build list of repo_cron_names] *****************************
ok: [prd-inner-mgmt202] => (item={u'crontab_rules': u'11 1 * * * find /home/ansible/.ansible/tmp/ -atime +10 -delete\n', u'user': u'root'}) => {
"ansible_facts": {
"repo_cron_names": [
"root"
]
},
"changed": false,
"invocation": {
"module_args": {
"repo_cron_names": [
"root",
"other"
]
},
"module_name": "set_fact"
},
"item": {
"crontab_rules": "11 1 * * * find /home/ansible/.ansible/tmp/ -atime +10 -delete\n",
"user": "root"
}
}
Assert the required check, use intersect jinja filter. In my case a system defined object (cron record) should exist in my repository - so the list should have 1 element.
TASK [mid_crontab : assert check if system_crontab is defined in repo_crontab] *
ok: [prd-inner-mgmt202] => (item=root) => {
"changed": false,
"invocation": {
"module_args": {
"that": "1 == 1"
},
"module_name": "assert"
},
"item": "root",
"msg": "All assertions passed"
}
Seems you want to reduce original dict to list of strings (names) and compare the difference:
---
- hosts: localhost
gather_facts: no
vars:
crontabs:
- user: root
crontab_rules: xxx
tasks:
- find:
path: /tmp/test
register: myfiles
- assert:
that: sys_crons_violation | count == 0
msg: "This crons are not defined in repo : {{ sys_crons_violation | join(', ') }}"
vars:
sys_crons: "{{ myfiles.files | map(attribute='path') | map('basename') | list }}"
repo_crons: "{{ crontabs | map(attribute='user') | list }}"
sys_crons_violation: "{{ sys_crons | difference(repo_crons) }}"
result:
TASK [find] ************************
TASK [assert] **********************
fatal: [localhost]: FAILED! => {
"assertion": "sys_crons_violation | count == 0",
"changed": false,
"evaluated_to": false,
"msg": "This crons are not defined in repo : another_file"
}