Ansible - Combining Lists of Dictionaries - list

Question

I have two lists of dictionaries. The first list of dict contains detailed information for all systems in an environment.
"cluster_detail": [
{
"env_name": "env_1",
"cluster_name": "cluster_1",
"primary": "server1",
"secondary": "server2,server3",
"Standby": "server4,server5",
"TieBreakers": "server6,server7"
},
{
"env_name": "env_1",
"cluster_name": "cluster_2",
"primary": "server11",
"secondary": "server12,server13",
"Standby": "server14,server15",
"TieBreakers": "server16,server17"
},
{
"env_name": "env_2",
"cluster_name": "cluster_7",
"primary": "server21",
"secondary": "server22,server23",
"Standby": "server24,server25",
"TieBreakers": "server26,server27"
},
{
"env_name": "env_2",
"cluster_name": "cluster_8",
"primary": "server31",
"secondary": "server32,server33",
"Standby": "server34,server35",
"TieBreakers": "server36,server37"
}
]
The second list of dict contains partial detail for some parts of the same environment.
"cluster_partial": [
{
"key": "env_1",
"value": [
"cluster_1",
"cluster_2",
"cluster_3",
"cluster_4"
]
},
{
"key": "env_2",
"value": [
"cluster_5",
"cluster_6",
"cluster_7",
"cluster_8"
]
}
]
I would like to add detail to "cluster_partial" by searching for its values (cluster_1, Cluster_2, etc...) in "Cluster_detail", and where there's a match, add in the standby or tiebreaker server names.
I've been going through https://docs.ansible.com/ansible/latest/playbook_guide/complex_data_manipulation.html and attempting various combinations of the examples given but am not having any success.
I've also looked at ansible - combine three lists of dictionaries but need to be more selective I think.
What I would like to end up with is
{
end_result:
[
{
"key": "env_1",
"value": [
{
"cluster_1": {
"tiebreaker": [
"server6",
"server7"
]
}
},
{
"cluster_2": {
"tiebreaker": [
"server16",
"server17"
]
}
},
"cluster_3",
"cluster_4"
]
},
{
"key": "env_2",
"value": [
"cluster_5",
"cluster_6",
{
"cluster_7": {
"tiebreaker": [
"server26",
"server27"
]
}
},
{
"cluster_8": {
"tiebreaker": [
"server36",
"server37"
]
}
}
]
}
]
}
Valid YAML
end_result:
- key: env_1
value:
- cluster_1:
tiebreaker: [server6, server7]
- cluster_2:
tiebreaker: [server16, server17]
- cluster_3
- cluster_4
- key: env_2
value:
- cluster_5
- cluster_6
- cluster_7:
tiebreaker: [server26, server27]
- cluster_8:
tiebreaker: [server36, server37]
I've tried the following
- name: Add tiebreaker Value to list of Dictionary
ansible.builtin.set_fact:
cluster_partial: |
{% for item in cluster_partialm %}
"tiebreaker": {{item}}
{% endfor %}
when: cluster_detail.cluster_name is defined and cluster_detail.cluster_name == cluster_partial.value`

Answer 1

It looks like cluster_partial is unnecessary for what you're doing; we can achieve something that almost matches your desired end state by using only the information in cluster_detail:
- hosts: localhost
gather_facts: false
tasks:
- set_fact:
end_result: >-
{{
end_result|combine({
item.env_name: {
item.cluster_name: {
"tiebreaker": item.TieBreakers.split(',')
}
}
}, recursive=True)
}}
vars:
end_result: {}
loop: "{{ cluster_detail }}"
- debug:
var: end_result
Which produces:
TASK [debug] *******************************************************************
ok: [localhost] => {
"end_result": {
"env_1": {
"cluster_1": {
"tiebreaker": [
"server6",
"server7"
]
},
"cluster_2": {
"tiebreaker": [
"server16",
"server17"
]
}
},
"env_2": {
"cluster_7": {
"tiebreaker": [
"server26",
"server27"
]
},
"cluster_8": {
"tiebreaker": [
"server36",
"server37"
]
}
}
}
}

Answer 2

Create the dictionary of available clusters
cluster_name_tiebreaker: "{{ cluster_detail|
items2dict(key_name='cluster_name',
value_name='TieBreakers') }}"
gives
cluster_name_tiebreaker:
cluster_1: server6,server7
cluster_2: server16,server17
cluster_7: server26,server27
cluster_8: server36,server37
Use Jinja to create the structure
end_result_str: |
{% for e in cluster_partial %}
{{ e.key }}:
{% for cluster in e.value %}
{% if cluster in cluster_name_tiebreaker %}
{{ cluster }}:
tiebreaker: {{ cluster_name_tiebreaker[cluster]|split(',') }}
{% else %}
{{ cluster }}:
{% endif %}
{% endfor %}
{% endfor %}
end_result_dict: "{{ end_result_str|from_yaml }}"
gives the dictionary
end_result_dict:
env_1:
cluster_1:
tiebreaker: [server6, server7]
cluster_2:
tiebreaker: [server16, server17]
cluster_3: null
cluster_4: null
env_2:
cluster_5: null
cluster_6: null
cluster_7:
tiebreaker: [server26, server27]
cluster_8:
tiebreaker: [server36, server37]
Convert the dictionary to the list
end_result: "{{ end_result_dict|dict2items }}"
gives
end_result:
- key: env_1
value:
cluster_1:
tiebreaker: [server6, server7]
cluster_2:
tiebreaker: [server16, server17]
cluster_3: null
cluster_4: null
- key: env_2
value:
cluster_5: null
cluster_6: null
cluster_7:
tiebreaker: [server26, server27]
cluster_8:
tiebreaker: [server36, server37]
This is not exactly what you want. The attribute value is a dictionary. If you really want a list change the template
end_result_str: |
{% for e in cluster_partial %}
{{ e.key }}:
{% for cluster in e.value %}
{% if cluster in cluster_name_tiebreaker %}
- {{ cluster }}:
tiebreaker: {{ cluster_name_tiebreaker[cluster]|split(',') }}
{% else %}
- {{ cluster }}:
{% endif %}
{% endfor %}
{% endfor %}
gives
end_result:
- key: env_1
value:
- cluster_1:
tiebreaker: [server6, server7]
- cluster_2:
tiebreaker: [server16, server17]
- {cluster_3: null}
- {cluster_4: null}
- key: env_2
value:
- {cluster_5: null}
- {cluster_6: null}
- cluster_7:
tiebreaker: [server26, server27]
- cluster_8:
tiebreaker: [server36, server37]
Fit the template further if you really want to have various types of items in the lists. Remove the colon : to get strings instead of dictionaries with undefined (null) values
{% else %}
- {{ cluster }}
gives what you want
end_result:
- key: env_1
value:
- cluster_1:
tiebreaker: [server6, server7]
- cluster_2:
tiebreaker: [server16, server17]
- cluster_3
- cluster_4
- key: env_2
value:
- cluster_5
- cluster_6
- cluster_7:
tiebreaker: [server26, server27]
- cluster_8:
tiebreaker: [server36, server37]
Example of a complete playbook for testing
- hosts: localhost
vars:
cluster_detail:
- Standby: server4,server5
TieBreakers: server6,server7
cluster_name: cluster_1
env_name: env_1
primary: server1
secondary: server2,server3
- Standby: server14,server15
TieBreakers: server16,server17
cluster_name: cluster_2
env_name: env_1
primary: server11
secondary: server12,server13
- Standby: server24,server25
TieBreakers: server26,server27
cluster_name: cluster_7
env_name: env_2
primary: server21
secondary: server22,server23
- Standby: server34,server35
TieBreakers: server36,server37
cluster_name: cluster_8
env_name: env_2
primary: server31
secondary: server32,server33
cluster_partial:
- key: env_1
value: [cluster_1, cluster_2, cluster_3, cluster_4]
- key: env_2
value: [cluster_5, cluster_6, cluster_7, cluster_8]
cluster_name_tiebreaker: "{{ cluster_detail|
items2dict(key_name='cluster_name',
value_name='TieBreakers') }}"
end_result_str: |
{% for e in cluster_partial %}
{{ e.key }}:
{% for cluster in e.value %}
{% if cluster in cluster_name_tiebreaker %}
- {{ cluster }}:
tiebreaker: {{ cluster_name_tiebreaker[cluster]|split(',') }}
{% else %}
- {{ cluster }}
{% endif %}
{% endfor %}
{% endfor %}
end_result_dict: "{{ end_result_str|from_yaml }}"
end_result: "{{ end_result_dict|dict2items }}"
tasks:
- debug:
var: cluster_name_tiebreaker
- debug:
var: end_result_str
- debug:
var: end_result_dict|to_yaml
- debug:
var: end_result|to_yaml