Convert nested list of dicts to dict in Ansible - list

I'm trying to convert a list of dictionary in dictionary. My defined vars:
users:
- name: bruce
age: 22
params:
- key: "k1"
value: "v1bruce"
- key: "k2"
value: "v2bruce"
- name: alan
age: 25
params:
- key: "k1"
value: "v1alan"
- key: "k5"
value: "v5alan"
- name: carl
age: 43
params:
- key: "k1"
value: "v1carl"
- key: "k2"
value: "v2carl"
- key: "k5"
value: "v5carl"
I need to create a new structure in Ansible starting from the previous converting params to dictionaries like the following:
employee:
- name: bruce
age: 22
params:
"k1": "v1bruce"
"k2": "v2bruce"
- name: alan
age: 25
params:
"k1": "v1alan"
"k5": "v5alan"
- name: carl
age: 43
params:
"k1": "v1carl"
"k2": "v2carl"
"k5": "v5carl"
The key point is that I don't know how many elements my starting list is composed.
Is it possible?
Thanks.

Use items2dict in loop. For example
- set_fact:
employee: "{{ employee|default([]) +
[{'name': item.name,
'age': item.age,
'params': item.params|items2dict}] }}"
loop: "{{ users }}"

Warning: ugly solution ahead.
The base to understand is that your params looks like a transformation of a dict via the dict2items filter. So we have to reverse the operation with items2dict. But since it is nested inside your list of hash, it is not so easy. This is what I came up with:
---
- hosts: localhost
gather_facts: False
vars:
users:
- name: bruce
age: 22
params:
- key: "k1"
value: "v1bruce"
- key: "k2"
value: "v2bruce"
- name: alan
age: 25
params:
- key: "k1"
value: "v1alan"
- key: "k5"
value: "v5alan"
- name: carl
age: 43
params:
- key: "k1"
value: "v1carl"
- key: "k2"
value: "v2carl"
- key: "k5"
value: "v5carl"
tasks:
- name: Show raw
debug:
var: users
- name: Transform
set_fact:
user_transformed: >-
{%- set result = [] -%}
{%- for user in users -%}
{{
result.append({
'name': user.name,
'age': user.age,
'params': user.params | items2dict
})
}}
{%- endfor -%}
{{ result | to_json | from_json }}
- name: Show transformed
debug:
var: user_transformed

Related

ansible - need output in csv in multiple columns

I have playbook as below:
tasks:
- name: To Get list of all ACM
aws_acm_info:
region: "{{ region }}"
register: acm
- name: Cname Names
set_fact:
cname: "{{ acm | json_query(jmesquery) }}"
vars:
jmesquery: 'certificates[*].domain_validation_options[].resource_record.name'
- name: Cname Values
set_fact:
value: "{{ acm | json_query(jmesquery) }}"
vars:
jmesquery: 'certificates[*].domain_validation_options[].resource_record.value'
- name: set file header
shell: echo 'cname, value'> {{ path }}
run_once: true
- name: CSV - Write information into .csv file
lineinfile:
insertafter: ','
dest: "{{ path }}"
line: "{{ item }}"
with_items:
- "{{ cname }}"
- "{{ value }}"
I am getting output in single column as cname, but I need values in 2nd column as value.
required output format world be
enter image description here
I really appreciate any help you can provide.
Given the data below for testing
acm:
certificates:
- domain_validation_options:
- resource_record: {name: aaa, value: 1}
- domain_validation_options:
- resource_record: {name: bbb, value: 2}
- domain_validation_options:
- resource_record: {name: ccc, value: 3}
Get the name/value pairs in a single query
_query: 'certificates[*].domain_validation_options[].[resource_record.name,
resource_record.value]'
csv: "{{ acm|json_query(_query)|map('join', ',')|list }}"
give
csv:
- aaa,1
- bbb,2
- ccc,3
You can write the lines into a file
- lineinfile:
dest: /tmp/cname.csv
line: "{{ item }}"
loop: "{{ csv }}"
gives
shell> cat /tmp/cname.csv
aaa,1
bbb,2
ccc,3
Example of a complete playbook
- hosts: localhost
vars:
acm:
certificates:
- domain_validation_options:
- resource_record: {name: aaa, value: 1}
- domain_validation_options:
- resource_record: {name: bbb, value: 2}
- domain_validation_options:
- resource_record: {name: ccc, value: 3}
_query: 'certificates[*].domain_validation_options[].[resource_record.name,
resource_record.value]'
csv: "{{ acm|json_query(_query)|map('join', ',')|list }}"
tasks:
- lineinfile:
dest: /tmp/cname.csv
line: "{{ item }}"
loop: "{{ csv }}"
Read the file
- hosts: localhost
tasks:
- community.general.read_csv:
path: /tmp/cname.csv
fieldnames: cname,value
delimiter: ','
register: cname
- debug:
var: cname.list
gives
cname.list:
- {cname: aaa, value: '1'}
- {cname: bbb, value: '2'}
- {cname: ccc, value: '3'}
You can see that the values are strings now. If you want to keep values as integers store the data in YAML format
- hosts: localhost
vars:
acm:
certificates:
- domain_validation_options:
- resource_record: {name: aaa, value: 1}
- domain_validation_options:
- resource_record: {name: bbb, value: 2}
- domain_validation_options:
- resource_record: {name: ccc, value: 3}
_query: 'certificates[*].domain_validation_options[].[resource_record.name,
resource_record.value]'
csv: "{{ acm|json_query(_query)|map('join', ': ')|list }}"
tasks:
- lineinfile:
dest: /tmp/cname.yml
line: "{{ item }}"
loop: "{{ csv }}"
gives
shell> cat /tmp/cname.yml
aaa: 1
bbb: 2
ccc: 3
Read the file
- hosts: localhost
tasks:
- include_vars:
file: /tmp/cname.yml
name: cname_dict
- debug:
var: cname_dict
gives
cname_dict:
aaa: 1
bbb: 2
ccc: 3

How to concatenate two lists of dictonaries in Ansible

I have two list of dicts:
dev_users:
- name: cs3141
key:
cs513e_key1.pub
cs513e_key2.pub
- name: ab1234
key:
ab1234.pub
- name: cd5678
key:
ab1234.pub
and
sys_admin_users:
- name: xy3141
key:
xy3141.pub
- name: cd1234
key:
cd1234.pub
- name: ef5678
key:
ef5678.pub
When I try to concatenate them:
- set_fact: users= "{{ dev_users + sys_admin_users }}"
I get this error:
ERROR! failed to combine variables, expected dicts but got a 'dict' and a 'AnsibleSequence':
{}
[{"set_fact": "users= \"{{ dev_users + sys_admin_users }}\""}]
How can I concatenate these two lists?
The problem was that I was trying to concatenate the two list in the variables section, as opposed to the task section. Tadeboro in #ansible gave me this code that worked:
---
- hosts: localhost
gather_facts: false
vars:
dev_users:
- name: cs3141
key:
cs513e_key1.pub
cs513e_key2.pub
- name: ab1234
key:
ab1234.pub
- name: cd5678
key:
ab1234.pub
sys_admin_users:
- name: xy3141
key:
xy3141.pub
- name: cd1234
key:
cd1234.pub
- name: ef5678
key:
ef5678.pub
tasks:
- name: Test
set_fact:
users: "{{ dev_users + sys_admin_users }}"
This command works fine: ansible-playbook -v x.yaml
Here is another solution, that iteratively concatenates a list of list. I like it better, because it is more general. It only works in more recent versions of ansible, because there was a bug in version 2.5 which took some time to fix.
---
# ansible-playbook -v manyx.yaml
- hosts: localhost
gather_facts: false
vars:
dev_users:
- name: cs3141
key:
cs513e_key1.pub
cs513e_key2.pub
- name: ab1234
key:
ab1234.pub
- name: cd5678
key:
ab1234.pub
sys_admin_users:
- name: xy3141
key:
xy3141.pub
- name: cd1234
key:
cd1234.pub
- name: ef5678
key:
ef5678.pub
other_users:
- name: fe9876
key:
fe9876.pub
list_of_users_list:
- "{{ dev_users }}"
- "{{ sys_admin_users }}"
- "{{ other_users }}"
all_users: []
tasks:
- name: Test
set_fact:
all_users: "{{ item + all_users }}"
loop: "{{ list_of_users_list }}"
- name: print all_users
debug:
msg: "{{ item }}"
loop: "{{ all_users }}"

Convert a list to dictionary - Ansible YAML

I have a playbook, where I get an error message
fatal: [localhost]: FAILED! => {"ansible_facts": {"tasks": {}}, "ansible_included_var_files": [], "changed": false, "message": "/home/user/invoke_api/automation/tmp/task.yml must be stored as a dictionary/hash"}
task.yml
The file task.yml is dynamically created, and filtered all the time from another source to give the output below.
- key: gTest101
value:
Comments: FWP - Testing this
IP: 10.1.2.3
Name: gTest101
- key: gTest102
value:
Comments: FWP - Applying this
IP: 10.1.2.4
Name: gTest102
Question: How do I convert the list in my task.yml to a dictionary? What's the code to convert from list to dictionary
playbook.yml
---
- name: Global Objects
hosts: check_point
connection: httpapi
gather_facts: False
vars_files:
- 'credentials/my_var.yml'
- 'credentials/login.yml'
tasks:
- name: read-new-tmp-file
include_vars:
file: tmp/task.yml
name: tasks
register: new_host
- name: add-host-object-to-group
check_point.mgmt.cp_mgmt_host:
name: "{{ item.value.Name | quote }}"
ip_address: "{{ item.value.IP | quote }}"
comments: "{{ item.value.Comments }}"
groups: gTest1A
state: present
auto_publish_session: yes
loop: "{{ new_host.dict | dict2items }}"
delegate_to: Global
ignore_errors: yes
Ansible core 2.9.13
python version = 2.7.17
Q: "How do I convert the list in my task.yml to a dictionary?"
A: Use items2dict. For example, read the file and create the list
- set_fact:
l1: "{{ lookup('file', 'task.yml')|from_yaml }}"
gives
l1:
- key: gTest101
value:
Comments: FWP - Testing this
IP: 10.1.2.3
Name: gTest101
- key: gTest102
value:
Comments: FWP - Applying this
IP: 10.1.2.4
Name: gTest102
Then, the task below
- set_fact:
d1: "{{ l1|items2dict }}"
creates the dictionary
d1:
gTest101:
Comments: FWP - Testing this
IP: 10.1.2.3
Name: gTest101
gTest102:
Comments: FWP - Applying this
IP: 10.1.2.4
Name: gTest102
A vars file is a yaml dict file, so your list have to be a fields of a vars:
my_vars:
- Comments: FWP - Testing this
IP: 10.1.2.3
Name: gTest101
- Comments: FWP - Applying this
IP: 10.1.2.4
Name: gTest102
and you dont need to register the include_vars tasks. Just loop on the list var name (here my_vars).
---
- name: Global Objects
hosts: check_point
connection: httpapi
gather_facts: False
vars_files:
- 'credentials/my_var.yml'
- 'credentials/login.yml'
tasks:
- name: read-new-tmp-file
include_vars:
file: tmp/task.yml
- name: add-host-object-to-group
check_point.mgmt.cp_mgmt_host:
name: "{{ item.Name }}"
ip_address: "{{ item.IP }}"
comments: "{{ item.Comments }}"
groups: gTest1A
state: present
auto_publish_session: yes
loop: "{{ my_vars }}"
delegate_to: Global
ignore_errors: yes

Ansible - remove user from group x if already in other group

I am looking for the easiest way to remove users from group x when they are already in group google-sudo. I store users in group vars in this kind of list and dictionary combination:
user_account:
- name: jenny
authorized_keys:
- jenny_01
groups:
- "{% if not googlesudo.stat.exists %}sudo{% else %}google-sudo{% endif %}"
- name: jerry
authorized_keys:
- jerry_01
groups:
- "{% if not googlesudo.stat.exists %}sudo{% else %}google-sudo{% endif %}"
...
These are tasks I already created:
- name: Check if google-sudo file exists
stat:
path: /etc/sudoers.d/google_sudo
register: googlesudo
tags:
- add_user_group
- remove_from_x
- debug: var=googlesudo verbosity=2
tags:
- add_user_group
- remove_from_x
- debug:
msg: "User account to create: {{ item.name }}"
with_items: "{{ user_account }}"
- name: "Creating user"
user:
name: "{{ item.name }}"
group: users
shell: /bin/bash
with_items: "{{ user_account }}"
- name: Add user to additional groups
user:
name: "{{ item.0.name }}"
groups: "{{ item.1 }}"
append: yes
with_subelements:
- "{{ user_account }}"
- groups
tags:
- remove_from_x
- name: Check if user already in google-sudo
command: "groups {{ item.name }}"
with_items:
- "{{ user_account }}"
register: root_users
tags:
- remove_from_x
- name: View root users
debug:
msg: "{{ item }}"
verbosity: 2
with_items:
- "{{ root_users }}"
tags:
- remove_from_x
- name: Save state
set_fact:
is_in_googlesudo: "{{ root_users.results.0.stdout_lines }}"
tags:
- remove_from_x
- name: List
debug: msg='{{ user_account |json_query("[?groups==`google-sudo`]")}}'
tags:
- remove_from_x
- name: Remove from x group
shell: "deluser {{ item.name }} x"
with_items:
- "{{ user_account }}"
when: "'x in is_in_googlesudo' and 'google-sudo in is_in_googlesudo'"
tags:
- remove_from_x
I was testing json_query to extract user name if he is in google-sudo and x group, but without success. Tried to list users when group is defined, however I get empty output, using this:
msg='{{ user_account |json_query("[?groups==`google-sudo`]")}}'
I wonder if is there any shorter way to remove users from group x after checking on server (using eg. groups command) or in user_account -> group if he's already in google-sudo.
Probably there is some nice and elegant way to write this code, I will appreciate any ideas how to deal with it.
Let's simplify the data, e.g.
vars:
my_group: "{{ googlesudo.stat.exists|ternary('google-sudo', 'sudo') }}"
user_account:
- name: jenny
groups: "{{ my_group }}"
- name: jerry
groups: "{{ my_group }}"
The code works as expected, e.g.
- stat:
path: /tmp/google_sudo
register: googlesudo
- debug:
var: user_account
- debug:
msg: '{{ user_account|json_query("[?groups==`google-sudo`].name") }}'
gives if /tmp/google_sudo exists
user_account:
- groups: google-sudo
name: jenny
- groups: google-sudo
name: jerry
msg:
- jenny
- jerry
otherwise
user_account:
- groups: sudo
name: jenny
- groups: sudo
name: jerry
msg: []
Q: "Remove user from group x if already in another group."
A: Use getent to get the list of users in a group. For example, to list users in the group sudo
- getent:
database: group
- debug:
var: getent_group.sudo.2
Test action if a user is a member of the group sudo, e.g.
- debug:
msg: "Remove user {{ item.name }} from group x"
loop: "{{ user_account }}"
when: item.name in getent_group.sudo.2
Q: "Have groups as a list. (Probably I should use contains function.)"
A: Yes. Use contains, e.g given the lists
vars:
user_account:
- name: jenny
groups: [google-sudo, group2]
- name: jerry
groups: [google-sudo, group3]
the task
- debug:
msg: "{{ user_account|json_query('[?contains(groups, `group2`)].name') }}"
gives
msg:
- jenny

Ansible 2.5.2, AWS Lambda, Want to create a template that works whether or not I have subnets or security groups assigned

I'm using Ansible 2.5.2 to try and automate deployment of Lambda into AWS.
How do I create template so that if the security groups section is blank, the code deploys?
Below results in the error EC2 Error Message: The subnet ID '' does not exist
---
- name: Deploy and Update Lambda
hosts: localhost
gather_facts: no
connection: local
tasks:
- name: Lambda Deploy
lambda:
profile: "{{ profile }}"
name: '{{ item.name }}'
state: present #absent or present
zip_file: '{{ item.zip_file }}'
runtime: 'python2.7'
role: '{{ item.role }}'
handler: 'hello_python.my_handler'
vpc_subnet_ids: '{{ item.vpc_subnet_ids }}'
vpc_security_group_ids: '{{ item.vpc_security_group_ids }}'
environment_variables: '{{ item.env_vars }}'
tags: "{{ item.tags }}"
with_items:
- name: AnsibleTest
role: 'arn:aws:iam::xxxxxxxxxx:role/Dev-LambdaRole'
zip_file: hello-code.zip
vpc_subnet_ids:
# - subnet-080802e6660be744c
# - subnet-00a8380a28ae0528c
# - subnet-0723ad3c29a435ee0
vpc_security_group_ids:
- sg-0fa788da8ecd36fe5
env_vars:
key1: "first"
key2: "second"
tags:
x: "133"
xx: "1"
project-name: "x"
xxx: "Ansible"
app-function: "automation"
Name: "AnsibleTest"