How to concatenate two lists of dictonaries in Ansible - list

I have two list of dicts:
dev_users:
- name: cs3141
key:
cs513e_key1.pub
cs513e_key2.pub
- name: ab1234
key:
ab1234.pub
- name: cd5678
key:
ab1234.pub
and
sys_admin_users:
- name: xy3141
key:
xy3141.pub
- name: cd1234
key:
cd1234.pub
- name: ef5678
key:
ef5678.pub
When I try to concatenate them:
- set_fact: users= "{{ dev_users + sys_admin_users }}"
I get this error:
ERROR! failed to combine variables, expected dicts but got a 'dict' and a 'AnsibleSequence':
{}
[{"set_fact": "users= \"{{ dev_users + sys_admin_users }}\""}]
How can I concatenate these two lists?

The problem was that I was trying to concatenate the two list in the variables section, as opposed to the task section. Tadeboro in #ansible gave me this code that worked:
---
- hosts: localhost
gather_facts: false
vars:
dev_users:
- name: cs3141
key:
cs513e_key1.pub
cs513e_key2.pub
- name: ab1234
key:
ab1234.pub
- name: cd5678
key:
ab1234.pub
sys_admin_users:
- name: xy3141
key:
xy3141.pub
- name: cd1234
key:
cd1234.pub
- name: ef5678
key:
ef5678.pub
tasks:
- name: Test
set_fact:
users: "{{ dev_users + sys_admin_users }}"
This command works fine: ansible-playbook -v x.yaml

Here is another solution, that iteratively concatenates a list of list. I like it better, because it is more general. It only works in more recent versions of ansible, because there was a bug in version 2.5 which took some time to fix.
---
# ansible-playbook -v manyx.yaml
- hosts: localhost
gather_facts: false
vars:
dev_users:
- name: cs3141
key:
cs513e_key1.pub
cs513e_key2.pub
- name: ab1234
key:
ab1234.pub
- name: cd5678
key:
ab1234.pub
sys_admin_users:
- name: xy3141
key:
xy3141.pub
- name: cd1234
key:
cd1234.pub
- name: ef5678
key:
ef5678.pub
other_users:
- name: fe9876
key:
fe9876.pub
list_of_users_list:
- "{{ dev_users }}"
- "{{ sys_admin_users }}"
- "{{ other_users }}"
all_users: []
tasks:
- name: Test
set_fact:
all_users: "{{ item + all_users }}"
loop: "{{ list_of_users_list }}"
- name: print all_users
debug:
msg: "{{ item }}"
loop: "{{ all_users }}"

Related

ansible - need output in csv in multiple columns

I have playbook as below:
tasks:
- name: To Get list of all ACM
aws_acm_info:
region: "{{ region }}"
register: acm
- name: Cname Names
set_fact:
cname: "{{ acm | json_query(jmesquery) }}"
vars:
jmesquery: 'certificates[*].domain_validation_options[].resource_record.name'
- name: Cname Values
set_fact:
value: "{{ acm | json_query(jmesquery) }}"
vars:
jmesquery: 'certificates[*].domain_validation_options[].resource_record.value'
- name: set file header
shell: echo 'cname, value'> {{ path }}
run_once: true
- name: CSV - Write information into .csv file
lineinfile:
insertafter: ','
dest: "{{ path }}"
line: "{{ item }}"
with_items:
- "{{ cname }}"
- "{{ value }}"
I am getting output in single column as cname, but I need values in 2nd column as value.
required output format world be
enter image description here
I really appreciate any help you can provide.
Given the data below for testing
acm:
certificates:
- domain_validation_options:
- resource_record: {name: aaa, value: 1}
- domain_validation_options:
- resource_record: {name: bbb, value: 2}
- domain_validation_options:
- resource_record: {name: ccc, value: 3}
Get the name/value pairs in a single query
_query: 'certificates[*].domain_validation_options[].[resource_record.name,
resource_record.value]'
csv: "{{ acm|json_query(_query)|map('join', ',')|list }}"
give
csv:
- aaa,1
- bbb,2
- ccc,3
You can write the lines into a file
- lineinfile:
dest: /tmp/cname.csv
line: "{{ item }}"
loop: "{{ csv }}"
gives
shell> cat /tmp/cname.csv
aaa,1
bbb,2
ccc,3
Example of a complete playbook
- hosts: localhost
vars:
acm:
certificates:
- domain_validation_options:
- resource_record: {name: aaa, value: 1}
- domain_validation_options:
- resource_record: {name: bbb, value: 2}
- domain_validation_options:
- resource_record: {name: ccc, value: 3}
_query: 'certificates[*].domain_validation_options[].[resource_record.name,
resource_record.value]'
csv: "{{ acm|json_query(_query)|map('join', ',')|list }}"
tasks:
- lineinfile:
dest: /tmp/cname.csv
line: "{{ item }}"
loop: "{{ csv }}"
Read the file
- hosts: localhost
tasks:
- community.general.read_csv:
path: /tmp/cname.csv
fieldnames: cname,value
delimiter: ','
register: cname
- debug:
var: cname.list
gives
cname.list:
- {cname: aaa, value: '1'}
- {cname: bbb, value: '2'}
- {cname: ccc, value: '3'}
You can see that the values are strings now. If you want to keep values as integers store the data in YAML format
- hosts: localhost
vars:
acm:
certificates:
- domain_validation_options:
- resource_record: {name: aaa, value: 1}
- domain_validation_options:
- resource_record: {name: bbb, value: 2}
- domain_validation_options:
- resource_record: {name: ccc, value: 3}
_query: 'certificates[*].domain_validation_options[].[resource_record.name,
resource_record.value]'
csv: "{{ acm|json_query(_query)|map('join', ': ')|list }}"
tasks:
- lineinfile:
dest: /tmp/cname.yml
line: "{{ item }}"
loop: "{{ csv }}"
gives
shell> cat /tmp/cname.yml
aaa: 1
bbb: 2
ccc: 3
Read the file
- hosts: localhost
tasks:
- include_vars:
file: /tmp/cname.yml
name: cname_dict
- debug:
var: cname_dict
gives
cname_dict:
aaa: 1
bbb: 2
ccc: 3

Convert a list to dictionary - Ansible YAML

I have a playbook, where I get an error message
fatal: [localhost]: FAILED! => {"ansible_facts": {"tasks": {}}, "ansible_included_var_files": [], "changed": false, "message": "/home/user/invoke_api/automation/tmp/task.yml must be stored as a dictionary/hash"}
task.yml
The file task.yml is dynamically created, and filtered all the time from another source to give the output below.
- key: gTest101
value:
Comments: FWP - Testing this
IP: 10.1.2.3
Name: gTest101
- key: gTest102
value:
Comments: FWP - Applying this
IP: 10.1.2.4
Name: gTest102
Question: How do I convert the list in my task.yml to a dictionary? What's the code to convert from list to dictionary
playbook.yml
---
- name: Global Objects
hosts: check_point
connection: httpapi
gather_facts: False
vars_files:
- 'credentials/my_var.yml'
- 'credentials/login.yml'
tasks:
- name: read-new-tmp-file
include_vars:
file: tmp/task.yml
name: tasks
register: new_host
- name: add-host-object-to-group
check_point.mgmt.cp_mgmt_host:
name: "{{ item.value.Name | quote }}"
ip_address: "{{ item.value.IP | quote }}"
comments: "{{ item.value.Comments }}"
groups: gTest1A
state: present
auto_publish_session: yes
loop: "{{ new_host.dict | dict2items }}"
delegate_to: Global
ignore_errors: yes
Ansible core 2.9.13
python version = 2.7.17
Q: "How do I convert the list in my task.yml to a dictionary?"
A: Use items2dict. For example, read the file and create the list
- set_fact:
l1: "{{ lookup('file', 'task.yml')|from_yaml }}"
gives
l1:
- key: gTest101
value:
Comments: FWP - Testing this
IP: 10.1.2.3
Name: gTest101
- key: gTest102
value:
Comments: FWP - Applying this
IP: 10.1.2.4
Name: gTest102
Then, the task below
- set_fact:
d1: "{{ l1|items2dict }}"
creates the dictionary
d1:
gTest101:
Comments: FWP - Testing this
IP: 10.1.2.3
Name: gTest101
gTest102:
Comments: FWP - Applying this
IP: 10.1.2.4
Name: gTest102
A vars file is a yaml dict file, so your list have to be a fields of a vars:
my_vars:
- Comments: FWP - Testing this
IP: 10.1.2.3
Name: gTest101
- Comments: FWP - Applying this
IP: 10.1.2.4
Name: gTest102
and you dont need to register the include_vars tasks. Just loop on the list var name (here my_vars).
---
- name: Global Objects
hosts: check_point
connection: httpapi
gather_facts: False
vars_files:
- 'credentials/my_var.yml'
- 'credentials/login.yml'
tasks:
- name: read-new-tmp-file
include_vars:
file: tmp/task.yml
- name: add-host-object-to-group
check_point.mgmt.cp_mgmt_host:
name: "{{ item.Name }}"
ip_address: "{{ item.IP }}"
comments: "{{ item.Comments }}"
groups: gTest1A
state: present
auto_publish_session: yes
loop: "{{ my_vars }}"
delegate_to: Global
ignore_errors: yes

Ansible - remove user from group x if already in other group

I am looking for the easiest way to remove users from group x when they are already in group google-sudo. I store users in group vars in this kind of list and dictionary combination:
user_account:
- name: jenny
authorized_keys:
- jenny_01
groups:
- "{% if not googlesudo.stat.exists %}sudo{% else %}google-sudo{% endif %}"
- name: jerry
authorized_keys:
- jerry_01
groups:
- "{% if not googlesudo.stat.exists %}sudo{% else %}google-sudo{% endif %}"
...
These are tasks I already created:
- name: Check if google-sudo file exists
stat:
path: /etc/sudoers.d/google_sudo
register: googlesudo
tags:
- add_user_group
- remove_from_x
- debug: var=googlesudo verbosity=2
tags:
- add_user_group
- remove_from_x
- debug:
msg: "User account to create: {{ item.name }}"
with_items: "{{ user_account }}"
- name: "Creating user"
user:
name: "{{ item.name }}"
group: users
shell: /bin/bash
with_items: "{{ user_account }}"
- name: Add user to additional groups
user:
name: "{{ item.0.name }}"
groups: "{{ item.1 }}"
append: yes
with_subelements:
- "{{ user_account }}"
- groups
tags:
- remove_from_x
- name: Check if user already in google-sudo
command: "groups {{ item.name }}"
with_items:
- "{{ user_account }}"
register: root_users
tags:
- remove_from_x
- name: View root users
debug:
msg: "{{ item }}"
verbosity: 2
with_items:
- "{{ root_users }}"
tags:
- remove_from_x
- name: Save state
set_fact:
is_in_googlesudo: "{{ root_users.results.0.stdout_lines }}"
tags:
- remove_from_x
- name: List
debug: msg='{{ user_account |json_query("[?groups==`google-sudo`]")}}'
tags:
- remove_from_x
- name: Remove from x group
shell: "deluser {{ item.name }} x"
with_items:
- "{{ user_account }}"
when: "'x in is_in_googlesudo' and 'google-sudo in is_in_googlesudo'"
tags:
- remove_from_x
I was testing json_query to extract user name if he is in google-sudo and x group, but without success. Tried to list users when group is defined, however I get empty output, using this:
msg='{{ user_account |json_query("[?groups==`google-sudo`]")}}'
I wonder if is there any shorter way to remove users from group x after checking on server (using eg. groups command) or in user_account -> group if he's already in google-sudo.
Probably there is some nice and elegant way to write this code, I will appreciate any ideas how to deal with it.
Let's simplify the data, e.g.
vars:
my_group: "{{ googlesudo.stat.exists|ternary('google-sudo', 'sudo') }}"
user_account:
- name: jenny
groups: "{{ my_group }}"
- name: jerry
groups: "{{ my_group }}"
The code works as expected, e.g.
- stat:
path: /tmp/google_sudo
register: googlesudo
- debug:
var: user_account
- debug:
msg: '{{ user_account|json_query("[?groups==`google-sudo`].name") }}'
gives if /tmp/google_sudo exists
user_account:
- groups: google-sudo
name: jenny
- groups: google-sudo
name: jerry
msg:
- jenny
- jerry
otherwise
user_account:
- groups: sudo
name: jenny
- groups: sudo
name: jerry
msg: []
Q: "Remove user from group x if already in another group."
A: Use getent to get the list of users in a group. For example, to list users in the group sudo
- getent:
database: group
- debug:
var: getent_group.sudo.2
Test action if a user is a member of the group sudo, e.g.
- debug:
msg: "Remove user {{ item.name }} from group x"
loop: "{{ user_account }}"
when: item.name in getent_group.sudo.2
Q: "Have groups as a list. (Probably I should use contains function.)"
A: Yes. Use contains, e.g given the lists
vars:
user_account:
- name: jenny
groups: [google-sudo, group2]
- name: jerry
groups: [google-sudo, group3]
the task
- debug:
msg: "{{ user_account|json_query('[?contains(groups, `group2`)].name') }}"
gives
msg:
- jenny

Convert nested list of dicts to dict in Ansible

I'm trying to convert a list of dictionary in dictionary. My defined vars:
users:
- name: bruce
age: 22
params:
- key: "k1"
value: "v1bruce"
- key: "k2"
value: "v2bruce"
- name: alan
age: 25
params:
- key: "k1"
value: "v1alan"
- key: "k5"
value: "v5alan"
- name: carl
age: 43
params:
- key: "k1"
value: "v1carl"
- key: "k2"
value: "v2carl"
- key: "k5"
value: "v5carl"
I need to create a new structure in Ansible starting from the previous converting params to dictionaries like the following:
employee:
- name: bruce
age: 22
params:
"k1": "v1bruce"
"k2": "v2bruce"
- name: alan
age: 25
params:
"k1": "v1alan"
"k5": "v5alan"
- name: carl
age: 43
params:
"k1": "v1carl"
"k2": "v2carl"
"k5": "v5carl"
The key point is that I don't know how many elements my starting list is composed.
Is it possible?
Thanks.
Use items2dict in loop. For example
- set_fact:
employee: "{{ employee|default([]) +
[{'name': item.name,
'age': item.age,
'params': item.params|items2dict}] }}"
loop: "{{ users }}"
Warning: ugly solution ahead.
The base to understand is that your params looks like a transformation of a dict via the dict2items filter. So we have to reverse the operation with items2dict. But since it is nested inside your list of hash, it is not so easy. This is what I came up with:
---
- hosts: localhost
gather_facts: False
vars:
users:
- name: bruce
age: 22
params:
- key: "k1"
value: "v1bruce"
- key: "k2"
value: "v2bruce"
- name: alan
age: 25
params:
- key: "k1"
value: "v1alan"
- key: "k5"
value: "v5alan"
- name: carl
age: 43
params:
- key: "k1"
value: "v1carl"
- key: "k2"
value: "v2carl"
- key: "k5"
value: "v5carl"
tasks:
- name: Show raw
debug:
var: users
- name: Transform
set_fact:
user_transformed: >-
{%- set result = [] -%}
{%- for user in users -%}
{{
result.append({
'name': user.name,
'age': user.age,
'params': user.params | items2dict
})
}}
{%- endfor -%}
{{ result | to_json | from_json }}
- name: Show transformed
debug:
var: user_transformed

Iterating in a with_items loop ansible

Hi there i am atm trying to get the hang of ansible with aws and i am really likeing the flexibility of it so far.
Sadly now i am starting to hit a brick wall with my experiments. I am trying to tag volumes for specific instances that have the tag environment: test with the tages environment: test and backup: true . The playbook is working as intended if i specify every single Index of the array in the with_items loop. Here's my playbook so far:
---
- name: Tag the EBS Volumes
hosts: tag_environment_test
gather_facts: False
tags: tag
vars_files:
- /etc/ansible/vars/aws.yml
tasks:
- name: Gather instance instance_ids
local_action:
module: ec2_remote_facts
region: '{{ aws_region }}'
filters:
instance-state-name: running
"tag:environment": test
register: test_id
- name: Gather volume information for instance
local_action:
module: ec2_vol
region: '{{ aws_region }}'
instance: "{{ item.id }}"
state: list
with_items:
- "{{ test_id.instances }}"
register: ec2_volumes
- debug:
var: ec2_volumes
- name: Do some actual tagging
local_action:
module: ec2_tag
region: '{{ aws_region }}'
resource: "{{ item.id }}"
args:
tags:
environment: test
backup: true
with_items:
- "{{ ec2_volumes.results[0].volumes }}"
# - "{{ ec2_volumes.results[1].volumes }}"
My question now is it possible to iterate over the full array in ec2_volumes.results without having the need to specify every single value in the array. Like for example _ec2_volumes.results[X].volumes X=X+1_ so every time he goes through the loop he iterates +1 until the end of the array.
Every Input also on the rest of the playbook would be very appriciated (like i said still trying to get the hang of ansible. :)
Greeting
Drees
You can iterate over your list of results:
- name: Do some actual tagging
delegate_to: localhost
ec2_tag:
region: '{{ aws_region }}'
resource: "{{ item.volume.id }}"
tags:
environment: test
backup: true
with_items: "{{ ec2_volumes.results }}"
Every Input also on the rest of the playbook would be very appreciated
Consider using delegate_to: localhost instead of local_action. Consider this task:
- name: an example
command: do something
Using delegate_to, I only need to add a single line:
- name: an example
delegate_to: localhost
command: do something
Whereas using local_action I need to rewrite the task:
- name: an example
local_action:
module: command do something
And of course, delegate_to is more flexible: you can use it to delegate to hosts other than localhost.
Update
Without seeing your actual playbook it's hard to identify the source of the error. Here's a complete playbook that runs successfully (using synthesized data and wrapping your ec2_tag task in a debug task):
---
- hosts: localhost
gather_facts: false
vars:
aws_region: example
ec2_volumes:
results:
- volume:
id: 1
- volume:
id: 2
tasks:
- name: Do some actual tagging
debug:
msg:
ec2_tag:
region: '{{ aws_region }}'
resource: '{{ item.volume.id }}'
tags:
environment: test
backup: true
with_items: "{{ ec2_volumes.results }}"