I have a task to install monitoring agent in multiple hosts.
we are using three templates(yml files) during this installations. these template contain multiple tags. such as app name, teamdl, server IP, datacenter and so on...
for bulk installation we are using csv file which contains value for each tag.
Ask is to use each item in the csv file and use them as variable for all three templates.
I am able to read these items, but not able to use them. please support.
main.yml
- name: copy the csv file
copy:
src: "/u00/ansible/Playbooks/test.csv". # present in ansible controller
dest: "/u00/app/monitor/test.csv" #target server
become: yes
become_user: root
vars:
contents: "{{ lookup('file', '/u00/app/monitor/test.csv').split('\n') }}"
- name: Update config yml files
template: src={{item.src}} dest={{item.dest}}
loop:
- { src: '/u00/ansible/Playbooks/files/infra.yml_template', dest: '/u00/app/monitor/infra.yml' }
infra.yml_template -
custom_attributes:
application : {{ item.Application }}
env : {{ item.env }}
datacenter : {{ item.Datacenter }}
log:
file: /u00/app/monitor/infra.log
format: text
level: smart
forward: false
stdout: false
smart_level_entry_limit: 500
rotate:
max_size_mb: 100
max_files: 5
compression_enabled: true
file_pattern: rotated.YYYY-MM-DD_hh-mm-ss.log
error -
TASK [infra-integration : copy the csv file] ********************************
[0;32mok: [testserver1][0m
[0;32mok: [Testserver2][0m
TASK [Infra-integration : Update config yml files] **************************
[0;31mAn exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible.errors.AnsibleUndefinedVariable: 'dict object' has no attribute 'Application'[0m
[0;31mfailed: [testserver1] (item={'src': '/u00/ansible/Playbooks/files/infra.yml_template', 'dest': '/u00/app/monitor/infra.yml'}) => {"ansible_loop_var": "item", "changed": false, "item": {"dest": "/u00/app/monitor/infra.yml", "src": "/u00/ansible/Playbooks/files/infra.yml_template"}, "msg": "AnsibleUndefinedVariable: 'dict object' has no attribute 'Application'"}[0m
Expectation is to read csv file and use variables in three different template located at three different location.
Related
I have roughly formatted yml files with key/value pairs in them. I then imported the values of both of these files successfully into a running playbook using the include_vars module.
Now, I want to be able to compare the value of the key/value pair from file/list 1, to all of the keys of file/list 2. Then finally when there is a match, print and preferably save/register the value of the matching key from file/list 2.
Essentially I am comparing a machine name to an IP list to try to grab the IP the machine needs out of that list. The name is "dynamic" and is different each time the playbook is run, as file/list 1 is always dynamically populated on each run.
Examples:
file/list 1 contents
machine_serial: m60
s_iteration: a
site_name: dud
t_number: '001'
file/list 2 contents
m51: 10.2.5.201
m52: 10.2.5.202
m53: 10.2.5.203
m54: 10.2.5.204
m55: 10.2.5.205
m56: 10.2.5.206
m57: 10.2.5.207
m58: 10.2.5.208
m59: 10.2.5.209
m60: 10.2.5.210
m61: 10.2.5.211
In a nutshell, I want to be able to get the file/list 1 ct_machine_serial key who's value is currently: m60 to be able to find it's key match in file/list 2, and then print and/or preferably register it's value of 10.2.5.210.
What I've tried so far:
Playbook:
- name: IP gleaning comparison.
hosts: localhost
remote_user: ansible
become: yes
become_method: sudo
vars:
ansible_ssh_pipelining: yes
tasks:
- name: Try to do a variable import of the file1 file.
include_vars:
file: ~/active_ct-scanner-vars.yml
name: ctfile1_vars
become: no
- name: Try to do an import of file2 file for lookup comparison to get an IP match.
include_vars:
file: ~/machine-ip-conversion.yml
name: ip_vars
become: no
- name: Best, but failing attempt to get the value of the match-up IP.
debug:
msg: "{{ item }}"
when: ctfile1_vars.machine_serial == ip_vars
with_items:
- "{{ ip_vars }}"
Every task except the final one works perfectly.
My failed output final task:
TASK [Best, but failing attempt to get the value of the match-up IP.] ***********************************************************************************
skipping: [localhost] => (item={'m51': '10.200.5.201', 'm52': '10.200.5.202', 'm53': '10.200.5.203', 'm54': '10.200.5.204', 'm55': '10.200.5.205', 'm56': '10.200.5.206', 'm57': '10.200.5.207', 'm58': '10.200.5.208', 'm59': '10.200.5.209', 'm60': '10.200.5.210', 'm61': '10.200.5.211'})
skipping: [localhost]
What I hoped for hasn't happened, it simply skips the task, and doesn't iterate over the list like I was hoping, so there must be a problem somewhere. Hopefully there is an easy solution to this I just missed. What could be the correct answer?
Given the files
shell> cat active_ct-scanner-vars.yml
machine_serial: m60
s_iteration: a
site_name: dud
t_number: '001'
shell> cat machine-ip-conversion.yml
m58: 10.2.5.208
m59: 10.2.5.209
m60: 10.2.5.210
m61: 10.2.5.211
Read the files
- include_vars:
file: active_ct-scanner-vars.yml
name: ctfile1_vars
- include_vars:
file: machine-ip-conversion.yml
name: ip_vars
Q: "Compare the machine name to an IP list and grab the IP."
A: Both variables ip_vars and ctfile1_vars are dictionaries. Use ctfile1_vars.machine_serial as index in ip_vars
match_up_IP: "{{ ip_vars[ctfile1_vars.machine_serial] }}"
gives
match_up_IP: 10.2.5.210
Example of a complete playbook for testing
- hosts: localhost
gather_facts: false
vars:
match_up_IP: "{{ ip_vars[ctfile1_vars.machine_serial] }}"
tasks:
- include_vars:
file: active_ct-scanner-vars.yml
name: ctfile1_vars
- include_vars:
file: machine-ip-conversion.yml
name: ip_vars
- debug:
var: match_up_IP
I've been hitting a wall trying to get /etc/exports managed via Ansible.
I've got a role that installs a piece of software on a VM, and I want to then add an entry ot /etc/exports on the NFS server, for that specific VM, so it's able to access the NFS shares needed.
Lineinfile sounds like the way to go, but sofar I can't figure out how to properly write this.
I want this to:
not modify if the host is in the line, no matter where
add the NFS share and the host if there's no line for the NFS share
add the host to the share in case it isn't in there.
The latest installment of my 'add to /etc/exports' that thought should work, but doesn't, is:
- name: Add hosts to mountpoint line
ansible.builtin.lineinfile:
path: /etc/exports
line: '\1 {{ host_ip }}(root_squash,no_subtree_check)'
regex: '^((?!{{ volume_mountpoint }}.*{{ host_ip }}\(root_squash,no_subtree_check\).*).*)$'
backrefs: yes
but i'm still getting all kinds of weird side effects. I've used backreferences etc before, but somehow this one keeps tripping me up.
Anyone who sees what's gong wrong?
Typical /etc/exports entry:
/srv/files 172.16.0.14(rw,no_root_squash,no_subtree_check)
It's not possible in one step to modify a line using backreferences or add the line if missing. To modify the existing mount point the back-references are needed. For example, given the files for testing
shell> cat etc/export1
/srv/files 172.16.0.14(rw,no_root_squash,no_subtree_check)
shell> cat etc/export2
/srv/files 172.16.0.15(rw,no_root_squash,no_subtree_check)
shell> cat etc/export3
/srv/download 172.16.0.14(rw,no_root_squash,no_subtree_check)
the task
tasks:
- lineinfile:
path: "etc/{{ item }}"
regex: '^{{ mount }}(\s+)({{ ipr }})*({{ optionsr }})*(\s*)(.*)$'
line: '{{ mount }}\g<1>{{ ip }}{{ options }} \g<5>'
backrefs: true
vars:
mount: /srv/files
ipr: '172\.16\.0\.14'
ip: '172.16.0.14'
optionsr: '\(.*?\)'
options: '(root_squash,no_subtree_check)'
loop:
- export1
- export2
- export3
gives
--- before: etc/export1 (content)
+++ after: etc/export1 (content)
## -1 +1 ##
-/srv/files 172.16.0.14(rw,no_root_squash,no_subtree_check)
+/srv/files 172.16.0.14(root_squash,no_subtree_check)
changed: [localhost] => (item=export1)
--- before: etc/export2 (content)
+++ after: etc/export2 (content)
## -1 +1 ##
-/srv/files 172.16.0.15(rw,no_root_squash,no_subtree_check)
+/srv/files 172.16.0.14(root_squash,no_subtree_check) 172.16.0.15(rw,no_root_squash,no_subtree_check)
changed: [localhost] => (item=export2)
ok: [localhost] => (item=export3)
The first two files are all right. The problem is the third file. The line hasn't been added to the file. Quoting from backrefs
"... if the regexp does not match anywhere in the file, the file will be left unchanged."
The explanation is simple. There are no groups if the regex doesn't match. If there are no groups the line can't be created.
On the other hand, quoting from regexp
"... If the regular expression is not matched, the line will be added to the file ..."
As a result, it's not possible to ask lineinfile to add a line if the regexp does not match and, at the same time, to do nothing if the regexp is matched. If the regexp is matched you need back-references. If you use back-references you can't add a missing line.
To solve this problem read the content of the files and create a dictionary
- command: "cat etc/{{ item }}"
register: result
loop: [export1, export2, export3]
- set_fact:
content: "{{ dict(_files|zip(_lines)) }}"
vars:
_lines: "{{ result.results|map(attribute='stdout_lines')|list }}"
_files: "{{ result.results|map(attribute='item')|list }}"
gives
content:
export1:
- /srv/files 172.16.0.14(rw,no_root_squash,no_subtree_check)
export2:
- /srv/files 172.16.0.15(rw,no_root_squash,no_subtree_check)
export3:
- /srv/download 172.16.0.14(rw,no_root_squash,no_subtree_check)
Now add the line only if missing, i.e. do not replace the line if the mount point is already there
- lineinfile:
path: "etc/{{ item }}"
line: '{{ mount }} {{ ip }}{{ options }}'
vars:
mount: /srv/files
ip: '172.16.0.14'
options: '(root_squash,no_subtree_check)'
loop: "{{ content|list }}"
when: content[item]|select('search', mount)|length == 0
gives
skipping: [localhost] => (item=export1)
skipping: [localhost] => (item=export2)
--- before: etc/export3 (content)
+++ after: etc/export3 (content)
## -1 +1,2 ##
/srv/download 172.16.0.14(rw,no_root_squash,no_subtree_check)
+/srv/files 172.16.0.14(root_squash,no_subtree_check)
I need to copy the SSH public key from a local file, then use it in a uri task in my playbook.
Keep in mind, I cannot use "authorized_key" module as this is a system I must use the API to configure public keys for users.
Code below keeps failing, I am 100% sure its because of the filter I am using. I am including the commented out section that does work for the body.
Trying to use a lookup with a regex_search, I used [^\s]\s[^\s] which works in python. Also the key is in a different directory in my local host (../../ssh/ssh_key/key.pub)
Any ideas?
- name: copy public key to gitea
hosts: localhost
tasks:
- name: include user to add as variable
include_vars:
file: users.yaml
name: users
- name: Gather users key contents and create variable
# shell: "cat ../keys/ssh_keys/zz123z.pub | awk '{print $1 FS $2}'"
shell: "cat ../keys/ssh_keys/{{item.username}}.pub | awk '{print $1 FS $2}'"
register: key
with_items:
- "{{users.user}}"
- name: Add user's key to gitea
uri:
url: https://10.10.10.10/api/v1/admin/users/{{ item.username }}/keys
headers:
Authorization: "token {{ users.GiteaApiToken }}"
validate_certs: no
return_content: yes
status_code: 201
method: POST
body: "{\"key\": \"{{ key.stdout }}\", \"read_only\": true, \"title\": \"{{ item.username }} shared
body_format: json
with_items:
- "{{users.user}}"
This is the error I receive when using -vvv
TASK [Add user's key to gitea] *************************************************
task path: /home/dave/projects/Infrastructure/ansible/AddTempUsers/addusers.yaml:275
Wednesday 04 March 2020 18:14:29 -0500 (0:00:00.537) 0:00:01.991 *******
fatal: [localhost]: FAILED! => {
"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'stdout'\n\nThe error appears to be in '/home/dave/projects/Infrastructure/ansible/AddTempUsers/addusers.yaml': line 275, column 13, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: Add user's key to gitea\n ^ here\n"
}
I FIGURED IT OUT!
used shell with an awk command to gather the keys. (Note: including an awk for RSA keys, and one for id_ed25519, which we use. RSA is commented out but others can comment if they wish to use.)
Used loop control to iterate through the results.
Code below:
- name: copy public key to gitea
hosts: localhost
tasks:
- name: include user to add as variable
include_vars:
file: users.yaml
name: users
- name: Gather users key contents and create variable
# For RSA Keys
# shell: "cat ../keys/ssh_keys/{{item.username}}.pub | awk '/-END PUBLIC KEY-/ { p = 0 }; p; /-BEGIN PUBLIC KEY-/ { p = 1 }'
# For id_ed5519 Keys
shell: "cat ../keys/ssh_keys/{{item.username}}.pub | awk '{print $1 FS $2}'"
register: key
with_items:
- "{{users.user}}"
- name: Add user's key to gitea
uri:
url: https://10.10.10.10/api/v1/admin/users/{{ item.username }}/keys
headers:
Authorization: "token {{ users.GiteaApiToken }}"
validate_certs: no
return_content: yes
status_code: 201
method: POST
body: "{\"key\": \"{{ key.results[ndx].stdout }}\", \"read_only\": true, \"title\": \"{{ item.username }} shared VM\"}"
body_format: json
with_items:
- "{{users.user}}"
loop_control:
index_var: ndx
I wrote a task that is responsible for changing supervisor config file. The case is that on some servers we have more than one app running workers, so sometimes more than one path needs to be added to include section of supervisor.conf.
Currently I wrote this task in /roles/supervisor/tasks/main.yml/:
- name: Add apps paths in include section
lineinfile:
dest: /etc/supervisor/supervisord.conf
regex: '^files ='
line: 'files = /etc/supervisor/conf.d/*.conf /home/app/{{ app_name }}/releases/app/shared/supervisor/*.conf /home/dev/{{ app_name2 }}/releases/dev/shared/supervisor/*.conf'
when: ansible_hostname = 'ser-db-10'
notify: restart supervisor
tags: multi_workers
... and added in /roles/supervisor/defaults/main.yml/ this:
app_name: bla
app_name2: blabla
It works, but I don't like the thing that there are two application paths hardcoded in line and maybe I should also add variable in place of ser-db-10.
I am wondering how to rebuild this task to make it more independent.
What I mean is, if there are 4 apps, add 4 paths, if there are 2 apps, add 2 paths.
What is the most efficient way to do this?
As an example of how to put together the parameter line, the play below
- hosts: test_01
vars:
app_name1: A
app_name2: B
my_conf:
test_01:
lines:
- '/etc/*.conf'
- '/etc/{{ app_name1 }}/*.conf'
- '/etc/{{ app_name2 }}/*.conf'
tasks:
- debug:
msg: "files = {{ my_conf[inventory_hostname].lines|join(' ') }}"
gives
"msg": "files = /etc/*.conf /etc/A/*.conf /etc/B/*.conf"
With appropriate dictionary my_conf the task below should do the job
- name: Add apps paths in include section
lineinfile:
dest: /etc/supervisor/supervisord.conf
regex: '^files ='
line: "files = {{ my_conf[inventory_hostname].lines|join(' ') }}"
notify: restart supervisor
tags: multi_workers
(not tested)
I have a users.ini file having below content:
[integration]
# My integration information
user=gertrude
pass=anotherpassword
I am trying to fetch the value in my below yml file using lookup plugin for INI:
- hosts: "{{vnf_ip}}"
connection: local
tasks:
debug: msg="User in integration is {{ lookup('ini', 'user section=integration file=users.ini') }}"
But I am getting output as
TASK [debug] ***********************************************************************************************************************************
ok: [10.10.10.10] => {
"msg": "User in integration is ['gertrude']"
}
Instead of ['gertrude'] it should simply be gertrude.
How to get gertrude simply????
What Ansible version do you use? On modern 2.3.2 it works as expected and returns just gertrude.
If you can't upgrade, you can use first filter to get an element from your resulting list:
{{ lookup('ini', 'user section=integration file=users.ini') | first }}