ansible parse text string from stdout - regex

My problem is with ansible and parsing stdout. I need to capture the stdout from an ansible play and parse this output for a specific substring within stdout and save into a var. My specific use case is below
- shell: "vault.sh --keystore EAP_HOME/vault/vault.keystore |
--keystore-password vault22 --alias vault --vault-block |
vb --attribute password --sec-attr 0penS3sam3 --enc-dir |
EAP_HOME/vault/ --iteration 120 --salt 1234abcd"
register: results
become: true
This generates an output with the following line, the goal is to capture the masked key that jboss vault generates and save that in an ansible var so I can use it to configure the standalone.xml template:
vault-option name="KEYSTORE_PASSWORD" value="MASK-5dOaAVafCSd"/>
I need a way parse this string with possibly regex and save the "MASK-5dOaAVafCSd" substring into an ansible var using set_facts module or any other ansible module.
Currently my code looks like this
#example stdout
results: vault-option name=\"KEYSTORE_PASSWORD\" value=\"MASK-5dOaAVafCSd\"/>
- name: JBOSS_VAULT:define keystore password masked value variable
set_fact:
masked_value: |
"{{ results.stdout |
regex_replace('^.+(MASK-.+?)\\.+','\\\1') }}"
This code is defining masked_value as the results.stdout, not the expected capture group.

You are very close. I advice you to use regex101.com to test regular expressions.
Here is my solution:
---
- hosts: localhost
gather_facts: no
tasks:
- shell: echo 'vault-option name="KEYSTORE_PASSWORD" value="MASK-5dOaAVafCSd"'
register: results
- set_fact:
myvalue: "{{ results.stdout | regex_search(regexp,'\\1') }}"
vars:
regexp: 'value=\"([^"]+)'
- debug:
var: myvalue
result:
ok: [localhost] => {
"myvalue": [
"MASK-5dOaAVafCSd"
]
}
Update:
regex_search returns a list of found matches, so to get only first one use:
{{ results.stdout | regex_search(regexp,'\\1') | first }}

The above solution worked for me, however I had to do some extra logic to filter shell command output to get to the line which contains following
<vault-option name="KEYSTORE_PASSWORD" value="MASK-6qcNdkIprlA"/>
because vault command output has many lines in it. Once this line is captured, the solution given by Konstantin works just fine. Below is the whole thing that needs to done in one place.
- name: Creating jboss vault
shell: |
{{ baseDir }}/bin/vault.sh -e {{ vaultDir }} -k {{ keystoreURL }} -p {{ keystorePassword }} \
-s {{ keystoreSalt }} -i {{ iterationCount }} -v {{ keystoreAlias }} -b {{ vaultBlock }} \
-a {{ attributeName }} -x {{ attributeValue }}
register: vaultResult
- set_fact:
jbossKeystorePassword: "{{ item | regex_search('value=\"([^\"]+)','\\1') | first }}"
when: item | trim | match('.*KEYSTORE_PASSWORD.*')
with_items:
- "{{ vaultResult.stdout_lines }}"
- debug:
var: jbossKeystorePassword
Be sure to replace all variables with your values in above vault.sh command.

Related

Convert Ansible variable to integer for arithmetic

I would like to execute a command to obtain the current AWS EC2 launch template version in an integer format so I can do basic arithmetic on it to use in subsequent queries / deletes.
For example:
tasks:
- name: Lookup current default version of EC2 launch template
command: aws ec2 describe-launch-template-versions --launch-template-id lt-xxx --filters Name=is-default-version,Value=true --query 'LaunchTemplateVersions[*].[VersionNumber]'
delegate_to: localhost
register: result
- name: Show results
debug:
msg: '{{ result.stdout }}'
delegate_to: localhost
If this output is '5' I would like to subtract 1 from it so I can execute an additional command to do the following:
aws ec2 delete-launch-template-versions --launch-template-id lt-xxx --versions {{result - 1}}
I realize this will not work as written, but this is the premise I'm going for.
Convert the string to an integer. For example
- command: echo 5
register: result
- command: "echo {{ result.stdout|int - 1 }}"
register: result
- debug:
var: result.stdout
gives
result.stdout: '4'
The type of the command return values' attribute stdout is string. See the results below
- command: echo 5
register: result
- debug:
msg: |-
result.stdout: {{ result.stdout }}
result.stdout|type_debug: {{ result.stdout|type_debug }}
result.stdout|int|type_debug: {{ result.stdout|int|type_debug }}
msg: |-
result.stdout: 5
result.stdout|type_debug: AnsibleUnsafeText
result.stdout|int|type_debug: int

Remove a given hostname+URL from a line containing 3, separated by commas, in any position, using Ansible playbook

Scenario: I have a configuration file for etcd, and one of the nodes in the cluster has failed. I know the name of the failed node, but not its IP address nor the names of the other two hosts in the cluster. I need to write an Ansible play to remove the failed node from a line in the etcd config file, (presumably) using the Ansible builtin replace which (I believe) uses Python as its RE engine.
I have managed to create something that works, with one caveat: If the failed host is the third one listed, the RE leaves a dangling comma at the end of the line. I'm hoping that someone smarter than I am can edit or replace my regex to cover all three positional cases.
The hostname of the failed node is passed into the playbook as a variable, so {{ failed_node }} would be substituted for the actual hostname of the failed node, let's call it app-failedhost-eeeeeeeeee.node.consul in my example.
Given a regex
((?:^ETCD_INITIAL_CLUSTER=)(?:[a-z0-9-.]{15,}=https:\/\/[0-9]+(?:\.[0-9]+){3}:2380,?){0,2})(,?{{ failed_node }}=https:\/\/[0-9]+(?:[.][0-9]+){3}:2380,?)((?:,?[a-z0-9-.]{15,}=https:\/\/[0-9]+(?:\.[0-9]+){3}:2380,?){0,2})
which when being actually run would be (if failed_node=app-failedhost-eeeeeeeeee.node.consul)
((?:^ETCD_INITIAL_CLUSTER=)(?:[a-z0-9-.]{15,}=https:\/\/[0-9]+(?:\.[0-9]+){3}:2380,?){0,2})(,?app-failedhost-eeeeeeeeee.node.consul=https:\/\/[0-9]+(?:[.][0-9]+){3}:2380,?)((?:,?[a-z0-9-.]{15,}=https:\/\/[0-9]+(?:\.[0-9]+){3}:2380,?){0,2})
if run against one of these lines,
ETCD_INITIAL_CLUSTER=app-failedhost-eeeeeeeeee.node.consul=https://192.168.18.39:2380,app-instance-de24a5c1aefb.node.consul=https://192.168.18.92:2380,app-instance-6cc297ab3cc.node.consul=https://192.168.18.11:2380
ETCD_INITIAL_CLUSTER=app-instance-de24a5c1aefb.node.consul=https://192.168.18.92:2380,app-failedhost-eeeeeeeeee.node.consul=https://192.168.18.39:2380,app-instance-6cc297ab3cc.node.consul=https://192.168.18.11:2380
ETCD_INITIAL_CLUSTER=app-instance-de24a5c1aefb.node.consul=https://192.168.18.92:2380,app-instance-6cc297ab3cc.node.consul=https://192.168.18.11:2380,app-failedhost-eeeeeeeeee.node.consul=https://192.168.18.39:2380
(which if you simplify, is ETCD_INITIAL_CLUSTER= followed by three pairs of values, comma-separated, FQDN=https://[IP address]:2380 with the failed node in position 0, 1, or 2)
and the replace: is '\1\3', you get
ETCD_INITIAL_CLUSTER=app-instance-de24a5c1aefb.node.consul=https://192.168.18.92:2380,app-instance-6cc297ab3cc.node.consul=https://192.168.18.11:2380
ETCD_INITIAL_CLUSTER=app-instance-de24a5c1aefb.node.consul=https://192.168.18.92:2380,app-instance-6cc297ab3cc.node.consul=https://192.168.18.11:2380
ETCD_INITIAL_CLUSTER=app-instance-de24a5c1aefb.node.consul=https://192.168.18.92:2380,app-instance-6cc297ab3cc.node.consul=https://192.168.18.11:2380,
That's correct for the first two cases (failed node in first or second position) but if the failed node is in the third (last) position as in the third example line, then the final comma is left behind.
https://regex101.com/r/f635Wv/1 has the same examples as above.
Playbook, in case the full situation is not clear from the regex above, called node-cleanup.yaml, is called with ansible-playbook node-cleanup.yaml --extra-vars "failed_node=app-failedhost-eeeeeeeeee.node.consul" in the above examples:
---
- name: Clean up failed etcd node
hosts: etcd
become: true
tasks:
- name: Remove failed host from ETCD_INITIAL_CLUSTER line
replace:
path: "/etc/etcd/etcd.conf"
regexp: '((?:^ETCD_INITIAL_CLUSTER=)(?:[a-z0-9-.]{15,}=https:\/\/[0-9]+(?:\.[0-9]+){3}:2380,?){0,2})(,?{{ failed_node }}=https:\/\/[0-9]+(?:[.][0-9]+){3}:2380,?)((?:,?[a-z0-9-.]{15,}=https:\/\/[0-9]+(?:\.[0-9]+){3}:2380,?){0,2})'
replace: '\1\3'
but I think that part is fine, I just need some help with that beast of a regex.
If the line in the file before is simplified as
ETCD_INITIAL_CLUSTER=host1=IP,host2=IP,host3=IP
and I pass in “host3” for {{ failed_node }}, then I want
ETCD_INITIAL_CLUSTER=host1=IP,host2=IP
to come out, but what I actually get is
ETCD_INITIAL_CLUSTER=host1=IP,host2=IP,
(note the trailing comma)
Given the file
shell> cat test.conf
ETCD_INITIAL_CLUSTER=host1=IP,host2=IP,host3=IP
and the variable
failed_node: host3
Get the line from the configuration file. There are many options depending on the file is local or remote, e.g.
- shell: cat test.conf | grep ETCD_INITIAL_CLUSTER
register: result
check_mode: false
- set_fact:
eic: "{{ result.stdout }}"
gives
eic: ETCD_INITIAL_CLUSTER=host1=IP,host2=IP,host3=IP
Split the key/value pair and create a new value by rejecting the failed node
- set_fact:
_value: "{{ eic|regex_replace('^(.*?)=(.*)$', '\\2') }}"
_key: "{{ eic|regex_replace('^(.*?)=(.*)$', '\\1') }}"
- set_fact:
_new_value: "{{ _hip|reject('search', failed_node) }}"
vars:
_hip: "{{ _value.split(',') }}"
gives
_new_value:
- host1=IP
- host2=IP
Now update the key in the configuration file, e.g.
- replace:
path: test.conf
regexp: '{{ _key }}\s*=\s*{{ _value }}'
replace: '{{ _key }}={{ _new_value|join(",") }}'
running the playbook in the check mode (--check --diff) gives
+++ after: test.conf
## -1 +1 ##
-ETCD_INITIAL_CLUSTER=host1=IP,host2=IP,host3=IP
+ETCD_INITIAL_CLUSTER=host1=IP,host2=IP
The procedure can be optimized. The tasks below do the same job
- shell: cat test.conf | grep ETCD_INITIAL_CLUSTER
register: result
check_mode: false
- replace:
path: test.conf
regexp: '{{ _key }}\s*=\s*{{ _value }}'
replace: '{{ _key }}={{ _new_value|join(",") }}'
vars:
_key: "{{ result.stdout|regex_replace('^(.*?)=(.*)$', '\\1') }}"
_value: "{{ result.stdout|regex_replace('^(.*?)=(.*)$', '\\2') }}"
_new_value: "{{ _value.split(',')|reject('search', failed_node) }}"
There are other options on how to get the line from the configuration file. For example, if the file is local, the Ansible way would be lookup plugin, e.g.
- debug:
msg: "{{ lookup('ini', 'ETCD_INITIAL_CLUSTER type=properties file=test.conf') }}"
gives the value of ETCD_INITIAL_CLUSTER
msg: host1=IP,host2=IP,host3=IP
This would further reduce the job to a single task
- replace:
path: test.conf
regexp: '{{ _key }}\s*=\s*{{ _value }}'
replace: '{{ _key }}={{ _new_value|join(",") }}'
vars:
_key: ETCD_INITIAL_CLUSTER
_value: "{{ lookup('ini', _key ~ ' type=properties file=test.conf') }}"
_new_value: "{{ _value.split(',')|reject('search', failed_node) }}"

Dictionary with list remove u ticks and brackets

I'm having an issue accessing a list within a dictionay in ansible.
It's actually more complex than this, I've made a simplified version of the issue for brevity.
ansible version is 2.9
The dictionary looks like:
data_dict:
data:
arch_paths:
- /opt/data1/20201007_test1
- /opt/data1/20201013_test2
- /opt/data1/20201029_test3
bucket: host-data
path: archives/host
src_path: /opt/data1
targz:
arch_paths:
- /opt/data2/20201005.tar.gz
- /opt/data2/20201013.tar.gz
- /opt/data2/20201029.tar.gz
bucket: app-data
path: archives/app
src_path: /opt/data2
My task examples:
- name: print files to archive
debug:
msg: "{{item.value.arch_paths}}"
loop: "{{ data_dict|dict2items }}"
- name: Archive the things
shell: echo "{{item.value.bucket}} -k {{item.value.path}} -p {{item.value.src_path}} {{item.value.arch_paths}}" >> /var/log/put.log
loop: "{{ data_dict|dict2items }}"
These run as expected. But, what ends up in the log file is:
host-data -k archives/host -p /opt/data1 [u'/opt/data1/20201007_test1', u'/opt/data1/20201013_test2', u'/opt/data1/20201029_test3']
app-data -k archives/app -p /opt/data2 [u'/opt/data2/20201005.tar.gz', u'/opt/data2/20201013.tar.gz', u'/opt/data2/20201029.tar.gz']
I want to remove the u'...' and commas so I just have the values with spaces e.g.
host-data -k archives/host -p /opt/data1 /opt/data1/20201007_test1 /opt/data1/20201013_test2 /opt/data1/20201029_test3
app-data -k archives/app -p /opt/data2 /opt/data2/20201005.tar.gz /opt/data2/20201013.tar.gz /opt/data2/20201029.tar.gz
I've seen similar problems on this site but have been unable to adapt any of the solutions.
This happens because arch_paths is a list, so, you just need to join the list items: item.value.arch_paths | join(' ').
With the task:
- shell: echo "{{ item.value.bucket }} -k {{ item.value.path }} -p {{ item.value.src_path }} {{ item.value.arch_paths | join(' ') }}" >> /var/log/put.log
loop: "{{ data_dict|dict2items }}"
It gives a file /var/log/put.log containing:
host-data -k archives/host -p /opt/data1 /opt/data1/20201007_test1 /opt/data1/20201013_test2 /opt/data1/20201029_test3
app-data -k archives/app -p /opt/data2 /opt/data2/20201005.tar.gz /opt/data2/20201013.tar.gz /opt/data2/20201029.tar.gz

Ansible - Read a file, extract specific line, extract a column and assign to variable

I need some help with extracting a specific line from a file and then extracting a column, assign it to a variable and then use that variable in the next task.
I have the file with this format on the confluent broker server
Save the key. It cannot be retrieved later.
+------------+----------------------------------------------+
| Enc Key | omykeyvaluecontinuousstringgoeshereandmakelong= |
+------------+----------------------------------------------+
I am trying to write Ansible task that will read the third line and then extract the key into a variable which I need to export as an environment variable in the task. In the next task I will be executing a confluent command as a shell command.
I tried something like below, but it doesn't work - I get error
vars:
ansible_ssh_extra_args: "-o StrictHostKeyChecking=no"
ansible_host_key_checking: false
contents: "{{ lookup('file', '/etc/kafka/info.txt') }}"
contents2: "{{ lookup('file', '/etc/kafka/info.txt').splitlines() }}"
- name: set fact
set_fact:
extract_key: "{{ contents.split('\n')[2] }}"
- name: Display output
debug: msg="{{ extract_key }}"
And then extract the key value from extract_key variable
How can I achieve this?
Thank you
The task below does the job
- set_fact:
extract_key: "{{ contents.split('\n').2.split('|').2|trim }}"
gives
extract_key: omykeyvaluecontinuousstringgoeshereandmakelong=
You can use this filter if only text lines are fixed:
- name: capturing Key
shell: echo {{ contents }} | head -3 | tail -1 | sed 's/|/\n/g' | sed -n 3p
register: extract_key
- name: Display output
debug: msg="{{ extract_key.stdout }}"
This returns omykeyvaluecontinuousstringgoeshereandmakelong=

Ansible regex_replace or regex_search

I am trying to match the "OK" from the following output with regex and store it in a varible:
System 'server.mylabserver.com'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
load average [0.00] [0.01] [0.05]
cpu 0.1%us 0.1%sy 0.0%wa
memory usage 367.9 MB [20.0%]
swap usage 0 B [0.0%]
uptime 2h 10m
boot time Mon, 02 Apr 2018 06:51:01
data collected Mon, 02 Apr 2018 09:01:02
Ansible code with "regex_replace" that I've tried:
- name: Fetch the monit status
shell: "monit status | tail -n +3"
register: monit_status_raw
tags: basic_monitoring
- name: Extract monit variables
set_fact:
vmstatus: "{{ monit_status_raw | regex_replace('^\s\s([a-z]*)\s+', '\\1:')}}"
Error:
The offending line appears to be:
set_fact:
vmstatus: "{{ monit_status_raw | regex_replace('^\s\s([a-z]*)\s+', '\\1')}}"
^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:
with_items:
- {{ foo }}
Should be written as:
with_items:
- "{{ foo }}"
Ansible code with "regex_search" that I've tried:
- name: Fetch the monit status
shell: "monit status | tail -n +3"
register: monit_status_raw
- name: Extract monit variables
set_fact:
vmstatus: "{{ monit_status_raw | regex_search('^\s\sstatus\s+(.*)$') }}"
Error:
The offending line appears to be:
set_fact:
vmstatus: "{{ monit_status_raw | regex_search('^\s\sstatus\s+(.*)$') }}"
^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:
with_items:
- {{ foo }}
Should be written as:
with_items:
- "{{ foo }}"
Any idea what it's wrong in the regexes?
Thank you,
Dan
I think if you'd like to use regexp_search - you need to give a string and think about escaping characters, and then you need to use some construction as:
with_items
- "{{ monit_status_raw.stdout_lines }}"
But I think it will be simpler:
- name: Fetch the monit status
shell: 'monit status | tail -n +2 | grep "^\s*status" '
register: monit_status_raw
- set_fact:
vmstatus: "{{ monit_status_raw.stdout.split('status')[1]| replace(' ','')}}"
You will get vmstatus = 'Ok', if you use your sample.