I want to append to the EOL of a log4j.propertis file a string, by using ansible. The problem is that the file has new lines and when trying to use the "backrefs" of the ansible's lineinfile module it adds the string under the line instead of appending to EOL.
The file is log4j properties file which have a line I want to edit. The line start with 'log4j.rootLogger' and I want to append to the EOL a string 'bla'
This is how the line looks like now: log4j.rootLogger=WARN, memory, servlet
Expected after the change: log4j.rootLogger=WARN, memory, servlet, bla
The ansible code I used where rootLoggerAppender is the attribute:
lineinfile:
path: "{{ tomcat_path }}/webapps/ROOT/WEB-INF/config/log4j.properties"
regexp: '^(log4j.rootLogger=\.*)'
line: '\1, {{ rootLoggerAppender }}'
backrefs: yes
update:
when I cat log4j.properties the output as follows:
# servlet appender - logs in memory only, allowing remote read of logs
log4j.rootLogger=DEBUG, memory, servlet
When I have update the ansible code to match the exact line the output was as expected.
lineinfile:
path: "{{ tomcat_path }}/webapps/ROOT/WEB-INF/config/log4j.properties"
regexp: '^(log4j\.rootLogger=DEBUG, memory, servlet)'
line: '\1, {{ rootLoggerAppender }}'
backrefs: yes
I have also noticed the file is a windows file, and when converting it to a linux it worked as expected
Correct regexp is below. Escaped dot match a dot only. The first dot should be escaped, but not the second one. This should match any character.
regexp: '^(log4j\.rootLogger=.*)$'
But this solution is not idempotent. Running this task will add the appender repeatedly
$ cat log4j.properties
log4j.rootLogger=WARN, memory, servlet, bla, bla
The regexp below makes the task idempotent
regexp: '^(log4j\.rootLogger=.*?)(, {{ rootLoggerAppender }})?$'
make the first group non-gready
the second group match 0 or 1 delimited appender
Given the file
$ cat log4j.properties
log4j.rootLogger=WARN, memory, servlet
The idempotent playbook
- hosts: localhost
vars:
rootLoggerAppender: bla
tasks:
- lineinfile:
path: log4j.properties
regexp: '^(log4j\.rootLogger=.*?)(, {{ rootLoggerAppender }})?$'
line: '\1, {{ rootLoggerAppender }}'
backrefs: true
gives
$ cat log4j.properties
log4j.rootLogger=WARN, memory, servlet, bla
The non-idempotent version of the playbook
- hosts: localhost
vars:
rootLoggerAppender: bla
tasks:
- lineinfile:
path: log4j.properties
regexp: '^(log4j\.rootLogger=.*)$'
line: '\1, {{ rootLoggerAppender }}'
backrefs: true
gives the same result
$ cat log4j.properties
log4j.rootLogger=WARN, memory, servlet, bla
, but when running repeatedly it keeps adding the delimited appender
$ cat log4j.properties
log4j.rootLogger=WARN, memory, servlet, bla, bla
Related
I've read a thousand of the Ansible regex_search questions on here and have not found a satisfying answer.
Here's the test playbook. backup_stdout is set identically to what I get from the backup utility:
---
- hosts: localhost
connection: local
gather_facts: no
vars:
backup_stdout: |-
Saving active configuration...
/var/local/ucs/f5-apm-1625-081021.ucs is saved.
tasks:
- name: Get the backup name
ansible.builtin.set_fact:
backup_name: "{{ backup_stdout | regex_search(stdout_regex, multiline=True) }}"
vars:
stdout_regex: '"\/var.*ucs\b"gm'
failed_when: backup_name == ''
- debug:
var: backup_name
I can't get the regex_search to produce a match. Here's the same code on regex101, which shows that it does match. I've tried the following:
with/without the multiline
with/without the trailing '\\1' in the expression
with/without passing the result to the | first filter
using ^\/var.*ucs instead of the word boundary (also matches on regex101)
So far, no matter what I've tried, I can't get the Ansible to match. Any help appreciated.
You've got some weird quoting in your regular expression that is causing problems. Because you've written:
stdout_regex: '"\/var.*ucs\b"gm'
You're passing the literal value "\/var.*ucs\b"gm to regex_search. There are no quotes (nor is there a gm) in the content of backup_stdout, so this is never going to match.
I think you want:
- hosts: localhost
connection: local
gather_facts: no
vars:
backup_stdout: |-
Saving active configuration...
/var/local/ucs/f5-apm-1625-081021.ucs is saved.
tasks:
- name: Get the backup name
ansible.builtin.set_fact:
backup_name: "{{ backup_stdout | regex_search(stdout_regex, multiline=True) }}"
vars:
stdout_regex: '/var.*ucs\b'
failed_when: backup_name == ''
- debug:
var: backup_name
Which produces:
TASK [debug] *******************************************************************
ok: [localhost] => {
"backup_name": "/var/local/ucs/f5-apm-1625-081021.ucs"
}
I've been hitting a wall trying to get /etc/exports managed via Ansible.
I've got a role that installs a piece of software on a VM, and I want to then add an entry ot /etc/exports on the NFS server, for that specific VM, so it's able to access the NFS shares needed.
Lineinfile sounds like the way to go, but sofar I can't figure out how to properly write this.
I want this to:
not modify if the host is in the line, no matter where
add the NFS share and the host if there's no line for the NFS share
add the host to the share in case it isn't in there.
The latest installment of my 'add to /etc/exports' that thought should work, but doesn't, is:
- name: Add hosts to mountpoint line
ansible.builtin.lineinfile:
path: /etc/exports
line: '\1 {{ host_ip }}(root_squash,no_subtree_check)'
regex: '^((?!{{ volume_mountpoint }}.*{{ host_ip }}\(root_squash,no_subtree_check\).*).*)$'
backrefs: yes
but i'm still getting all kinds of weird side effects. I've used backreferences etc before, but somehow this one keeps tripping me up.
Anyone who sees what's gong wrong?
Typical /etc/exports entry:
/srv/files 172.16.0.14(rw,no_root_squash,no_subtree_check)
It's not possible in one step to modify a line using backreferences or add the line if missing. To modify the existing mount point the back-references are needed. For example, given the files for testing
shell> cat etc/export1
/srv/files 172.16.0.14(rw,no_root_squash,no_subtree_check)
shell> cat etc/export2
/srv/files 172.16.0.15(rw,no_root_squash,no_subtree_check)
shell> cat etc/export3
/srv/download 172.16.0.14(rw,no_root_squash,no_subtree_check)
the task
tasks:
- lineinfile:
path: "etc/{{ item }}"
regex: '^{{ mount }}(\s+)({{ ipr }})*({{ optionsr }})*(\s*)(.*)$'
line: '{{ mount }}\g<1>{{ ip }}{{ options }} \g<5>'
backrefs: true
vars:
mount: /srv/files
ipr: '172\.16\.0\.14'
ip: '172.16.0.14'
optionsr: '\(.*?\)'
options: '(root_squash,no_subtree_check)'
loop:
- export1
- export2
- export3
gives
--- before: etc/export1 (content)
+++ after: etc/export1 (content)
## -1 +1 ##
-/srv/files 172.16.0.14(rw,no_root_squash,no_subtree_check)
+/srv/files 172.16.0.14(root_squash,no_subtree_check)
changed: [localhost] => (item=export1)
--- before: etc/export2 (content)
+++ after: etc/export2 (content)
## -1 +1 ##
-/srv/files 172.16.0.15(rw,no_root_squash,no_subtree_check)
+/srv/files 172.16.0.14(root_squash,no_subtree_check) 172.16.0.15(rw,no_root_squash,no_subtree_check)
changed: [localhost] => (item=export2)
ok: [localhost] => (item=export3)
The first two files are all right. The problem is the third file. The line hasn't been added to the file. Quoting from backrefs
"... if the regexp does not match anywhere in the file, the file will be left unchanged."
The explanation is simple. There are no groups if the regex doesn't match. If there are no groups the line can't be created.
On the other hand, quoting from regexp
"... If the regular expression is not matched, the line will be added to the file ..."
As a result, it's not possible to ask lineinfile to add a line if the regexp does not match and, at the same time, to do nothing if the regexp is matched. If the regexp is matched you need back-references. If you use back-references you can't add a missing line.
To solve this problem read the content of the files and create a dictionary
- command: "cat etc/{{ item }}"
register: result
loop: [export1, export2, export3]
- set_fact:
content: "{{ dict(_files|zip(_lines)) }}"
vars:
_lines: "{{ result.results|map(attribute='stdout_lines')|list }}"
_files: "{{ result.results|map(attribute='item')|list }}"
gives
content:
export1:
- /srv/files 172.16.0.14(rw,no_root_squash,no_subtree_check)
export2:
- /srv/files 172.16.0.15(rw,no_root_squash,no_subtree_check)
export3:
- /srv/download 172.16.0.14(rw,no_root_squash,no_subtree_check)
Now add the line only if missing, i.e. do not replace the line if the mount point is already there
- lineinfile:
path: "etc/{{ item }}"
line: '{{ mount }} {{ ip }}{{ options }}'
vars:
mount: /srv/files
ip: '172.16.0.14'
options: '(root_squash,no_subtree_check)'
loop: "{{ content|list }}"
when: content[item]|select('search', mount)|length == 0
gives
skipping: [localhost] => (item=export1)
skipping: [localhost] => (item=export2)
--- before: etc/export3 (content)
+++ after: etc/export3 (content)
## -1 +1,2 ##
/srv/download 172.16.0.14(rw,no_root_squash,no_subtree_check)
+/srv/files 172.16.0.14(root_squash,no_subtree_check)
I have a web application that uses a YAML file for configuration. This is an except from the file:
---
settings:
domain: 127.0.0.1
I have an Ansible playbook that uses the lineinfile module to replace the IP address in the YAML file above with the server's public IP address.
- name: Discovering Public Internet Protocol Address
ipify_facts:
register: public_ip
- name: Configuring Application with discovered Public IP
lineinfile:
dest: /application/path/settings.yml
regexp: '^(.*)domain: (.*)$'
line: 'domain: {{ ipify_public_ip }}'
This finds and replaces the 127.0.0.1 IP with the public server's IP but it breaks the YAML indentation as follows:
---
settings:
domain: 54.12.33.3
Problem: "domain" is moved to the same line with "settings" and my ruby app fails to run migrations because it identifies a YAML syntax error.
I do not mind replacing lineinfile with another module, but I'd like to keep it if possible. I've been struggling with this for hours and will appreciate any assistance.
As a quick solution, try to use the 2 spaces () for a better match and substitution:
regexp: '^.*domain: (.*)$'
line: ' domain: {{ ipify_public_ip }}'
I'm sure other improvements can be made to the regex, to use \s or [:space:].
UPDATE: .* from the beginning of regexp shouldn't be needed. Updated per comment requested.
You could just create a yaml template verision.
- template:
src: /path/to/settings.tpl.yml
dest: /path/to/settings.yml
settings.tpl.yml
---
settings:
domain: {{ public_ip }}
I try to use Ansible regex_replace to filter the sub-string "application_1514971620021_4505" from a status message.
In the shell the message looks like this:
I run this code in Ansible:
---
- hosts: [npif]
remote_user: root
tasks:
- block:
- name: Admin submit check
command: chdir=/usr/spring-xd-1.3.1.RELEASE-yarn/ bin/xd-yarn submitted
register: admininfo
- debug: msg="{{ admininfo.stdout }}"
- debug: msg="{{ admininfo.stdout | regex_replace('^.*(application\_\d.*\_\d*)\s.*', '\\1') }}"
become: yes
become_user: ingestdev
debug: msg="{{ admininfo.stdout }}" returns the status message in the different format than in the shell:
ok: [npif] => {
"msg": " APPLICATION ID USER NAME QUEUE TYPE STARTTIME FINISHTIME STATE FINALSTATUS ORIGINAL TRACKING URL\n ------------------------------ --------- --------- -------- ---- -------------- ---------- ------- ----------- ------------------------\n application_1514971620021_4505 ingestdev spring-xd batch_cb XD 1/3/18 2:49 PM N/A RUNNING UNDEFINED http://x.x.x.x:9394"
}
When I run the second debug with regex_replace, I get the identical output to the first debug output - no regex_replace filter has been applied. The regex filter is correct - I've tested it externally. Basically the Ansible code is working too - I have tested with line below and got "test" as expected.
- debug: msg="{{ 'test.home.com' | regex_replace('^([^.]*).*', '\\1') }}"
Do you have an idea, what is wrong with my approach?
Your first problem is that .* doesn't appear to match newlines. Consider this:
- debug:
msg: "{{ admininfo.stdout | regex_replace('.*application', 'foo') }}"
This will replace application with foo, but will leave the header lines intact. Since the ^ anchors your regular expression to the beginning of the text (not the beginning of a line) your expression will never match.
You can take advantage of the fact that ansible has already provided you with individual lines in the stdout_lines key of your registered output. In this case, you would use something like:
- debug:
msg: >
{{ admininfo.stdout_lines[2] | regex_replace('^.*(application_\d.*_\d*)\s.*', '\1') }}
Note here that I've made a few changes in how things are quoted and escaped. In particular, I'm using the folded literal operator > in place of double quotes, and you neede to use \1 instead of \\1 for your replacement string.
This gives me:
ok: [localhost] => {
"msg": "application_1514971620021_4505\n"
}
I am trying to replace or append a path part to the path definition in /etc/environment on a linux box.
Here's what I have:
//all.yml
my_path: "/usr/bin:/usr/sbin"
my_extra_path: "/usr/extra/path"
In my role-file:
//updatePath.yml
- name: update /etc/environment
lineinfile:
dest=/etc/environment
state=present
backrefs=yes
regexp='PATH=({{ my_path }}:?)?({{ my_extra_path }}:?)?(.*)'
line='PATH={{ my_extra_path }}:{{ my_extra_path }}:\3'
Now when I run the role, it works fine updating an existing PATH-line, but not creating duplicates within the line or even duplicate lines. So far so good.
When there is no line with "PATH=" present, I would expect it to add a new one. But it doesn't.
Is my expectation wrong or where lies the problem?
You are using the backrefs: true flag, which prevents lineinfile from changing the file if the line does not already exist. From the docs:
Used with state=present. If set, line can contain backreferences (both
positional and named) that will get populated if the regexp matches.
This flag changes the operation of the module slightly; insertbefore
and insertafter will be ignored, and if the regexp doesn't match
anywhere in the file, the file will be left unchanged. If the regexp
does match, the last matching line will be replaced by the expanded
line parameter.
Since you need to create the line if it does not exist, you should use:
- name: Check whether /etc/environment contains PATH
command: grep -Fxq "PATH=" /etc/environment
register: checkpath
ignore_errors: True
changed_when: False
//updatePath.yml
- name: Add path to /etc/environment
lineinfile:
dest=/etc/environment
state=present
regexp='^PATH='
line='PATH={{ my_extra_path }}'
when: not checkpath.rc == 0
- name: update /etc/environment
lineinfile:
dest=/etc/environment
state=present
backrefs=yes
regexp='PATH=({{ my_path }}:?)?({{ my_extra_path }}:?)?(.*)'
line='PATH={{ my_extra_path }}:{{ my_extra_path }}:\3'
when: checkpath.rc == 0
Same idea as here : https://stackoverflow.com/a/40890850/7231194 or here : https://stackoverflow.com/a/40891927/7231194
Steps are:
Try to replace the line.
If replace mod change it. Cool, it's over !
If replace mod doesn't change, add the line
Example
# Vars
- name: Set parameters
set_fact:
my_path: "/usr/bin:/usr/sbin"
my_extra_path: "/usr/extra/path"
# Tasks
- name: Try to replace the line if it exists
replace:
dest : /dir/file
replace : 'PATH={{ my_extra_path }}'
regexp : '^PATH=.*'
backup : yes
register : tryToReplace
# If the line not is here, I add it
- name: Add line
lineinfile:
state : present
dest : /dir/file
line : '{{ my_extra_path }}'
regexp : ''
insertafter: EOF
when: tryToReplace.changed == false