blockinfile Ansible module does not insert at the given regex - regex

Below is the test.conf where i wish to add a block before the line closing tags i.e. before the line which starts with </VirtualHost>
cat test.conf
#
##<VirtualHost _default_:443>
<VirtualHost *:443>
#ProxyPreserveHost On
</VirtualHost>
Below is my playbook to add the block:
cat /tmp/test.yml
---
- name: "Play 1"
hosts: localhost
tasks:
- name: Debug
blockinfile:
path: "/tmp/test.conf"
marker: "#"
state: present
block: |
<FilesMatch "^.*\.(css|html?|js|pdf|txt|xml|xsl|gif|ico|jpe?g|png)$">
Require all granted
</FilesMatch>
insertbefore: '^[^#]*</VirtualHost>'
I checked my test.conf and regex ^[^#]*<\/VirtualHost> on online python editor https://regex101.com and it gets the correct line matched.
Online regex tester and debugger: PHP, PCRE, Python, Golang and JavaScript
Online regex tester, debugger with highlighting for PHP, PCRE, Python, Golang and JavaScript.
regex101.com
The file gets changed and the block gets inserted however in the wrong place as you can see below:
TASK [Debug] ************************************************************************************************************************************************
changed: [localhost]
PLAY RECAP **************************************************************************************************************************************************
localhost : ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
cat /tmp/test.conf
#
<FilesMatch "^.*\.(css|html?|js|pdf|txt|xml|xsl|gif|ico|jpe?g|png)$">
Require all granted
</FilesMatch>
#
##<VirtualHost _default_:443>
<VirtualHost *:443>
#ProxyPreserveHost On
</VirtualHost>
Can you please suggest what is wrong with my playbook and how to get this to work ?

It's because ansible specifies in the fine manual that marker: is exactly what it says -- the way it knows where the managed blocks begin and end. Since you chose to use text that is found throughout your file but is unrelated to the managed block sections, ansible just shrugged its shoulders and gave GIGO.
They even have a dedicated warning about leaving out the magic {mark} template param from marker::
Using a custom marker without the {mark} variable may result in the block being repeatedly inserted on subsequent playbook runs.
If you change your marker: to even something like marker: "#*#*#*" it will start to work ... or at least will work once.

Related

Add a line in a file at multiple places upon condition using Ansible

I wish to add the line "Hi There" in file1.txt after i find a line that starts with Directory tag if it is already not present.
I have multiple Directory tag entries in my file1.txt a few of them are below:
<Directory "/web/htdocs">
<Directory />
.....
.....
<Directory "/web/cgi-bin">
With the below code I was able to a add the line "Hi There" after only in the last Directory tag entry.
- name: Insert After string
replace:
path: "/tmp/file1.txt"
state: present
line: 'Hi There'
insertafter: '^<Directory '
Results:
<Directory "/web/cgi-bin">
Hi There
However, I wish to get "Hi There" on the next line of each of the / all of the Directory entries.
I understand that the "replace" module replaces all the occurances found. However, my limited knowledge says that the replace module needs a string to be replaced.
My requirement is not to replace anything but to insert a line after each of the Directory tag occurance are found.
The idea: match any line like '' except if the next line is 'Hi there'. Replace with the same line followed by 'Hi there'. This is a regexp concept known as negative lookahead
My test.txt file at start
<Directory "/web/htdocs">
<Directory />
<Directory "/web/cgi-bin">
<Directory />
<Directory "/some/other">
Hi there
</Directory>
My test.yml playbook
---
- name: Replace several lines
hosts: localhost
tasks:
- name: Add 'Hi there' after directory def if not present
replace:
path: test.txt
regexp: '^(<Directory ".*">\n)(?!Hi there)'
replace: '\1Hi there\n'
First run:
$ ansible-playbook test.yml
PLAY [Replace several lines] ********************************************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] **************************************************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Add 'Hi there' after directory def if not present] ****************************************************************************************************************************************************************************************************************
changed: [localhost]
PLAY RECAP **************************************************************************************************************************************************************************************************************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
test.txt file after first run. Note that lines where added only where needed.
<Directory "/web/htdocs">
Hi there
<Directory />
<Directory "/web/cgi-bin">
Hi there
<Directory />
<Directory "/some/other">
Hi there
</Directory>
Second run:
$ ansible-playbook test.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
PLAY [Replace several lines] ********************************************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] **************************************************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Add 'Hi there' after directory def if not present] ****************************************************************************************************************************************************************************************************************
ok: [localhost]
PLAY RECAP **************************************************************************************************************************************************************************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
As you can see, this time the file is unchanged since all possible lines were added.

Replace Specific String in YAML File using Ansible

I have a web application that uses a YAML file for configuration. This is an except from the file:
---
settings:
domain: 127.0.0.1
I have an Ansible playbook that uses the lineinfile module to replace the IP address in the YAML file above with the server's public IP address.
- name: Discovering Public Internet Protocol Address
ipify_facts:
register: public_ip
- name: Configuring Application with discovered Public IP
lineinfile:
dest: /application/path/settings.yml
regexp: '^(.*)domain: (.*)$'
line: 'domain: {{ ipify_public_ip }}'
This finds and replaces the 127.0.0.1 IP with the public server's IP but it breaks the YAML indentation as follows:
---
settings:
domain: 54.12.33.3
Problem: "domain" is moved to the same line with "settings" and my ruby app fails to run migrations because it identifies a YAML syntax error.
I do not mind replacing lineinfile with another module, but I'd like to keep it if possible. I've been struggling with this for hours and will appreciate any assistance.
As a quick solution, try to use the 2 spaces () for a better match and substitution:
regexp: '^.*domain: (.*)$'
line: ' domain: {{ ipify_public_ip }}'
I'm sure other improvements can be made to the regex, to use \s or [:space:].
UPDATE: .* from the beginning of regexp shouldn't be needed. Updated per comment requested.
You could just create a yaml template verision.
- template:
src: /path/to/settings.tpl.yml
dest: /path/to/settings.yml
settings.tpl.yml
---
settings:
domain: {{ public_ip }}

HAProxy 1.6+: rewrite host based on path

I'm trying to redirect all requested of type:
static.domain.com/site1/resource.jpg
static.domain.com/site1/resource2.js
static.domain.com/site2/resource3.gif
static.domain.com/site2/someDir/resource4.txt
to
site1.domain.com/resource.jpg
site1.domain.com/resource2.js
site2.domain.com/resource3.gif
site2.domain.com/someDir/resource4.txt
Basically, if the host is static.domain.com:
New subdomain is based on the the first part of the original path, with same TLD
New path is the original path not including the first part
I am pretty sure regexps can solve this, just not sure how to modify one header based on another..
At first I thought this might work:
# Detect hosts of the format static.*
acl host_static hdr_beg(host) -i static.
# Style using reqirep
# -------------
# Replace "static.domain.com" with "someFolder.domain.com" if the host is static.* and the path has at least two / symbols
# This causes: static.domain.com ===> whatever3.domain.com
#reqirep ^([^\ :]*\ /)([^/]+)(/.*\n)(^(?:[a-zA-Z0-9()\-=\*\.\?;,+\/&_]+:\ .+\n)+)*Host:\ static\.([^/]+?)$ \1\2\3\4Host:\ \2.\5 if host_static
#
# Replace "/someFolder/" with "/" at the beginning of any request path, if the host is static.*
# This causes: /whatever3/another/long/path ===> /another/long/path
#reqirep ^([^\ :]*)\ /[^/]+/(.*) \1\ /\2 if host_static
#---------------
but it doesn't work as expected. The regexp works properly in controlled tests, but not in haproxy itself. Probably an issue of directive processing and execution order. (perhaps the modification of the request path screws the first regexp?)
I then tried this:
# Style using set-var, set-path etc
#---------------
#http-request set-var(req.first_path_part) path,field(2,/) if host_static
#http-request set-var(req.last_host_part) hdr(host),regsub(^static\.,) if host_static
#http-request replace-header Host .* %[var(req.first_path_part)].%[var(req.last_host_part)] if host_static
#http-request set-path %[path,regsub(^/.*?/,/)] if host_static
#---------------
Once again, it almost works, but for some reason the host doesn't get replaced properly.
Since this was only used by the QA env, and the behaviour is different from Production anyways (static.*, in my case, would point to a CDN), I decided this is a sufficient solution for now:
# New style, using set-var and redirection.
#---------------
http-request set-var(req.first_path_part) path,field(2,/) if host_static
http-request set-var(req.last_host_part) hdr(host),regsub(^static\.,) if host_static
http-request redirect location https://%[var(req.first_path_part)].%[var(req.last_host_part)]%[path,regsub(^/.*?/,/)] code 302 if host_static
#---------------
I'm not sure how HAProxy works, but I can help you with the regex.
Try: ^static\.([^/]+)/([^/]+)/(.*)$
Your new URL will be \2.\1/\3.
Note that you may need to escape the /s in the regex (which would make it \/).

Apache 2.4 setenvif dontlog pingdom.com

Many of webmasters use pingdom.com as a monitoring ping service.
But the problem is that /httpd/access_log is full of
208.64.28.194 - - [06/Aug/2015:12:20:22 -0500] "GET / HTTP/1.1" 200 2917 "-" "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)"
I set
CustomLog "logs/access_log" combined env=!dontlog
and tried to get rid of it using variations like
SetEnvIf Remote_Host "^pingdom\.com$" dontlog
SetEnvIFNoCase Remote_Host "pingdom.com$" dontlog
SetEnvIfNoCase Referer "www\.pingdom\.com" dontlog
SetEnvIFNoCase Host "^pingdom.com$" dontlog
but still no a success with any of them - so thanks for any else hint to try.
I'll put my comment here as answer so anyone will find this easier.
Since one can see from the log file, the host name is not Pingdom.com but a part of the user agent string.
Solutions to try:
First be sure you have enabled the setenvif-module. Write the command
sudo apache2ctl -M | grep setenv
It should return something like "setenvif_module (shared)"
Then you can try setting by remote address
SetEnvIf Remote_Addr "208\.64\.28\.194$" dontlog
The final working solution is this, dont log if the user agent string contains Pingdom string:
SetEnvIfNoCase User-Agent "^Pingdom" dontlog
Edit: enhanced some parts of the answer.

How can I make multiple replacements on the same file using one saltstack state?

Here's my target file:
Sonatype Nexus
# ==============
# This is the most basic configuration of Nexus.
# Jetty section
application-port=8081
application-host=0.0.0.0
nexus-webapp=${bundleBasedir}/nexus
nexus-webapp-context-path=/nexus
# Nexus section
nexus-work=/opt/nexuswork
runtime=${bundleBasedir}/nexus/WEB-INF
I know there's an easy way to do this with regex or a simple sed script:
sed -i 's/${bundleBasedir}\/..\/my\/second\/path\/002\/\/nexus/\/myfirstdir001\/g'
However, I would, ideally, prefer the saltstack way.
I would like it to look something like this:
Sonatype Nexus
# ==============
# This is the most basic configuration of Nexus.
# Jetty section
application-port=8081
application-host=0.0.0.0
nexus-webapp=/my/second/path/002/nexus # changed
nexus-webapp-context-path=/nexus
# Nexus section
nexus-work=/opt/nexuswork
runtime=/myfirstdir001/nexus/WEB-INF # changed
I haven't yet made sense of the saltstack documentation on this.
Saltstack's documentation for salt.states.file.replace seems fairly straightforward:
http://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html#salt.states.file.replace
Here's what I tried:
/opt/nexus-2.8.0/conf/nexus.properties
file: # state
- replace
- pattern: '\$\{bundleBasedir\}' # without escapes: '${bundleBasedir}/nexus'
- repl: '/my/second/path/002/nexus'
# - name: /opt/nexus-2.8.0/conf/nexus.properties
# - count=0
# - append_if_not_found=False
# - prepend_if_not_found=False
# - not_found_content=None
# - backup='.bak'
# - show_changes=True
- pattern: '\$\{bundleBasedir\}\/WEB-INF' # without escapes: ${bundleBasedir}/WEB-INF
- repl: '/myfirstdir001/'
I could maybe try multiple state IDs, but that seems inelegant.
If there's anything else I'm fuffing up, please advise!
I'd shore love to find a solution to this.
Also, if there's any demand for people improving the salt documentation, I think my team could be convinced to pitch in some.
Here's the closest thing I've found to someone else asking this question:
http://comments.gmane.org/gmane.comp.sysutils.salt.user/15138
For such a small file I would probably go with a template as ahus1 suggested.
If the file was bigger and/or we didn't want to control other lines just ensure that those two are correct, I think multiple state IDs (as mentioned by OP) is a good way to go. Something like:
/opt/nexus-2.8.0/conf/nexus.properties-jetty:
file:
- replace
- name: /opt/nexus-2.8.0/conf/nexus.properties
- pattern: '\$\{bundleBasedir\}' # without escapes: '${bundleBasedir}/nexus'
- repl: '/my/second/path/002/nexus'
/opt/nexus-2.8.0/conf/nexus.properties-nexus:
file:
- replace:
- name: /opt/nexus-2.8.0/conf/nexus.properties
- pattern: '\$\{bundleBasedir\}\/WEB-INF' # without escapes: ${bundleBasedir}/WEB-INF
- repl: '/myfirstdir001/'
I have a similar setup in my configuration but I use salt.states.file.line to replace some lines with my values. In addition I used salt.states.file.managed with a template and replace: False to initialize the file if it's missing but once it exists, only the line states are doing changes.
The salt way to do this as I understand it: Place a template file for nexus.properties inside salt and use file.managed like shown in the docs http://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html
You will end up with something like:
/opt/nexus-2.8.0/conf/nexus.properties:
file.managed:
- source: salt://nexus/nexus.properties.jinja
- template: jinja
- defaults:
bundleBasedir: "..."
You'll then use Jinja templating in your file:
# Jetty section
application-port=8081
application-host=0.0.0.0
nexus-webapp={{ bundleBasedir }}/nexus
nexus-webapp-context-path=/nexus
See here for Jinja templating: http://docs.saltstack.com/en/latest/ref/renderers/all/salt.renderers.jinja.html
I hope it helps.