Print a block of text until the first blank line using awk - regex

here is my sample file:
Host dns2
HostName 172.20.4.80
User root
Port 22
Host dns1
HostName 172.20.4.75
User root
Port 22
Host dns3
HostName 172.20.4.76
User root
Port 22
Host dns4
HostName 172.20.4.77
User root
Port 22
Host dns5
HostName 172.20.4.78
User root
Port 22
Host dns6
HostName 172.20.4.79
User root
Port 22
i want to print only one block means for example
Host dns1
HostName 172.20.4.75
User root
Port 22
output :
Host: dns2 HostName: 172.20.4.80 User: root Port: 22
but in this example all blocks have 4 lines maybe they reach to 5 or more lines later so i want to print from Host to first blank line or remove from Host to first blank line
im really bad at regex and need this to complete my script
thank you

I think that you basically want this:
awk -v RS='' '/dns1/' file
Unset the record separator so that each block is treated as a record, then print whichever record matches the pattern.
Or to use a shell variable:
host=dns1
awk -v host="$host" -v RS='' '$0 ~ host' file
In both of these examples, I'm using the fact that the default action is { print }. As you will likely be changing the output by using { printf ... } you may want to consider adding an exit statement to avoid processing the rest of the file unnecessarily.

a similar awk
$ awk -v RS= -v OFS=' ' '{for(i=1;i<NF;i+=2) $i=$i":"}1' hosts
Host: dns2 HostName: 172.20.4.80 User: root Port: 22
Host: dns1 HostName: 172.20.4.75 User: root Port: 22
Host: dns3 HostName: 172.20.4.76 User: root Port: 22
Host: dns4 HostName: 172.20.4.77 User: root Port: 22
Host: dns5 HostName: 172.20.4.78 User: root Port: 22
Host: dns6 HostName: 172.20.4.79 User: root Port: 22
will give you all records in the desired output format. You can filter either this output further or add a pattern such as
$ awk -v RS= -v OFS=' ' '{for(i=1;i<NF;i+=2) $i=$i":"} /dns2/' hosts
Host: dns2 HostName: 172.20.4.80 User: root Port: 22
if want to exit after processing the selected record, you need to slightly change the script
$ awk -v RS= -v OFS=' ' '/dns2/{for(i=1;i<NF;i+=2) $i=$i":"; print; exit}' hosts
Host: dns2 HostName: 172.20.4.80 User: root Port: 22
If you want to select everything except one record you can negate the pattern (and remove exit)
$ awk -v RS= -v OFS=' ' '!/dns2/{for(i=1;i<NF;i+=2) $i=$i":"; print}' hosts
Host: dns1 HostName: 172.20.4.75 User: root Port: 22
Host: dns3 HostName: 172.20.4.76 User: root Port: 22
Host: dns4 HostName: 172.20.4.77 User: root Port: 22
Host: dns5 HostName: 172.20.4.78 User: root Port: 22
Host: dns6 HostName: 172.20.4.79 User: root Port: 22
Note that sed inplace replacement needs an intermediary file. If you want to replace the original file with the formatted one sans one record, you can use this command pattern on the last awk statement
$ awk ... > temp && mv temp original
UPDATE:
setting OFS will change all the separators between fields. You want to logically group them by name: value, so change the script as such
$ awk -v RS= '{for(i=1;i<NF;i++) $i=$i (i%2?":":"\t")}1' hosts
Host: dns2 HostName: 172.20.4.80 User: root Port: 22
Host: dns1 HostName: 172.20.4.75 User: root Port: 22
Host: dns3 HostName: 172.20.4.76 User: root Port: 22
Host: dns4 HostName: 172.20.4.77 User: root Port: 22
Host: dns5 HostName: 172.20.4.78 User: root Port: 22
Host: dns6 HostName: 172.20.4.79 User: root Port: 22
which sets a tab delimiter after even positioned fields.

To print the 3rd record:
$ awk -v RS= -F'\n[[:blank:]]+' -v OFS='\t' 'NR==3{$1=$1; gsub(/ +/,": "); print}' file
Host: dns3 HostName: 172.20.4.76 User: root Port: 22
To print the records that contains dns4:
$ awk -v RS= -F'\n[[:blank:]]+' -v OFS='\t' '/dns4/{$1=$1; gsub(/ +/,": "); print}' file
Host: dns4 HostName: 172.20.4.77 User: root Port: 22
To print all records except those that contain dns3, dns4, or dns5:
$ awk -v RS= -F'\n[[:blank:]]+' -v OFS='\t' '!/dns[345]/{$1=$1; gsub(/ +/,": "); print}' file
Host: dns2 HostName: 172.20.4.80 User: root Port: 22
Host: dns1 HostName: 172.20.4.75 User: root Port: 22
Host: dns6 HostName: 172.20.4.79 User: root Port: 22

Not different than Tom Fenech approach, since it uses the record separator, but it plays with the field separator too to obtain the desired output:
awk -v RS='' -F'\n[\t ]*' -v OFS=' ' '/dns1/{$1=$1;print}' file
When you change the output field separator, you need to use $1=$1 (or $0=$0 or with any other field) to force awk to re-evaluate the record and to take in account the new field separator.
Notes: you can exit awk when a matching block is found with the exit command. This avoids to process all the end of the file.
You can also only test the pattern /dns1/ with the first field.
awk -v RS='' -F'\n[\t ]*' -v OFS=' ' '$1~/dns1/{$1=$1;print;exit}' file
If you add semi-colons to the result, since you modify the fields, the $1=$1 trick becomes useless. You can write:
awk -v RS='' -F'\n[\t ]*' -v OFS=' ' '$1~/dns1/{for(i=1;i<=NF;i++){sub(" ", ": ", $i)};print;exit}' file

This might work for you (GNU sed):
sed -n '/Host dns1/{:a;N;/^\s*$/M!ba;s/\n\s*/ /g;s/\s*$//p}' file
This focuses in on the required string, then appends following lines until a blank one and finally manipulates the new string that is gathered into the required output.

Related

Updating letsencrypt cert-manager ClusterIssuer server url (with kubectl patch)

A ClustrerIssuer is running. It is configured with the lestencrypt staging url. How do I change the server url to production? Is it possible to patch or is replacing the complete issuer mandatory?
I tried:
kubectl patch ClusterIssuer letsencrypt --type json -p '[{"op": "replace", "path": "/spec/acme/sever", "value":"https://acme-v02.api.letsencrypt.org/directory"}]'
The result is:
clusterissuer.cert-manager.io/letsencrypt patched (no change)
kubectl describe ClusterIssuer letsencrypt | grep letsencrypt.org
Server: https://acme-staging-v02.api.letsencrypt.org/directory
Uri: https://acme-staging-v02.api.letsencrypt.org/acme/acct/xyz
*** This works, but I am trying to make the path work:
cat ../include/letsencypt.yaml | sed -E "s/acme-staging-v02/acme-v02/g" | kubectl apply -n cert-manager -f -

Ansible regex to replace the DNS address in Ubuntu netplan file

I would like to change DNS IP address from 192.168.86.14 to 192.168.86.16 in Ubuntu netplan yaml file:
link: ens3
addresses: [192.168.86.12/24]
gateway4: 192.168.86.1
nameservers:
addresses: [192.168.86.14,8.8.8.8,8.8.4.4]
Here is my ansible playbook:
- name: test
ansible.builtin.replace:
path: /etc/netplan/00-installer-config.yaml
regexp: '(addresses: \[)+192.168.86.14,'
replace: '\1192.168.86.16,'
My playbook doesn't change anything in the file. Tried to escape comma but doesn't match anything as well.
For some reason I need to make sure the IP address is between "addresses [" and "," so I can't just use the syntax like this :
- name: test
ansible.builtin.replace:
path: /etc/netplan/00-installer-config.yaml
regexp: '192.168.86.14'
replace: '192.168.86.16'
I am very new to Ansible, any help is appreciated!
The dictionaries are immutable in YAML. But, you can update dictionaries in Jinja2. Let's take a complete example of a netplan configuration file, e.g.
shell> cat 00-installer-config.yaml
network:
version: 2
renderer: networkd
ethernets:
ens3:
mtu: 9000
enp3s0:
link: ens3
addresses: [192.168.86.12/24]
gateway4: 192.168.86.1
nameservers:
addresses: [192.168.86.14,8.8.8.8,8.8.4.4]
Read the dictionary into a variable
- include_vars:
file: 00-installer-config.yaml
name: netplan_conf
gives
netplan_conf:
network:
ethernets:
enp3s0:
addresses:
- 192.168.86.12/24
gateway4: 192.168.86.1
link: ens3
nameservers:
addresses:
- 192.168.86.14
- 8.8.8.8
- 8.8.4.4
ens3:
mtu: 9000
renderer: networkd
version: 2
Create a template that updates the nameservers
shell> cat 00-installer-config.yaml.j2
{% set _dummy = netplan_conf.network.ethernets.enp3s0.nameservers.update({'addresses': _addresses}) %}
{{ netplan_conf|to_nice_yaml }}
The task below
- template:
src: 00-installer-config.yaml.j2
dest: 00-installer-config.yaml
vars:
_addresses: "{{ netplan_conf.network.ethernets.enp3s0.nameservers.addresses|
regex_replace('192.168.86.14', '192.168.86.16') }}"
will update the configuration file
shell> cat 00-installer-config.yaml
network:
ethernets:
enp3s0:
addresses:
- 192.168.86.12/24
gateway4: 192.168.86.1
link: ens3
nameservers:
addresses:
- 192.168.86.16
- 8.8.8.8
- 8.8.4.4
ens3:
mtu: 9000
renderer: networkd
version: 2

Not able to run express-gateway on heroku - failed to bind to $PORT

I'm trying to have the basic example express-gateway running on Heroku
http:
port: ${HTTP_PORT:-8080}
hostname: ${HOST:-localhost}
admin:
port: 9876
host: localhost
apiEndpoints:
api:
host: 'localhost'
paths: '/ip'
serviceEndpoints:
httpbin:
url: 'https://httpbin.org'
...
Looking the logs I got the following error:
2021-02-15T18:53:09.947569+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch
Any ideas?
I managed to fix that with the following config:
http:
port: ${PORT:-8080}
https:
port: ${PORT:-8081}
admin:
port: 9876
apiEndpoints:
api:
paths: '/ip'
So the default port variable is called PORT and by removing all references to localhost 🙌

Parsing .ssh/config for proxy information

I would like to parse the .ssh/config file for proxy information contained therein and display each host that has related proxy information with the proxy information. Hosts that do not have any proxy information should be filtered out. Man page for .ssh/config: http://man-wiki.net/index.php/5:ssh_config
This should be done from a Unix shell script under Bash, so a standard tool like Perl, awk or sed is preferred.
Example input file:
Host ssh.foo.com
User ssh
HostName ssh.foo.com
Port 443
ProxyCommand /usr/local/bin/corkscrew proxy 8080 %h %p ~/.ssh/proxyauth
Host ci
HostName 127.0.0.2
User ci
Host nightly
HostName 192.168.1.1
User goodnight
Host foobar.org
User git
HostName foobar.org
Port 443
ProxyCommand /usr/local/bin/corkscrew proxy 8080 %h %p ~/.ssh/proxyauth
Host integration
HostName 192.168.1.2
User int
The expected output should look like this:
Host: ssh.foo.com - Proxy: /usr/local/bin/corkscrew proxy 8080 %h %p ~/.ssh/proxyauth
Host: foobar.org - Proxy: /usr/local/bin/corkscrew proxy 8080 %h %p ~/.ssh/proxyauth
The difficulty here is that the search has to cover multiple lines.
Try following awk command:
awk '
$1 == "Host" {
host = $1 ": " $2;
next;
}
$1 == "ProxyCommand" {
$1 = "";
sub( /^[[:space:]]*/, "" );
printf "%s - Proxy: %s\n", host, $0;
}
' .ssh/config
It yields:
Host: ssh.foo.com - Proxy: /usr/local/bin/corkscrew proxy 8080 %h %p ~/.ssh/proxyauth
Host: foobar.org - Proxy: /usr/local/bin/corkscrew proxy 8080 %h %p ~/.ssh/proxyauth
an awk oneliner may work for your requirement:
awk -v RS="" '/Proxy/{gsub(/\n/,"");gsub(/\s*User.*ProxyCommand/,"- Proxy:");print}'file
test (a.txt is your input file)
kent$ awk -v RS="" '/Proxy/{gsub(/\n/,"");gsub(/\s*User.*ProxyCommand/,"- Proxy:");print}' a.txt
Host ssh.foo.com - Proxy: /usr/local/bin/corkscrew proxy 8080 %h %p ~/.ssh/proxyauth
Host foobar.org - Proxy: /usr/local/bin/corkscrew proxy 8080 %h %p ~/.ssh/proxyauth
Just for kicks, here is a nasty sed command that seems to accomplish this.
sed -n '/^[Hh]ost/{:r;h;:l;n;/^[Hh]ost/!{/[Pp]roxy/H;bl};/^[Hh]ost/{x;/[Pp]roxy/{s/\n\s*/ - /;s/[Cc]ommand/:/;s/[Hh]ost/\0:/;p};x;br}}' ~/.ssh/config
This is more for demonstration purposes anyway since the other solutions are much easier to grasp. A commented view of this:
/^[Hh]ost/ { # Begin on a Host line
:restart # Label to jump back to
h # Copy the pattern space to the save space
:loop # Label to jump to when we don't want to wipe out the save space
n # Read in the next line
/^[Hh]ost/!{ # For lines that aren't host definitions
/[Pp]roxy/H # If they are proxy configurations, append the line to the save space
b loop # Go to :loop
}
/^[Hh]ost/{ # If they are *not* host definitions
x # Exchange the pattern and save spaces
/[Pp]roxy/{ # If the new pattern space contains a proxy configuration
s/\n\s*/ - / # Insert the required separator
s/[Cc]ommand/:/ # Change "ProxyCommand" to "Proxy:"
s/[Hh]ost/\0:/ # Change "Host" to "Host:"
p # Print the pattern space
}
x # Exchange the pattern and save spaces
b restart # Go to :restart
}
}
Example output:
Host: ssh.foo.com - Proxy: /usr/local/bin/corkscrew proxy 8080 %h %p ~/.ssh/proxyauth
Host: foobar.org - Proxy: /usr/local/bin/corkscrew proxy 8080 %h %p ~/.ssh/proxyauth
Have you checked out the Net::SSH::Perl::Config module ? It claims to read configs in the standard ssh format - albeit with a subset of keywords.
The configuration files should be in the same format used for the ssh
command line program; see the ssh manpage for information on this
format. Net::SSH::Perl::Config understands a subset of the
configuration directives that can live in these files; this subset
matches up with the functionality that Net::SSH::Perl can support.
Unknown keywords will simply be skipped.
One way using GNU sed:
sed -nre '/^Host/h;/ProxyCommand/{H;x;s/(Host)(.*)\n +[^ ]+ /\1:\2 - Proxy: /p}' file.txt
Results:
Host: ssh.foo.com - Proxy: /usr/local/bin/corkscrew proxy 8080 %h %p ~/.ssh/proxyauth
Host: foobar.org - Proxy: /usr/local/bin/corkscrew proxy 8080 %h %p ~/.ssh/proxyauth
try this:
#! /usr/bin/perl
use strict;
use warnings;
open FD, "DATA" || die $!;
my $line = do {local $/; <FD>};
my #a = split /Host\b/, $line;
foreach (#a) {
if (defined $_ and /^\s*(\S+).*ProxyCommand\s+([^\n]+)/s) {
print "$1 - Proxy: $2\n";
}
}

Replace Line in config file

I want to replace lines of the Redmine config file automatically. The config file looks like this:
# MySQL (default setup).
....
production:
adapter: mysql
database: redmine
host: localhost
username: root
password:
encoding: utf8
development:
adapter: mysql
database: redmine_development
host: localhost
username: root
password:
encoding: utf8
....
I only want to replace lines after the production section like password, database,...
How can I do that with sed?
based on your example file:
sed -r '/^production:/,/^$/{s/(\s*database:).*/\1newDB/;
s/(\s*password:).*/\1newPWD/}' yourConfig
the above cmd will change "database" and "password" setting in "production" section to newDB and newPWD.
output:
...
production:
adapter: mysql
database:newDB
host: localhost
username: root
password:newPWD
encoding: utf8
development:
adapter: mysql
database: redmine_development
host: localhost
username: root
password:
encoding: utf8
...