I currently am trying to get specific parameters from a url.
My url looks like: https://private.io/report-artifact/dsop-pipeline-artifacts/container-scan-reports/redhat/ubi/ubi7/7.8/2020-02-14T222203.548_2868/ubi7-7.8.tar
I want just redhat/ubi/ubi7/7.8
I can get redhat/ubi/ubi7/7.8/2020-02-14T222203.548_2868/ubi7-7.8.tar by doing,
echo https://private.io/report-artifact/dsop-pipeline-artifacts/container-scan-reports/redhat/ubi/ubi7/7.8/2020-02-14T222203.548_2868/ubi7-7.8.tar | sed 's|.*/container-scan-reports/||'
Thus I want to remove /2020-02-14T222203.548_2868/ubi7-7.8.tar
I also would like to change the / to a - so that I have redhat-ubi-ubi7-7.8
With GNU sed:
Get the 4 following path elements after .*/container-scan-reports/ and replace all / with -:
url='https://private.io/report-artifact/dsop-pipeline-artifacts/container-scan-reports/redhat/ubi/ubi7/7.8/2020-02-14T222203.548_2868/ubi7-7.8.tar'
echo "$url" | sed -E 's|.*/container-scan-reports/(([^/]*/){3}[^/]*).*|\1|;s|/|-|g'
Or you could get everything after .*/container-scan-reports/, but not the last two path elements:
echo "$url" | sed -E 's|.*/container-scan-reports/(.*)/[^/]*/[^/]*|\1|;s|/|-|g'
When you know the position in the string you can use cut
echo "${string}" | cut -d/ -f 7-10 | tr '/' '-'
Another way with sed is
echo "${string}" | sed -E 's#([^/]*/){6}([^/]*)/([^/]*)/([^/]*)/([^/]*).*#\2-\3-\4-\5#'
Related
I have text like:
TEXT="I need to replace the hostname [[google.com]] with it's ip in side the text"
Is there a way to use something like below, but working?
sed -Ee "s/\[\[(.*)\]\]/`host -t A \1 | rev | cut -d " " -f1 | rev`/g" <<< $TEXT
looks like the value of \1 is not being passed to the shell command used inside sed.
Thanks
Backquote interpolation is performed by the shell, not by sed. This means that your backquotes will either be replaced by the output of a command before the sed command is run, or (if you correctly quote them) they will not be replaced at all, and sed will see the backquotes.
You appear to be trying to have sed perform a replacement, then have the shell perform backquote interpolation.
You can get the backquotes past the shell by quoting them properly:
$ echo "" | sed -e 's/^/`hostname`/'
`hostname`
However, in that case you will have to use the resulting string in a shell command line to cause backquote interpolation again.
Depending on how you feel about awk, perl, or python, I'd suggest you use one of them to do this job in a single pass. Alternatively, you could make a first pass extracting the hostnames into a command without backquotes, then execute the commands to get the IP addresses you want, then replace them in another pass.
It's got to be a two part command, one to get a variable that bash can use, the other to do a straight-up /s/ replacement with sed.
TEXT="I need to replace the hostname [[google.com]] with it's ip in side the text"
DOMAIN=$(echo $TEXT | sed -e 's/^.*\[\[//' -e 's/\]\].*$//')
echo $TEXT | sed -e 's/\[\[.*\]\]/'$(host -tA $DOMAIN | rev | cut -d " " -f1 | rev)'/'
But, more cleanly using how to split a string in shell and get the last field
TEXT="I need to replace the hostname [[google.com]] with it's ip in side the text"
DOMAIN=$(echo $TEXT | sed -e 's/^.*\[\[//' -e 's/\]\].*$//')
HOSTLOOKUP=$(host -tA $DOMAIN)
echo $TEXT | sed -e 's/\[\[.*\]\]/'${HOSTLOOKUP##* }/
The short version is that you can't mix sed and bash the way you're expecting to.
This works:
#!/bin/bash
txt="I need to replace the hostname [[google.com]] with it's ip in side the text"
host_name=$(sed -E 's/^[^[]*\[\[//; s/^(.*)\]\].*$/\1/' <<<"$txt")
ip_addr=$(host -tA "$host_name" | sed -E 's/.* ([0-9.]*)$/\1/')
echo "$txt" | sed -E 's/\[\[.*\]\]/'"$ip_addr/"
# I need to replace the hostname 172.217.4.174 with it's ip in side the text
Thank you all,
I made the below solution:
function host_to_ip () {
echo $(host -t A $1 | head -n 1 | rev | cut -d" " -f1 | rev)
}
function resolve_hosts () {
local host_placeholders=$(grep -o -e "##.*##" $1)
for HOST in ${host_placeholders[#]}
do
sed -i -e "s/$HOST/$(host_to_ip $(sed -Ee 's/##(.*)##/\1/g' <<< $HOST))/g" $1
done
}
Where resolve_hosts gets a text file as an argument
i have a range of strings which conform to one of the two following patters:
("string with spaces",4)
or
(string_without_spaces,4)
I need to extract the "string" via a bash command, and so far have found a pattern that works for each, but not for both.
echo "(\"string with spaces\",4)" | sed -n 's/("\(.*\)",.*)/\1/ip'
output:string with spaces
echo "(string_without_spaces,4)" | sed -n 's/(\(.*\),.*)/\1/ip'
output:string_without_spaces
I have tried using "\? however it does not match the " if it is there:
echo "(SIM,0)" | sed -n 's/("\?\(.*\)"\?,.*)/\1/ip'
output: SIM
echo "(\"SIM\",0)" | sed -n 's/("\?\(.*\)"\?,.*)/\1/ip'
output: SIM"
can anyone suggest a pattern that would extract the string in both scenarios? I am not tied to sed but would prefer to not have to install perl in this environment.
How about using [^"] instead of . to exclude " to be matched.
$ echo '("string with spaces",4)' | sed -n 's/("\?\([^"]*\)"\?,.*)/\1/p'
string with spaces
$ echo "(string_without_spaces,4)" | sed -n 's/("\?\([^"]*\)"\?,.*)/\1/p'
string_without_spaces
$ echo "(SIM,0)" | sed -n 's/("\?\([^"]*\)"\?,.*)/\1/p'
SIM
$ echo '("SIM",0)' | sed -n 's/("\?\([^"]*\)"\?,.*)/\1/p'
SIM
I have the following bash:
#!/bin/bash
if ["$#" -ne "1"]; then
echo "Usage: `basename $0` <HOSTNAME>"
exit 1
fi
IPADDR=`ifconfig | head -2 | tail -1 | cut -d: -f2 | rev | cut -c8-23 | rev`
sed -i -e '1i$IPADDR $1\' /etc/hosts
But when I cat /etc/hosts:
$IPADDR
How can I deal with such issues?
Your problem is that variables inside single quotes ' aren't expanded by the shell, but left unchanged. To quote variables you want expanded use double quotes " or just leave off the quotes if they are unneeded like here, e.g.
sed -i -e '1i'$IPADDR' '$1'\' /etc/hosts
In above line $IPADDR and $1 are outside of quotes and will be expanded by the shell before the arguments are being feed to sed.
The single quotes mean the string isn't interpolated as a variable.
#!/bin/bash
IPADDR=$(/sbin/ifconfig | head -2 | tail -1 | cut -d: -f2 | rev | cut -c8-23 | rev)
sed -i -e "1i${IPADDR} ${1}" /etc/hosts
I also did the command in $(...) out of habit!
Refer to sed manual:https://www.gnu.org/software/sed/manual/sed.html
As a GNU extension, the i command and text can be separated into two
-e parameters, enabling easier scripting:
The formal usage should be:
sed -i -e "1i$IPADDR\\" -e "$1" /etc/hosts
I need to replace characters from 10th to 20th in the string which looks like that:
123456789012345678901234567890
So far I've tried:
a)
Works for the 10th character ONLY:
echo "123456789012345678901234567890" | sed 's/./X/10'
b)
Doesn't work on the range:
echo "123456789012345678901234567890" | sed 's/./X/10,20'
echo "123456789012345678901234567890" | sed 's/./X/10\,20'
echo "123456789012345678901234567890" | sed 's/./X/\{10,20\}'
echo "123456789012345678901234567890" | sed 's/./X/\{10\,20\}'
Does not work and I get error
unknown option to `s'
So - the question is - how do I make this to work:
echo "123456789012345678901234567890" | sed 's/./X/10,20'
Try:
$ sed -r "s/^(.{9})(.{11})/\1XXXXXXXXXX/" <<< 123456789012345678901234567890
123456789XXXXXXXXXX1234567890
It is a complex sed problem, I could just find this solution:
$ sed 's/^\(.\{10\}\)\(.\{10\}\)/\1XXXXXXXXXX/' <<< 123456789012345678901234567890
1234567890XXXXXXXXXX1234567890
With awk it looks nicer:
$ awk 'BEGIN{FS=OFS=""} {for (i=10;i<=20;i++) $i="X"} {print}' <<< 123456789012345678901234567890
123456789XXXXXXXXXXX1234567890
You can do it with bash parameter substitution like this:
#!/bin/bash
s="123456789012345678901234567890"
l=${s:0:9} # Extract left part
m=${s:10:11} # Extract middle part
r=${s:20} # Extract right part
# Diddle with middle part to your heart's content and re-assemble "$l$m$r" when done
m=$(sed 's/./X/g' <<<$m)
See here for more explanation and examples.
Or, you can do this:
transform the row of letters into a column so each is on its own line
apply your edits to LINES 10 through 20 (as opposed to characters 10 through 20)
transform column of letters back into a row (by deleting linefeeds)
as shown in the one-liner below:
$ echo "123456789012345678901234567890" | sed "s/\(.\)/\1\n/g" | sed "10,20s/./X/" | tr -d "\n"
I know, that it looks ugly, but:
echo "123456789012345678901234567890" | \
sed 's/^\(.\{10\}\).\{10\}\(.*\)/\1XXXXXXXXXX\2/'
Without placing multiple X in sed command:
sed -r 's/^(.{9})(.{10,20})(.*)$/\1\n\2\n\3/' | sed -e '2s/./X/g' -e 'N;N;s/\n//g'
To replace the 10th to 20th characters, inclusive, try:
echo 123456789012345678901234567890 | sed 's/\(.\{9\}\).\{11\}/\1XXXXXXXXXX/'
123456789XXXXXXXXXX1234567890
With the GNU sed, you can use the -r switch to remove most of the backslashes:
echo 123456789012345678901234567890 | sed -r 's/(.{9}).{11}/\1XXXXXXXXXX/'
Or the naive approach also works here:
echo 123456789012345678901234567890 | sed 's/\(.........\).........../\1XXXXXXXXXX/'
This might work for you (GNU sed):
sed ':a;/.\{9\}X\{11\}/!s/\(.\{9\}X*\)./\1X/;ta' file
or with a bit of syntactic sugar:
sed -r ':a;/.{9}X{11}/!s/(.{9}X*)./\1X/;ta' file
Could anybody suggest a reg ex for checking versions
What if I like to check any update version in 1.0.0 release may be 1.0.0-1 or 1.0.0-2 or 1.0.0-3 and I just need to check for what update version is it -1 or -2 or -3 1.0.0-1 or 1.0.0-2 or 1.0.0-3 what regex can I use ?
tried
sed -ne s/1.0.0\-[0-9]\1/p
Thanks !
Perhaps something like this?:
$ echo "1.0.0-1" | sed -ne 's/1.0.0\(\-[0-9]\)/\1/p'
-1
The regex to get the update number for any version would be:
$ echo "1.0.0-1" | sed -ne 's/[0-9]\+\.[0-9]\+\.[0-9]\+\(\-[0-9]\)/\1/p'
-1
The trick is to capture the output you want between \( and \) grouping parentheses and the use \1 to output the pattern in the replace part of the command.
You don't really need a regular expression here; just split the string on the correct character:
version=1.0.0-2
major=$(echo $version | cut -d. -f1 )
minor=$(echo $version | cut -d. -f1 )
patchlvl=$(echo $version | cut -d. -f1 | cut -d - -f1 )
build=$(echo $version | cut -d - -f2)
This will leave only the update version number:
echo "1.0.0-1" | sed -e 's/^[1-9][0-9.]*-//'
And this will leave the leading '-' on it:
echo "1.0.0-1" | sed -e 's/^[1-9][0-9.]*//'
The p command isn't needed in the above so I left it, and the -n option off, but you can restore them if this part of some more elaborate sed script.
Edit: OP apparently just wants to know if the version contains an update. Here are a couple of ways in classic bourne shell:
#!/bin/sh
version='1.0.0-1'
case "$version" in
*-[1-9]*) # Not a regex, this is a (file) "pattern"
# Update triggered code here
;;
esac
# Or:
if echo "$version" | grep '-[1-9][0-9]*$' >/dev/null ; then
# Update triggered code here
fi
There are other ways to do it in other shells/environments of course.