I currently am trying to get specific parameters from a url.
My url looks like: https://private.io/report-artifact/dsop-pipeline-artifacts/container-scan-reports/redhat/ubi/ubi7/7.8/2020-02-14T222203.548_2868/ubi7-7.8.tar
I want just redhat/ubi/ubi7/7.8
I can get redhat/ubi/ubi7/7.8/2020-02-14T222203.548_2868/ubi7-7.8.tar by doing,
echo https://private.io/report-artifact/dsop-pipeline-artifacts/container-scan-reports/redhat/ubi/ubi7/7.8/2020-02-14T222203.548_2868/ubi7-7.8.tar | sed 's|.*/container-scan-reports/||'
Thus I want to remove /2020-02-14T222203.548_2868/ubi7-7.8.tar
I also would like to change the / to a - so that I have redhat-ubi-ubi7-7.8
With GNU sed:
Get the 4 following path elements after .*/container-scan-reports/ and replace all / with -:
url='https://private.io/report-artifact/dsop-pipeline-artifacts/container-scan-reports/redhat/ubi/ubi7/7.8/2020-02-14T222203.548_2868/ubi7-7.8.tar'
echo "$url" | sed -E 's|.*/container-scan-reports/(([^/]*/){3}[^/]*).*|\1|;s|/|-|g'
Or you could get everything after .*/container-scan-reports/, but not the last two path elements:
echo "$url" | sed -E 's|.*/container-scan-reports/(.*)/[^/]*/[^/]*|\1|;s|/|-|g'
When you know the position in the string you can use cut
echo "${string}" | cut -d/ -f 7-10 | tr '/' '-'
Another way with sed is
echo "${string}" | sed -E 's#([^/]*/){6}([^/]*)/([^/]*)/([^/]*)/([^/]*).*#\2-\3-\4-\5#'
I have a bash variable, some file path (with spaces) and filename, e.g:
$ echo $tmp
/home/xyz/some/path/with spaces/AlbumArt_{random-number-sequence}_Large.jpg
When I attempt to identify the filename part with grep, e.g:
$ echo "$tmp" | egrep 'AlbumArt.*Large.jpe?g$'
/home/xyz/some/path/with spaces/**AlbumArt_{random-number-sequence}_Large.jpg**
The filename part appears to be identified correctly, but when I attempt to convert this to a sed substitution expression, e.g:
$ echo "$tmp" | sed 's#AlbumArt.*Large.jpe?g$#NewString#'
/home/xyz/some/path/with spaces/AlbumArt_{random-number-sequence}_Large.jpg
The expected substitution isn't happening. Thanks in advance for any help.
In fact egrep is a variant of grep -E, allowing to 'activate' extended regular expression (you can see: https://en.wikipedia.org/wiki/Regular_expression#Standards).
Thus, you just need to use the same option with sed:
echo "$tmp" | sed -E 's#AlbumArt.*Large.jpe?g$#NewString#'
I have text like:
TEXT="I need to replace the hostname [[google.com]] with it's ip in side the text"
Is there a way to use something like below, but working?
sed -Ee "s/\[\[(.*)\]\]/`host -t A \1 | rev | cut -d " " -f1 | rev`/g" <<< $TEXT
looks like the value of \1 is not being passed to the shell command used inside sed.
Thanks
Backquote interpolation is performed by the shell, not by sed. This means that your backquotes will either be replaced by the output of a command before the sed command is run, or (if you correctly quote them) they will not be replaced at all, and sed will see the backquotes.
You appear to be trying to have sed perform a replacement, then have the shell perform backquote interpolation.
You can get the backquotes past the shell by quoting them properly:
$ echo "" | sed -e 's/^/`hostname`/'
`hostname`
However, in that case you will have to use the resulting string in a shell command line to cause backquote interpolation again.
Depending on how you feel about awk, perl, or python, I'd suggest you use one of them to do this job in a single pass. Alternatively, you could make a first pass extracting the hostnames into a command without backquotes, then execute the commands to get the IP addresses you want, then replace them in another pass.
It's got to be a two part command, one to get a variable that bash can use, the other to do a straight-up /s/ replacement with sed.
TEXT="I need to replace the hostname [[google.com]] with it's ip in side the text"
DOMAIN=$(echo $TEXT | sed -e 's/^.*\[\[//' -e 's/\]\].*$//')
echo $TEXT | sed -e 's/\[\[.*\]\]/'$(host -tA $DOMAIN | rev | cut -d " " -f1 | rev)'/'
But, more cleanly using how to split a string in shell and get the last field
TEXT="I need to replace the hostname [[google.com]] with it's ip in side the text"
DOMAIN=$(echo $TEXT | sed -e 's/^.*\[\[//' -e 's/\]\].*$//')
HOSTLOOKUP=$(host -tA $DOMAIN)
echo $TEXT | sed -e 's/\[\[.*\]\]/'${HOSTLOOKUP##* }/
The short version is that you can't mix sed and bash the way you're expecting to.
This works:
#!/bin/bash
txt="I need to replace the hostname [[google.com]] with it's ip in side the text"
host_name=$(sed -E 's/^[^[]*\[\[//; s/^(.*)\]\].*$/\1/' <<<"$txt")
ip_addr=$(host -tA "$host_name" | sed -E 's/.* ([0-9.]*)$/\1/')
echo "$txt" | sed -E 's/\[\[.*\]\]/'"$ip_addr/"
# I need to replace the hostname 172.217.4.174 with it's ip in side the text
Thank you all,
I made the below solution:
function host_to_ip () {
echo $(host -t A $1 | head -n 1 | rev | cut -d" " -f1 | rev)
}
function resolve_hosts () {
local host_placeholders=$(grep -o -e "##.*##" $1)
for HOST in ${host_placeholders[#]}
do
sed -i -e "s/$HOST/$(host_to_ip $(sed -Ee 's/##(.*)##/\1/g' <<< $HOST))/g" $1
done
}
Where resolve_hosts gets a text file as an argument
I'm am trying to replace a series of asterix symbols in a text file with a -999.9 using sed. However I can't figure out how to properly escape the wildcard symbol.
e.g.
$ echo "2006.0,1.0,************,-5.0" | sed 's/************/-999.9/g'
sed: 1: "s/************/-999.9/g": RE error: repetition-operator operand invalid
Doesn't work. And
$ echo "2006.0,1.0,************,-5.0" | sed 's/[************]/-999.9/g'
2006.0,1.0,-999.9-999.9-999.9-999.9-999.9-999.9-999.9-999.9-999.9-999.9-999.9-999.9,-5.0
puts a -999.9 for every * which isn't what I intended either.
Thanks!
Use this:
echo "2006.0,1.0,************,-5.0" | sed 's/[*]\+/-999.9/g'
Test:
$ echo "2006.0,1.0,************,-5.0" | sed 's/[*]\+/-999.9/g'
2006.0,1.0,-999.9,-5.0
Any of these (and more) is a regexp that will modify that line as you want:
$ echo "2006.0,1.0,************,-5.0" | sed 's/\*\**/999.9/g'
2006.0,1.0,999.9,-5.0
$ echo "2006.0,1.0,************,-5.0" | sed 's/\*\+/999.9/g'
2006.0,1.0,999.9,-5.0
$ echo "2006.0,1.0,************,-5.0" | sed -r 's/\*+/999.9/g'
2006.0,1.0,999.9,-5.0
$ echo "2006.0,1.0,************,-5.0" | sed 's/\*\{12\}/999.9/g'
2006.0,1.0,999.9,-5.0
$ echo "2006.0,1.0,************,-5.0" | sed -r 's/\*{12}/999.9/g'
2006.0,1.0,999.9,-5.0
$ echo "2006.0,1.0,************,-5.0" | sed 's/\*\{1,\}/999.9/g'
2006.0,1.0,999.9,-5.0
$ echo "2006.0,1.0,************,-5.0" | sed -r 's/\*{1,}/999.9/g'
2006.0,1.0,999.9,-5.0
sed operates on regular expressions, not strings, so you need to learn regular expression syntax if you're going to use sed and in particular the difference between BREs (which sed uses by default) and EREs (which some seds can be told to use instead) and PCREs (which sed never uses but some other tools and "regexp checkers" do). Only the first solution above is a BRE that will work on all seds on all platforms. Google is your friend.
* is a regex symbol that needs to be escaped.
You can even use BASH string replacement:
s="2006.0,1.0,************,-5.0"
echo "${s/\**,/-999.9,}"
2006.0,1.0,-999.9,-5.0
Using sed:
sed 's/\*\+/999.9/g' <<< "$s"
2006.0,1.0,999.9,-5.0
Ya, * are special meta character which repeats the previous token zero or more times. Escape * in-order to match literal * characters.
sed 's/\*\*\*\*\*\*\*\*\*\*\*\*/-999.9/g'
When this possibility was introduced into gawk I have no idea!
gawk -F, '{sub(/************/,"-999.9",$3)}1' OFS=, file
2006.0,1.0,-999.9,-5.0
I am using sed with grep command to replace a string. Old string is in 8 files at home location and I want to replace all of these with new string. I am using this:
#! /bin/bash
read oldstring
read newstring
sed -i -e 's/'Soldstring'/'$newstring'/' grep "$oldstring" /home/*
Now this command works but I am getting an warning:
sed: can't read grep: No such file or directory
sed: can't read oldstring: No such file or directory
Any ideas?
You probably wanted
sed -i -e "s|Soldstring|$newstring|" $(grep -l "$oldstring" /home/*)
However that form is unsafe. Better use xargs:
grep -l "$oldstring" /home/* | xargs sed -i -e "s|Soldstring|$newstring|"
And another if possible is to store on arrays:
readarray -t files < <(exec grep -l "$oldstring" /home/*)
sed -i -e "s|Soldstring|$newstring|" "${files[#]}"
You are not executing grep, you are giving it as a parameter to sed.
are you missing backticks?
sed -i -e 's/'Soldstring'/'$newstring'/' `grep "$oldstring" /home/*`
sed -i -e "s/$oldstring/$newstring/g" `grep -l "$oldstring" /home/*`
Just in order to clearly point out the various typos in your code:
#! /bin/bash
# ^
# extra space here (not really an error I think -- but unusual)
read oldstring
read newstring
sed -i -e 's/'Soldstring'/'$newstring'/' grep "$oldstring" /home/*
# ^ ^ ^
# `S` instead of `$` here | |
# here and there
# missing backticks (`)
As a side note, I suggest backticks above, but, since you are using bash, the syntax $(grep ....) is probably better than the classic Bourne Shell syntax `grep ....`. Finally, as suggested by konsolebox, "command nesting" might be unsafe, for example, in this case, if some file names contain spaces.