I am attempting to use sed to replace a regular expression capture group. I've read that this requires enabling extended regular expressions with the -E flag. However, the following command is still not updating the text as expected.
echo "master-abcdef" | sed -i '' -E "s/IMAGE_TAG:\s*(\S+)$/\1/g" values.yaml
Where values.yaml has contents of:
global:
env: default
default:
IMAGE_TAG: dev-0be3323.zgm9a
... (more text below)
I am expecting values.yaml to be replaced to:
global:
env: default
default:
IMAGE_TAG: master-abcdef
... (more text below)
You may use this in any version sed:
sed -i.bak -E 's/(IMAGE_TAG:[[:blank:]]+)[^[:blank:]]+/\1master-abcdef/' file.yml
cat file.yml
global:
env: default
default:
IMAGE_TAG: master-abcdef
... (more text below)
Here [[:blank:]] matches a space or tab.
If you are using gnu-sed then use:
sed -i -E 's/(IMAGE_TAG:\s+)\S+/\1master-abcdef/' file.yml
With yq, you can write
yq eval '.default.IMAGE_TAG = "master-abcdef"' values.yaml
You can use
repl="master-abcdef"
sed -i '' -E "s/(IMAGE_TAG:[[:space:]]*).*/\\1$repl/" values.yaml
Here,
(IMAGE_TAG:[[:space:]]*) - captures into \1 an IMAGE_TAG: string and then any zero or more whitespaces
.* - matches the rest of the string (here, line).
The \1$repl replacement puts the captured value + the repl value in place of the matched text.
Related
I am trying to get a value from a JSON from my local server (https://regex101.com/r/qeGcGu/1) on a headless mac mini (catalina), via sed. However, with the sed command I'd expect to work:
usr#mcMini ~/Documents/qBitTorrent cat /tmp/json.out | sed -i.bak '"hash":"(.*?)"'
sed: 1: ""hash":"(.*?)"": invalid command code "
usr#mcMini ~/Documents/qBitTorrent cat /tmp/json.out | sed -i.bak '\"hash\":\"(.*?)\"'
sed: 1: "\"hash\":\"(.*?)\"": unterminated regular expression
usr#mcMini ~/Documents/qBitTorrent cat /tmp/json.out | sed -i '' '\"hash\":\"(.*?)\"'
sed: 1: "\"hash\":\"(.*?)\"": unterminated regular expression
usr#mcMini ~/Documents/qBitTorrent cat /tmp/json.out | sed -i '' '"hash":"(.*?)"'
sed: 1: ""hash":"(.*?)"": invalid command code "
The file that I am trying to get the string from is a raw json.
[{"added_on":1587102956,"amount_left":0,"auto_tmm":false,"availability":-1,"category":"radarr","completed":1218638934,"completion_on":1587108704,"dl_limit":-1,"dlspeed":0,"downloaded":1220894674,"downloaded_session":0,"eta":8640000,"f_l_piece_prio":false,"force_start":true,"hash":"87802183fc647548ec6efe18feb16149522f6aa0","last_activity":1587119220,"magnet_uri":"magnet:?xt=urn:btih:87802183fc647548ec6efe18feb16149522f6aa0&dn=Fantasia%202000%20(1999)%20%5b1080p%5d%20%5bYTS.AG%5d&tr=udp%3a%2f%2ftracker.coppersurfer.tk%3a6969%2fannounce&tr=udp%3a%2f%2f9.rarbg.com%3a2710%2fannounce&tr=udp%3a%2f%2fp4p.arenabg.com%3a1337&tr=udp%3a%2f%2ftracker.leechers-paradise.org%3a6969&tr=udp%3a%2f%2ftracker.internetwarriors.net%3a1337&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce&tr=udp%3a%2f%2ftracker.zer0day.to%3a1337%2fannounce&tr=udp%3a%2f%2ftracker.leechers-paradise.org%3a6969%2fannounce&tr=udp%3a%2f%2fcoppersurfer.tk%3a6969%2fannounce","max_ratio":-1,"max_seeding_time":-1,"name":"Fantasia 2000 (1999) [1080p] [YTS.AG]","num_complete":22,"num_incomplete":4,"num_leechs":0,"num_seeds":0,"priority":0,"progress":1,"ratio":0.1782183661159947,"ratio_limit":-2,"save_path":"/Volumes/1049/Media/","seeding_time_limit":-2,"seen_complete":1587118087,"seq_dl":false,"size":1218638934,"state":"forcedUP","super_seeding":false,"tags":"","time_active":13224,"total_size":1218638934,"tracker":"udp://tracker.coppersurfer.tk:6969/announce","up_limit":-1,"uploaded":217585854,"uploaded_session":128831791,"upspeed":0}]
Actually what I want to accomplish is to get the first 6 chars from hash:
"hash":"87802183fc647548ec6efe18feb16149522f6aa0"
In this case my desired value is 878021
Could you please guide me in the correct direction?
You may use
sed -n 's/.*"hash":"\([^"]*\).*/\1/p' /tmp/json.out
Here, note that the file can be provided directly to the sed command, no need piping it with cat.
How it works
-n - option that suppresses the default line output (by default, sed will output non-matching lines)
s/ - substitute command (we are replacing)
.*"hash":"\([^"]*\).* - matches
.* - 0+ chars
"hash":" - "hash":" substring
\([^"]*\) - Group 1 (capturing group, \1 is used in the replacement part to refer to this value) - any 0+ chars other than "
.* - 0+ chars
\1 - the replacement is Group 1 value (it is all that remains on the matching line)
p - if there was a valid replacement print the result after replacement only.
I am working on manipulating a string (BRANCH_NAME) and removing the front characters up to the forward slash. This is being used to change commit messages in git.
For example, the branch 'feature/pux-1234' is to be changed to pux-1234. There are different value that may exist, exclusive of each other. In this first attempt, I am checking for two values: feature/ and hotfix/.
Here is the code:
# Remove everything up to the first slash if it exists
if [[ $BRANCH_NAME == *"/"* ]]; then
PRETRIM=$(echo $BRANCH_NAME | sed -E 's/(?:(?<=feature\/)|(?<=hotfix\/)).+/' )
else
PRETRIM = $BRANCH_NAME
fi
The error I am receiving is this:
sed: 1: "s/(?:(?<=feature/)|(?< ...": RE error: repetition-operator operand invalid
Ideas to help resolve?
For example, the branch feature/pux-1234 is to be changed to pux-1234
You don't need a lookbehind here and anyway sed doesn't support look arounds.
You can use capture groups in sed to match all options in one group and replace with empty string:
s='feature/pux-1234'
sed -E 's~(feature|hotfix)/~~' <<< "$s"
pux-1234
and
s='hotfix/pux-1234'
sed -E 's~(feature|hotfix)/~~' <<< "$s"
pux-1234
Using extglob, you can can do this in bash itself:
shopt -s extglob
echo "${s/+(feature|hotfix)\/}"
pux-1234
Given a file with many ENV variables.
Example: DEV.env
TRUST_STORE=/run/secret/truststore.jks
#TRUST_STORE_PASSWORD=changeit
TRUST_STORE_PASSWORD=ch#nge1t
I know the command sed "s/^/export /g" will add export at the beginning of each line.
But I don't want to do this when the first character is "#".
How can I do with sed?
Something like this:
$ sed "s/^[^#]/export &/g"
export TRUST_STORE=/run/secret/truststore.jks
#TRUST_STORE_PASSWORD=changeit
export TRUST_STORE_PASSWORD=ch#nge1t
Alternatives:
$ sed "/^[^#]/ s/^/export /g"
$ sed "/^#/n; s/^/export /g" #n : read the next line
You may use a regex that will capture a char other than # or end of line into Group 1 and then restore the value captured with a \1 backreference
sed -r "s/^([^#]|$)/export \1/"
The -r option (also -E on some OSes) enables the POSIX ERE syntax in sed.
^ - matches start of a line
([^#]|$) - a capturing group #1 matching a char other than # (with [^#]) or (|) end of line ($).
The \1 holds the value captured with Group 1.
See the online demo.
You can use this sed:
sed -i.bak '/^[[:blank:]]*#/!s/^/export /' file
cat file
export TRUST_STORE=/run/secret/truststore.jks
#TRUST_STORE_PASSWORD=changeit
export TRUST_STORE_PASSWORD=ch#nge1t
Here:
/^[[:blank:]]*#/!: Do this only if we have # at line start following optional whitespaces
s/^/export /: Replace line start with export
echo "abc" | perl -ne 's/^(?!#)/export /g; print;'
No capturing needed.
I have lines like these
my_list=cloning/cloning-1.7.jar,commons/commons-lang-2.5.jar
my_lib_list=antlr/antlr-1.0.jar,aopa/aopa-1.0.jar
and I need to remove the part before '/' like this:
my_list=cloning-1.7.jar,commons-lang-2.5.jar
my_lib_list=antlr-1.0.jar,aopa-1.0.jar
I tried this
sed -i -e "s/(?<=\/).*?(\.jar)//g"
Nothing happens. Regex seems to be right (might need to be inverted), but atleast something should change in the file, right?
Your pattern - (?<=\/).*?(\.jar) - contains a lookbehind ((?<=...)) and lazy matching quantifier (*?). Neither are supported by sed.
You can use
sed -E 's/[[:alnum:]]+\///g'
See the IDEONE demo
Pattern details:
[[:alnum:]]+ - 1 or more alphanumeric symbols
\/ - a literal /
You can do:
sed -r 's#^([^=]+=)[^/]+/([^,]+,)[^/]+/(.*)#\1\2\3#'
Example:
$ sed -r 's#^([^=]+=)[^/]+/([^,]+,)[^/]+/(.*)#\1\2\3#' <<<'my_list=cloning/cloning-1.7.jar,commons/commons-lang-2.5.jar'
my_list=cloning-1.7.jar,commons-lang-2.5.jar
$ sed -r 's#^([^=]+=)[^/]+/([^,]+,)[^/]+/(.*)#\1\2\3#' <<<'my_lib_list=antlr/antlr-1.0.jar,aopa/aopa-1.0.jar'
my_lib_list=antlr-1.0.jar,aopa-1.0.jar
I have a bash script that I copy and run on both linux and AIX servers.
This script gets a "name" parameter which represents a file name, and I need to manipulate this name via regex (the purpose is irrelevant and very hard to explain).
From the name parameter I need to take the beginning until the first "-" character that is followed by a digit, and then concat it with the last "." character until the end of the string.
For example:
name: abcd-efg-1.23.4567-8.jar will become: abcd-efg.jar
name: abc123-abc3.jar will remain: abc123-abc3.jar
name: abc-890.jar will become: abc.jar
I've tried several variations of:
name=$1
regExpr="^(.*?)-\d.*\.(.*?)$/g"
echo $name
echo $(printf ${name} | sed -e $regExpr)
Also I cant use sed -r (seen on some examples) because AIX sed does not support the -r flag.
The last line is the problem of course; I think I need to somehow use $1 + $2 placeholders, but I can't seem to get it right.
How can I change my regex so that it does what I want?
Given the file:
abcd-efg-1.23.4567-8.jar
abc123-abc3.jar
abc-890.jar
This is a way to change the names you give:
$ sed 's/\(.\?\)-[0-9].*\(\.[a-z]*\)$/\1\2/' file
abcd-efg.jar
abc123-abc3.jar
abc.jar
Which is equivalent to (if you could use -r):
$ sed -r 's/(.?)-[0-9].*(\.[a-z]*)$/\1\2/' file
abcd-efg.jar
abc123-abc3.jar
abc.jar
It gets everything up to - + digit and "stores" in \1.
It gets from last . + letters and "stores" in \2.
Finally it prints those blocks back.
Note the extension could also be fetched with the basename builtin or with something like `"${line##*.}".
You could use this:
perl -F'(-(?:\d)|\.)' -ane 'print "$F[0].$F[$#F]"'
It splits the input on any - followed by a digit, or any .. Then it prints the first field, followed by a dot, followed by the last field.
Testing it out:
$ cat file
abcd-efg-1.23.4567-8.jar
abc123-abc3.jar
abc-890.jar
$ perl -F'(-(?:\d)|\.)' -ane 'print "$F[0].$F[$#F]"' file
abcd-efg.jar
abc123-abc3.jar
abc.jar
In sed, you could simply use the following.
#!/bin/sh
STRING=$( cat <<EOF
abcd-efg-1.23.4567-8.jar
abc123-abc3.jar
abc-890.jar
EOF
)
echo "$STRING" | sed 's/-[0-9].*\(\.[^.]\+\)$/\1/'
# abcd-efg.jar
# abc123-abc3.jar
# abc.jar
This matches a hyphen followed by a number and everything after and replaces with the file extension.
Or you may consider using a Perl one-liner:
echo "$STRING" | perl -pe 's/-\d.*(?=\.[^.]+$)//'
# abcd-efg.jar
# abc123-abc3.jar
# abc.jar
when a successful regex match is made, perl will capture whatever is matched in parenthesis ( .. ) as $1, $2, etc.
$ perl -e 'my $arg = $ARGV[0]; $arg =~ /^(.*?)-\d.*\.(.*?)$/; print "$1.$2\n"; ' abc-890.jar
abc.jar