sed regular expression does not work as expected. Differs on pipe and file - regex

I have a string in text file where i want to replace the version number. Quotation marks can vary from ' to ". Also spaces around = can be there and can be not as well:
$data['MODULEXXX_VERSION'] = "1.0.0";
For testing i use
echo "_VERSION'] = \"1.1.1\"" | sed "s/\(_VERSION.*\)[1-9]\.[1-9]\.[1-9]/\11.1.2/"
which works perfectly.
When i change it to search in the file (the file has the same string):
sed "s/\(_VERSION.*\)[1-9]\.[1-9]\.[1-9]/\11.1.2/" -i test.php
, it does not find anything.
After after playing with the search part of regex, i found one more odd thing:
sed "s/\(_VERSION.*\)[1-9]\./\1***/" -i test.php
works and changes the string to $data['MODULEXXX_VERSION'] = "***0.0";, but
sed "s/\(_VERSION.*\)[1-9]\.[1-9]/\1***/" -i test.php
does not find anything anymore. Why?
I am using Ubuntu 17.04 desktop.
Anyone can explain what am I doing wrong? What would be the best command for replacing version numbers in the file for the string $data['MODULEXXX_VERSION'] = "***0.0";?

The main problem is that [1-9] doesn't match the 0s in the version number. You need to use [0-9].
Besides that, you may use the following sed command:
sed -r 's/(.*_VERSION['\''"]]\s*=\s*).*/\1"1.0.1";/' conf.php
This doesn't look at the current value, it simply replaces everything after the =.
I've used -r which enables extended posix regular expressions which makes it a bit simpler to formulate the pattern.
Another, probably cleaner attempt is to store the conf.php as a template like conf.php.tpl and then use a template engine to render the file. Or if you really want to use sed, the file may look like:
$data['FOO_VERSION'] = "FOO_VERSION_TPL";
Then just use:
sed 's/FOO_VERSION_TPL/1.0.1/' conf.php.tpl > conf.php
If there are multiple values to replace:
sed \
-e 's/FOO/BAR/' \
-e 's/HELLO/WORLD/' \
conf.php.tpl > conf.php
But I recommend a template engine instead of sed. That becomes more important when the content of the variables to replace may contain characters special to regular expressions.

Related

sed : match all instances of regex in infile1.txt, and output only these to outfile2.txt

I have a text file infile1 with 1,000's of lines.
I wish to use sed to extract the occuring instances of a regex pattern match to outfile2.
NB
Each instance of the regex pattern match may occur more than once on each line of infile1.
Each instance of the extracted regex pattern should be printed to a new line in outfile2.
Does anyone know the syntax within sed to place the regex into?
ps the regex pattern is
\(Google[ ]{1,3}“[a-zA-Z0-9 ]{1,100}[., ]{0,3}”\)
Thank you :)
I think you want
grep -oE 'Google[ ]{1,3}"[a-zA-Z0-9 ]{1,100}[., ]{0,3}"' filename
-o tells grep to print only the matches, each on a line of its own, and -E instructs it to interpret the regex in extended POSIX syntax, which your regex appears to be.
Note that [ ] could be replaced with just a space, and you might want to use [[:alnum:] ] instead of [a-zA-Z0-9 ] to cover umlauts and suchlike if they exist in the current locale.
Addendum: It is also possible to do this with sed. I don't recommend it, but you could write (using GNU sed):
sed -rn 's/Google[ ]{1,3}"[A-Za-z0-9 ]{1,100}[., ]{0,3}"/\n&\n/g; s/[^\n]*\n([^\n]*\n)/\1/g; s/\n[^\n]*$//p' filename
To make this work with older versions of BSD sed, use -En instead of -rn. -r and -E enable extended regex syntax. -r was historically used by GNU sed, -E by BSD sed; newer versions of them support both for compatibility. -n disables auto-printing.
The code works as follows:
# mark all occurrences of the regex by circumscribing them with newlines
s/Google[ ]{1,3}"[A-Za-z0-9 ]{1,100}[., ]{0,3}"/\n&\n/g
# Isolate every other line from the pattern space (the matches). This will
# leave the part behind the last match...
s/[^\n]*\n([^\n]*\n)/\1/g
# ...so we remove it afterwards and print the result of the transformation if it
# happened (the s///p flag does that). The transformation will not happen if
# there were no matches in the line (because then no newlines will have been
# inserted), so in those cases nothing will be printed.
s/\n[^\n]*$//p
It can be done with sed too, but it isn't pretty:
sed -n ':start /foo/{ h; s/\(foo\).*/\1/; s/.*\(foo\)/\1/; p; g; s/foo\(.*\)/\1/; b start; }' infile1 >outfile2
-- provided that you replace the four occurences of foo above with your pattern Google {1,3}“[a-zA-Z0-9 ]{1,100}[., ]{0,3}”.
Yeah, I told you it isn't pretty. :)

sed replace exact match

I want to change some names in a file using sed. This is how the file looks like:
#! /bin/bash
SAMPLE="sample_name"
FULLSAMPLE="full_sample_name"
...
Now I only want to change sample_name & not full_sample_name using sed
I tried this
sed s/\<sample_name\>/sample_01/g ...
I thought \<> could be used to find an exact match, but when I use this, nothing is changed.
Adding '' helped to only change the sample_name. However there is another problem now: my situation was a bit more complicated than explained above since my sed command is embedded in a loop:
while read SAMPLE
do
name=$SAMPLE
sed -e 's/\<sample_name\>/$SAMPLE/g' /path/coverage.sh > path/new_coverage.sh
done < $1
So sample_name should be changed with the value attached to $SAMPLE. However when running the command sample_name is changed to $SAMPLE and not to the value attached to $SAMPLE.
I believe \< and \> work with gnu sed, you just need to quote the sed command:
sed -i.bak 's/\<sample_name\>/sample_01/g' file
In GNU sed, the following command works:
sed 's/\<sample_name\>/sample_01/' file
The only difference here is that I've enclosed the command in single quotes. Even when it is not necessary to quote a sed command, I see very little disadvantage to doing so (and it helps avoid these kinds of problems).
Another way of achieving what you want more portably is by adding the quotes to the pattern and replacement:
sed 's/"sample_name"/"sample_01"/' script.sh
Alternatively, the syntax you have proposed also works in GNU awk:
awk '{sub(/\<sample_name\>/, "sample_01")}1' file
If you want to use a variable in the replacement string, you will have to use double quotes instead of single, for example:
sed "s/\<sample_name\>/$var/" file
Variables are not expanded within single quotes, which is why you are getting the the name of your variable rather than its contents.
#user1987607
You can do this the following way:
sed s/"sample_name">/sample_01/g
where having "sample_name" in quotes " " matches the exact string value.
/g is for global replacement.
If "sample_name" occurs like this ifsample_name and you want to replace that as well
then you should use the following:
sed s/"sample_name ">/"sample_01 "/g
So that it replaces only the desired word. For example the above syntax will replace word "the" from a text file and not from words like thereby.
If you are interested in replacing only first occurence, then this would work fine
sed s/"sample_name"/sample_01/
Hope it helps

How can I use `sed` to replace the single quotes enclosing a directory with double quotes

What I want to achieve:
Suppose I have a file file with the following content:
ENV_VAR='/foo/`whoami`/bar/'
sh my_script.sh 'LOL'
I want to replace - using sed - the single quotes that surrounds the directory names, but not the ones that surrounds stuff that does not seem like a directory, for example, the arguments of a script.
That is, after running the sed command, I would expect the following output:
ENV_VAR="/foo/`whoami`/bar/"
sh my_script.sh 'LOL'
The idea is to make this happen without using tr to replace ' with ", nor sed like s/'/"/g, as I don't want to replace the lines that does not seem to be directories.
Please note that sed is running on AIX, so no GNU sed is available.
What I have tried:
If I use sed like this:
sed "s;'=.*/.*';&;g" file
... the & variable hold the regex previously matched, that is: ='/foo/`whoami`/bar/'. However, I can't figure out how to make the replacement so the single quotes gets transformed into double quotes.
I wonder if there's a way to make this work using sed only, via a one-liner.
This will do the job:
/usr/bin/sed -e "/='.*\/.*'/ s/'/\"/g" file
Basically, you just want the plain ' => " replacement, but not for all lines, just for those that match the pattern ='.*\/.*'/. And, in the s command you just need to escape the ".
This should work:
sed "s/'\(.*\/.*\)'/\"\1\"/g"
Captures the part between ' and uses a backreference.

search and replace substring in string in bash

I have the following task:
I have to replace several links, but only the links which ends with .do
Important: the files have also other links within, but they should stay untouched.
<li>Einstellungen verwalten</li>
to
<li>Einstellungen verwalten</li>
So I have to search for links with .do, take the part before and remember it for example as $a , replace the whole link with
<s:url action=' '/>
and past $a between the quotes.
I thought about sed, but sed as I know does only search a whole string and replace it complete.
I also tried bash Parameter Expansions in combination with sed but got severel problems with the quotes and the variables.
cat ./src/main/webapp/include/stoBox2.jsp | grep -e '<a href=".*\.do">' | while read a;
do
b=${a#*href=\"};
c=${b%.do*};
sed -i 's/href=\"$a.do\"/href=\"<s:url action=\'$a\'/>\"/g' ./src/main/webapp/include/stoBox2.jsp;
done;
any ideas ?
Thanks a lot.
sed -i sed 's#href="\(.*\)\.do"#href="<s:url action='"'\1'"'/>"#g' ./src/main/webapp/include/stoBox2.jsp
Use patterns with parentheses to get the link without .do, and here single and double quotes separate the sed command with 3 parts (but in fact join with one command) to escape the quotes in your text.
's#href="\(.*\)\.do"#href="<s:url action='
"'\1'"
'/>"#g'
parameters -i is used for modify your file derectly. If you don't want to do this just remove it. and save results to a tmp file with > tmp.
Try this one:
sed -i "s%\(href=\"\)\([^\"]\+\)\.do%\1<s:url action='\2'/>%g" \
./src/main/webapp/include/stoBox2.jsp;
You can capture patterns with parenthesis (\(,\)) and use it in the replacement pattern.
Here I catch a string without any " but preceding .do (\([^\"]\+\)\.do), and insert it without the .do suffix (\2).
There is a / in the second pattern, so I used %s to delimit expressions instead of traditional /.

Understand sed usage

I use sed to automaticaly update the version in my doxyfile using this :
sed -i -e "s/PROJECT_NUMBER.([ ]{2,}=.*)/PROJECT_NUMBER = $$VERSION/g" ".doxygen"
with $$VERSION = 1.1.0 (for example)
and as a source :
PROJECT_NUMBER = 1.0.10
But it generate an copy version of my .doxygen named .doxygen-e and don't change the line. I've tested my regex here.
I don't understand what's wrong given the fact that it works with my plist file using this :
sed -i -e "s/#VERSION#/$$VERSION/g" "./$${TARGET}.app/Contents/Info.plist"
There are a couple of problems here:
You need to refer to a shell variable $FOO as $$FOO in a Makefile. If you are attempting to do it in bash or any other shell, saying:
$$FOO
would result in the numeric PID of the current process concatenated with FOO, e.g. if the PID of the current process is 1234, then you'd get:
1234FOO
That said, your regex seems to be wrong on more than one count. You say:
PROJECT_NUMBER.([ ]{2,}=.*)
Since you are not using any option for sed that would enable the use of Extended Regular Expressions, this would match the string PROJECT_NUMBER, followed by one character, followed by (, followed by 2 or more whitespaces, an = sign, until it encounters the last ) in the string.
Since you haven't mentioned anything about how the line in the file looks like, I'd assume that it's of the form:
PROJECT_NUMBER = 42.42
The following might work for you:
sed 's/\(PROJECT_NUMBER[ ]*=[ ]*\)[^ ]*/\1$VERSION/' filename
If invoking from within a Makefile, you'd need to double the $.