I am trying to replace a string in the config file.
I would like to run something like this:
OS
(docker image php:8.1-apache-buster)
Debian GNU/Linux 10 (buster)
sed (GNU sed) 4.7 Packaged by Debian
Possible inputs:
post_max_size = 4M
post_max_size = 24M
post_max_size = 248M
...
Example output (any user given value):
post_max_size = 128M
Example cmd:
sed -i 's/(post_max_size = ([0-9]{1,})M/post_max_size = 128M/g' /usr/local/etc/php/php.ini
Joining regex with strings does not work here.
It works when I run string replace without any regex
sed -i 's/post_max_size = 8M/post_max_size = 128M/g' /usr/local/etc/php/php.ini
This works only if the value of the post_max_size is set exactly to 2M. I would like to be able to make a change with regex regardless of the value set.
I searched the Internet and sed cmd docs but did not find anything which fits my use case.
The following should work:
sed -i 's/^post_max_size = .*/post_max_size = 128M/g' /usr/local/etc/php/php.ini
You can match optional spaces with [[:space:]]*, the -E for extended-regexp and use group 1 noted as \1 followed by your replacement like \1128M
sed -E -i 's/(post_max_size[[:space:]]*=[[:space:]]*)[0-9]+M/\1128M/g' /usr/local/etc/php/php.ini
Related
I have a file that contains somehting like this:
[project]
name = "sinntelligence"
version = "1.1.dev12"
dependencies = [
"opencv-python",
"matplotlib",
"PySide6",
"numpy",
"numba"
]
Now I want to find the "version" string and increment the last number after "dev". Thus in the above example I would like to change
version = "1.1.dev12"
to
version = "1.1.dev13"
and so forth. With grep I was able to get this line with this regular expression:
grep -P "^version.*dev[0-9]+"
But since I want to replace something in a file I thought it would make more sense to use sed instead. However, with sed I don't even find that line (i.e. nothing is replaced) with this:
sed -i "s/^version.*dev[0-9]+/test/g" sed-test.txt
Any ideas 1) what I am doing wrong here with sed and 2) how can increase that "dev" number by one and write that back to the file (with just typical Ubuntu Linux command line tools)?
You used grep with -P option that enables the PCRE regex engine, and with sed you are using a POSIX BRE pattern. That is why you do not even match that line.
Then, with sed, you won't be able to easily eval and change the number, you can do that with perl:
perl -i -pe 's/^version.*dev\K(\d+)/$1 + 1/e' sed-test.txt
See the online demo:
#!/bin/bash
s='[project]
name = "sinntelligence"
version = "1.1.dev12"
dependencies = [
"opencv-python",
"matplotlib",
"PySide6",
"numpy",
"numba"
]'
perl -pe 's/^version.*dev\K(\d+)/$1 + 1/e' <<< "$s"
Output:
[project]
name = "sinntelligence"
version = "1.1.dev13"
dependencies = [
"opencv-python",
"matplotlib",
"PySide6",
"numpy",
"numba"
]
what I am doing wrong here with sed
You have to use -E option to enable extended regular expressions:
$ sed -E "s/^version.*dev[0-9]+/test/g" sed-test.txt
[project]
name = "sinntelligence"
test"
dependencies = [
"opencv-python",
"matplotlib",
"PySide6",
"numpy",
"numba"
]
how can increase that "dev" number by one and write that back to the
file (with just typical Ubuntu Linux command line tools)?
I'd use awk, below is the adaptation of solution in this Ed Morton's
answer:
awk -i inplace '/^version/ {split($3,lets,/[0-9]+"$/,digs); $3=lets[1] digs[1]+1 "\""} 1' sed-test.txt
I have a bunch of Jupyter Notebooks with equations written in LaTex. I know that I can convert the notebook to HTML as follows.
jupyter nbconvert --to html --template basic test.ipynb test.html
However, the LaTex markup are preserved. For example, if I have $y = w'x$, then that still shows up in the output HTML. I want to get this generated HTML into WordPress (basically, copy/paste), but WordPress delimits LaTex like this $latex y = w'x$.
How can I use perl or sed (or anything else) to convert $y = w'x to $latex y = w'x$?
I know I can just write a program to do it, but I think that's an overkill because I am sure these available command line tools can do it to. Additionally, any tool that is available on both Windows and Mac/Linux would be a bonus, since I work on both environment types, and do not want to have to resort to a *nix like environment to do this conversion (though, Windows does have Windows Linux Subsystem now, so I guess that might be ok if it's just a Linux tool).
I tried to modify this sed expression from this post (on Mac), but it did not work.
sed -e ' /\$\$/{s/\$\$/{\$latex }/;:a;N;/\$\$/!ba;s/\$\$/{\$}/};s/^\(\$\)\(.*\)\(\$\)$/{\$latex }\2{\$}/' test.html
unexpected EOF (pending }'s)
You can use the following sed command:
sed -e 's/\$\([^$]*\)\$/$\\latex \1$/g'
INPUT:
$ echo -e "abc \$y = w'x$ toto\n123 \$u = v'w$ xyz"
abc $y = w'x$ toto
123 $u = v'w$ xyz
OUTPUT:
echo -e "abc \$y = w'x$ toto\n123 \$u = v'w$ xyz" | sed -e 's/\$\([^$]*\)\$/$\\latex \1$/g'
abc $\latex y = w'x$ toto
123 $\latex u = v'w$ xyz
Explanations:
You use sed in find and replace mode and it will replace everything between two $ characters via this regex: \$\([^$]*\)\$ by what is already here (backreference) and add \latex at the beginning.
Last but not least, the following sed -e 's/\$\([^$]*\)\$/$latex \1$/g' sed command does the $latex replacement without the \
I am not sure what the whole format of your file might be. If I take your example at face value and assume that the first $ on any line is in column 1 then
sed -e's/^\$/$\\latex /'
works for me. But if the string to convert is anywhere in the file then things get harder.
I've been working on a script to update the PRODUCT_BUNDLE_IDENTIFIER in a pbxproj file with a new value using a build script. The regex I've come up with selects everything between 'PRODUCT_BUNDLE_IDENTIFIER = ' and any text following up to 2 occurrences of '.' which is what I want.
The regex I've put together to find these occurences is shown here:
(?<=PRODUCT_BUNDLE_IDENTIFIER = )([a-zA-Z0-9_]+(?:\.[a-zA-Z0-9_]+){0,2})
I've tested it with a validator here: https://regex101.com/r/jUhJm7/1
To save time, here's a screenshot with the regex applied and the green portions selected as desired, so the regex seems to be working and recognizes the bundle id portion of the following examples as expected:
The issue I'm experiencing is that when using this regex with grep, grep -e, egrep, or sed it doesn't seem to be working in the same manner. I would like to use sed to run the string replacement and have tried the following methods to achieve this:
# variable definitions
BUNDLE_ID='mynew.bundle.id'
PBXFILE="$SRCROOT/myproject.xcodeproj/project.pbxproj"
# check if the test bundle id is currently in the file
if grep -Fq "REPLACEABLE_BUNDLE_ID" $PBXFILE; then
# this commented version works as expected as it's using simple string replacement
#sed -i '' "s/REPLACEABLE_BUNDLE_ID/$BUNDLE_ID/g" $PBXFILE
# these are the versions of the regex I've tried with sed #
# basic version working in validator & testing with sublime text regex engine
(?<=PRODUCT_BUNDLE_IDENTIFIER = )([a-zA-Z0-9_]+(?:\.[a-zA-Z0-9_]+){0,2})
# added extra parentheses around product id first portion
(?<=(PRODUCT_BUNDLE_IDENTIFIER = ))([a-zA-Z0-9_]+(?:\.[a-zA-Z0-9_]+){0,2})
# escaped version
\(?<=PRODUCT_BUNDLE_IDENTIFIER = \)\([a-zA-Z0-9_]+\(?:\.[a-zA-Z0-9_]+\){0,2}\)
# try replacing the current bundle id using the regex
sed -i -E '' "s/I put the regex here/$BUNDLE_ID/g" $PBXFILE
fi
I'm fairly new with regex and have not used sed before. I've read about extended regular expressions here: http://www.grymoire.com/Unix/Regular.html#uh-12 and feel like I'm just failing to put the pieces together properly.
Try this for GNU sed (use -E for unix):
$ sed -r "s/(PRODUCT_BUNDLE_IDENTIFIER = )[a-zA-Z0-9_]+(\.[a-zA-Z0-9_]+){0,2}/\1${BUNDLE_ID}/"
for example:
$ cat test.txt
PRODUCT_BUNDLE_IDENTIFIER = com.test.mybundle.keyboard;
PRODUCT_BUNDLE_IDENTIFIER = com.test.mybundle.iMessage;
PRODUCT_BUNDLE_IDENTIFIER = com.test;
PRODUCT_BUNDLE_IDENTIFIER = replaceable;
$ BUNDLE_ID='mynew.bundle.id'
$ sed -r "s/(PRODUCT_BUNDLE_IDENTIFIER = )[a-zA-Z0-9_]+(\.[a-zA-Z0-9_]+){0,2}/\1${BUNDLE_ID}/" test.txt
PRODUCT_BUNDLE_IDENTIFIER = mynew.bundle.id.keyboard;
PRODUCT_BUNDLE_IDENTIFIER = mynew.bundle.id.iMessage;
PRODUCT_BUNDLE_IDENTIFIER = mynew.bundle.id;
PRODUCT_BUNDLE_IDENTIFIER = mynew.bundle.id;
I have separate files include path string for each like ;
path = /aaa/bbb/ccc.com/user#ccc.com/dddd/user#yahoo.com/
path = /aaa/bbb/ccc.com/user#ccc.com/dddd/user#hotmail.co.uk/
path = /aaa/bbb/ccc.com/user#ccc.com/dddd/user#abc.xxx.co.uk/
path = /aaa/bbb/ccc.com/user#ccc.com/dddd/user55#ccc.com/
what i want to trim lines like;
path = /aaa/bbb/ccc.com/user/dddd/.user#yahoo/
path = /aaa/bbb/ccc.com/user/dddd/.user#hotmail/
path = /aaa/bbb/ccc.com/user/dddd/.user#abc/
path = /aaa/bbb/ccc.com/user/dddd/.user55#ccc.com/
I am almost be able to achieve with below (all strings are in separate files but at the 15th line)
sed -r '15s!#[^/]+(/[^/]+/[^.#]+#[^.]+).*$!\1/!g' $file
however, i have an issue with dot part that cuts it as ;
path = /aaa/bbb/ccc.com/user/dddd/user55#ccc/
instead, it should have been;
path = /aaa/bbb/ccc.com/user/dddd/.user55#ccc/
Thanks in advance,
Using a pattern with three capture groups should do what you need. The first group will capture the portion behind the initial # (as a group we omit from the replacement), the second group will include the /dddd/ portion, and the third being the complete user#somewhere with a prepended .
's!(#.+\..+)(/.+/)(.+#.+)!\2.\3!g'
Depending on your version of bash you could use it like this:
sed -i.bak -r 's!(#.+\..+)(/.+/)(.+#.+)!\2.\3!g' $file
↳ (GNU bash, version 4.1.2(1)-release (x86_64-redhat-linux-gnu)
sed -i bak -E 's!(#.+\..+)(/.+/)(.+#.+)!\2.\3!g' $file
↳ GNU bash, version 3.2.48(1)-release (x86_64-apple-darwin12)
result:
path = /aaa/bbb/ccc.com/user/dddd/.user#yahoo.com/
path = /aaa/bbb/ccc.com/user/dddd/.user#hotmail.co.uk/
path = /aaa/bbb/ccc.com/user/dddd/.user#abc.xxx.co.uk/
path = /aaa/bbb/ccc.com/user/dddd/.user55#ccc.com/
It's a little unclear if you want to keep the full extension on the end of the last match; if not sed is probably not the best choice because it can't do look-ahead, look-behind assertions, nor toggle greedy in any straight-forward manner. In the case that's a deal breaker, you could use this pattern on one of many other avenues:
(#.+\..+)(/.+/)(.+#.+?)(\..*/)
result:
path = /aaa/bbb/ccc.com/user/dddd/.user#yahoo
path = /aaa/bbb/ccc.com/user/dddd/.user#hotmail
path = /aaa/bbb/ccc.com/user/dddd/.user#abc
path = /aaa/bbb/ccc.com/user/dddd/.user55#ccc
You would have to use two matches:
sed -E 's/(.*?\..*?)\/(.*?)#\1/\1\/\2/g'
Regex: (.*?\..*?)\/(.*?)#\1
Replacement: \1\/\2
Flags: g (Global)
Result:
path = /aaa/bbb/ccc.com/user/dddd/user#yahoo.com/
path = /aaa/bbb/ccc.com/user/dddd/user#hotmail.co.uk/
path = /aaa/bbb/ccc.com/user/dddd/user#abc.xxx.co.uk/
path = /aaa/bbb/ccc.com/user/dddd/user55#ccc.com/
sed -E 's/(\w+#\w+)[\w\.]*/\1/g'
Regex: (\w+#\w+)[\w\.]*
Replacement: \1
Flags: g (Global)
Result:
path = /aaa/bbb/ccc.com/user/dddd/user#yahoo/
path = /aaa/bbb/ccc.com/user/dddd/user#hotmail/
path = /aaa/bbb/ccc.com/user/dddd/user#abc/
path = /aaa/bbb/ccc.com/user/dddd/user55#ccc/
If the -E switch is not available on your version of sed, then you might have to use perl.
Example:
perl -pe 's/(.*?\..*?)\/(.*?)#\1/\1\/\2/g' -i filename.ext
If I try this in bash, I get the following result:
root#home [~]# echo "path = /aaa/bbb/ccc.com/user#ccc.com/dddd/user55/" | sed -E 's/(.*?\..*?)\/(.*?)#\1/\1\/\2/g'
path = /aaa/bbb/ccc.com/user/dddd/user55/
root#home [~]# echo "path = /aaa/bbb/ccc.com/user/dddd/user55/" | sed -E 's/(\w+#\w+)[\w\.]*/\1/g'
path = /aaa/bbb/ccc.com/user/dddd/user55/
I have this existing pattern:
ethernet0.generatedAddress = "00:50:56:bf:71:06"
I need to replace a mac address in the above expression with a new mac address using a sed pattern.
Note: The mac-address that needs to replaced is not same everytime.
I tried this sed expression , but no luck..
sed 's/ethernet0.generatedAddress.*=.*\"([[:xdigit:]]{1,2}:){5}[[:xdigit:]]{1,2}/ethernet0.generatedAddress = \"00:16:3e:5e:1d:01'
Thanks in Advance
Pattern:
([a-z0-9]{2}:[a-z0-9]{2}:[a-z0-9]{2}:[a-z0-9]{2}:[a-z0-9]{2}:[a-z0-9]{2})
Or the following one if uppercase letters are used
([a-zA-Z0-9]{2}:[a-zA-Z0-9]{2}:[a-zA-Z0-9]{2}:[a-zA-Z0-9]{2}:[a-zA-Z0-9]{2}:[a-zA-Z0-9]{2})
Replacement:
new_mac_address // for instance 00:f6:a0:ff:f1:06
Side note: As has been pointer in the comments below, escape parentheses and curly brackets if needed or use -r option
Using sed it would be something like this (just tested)
sed -r 's/(.*)([a-zA-Z0-9]{2}:[a-zA-Z0-9]{2}:[a-zA-Z0-9]{2}:[a-zA-Z0-9]{2}:[a-zA-Z0-9]{2}:[a-zA-Z0-9]{2})(.*)/\1\NEW_MAC_ADDRESS\3/g' file.txt
Use -i option in addition if you want to replace the file on-the-fly
Content of the tested file (file.txt)
something before ethernet0.generatedAddress = "00:50:56:bf:71:06" and something after
Demo
Why not use awk? It gives simple and easy to understand solution.
cat file
some data
ethernet0.generatedAddress = "00:50:56:bf:71:06"
more data
mac="ab:11:23:55:11:cc"
awk -v m="$mac" -F' "|"' '/ethernet0.generatedAddress/ {$2="\""m"\""}1' file
some data
ethernet0.generatedAddress = "ab:11:23:55:11:cc"
more data
It search for ethernet0.generatedAddress, if found, replace field #2 separated by " with new mac.
If one extra space is not problem, this would be more clean:
awk -v m="$mac" -F\" '/ethernet0.generatedAddress/ {$2=FS m FS}1' file
some data
ethernet0.generatedAddress = "ab:11:23:55:11:cc"
more data
Or this:
awk -v m="\"$mac\"" -F\" '/ethernet0.generatedAddress/ {$2=m}1' file