search and replace files in linux(sed) - regex

I'm trying to search and replace the following:
<?php
<!DOCTYPE HTML>
with
<!DOCTYPE HTML>
so far I have tried this:
find . \( -name "*.php" \) -exec grep -Hn "<?php <\!DOCTYPE HTML>" {} \; -exec sed -i 's/<?php <\!DOCTYPE HTML>/<\!DOCTYPE HTML>/g' {} \;
But it's not finding any instances of files with my needle string which exists on my server.

find . -name "*.php" -exec grep -lZz '^<?php[[:space:]]\+<!DOCTYPE HTML>' {} + |
xargs -r0 sed -i '^<?php[[:space:]]*$/,1d'
Edit: The previous version didn't work due to the character \n in the pattern. The updated version avoid this character.

With GNU awk (for RS='\0' to read the whole file as one record) and assuming your file names don't contain newlines all you need is the clear, simple:
find . -name '*.php' -print |
while IFS= read -r file; do
gawk -v RS='\0' '{gsub(/<\?php\n<!DOCTYPE HTML>/,"<!DOCTYPE HTML>"); print}' "$file" > tmp &&
mv tmp "$file"
done

Related

SED: replace semvers of multiple files

[context] My script needs to replace semvers of multiple .car names with commit sha. In short, I would like that every dev_CA_1.0.0.car became dev_CA_6a8zt5d832.car
ADDING commit sha right before .car was pretty trivial. With this, I end up with dev_CA_1.0.0_6a8zt5d832.car
find . -depth -name "*.car" -exec sh -c 'f="{}"; \
mv -- "$f" $(echo $f | sed -e 's/.car/_${CI_COMMIT_SHORT_SHA}.car/g')' \;
But I find it incredibly difficult to REPLACE. What aspect of sed am I misconceiving trying this:
find . -depth -name "*.car" -exec sh -c 'f="{}"; \
mv -- "$f" $(echo $f | sed -r -E 's/[0-9\.]+.car/${CI_COMMIT_SHORT_SHA}.car/g')
or this
find . -depth -name "*.car" -exec sh -c 'f="{}"; \
mv -- "$f" $(echo $f | sed -r -E 's/^(.*_)[0-9\.]+\.car/\1${CI_COMMIT_SHORT_SHA}\.car/g')' \;
no matches found: f="{}"; mv -- "$f" $(echo $f | sed -r -E ^(.*_)[0-9.]+.car/1684981321531.car/g)
or multiple variants:
\ escaping (e.g. \\.)
( and ) escaping (e.g. \() (I read somewhere that regex grouping with sed requires some care with ())
Is there a more direct way to do it?
Edit
$f getting in sed are path looking like
./somewhere/some_project_CA_1.2.3.car
./somewhere_else/other_project_CE_9.2.3.car
You may use
sed "s/_[0-9.]\{1,\}\.car$/_${CI_COMMIT_SHORT_SHA}.car/g"
See the online demo
Here, sed is used with a POSIX ERE expression, that matches
_ - an underscore
[0-9.]\{1,\} - 1 or more digits or dots
\.car - .car (note that a literal . must be escaped! a . pattern matches any char)
$ - end of string.
Can you try this :
export CI_COMMIT_SHORT_SHA=6a8zt5d832
find . -depth -name "*.car" -exec sh -c \
'f="{}"; echo mv "$f" "${f%_*}_${CI_COMMIT_SHORT_SHA}.car"' \;
Remove echo once you are satisfied of the result.

Nesting Find and sed command in if else

I am using a find and sed command to replace characters in a file. see the code 1 below
find . -type f -exec sed -i '/Subject/{:a;s/(Subject.*)Subject/\1SecondSubject/;tb;N;ba;:b}' {} +
Given that I have multiple files I need to replace. In a given situation, the Subject I am trying to replace is not available.
Is there a way I can first check if the file contains the attribute 'Subject' if not I need to execute another command. i.e
Check if the file contains character 'Subject'
If true then execute code1 above
If there is no instance of Subject execute code 2 below
find . -name "*.html" -exec rename 's/.html$/.xml/' {} ;
Any Ideas? Thanks in advance
Something like this should work.
find . -type f \( \
-exec grep -q "Subject" {} \; \
-exec sed -i '/Subject/{:a;s/(Subject.*)Subject/\1SecondSubject/;tb;N;ba;:b}' {} \; \
-o \
-exec rename 's/.html$/.xml/' {} \; \)
-exec takes the exit code of the command it executes, so -exec grep -q "Subject" {} \; will only be true if the grep is true. And since the short circuit -o (or) has a lower precedence than the implied -a (and) between the other operators it should conversely only get executed if the grep is false.
You can use find in a process substitution like this:
while IFS= read -d'' -r file; do
echo "processing $file ..."
if grep -q "/Subject/" "$file"; then
sed -i '{:a;s/(Subject.*)Subject/\1SecondSubject/;tb;N;ba;:b}' "$file"
else if [[ $file == *.html ]]; then
rename 's/.html$/.xml/' "$file"
fi
done < <(find . -type f -print0)

need to rename many files in directory using sed and find

I would like to rename all files named *-6.0.dll with *-6.1.dll
I tried:
find . -name '*-6.0.dll*' -exec mv {} $(echo {} | sed -e 's/-6.0.dll/-6.1.dll/g') \;
but this didn't work; the file names didn't change.
Any ideas?
for x in *-6.0.dll; do y=$(echo $x | sed -e 's/-6\.0\.dll$/-6.1.dll/'); echo mv $x $y; done
Remove the echo once you are satisfied the results are correct.
use this:
find . -name '*-6.0.dll*' -exec sh -c 'mv {} $(echo {} | sed -e 's/\-6\.0\.dll/\-6\.1\.dll/g')' \;
an explanation of using the sh -c vs mv can be found here http://linuxplayer.org/2010/05/shell-programming-trap-batch-rename-with-find
I also modified your regex, some of the characters need to be escaped for proper matching.

How to match once per file in grep?

Is there any grep option that let's me control total number of matches but stops at first match on each file?
Example:
If I do this grep -ri --include '*.coffee' 're' . I get this:
./app.coffee:express = require 'express'
./app.coffee:passport = require 'passport'
./app.coffee:BrowserIDStrategy = require('passport-browserid').Strategy
./app.coffee:app = express()
./config.coffee: session_secret: 'nyan cat'
And if I do grep -ri -m2 --include '*.coffee' 're' ., I get this:
./app.coffee:config = require './config'
./app.coffee:passport = require 'passport'
But, what I really want is this output:
./app.coffee:express = require 'express'
./config.coffee: session_secret: 'nyan cat'
Doing -m1 does not work as I get this for grep -ri -m1 --include '*.coffee' 're' .
./app.coffee:express = require 'express'
Tried not using grep e.g. this find . -name '*.coffee' -exec awk '/re/ {print;exit}' {} \; produced:
config = require './config'
session_secret: 'nyan cat'
UPDATE: As noted below the GNU grep -m option treats counts per file whereas -m for BSD grep treats it as global match count
So, using grep, you just need the option -l, --files-with-matches.
All those answers about find, awk or shell scripts are away from the question.
I think you can just do something like
grep -ri -m1 --include '*.coffee' 're' . | head -n 2
to e.g. pick the first match from each file, and pick at most two matches total.
Note that this requires your grep to treat -m as a per-file match limit; GNU grep does do this, but BSD grep apparently treats it as a global match limit.
I would do this in awk instead.
find . -name \*.coffee -exec awk '/re/ {print FILENAME ":" $0;exit}' {} \;
If you didn't need to recurse, you could just do it with awk:
awk '/re/ {print FILENAME ":" $0;nextfile}' *.coffee
Or, if you're using a current enough bash, you can use globstar:
shopt -s globstar
awk '/re/ {print FILENAME ":" $0;nextfile}' **/*.coffee
using find and xargs.
find every .coffee files and excute -m1 grep to each of them
find . -print0 -name '*.coffee'|xargs -0 grep -m1 -ri 're'
test
without -m1
linux# find . -name '*.txt'|xargs grep -ri 'oyss'
./test1.txt:oyss
./test1.txt:oyss1
./test1.txt:oyss2
./test2.txt:oyss1
./test2.txt:oyss2
./test2.txt:oyss3
add -m1
linux# find . -name '*.txt'|xargs grep -m1 -ri 'oyss'
./test1.txt:oyss
./test2.txt:oyss1
find . -name \*.coffee -exec grep -m1 -i 're' {} \;
find's -exec option runs the command once for each matched file (unless you use + instead of \;, which makes it act like xargs).
You can do this easily in perl, and no messy cross platform issues!
use strict;
use warnings;
use autodie;
my $match = shift;
# Compile the match so it will run faster
my $match_re = qr{$match};
FILES: for my $file (#ARGV) {
open my $fh, "<", $file;
FILE: while(my $line = <$fh>) {
chomp $line;
if( $line =~ $match_re ) {
print "$file: $line\n";
last FILE;
}
}
}
The only difference is you have to use Perl style regular expressions instead of GNU style. They're not much different.
You can do the recursive part in Perl using File::Find, or use find feed it files.
find /some/path -name '*.coffee' -print0 | xargs -0 perl /path/to/your/program

Find and replace using xargs sed - can I save the results to a file?

My code to replace all instances of 'foo' with 'bar' :
find . -type f |
xargs grep 'foo' -l |
xargs sed -i 's|foo|bar|g'
I'd like to save a list of the modified files to a text document. Is it possible?
EDIT :
This is the final code that worked for me :
find . -type f -print0 |
xargs -0 grep 'foo' -l |
tee result.txt |
xargs -0 sed -i 's|foo|bar|g'
Not sure whether this is the quickest way, but for a few thousand files the difference in speed between this and other suggested methods is probably very small.
Looks like a useless use of xargs, as often in combination with find.
find . -type f -exec grep 'foo' -l {} ";" -exec sed -i 's|foo|bar|g' {} ";" -ls > file.lst
Use it with care, since I didn't test it. I'm not sure, whether you like to change the list of filenames, or the file content. Since you search with grep and sed, I think only working with sed should be sufficient:
find . -type f -exec sed -i 's|foo|bar|g' {} ";" -ls > file.lst