I have the command
find . -type f \( ! -name "*.png" \) -print0 | \
xargs -0 sed -i 's#word#replace#g'
awk/sed: How to do a recursive find/replace of a string?
BASH: recursive program to replace text in a tree of files
This command works so far but i want to show the files in which sed replaces text.
Is there some parameter which allows that?
You can use the print and exec options together and print out the files that it processes:
find . -type f \( ! -name "*.png" \) -print -exec sed -i 's#word#replace#g' {} \; 2>/dev/null
You are missing cmp to compare two files pre and post change. Something on these lines you could try:
find . -type f \( ! -name "*.png" \) -exec sh -c 'sed "s/word/replace/g" $0 > $0.$$; if ! cmp $0.$$ $0; then echo $0; mv $0.$$ $0; else rm $0.$$; fi' {} \;
find - in current directory all the files name not like *.png
exec - sed search and replace word by relace word in file found by find command and put it in temp file appended by process id
if !cmp statement - compare both the files new and old and if they are not same then print file name along with the output and at the end move temp file to orginal file if they are not same else delete the temp file.
I don't know what platform you are on, but this is a possibility. Change your xargs to run a shell so you can do multiple commands and then inside that shell, test if the file is newer than some arbitrary file you created at the start - i.e. it is changed
touch /tmp/go
find ... | xargs ... -I % sh -c 'sed "%"; [ "%" -nt /tmp/go ] && echo %'
Related
Hi I've got a list of csv files which need to be formatted properly by getting rid of some unwanted characters.
original:
9: ["2019-4-24",-7.101458109105941]
10: ["2019-5-6",-7.050609022950812]
100: ["2019-5-6",-7.050609022950812]
I'd like to modify as:
2019-4-24,-7.101458109105941
2019-5-6,-7.050609022950812
2019-5-6,-7.050609022950812
There are dozens of files in this format and I was thinking of writing a sed command which makes a series of null substitutions for all the files in directory, but these don't seem to work.
find ./ -type f -exec sed -i '' -e "s/^[[:space:]]*//" {} \;
find ./ -type f -exec sed -i '' -e "s/\[//" {} \;
find ./ -type f -exec sed -i '' -e "s/\]//" {} \;
Many thanks for suggestions.
I found this to work on my linux machine.
find ./ -type f -exec sed -i "s/^.\+\[//;s/\"//g;s/\]//" {} \;
Which, from what I gather is equivalent to the following in macOS:
find ./ -type f -exec sed -i '' "s/^.\+\[//;s/\"//g;s/\]//" {} \;
It comprises of 3 substitutions(separated by semicolon):
s/^.\+\[// deletes everything from the start to the "[" character.
s/\"//g deletes all occurences of the double quote character.
s/\]// deletes the final "]" at the end.
And please make a backup or something if you are going to use sed -i.
I have a Git project that contains a lot of files with no trailing newline. I want to add trailing newlines without adding superfluous newlines. How can I do this?
I found this surprisingly tricky to do using the tools I would usually use (grep, sed), but an elegant solution does exist using standard shell commands:
tail -c 1 file.txt | read || echo >> file.txt
tail Outputs the last byte of file
read Reads a line into a variable. With no variable specified, does nothing, but if an EOF occurs before a newline, exits with code 1.
echo Runs only if read fails (i.e. if the last character was not a newline), and appends a newline to file.txt
To find the files we want to change you can use find:
find -not -path "./.git/*" -type f -exec sh -c "grep -Iq . {} && (tail -c 1 {} | read || echo >> {})" \;
-not -path excludes .git/, which we don't want to mess with
-type f restricts the search to files
-exec sh -c "..." is required to bundle together commands that include pipes
grep -Iq . searches for anything (.) so is a no-op, but exits with code 1 if file is binary
{} marks the position where find will insert the file name
I would advise checking the list of files before running the command. Do this by replacing echo >> {} with echo {}:
find -not -path "./.git/*" -type f -exec sh -c "grep -Iq . {} && (tail -c 1 {} | read || echo {})" \;
I am using a find and sed command to replace characters in a file. see the code 1 below
find . -type f -exec sed -i '/Subject/{:a;s/(Subject.*)Subject/\1SecondSubject/;tb;N;ba;:b}' {} +
Given that I have multiple files I need to replace. In a given situation, the Subject I am trying to replace is not available.
Is there a way I can first check if the file contains the attribute 'Subject' if not I need to execute another command. i.e
Check if the file contains character 'Subject'
If true then execute code1 above
If there is no instance of Subject execute code 2 below
find . -name "*.html" -exec rename 's/.html$/.xml/' {} ;
Any Ideas? Thanks in advance
Something like this should work.
find . -type f \( \
-exec grep -q "Subject" {} \; \
-exec sed -i '/Subject/{:a;s/(Subject.*)Subject/\1SecondSubject/;tb;N;ba;:b}' {} \; \
-o \
-exec rename 's/.html$/.xml/' {} \; \)
-exec takes the exit code of the command it executes, so -exec grep -q "Subject" {} \; will only be true if the grep is true. And since the short circuit -o (or) has a lower precedence than the implied -a (and) between the other operators it should conversely only get executed if the grep is false.
You can use find in a process substitution like this:
while IFS= read -d'' -r file; do
echo "processing $file ..."
if grep -q "/Subject/" "$file"; then
sed -i '{:a;s/(Subject.*)Subject/\1SecondSubject/;tb;N;ba;:b}' "$file"
else if [[ $file == *.html ]]; then
rename 's/.html$/.xml/' "$file"
fi
done < <(find . -type f -print0)
I have a series of files that I would like to clean up using commandline tools available on a *nix system. The existing files are named like so.
filecopy2.txt?filename=3
filecopy4.txt?filename=33
filecopy6.txt?filename=198
filecopy8.txt?filename=188
filecopy3.txt?filename=19
filecopy5.txt?filename=1
filecopy7.txt?filename=5555
I would like them to be renamed removing all characters after and including the "?".
filecopy2.txt
filecopy4.txt
filecopy6.txt
filecopy8.txt
filecopy3.txt
filecopy5.txt
filecopy7.txt
I believe the following regex will grab the bit I want to remove from the name,
\?(.*)
I just can't figure out how to accomplish this task beyond this.
A bash command:
for file in *; do
mv $file ${file%%\?filename=*}
done
find . -depth -name '*[?]*' -exec sh -c 'for i do
mv "$i" "${i%[?]*}"; done' sh {} +
With zsh:
autoload zmv
zmv '(**/)(*)\?*' '$1$2'
Change it to:
zmv -Q '(**/)(*)\?*(D)' '$1$2'
if you want to rename dot files as well.
Note that if filenames may contain more than one ? character, both will only trim from the rightmost one.
If all files are in the same directory (ignoring .dotfiles):
$ rename -n 's/\?filename=\d+$//' -- *
If you want to rename files recursively in a directory hierarchy:
$ find . -type f -exec rename -n 's/\?filename=\d+$//' {} +
Remove -n option, to do the renaming.
I this case you can use the cut command:
echo 'filecopy2.txt?filename=3' | cut -d? -f1
example:
find . -type f -name "*\?*" -exec sh -c 'mv $1 $(echo $1 | cut -d\? -f1)' mv {} \;
You can use rename if you have it:
rename 's/\?.*$//' *
I use this after downloading a bunch of files where the URL included parameters and those parameters ended up in the file name.
This is a Bash script.
for file in *; do
mv $file ${file%%\?*};
done
I need to go recursively through directories. First argument must be directory in which I need to start from, second argument is regex which describes name of the file.
ex. ./myscript.sh directory "regex"
While script recursively goes through directories and files, it must use wc -l to count lines in the files which are described by regex.
How can I use find with -exec to do that? Or there is maybe some other way to do it? Please help.
Thanks
Yes, you can use find:
$ find DIR -iname "regex" -type f -exec wc -l '{}' \;
Or, if you want to count the total number of lines, in all files:
$ find DIR -iname "regex" -type f -exec wc -l '{}' \; | awk '{ SUM += $1 } END { print SUM }'
Your script would then look like:
#!/bin/bash
# $1 - name of the directory - first argument
# $2 - regex - second argument
if [ $# -lt 2 ]; then
echo Usage: ./myscript.sh DIR "REGEX"
exit
fi
find "$1" -iname "$2" -type f -exec wc -l '{}' \;
Edit: - if you need more fancy regular expressions, use -regextype posix-extended and -regex instead of -iname as noted by #sudo_O in his answer.