I've got to go through 100+ sites and add two lines to the same file in all of them (sendmail1.php).
The boss wants me to hand copy/paste this stuff, but there's GOT to be an easier way, so I'm trying to do it with find and sed, both of which apparently I'm not using well. I just want to run a script in the dir housing the directories that have all of the sites in them.
I have this:
#!/bin/bash
read -p "Which file, sir? : " file
# find . -depth 1 -type f -name 'sendmail1.php' -exec \
sed -i 's/require\ dirname\(__FILE__\).\'\/includes\/validate.php\';/ \
a require_once dirname\(__FILE__\).\'\/includes\/carriersoft_email.php\';' $file
sed -i 's/\else\ if\($_POST[\'email\']\ \&\&\ \($_POST\'work_email\'\]\ ==\ \"\"\)\){/ \
a\t$carriersoft_sent = carriersoft_email\(\);' $file
exit 0
At the moment, I have the find commented out while trying to sort out sed here and testing the script, but I'd like to solve both.
I think I'm not escaping something necessary in the sed bits, but I keep going over it and changing it and getting different errors (sometimes "unfinished s/ statement" other times. other stuff.
The point is I have to do this:
Right below require dirname(__FILE__).'/includes/validate.php';, add this line:
require_once dirname(__FILE__).'/includes/carriersoft_email.php';
AND
Under else if($_POST['email'] && ($_POST['work_email'] == "")){, add this line:
$carriersoft_sent = carriersoft_email();
I'd like to turn this 4 hours copy/pasta nightmare into a 2 minutes lazy admin type script it and get it done job.
But my fu is not strong with the sed or the find...
As for the find, I get "path must preceed expression: 1"
I've found questions here addressing that error, but indicating that using the '' to surround the filename should resolve it, but it's not working.
Keep it simple and just use awk since awk can operate with strings, unlike sed which only works on REs with additional caveats:
find whatever |
while IFS= read -r file
do
awk '
{ print }
index($0,"require dirname(__FILE__).\047/includes/validate.php\047;") {
print "require_once dirname(__FILE__).\047/includes/carriersoft_email.php\047;"
}
index($0,"else if($_POST[\047email\047] && ($_POST[\04work_email\047] == "")){") {
print "$carriersoft_sent = carriersoft_email();"
}
' "$file" > /usr/tmp/tmp_$$ &&
mv /usr/tmp/tmp_$$ "$file"
done
With GNU awk you can use -i inplace to avoid manually specifying the tmp file name if you like, just like with sed -i.
The \047s are one way to specify a single quote inside a single-quote-delimited script.
Try this:
sed -e "s/require dirname(__FILE__).'\/includes\/validate.php';/&\nrequire_once dirname(__FILE__).'\/includes\/carriersoft_email.php'\;/" \
-e "s/else if(\$_POST\['email'\] && (\$_POST\['work_email'\] == \"\")){/&\n\$carriersoft_sent = carriersoft_email();/" \
file
Note: I haven't used the -i flag. Once you are confirmed that it works for you, you can use the -i flag. Also I have combined your two sed command into one with -e option.
I think it would be clearer if instead of s you used another fine command, a. To output one changed file, create a script (eg. script.sed) with the following content:
/require dirname(__FILE__)\.'\/includes\/validate.php';/a\
require_once dirname(__FILE__).'/includes/carriersoft_email.php';
/else if(\$_POST\['email'\] && (\$_POST\['work_email'\] == "")){/a\
$carriersoft_sent = carriersoft_email();
and run sed -f script.sed sendmail1.php.
To apply changes in all files, run:
find . -name 'sendmail1.php' -exec sed -i -f script.sed {} \;
(-i causes sed to change file in-place).
It is always advisable in such operations to do a backup and check out the exact changes after running the command. :)
Related
I have a collection of plain text files which are named as yymmdd_nnnnnnnnnn.txt, which I want to append another number sequence to the filenames, so that they each become named as yymmdd_nnnnnnnnnn_iiiiiiiii.txt instead, where the iiiiiiiii is taken from the one line in each file which contains the text "GST: 123456789⏎" (or similar) at the end of the line. While I am sure that there will only be one such matching line within each file, I don't know exactly which line it will be on.
I need an elegant one-liner solution that I can run over the collection of files in a folder, from a bash script file, to rename each file in the collection by appending the specific GST number for each filename, as found within the files themselves.
Before even getting to the renaming stage, I have encountered a problem with this. Here is what I tried, which didn't work...
# awk '/\d+$/' | grep -E 'GST: ' 150101_2224567890.txt
The grep command alone works perfectly to find the relevant line within the file, but the awk doesn't return just the final digits group. It fails with the error "warning: regexp escape sequence \d is not a known regexp operator". I had assumed that this regex should return any number of digits which are at the end of the line. The text file in question contains a line which ends with "GST: 112060340⏎". Can someone please show me how to make this work, and maybe also to help with the appropriate coding to move the collection of files to the new filenames? Thanks.
Thanks to a comment from #Renaud, I now have the following code working to obtain just the GST registration number from within a text file, which puts me a step closer towards a workable solution.
awk '/GST: / {printf $NF}' 150101_2224567890.txt
I still need to loop this over the collection instead of just specifying one filename. I also need to be able to use the output from #Renaud's contribution, to rename the files. I'm getting closer to a working solution, thanks!
This awk should work for you:
awk '$1=="GST:" {fn=FILENAME; sub(/\.txt$/, "", fn); print "mv", FILENAME, fn "_" $2 ".txt"; nextfile}' *_*.txt | sh
To make it more readable:
awk '$1 == "GST:" {
fn = FILENAME
sub(/\.txt$/, "", fn)
print "mv", FILENAME, fn "_" $2 ".txt"
nextfile
}' *_*.txt | sh
Remove | sh from above to see all mv commands together.
You may try
for f in *_*.txt; do echo mv "$f" "${f%.txt}_$(sed '/.*GST: /!d; s///; q' "$f").txt"; done
Drop the echo if you're satisfied with the output.
As you are sure there is only one matching line, you can try:
$ n=$(awk '/GST:/ {print $NF}' 150101_2224567890.txt)
$ mv 150101_2224567890.txt "150101_2224567890_$n.txt"
Or, for all .txt files:
for f in *.txt; do
n=$(awk '/GST:/ {print $NF}' "$f")
if [[ -z "$n" ]]; then
printf '%s: GST not found\n' "$f"
continue
fi
mv "$f" "$f{%.txt}_$n.txt"
done
Another one-line solution to consider, although perhaps not so elegant.
for original_filename in *_*.txt; do \
new_filename=${original_filename%'.txt'}_$(
grep -E 'GST: ' "$original_filename" | \
sed -E 's/.*GST//g; s/[^0-9]//g'
)'.txt' && \
mv "$original_filename" "$new_filename"; \
done
Output:
150101_2224567890_123456789.txt
If you are open to a multi line script:-
#!/bin/sh
for f in *.txt; do
prefix=$(echo "${f}" | sed s'#\.txt##')
cp "${f}" f1
sed -i s'#GST#%GST#' "./f1"
cat "./f1" | tr '%' '\n' > f2
number=$(cat "./f2" | sed -n '/GST/'p | cut -d':' -f2 | tr -d ' ')
newname="${prefix}_${number}.txt"
mv -v "${f}" "${newname}"
rm -v "./f1"
rm -v "./f2"
done
In general, if you want to make your files easy to work with, then leave as many potential places for them to be split with newlines as possible. It is much easier to alter files by simply being able to put what you want to delete or print on its' own line, than it is to search for things horizontally with regular expressions.
I'm using find to grab a bunch of files in a directory, and then awk to try and find/replace a string in them. However, awk is just printing the entire contents of the files to stdout instead of overwriting the files. I'm a bit confused as to what syntax I should use here to accomplish my goal.
My command so far is:
find -iname '*_input.xml' -exec awk -P '/"prop_name" : "ID",/ , /}/ {print sub(/SKU/,"SKU+ProductId")} 1' {} +
THis is replacing the string SKU with SKU+ProductId but not overwriting the file.
Any one have any ideas as to what Im missing?
Cheers!
Update:
Example Content:
},
"prop_name" : "ID",
"rule" : "SKU"
},
awk writes to standard output, so you will need to redirect standard output. You can do that with find by executing an inner shell command:
find -iname '*_input.xml' \
-exec sh -c 'for file; do
./awk_script "$file" > "$file.tmp" && mv "$file.tmp" "$file"
done' _ {} +
I would move the awk command to its own file to avoid problems with quoting. You may also want to consider saving the task of overwriting the original files until you've checked the results.
You could create an awk file like this, I believe. Don't forget to chmod +x it.
#!/usr/bin/awk -f
/"prop_name" : "ID",/ , /}/ {
print sub(/SKU/,"SKU+ProductId")
} 1
GNU sed has a -i for in place editing that makes this a lot easier:
find -iname '*_input.xml' \
-exec sed -i '/"prop_name" : "ID",/ , /}/ s/SKU/SKU+ProductId/' {} +
You need to use a new-ish version of GNU awk (4.?) with the -i infile option and get rid of the print.
Problem
As I am trying to write a script to rename massive files according to some regex requirement, the command work ok on my iTerm2 succeeds but the same command fails to do the work in the script.
Plus some of my file names includes some Chinese and Korean characters.(don't know whether that is the problem or not)
code
So My code takes three input: Old regex, New regex and the files that need to be renamed.
Here is not code:
#!/bin/bash
# we have less than 3 arguments. Print the help text:
if [ $# -lt 3 ] ; then
cat << HELP
ren -- renames a number of files using sed regular expressions USAGE: ren 'regexp'
'replacement' files...
EXAMPLE: rename all *.HTM files into *.html:
ren 'HTM' 'html' *.HTM
HELP
exit 0
fi
OLD="$1"
NEW="$2"
# The shift command removes one argument from the list of
# command line arguments.
shift
shift
# $# contains now all the files:
for file in "$#"; do
if [ -f "$file" ] ; then
newfile=`echo "$file" | sed "s/${OLD}/${NEW}/g"`
if [ -f "$newfile" ]; then
echo "ERROR: $newfile exists already"
else
echo "renaming $file to $newfile ..."
mv "$file" "$newfile"
fi
fi
done
I register the bash command in the .profile as:
alias ren="bash /pathtothefile/ren.sh"
Test
The original file name is "제01과.mp3" and I want it to become "第01课.mp3".
So with my script I use:
$ ren "제\([0-9]*\)과" "第\1课" *.mp3
And it seems that the sed in the script has not worked successfully.
But the following which is exactly the same, works to replaces the name:
$ echo "제01과.mp3" | sed s/"제\([0-9]*\)과\.mp3"/"第\1课\.mp3"/g
Any thoughts? Thx
Print the result
I have make the following change in the script so that it could print the process information:
newfile=`echo "$file" | sed "s/${OLD}/${NEW}/g"`
echo "The ${file} is changed to ${newfile}"
And the result for my test is:
The 제01과.mp3 is changed into 제01과.mp3
ERROR: 제01과.mp3 exists already
So there is no format problem.
Updating(all done under bash 4.2.45(2), Mac OS 10.9)
Testing
As I try to execute the command from the bash directly. I mean with the for loop. There is something interesting. I first stored all the names into a files.txt file using:
$ ls | grep mp3 > files.txt
And do the sed and bla bla. While single command in bash interactive mode like:
$ file="제01과.mp3"
$ echo $file | sed s/"제\([0-9]*\)과\.mp3"/"第\1课\.mp3"/g
gives
第01课.mp3
While in the following in the interactive mode:
files=`cat files.txt`
for file in $files
do
echo $file | sed s/"제\([0-9]*\)과\.mp3"/"第\1课\.mp3"/g
done
gives no changes!
And by now:
echo $file
gives:
$ 제30과.mp3
(There are only 30 files)
Problem Part
And I tried the first command which worked before:
$ echo $file | sed s/"제\([0-9]*\)과\.mp3"/"第\1课\.mp3"/g
It gives no changes as:
$ 제30과.mp3
So I create a new newfile and tried again as:
$ newfile="제30과.mp3"
$ echo $newfile | sed s/"제\([0-9]*\)과\.mp3"/"第\1课\.mp3"/g
And it gives correctly:
$第30课.mp3
WOW ORZ... Why! Why ! Why! And I try to see whether file and newfile are the same, and of course, they are not:
if [[ $file == $new ]]; then
echo True
else
echo False
fi
gives:
False
My guess
I guess there are some encoding problems , but I have found non reference, could anyone help? Thx again.
Update 2
I seem to understand that there are a huge difference between string and the file name. To be specific, it I directly use a variable like:
file="제30과.mp3"
in the script, the sed works fine. However, if the variable was passed from the $# or set the variable like:
file=./*mp3
Then the sed fails to work. I don't know why. And btw, mac sed has no -r option and in ubuntu -r does not solve the question I mention above.
Some errors combined:
In order to use groups in a regex, you need extended regex -r in sed, -E in grep
escaping correctly is a beast :)
Example
files="제2과.mp3 제30과.mp3"
for file in $files
do
echo $file | sed -r 's/제([0-9]*)과\.mp3/第\1课.mp3/g'
done
outputs
第2课.mp3
第30课.mp3
If you are not doing this as a programming project, but want to skip ahead to the part where it just works, I found these resources listed at http://www.tldp.org/LDP/GNU-Linux-Tools-Summary/html/x4055.htm:
MMV (and MCP, MLN, ...) utilities use a specialized syntax to perform bulk file operations on paths. (http://linux.maruhn.com/sec/mmv.html)
mmv before\*after.mp3 Before\#1After.mp3
Esomaniac, a Java alternative that also works on Windows, is apparently dead (home page is parked).
rename is a perl script you can download from CPAN: https://metacpan.org/release/File-Rename
rename 's/\.JPG$/.jpg/' *.JPG
I have a list of files (thousands of them) like this:
/path/2010 - filename.txt
/path/2011 - another file name.txt
Always following this pattern: #### - string.txt
I need to change them to look like this:
/path/filename (2010).txt
/path/another file name (2011).txt
How can I do this quickly with bash, shell, terminal, etc.?
Try rename command:
rename -n 's/(.*) - (.*)(\.txt)/$2 ($1)$3/' *.txt
-n(--no-act) option is for preview.
Remove -n to perform substitution.
Untested.
find /path -name '???? - *.txt' -print0 | while read -d ''; do
[[ $REPLY =~ (.*)/(....)\ -\ (.*)\.txt$ ]] || continue
path=${BASH_REMATCH[1]}
year=${BASH_REMATCH[2]}
str=${BASH_REMATCH[3]}
echo mv "$REPLY" "$path/$str ($year).txt"
done
Remove the echo once the generated mv commands look right.
I know you didn't tag it with zsh but you did say shell. Anyway here's how to do it with the zmv function in zsh:
autoload zmv # It's not loaded by default
zmv -nvw '* - *.*' '$2 ($1).$3'
Remove -n when you're happy with the output.
-v makes zmv verbose. -w implicitly makes a group of each wildcard.
I'd prefer to add this as a comment, but I'm not yet allowed to.
I asked a similar question and received a number of helpful answers over here:
https://unix.stackexchange.com/questions/37355/recursively-rename-subdirectories-that-match-a-regex
Perhaps one of those solutions can be adapted to suit you needs.
First, I know this sounds ass backwards. It is. But I'm looking to convert (on the BASH command line) a bunch of script-generated thumbnail filenames that do have a "%20" in them to the equivalent without filenames. In case you're curious, the reason is because the script I'm using created the thumbnail filenames from their current URLs, and it added the %20 in the process. But now WordPress is looking for files like "This%20Filename.jpg" and the browser is, of course, removing the escape character and replacing it with spaces. Which is why one shouldn't have spaces in filenames.
But since I'm stuck here, I'd love to convert my existing thumbnails over. Next, I will post a question for help fixing the problem in the script mentioned above. What I'm looking for now is a quick script to do the bad thing and create filenames with spaces out of filenames with "%20"s.
Thanks!
If you only want to replace each literal %20 with one space:
for i in *; do
mv "$i" "${i//\%20/ }"
done
(for instance this will rename file%with%20two%20spaces to file%with two spaces).
You'll probably need to apply %25->% too though, and other similar transforms.
convmv can do this, no script needed.
$ ls
a%20b.txt
$ convmv --unescape *.txt --notest
mv "./a%20b.txt" "./a b.txt"
Ready!
$ ls
a b.txt
personally, I don't like file names with spaces - beware you will have to treat them specially in future scripts. Anyway, here is the script that will do what you want to achieve.
#!/bin/sh
for fname in `ls *%20*`
do
newfname=`echo $fname | sed 's/%20/ /g'`
mv $fname "$newfname"
done;
Place this to a file, add execute permission and run this from the directory where you have file with %20 in their names.
Code :
#!/bin/bash
# This is where your files currently are
DPATH="/home/you/foo/*.txt"
# This is where your new files will be created
BPATH="/home/you/new_foo"
TFILE="/tmp/out.tmp.$$"
[ ! -d $BPATH ] && mkdir -p $BPATH || :
for f in $DPATH
do
if [ -f $f -a -r $f ]; then
/bin/cp -f $f $BPATH
sed "s/%20/ /g" "$f" > $TFILE && mv $TFILE "$f"
else
echo "Error: Cannot read $f"
fi
done
/bin/rm $TFILE
Not bash, but for the more general case of %hh (encoded hex) in names.
#!/usr/bin/perl
foreach $c(#ARGV){
$d=$c;
$d=~s/%([a-fA-F0-9][a-fA-F0-9])/my $a=pack('C',hex($1));$a="\\$a"/eg;
print `mv $c $d` if ($c ne $d);
}