First, I know this sounds ass backwards. It is. But I'm looking to convert (on the BASH command line) a bunch of script-generated thumbnail filenames that do have a "%20" in them to the equivalent without filenames. In case you're curious, the reason is because the script I'm using created the thumbnail filenames from their current URLs, and it added the %20 in the process. But now WordPress is looking for files like "This%20Filename.jpg" and the browser is, of course, removing the escape character and replacing it with spaces. Which is why one shouldn't have spaces in filenames.
But since I'm stuck here, I'd love to convert my existing thumbnails over. Next, I will post a question for help fixing the problem in the script mentioned above. What I'm looking for now is a quick script to do the bad thing and create filenames with spaces out of filenames with "%20"s.
Thanks!
If you only want to replace each literal %20 with one space:
for i in *; do
mv "$i" "${i//\%20/ }"
done
(for instance this will rename file%with%20two%20spaces to file%with two spaces).
You'll probably need to apply %25->% too though, and other similar transforms.
convmv can do this, no script needed.
$ ls
a%20b.txt
$ convmv --unescape *.txt --notest
mv "./a%20b.txt" "./a b.txt"
Ready!
$ ls
a b.txt
personally, I don't like file names with spaces - beware you will have to treat them specially in future scripts. Anyway, here is the script that will do what you want to achieve.
#!/bin/sh
for fname in `ls *%20*`
do
newfname=`echo $fname | sed 's/%20/ /g'`
mv $fname "$newfname"
done;
Place this to a file, add execute permission and run this from the directory where you have file with %20 in their names.
Code :
#!/bin/bash
# This is where your files currently are
DPATH="/home/you/foo/*.txt"
# This is where your new files will be created
BPATH="/home/you/new_foo"
TFILE="/tmp/out.tmp.$$"
[ ! -d $BPATH ] && mkdir -p $BPATH || :
for f in $DPATH
do
if [ -f $f -a -r $f ]; then
/bin/cp -f $f $BPATH
sed "s/%20/ /g" "$f" > $TFILE && mv $TFILE "$f"
else
echo "Error: Cannot read $f"
fi
done
/bin/rm $TFILE
Not bash, but for the more general case of %hh (encoded hex) in names.
#!/usr/bin/perl
foreach $c(#ARGV){
$d=$c;
$d=~s/%([a-fA-F0-9][a-fA-F0-9])/my $a=pack('C',hex($1));$a="\\$a"/eg;
print `mv $c $d` if ($c ne $d);
}
Related
I want to create a list:
0123456789abcdefghijklmnopqrstuvxwyzABCDEFGHIJKLMNOPQRSTUVXWYZ-._
And then iterate trough all files in folder, replacing every character in the file names that is not present in this list with an underscore _.
That includes blank spaces.
But using Bash and GNU tools only.
Is that possible?
Bash internals only:
for file in *; do
repl=${file//[!$permitted]/_}
case $file in "$repl") continue;; esac # skip if identical
# Safety: add a suffix to avoid overwriting
while [ -e "$repl" ]; do
repl=${repl}_
done
mv "$file" "$repl"
done
If $permitted contains a slash, you will need to backslash-escape it.
Having the Perl version of the rename command, you can use this:
rename 's/[^[:alnum:]._-]/_/g' *
Yes it is possible :)
for filename in *; do
newfilename=$(echo "$filename" | sed 's/[^0123456789abcdefghijklmnopqrstuvxwyzABCDEFGHIJKLMNOPQRSTUVXWYZ._-]/_/g')
mv "$filename" "$newfilename"
done
I've got to go through 100+ sites and add two lines to the same file in all of them (sendmail1.php).
The boss wants me to hand copy/paste this stuff, but there's GOT to be an easier way, so I'm trying to do it with find and sed, both of which apparently I'm not using well. I just want to run a script in the dir housing the directories that have all of the sites in them.
I have this:
#!/bin/bash
read -p "Which file, sir? : " file
# find . -depth 1 -type f -name 'sendmail1.php' -exec \
sed -i 's/require\ dirname\(__FILE__\).\'\/includes\/validate.php\';/ \
a require_once dirname\(__FILE__\).\'\/includes\/carriersoft_email.php\';' $file
sed -i 's/\else\ if\($_POST[\'email\']\ \&\&\ \($_POST\'work_email\'\]\ ==\ \"\"\)\){/ \
a\t$carriersoft_sent = carriersoft_email\(\);' $file
exit 0
At the moment, I have the find commented out while trying to sort out sed here and testing the script, but I'd like to solve both.
I think I'm not escaping something necessary in the sed bits, but I keep going over it and changing it and getting different errors (sometimes "unfinished s/ statement" other times. other stuff.
The point is I have to do this:
Right below require dirname(__FILE__).'/includes/validate.php';, add this line:
require_once dirname(__FILE__).'/includes/carriersoft_email.php';
AND
Under else if($_POST['email'] && ($_POST['work_email'] == "")){, add this line:
$carriersoft_sent = carriersoft_email();
I'd like to turn this 4 hours copy/pasta nightmare into a 2 minutes lazy admin type script it and get it done job.
But my fu is not strong with the sed or the find...
As for the find, I get "path must preceed expression: 1"
I've found questions here addressing that error, but indicating that using the '' to surround the filename should resolve it, but it's not working.
Keep it simple and just use awk since awk can operate with strings, unlike sed which only works on REs with additional caveats:
find whatever |
while IFS= read -r file
do
awk '
{ print }
index($0,"require dirname(__FILE__).\047/includes/validate.php\047;") {
print "require_once dirname(__FILE__).\047/includes/carriersoft_email.php\047;"
}
index($0,"else if($_POST[\047email\047] && ($_POST[\04work_email\047] == "")){") {
print "$carriersoft_sent = carriersoft_email();"
}
' "$file" > /usr/tmp/tmp_$$ &&
mv /usr/tmp/tmp_$$ "$file"
done
With GNU awk you can use -i inplace to avoid manually specifying the tmp file name if you like, just like with sed -i.
The \047s are one way to specify a single quote inside a single-quote-delimited script.
Try this:
sed -e "s/require dirname(__FILE__).'\/includes\/validate.php';/&\nrequire_once dirname(__FILE__).'\/includes\/carriersoft_email.php'\;/" \
-e "s/else if(\$_POST\['email'\] && (\$_POST\['work_email'\] == \"\")){/&\n\$carriersoft_sent = carriersoft_email();/" \
file
Note: I haven't used the -i flag. Once you are confirmed that it works for you, you can use the -i flag. Also I have combined your two sed command into one with -e option.
I think it would be clearer if instead of s you used another fine command, a. To output one changed file, create a script (eg. script.sed) with the following content:
/require dirname(__FILE__)\.'\/includes\/validate.php';/a\
require_once dirname(__FILE__).'/includes/carriersoft_email.php';
/else if(\$_POST\['email'\] && (\$_POST\['work_email'\] == "")){/a\
$carriersoft_sent = carriersoft_email();
and run sed -f script.sed sendmail1.php.
To apply changes in all files, run:
find . -name 'sendmail1.php' -exec sed -i -f script.sed {} \;
(-i causes sed to change file in-place).
It is always advisable in such operations to do a backup and check out the exact changes after running the command. :)
Problem
As I am trying to write a script to rename massive files according to some regex requirement, the command work ok on my iTerm2 succeeds but the same command fails to do the work in the script.
Plus some of my file names includes some Chinese and Korean characters.(don't know whether that is the problem or not)
code
So My code takes three input: Old regex, New regex and the files that need to be renamed.
Here is not code:
#!/bin/bash
# we have less than 3 arguments. Print the help text:
if [ $# -lt 3 ] ; then
cat << HELP
ren -- renames a number of files using sed regular expressions USAGE: ren 'regexp'
'replacement' files...
EXAMPLE: rename all *.HTM files into *.html:
ren 'HTM' 'html' *.HTM
HELP
exit 0
fi
OLD="$1"
NEW="$2"
# The shift command removes one argument from the list of
# command line arguments.
shift
shift
# $# contains now all the files:
for file in "$#"; do
if [ -f "$file" ] ; then
newfile=`echo "$file" | sed "s/${OLD}/${NEW}/g"`
if [ -f "$newfile" ]; then
echo "ERROR: $newfile exists already"
else
echo "renaming $file to $newfile ..."
mv "$file" "$newfile"
fi
fi
done
I register the bash command in the .profile as:
alias ren="bash /pathtothefile/ren.sh"
Test
The original file name is "제01과.mp3" and I want it to become "第01课.mp3".
So with my script I use:
$ ren "제\([0-9]*\)과" "第\1课" *.mp3
And it seems that the sed in the script has not worked successfully.
But the following which is exactly the same, works to replaces the name:
$ echo "제01과.mp3" | sed s/"제\([0-9]*\)과\.mp3"/"第\1课\.mp3"/g
Any thoughts? Thx
Print the result
I have make the following change in the script so that it could print the process information:
newfile=`echo "$file" | sed "s/${OLD}/${NEW}/g"`
echo "The ${file} is changed to ${newfile}"
And the result for my test is:
The 제01과.mp3 is changed into 제01과.mp3
ERROR: 제01과.mp3 exists already
So there is no format problem.
Updating(all done under bash 4.2.45(2), Mac OS 10.9)
Testing
As I try to execute the command from the bash directly. I mean with the for loop. There is something interesting. I first stored all the names into a files.txt file using:
$ ls | grep mp3 > files.txt
And do the sed and bla bla. While single command in bash interactive mode like:
$ file="제01과.mp3"
$ echo $file | sed s/"제\([0-9]*\)과\.mp3"/"第\1课\.mp3"/g
gives
第01课.mp3
While in the following in the interactive mode:
files=`cat files.txt`
for file in $files
do
echo $file | sed s/"제\([0-9]*\)과\.mp3"/"第\1课\.mp3"/g
done
gives no changes!
And by now:
echo $file
gives:
$ 제30과.mp3
(There are only 30 files)
Problem Part
And I tried the first command which worked before:
$ echo $file | sed s/"제\([0-9]*\)과\.mp3"/"第\1课\.mp3"/g
It gives no changes as:
$ 제30과.mp3
So I create a new newfile and tried again as:
$ newfile="제30과.mp3"
$ echo $newfile | sed s/"제\([0-9]*\)과\.mp3"/"第\1课\.mp3"/g
And it gives correctly:
$第30课.mp3
WOW ORZ... Why! Why ! Why! And I try to see whether file and newfile are the same, and of course, they are not:
if [[ $file == $new ]]; then
echo True
else
echo False
fi
gives:
False
My guess
I guess there are some encoding problems , but I have found non reference, could anyone help? Thx again.
Update 2
I seem to understand that there are a huge difference between string and the file name. To be specific, it I directly use a variable like:
file="제30과.mp3"
in the script, the sed works fine. However, if the variable was passed from the $# or set the variable like:
file=./*mp3
Then the sed fails to work. I don't know why. And btw, mac sed has no -r option and in ubuntu -r does not solve the question I mention above.
Some errors combined:
In order to use groups in a regex, you need extended regex -r in sed, -E in grep
escaping correctly is a beast :)
Example
files="제2과.mp3 제30과.mp3"
for file in $files
do
echo $file | sed -r 's/제([0-9]*)과\.mp3/第\1课.mp3/g'
done
outputs
第2课.mp3
第30课.mp3
If you are not doing this as a programming project, but want to skip ahead to the part where it just works, I found these resources listed at http://www.tldp.org/LDP/GNU-Linux-Tools-Summary/html/x4055.htm:
MMV (and MCP, MLN, ...) utilities use a specialized syntax to perform bulk file operations on paths. (http://linux.maruhn.com/sec/mmv.html)
mmv before\*after.mp3 Before\#1After.mp3
Esomaniac, a Java alternative that also works on Windows, is apparently dead (home page is parked).
rename is a perl script you can download from CPAN: https://metacpan.org/release/File-Rename
rename 's/\.JPG$/.jpg/' *.JPG
I'm trying to write a bash script to remove spaces, underscores and dots and replace them with dashes. I also set to lowercase and remove brackets. That's the (long) second sed command, which seems to work.
The first sed call escapes the original names with spaces with '\ ' like when I tab complete, and this is the issue I think.
If I replace 'mv -i' with 'echo' I get what I think I want: the original filename escaped with backslashes and then the new name. If I paste this into the terminal it works, but with mv in the script the spaces cause problems. The escaping doesn't work.
#!/bin/bash
for a in "$#"; do
mv -i $(echo "$a" | sed -e 's/ /\\\ /g') $(echo "$a" | sed -e 's/\(.*\)/\L\1/' -e 's/_/-/g' -e 's/ /-/g' -e 's/---/--/g' -e 's/(//g' -e 's/)//g' -e 's/\[//g' -e 's/\]//g' -e 's/\./-/g' -e 's/-\([^-]*\)$/\.\1/')
done
The other solution is to put quotes around the names, but I can't work out how I would do this. I feel like I've got close, but I'm stumped.
I've also considered the 'rename' command, but you cannot do multiple operations like you can with sed.
Please point out any other issues, this is one of my first scripts. I'm not sure I got the "$#" or "$a" bits completely correct.
Cheers.
edit:
sample input filename
I am a Badly [named] (file) - PLEASE.rename_me.JPG
should become
i-am-a-badly-named-file--please-rename-me.jpg
edit2: my solution, tweaked from gniourf_gniourf's really helpful pure bash answer:
#!/bin/bash
for a in "$#"; do
b=${a,,} #lowercase
b=${b//[_[:space:]\.]/-} #subst dot,space,underscore with dash
b=${b//---/--} #remove triple dash
b=${b//[()\[\]]/} #remove brackets
if [ "${b%-*}" != "$b" ]; then #if there is a dash (prevents filename.filename)
b=${b%-*}.${b##*-} #replace final dash with a dot for extension
fi
if [ "$a" != "$b" ]; then #if there has been a change
echo '--->' "$b" #
#mv -i -- "$a" "$b" #rename
fi
done
This only fails if the file had spaces etc and no extension (e.g this BAD_filename becomes this-bad.filename. But these are media files and should have an extension, so I would have to sort them anyway.
Again, corrections and improvements welcome. I'm new at this stuff
Try doing this with rename :
rename 's/[_\s\.]/-/g' *files
from the shell prompt. It's very useful, you can put some perl code inside if needed.
You can remove the -n (dry-run mode switch) when your tests become valids.
There are other tools with the same name which may or may not be able to do this, so be careful.
If you run the following command (linux)
$ file $(readlink -f $(type -p rename))
and you have a result like
.../rename: Perl script, ASCII text executable
then this seems to be the right tool =)
If not, to make it the default (usually already the case) on Debian and derivative like Ubuntu :
$ sudo update-alternatives --set rename /path/to/rename
(replace /path/to/rename to the path of your perl's rename command.
Last but not least, this tool was originally written by Larry Wall, the Perl's dad.
Just for the records, look:
$ a='I am a Badly [named] (file) - PLEASE.rename_me.JPG'
$ # lowercase that
$ echo "${a,,}"
i am a badly [named] (file) - please.rename_me.jpg
$ # Cool! let's save that somewhere
$ b=${a,,}
$ # substitution 's/[_ ]/-/g:
$ echo "${b//[_ ]/-}"
i-am-a-badly-[named]-(file)---please.rename-me.jpg
$ # or better, yet:
$ echo "${b//[_[:space:]]/-}"
i-am-a-badly-[named]-(file)---please.rename-me.jpg
$ # Cool! let's save that somewhere
$ c=${b//[_[:space:]]/-}
$ # substitution 's/---/--/g' (??)
$ echo "${c//---/--}"
i-am-a-badly-[named]-(file)--please.rename-me.jpg
$ d=${c//---/--}
$ # substitution 's/()[]//g':
$ echo "${d//[()\[\]]/}"
i-am-a-badly-named-file--please.rename-me.jpg
$ e="${d//[()\[\]]/}"
$ # substitution 's/\./-/g':
$ echo "${e//\./-}"
i-am-a-badly-named-file--please-rename-me-jpg
$ f=${e//\./-}
$ # substitution 's/-\([^-]*\)$/\.\1/':
$ echo "${f%-*}.${f##*-}"
i-am-a-badly-named-file--please-rename-me.jpg
$ # Done!
Now, here's a 100% bash implementation of what you're trying to achieve:
#!/bin/bash
for a in "$#"; do
b=${a,,}
b=${b//[_[:space:]]/-}
b=${b//---/--}
b=${b//[()\[\]]/}
b=${b//\./-}
b=${b%-*}.${b##*-}
mv -i -- "$a" "$b"
done
yeah, done!
All this standard and known as shell parameter expansion.
Remark. For a more robust script, you could check whether a has an extension (read: a period in its name), otherwise the last substitution of the algorithm fails a little bit. For this, put the following line just below the for statement:
[[ a != *.* ]] && { echo "Oh no, file \`$a' has no extension..."; continue; }
(and isn't the *.* part of this line so cute?)
I have a list of files (thousands of them) like this:
/path/2010 - filename.txt
/path/2011 - another file name.txt
Always following this pattern: #### - string.txt
I need to change them to look like this:
/path/filename (2010).txt
/path/another file name (2011).txt
How can I do this quickly with bash, shell, terminal, etc.?
Try rename command:
rename -n 's/(.*) - (.*)(\.txt)/$2 ($1)$3/' *.txt
-n(--no-act) option is for preview.
Remove -n to perform substitution.
Untested.
find /path -name '???? - *.txt' -print0 | while read -d ''; do
[[ $REPLY =~ (.*)/(....)\ -\ (.*)\.txt$ ]] || continue
path=${BASH_REMATCH[1]}
year=${BASH_REMATCH[2]}
str=${BASH_REMATCH[3]}
echo mv "$REPLY" "$path/$str ($year).txt"
done
Remove the echo once the generated mv commands look right.
I know you didn't tag it with zsh but you did say shell. Anyway here's how to do it with the zmv function in zsh:
autoload zmv # It's not loaded by default
zmv -nvw '* - *.*' '$2 ($1).$3'
Remove -n when you're happy with the output.
-v makes zmv verbose. -w implicitly makes a group of each wildcard.
I'd prefer to add this as a comment, but I'm not yet allowed to.
I asked a similar question and received a number of helpful answers over here:
https://unix.stackexchange.com/questions/37355/recursively-rename-subdirectories-that-match-a-regex
Perhaps one of those solutions can be adapted to suit you needs.