Unix CLI script to rename folders using their pre-existing names - regex

I have a directory with folder structure as follows:
-- DATA -- ABD 1231345 -- 01-08-12 // date in mm-dd-yy format
-- 03-09-12
-- 06-11-12
-- DEF 4859480 -- 02-10-12
-- 05-10-12
-- 07-10-12
I would like to batch rename this DATA folder as follows
-- DATA -- ABD 1231345 -- 2012_01_08 // date in yyyy_mm_dd format with underscore
-- 2012_03_09
-- 2012_06_11
-- DEF 4859480 -- 2012_02_10
-- 2012_05_10
-- 2012_07_10
Do you have a suggestion on how to accomplish using command line on Mac OSX / unix?

You could use a for loop and awk, parsing each file-name into your specified format and then mv to rename the original to the new name:
for dir in DATA/*; do \
pushd "$dir"; # change directory \
for file in *; do \
path=`echo $file | awk -F- '{print "20"$3"_"$1"_"$2}'`; \
mv $file $path; # "rename" the file \
done; \
popd; # restore original directory \
done;
This can be executed in folder above DATA. If you want to execute it directly in DATA, update the first loop to read for dir in *; do instead of DATA/*. It tells awk to use the - as the delimiter (instead of whitespace), and then reconstructs a string from "mm-dd-yy" to "20yy_mm_dd".
Using pushd and popd will enable the script to change the current directory to each subdirectory inside DATA (pushd) and then, after moving all necessary files will change back to the original (popd). Doing this will save you a lot of parsing-effort trying to save directory paths / etc.

You could use string manipulations and arrays to do that with bash only.
Something like:
for f in * ; do
parts=(${f//-/ })
mv "$f" "20${parts[2]}_${parts[1]}_${parts[0]}"
done
Search this site for various options to recurse into directories e.g.:
Shell script to traverse directories

Use the date command to convert the file name:
$ date -j -f %m-%d-%y 01-08-12 +%Y_%m_%d
2012_01_08
Getting to the files is a little tricker. We'll just switch directories to avoid dealing with long file paths.
for d in DATA; do
pushd "$d"
for f in *; do
new_f=$(date -j -f %m-%d-%y $f +%Y_%m_%d)
mv "$f" "$new_f"
done
popd
done

This site gives a good snippet
for i in *.avi; do j=`echo $i | sed 's/(\d{2})-(\d{2})-(\d{2})/20\3_\1_\2/g'`; mv "$i" "$j"; done

Related

How can I auto-format Rust (and C++) code on commit automatically?

I would like to automatically format the code when I do commit using rustfmt the same way as I did it before for clang-format -i. I.e. format only the lines of code which has been updated in the commit without touching other code. How to do it?
It might be done using git pre-commit hook in the following way:
Add file pre-commit to the folder .githooks in your repo with the following text:
#!/bin/bash
exe=$(which rustfmt)
if [ -n "$exe" ]
then
# field separator to the new line
IFS=$'\n'
for line in $(git status -s)
do
# if added or modified
if [[ $line == A* || $line == M* ]]
then
# check file extension
if [[ $line == *.rs ]]
then
# format file
rustfmt $(pwd)/${line:3}
# add changes
git add $(pwd)/${line:3}
fi
fi
done
else
echo "rustfmt was not found"
fi
Run in your repo folder:
chmod +x .githooks/pre-commit
git config core.hooksPath .githooks
To make it work for clang-format you need to replace rustfmt with clang-format -i and do corresponding modifications in the check for file extension (cpp\h\hpp\etc).

Bash script to 7za all files in a directory then rename files

I am working on using 7za to compress and password protect all files in a directory then rename the files.
Here is my code :
#!/bin/bash
rename 's/ /_/g' *
rename 's/_-_/_/g' *
Password_string=My_Password_String
a=1
for i in *;
do 7za a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=off -p$Password_string -mhe=on $i.7z $i
new=$(printf "Summer_Vacat_%04d.7z" "$a")
mv -i -- "$i" "$new"
let a=a+1 #increment counter
done
I am in the directory with all the folders I want to 7za and password protect, the mv command renames the folders before they are compressed. I want to rename $i.7z to Summer_Vacat_0000.7z, what am I missing here? This is only the first phase of this program...
Try this:
#!/bin/bash
Password_string=My_Password_String
a=1
for i in *;
do
7za a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=off -p$Password_string -mhe=on $i.7z $i
if [ -f "${i}.7z" ];then
new=$(printf "Summer_Vacat_%04d.7z" "$a")
mv -i "${i}.7z" "$new"
rm -f "$i" # this will remove the unencrypted or original file
let a=a+1 #increment counter
fi
done
Note that if your script is in this folder, it will also be encrypted and renamed.. so you need to check for the name and exclude it from the loop

How to write unix regular expression to select for specific files in a cp for-loop

I've got a directory with a bunch of files. Instead of describing the filenames and extensions, I'll just show you what is in the directory:
P01_1.atag P03_3.tgt P05_6.src P08_3.atag P10_5.tgt
P01_1.src P03_4.atag P05_6.tgt P08_3.src P10_6.atag
P01_1.tgt P03_4.src P06_1.atag P08_3.tgt P10_6.src
P01_2.atag P03_4.tgt P06_1.src P08_4.atag P10_6.tgt
P01_2.src P03_5.atag P06_1.tgt P08_4.src P11_1.atag
P01_2.tgt P03_5.src P06_2.atag P08_4.tgt P11_1.src
P01_3.atag P03_5.tgt P06_2.src P08_5.atag P11_1.tgt
P01_3.src P03_6.atag P06_2.tgt P08_5.src P11_2.atag
P01_3.tgt P03_6.src P06_3.atag P08_5.tgt P11_2.src
P01_4.atag P03_6.tgt P06_3.src P08_6.atag P11_2.tgt
P01_4.src P04_1.atag P06_3.tgt P08_6.src P11_3.atag
P01_4.tgt P04_1.src P06_4.atag P08_6.tgt P11_3.src
P01_5.atag P04_1.tgt P06_4.src P09_1.atag P11_3.tgt
P01_5.src P04_2.atag P06_4.tgt P09_1.src P11_4.atag
P01_5.tgt P04_2.src P06_5.atag P09_1.tgt P11_4.src
P01_6.atag P04_2.tgt P06_5.src P09_2.atag P11_4.tgt
P01_6.src P04_3.atag P06_5.tgt P09_2.src P11_5.atag
P01_6.tgt P04_3.src P06_6.atag P09_2.tgt P11_5.src
P02_1.atag P04_3.tgt P06_6.src P09_3.atag P11_5.tgt
P02_1.src P04_4.atag P06_6.tgt P09_3.src P11_6.atag
P02_1.tgt P04_4.src P07_1.atag P09_3.tgt P11_6.src
P02_2.atag P04_4.tgt P07_1.src P09_4.atag P11_6.tgt
P02_2.src P04_5.atag P07_1.tgt P09_4.src P12_1.atag
P02_2.tgt P04_5.src P07_2.atag P09_4.tgt P12_1.src
P02_3.atag P04_5.tgt P07_2.src P09_5.atag P12_1.tgt
P02_3.src P04_6.atag P07_2.tgt P09_5.src P12_2.atag
P02_3.tgt P04_6.src P07_3.atag P09_5.tgt P12_2.src
P02_4.atag P04_6.tgt P07_3.src P09_6.atag P12_2.tgt
P02_4.src P05_1.atag P07_3.tgt P09_6.src P12_3.atag
P02_4.tgt P05_1.src P07_4.atag P09_6.tgt P12_3.src
P02_5.atag P05_1.tgt P07_4.src P10_1.atag P12_3.tgt
P02_5.src P05_2.atag P07_4.tgt P10_1.src P12_4.atag
P02_5.tgt P05_2.src P07_5.atag P10_1.tgt P12_4.src
P02_6.atag P05_2.tgt P07_5.src P10_2.atag P12_4.tgt
P02_6.src P05_3.atag P07_5.tgt P10_2.src P12_5.atag
P02_6.tgt P05_3.src P07_6.atag P10_2.tgt P12_5.src
P03_1.atag P05_3.tgt P07_6.src P10_3.atag P12_5.tgt
P03_1.src P05_4.atag P07_6.tgt P10_3.src P12_6.atag
P03_1.tgt P05_4.src P08_1.atag P10_3.tgt P12_6.src
P03_2.atag P05_4.tgt P08_1.src P10_4.atag P12_6.tgt
P03_2.src P05_5.atag P08_1.tgt P10_4.src
P03_2.tgt P05_5.src P08_2.atag P10_4.tgt
P03_3.atag P05_5.tgt P08_2.src P10_5.atag
P03_3.src P05_6.atag P08_2.tgt P10_5.src
I have a file that is just outside of this directory that I need to copy to all of the files that end with "_1.src" inside the directory.
I'm working with unix in the Terminal app, so I tried writing this for loop, but it rejected my regular expression:
for .*1.src in ./
> do
> cp ../1.src
> done
I've only written regular expressions in Python before and have minimal experience, but I was under the impression that .* would match any combination of characters. However, I got the following error message:
-bash: `.*1.src': not a valid identifier
I then tried the same for loop with the following regular expression:
^[a-zA-Z0-9_]*1.src$
But I got the same error message:
-bash: `^[a-zA-Z0-9_]*1.src$': not a valid identifier
I tried the same regular expression with and without quotation marks, but it always gives the same 'not a valid identifier' error message.
Tested on Bash 4.4.12, the following is possible:
$ for i in ./*_1.src; do echo "$i" ; done
This will echo every file ending with _1.src to the screen, thus moving it will be possible as well.
$ mkdir tmp
$ for i in ./*_1.src; do mv "$i" tmp/.; done
I've tested with the following data:
$ touch P{1,2}{0,1,2}_{0..6}.{src,tgt,atag}
$ ls
P10_0.atag P10_5.src P11_3.tgt P12_2.atag P20_0.src P20_5.tgt P21_4.atag P22_2.src
P10_0.src P10_5.tgt P11_4.atag P12_2.src P20_0.tgt P20_6.atag P21_4.src P22_2.tgt
P10_0.tgt P10_6.atag P11_4.src P12_2.tgt P20_1.atag P20_6.src P21_4.tgt P22_3.atag
P10_1.atag P10_6.src P11_4.tgt P12_3.atag P20_1.src P20_6.tgt P21_5.atag P22_3.src
P10_1.src P10_6.tgt P11_5.atag P12_3.src P20_1.tgt P21_0.atag P21_5.src P22_3.tgt
P10_1.tgt P11_0.atag P11_5.src P12_3.tgt P20_2.atag P21_0.src P21_5.tgt P22_4.atag
P10_2.atag P11_0.src P11_5.tgt P12_4.atag P20_2.src P21_0.tgt P21_6.atag P22_4.src
P10_2.src P11_0.tgt P11_6.atag P12_4.src P20_2.tgt P21_1.atag P21_6.src P22_4.tgt
P10_2.tgt P11_1.atag P11_6.src P12_4.tgt P20_3.atag P21_1.src P21_6.tgt P22_5.atag
P10_3.atag P11_1.src P11_6.tgt P12_5.atag P20_3.src P21_1.tgt P22_0.atag P22_5.src
P10_3.src P11_1.tgt P12_0.atag P12_5.src P20_3.tgt P21_2.atag P22_0.src P22_5.tgt
P10_3.tgt P11_2.atag P12_0.src P12_5.tgt P20_4.atag P21_2.src P22_0.tgt P22_6.atag
P10_4.atag P11_2.src P12_0.tgt P12_6.atag P20_4.src P21_2.tgt P22_1.atag P22_6.src
P10_4.src P11_2.tgt P12_1.atag P12_6.src P20_4.tgt P21_3.atag P22_1.src P22_6.tgt
P10_4.tgt P11_3.atag P12_1.src P12_6.tgt P20_5.atag P21_3.src P22_1.tgt P10_5.atag
P11_3.src P12_1.tgt P20_0.atag P20_5.src P21_3.tgt P22_2.atag
Apparently, my previous answer didn't work. But this seems to:
$ for x in `echo ./P[01][012]_1.src`; do echo "$x"; done
./P01_1.src
./P02_1.src
So, when you run this echo alone, this pattern gets expanded into many names:
$ echo ./P[01][012]_1.src # note that the 'regex' is not enclosed in quotes
./P01_1.src ./P02_1.src
And then you can iterate over these names in a loop.
BTW, as noted in the comments, you don't even need that echo, so you can plug the pattern right into the loop:
for x in ./P[01][012]_1.src; do echo "$x"; done
Please correct me if your goal is something other than
"overwrite many existing files sharing a common suffix with the contents of a single file"
find /path/to/dest_dir -type f -name "*_1.src" |xargs -n1 cp /path/to/source_file
Note that without the -maxdepth 1 option, find will recurse through your destination directory.
Thanks to everyone; this is what ended up working:
for x in `echo ./P[0-9]*_1.src`
> do
> cp ../1.src "$x"
> done
This loop allowed me to copy the contents of the one file to all of the files in the subdirectory that ended with "_1.src"

How to move fossil repository subdirectory tree (to elsewhere within same repository, retaining tree levels)

I have a directory with multiple levels of subdirectories inside a fossil checkout, that I want to move to another location in a different subdirectory and retain the multiple-level directory structure.
For instance, to move a1 into a2 below, to go from having (handwritten like an abbreviated find command output):
a1/
a1/b/
a1/b/files
a1/c/
a1/c/d/
a1/c/d/more-files
a2/
I want fossil mv --hard a1 a2 to result in:
a2/a1/
a2/a1/b/
a2/a1/b/files
a2/a1/c/
a2/a1/c/d/
a2/a1/c/d/more-files
Just like the normal unix mv command would result in. Ideally with the history of the mv kept so it can be merged into another branch with any changes to files and more-files intact; as I could just fossil remove the files then re-add them as fresh files, but this is an uglier solution than I'd like.
fossil mv command (in v1.33 on Linux) loses the multiple levels and I end up with all files from lower level subdirectories moved into the top level directory of the new location.
One solution was to write a script to move each directoy individually, a level at a time, so it retained the structure. I would like to suggest this functionality to the fossil developer(s). I may post the script (below, with another dependency script included below that) to my github (user jgbreezer) sometime, but for now, this script (which I called fossilmvtree). It ignores files in the checkout not in fossil and will leave the old files/dirs where there are any (I don't believe it deletes them):
#!/bin/bash
# $1=source tree
# $2=dest. dir
# supports fossil mv options
# moves single source tree as-is to under/new dest.dir (not reducing dir levels to flat structure under dest dir)
exclude=''
usage () {
cat >&2 <<EOF
Usage: fossilmvtree [-x|--exclude= exclude_dirname] source dest"
-x option may be specified multiple times; do not specify full paths, just last
(filename/aka basename) of a directory to exclude from the move.
Command-line arguments are always included.
EOF
}
while [ -z "${1##-*}" ]
do
case "$1" in
-x|--exclude|--exclude=*)
if [[ "${1#--exclude=}" == "$1" ]]
then
# separate arg, '--exclude=' not used
shift
arg="$1"
else
arg="${1#--exclude=}"
fi
excinfo="$excinfo $arg"
# pruning is efficient
exclude="$exclude -type d -name '${arg//\'/\\\'}' -prune -o"
;;
--case-sensitive)
fossilopts="$fossilopts $1 $2"; shift;;
-*)
fossilopts="$fossilopts $1";;
esac
shift
done
echo "excluding paths: $excinfo"
echo "fossil mv options: $fossilopts"
[ $# -eq 2 ] || { usage; exit 1; }
mv="$(which fossilmvrev 2>/dev/null)" || { usage; echo "error:Missing fossilmvrev" >&2; exit 1; }
src="$1"
srcdir="$(basename "$src")"
dst="$2"
if [ -f "$dst" ]
then
# move src to new subdir of dst; otherwise we're renaming and moving
[ -d "$dst" ] || { echo "error:Destination '$dst' exists but is not a directory" >&2; exit 1; }
dst="$dst/$srcdir"
fi
#could set safe PATH (-execdir is cautious of relative/empty paths in $PATH but fossil binary might not be in std.location): PATH=/bin:/usr/bin:/usr/local/bin
eval find "$src" $exclude -type d -printf '%P\\n' | {
while read -r dir
do
[ -z "$dir" ] || [[ "$src/$dir" == "$dst/$dir" ]] && continue
echo
echo "fossil mv $src/$dir/* $dst/$dir/"
mkdir -p "$dst/$dir" || exit 1
find "$src/$dir" -maxdepth 1 \! -type d -exec "$mv" $fossilopts "$dst/$dir" '{}' +
rmdir "$src/$dir" # tidy up, as we only moved the files above (fossil doesn't really manage dirs)
# if rmdir fails due to remaining files, let user manage that (rmdir will complain to stderr if so) -
# likely to be unversioned files they might have forgotten about, shouldn't delete without user's knowledge.
done
}
It was only really tested once or twice on my specific fossil checkout, though written ready to be a re-usable script; please check the diffs (suggest do a clean checkout somewhere else and run it on that, then diff against your regular one using "diff -qr" or something before committing to check it behaved itself).
Careful if using the -x/exclude option, I wasn't sure that worked properly.
It depends on fossilmvrev script:
#!/bin/sh
# switch order of move arguments around to work with find -exec ... +
opts=''
while [ -z "${1##-*}" ]
do
case "$1" in
--case-sensitive) opts="$opts $1 $2"; shift 2;;
*) opts="$opts $1"; shift;;
esac
done
destdir="$1"
shift
fossil mv $opts "$#" $destdir
I think there is a much simpler solution
(this is under Windows but similarly for Linux)
md a2\a1
fossil mv a1 a2/a1 --hard

VisualSVN post-commit hook with batch file

I'm running VisualSVN on a Windows server.
I'm trying to add a post-commit hook to update our staging project whenever a commit happens.
In VisualSVN, if I type the command in the hook/post-commit dialog, everything works great.
However, if I make a batch file with the exact same command, I get an error that says the post-commit hook has failed. There is no additional information.
My command uses absolute paths.
I've tried putting the batch file in the VisualSVN/bin directory, I get the same error there.
I've made sure VisualSVN has permissions for the directories where the batch file is.
The only thing I can think of is I'm not calling it correctly from VisualSVN. I'm just replacing the svn update command in the hook/post-commit dialog with the batch file name ("c:\VisualSVN\bin\my-batch-file.bat") I've tried it with and without the path (without the path it doesn't find the file at all).
Do I need to use a different syntax in the SVNCommit dialog to call the batch file? What about within the batch file (It just has my svn update command. It works if I run the batch file from the command line.)
Ultimately I want to use a batch file because I want to do a few more things after the commit.
When using VisualSVN > Select the Repo > Properties > Hooks > Post-commit hook.
Where is the code I use for Sending an Email then running a script, which has commands I want to customize
"%VISUALSVN_SERVER%\bin\VisualSVNServerHooks.exe" ^
commit-notification "%1" -r %2 ^
--from support#domainname.com --to "support#domainname.com" ^
--smtp-server mail.domainname.com ^
--no-diffs ^
--detailed-subject
--no-html
set PWSH=%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe
%PWSH% -command $input ^| C:\ServerScripts\SVNScripts\post-commit-wp.ps1 %1 %2
if errorlevel 1 exit %errorlevel%
The script file is located on C:\ServerScripts\SVNScripts\
post-commit-wp.ps1 and I pass in two VisualSVN variables as %1 and %2
%1 = serverpathwithrep
%2 = revision number
The script file is written in Windows PowerShell
# PATH TO SVN.EXE
$svn = "C:\Program Files\VisualSVN Server\bin\svn.exe"
$pathtowebistesWP = "c:\websites-wp\"
# STORE HOOK ARGUMENTS INTO FRIENDLY NAMES
$serverpathwithrep = $args[0]
$revision = $args[1]
# GET DIR NAME ONLY FROM REPO-PATH STRING
# EXAMPLE: C:\REPOSITORIES\DEVHOOKTEST
# RETURNS 'DEVHOOKTEST'
$dirname = ($serverpathwithrep -split '\\')[-1]
# Combine ServerPath with Dir name
$exportpath = -join($pathtowebistesWP, $dirname);
# BUILD URL TO REPOSITORY
$urepos = $serverpathwithrep -replace "\\", "/"
$url = "file:///$urepos/"
# --------------------------------
# SOME TESTING SCRIPTS
# --------------------------------
# STRING BUILDER PATH + DIRNAME
$name = -join($pathtowebistesWP, "testscript.txt");
# CREATE FILE ON SERVER
New-Item $name -ItemType file
# APPEND TEXT TO FILE
Add-Content $name $pathtowebistesWP
Add-Content $name $exportpath
# --------------------------------
# DO EXPORT REPOSITORY REVISION $REVISION TO THE ExportPath
&"$svn" export -r $revision --force "$url" $exportpath
I added comments to explain each line and what it does. In a nutshell, the scripts:
Gets all the parameters
Build a local dir path
Runs SVN export
Places files to a website/publish directory.
Its a simple way of Deploying your newly committed code to a website.
Did you try to execute batch file using 'call' command? I mean:
call C:\Script\myscript.bat
I was trying the same thing and found that you also must have the script in the hooks folder.. the bat file that is.