First of all, the server runs Solaris.
The context of my question is Informatica PowerCenter.
I need to list files situated in the Inbox directory. Basically, the outcome should be one file list by type of file. The different file types are distinguished by the file name. I don't want to update the script every time a new file type starts to exist so I was thinking of a parameterized shell script with the regex, the inbox directory and the file list
An example:
/Inbox/ABC.DEFGHI.PAC.AE.1236547.49566
/Inbox/ABC.DEFGHI.PAC.AE.9876543.21036
/Inbox/DEF.JKLMNO.PAC.AI.1236547.49566
... has to result in 2 list files containing the path and file name of the listed files:
/Inbox/PAC.AE.FILELIST
-->/Inbox/ABC.DEFGHI.PAC.AE.1236547.49566
-->/Inbox/ABC.DEFGHI.PAC.AE.9876543.21036
/Inbox/PAC.AI.FILELIST
-->/Inbox/DEF.JKLMNO.PAC.AI.1236547.49566
Assuming all input files follow the convention you indicate (when splitting on dots, the 3rd and 4th column determine the type), this script might do the trick:
#! /usr/bin/env bash
# First parameter or current directory
INPUTDIR=${1:-.}
# Second parameter (or first input directory if not given)
OUTPUTDIR=${2:-$INPUTDIR}
# Filter out directories
INPUTFILES=$(ls -p $INPUTDIR | grep -v "/")
echo "Input: $INPUTDIR, output: $OUTPUTDIR"
for FILE in $INPUTFILES; do
FILETYPE=$(echo $FILE | cut -d. -f3,4)
COLLECTION_FILENAME="$OUTPUTDIR/${FILETYPE:-UNKNOWN}.FILELIST"
echo "$FILE" >> $COLLECTION_FILENAME
done
Usage:
./script.sh Inbox Inbox/collections
Will read all files (not directories) from Inbox, and write the collection files to Inbox/collections. Filenames inside collections should be sorted alphabetically.
Related
I have a file containing various paths such as:
/home/user/Desktop/Bash/file1.txt
/home/user/Desktop/Bash/file2.txt
/home/user/Desktop/Bash/file3.txt
/home/user/Desktop/Bash/moreFiles/anotherFile.txt
/home/user/Desktop/text/prettyFile.txt
And I receive a input from user that contains the directory, such as:
/home/user/Desktop/Bash/
And I usually save this expression into regex to find all the files in the directory by grep. However, if the folder has more folders, it includes them as well, but I want to only include the files in the directory that was entered by the user. My desired output is should be this:
/home/user/Desktop/Bash/file1.txt
/home/user/Desktop/Bash/file2.txt
/home/user/Desktop/Bash/file3.txt
But it keeps including
/home/user/Desktop/Bash/moreFiles/anotherFile.txt
which I don't want and I need to do it inside a bash script.
You can use this grep command to get just the files directly under given path skipping sub-directories:
s='/home/user/Desktop/Bash/'
grep -E "$s[^/]+/?$" file
/home/user/Desktop/Bash/file1.txt
/home/user/Desktop/Bash/file2.txt
/home/user/Desktop/Bash/file3.txt
Can we grep for multiple patterns in a folder containing n number of files. And if a match found for each and every pattern create a directory and push the files of similar pattern type into same directory likewise the others.
For example : I am having a folder name : X. X can have multiple sub folders and multiple files inside them.
I want to search for a pattern like This code is from. If a match of this string is found in multiple files in X folder create a directory named dir1 and push all the matched files into dir1.
And the same for other patterns matches also if the matches are found create directories and push the files into respective directories.
I tried of searching with grep can found all pattern matched files but parallely I can't do mkdir . In this way for n matches of patterns in X n dir it should create. Searching is fine but having issue with directories creation parallely.
one way to get the same folder structure is, unfortunately, not to use xargs cp -t dir, but instead copy one-by-one with rsync, e.g.,
grep -irl "Version" | xargs -I{} rsync -a "{}" "dir/{}"
I mean, it's not elegant, but you could use embedded for loops with an array of search strings.
EDIT: Missed the part about separate folders for different match strings. Changes are below.
#!/bin/bash
#Assuming:
#patarr is an array containing all paterns
#test/files is the location of files to be searched
#test/found is the location of matching files
for file in test/files/* #*/
#This loop runs for every file in test/files/. $file holds the filename of the current file
do
for ((i=0;i<${#patarr[#]};i++))
#This loop runs once for every index in the patarr array. $i holds the current loop number
do
if [[ $(cat $file | grep ${patarr[$i]} | wc -l) -gt 0 ]]
#if grep finds at least one match using the pattern in patarr with index "i"
then
#cp $file temp/found/ #Old code, see edit above
mkdir -p temp/found/${pararr[$i]}
#Makes a folder with the name as our search string. -p means no error if the folder already exists.
cp $file temp/found/${pararr[$i]}/
#Copies the file into said folder
fi
done
done
I am trying to take a text file that contains a list of files and copy them all to a directory. Within this directory, they will have unique directory names. An example of text file the structure can be seen below:
/data/isip/data/tuh_eeg/v0.6.0/edf/001/00000003/s01_2011_11_01/a_.edf
/data/isip/data/tuh_eeg/v0.6.0/edf/001/00000003/s01_2011_11_01/a_1.edf
/data/isip/data/tuh_eeg/v0.6.0/edf/001/00000003/s02_2011_11_11/a_.edf
/data/isip/data/tuh_eeg/v0.6.0/edf/001/00000003/s02_2011_11_11/a_1.edf
/data/isip/data/tuh_eeg/v0.6.0/edf/001/00000005/s01_2009_02_13/a_.edf
/data/isip/data/tuh_eeg/v0.6.0/edf/001/00000005/s02_2010_10_02/a_.edf
/data/isip/data/tuh_eeg/v0.6.0/edf/001/00000005/s03_2010_10_02/a_.edf
/data/isip/data/tuh_eeg/v0.6.0/edf/001/00000005/s04_2010_10_03/a_.edf
/data/isip/data/tuh_eeg/v0.6.0/edf/001/00000005/s04_2010_10_03/a_1.edf
/data/isip/data/tuh_eeg/v0.6.0/edf/001/00000005/s04_2010_10_03/a_2.edf
/data/isip/data/tuh_eeg/v0.6.0/edf/001/00000005/s04_2010_10_03/a_3.edf
/data/isip/data/tuh_eeg/v0.6.0/edf/001/00000005/s04_2010_10_03/a_4.edf
I need a shell command or an EMACS macro to go through this list and copy them all to unique directories within the current working directory. The unique directory will depend on the file; for example, for the first two files, the directory would be
/001/00000003/s01_2011_11_01/
I have tried doing this using an EMACS macro, but I was not able to get it to work. A shell command or EMACs macro would work.
Something as simple as:
cat list | sed "s/^.*edf\/\(.*\)\/\(.*\)$/mkdir -p root_dir\/\1 \&\& cp \0 root_dir\/\1\/\2/" | sh
If on OSX - install gnu-sed and use gsed instead of sed. Run command without | sh to see what it'll do. Make sure to tweak root_dir, of course.
I'm very new to shell script and therefore I don't now very much about it.
I have an application, which creates a java file with a half unknown name, and now I try to write a script, which needs this name.
The known name of the file is /target/plugin-<dyn>.jar, the <dyn> part is unknown and could be nearly anything (btw it is mostly a version number with variable text parts).
Now I want to save plugin-<dyn> (without the .jar) in a variable for later use. I looked very much in the internet, but I can't find a solution.
If you need get file name without extension .jar. You can refer my bash script below:
# for loop all files in target directory that matched plugin-*.jar
for f in target/plugin-*.jar
do
# print file name without extension .jar
echo ${f%.*}
done
UPDATED:
# for loop all files in target directory that matched plugin-*.jar
for f in target/plugin-*.jar
do
# print file name without extension .jar
filename="${f##*/}" # get plugin-*.jar
echo ${filename%.*} # print plugin-* without jar
done
try this
filename="/target/plugin-<dyn>.jar"
echo ${filename} | awk -F [/.] '{print $(NF - 1)}'
echo ${filename} | sed 's/.*\/\([^/]*\)\.jar/\1/'
but if <dyn> has a slash, comma or point. it may not work
I'm (still) not a shell-wizard, but I'm trying to find a way to create directories and files from a list of file names.
Let's take this source file (source.txt) as an example:
README.md
foo/index.html
foo/bar/README.md
foo/bar/index.html
foo/baz/README.md
I'll use this command to remove empty lines and trim useless spaces:
$ more source.txt | sed '/^$/d;s/^ *//;s/ *$//'
It will give me this list:
README.md
foo/index.html
foo/bar/README.md
foo/bar/index.html
foo/baz/README.md
Now I'm trying to loop on every line and create the related file (it it doesn't already exists), with it's parents directories.
How could I do this?
Ideally, I would put this script in an alias to quickly use it.
As always, posting a question brings me to the end of the problem...
I came to a satisfying solution, using dirname and basename in a for .. in loop:
for i in `cat source.txt | sed '/^$/d;s/^ *//;s/ *$//'`;
do mkdir -p `dirname $i`;
touch `echo $(dirname $i)$(echo "/")$(basename $i)`;
done
This one-line command will:
read the file names list
create directories
create empty files in their own directory