Get specific file with name like abc_yyyymmdd_hhmmss.csv from directory using mget.
Example files in a folder:
abc_20221202_145911.csv
abc_20221202_145921.csv
abc_20221202_145941.csv
abc_20181202_145941.csv
But, I want to ignore hhmmss part. I want to get all files with abc_20221202_*.csv
How to include * in mget.
My code below:
File=abc_
Date=20221202
Filename=$File$Date"_*".csv
// Assume I have sftp connection established and I am in directory //where files with above naming convention are present. As I can //download the file when hardcoding exact file name during testing
conn=`lftp $protocol://$user:$password#$sftp_server -p $port <<EOF>/error.log
cd $path
mget $Filename
EOF`
The script is able to find the file but not able to retrieve it from the server.
But, if I remove * and provide the entire file name abc_20221202_145941.csv it will download the file. Why is * causing issue in retrieving the file
Assuming mget actually accepts regex:
Currently your regexp is looking for files that match abc_20221202_(underscore any number of times).csv
Just add a . before the * so it matches any character after the underscore any number of times before the .csv
Like so:
Filename=$File$Date"_.*".csv
If mget doesn't actually support regex, just use wget instead:
wget -r -np -nH -A "abc_20221202_.*\.csv" --ftp-user=user --ftp-password=psd ftp://ip/*
I think the backtick symbol was causing the problem when using *. Once I removed the ` (backtick) and used below command, it worked fine.
lftp -p $port $protocol://$user:$password#$sftp_server <<EOF>/error.log
cd $path
lcd $targetPath
mget $Filename
EOF
You probably missed an underscore between File and Date. A good way to debug such problems is to enable debug (“debug” command) and command logging (set cmd:trace true)
Related
I would like to use Vim to find certain string and replace it with another. For every replacements, it should ask for confirmation similar to what %s/foo/replace/gc does for a single file in Vim.
What have I tried?
sed: It doesn't do interactive replacements.
One of the comments in the following this link suggests vim -esnc '%s/foo/bar/g|:wq' file.txt. I tried vim -esnc '%s/foo/bar/gc|:wq' file.txt (used gc instead of g). Now the terminal gets stuck.
Emacs xah-find-replace package. Unfortunately it didn't do interactive replacements as promised in the link.
Combining :argdo with the substitute command would be the recommended way to do this.
You can populate the args by either opening all the files vim *.txt or manually populate this after opening vim using the command:
:args `find . -type f -name '*.txt'`
Now set hidden using the command:
:set hidden
this is required so that you're not prompted to save the file when switching from one buffer to the other. Refer, :h hidden for more information.
Now use the substitute command like you're used to, prefixing the argdo to perform this for every file in the argslist
:silent argdo %s/pattern/replace/gec
The silent is optional and just mutes the reporting. The e flag is to stop reporting the error no matches found message in some of the buffers
Now after replace, you can write the changes using the following command
:argdo update
This will write buffers that were modified only.
If you are looking for an interactive mode of replacement, it is easier to do it with vim.
vim -c '%s/PATTERN/REPLACEMENT/gc' -c 'wq' FILENAME
The stuck terminal in your case is due to piping the save command to the replacement string, as it does not allow the interactive mode to come in to action. And it is not a stuck terminal, if you type "yes" and press enter it should still show you the expected result.
In case multiple files are involved which is spread across multiple subdirectories, using find command with for loop will help as mentioned below:
for FILENAME in `find DIRECTORYPATH -type f -name *.txt`
do
vim -c '%s/PATTERN/REPLACEMENT/gc' -c 'wq' $FILENAME
done
In bash turn on double star to list all files in all subdirectories:
shopt -s globstar
Now start vim once with all files and run the substitute command for all files, then save and exit:
vim -c 'set nomore' -c 'argdo %s/foo/bar/gc' -c xa **/*.txt
I try to get specific files with wget and a regular expression.
The files are infographics.jpg and informatics.jpg
Here the command line:
wget -r -nd -P test -A jpg --accept-regex '.*\/i.*.jpg'
It downloads every jpg it can find instead of just the two files beginning with an "i".
If I add an "n" after the "i"...
wget -r -nd -P test -A jpg --accept-regex '.*\/in.*.jpg'
... that works perfectly, downloading the two files beginning with "in".
But not otherwise. I found that "/i" means case-insensitive. So I tried different ways to make sure the "i" is taken as a letter, not a switch, like "[i]", ... No luck.
Is this a /i problem? And may I get rid of that?
More probably it's for greediness of regex. Yours matches from beginning up to an i (that might exist on a path, not within filename) then up to a .jpg sequence of characters - could not be at the end. So you need to restrict regex a bit:
/i[^/]*\.jpg$
This matches a / immediately followed by an i then without jumping over different parts of URL ([^/]*, filename only) ends to .jpg that meets end of URL as well.
How can I rename many files. Remove the digits at the beginning.
I have a Mac. All the files are in the same folder.
The pattern is:
1, 2 or 3 digits - any name.php
With Regular Expression, I think it would be:
\d*-(.*).php
For example:
1-marketing.php
2-3D.php
3-without.php
I want to remove the numbers and the dash at the beginning.
In the example it would be:
marketing.php
3D.php
without.php
What I have explored two ways:
Select the files > ctrl click > rename items. This is a fantastic method to change the name of files. But I think it cannot be used in this case. If I understand, it does not support Regex. Am I right?
Terminal. I am not very familiar with terminal. I tried mv 1-marketing.php marketing.php It works for 1 file, but how can I do the same for many? I am new with the terminal. If it can be done, please explain the basic.
Open the terminal app in Mac OS X and navigate to the folder containing the .php files
cd /my/path/to-php-files/
and run the below command on the command-line.
for file in *.php; do mv -v "$file" "${file#*-}"; done
The bash parameter expansion syntax ${file#*-} removes the characters before - from the beginning, so ideally 3-number-without.php becomes number-without.php
(or) use the perl rename utility not available by default in Mac OS, you can download and install it with homebrew 🍺:
brew install rename
and do
rename -n 's/^(\d+)-(.*)/$2/' *.php
The -n is just for a dry-run to see how the files are to be renamed, remove it as
rename 's/^(\d+)-(.*)/$2/' *.php
for the actual renaming.
I'm very new to shell script and therefore I don't now very much about it.
I have an application, which creates a java file with a half unknown name, and now I try to write a script, which needs this name.
The known name of the file is /target/plugin-<dyn>.jar, the <dyn> part is unknown and could be nearly anything (btw it is mostly a version number with variable text parts).
Now I want to save plugin-<dyn> (without the .jar) in a variable for later use. I looked very much in the internet, but I can't find a solution.
If you need get file name without extension .jar. You can refer my bash script below:
# for loop all files in target directory that matched plugin-*.jar
for f in target/plugin-*.jar
do
# print file name without extension .jar
echo ${f%.*}
done
UPDATED:
# for loop all files in target directory that matched plugin-*.jar
for f in target/plugin-*.jar
do
# print file name without extension .jar
filename="${f##*/}" # get plugin-*.jar
echo ${filename%.*} # print plugin-* without jar
done
try this
filename="/target/plugin-<dyn>.jar"
echo ${filename} | awk -F [/.] '{print $(NF - 1)}'
echo ${filename} | sed 's/.*\/\([^/]*\)\.jar/\1/'
but if <dyn> has a slash, comma or point. it may not work
I've got a file structure that looks like:
A/
2098765.1ext
2098765.2ext
2098765.3ext
2098765.4ext
12345.1ext
12345.2ext
12345.3ext
12345.4ext
B/
2056789.1ext
2056789.2ext
2056789.3ext
2056789.4ext
54321.1ext
54321.2ext
54321.3ext
54321.4ext
I need to rename all the files that begin with 20 to start with 10; i.e., I need to rename B/2022222.1ext to B/1022222.1ext
I've seen many of the other questions regarding renaming multiple files, but couldn't seem to make it work for my case. Just to see if I can figure out what I'm doing before I actually try to do the copy/renaming I've done:
for file in "*/20?????.*"; do
echo "{$file/20/10}";
done
but all I get is
{*/20?????.*/20/10}
Can someone show me how to do this?
You just have a little bit of incorrect syntax is all:
for file in */20?????.*; do mv $file ${file/20/10}; done
Remove quotes from the argument to in. Otherwise, the filename expansion does not occur.
The $ in the substitution should go before the bracket
Here is a solution which use the find command:
find . -name '20*' | while read oldname; do echo mv "$oldname" "${oldname/20/10}"; done
This command does not actually do your bidding, it only prints out what should be done. Review the output and if you are happy, remove the echo command and run it for real.
Just wanna add to Explosion Pill's answer.
On OS X though, you must say
mv "${file}" "${file_expression}"
Or the mv command does not recognize it.
Brace expansions like :
{*/20?????.*/20/10}
can't be surrounded by quotes.
Instead, try doing (with Perl rename) :
rename 's/^10/^20/' */*.ext
You can do this using the Perl tool rename from the shell prompt. (There are other tools with the same name which may or may not be able to do this, so be careful.)
If you want to do a dry run to make sure you don't clobber any files, add the -n switch to the command.
note
If you run the following command (linux)
$ file $(readlink -f $(type -p rename))
and you have a result like
.../rename: Perl script, ASCII text executable
then this seems to be the right tool =)
This seems to be the default rename command on Ubuntu.
To make it the default on Debian and derivative like Ubuntu :
sudo update-alternatives --set rename /path/to/rename
The glob behavior of * is suppressed in double quotes. Try:
for file in */20?????.*; do
echo "${file/20/10}";
done