I have three text file.
C:\content1.txt
C:\content2.txt
C:\content3.txt
text file contains random links
https://example2.com/file/casdqwe/test1.rar
https://example2.com/file/casdqwe/test2.rar
https://example5.com/file/casdqwe/test3.rar
i want to remove the the domain names and save the last part only. output
test1.rar
test2.rar
test3.rar
and save it in original file no need any confirmation.
notepad++ regex replace work but i am not sure how to do it in
using Powershell/VBS script any will work fine.
Find What: https://example\.com[^\s\[\]<'\"]+/([^\s\[\]<'\"]+)
Replace with: $1
and powershell i have something like that
(Get-Content -path C:\content1.txt -Raw) -replace "https://example2.com/file/asdcb6af26/test.rar" -replace "https://example2.com/file/(.*)\/(.*)", '$2'
Use these simple powershell commands:
(Get-Content C:\content1.txt) -creplace '(?s)^.*/', '' | Set-Content C:\content1.txt
(Get-Content C:\content2.txt) -creplace '(?s)^.*/', '' | Set-Content C:\content2.txt
(Get-Content C:\content3.txt) -creplace '(?s)^.*/', '' | Set-Content C:\content3.txt
Let me know if you have any questions.
Refer to your previous question here, you can give a try for this batch file that use Regex in vbscript.
#echo off
Mode 85,35 & color 0A
Title Replace Multi String using Regex with vbscript into Folder with text files
Set "Source_Folder=C:\test"
Set "Backup_Folder=%userprofile%\Backup_Contents\"
Rem :: Just make a backup of your folder and its contents if something went wrong!
If Not Exist "%Backup_Folder%" XCopy "%Source_Folder%" "%Backup_Folder%" /D /Y /E /F >%~dp0BackupLogFile.txt
Set "VBSFILE=%tmp%\%~n0.vbs" & Call :CreateVBS
Set "TmpFile=%Temp%\%~n0.tmp"
for /R "%Source_Folder%" %%f in (*.txt) do (
echo( ------------------------------------------
echo Replacing Contents of "%%f"
echo( ------------------------------------------
Call :Search_Replace "%%f" "%TmpFile%"
Move /Y "%TmpFile%" "%%f">nul
)
If Exist "%VBSFILE%" Del "%VBSFILE%"
Timeout /T 3 /NoBreak>nul & Exit
::-----------------------------------------------------------------------------
:CreateVBS
(
echo WScript.StdOut.WriteLine Search_Replace(Data^)
echo Function Search_Replace(Data^)
echo Dim strPattern, strReplace, strResult,oRegExp
echo Data = "%~1"
echo Data = WScript.StdIn.ReadAll
echo strPattern1 = "(\x22<|<)([\s\S]*?)(/>\x22|>| />\x22| />| \x22>)"
echo strReplace1 = "[abc]$2[/abc]"
echo strPattern2 = "(http:\/\/|https:\/\/)(.+)[^\s\[\]<]\/([^\s\[\]<].+)"
echo strReplace2 = "$3"
echo Set oRegExp = New RegExp
echo oRegExp.Global = True
echo oRegExp.IgnoreCase = True
echo oRegExp.Pattern = strPattern1
echo strResult1 = oRegExp.Replace(Data,strReplace1^)
echo oRegExp.Pattern = strPattern2
echo strResult2 = oRegExp.Replace(strResult1,strReplace2^)
echo Search_Replace = strResult2
echo End Function
)>"%VBSFILE%"
Exit /b
::----------------------------------------------------------------------------
:Search_Replace <InputFile> <OutPutFile>
Cscript /nologo "%VBSFILE%" < "%~1" > "%~2"
Exit /B
::----------------------------------------------------------------------------
I'm guessing that maybe,
\bhttps?:\/\/.*\/
simply replaced with an empty string might work just fine.
If you wish to simplify/modify/explore the expression, it's been explained on the top right panel of regex101.com. If you'd like, you can also watch in this link, how it would match against some sample inputs.
Related
My bash script uses a perl command to replace a variable within a file, with the contents of another file. Pretty standard I thought, yet I am really struggling here.
#/bin/bash
display_usage() {
echo -e "\nUsage: This script must be run with an environment parameter (file in directory env/)."
echo -e "Example: ./configureEnv <env parameter>\n"
}
# If no env argument supplied, display usage.
if [ $# -eq 0 ]; then
display_usage
exit 1
fi
# Replace the placeholder of FIREBASE_ENV in index.html with the Firebase env settings.
perl -pi -e 's/%FIREBASE_ENV%/`cat testEnvConfig`/g;' index.html
if [ $? -eq 0 ]; then
echo "Updated Firebase settings based on environment file: $1"
exit 0
else
echo "[Error] Environment settings configuration failed. Please check parameters are correct."
exit 1
fi
As you can see, the key line is:
perl -pi -e 's/%FIREBASE_ENV%/`cat testEnvConfig`/g;' index.html
It should replace the placeholder string of %FIREBASE_ENV% with the contents of the file, but instead it replaces replaces %FIREBASE_ENV% with `cat testEnvConfig`.
The replacement in a substitution interpolates variables like double quotes, but it doesn't interpret backticks. You need to specify the /e modifier to evaluate the replacement as code.
s/%FIREBASE_ENV%/`cat testEnvConfig`/ge
You don't need to shell out for this, though. Perl can read a file and store its contents in a variable:
my $config = do { local( #ARGV, $/ ) = 'testEnvConfig'; <> };
# ...
s/%FIREBASE_ENV%/$config/g;
With a (very useful) Perl module Path::Tiny reading a file takes one statement and we have
perl -MPath::Tiny -i.bak -pe'
BEGIN { $f = path("testEnvConfig")->slurp; chomp $f };
s/%FIREBASE_ENV%/$f/g
' index.html
I've added a backup .bak for your safety while testing, and split code over lines for readability.
I remove the final newline from the file. If you actually want it remove that chomp $f
Why your honest attempt doesn't work has been explained, as you need to evaluate the replacement part as code, with /e. However, once you do that, look at the situation:
out of a bash script call a Perl program (the one-liner)
which goes out to the system (a syscall at least, perhaps another shell!)
where yet another program runs (cat)
with the file content sent back to Perl, which then does its thing.
Huh. Better just read that file in Perl, no?
By default the right-hand side of a substitution (s///) works like a double-quoted string. "`cat whatever`" doesn't do anything special in a string; it doesn't run any commands.
You need to use the /e flag:
s/%FIREBASE_ENV%/`cat testEnvConfig`/eg
This tells Perl that the right-hand side is to be evaluated as a block of code, not a string.
Alternatively you can avoid shelling out to cat by reading the file in Perl:
perl -pi -e 'BEGIN { my $file = "testEnvConfig"; open my $fh, "<", $file or die "$file: $!\n"; local $/; $config = readline $fh } s/%FIREBASE_ENV%/$config/g'
This also avoids having to re-read the file for each occurrence of %FIREBASE_ENV%.
I have a large file that I am searching through to locate and replace invalid dates. I’m using a REGEX expression to locate the dates and then determining if they are valid or not. If the script finds an invalid date it needs to replace the date with the current date. For audit purposes I need to record the invalid string and the line number on which the error was found. So far (with some prior help to SO) I have been able to locate the invalid dates, but I have not been able to successfully change them.
This is the code I’m using to locate the invalid dates. How can I locate and change the date in a single pass?
$matchInfos = #(Select-String -Pattern $regex -AllMatches -Path $file)
foreach ($minfo in $matchInfos)
{
#"LineNumber $($minfo.LineNumber)"
foreach ($match in #($minfo.Matches | Foreach {$_.Groups[0].value}))
{
if (([Boolean]($match -as [DateTime]) -eq $false ) -or ([DateTime]::parseexact($match,"MM-dd-yyyy",$null).Year -lt "1800")) {
Write-host "Invalid date on line $($minfo.LineNumber) - $match"
#Add-Content -Path $LOGFILE -Value "Invalid date on line $($minfo.LineNumber) - $match"
# Replace the invalid date with a corrected one
Write-Host "Replacing $match with $(Get-Date -Format "MM-dd-yyyy")"
#Add-Content -Path $LOGFILE -Value "Replacing $match with $(Get-Date -Format "MM-dd-yyyy")"
}
}
}
You have to write out a temporary file with the changes and replace the file with the temporary. Here's one I wrote that will do that part for you:
Windows IT Pro: Replacing Strings in Files Using PowerShell
Example of use:
replace-filestring -pattern 'find' -replacement 'replace' -path myfile.txt -overwrite
With this command, the script will read myfile.txt, replace 'find' with 'replace', write the output to a temporary file, and then replace myfile.txt with the temporary file. (Without the -Overwrite parameter, the script will only output the contents of myfile.txt with the changes.)
Bill
$lines = get-content $file
$len = $lines.count
$bad = #{}
for($i=0;$i-lt$len;$i++){
if($lines[$i] -match ""){
$bad_date = $lines[$i].substring(10) #Get the bad date
$good_date = Get-Date -Format G
$bad["$i"] += #($line[$i])
$lines[$i] = $lines[$i].Replace($bad_date,$good_date)
}
}
$lines > $NewFile
$bad > $bad_date_file
Here is some pseudo code of how I would combat this problem. Not sure how big your file is. Reading and writing could be slow.
How can I return the matched value from a regular expression Perl execution?
I want to further use the match in my batch file.
Batch file:
perl -p -e "ab{2}" c:\file.txt
REM echo %var% how can I get the result from perl?
REM do something with %var%
File file.txt:
abbreviation
Update for zb226:
test
Filename: dynamicFile.txt
Property: some property to neglect
Message: the message I want
Time: dynamicTime
You need the FOR command for that:
#ECHO OFF
FOR /F "delims=" %%I IN ('perl -ne "print $1.\"\n\" if ~/(ab{2})/" c:\file.txt') DO SET match=%%I
ECHO match: %match%
Note that this will return the last match - which does not matter with your example regexp ab{2}, as that will always return abb or nothing.
Edit 1: Forgot to add a newline in the perl onliner - alternatively you can use Andrey's perl -pe ... version, which does not require printing the newline.
Edit 2: Then again, Andrey's one-liner matches the regexp only at the beginning of a line, which is not what your question suggests. Also it prints non-matching lines, as you already noticed. I can't think of an easy fix for the perl -pe version...
Edit 3: Here's a quick and dirty solution for the additional information you've provided in the comment.
#ECHO OFF
FOR /F "usebackq delims=" %%I IN (`perl -e "local $/ = undef; open my $h, '<', $ARGV[0]; my $x = <$h>; print $1 if $x=~/Filename: dynamicFile.txt.*?Message: (.*?)Time/s;" c:\file.txt`) DO SET match=%%I
ECHO match: %match%
With a test file of...
test
Filename: dynamicFile.txt
Property: some property to neglect
Message: the message I want
Time: dynamicTime
----
Filename: someOtherFile.txt
Property: some property to neglect
Message: someMessage
Time: dynamicTime
...this outputs:
match: the message I want
perl -p -e "s/^(ab{2}).*$/\1/" c:\file.txt
perl -lne "print $1 if /(ab{2})/" C:\file.txt
I have a Bash script that is working on my OpenSuSE box, but when copied across to my Ubuntu box, is not working. The script reads in from a file. The file has fields separated by white space (tabs and spaces).
#!/bin/bash
function test1()
{
while read LINE
do
if [[ $LINE =~ "^$" || $LINE =~ "^#.*" ]] ; then
continue;
fi
set -- $LINE
local field1=$1
local field2=$2
done < test.file
}
test1
with test.file containing:
# Field1Header Field2Header
abcdef A-2
ghijkl B-3
There seem to be two problems:
(1) $field2, the one with the hyphen, is blank
(2) The regex to strip out the blank lines and lines that start with # is not working
Anyone know what's wrong? As I said, it works fine on OpenSuSE.
Thanks,
Paul
Apparently, as of bash 3.2 the regular expression should not be quoted. So this should work:
#!/bin/bash
while read LINE
do
if [[ $LINE =~ ^$ || $LINE =~ ^#.* ]] ; then
continue;
fi
set -- $LINE
local field1=$1
local field2=$2
done < test.file
Edit: you should probably use Jo So's answer as it's definitely cleaner. But I was explaining why the regex fails and the reason behind the different behavior between OpenSuse and Ubuntu(different version of bash, very probably)
Quoting is wrong, that probably accounts for the regex failing.
No need to use bashisms.
No need to use set
Try
while read field1 field2 dummy
do
if ! test "${field1%%#*}"
then
continue
fi
# do stuff here
done
EDIT: The obvious version using set
while read -r line
do
if ! test "${line%%#*}"
then
continue
fi
set -- $line
do_stuff_with "$#"
done
On my ubuntu there is no expresion like "=~" for test command. Just use this one:
if [[ $LINE = "" || ${LINE:0:1} = "#" ]] ; then
continue;
fi
The file 'MyFile.txt' has a line in it and a part of that line I need replaced.
Example:
The line in the file is like this
53544THOIN91111160000000
I want to replace '111116' from the existing line in 'MyFile.txt', the thing here is '111116' is a variable and would keep changing. Its basically a Date with the format YYMMDD, i want to read the modified date from another file and replace these numbers in 'MyFile.txt'
Here is the code i tried.
set b=MyFile.txt
for /f "tokens= 1" %%c in (%b%) do (set line=%%c)
Set OLDDate=%line:~11,6%
SET filename="AnotherFile.txt"
FOR %%f IN (%filename%) DO SET filedatetime=%%~tf
SET Month=%filedatetime:~0,2%
SET Date=%filedatetime:~3,2%
SET Year=%filedatetime:~8,2%
SET NEWDate=%Year%%Month%%date%
ECHO OLD DATE = %OLDDate%
ECHO NEW DATE = %NEWDate%
I need %OLDDate% to be replaced by %NEWDate% in 'MyFile.txt' in the position ~11,6
Any reason why powershell couldn't do it?
# Example of PowerShell -replace parameter
clear-Host
$file = Get-ChildItem "D:\powershell\snippets\g*.txt"
foreach ($str in $file)
{
$content = Get-Content -path $str
$content | foreach {$_ -replace "the the", "the"} | Set-Content $str
}
write-Host "After replace `n"
$file
In the foreach loop you can replace any string with another.
Date logic can be used as follows:
$anotherfile = gi anotherfile.txt
Retrieving date info
$year = $anotherfile.LastWriteTime.Year