I have a file, input.txt, containing text like this:
GRP123456789
123456789012
GRP234567890
234567890123
GRP456789012
"A lot of text. More text. Blah blah blah: Foobar." (Source Error) (Blah blah blah)
GRP567890123
Source Error
GRP678901234
Source Error
GRP789012345
345678901234
456789012345
I'm attempting to capture all occurrences of "GRP#########" on the condition that at least one number is on the next line.
So GRP123456789 is valid, but GRP456789012 and GRP678901234 are not.
The RegEx pattern I came up with on http://regexstorm.net/tester is: (GRP[0-9]{9})\s\n\s+[0-9]
The PowerShell script I have so far, based off this site http://techtalk.gfi.com/windows-powershell-extracting-strings-using-regular-expressions/, is:
$input_path = 'C:\Users\rtaite\Desktop\input.txt'
$output_file = 'C:\Users\rtaite\Desktop\output.txt'
$regex = '(GRP[0-9]{9})\s\n\s+[0-9]'
select-string -Path $input_path -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Values } > $output_file
I'm not getting any output, and I'm not sure why.
Any help with this would be appreciated as I'm just trying to understand this better.
You need to turn the text input into a single string before passing it to Select-String, otherwise the cmdlet will operate on each line individually and thus never find a match.
Get-Content $input_path | Out-String |
Select-String $regex -AllMatches |
Select-Object -Expand Matches |
ForEach-Object { $_.Groups[1].Value } |
Set-Content $output_file
If you're using PowerShell v3 or newer you can replace Get-Content | Out-String with Get-Content -Raw.
To strip strings from a text file using a pattern, then the best tool for the job is the Select-String. This is also has a parameter called -Context which lets you capture lines before or after the matched line, ideal for just this problem.
So my solution would be something like this:
Select-String 'input.txt' -Pattern '^GRP[0-9]{9}' -Context 0, 1 | ? {
$_.Context.PostContext -match '\d'
} | Select -ExpandProperty line | Set-Content 'output_file.txt'
Using
[regex]::Matches($(Get-Content '.\Desktop\new 1.txt'), "GRP\d+(?=\s+\d)") |
% { $_.value | Out-File .\Desktop\new-1-matches.txt -Append }
I achieved the following output from your sample file:
GRP123456789
GRP234567890
GRP789012345
Related
I am new to powershell. I am trying to automate my work a bit and need simple extraction of following pattern from all filetypes:
([0-9A-Z]{2,4}.[0-9A-Z]{8}.[0-9A-Z]{8}.[0-9A-Z]{4})
Example:
*lots of text*
X-xdaemon-transaction-id: string=9971.0A67341C.6147B834.0043,ee=3,shh,rec=0.0,recu=0.0,reid=0.0,cu=3,cld=1
X-xdaemon-transaction-id: string=AA71.0A67341C.6147B442.0043,ee=3,shh,rec=0.0,recu=0.0,reip=0.0,cu=3,cld=1
*lots of text*
Unfortunately, I am receiving output like this:
1mAAAA-0005nG-TN-H:220:
X-xdaemon-transaction-id: string=AA71.0A67341C.6147B442.0043,ee=3,shh,rec=0.0,recu=0.0,reip=0.0,cu=3,cld=1
my 'code' is as following:
Select-String -Path C:\Samples\* -Pattern "(0001.[0-9A-Z]{8}.[0-9A-Z]{8}.[0-9A-Z]{4})" -CaseSensitive
And I'd like to receive only the patterns: AA71.0A67341C.6147B442.0043 without anything added
Thanks for any help!
You can use
$rx = '\b[0-9A-Z]{2,4}\.[0-9A-Z]{8}\.[0-9A-Z]{8}\.[0-9A-Z]{4}\b'
Select-String -AllMatches -Pattern $rx -Path 'C:\Samples\*' -CaseSensitive | % { $_.matches.value }
That is,
Add word boundaries to match your expected strings as whole words and escape the literal . chars
Use -AllMatches (to get multiple matches per line if any) and access each resulting object match value with $_.matches.value.
PS test:
PS C:\Users\admin> $B = Select-String -AllMatches -Pattern '\b[0-9A-Z]{2,4}\.[0-9A-Z]{8}\.[0-9A-Z]{8}\.[0-9A-Z]{4}\b' -Path 'C:\Samples\*' -CaseSensitive | % { $_.matches.value }
PS C:\Users\admin> $B
9971.0A67341C.6147B834.0043
AA71.0A67341C.6147B442.0043
PS C:\Users\admin>
try:
$find = Get-ChildItem *.txt | Select-String -Pattern '\b[0-9A-Z]{2,4}.[0-9A-Z]{8}.[0-9A-Z]{8}.[0-9A-Z]{4}\b' -CaseSensitive
$find.Matches.Value
I get different results.
In Powershell using:
$Matches = Select-String -InputObject (Get-Content "StevenBlackhosts-urls.txt") `
-Pattern "(^|\.)ad[sxvkdz]\-" -AllMatches #`
$Matches.Matches.Count
I get 12 matches and this is incorrect.
In Notepad++, find and count
"(^|\.)ad[sxvkdz]\-"
I have 62 matches and this is correct.
I do not know what's wrong?
the txt "StevenBlackhosts-urls.txt" contains 65106 lines ...
zeus.ad.intl.xiaomi.com
api.ad.intl.xiaomi.com
sdkconfig.ad.intl.xiaomi.com
adv.sec.intl.miui.com
zeus.ad.xiaomi.com
www.api.ad.intl.xiaomi.com
ampmetrics.engadget.com
c.adskeeper.co.uk
events3.adcolony.com
metrics.adage.com
ads.feedly.com
lepodownload.mediatek.com
ads.aerserv.com
ads.mp.mydas.mobi
ads.nexage.com
sdk.adincube.com
dasdada.fu.ck
i1.dl-ad.com
ad.api.kaffnet.com
ad.click.kaffnet.com
api.ad.snappea.com
etc..
testing in this way if I get the same result to Notepaq++; Why does this happen ??
$Matches = Select-String -InputObject (Get-Content "StevenBlackhosts-urls.txt") -Pattern "( |\.)ad[sxvkdz]\-" -AllMatches
$Matches.Matches.Count
It also works well like this, giving me the 62 lines
Get-Content "StevenBlackhosts-urls.txt" | Select-String -Pattern "(^|\.)ad[sxvkdz]\-" -AllMatches | set-content "test.txt"
Get-Content "StevenBlackhosts-urls.txt" | Select-String -Pattern "(^|\.)ad[sxvkdz]\-" -AllMatches | Measure-Object –Line | Select-Object -ExpandProperty Lines
this works right for me ..
thank you all for your suggestions . .
I am using a PowerShell command to find all *.vue files (it's a simple text format) in a directory, where I need to match this:
7,Id
6,Default
So, these are 2 consecutive lines. With Notepad++ I see CRLF at the end of the line. Following Google searches, this must be close:
Get-ChildItem "D:\Wim\TM1\TI processes" -Filter *.vue -Recurse |
Select-String -Pattern "7,Id\r\n6,Default" -CaseSensitive |
Out-File C:\test.txt
But it does not find the files. I checked that I can find the first part (7,Id) correctly, and also the second part (6,Default), but the combination with the newline is not working.
Any ideas please? Maybe an alternative?
I can have a workaround but it's inefficient and a lot of coding. For example, I could use PowerShell to provide a list of only the first sentence, then process these files to see if it matches the second sentence as well. I want to avoid that.
You need to pass the content of the file as a single string, otherwise Select-String will apply the pattern to each line separately.
Get-ChildItem "D:\Wim\TM1\TI processes" -Filter *.vue -Recurse | ForEach-Object {
Get-Content $_.FullName | Out-String |
Select-String -Pattern "7,Id\r\n6,Default" -CaseSensitive |
Select-Object -Expand Matches |
Select-Object -Expand Groups |
Select-Object -Expand Value
} | Out-File C:\test.txt
On PowerShell v3 and newer you can use Get-Content -Raw instead of Get-Content | Out-String.
As an alternative to Select-String you could use the -cmatch operator in a Where-Object filter:
Get-ChildItem "D:\Wim\TM1\TI processes" -Filter *.vue -Recurse | ForEach-Object {
Get-Content $_.FullName | Out-String | Where-Object {
$_ -cmatch "7,Id\r\n6,Default"
} | ForEach-Object {
$matches[0]
}
} | Out-File C:\test.txt
With Select-String, the -Pattern parameter is regex capable, so try this:
Get-ChildItem "D:\Wim\TM1\TI processes" -Filter *.vue -Recurse |
Select-String -Pattern "7,Id|6,Default" -CaseSensitive |
Out-File C:\test.txt
The vertical pipe bar (|) acts as an alternative separator, or in otherwords, an "or" operator. With the pattern it will match either.
I'm trying to extract the titles of events in one of my logs, which is just a text file with lots of data. The filename is eventlog-1-5-2016.txt (date is always the current date). Each line in the file is one event like this:
1-1-16(Commodore Rally)Address|Time
1-2-16(Open House)Address|Time
I just want to go through the file and extract the title in parentheses, excluding the parentheses themselves, and output the list to the console, or a text file.
I've also tried output to a txt file but I'm missing something. Can you tell me why this doesn't work:
Console:
Select-String -Path c:\log\eventlog-1-5-2016.txt -Pattern '\(([^\)]+)\)' -AllMatches |
% { $_.Matches }
To File:
Select-string -Path c:\log\eventlog-1-5-2016.txt -Pattern '\(([^\)]+)\)' -AllMatches |
% { $_.Matches } | { $_.Value > C:\log\results.txt
or even a better way to do this if this wrong.
Bonus question, could the path auto calculate the current date and correct the file name for easy future pasting? (not major!)
The current date can be determined like this:
(Get-Date).ToString('d-M-yyyy')
Also, your regular expression could be simplified a little by using a non-greedy match:
`\((.+?)\)
If you want just the text between the parentheses you need the value of the captured group instead of the complete match:
$date = (Get-Date).ToString('d-M-yyyy')
Select-String -Path "C:\log\eventlog-$date.txt" -Pattern '\((.+?)\)' -AllMatches |
ForEach-Object { $_.Matches } |
ForEach-Object { $_.Groups[1] } |
ForEach-Object { $_.Value } |
Out-File 'C:\log\results.txt'
If you have PowerShell v3 or newer you could collapse the ForEach-Object statements:
Select-String -Path "C:\log\eventlog-$date.txt" -Pattern '\((.+?)\)' -AllMatches |
ForEach-Object { $_.Matches.Groups[1].Value } |
Out-File 'C:\log\results.txt'
Or you could use the -match operator:
Get-Content "C:\log\eventlog-$date.txt" |
Where-Object { $_ -match '\((.+?)\)' } |
ForEach-Object { $matches[1] } |
Set-Content 'C:\log\results.txt'
What happens is that Select-String doesn't quite do what you think it does. It will match patterns, but instead of returning the matched part, it will return you the whole matching string. Thus, the statement returns you the whole matched row instead of just substring in parenthesis. This is common a cause for confusion.
As a simple example in case of link rot,
[regex]$rx = '\(([^\)]+)\)'
cat C:\Temp\logfile.txt | % { $rx.Matches( $_ ).value }
(Commodore Rally)
(Open House)
I am learning regex and am trying to get a better understanding by using a text file with the value $100,000 in it. What I am trying to do is to search the text file for the string "$100,000" and if it is there export the value out into a new CSV. this is what I'm using so far.
[io.file]::readalltext("c:\utilities\notes_$datetime.txt") -match("[$][0-9][0-9][0-9],[0-9][0-9][0-9]") | Out-File C:\utilities\amount.txt -Encoding ascii -Force
Which returns true. Can someone point me in the right direction as to grabbing the string value that it finds into a new CSV?
many thanks!
You're reading the file into a single string, not an array of lines, so you should use the Select-String -AllMatches instead of the -match operator:
[IO.File]::ReadAllText("c:\utilities\notes_$datetime.txt") |
Select-String '\$\d{3},\d{3}' -AllMatches |
% { $_.Matches.Groups.Value } |
Out-File C:\utilities\amount.txt -Encoding ascii -Force
As a side note, using Get-Content -Raw would be slightly more PoSh than using .Net methods, although .Net methods provide better performance.
Get-Content "c:\utilities\notes_$datetime.txt" -Raw |
Select-String '\$\d{3},\d{3}' -AllMatches |
% { $_.Matches.Groups.Value } |
Out-File C:\utilities\amount.txt -Encoding ascii -Force
I prefer to use [regex]::match for that:
$x = 'text bla $100,000 text text'
[regex]::Match($x,"\$[\d]{3},[\d]{3}").Groups[0].Value
I also changed the expression a little bit ($ followed by 3 numbers, followed by a "," and another 3 numbers).
So your script could look like this:
$fileContent = Get-Content "c:\utilities\notes_$datetime.txt"
[regex]::Match($fileContent,"\$[\d]{3},[\d]{3}").Groups[0].Value | Out-File C:\utilities\amount.txt -Encoding ascii -Force
Why not use the Select-String cmdlet - far easier:
Select-String .\infile.csv -pattern '\$[\d]{3},[\d]{3}' | Select Line | Out-File outfile.txt
You can then process multiple files like so:
Get-Childitem *.csv | Select-String -pattern '\$[\d]{3},[\d]{3}' | Select Line | Out-File outfile.txt
The Select-String has the following properties:
Line - the line where the regex found a match
LineNumber - the line number in the file where the match was found
Filename - the name of the file the match was found in