I get different results.
In Powershell using:
$Matches = Select-String -InputObject (Get-Content "StevenBlackhosts-urls.txt") `
-Pattern "(^|\.)ad[sxvkdz]\-" -AllMatches #`
$Matches.Matches.Count
I get 12 matches and this is incorrect.
In Notepad++, find and count
"(^|\.)ad[sxvkdz]\-"
I have 62 matches and this is correct.
I do not know what's wrong?
the txt "StevenBlackhosts-urls.txt" contains 65106 lines ...
zeus.ad.intl.xiaomi.com
api.ad.intl.xiaomi.com
sdkconfig.ad.intl.xiaomi.com
adv.sec.intl.miui.com
zeus.ad.xiaomi.com
www.api.ad.intl.xiaomi.com
ampmetrics.engadget.com
c.adskeeper.co.uk
events3.adcolony.com
metrics.adage.com
ads.feedly.com
lepodownload.mediatek.com
ads.aerserv.com
ads.mp.mydas.mobi
ads.nexage.com
sdk.adincube.com
dasdada.fu.ck
i1.dl-ad.com
ad.api.kaffnet.com
ad.click.kaffnet.com
api.ad.snappea.com
etc..
testing in this way if I get the same result to Notepaq++; Why does this happen ??
$Matches = Select-String -InputObject (Get-Content "StevenBlackhosts-urls.txt") -Pattern "( |\.)ad[sxvkdz]\-" -AllMatches
$Matches.Matches.Count
It also works well like this, giving me the 62 lines
Get-Content "StevenBlackhosts-urls.txt" | Select-String -Pattern "(^|\.)ad[sxvkdz]\-" -AllMatches | set-content "test.txt"
Get-Content "StevenBlackhosts-urls.txt" | Select-String -Pattern "(^|\.)ad[sxvkdz]\-" -AllMatches | Measure-Object –Line | Select-Object -ExpandProperty Lines
this works right for me ..
thank you all for your suggestions . .
Related
I am new to powershell. I am trying to automate my work a bit and need simple extraction of following pattern from all filetypes:
([0-9A-Z]{2,4}.[0-9A-Z]{8}.[0-9A-Z]{8}.[0-9A-Z]{4})
Example:
*lots of text*
X-xdaemon-transaction-id: string=9971.0A67341C.6147B834.0043,ee=3,shh,rec=0.0,recu=0.0,reid=0.0,cu=3,cld=1
X-xdaemon-transaction-id: string=AA71.0A67341C.6147B442.0043,ee=3,shh,rec=0.0,recu=0.0,reip=0.0,cu=3,cld=1
*lots of text*
Unfortunately, I am receiving output like this:
1mAAAA-0005nG-TN-H:220:
X-xdaemon-transaction-id: string=AA71.0A67341C.6147B442.0043,ee=3,shh,rec=0.0,recu=0.0,reip=0.0,cu=3,cld=1
my 'code' is as following:
Select-String -Path C:\Samples\* -Pattern "(0001.[0-9A-Z]{8}.[0-9A-Z]{8}.[0-9A-Z]{4})" -CaseSensitive
And I'd like to receive only the patterns: AA71.0A67341C.6147B442.0043 without anything added
Thanks for any help!
You can use
$rx = '\b[0-9A-Z]{2,4}\.[0-9A-Z]{8}\.[0-9A-Z]{8}\.[0-9A-Z]{4}\b'
Select-String -AllMatches -Pattern $rx -Path 'C:\Samples\*' -CaseSensitive | % { $_.matches.value }
That is,
Add word boundaries to match your expected strings as whole words and escape the literal . chars
Use -AllMatches (to get multiple matches per line if any) and access each resulting object match value with $_.matches.value.
PS test:
PS C:\Users\admin> $B = Select-String -AllMatches -Pattern '\b[0-9A-Z]{2,4}\.[0-9A-Z]{8}\.[0-9A-Z]{8}\.[0-9A-Z]{4}\b' -Path 'C:\Samples\*' -CaseSensitive | % { $_.matches.value }
PS C:\Users\admin> $B
9971.0A67341C.6147B834.0043
AA71.0A67341C.6147B442.0043
PS C:\Users\admin>
try:
$find = Get-ChildItem *.txt | Select-String -Pattern '\b[0-9A-Z]{2,4}.[0-9A-Z]{8}.[0-9A-Z]{8}.[0-9A-Z]{4}\b' -CaseSensitive
$find.Matches.Value
I try to scrape emailaddresses with Powershell from a directory, with subdirectories and within them .txt files. So i have this code:
$input_path = ‘C:\Users\Me\Documents\toscrape’
$output_file = ‘C:\Users\Me\Documents\toscrape\output.txt’
$regex = ‘\b[A-Za-z0-9._%-]+#[A-Za-z0-9.-]+\.[A-Za-z]{2,4}\b’
select-string -Path $input_path -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Value } > $output_file
But when I execute it, it gives me an error
select-string : The file C:\Users\Me\Documents\toscrape\ can not be read: Could not
path 'C:\Users\Me\Documents\toscrape\'.
At line:1 char:1
+ select-string -Path $input_path -Pattern $regex -AllMatches | % { $_.Matches } | ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Select-String], ArgumentException
+ FullyQualifiedErrorId : ProcessingFile,Microsoft.PowerShell.Commands.SelectStringCommand
I've tried variations to the $input_path, with Get-Item, Get-ChildItem, -Recurse, but nothing seems to work. Can anyone figure out how I need to scrape my location and all its subdirectories and files for the regex pattern?
The error is because Select-String assumes the -Path points to a file or is a wildcard pattern, and $input_path is pointing to a folder. You could use:
$input_path = 'C:\Users\Me\Documents\toscrape\*.txt'
Select-String $input_path ....
However, since you want to recurse through subdirectories, you'll need to use Get-ChildItem to do that.
$input_path = 'C:\Users\Me\Documents\toscrape'
$output_file = 'C:\Users\Me\Documents\toscrape\output.txt'
$regex = '\b[A-Za-z0-9._%-]+#[A-Za-z0-9.-]+\.[A-Za-z]{2,4}\b'
Get-ChildItem $input_path -Include *.txt -Recurse |
Select-String -Pattern $regex -AllMatches |
Select-Object -ExpandProperty Matches |
Select-Object -ExpandProperty Value |
Set-Content $output_file
Note that your regex may cause problems here. You're using \b for word boundary, but period ., hyphen -, and percent sign % are all non-word (\W) characters. The word characters (\w) are [A-Za-z0-9_].
For example:
PS C:\> '%username#example.com' -match '\b[A-Za-z0-9._%-]+#[A-Za-z0-9.-]+\.[A-Za-z]{2,4}\b'
True
PS C:\> $Matches.Values
username#example.com
If that's what you want the pattern to do, that's great, but it is something to be aware of. Regex for an email address is notoriously difficult.
Your correction didn't work but gave me another error, #Bacon Bits. However you put me on the right track. I adapted a bit and this seemed to work out for me.
$input_path = 'C:\Users\Me\Documents\toscrape'
$output_file = 'C:\Users\Me\Documents\toscrape\output.txt'
$regex = '\b[A-Za-z0-9._%-]+#[A-Za-z0-9.-]+\.[A-Za-z]{2,4}\b'
Get-ChildItem $input_path -Recurse | Select-String -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Value } > $output_file
I have a file, input.txt, containing text like this:
GRP123456789
123456789012
GRP234567890
234567890123
GRP456789012
"A lot of text. More text. Blah blah blah: Foobar." (Source Error) (Blah blah blah)
GRP567890123
Source Error
GRP678901234
Source Error
GRP789012345
345678901234
456789012345
I'm attempting to capture all occurrences of "GRP#########" on the condition that at least one number is on the next line.
So GRP123456789 is valid, but GRP456789012 and GRP678901234 are not.
The RegEx pattern I came up with on http://regexstorm.net/tester is: (GRP[0-9]{9})\s\n\s+[0-9]
The PowerShell script I have so far, based off this site http://techtalk.gfi.com/windows-powershell-extracting-strings-using-regular-expressions/, is:
$input_path = 'C:\Users\rtaite\Desktop\input.txt'
$output_file = 'C:\Users\rtaite\Desktop\output.txt'
$regex = '(GRP[0-9]{9})\s\n\s+[0-9]'
select-string -Path $input_path -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Values } > $output_file
I'm not getting any output, and I'm not sure why.
Any help with this would be appreciated as I'm just trying to understand this better.
You need to turn the text input into a single string before passing it to Select-String, otherwise the cmdlet will operate on each line individually and thus never find a match.
Get-Content $input_path | Out-String |
Select-String $regex -AllMatches |
Select-Object -Expand Matches |
ForEach-Object { $_.Groups[1].Value } |
Set-Content $output_file
If you're using PowerShell v3 or newer you can replace Get-Content | Out-String with Get-Content -Raw.
To strip strings from a text file using a pattern, then the best tool for the job is the Select-String. This is also has a parameter called -Context which lets you capture lines before or after the matched line, ideal for just this problem.
So my solution would be something like this:
Select-String 'input.txt' -Pattern '^GRP[0-9]{9}' -Context 0, 1 | ? {
$_.Context.PostContext -match '\d'
} | Select -ExpandProperty line | Set-Content 'output_file.txt'
Using
[regex]::Matches($(Get-Content '.\Desktop\new 1.txt'), "GRP\d+(?=\s+\d)") |
% { $_.value | Out-File .\Desktop\new-1-matches.txt -Append }
I achieved the following output from your sample file:
GRP123456789
GRP234567890
GRP789012345
I am using a PowerShell command to find all *.vue files (it's a simple text format) in a directory, where I need to match this:
7,Id
6,Default
So, these are 2 consecutive lines. With Notepad++ I see CRLF at the end of the line. Following Google searches, this must be close:
Get-ChildItem "D:\Wim\TM1\TI processes" -Filter *.vue -Recurse |
Select-String -Pattern "7,Id\r\n6,Default" -CaseSensitive |
Out-File C:\test.txt
But it does not find the files. I checked that I can find the first part (7,Id) correctly, and also the second part (6,Default), but the combination with the newline is not working.
Any ideas please? Maybe an alternative?
I can have a workaround but it's inefficient and a lot of coding. For example, I could use PowerShell to provide a list of only the first sentence, then process these files to see if it matches the second sentence as well. I want to avoid that.
You need to pass the content of the file as a single string, otherwise Select-String will apply the pattern to each line separately.
Get-ChildItem "D:\Wim\TM1\TI processes" -Filter *.vue -Recurse | ForEach-Object {
Get-Content $_.FullName | Out-String |
Select-String -Pattern "7,Id\r\n6,Default" -CaseSensitive |
Select-Object -Expand Matches |
Select-Object -Expand Groups |
Select-Object -Expand Value
} | Out-File C:\test.txt
On PowerShell v3 and newer you can use Get-Content -Raw instead of Get-Content | Out-String.
As an alternative to Select-String you could use the -cmatch operator in a Where-Object filter:
Get-ChildItem "D:\Wim\TM1\TI processes" -Filter *.vue -Recurse | ForEach-Object {
Get-Content $_.FullName | Out-String | Where-Object {
$_ -cmatch "7,Id\r\n6,Default"
} | ForEach-Object {
$matches[0]
}
} | Out-File C:\test.txt
With Select-String, the -Pattern parameter is regex capable, so try this:
Get-ChildItem "D:\Wim\TM1\TI processes" -Filter *.vue -Recurse |
Select-String -Pattern "7,Id|6,Default" -CaseSensitive |
Out-File C:\test.txt
The vertical pipe bar (|) acts as an alternative separator, or in otherwords, an "or" operator. With the pattern it will match either.
I am working on a code so that it find lines which has $control but should remove lines which start with * at first column
I am working with following but doesn't seem to work ..
$result = Get-Content $file.fullName | Select-String $control | Select-String -pattern "\^*" -notmatch
Thanks in advance
You're escaping the wrong character. You do not want to escape ^ as that's your anchor for "starting with". You'll want to escape the asterix, so try this:
$result = Get-Content $file.fullName | Select-String $control | select-string -pattern "^\*" -notmatch
Also, if all you want is the lines, you could also use this:
Get-Content $file.fullName | ? { $_ -match $control -and $_ -notmatch '^\*'}