The goal of this script is supposed to be to find any base64 encoded strings within a particular row of a CSV, grab only the base64, add the correct amount of padding charcters to it, decode it, and write it to an output.
function Get-FileName($initialDirectory) {
[System.Reflection.Assembly]::LoadWithPartialName("System.Sindows.Forms") | Out-Null
$OpenFileDialog = New-Object System.Windows.Forms.OpenFileDialog
$OpenFileDialog.InitialDirectory = $initialDirectory
$OpenFileDialog.Filter = "CSV (*.csv)| *.csv"
$OpenFileDialog.ShowDialog() | Out-Null
$OpenFileDialog.Filename
}
$inputfile = Get-FileName "C:\My Documents\"
$inputdata = Import-Csv $inputfile
$output = "C:\My Documents\output.csv"
$regex = [regex]::New('(?<=-[Ee]ncoded[Cc]ommand\s"??)[\w\d=\/!]{0,792}')
$csv1 = $inputdata | select -ExpandProperty 'NameOfRow' | ? {$_ -ne ""}
| Select-String -Pattern $regex -Allmatches | % {$_.Matches}
| ? {$_.Value -ne ""} | % {$_.Value}
foreach ($line in $csv1) {
$csvL = $line.Length
$csvM = $csvL %= 4
if ($csvM | % {$_ -eq 1}) {
$line | % {$_ + "==="}
} elseif ($csvM | % {$_ -eq 2}) {
$line | % {$_ + "=="}
} elseif ($csvM | % {$_ -eq 3}) {
$line | % {$_ + "="}
}
$decode = [System.Text.Encoding]::ASCII.GetString([System.Convert]::FromBase64String($line))
$decode > $output
}
Related
Two problems of regular replace
1.need to keep the front $character in the replacement result
2.Skipping the first two lines and the last line is not valid
Code:
$str = #'
#$start1 Random characters
#$start2 Random characters
$p1.AppendBreak($BreakType.LineBreak)
$doc.Protect($ProtectionType.AllowOnlyRevisions, "123")
$footerPara.AppendField("page", $FieldType.FieldPage)
$footerParagraph.AppendField("number of pages", $FieldType.FieldSectionPages)
$txtWatermark.Layout = $WatermarkLayout.Diagonal
$tr1.CharacterFormat.Border.BorderType = $BorderStyle.DashDotStroker
$stri.CharacterFormat.TextBackgroundColor = $Color.LightGray
$document.LoadFromFile(".\Template_HtmlFile.html", $FileFormat.Html, $XHTMLValidationType.None)
$docObject.DocumentObjectType -eq $DocumentObjectType.Picture
$document.Sections[0].Paragraphs[0].InsertSectionBreak($SectionBreakType.NoBreak)
$footerParagraph.Format.HorizontalAlignment = $Spire.Doc.Documents.HorizontalAlignment.Right
#end Random characters
'#
$str | Foreach-Object {
$_ -replace '\$\w+\.(\w+)', '"$1"'
} | Set-Content .\ok.txt
<# -Skip -SkipLast not valid
$str | Foreach-Object {
$_ -replace '\$\w+\.(\w+)', '"$1"'
} | Select-Object -Skip 2 | Select-Object -SkipLast 1 | Set-Content .\ok.txt
#>
Expected results:
At least for your example here string, you need to break it into a string array. Then for the replacement I was only successful when capturing both the beginning and the desired changed text.
$str -split '\r?\n' | Select-Object -Skip 2 |
Select-Object -SkipLast 1 | Foreach-Object {
$_ -replace '(^.+?)\$.+\.(\w+)', '$1"$2"'
} | Set-Content .\ok.txt
Contents of ok.txt
$p1.AppendBreak("LineBreak")
$doc.Protect("AllowOnlyRevisions", "123")
$footerPara.AppendField("page", "FieldPage")
$footerParagraph.AppendField("number of pages", "FieldSectionPages")
$txtWatermark.Layout = "Diagonal"
$tr1.CharacterFormat.Border.BorderType = "DashDotStroker"
$stri.CharacterFormat.TextBackgroundColor = "LightGray"
$document.LoadFromFile(".\Template_HtmlFile.html", "None")
$docObject.DocumentObjectType -eq "Picture"
$document.Sections[0].Paragraphs[0].InsertSectionBreak("NoBreak")
$footerParagraph.Format.HorizontalAlignment = "Right"
I need to get the total data transferred value from multiple log files. The data value is shown under the total column in the bytes row.
It is also respectively delimited with a g, m, t or k to show gb/mb/tb/kb.
Total Copied Skipped Mismatch FAILED Extras
Bytes : 54.414 g 54.414 g 0 0 0 0
Currently I have this script which can go through all the files and extract the value between Bytes: and the g but I need to be able to add more queries and to the `foreach file and sum them all to one consistent value.
This is what I have currently but only outputs the kb.
$pattern = "(?<=.*Bytes :.*)\w.+?(?= g.*)"
$pattern1 = "(?<=.*Bytes :.*)\w.+?(?= m.*)"
$pattern2 = "(?<=.*Bytes :.*)\w.+?(?= k.*)"
Get-ChildItem "C:\Users\logs" -Filter "BFR*" | ForEach-Object {
Get-Content "C:\Users\logs\*.log" | where-Object {$_ -match $pattern } | ForEach-Object {
[double] ($matches[0])
} | Measure-Object -Sum | Select-Object -ExpandProperty sum
} | ForEach-Object {
Get-Content "C:\Users\logs\*.log" | where-Object {$_ -match $pattern1 } | ForEach-Object {
[double] ($matches[0])
} | Measure-Object -Sum | Select-Object -ExpandProperty sum
} | ForEach-Object {
Get-Content "C:\Users\logs\*.log" | where-Object {$_ -match $pattern2 } | ForEach-Object {
[double] ($matches[0])
} | Measure-Object -Sum | Select-Object -ExpandProperty sum
}
Here is one possibility. (Sorry #JamesC, it's a 1-liner :-)):
Get-ChildItem .\LogFolder\*.log |
ForEach-Object {$totalBytes = 0}{
Get-Content $_ | Select-String -Pattern "^Bytes\s+:\s+(?<size>\d+\.\d+) (?<units>[tgmk]).*$" |
Foreach-Object {
$size = $_.Matches.Groups[1].Value
switch ($_.Matches.Groups[2].Value)
{
t {$totalbytes += (1tb * $size)}
g {$totalbytes += (1gb * $size)}
m {$totalbytes += (1mb * $size)}
k {$totalbytes += (1kb * $size)}
}
}
} {"Total Bytes: $totalBytes"}
I need to put a bunch of users through a regex query that look for 'IM' or '(IM)' at the end of their DisplayName and then removes it. I'll be exporting first of course for backup.
I'm have used the .contains method but if I use that any Tim's or Imogen's might get a little annoyed at me.
$users = Get-DistributionGroupMember 'DGName' | ? { $_.RecipientType -eq 'UserMailbox' }
$mailboxes = $users | Get-Mailbox
$output = #()
$mailboxes | Select-Object Alias, DisplayName | Export-Csv D:\Scripts\PreDisplayNames.csv -NoTypeInformation
foreach ($mb in $mailboxes){
if ($mb.DisplayName.EndsWith(" (IM)")){
$newString = $mb.DisplayName.TrimEnd(" (IM)")
$newStringTD = $mb.DisplayName.TrimEnd(" (IM)") + " (TD)"
$resultstring = "Setting DisplayName to $newString"
$mb | Set-Mailbox -DisplayName $newString -WhatIf
}
elseif($mb.DisplayName.EndsWith(" IM")){
$newString = $mb.DisplayName.TrimEnd(" IM")
$newStringTD = $mb.DisplayName.TrimEnd(" (IM)") + " (TD)"
$resultstring = "Setting DisplayName to $newString"
$mb | Set-Mailbox -DisplayName $newString -WhatIf
}
else{
$newString = 'DNIncorrect'
$resultstring = 'Incorrect DisplayName'
}
$props = [ordered]#{
'DisplayName' = $mb.DisplayName
'NewDisplayName' = $newString
'NewDisplayName TD' = $newStringTD
}
$object = New-Object -TypeName PSObject -Property $props
$output += $object
}
$output | Export-Csv "D:\scripts\DisplayNames.csv" -NoTypeInformation
Theres no need for a regex here, just use a -like to filter your objects when you collect the mailboxes
$mailboxes = $users | Get-Mailbox | Where-object {($_.DisplayName -like "*IM") -or ($_.DisplayName -like "*(IM)")}
If you still wanted a regex I normally use the following to generate a regex for specific words. It may work for you situation.
$Eliminate = #("bak", "error", "research","Retry")
[regex]$eliminate_regex = '(?i)(' + (($Eliminate | foreach { [regex]::escape($_) }) -join "|") + ')'
I'm new with powershell and in need of guidance. Been scouring the site for answers and coming up blank, decided to ask instead. If this has been answered please refer me to the link.
I have an application log (xml format) like below:
<log><identifier>123axr4x5</identifier><login>USER1</login><source>Order-Management</source><AddlInfo>Execution Time : 20ms</AddlInfo><Exception></Exception><timestamp>01/01/2015:22:00:00</timestamp><serverticks>643670855</serverticks><PID>1234</PID><Machine>PRD01X12mm</Machine></log>
<log><identifier>dd8jksl3g</identifier><login>USER2</login><source>Service-Assurance</source><AddlInfo>Execution Time : 80ms</AddlInfo><Exception></Exception><timestamp>01/01/2015:22:00:00</timestamp><serverticks>643680865</serverticks><PID>1234</PID><Machine>PRD01X12mm</Machine></log>
: and so on
I am creating a log parser that will scan a folder and its subfolder for matching regex pattern, and based on certain threshold, output into gridview/export to CSV. I am almost done, however i'm unable to solve 1 problem, which is to get the filename currently being parsed, to be displayed on the gridview.
Basically i am using piped Get-ChildItem as below
Get-ChildItem $Dir -recurse -Filter *logging*.txt|
Sort-Object LastWriteTime |
?{$_.LastWriteTime -gt (Get-Date).AddMinutes(-60)}|
Select-String -Pattern $Text |
Select-String -Pattern $Text3 |
Select-String -Pattern $Text2 -allmatches |
Foreach-Object {
$information = $_|Select-Object -Property API, Duration,DataRetrieved, ServerTime, ServerTicks , Identifier, Filename
$information.Filename = $_.Name
#$information.Filename = $_.FullName
} |
Out-GridView
Below is the full code:
$Dir = "C:\log\"
$threshold = 1 + 0
$StartTime = (Get-Date).ToString();
$EndTime = (Get-Date).ToString();
$Text = "abc"
$Text2 = "def"
$Text3 = "ghi"
$OutFile = "result"
$OutPath = $Dir + $OutFile + ".txt"
#ExtractionParameters
$AddlInnfoTagBegin = "AddlInfo"
$AddlInnfoTagEnd = "/AddlInfo"
$ServerTimeOfLogTagBegin = "ServerTimeOfLog"
$ServerTimeOfLogTagEnd = "/ServerTimeOfLog"
$ServerTicksTagBegin = "ServerTicks"
$ServerTicksTagEnd = "/ServerTicks"
$IdentifierTagBegin = "Identifier"
$IdentifierTagEnd = "/Identifier"
#parse file in folders
Get-ChildItem $Dir -recurse -Filter *logging*.txt|
Sort-Object LastWriteTime |
#?{$_.LastWriteTime -gt (Get-Date).AddMinutes(-60)}|
Select-String -Pattern $Text |
Select-String -Pattern $Text3 |
Select-String -Pattern $Text2 -allmatches |
Foreach-Object {
# take line and split it at tabulators
$parts = $_.Line
#write $parts
$indexOfAddlInfoBegin = $parts.IndexOf($AddlInnfoTagBegin) + $AddlInnfoTagBegin.Length +1
$indexOfAddlInfoEnd = $parts.IndexOf($AddlInnfoTagEnd) -1
$AddlInfoData = $parts.Substring($indexOfAddlInfoBegin, $indexOfAddlInfoEnd - $indexOfAddlInfoBegin)
$AddlInfoReplaced = $AddlInfoData.Replace(" seconds ","#")
$AddlInfoSplit = $AddlInfoReplaced.Split('#')
$information = $_|Select-Object -Property API, Duration,DataRetrieved, ServerTime, ServerTicks , Identifier, Filename
#get filename, which does not work
$information.Filename = $_.Name
#$information.Filename = $_.FullName
$information.API = $AddlInfoSplit[0].Split(':')[0]
$information.DataRetrieved = $AddlInfoSplit[1]
$information.Duration = $AddlInfoSplit[0].Split(':')[1]
$information.Duration = $information.Duration.Replace("Execution Time = ","")
$indexOfServerTimeBegin = $parts.IndexOf($ServerTimeOfLogTagBegin) + $ServerTimeOfLogTagBegin.Length +1
$indexOfServerTimeEnd = $parts.IndexOf($ServerTimeOfLogTagEnd) -1
$ServerTimeData = $parts.Substring($indexOfServerTimeBegin, $indexOfServerTimeEnd - $indexOfServerTimeBegin)
$information.ServerTime = $ServerTimeData
$indexOfServerTicksBegin = $parts.IndexOf($ServerTicksTagBegin) + $ServerTicksTagBegin.Length +1
$indexOfServerTicksEnd = $parts.IndexOf($ServerTicksTagEnd) -1
$ServerTickData = $parts.Substring($indexOfServerTicksBegin, $indexOfServerTicksEnd - $indexOfServerTicksBegin)
$information.ServerTicks = $ServerTickData
$indexOfIdentifierBegin = $parts.IndexOf($IdentifierTagBegin) + $IdentifierTagBegin.Length +1
$indexOfIdentifierEnd = $parts.IndexOf($IdentifierTagEnd) -1
$IdentifierData = $parts.Substring($indexOfIdentifierBegin, $indexOfIdentifierEnd - $indexOfIdentifierBegin)
$information.Identifier = $IdentifierData
$DurationAsInt = 0 + $information.Duration
if($DurationAsInt -gt $threshold) {
write $information
}
} |
Out-GridView
#Out-File -FilePath $OutPath -Append -Width 200
Any help is appreciated, thanks!!
-CL
The property you are looking for is "FileName".
$information.Filename = $_.FileName
Powershell provides a cmdlet "Get-Member" which would list all available properties/methods. You could enumerate the members to console and inspect what is available
Write-Host ( $_ | Get-Member)
I am just learning powershell and cant find how to run differents regex with powershell.
$input_path = 'C:\site-download\input.txt'
$output_file = 'C:\site-download\output.txt'
$regex = '(?<month>VPIP<\/span><span class=""right"">\d{2}.\d{1})'
$regex2 = '(?<month>VPIP<\/span><span class=""right"">\d{2}.\d{1})'
$regex3 = '(?<month>VPIP<\/span><span class=""right"">\d{2}.\d{1})'
$regex... = '(?<month>VPIP<\/span><span class=""right"">\d{2}.\d{1})'
select-string -Path $input_path -Pattern $regex -AllMatches | % { $_.Matches} | % { $_.Value }|
Foreach-Object {$_ -replace '</span><span class=""right"">', ' = '} > $output_file
$regex
works good, but how can i add $regex2 and $regex3 ... to outputfile?
Thanks
You just need a small change to your last section of your pipeline. Instead of using > $output_file just pipe the output of the foreach loop to Out-File cmdlet. So you should be able to have your last line of code look like this:
select-string -Path $input_path -Pattern $regex -AllMatches |
% { $_.Matches} | % { $_.Value } |
Foreach-Object {$_ -replace '</span><span class=""right"">', ' = '} |
Out-File $output_file