I'm new with powershell and in need of guidance. Been scouring the site for answers and coming up blank, decided to ask instead. If this has been answered please refer me to the link.
I have an application log (xml format) like below:
<log><identifier>123axr4x5</identifier><login>USER1</login><source>Order-Management</source><AddlInfo>Execution Time : 20ms</AddlInfo><Exception></Exception><timestamp>01/01/2015:22:00:00</timestamp><serverticks>643670855</serverticks><PID>1234</PID><Machine>PRD01X12mm</Machine></log>
<log><identifier>dd8jksl3g</identifier><login>USER2</login><source>Service-Assurance</source><AddlInfo>Execution Time : 80ms</AddlInfo><Exception></Exception><timestamp>01/01/2015:22:00:00</timestamp><serverticks>643680865</serverticks><PID>1234</PID><Machine>PRD01X12mm</Machine></log>
: and so on
I am creating a log parser that will scan a folder and its subfolder for matching regex pattern, and based on certain threshold, output into gridview/export to CSV. I am almost done, however i'm unable to solve 1 problem, which is to get the filename currently being parsed, to be displayed on the gridview.
Basically i am using piped Get-ChildItem as below
Get-ChildItem $Dir -recurse -Filter *logging*.txt|
Sort-Object LastWriteTime |
?{$_.LastWriteTime -gt (Get-Date).AddMinutes(-60)}|
Select-String -Pattern $Text |
Select-String -Pattern $Text3 |
Select-String -Pattern $Text2 -allmatches |
Foreach-Object {
$information = $_|Select-Object -Property API, Duration,DataRetrieved, ServerTime, ServerTicks , Identifier, Filename
$information.Filename = $_.Name
#$information.Filename = $_.FullName
} |
Out-GridView
Below is the full code:
$Dir = "C:\log\"
$threshold = 1 + 0
$StartTime = (Get-Date).ToString();
$EndTime = (Get-Date).ToString();
$Text = "abc"
$Text2 = "def"
$Text3 = "ghi"
$OutFile = "result"
$OutPath = $Dir + $OutFile + ".txt"
#ExtractionParameters
$AddlInnfoTagBegin = "AddlInfo"
$AddlInnfoTagEnd = "/AddlInfo"
$ServerTimeOfLogTagBegin = "ServerTimeOfLog"
$ServerTimeOfLogTagEnd = "/ServerTimeOfLog"
$ServerTicksTagBegin = "ServerTicks"
$ServerTicksTagEnd = "/ServerTicks"
$IdentifierTagBegin = "Identifier"
$IdentifierTagEnd = "/Identifier"
#parse file in folders
Get-ChildItem $Dir -recurse -Filter *logging*.txt|
Sort-Object LastWriteTime |
#?{$_.LastWriteTime -gt (Get-Date).AddMinutes(-60)}|
Select-String -Pattern $Text |
Select-String -Pattern $Text3 |
Select-String -Pattern $Text2 -allmatches |
Foreach-Object {
# take line and split it at tabulators
$parts = $_.Line
#write $parts
$indexOfAddlInfoBegin = $parts.IndexOf($AddlInnfoTagBegin) + $AddlInnfoTagBegin.Length +1
$indexOfAddlInfoEnd = $parts.IndexOf($AddlInnfoTagEnd) -1
$AddlInfoData = $parts.Substring($indexOfAddlInfoBegin, $indexOfAddlInfoEnd - $indexOfAddlInfoBegin)
$AddlInfoReplaced = $AddlInfoData.Replace(" seconds ","#")
$AddlInfoSplit = $AddlInfoReplaced.Split('#')
$information = $_|Select-Object -Property API, Duration,DataRetrieved, ServerTime, ServerTicks , Identifier, Filename
#get filename, which does not work
$information.Filename = $_.Name
#$information.Filename = $_.FullName
$information.API = $AddlInfoSplit[0].Split(':')[0]
$information.DataRetrieved = $AddlInfoSplit[1]
$information.Duration = $AddlInfoSplit[0].Split(':')[1]
$information.Duration = $information.Duration.Replace("Execution Time = ","")
$indexOfServerTimeBegin = $parts.IndexOf($ServerTimeOfLogTagBegin) + $ServerTimeOfLogTagBegin.Length +1
$indexOfServerTimeEnd = $parts.IndexOf($ServerTimeOfLogTagEnd) -1
$ServerTimeData = $parts.Substring($indexOfServerTimeBegin, $indexOfServerTimeEnd - $indexOfServerTimeBegin)
$information.ServerTime = $ServerTimeData
$indexOfServerTicksBegin = $parts.IndexOf($ServerTicksTagBegin) + $ServerTicksTagBegin.Length +1
$indexOfServerTicksEnd = $parts.IndexOf($ServerTicksTagEnd) -1
$ServerTickData = $parts.Substring($indexOfServerTicksBegin, $indexOfServerTicksEnd - $indexOfServerTicksBegin)
$information.ServerTicks = $ServerTickData
$indexOfIdentifierBegin = $parts.IndexOf($IdentifierTagBegin) + $IdentifierTagBegin.Length +1
$indexOfIdentifierEnd = $parts.IndexOf($IdentifierTagEnd) -1
$IdentifierData = $parts.Substring($indexOfIdentifierBegin, $indexOfIdentifierEnd - $indexOfIdentifierBegin)
$information.Identifier = $IdentifierData
$DurationAsInt = 0 + $information.Duration
if($DurationAsInt -gt $threshold) {
write $information
}
} |
Out-GridView
#Out-File -FilePath $OutPath -Append -Width 200
Any help is appreciated, thanks!!
-CL
The property you are looking for is "FileName".
$information.Filename = $_.FileName
Powershell provides a cmdlet "Get-Member" which would list all available properties/methods. You could enumerate the members to console and inspect what is available
Write-Host ( $_ | Get-Member)
Related
I try to remove " - 13234" from a bunch of files recursively. And if the file is already present after removing that part it should add (1), (2) and so on after the new file name.
I tested it on a small set of files and that was working fine. Now I tried it on a larger test set and it gave me some errors.
Like: Cannot rename file does not exist
Like: Cannot create a file when file already exist
For a lot of files it looks like it is doing fine actually.
$files = Get-ChildItem <location> -recurse -file | sort name | group-object -property {$_.fullname -replace "(.*)-.*?(\..*?)$",'$1$2'}
$files | foreach {
$inc = 0
if ($_.count -gt 1) {
rename-item -literalpath $_.group[0].fullname -NewName (($_.group[0].basename -replace "(.*)-.*?$",'$1') + $_.group[0].extension)
$inc++
for ($i = $inc; $i -lt $_.count; $i++) {
rename-item -literalpath $_.group[$i].fullname -NewName (($_.group[$i].basename -replace "(.*)-.*?$",'$1') + "($i)" + $_.group[$i].extension)
}
}
else {
rename-item -literalpath $_.group.fullname -NewName (($_.group.basename -replace "(.*)-.*?$",'$1') + $_.group.extension)
}
}
If the following files are in the same directory, the script probably will not work.
1 - 13234.txt
1.txt
The script first tries to change "1 - 13234.txt" to "1. txt".
You had better first check the existence of the file.
Get-ChildItem -File -Recurse |
where BaseName -match "(.*)-" |
Group-Object { $_.DirectoryName + "\" + $Matches[1].TrimEnd() + $_.Extension } |
foreach {
$fi = [IO.FileInfo]::new($_.Name)
$i = 0
$_.Group | Rename-Item -NewName {
do {
$newName = $fi.DirectoryName + "\" + $fi.BaseName + $(if($i){"($i)"}) + $fi.Extension
$script:i++
} while (Test-Path $newName)
$newName
} -PassThru
}
I would do something like this:
$Path = '<PATH TO THE ROOTFOLDER WHERE THE FILES TO RENAME ARE FOUND>'
Get-ChildItem -Path $Path -Filter '*-*' -File -Recurse | ForEach-Object {
# obtain the new file basename
$newBaseName = ($_.BaseName -split '-')[0].Trim()
$extension = $_.Extension # this includes the dot
$folder = $_.DirectoryName
# get an array of all filenames (name only) of the files with a similar name already present in the folder
$allFiles = #(Get-ChildItem $folder -Filter "$newBaseName*$extension" -File | Select-Object -ExpandProperty Name)
# for PowerShell version < 3.0 use this
# $allFiles = #(Get-ChildItem $folder-Filter "$newBaseName*$extension" | Where-Object { !($_.PSIsContainer) } | Select-Object -ExpandProperty Name)
# construct the new filename
$newFileName = $newBaseName + $extension
if ($allFiles.Count) {
$count = 1
while ($allFiles -contains $newFileName) {
$newFileName = "{0}({1}){2}" -f $newBaseName, $count++, $extension
}
}
Write-Host "Renaming '$($_.FullName)' to '$newFileName'"
$_ | Rename-Item -NewName $newFileName -Force
}
Hope that helps
The goal of this script is supposed to be to find any base64 encoded strings within a particular row of a CSV, grab only the base64, add the correct amount of padding charcters to it, decode it, and write it to an output.
function Get-FileName($initialDirectory) {
[System.Reflection.Assembly]::LoadWithPartialName("System.Sindows.Forms") | Out-Null
$OpenFileDialog = New-Object System.Windows.Forms.OpenFileDialog
$OpenFileDialog.InitialDirectory = $initialDirectory
$OpenFileDialog.Filter = "CSV (*.csv)| *.csv"
$OpenFileDialog.ShowDialog() | Out-Null
$OpenFileDialog.Filename
}
$inputfile = Get-FileName "C:\My Documents\"
$inputdata = Import-Csv $inputfile
$output = "C:\My Documents\output.csv"
$regex = [regex]::New('(?<=-[Ee]ncoded[Cc]ommand\s"??)[\w\d=\/!]{0,792}')
$csv1 = $inputdata | select -ExpandProperty 'NameOfRow' | ? {$_ -ne ""}
| Select-String -Pattern $regex -Allmatches | % {$_.Matches}
| ? {$_.Value -ne ""} | % {$_.Value}
foreach ($line in $csv1) {
$csvL = $line.Length
$csvM = $csvL %= 4
if ($csvM | % {$_ -eq 1}) {
$line | % {$_ + "==="}
} elseif ($csvM | % {$_ -eq 2}) {
$line | % {$_ + "=="}
} elseif ($csvM | % {$_ -eq 3}) {
$line | % {$_ + "="}
}
$decode = [System.Text.Encoding]::ASCII.GetString([System.Convert]::FromBase64String($line))
$decode > $output
}
Could someone please help me with extracting and counting the numbers from a text file with PowerShell?
Example: c:\temp\1.txt is some text with semicolon and numbers after them. I need to sum all of these numbers.
blablabl:5 dzfdsfdsfsdfsf:10
sdfsdfsdfdffs:8sdfsfsfdsfdsf:111
5+10+8+111...
What I've tried so far:
$LogText = "C:\temp\1.txt"
[regex]$Regex = "\. (\d+):[1]"
$Matches = $Regex.Matches($LogText)
$Matches | ForEach-Object {
Write-Host $Matches
}
#$array = #()
#$array = new-object collections.arraylist
$array = while ($Matches.Success) {
Write-Host $array[i++]
}
# -------------------------------------------------------------------
$text = Get-Content "C:\temp\1.txt"
[regex]$Regex = "\d"
$Matches = $Regex.Matches($text)
# -------------------------------------------------------------------
$pos = $text.IndexOf(":")
$rightPart = $text.Substring($pos+1)
Write-Host $rightPart
Use Select-String to extract the matches from the file and Measure-Object to do the calculation.
Select-String -Path 'C:\temp\1.txt' -Pattern '(?<=:)\d+' -AllMatches |
Select-Object -Expand Matches |
Select-Object -Expand Value |
Measure-Object -Sum |
Select-Object -Expand Sum
(?<=:) is a positive lookbehind assertion to match the colon preceding the number without making it part of the match.
Try it like that:
$txt=
#"
blablabl:5 dzfdsfdsfsdfsf:10
sdfsdfsdfdffs:8sdfsfsfdsfdsf:111
"#
[regex]$Regex = '\d+'
$sum=0;
$Regex.Matches($txt) | ForEach-Object {
$val = [int]$_.Value
$val
$sum+=$val
}
$sum
I have a text file that an application constantly errors to. I want to monitor this file with Powershell and log every error to another source.
Problem to solve: how do i pass multiline text when we are in -wait? Get-Content is passing arrays of strings.
$File = 'C:\Windows\Temp\test.txt'
$content = Get-Content -Path $file
# get stream of text
Get-Content $file -wait -Tail 0 | ForEach-Object {
if ($_ -match '(<ACVS_T>)((.|\n)*)(<\/ACVS_T>)+'){
write-host 'match found!'
}
}
Example of text junks that get drop:
<ACVS_T>
<ACVS_D>03/01/2017 17:24:03.602</ACVS_D>
<ACVS_TI>bf37ba1c9,iSTAR Server Compone</ACVS_TI>
<ACVS_C>ClusterPort</ACVS_C>
<ACVS_S>SoftwareHouse.NextGen.HardwareInterface.Nantucket.Framework.ClusterPort.HandleErrorState( )
</ACVS_S>
<ACVS_M>
ERROR MESSAGE FROM APP
</ACVS_M>
<ACVS_ST>
</ACVS_ST>
</ACVS_T>
solved it!
$File = 'D:\Program Files (x86)\Tyco\CrossFire\Logging\SystemTrace.Log'
$content = Get-Content -Path $file
# get stream of text
$text = ''
Get-Content $file -wait -Tail 0 | ForEach-Object {
$text +=$_
if ($text -match '(<ACVS_T>)((.|\n)*)(<\/ACVS_T>)+'){
[xml]$XML = "<Root>" + $text + "</Root>"
$text='' #clear it for next one
$XML.Root.ACVS_T | ForEach-Object {
$Obj = '' | Select-Object -Property ACVS_D, ACVS_TI, ACVS_C, ACVS_S, ACVS_M, ACVS_ST
$Obj.ACVS_D = $_.ACVS_D
$Obj.ACVS_ST = $_.ACVS_ST
$Obj.ACVS_C = $_.ACVS_C
$Obj.ACVS_S = $_.ACVS_S
$Obj.ACVS_M = $_.ACVS_M
$Obj.ACVS_ST = $_.ACVS_ST
write-host "`n`n$($Obj.ACVS_M)"
}
}
}
I need to put a bunch of users through a regex query that look for 'IM' or '(IM)' at the end of their DisplayName and then removes it. I'll be exporting first of course for backup.
I'm have used the .contains method but if I use that any Tim's or Imogen's might get a little annoyed at me.
$users = Get-DistributionGroupMember 'DGName' | ? { $_.RecipientType -eq 'UserMailbox' }
$mailboxes = $users | Get-Mailbox
$output = #()
$mailboxes | Select-Object Alias, DisplayName | Export-Csv D:\Scripts\PreDisplayNames.csv -NoTypeInformation
foreach ($mb in $mailboxes){
if ($mb.DisplayName.EndsWith(" (IM)")){
$newString = $mb.DisplayName.TrimEnd(" (IM)")
$newStringTD = $mb.DisplayName.TrimEnd(" (IM)") + " (TD)"
$resultstring = "Setting DisplayName to $newString"
$mb | Set-Mailbox -DisplayName $newString -WhatIf
}
elseif($mb.DisplayName.EndsWith(" IM")){
$newString = $mb.DisplayName.TrimEnd(" IM")
$newStringTD = $mb.DisplayName.TrimEnd(" (IM)") + " (TD)"
$resultstring = "Setting DisplayName to $newString"
$mb | Set-Mailbox -DisplayName $newString -WhatIf
}
else{
$newString = 'DNIncorrect'
$resultstring = 'Incorrect DisplayName'
}
$props = [ordered]#{
'DisplayName' = $mb.DisplayName
'NewDisplayName' = $newString
'NewDisplayName TD' = $newStringTD
}
$object = New-Object -TypeName PSObject -Property $props
$output += $object
}
$output | Export-Csv "D:\scripts\DisplayNames.csv" -NoTypeInformation
Theres no need for a regex here, just use a -like to filter your objects when you collect the mailboxes
$mailboxes = $users | Get-Mailbox | Where-object {($_.DisplayName -like "*IM") -or ($_.DisplayName -like "*(IM)")}
If you still wanted a regex I normally use the following to generate a regex for specific words. It may work for you situation.
$Eliminate = #("bak", "error", "research","Retry")
[regex]$eliminate_regex = '(?i)(' + (($Eliminate | foreach { [regex]::escape($_) }) -join "|") + ')'