## Declaire Variables
## File Path Variables
$savePath = $env:TEMP
$sourceFileAccellerator = "C:\ALL_SALES_WITH_uuid_07-09-2014.txt" ##Path to Source File goes here
$sourceFileLineBreaks = $($savePath + "NifaLeadsListCategoriesWLineBreaks.txt")
$categoryOutput = $($savePath + "CategoryParsedforPowerShell.txt")
$categoryOutputXtraSlash = $($savePath + "CategoriesXtraSlash.txt")
$categoryOutputNoNull = $($savePath + "CategoriesNoNULL.txt")
$searchedFile = $($savePath + $exactSearchTerm + "Temp.txt")
$uuidOutput = $($savePath + $exactSearchTerm + "uuid.SQL")
$uuidColumnSQL = $($savePath + $exactSearchTerm + "Table.sql")
## .sql file creation strings
$sqlUpdate = $("UPDATE ADC.dbo.Contacts SET [" + $exactSearchTerm + "] = 1 WHERE NifaID = '")
$columnAddLine1 = "ALTER TABLE ADC.dbo.Contacts"
$columnAdd = $("ADD [" + $exactSearchTerm + "] nvarchar(10)")
## Regex Variables
$uuidRegEx = '[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}'
$categoryRegEX = '(?<=\\)[^\\\r\n]*\\[^\\\r\n]*$'
## Get the original text file and add line breaks to the categories and output to a new file.
Get-Content $sourceFileAccellerator |% {$_-replace "`t","`n"} > $sourceFileLineBreaks
## Look for Categories and pars the last 2 categories
select-string -Path $sourceFileLineBreaks -Pattern $categoryRegEX -AllMatches | % { $_.Matches } | % { $_.Value } > $categoryOutput
## Cleanup the Categories for input into the ForEach loop.
$job1 = Start-Job { Get-Content $categoryOutput |% {$_-replace "\\\\", "\\\\\\\\"} > $categoryOutputXtraSlash }
Wait-Job $job1
Receive-Job $job1
$job2 = Start-Job { Get-Content $categoryOutputXtraSlash |% {$_-replace '/(?:(?:\r\n|\r|\n)\s*){2}/s', "\n\n"} > $categoryOutputNoNull }
Wait-Job $job2
Receive-Job $job2
## Import the SQL Cmdlets
Import-Module SQLPS
## Load the searchFile and run the script on each column
$SearchFile = New-Object System.IO.StreamReader -Arg $categoryOutputNoNull
while ($line = $file.ReadLine()) {
Select-String -Path $sourceFile -Pattern $line > $searchedFile
Select-String -Path $searchedFile -Pattern $uuidRegEx -AllMatches | % { $_.Matches } | % { $sqlUpdate + $_.Value + "';"} > $uuidOutput
## Create the sql script to add a column
Add-Content -Path $uuidColumnSQL -Value $($columnAddLine1 + " " + $columnAdd)
## Run the SQL scripts
Invoke-Sqlcmd -InputFile $uuidColumnSQL -OutputSqlErrors $true -ServerInstance DUMPSTER\DB_DEFAULT -QueryTimeout 0
Invoke-Sqlcmd -InputFile $uuidOutput -OutputSqlErrors $true -ServerInstance DUMPSTER\DB_DEFAULT -QueryTimeout 0
## Remove Temporary files
Remove-Item $searchedFile
Remove-Item $uuidOutput
Remove-Item $uuidColumnSQL
}
$file.close()
Remove-Item $categoryOutput
Remove-Item $sourceFileLineBreaks
Remove-Item $categoryOutputXtraSlash
I'm working on the above script that parses out data from a tab delimited .txt file and and have tested every component of the script and have had success, but when they're combined into a while loop, Powershell is throwing the errors below:
Output:
Id Name PSJobTypeName State HasMoreData Location Command
-- ---- ------------- ----- ----------- -------- -------
6 Job6 BackgroundJob Completed True localhost Get-Content $category...
Cannot bind argument to parameter 'Path' because it is null.
+ CategoryInfo : InvalidData: (:) [Get-Content], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.GetContentCommand
+ PSComputerName : localhost
8 Job8 BackgroundJob Completed True localhost Get-Content $category...
Cannot bind argument to parameter 'Path' because it is null.
+ CategoryInfo : InvalidData: (:) [Get-Content], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.GetContentCommand
+ PSComputerName : localhost
You cannot call a method on a null-valued expression.
At C:\CategoryParseAndSQLUpdate.ps1:53 char:8
+ while ($line = $file.ReadLine()) {
+ ~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
You cannot call a method on a null-valued expression.
At C:\CategoryParseAndSQLUpdate.ps1:66 char:1
+ $file.close()
+ ~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
When you use jobs (e.g. with Start-Job cmdlet) you have to look carefully at the scope of variable(s). Depending on the version: you can either:
access variable from parent scope with $using:VariableName (v. 3+)
use ArgumentList parameter from Start-Job and param() block inside script block
EDIT
Just noticed: I can't see where you define $file - looks like it's missing?
Related
Trying to parse a text file using PowerShell and create a list of sections present in the file.
While executing the below code snippet, error is thrown when the object is added to the list.
Method invocation failed because [System.Management.Automation.PSObject] does not contain a method named 'op_Addition'.
The first object gets added to the list but second object onward error is thrown.
Interestingly, the error does not occur if the program runs in debug mode.
function Process-Master-File {
[CmdletBinding()]
Param (
[Parameter(ValueFromPipeline = $true)][String[]]$PipeValue
)
Begin {
#create new section object
$codeSection = New-Object -TypeName psobject
$codeSection | Add-Member -MemberType NoteProperty -Name Name -Value $null
$codeSection | Add-Member -MemberType NoteProperty -Name SuppresionGroup -Value $null
$codeSection | Add-Member -MemberType NoteProperty -Name Mandatory -Value $False
$codeSection | Add-Member -MemberType NoteProperty -Name SectionBody -Value $null
[string]$out = ""
}
Process {
# Grab a line from the pipe
[string]$line = $_
# Find positions
try {
$pos = 0
$pos = $line.IndexOf('#Section')
if ($pos[0] -ne -1) {
if ($objTemp.Name) {
$objTemp.SectionBody = $section
$codeSectionList += $objTemp # Error is thrown here
$section = ""
}
$objTemp = $codeSection | Select-Object *
$objTemp.Name = $line.Substring($line.LastIndexOf(':') + 1).TrimEnd().TrimStart()
$objTemp.SuppresionGroup = $line.Substring($line.IndexOf('#SG') + 3, ($line.LastIndexOf(':') - $line.IndexOf('#SG') - 3)).TrimEnd().TrimStart()
if ($line.IndexOf('#Mandatory') -ne -1) {
$objTemp.Mandatory = $True
}
$section = $line
Write-Verbose $line
}
else {
$section += $line
}
}
Catch {
Write-Host "An error occurred while processing file:"
Write-Host $_
}
}
End {
$codeSectionList
}
}
You have to initialize $codesectionlist as an array. Otherwise, you're trying to add one pscustomobject to another one.
$codesectionlist += [pscustomobject]#{name='Joe'}
$codesectionlist += [pscustomobject]#{name='John'}
InvalidOperation: Method invocation failed because [System.Management.Automation.PSObject] does not contain a method named 'op_Addition'.
$codesectionlist.gettype().fullname
System.Management.Automation.PSCustomObject
$codesectionlist = #()
$codesectionlist += [pscustomobject]#{name='Joe'}
$codesectionlist += [pscustomobject]#{name='John'}
$codesectionlist.gettype().fullname
System.Object[]
The global variable might have been in included in the question. The script uses its own script scope, with the += operator. Debugging is like dot sourcing the script.
cat script.ps1
$a += 1
$a += 2
$a
$global:a = #()
.\script
3
. .\script
1
2
Weirdly, $a = $a + 1 works differently than $a += 1 in terms of scope.
cat ./script.ps1
$a
$a = $a + 1
$a = $a + 2
$a
$global:a = #()
./script
1
2
Actually I used a script instead of a function, but everything still applies.
Oh that's right. This is a bug I posted myself last December: +=, functions, and global variables #11297
Needing to replace a string in multiple text files with the same string , except with capture group 2 replaced by the sum of itself and capture group 4.
String: Total amount $11.39 | Change $0.21
Desired Result: Total amount $11.60 | Change $0.21
I have attempted several methods. Here is my last attempt which seems to run without error, but without any changes to the string .
$Originalfolder = "$ENV:userprofile\Documents\folder\"
$Originalfiles = Get-ChildItem -Path "$Originalfolder\*"
$RegexPattern = '\b(Total\s\amount\s\$)(\d?\d?\d?\d?\d\.?\d?\d?)(\s\|\sChange\s\$)(\d?\d?\d\.?\d?\d?)\b'
$Substitution = {
Param($Match)
$Result = $GP1 + $Sumtotal + $GP3 + $Change
$GP1 = $Match.Groups[1].Value
$Total = $Match.Groups[2].Value
$GP3 = $Match.Groups[3].Value
$Change = $Match.Groups[4].Value
$Sumtotal = ($Total + $Change)
return [string]$Result
}
foreach ($file in $Originalfiles) {
$Lines = Get-Content $file.FullName
$Lines | ForEach-Object {
[Regex]::Replace($_, $RegexPattern, $Substitution)
} | Set-Content $file.FullName
}
For one thing, your regular expression doesn't even match what you're trying to replace, because you escaped the a in amount:
\b(Total\s\amount\s\$)(\d?\d?\d?...
# ^^
\a is an escape sequence that matches the "alarm" or "bell" character \u0007.
Also, if you want to calculate the sum of two captures you need to convert them to numeric values first, otherwise the + operator would just concatenate the two strings.
$Total = $Match.Groups[2].Value
$Change = $Match.Groups[4].Value
$Sumtotal = $Total + $Change # gives 11.390.21
$Sumtotal = [double]$Total + [double]$Change # gives 11.6
And you need to build $Result after you defined the other variables, otherwise the replacement function would just return an empty string.
Change this:
$RegexPattern = '\b(Total\s\amount\s\$)(\d?\d?\d?\d?\d\.?\d?\d?)(\s\|\sChange\s\$)(\d?\d?\d\.?\d?\d?)\b'
$Substitution = {
param ($Match)
$Result = $GP1 + $Sumtotal + $GP3 + $Change
$GP1 = $Match.Groups[1].Value
$Total = $Match.Groups[2].Value
$GP3 = $Match.Groups[3].Value
$Change = $Match.Groups[4].Value
$Sumtotal = ($Total + $Change)
return [string]$Result
}
into this:
$RegexPattern = '\b(Total\samount\s\$)(\d?\d?\d?\d?\d\.?\d?\d?)(\s\|\sChange\s\$)(\d?\d?\d\.?\d?\d?)\b'
$Substitution = {
Param($Match)
$GP1 = $Match.Groups[1].Value
$Total = [double]$Match.Groups[2].Value
$GP3 = $Match.Groups[3].Value
$Change = [double]$Match.Groups[4].Value
$Sumtotal = ($Total + $Change)
$Result = $GP1 + $Sumtotal + $GP3 + $Change
return [string]$Result
}
and the code will mostly do what you want. "Mostly", because it will not format the calculated number to double decimals. You need to do that yourself. Use the format operator (-f) and change your replacement function to something like this:
$Substitution = {
Param($Match)
$GP1 = $Match.Groups[1].Value
$Total = [double]$Match.Groups[2].Value
$GP3 = $Match.Groups[3].Value
$Change = [double]$Match.Groups[4].Value
$Sumtotal = $Total + $Change
return ('{0}{1:n2}{2}{3:n2}' -f $GP1, $Sumtotal, $GP3, $Change)
}
As a side note: the sub-expression \d?\d?\d?\d?\d\.?\d?\d? could be shortened to \d+(?:\.\d+)? (one or more digit, optionally followed by a period and one or more digits) or, more exactly, to \d{1,4}(?:\.\d{0,2})? (one to four digits, optionally followed by a period and up to 2 digits).
here's how I'd do it: this is pulled out of a larger script that regularly scans a directory for files, then does a similar manipulation, and I've changed variables quickly to obfuscate, so shout if it doesn't work and I'll take a more detailed look tomorrow.
It takes a backup of each file as well, and works on a temp copy before renaming.
Note it also sends an email alert (code at the end) to say if any processing was done - this is because it's designed to run as as scheduled task in the original
$backupDir = "$pwd\backup"
$stringToReplace = "."
$newString = "."
$files = #(Get-ChildItem $directoryOfFiles)
$intFiles = $files.count
$tmpExt = ".tmpDataCorrection"
$DataCorrectionAppend = ".DataprocessBackup"
foreach ($file in $files) {
$content = Get-Content -Path ( $directoryOfFiles + $file )
# Check whether there are any instances of the string
If (!($content -match $stringToReplace)) {
# Do nothing if we didn't match
}
Else {
#Create another blank temporary file which the corrected file contents will be written to
$tmpFileName_DataCorrection = $file.Name + $tmpExt_DataCorrection
$tmpFile_DataCorrection = $directoryOfFiles + $tmpFileName_DataCorrection
New-Item -ItemType File -Path $tmpFile_DataCorrection
foreach ( $line in $content ) {
If ( $line.Contains("#")) {
Add-Content -Path $tmpFile_DataCorrection -Value $line.Replace($stringToReplace,$newString)
#Counter to know whether any processing was done or not
$processed++
}
Else {
Add-Content -Path $tmpFile_DataCorrection -Value $line
}
}
#Backup (rename) the original file, and rename the temp file to be the same name as the original
Rename-Item -Path $file.FullName -NewName ($file.FullName + $DataCorrectionAppend) -Force -Confirm:$false
Move-Item -Path ( $file.FullName + $DataCorrectionAppend ) -Destination backupDir -Force -Confirm:$false
Rename-Item -Path $tmpFile_DataCorrection -NewName $file.FullName -Force -Confirm:$false
# Check to see if anything was done, then populate a variable to use in final email alert if there was
If (!$processed) {
#no message as did nothing
}
Else {
New-Variable -Name ( "processed" + $file.Name) -Value $strProcessed
}
} # Out of If loop
}
I have a PowerShell script named script1.ps1 that work perfectly. Here is the script:
Write-Host Script to display members of the local -ForegroundColor Green
Write-Host Administators group of a remote server. -ForegroundColor Green
Write-Host "`n"
$strComputer = Read-Host "Please enter the computer name"
$computer = [ADSI]("WinNT://" + $strComputer + ",computer")
$group = $computer.PSBase.Children.Find("administrators")
Write-Host ""
Write-Host "Computer Name : "$computer.Name
Write-Host "_____________________________________"
Write-Host ""
Write-Host "Group Name : "$Group.Name
Write-Host "_____________________________________"
$domain = $group.Path.Split("/")[2]
$string1 = "WinNT://" + $domain + "/" + $strComputer + "/"
$string2 = $strComputer + "/"
$string3 = "WinNT://"
$members = ($group.PSBase.Invoke("Members") | Foreach-Object {$_.GetType().InvokeMember("Adspath", 'GetProperty', $null, $_, $null)}) -replace ($string1,$string2) -replace ($string3,"")
Write-Host ""
Write-Host "Members : "
Write-Host ""
$members
But as you can see, I'm obliged to write each time the name of computer I want.
I ask myself if there is a function or other things which take automatically the computers names from a text or CSV file?
Like this at the beginning of the script, it automatically take all the computers names & give me the members of the local 'Administators' group which then gonna be exported in one text file or CSV file too?
Updated script according to the answer given.
Write-Host Script to display members of the local -ForegroundColor Green
Write-Host Administators group of a remote server. -ForegroundColor Green
Write-Host "`n"
Get-Content 'C:\Users\herbautr\Desktop\List1.txt' | ForEach-Object {
Write-Host "-$_-"
$computer = [ADSI]("WinNT://" + $_ + ",computer")
$group = $computer.PSBase.Children.Find("administrators")
Write-Host ""
Write-Host "Computer Name : "$computer.Name
Write-Host "_____________________________________"
Write-Host ""
Write-Host "Group Name : "$Group.Name
Write-Host "_____________________________________"
$domain = $group.Path.Split("/")[2]
$string1 = "WinNT://" + $domain + "/" + $_ + "/"
$string2 = $_ + "/"
$string3 = "WinNT://"
$members = ($group.PSBase.Invoke("Members") | Foreach-Object {$_.GetType().InvokeMember("Adspath", 'GetProperty', $null, $_, $null)}) -replace ($string1,$string2) -replace ($string3,"")
Write-Host ""
Write-Host "Members : "
Write-Host ""
$members
} | Set-Content 'C:\Users\herbautr\Desktop\administrators.txt'
I have add 1 computer name to the List1.txt:
01SPEAI-TEST1
01SPEAI-TEST2
02SPHPV-TEST1
01SLCPTAAP-PROD
And it works (not) perfectly (Unreadable layout)
01SPEAI-PROD1/Administrator
VNF-PRINCIPAL/Admins du domaine
VNF-PRINCIPAL/svceri
01SPEAI-PROD2/Administrator
VNF-PRINCIPAL/Admins du domaine
VNF-PRINCIPAL/svceri
02SPHPV-PROD1/Administrator
VNF-PRINCIPAL/Admins du domaine
01SLCPTAAP-PROD/Administrator
VNF-PRINCIPAL/Admins du domaine
01SLCPTAAP-PROD/maint
VNF-PRINCIPAL/svcoraas
VNF-PRINCIPAL/svcvisionit
VNF-PRINCIPAL/GopOAS
VNF-PRINCIPAL/svcdigora
Note (15:18pm): I have tried with 5 names, it continue to work.
Why when adding just 1 name it "works"?
You're looking for Get-Content and Set-Content.
Get-Content 'C:\path\to\computers.txt' | ForEach-Object {
$computer = [ADSI]("WinNT://" + $_ + ",computer")
...
} | Set-Content 'C:\path\to\administrators.txt'
Note that you need to replace all occurrences of $strComputer inside the ForEach-Object loop with the current object automatic variable ($_).
If you want to use CSVs for input and output use the Import-Csv and Export-Csv cmdlets.
Import-Csv 'C:\path\to\computers.csv' | ForEach-Object {
$computer = [ADSI]("WinNT://" + $_.ComputerName + ",computer")
...
$members | ForEach-Object {
New-Object -Type PSObject -Property #{
Member = $_
}
}
} | Export-Csv 'C:\path\to\administrators.csv' -NoType
Note that CSVs have some advantages when you need to handle items with multiple properties, but they require somewhat more elaborate handling than simple strings (as you can see in my example above).
I'm new with powershell and in need of guidance. Been scouring the site for answers and coming up blank, decided to ask instead. If this has been answered please refer me to the link.
I have an application log (xml format) like below:
<log><identifier>123axr4x5</identifier><login>USER1</login><source>Order-Management</source><AddlInfo>Execution Time : 20ms</AddlInfo><Exception></Exception><timestamp>01/01/2015:22:00:00</timestamp><serverticks>643670855</serverticks><PID>1234</PID><Machine>PRD01X12mm</Machine></log>
<log><identifier>dd8jksl3g</identifier><login>USER2</login><source>Service-Assurance</source><AddlInfo>Execution Time : 80ms</AddlInfo><Exception></Exception><timestamp>01/01/2015:22:00:00</timestamp><serverticks>643680865</serverticks><PID>1234</PID><Machine>PRD01X12mm</Machine></log>
: and so on
I am creating a log parser that will scan a folder and its subfolder for matching regex pattern, and based on certain threshold, output into gridview/export to CSV. I am almost done, however i'm unable to solve 1 problem, which is to get the filename currently being parsed, to be displayed on the gridview.
Basically i am using piped Get-ChildItem as below
Get-ChildItem $Dir -recurse -Filter *logging*.txt|
Sort-Object LastWriteTime |
?{$_.LastWriteTime -gt (Get-Date).AddMinutes(-60)}|
Select-String -Pattern $Text |
Select-String -Pattern $Text3 |
Select-String -Pattern $Text2 -allmatches |
Foreach-Object {
$information = $_|Select-Object -Property API, Duration,DataRetrieved, ServerTime, ServerTicks , Identifier, Filename
$information.Filename = $_.Name
#$information.Filename = $_.FullName
} |
Out-GridView
Below is the full code:
$Dir = "C:\log\"
$threshold = 1 + 0
$StartTime = (Get-Date).ToString();
$EndTime = (Get-Date).ToString();
$Text = "abc"
$Text2 = "def"
$Text3 = "ghi"
$OutFile = "result"
$OutPath = $Dir + $OutFile + ".txt"
#ExtractionParameters
$AddlInnfoTagBegin = "AddlInfo"
$AddlInnfoTagEnd = "/AddlInfo"
$ServerTimeOfLogTagBegin = "ServerTimeOfLog"
$ServerTimeOfLogTagEnd = "/ServerTimeOfLog"
$ServerTicksTagBegin = "ServerTicks"
$ServerTicksTagEnd = "/ServerTicks"
$IdentifierTagBegin = "Identifier"
$IdentifierTagEnd = "/Identifier"
#parse file in folders
Get-ChildItem $Dir -recurse -Filter *logging*.txt|
Sort-Object LastWriteTime |
#?{$_.LastWriteTime -gt (Get-Date).AddMinutes(-60)}|
Select-String -Pattern $Text |
Select-String -Pattern $Text3 |
Select-String -Pattern $Text2 -allmatches |
Foreach-Object {
# take line and split it at tabulators
$parts = $_.Line
#write $parts
$indexOfAddlInfoBegin = $parts.IndexOf($AddlInnfoTagBegin) + $AddlInnfoTagBegin.Length +1
$indexOfAddlInfoEnd = $parts.IndexOf($AddlInnfoTagEnd) -1
$AddlInfoData = $parts.Substring($indexOfAddlInfoBegin, $indexOfAddlInfoEnd - $indexOfAddlInfoBegin)
$AddlInfoReplaced = $AddlInfoData.Replace(" seconds ","#")
$AddlInfoSplit = $AddlInfoReplaced.Split('#')
$information = $_|Select-Object -Property API, Duration,DataRetrieved, ServerTime, ServerTicks , Identifier, Filename
#get filename, which does not work
$information.Filename = $_.Name
#$information.Filename = $_.FullName
$information.API = $AddlInfoSplit[0].Split(':')[0]
$information.DataRetrieved = $AddlInfoSplit[1]
$information.Duration = $AddlInfoSplit[0].Split(':')[1]
$information.Duration = $information.Duration.Replace("Execution Time = ","")
$indexOfServerTimeBegin = $parts.IndexOf($ServerTimeOfLogTagBegin) + $ServerTimeOfLogTagBegin.Length +1
$indexOfServerTimeEnd = $parts.IndexOf($ServerTimeOfLogTagEnd) -1
$ServerTimeData = $parts.Substring($indexOfServerTimeBegin, $indexOfServerTimeEnd - $indexOfServerTimeBegin)
$information.ServerTime = $ServerTimeData
$indexOfServerTicksBegin = $parts.IndexOf($ServerTicksTagBegin) + $ServerTicksTagBegin.Length +1
$indexOfServerTicksEnd = $parts.IndexOf($ServerTicksTagEnd) -1
$ServerTickData = $parts.Substring($indexOfServerTicksBegin, $indexOfServerTicksEnd - $indexOfServerTicksBegin)
$information.ServerTicks = $ServerTickData
$indexOfIdentifierBegin = $parts.IndexOf($IdentifierTagBegin) + $IdentifierTagBegin.Length +1
$indexOfIdentifierEnd = $parts.IndexOf($IdentifierTagEnd) -1
$IdentifierData = $parts.Substring($indexOfIdentifierBegin, $indexOfIdentifierEnd - $indexOfIdentifierBegin)
$information.Identifier = $IdentifierData
$DurationAsInt = 0 + $information.Duration
if($DurationAsInt -gt $threshold) {
write $information
}
} |
Out-GridView
#Out-File -FilePath $OutPath -Append -Width 200
Any help is appreciated, thanks!!
-CL
The property you are looking for is "FileName".
$information.Filename = $_.FileName
Powershell provides a cmdlet "Get-Member" which would list all available properties/methods. You could enumerate the members to console and inspect what is available
Write-Host ( $_ | Get-Member)
I have written a log tailer with Powershell, the tailer loads in an xml file which contains configuration information regarding when to report on a word match in the log tail (basically if certain patterns occur X amount of times in the tail).
At the moment the tailer is not returning matches for many of the lines that contain matches.
For example we are retrieving a log file with many INFO lines, if I check for the word INFO nothing is detected, however if I look for the work shutdown it returns matches (the line with shutdown also contains INFO on the line).
The really strange thing is that using the same log file and same Powershell script seems to produce perfectly accurate results on my own machine but behaves strangely on the server.
I suspect that this might be an issue with the version of Powershell that is running on the server, so I was hoping someone here might know of issues that can come up with different versions. I have also noticed that when I print out the number of matches, if nothing is found the output is blank, perhaps this should be 0 and is causing some weird issue to trigger?
function Main()
{
#### GLOBAL SETTINGS
$DebugPreference = "Continue"
$serverName = $env:COMPUTERNAME
$scriptPath = Split-Path $script:MyInvocation.MyCommand.Path
$logConfigPath = "$scriptPath/config.xml"
#### VARIABLES RELATING TO THE LOG FILE
#contains the log path and log file mask
$logPaths = #()
$logFileMasks = #()
# the total number of lines grabbed from the end of the log file for evaluation
$numLinesToTail = 1000
# key value pair for the strings to match and the max count of matches before they are considered an issue
$keywords = #()
$maxCounts = #()
#### VARIABLES RELATING TO THE EMAIL SETTINGS
$smtpServer = "mail server"
$emailSubject = "$serverName log report"
$toEmailAddress = "email accounts"
$fromEmailAddress = ""
# any initial content you want in the email body should go here (e.g. the name of the server that this is on)
$htmlBodyContent = "<p><h3>SERVER $serverName : </h3></p><p>Items that appear in red have exceeded their match threshold and should be investigated.<br/>Tail Lines: $numLinesToTail</p>"
#### FUNCTION CALLS
LoadLogTailerConfig $logConfigPath ([ref]$logPaths) ([ref]$logFileMasks) ([ref]$keywords) ([ref]$maxCounts)
for ($i = 0; $i -lt $logPaths.Count; $i++)
{
$tail = GetLogTail $numLinesToTail $logPaths[$i] $logFileMasks[$i]
$tailIssueTable = CheckForKeywords $tail $keywords[$i] $maxCounts[$i]
if ($tailIssueTable -ne "")
{
$htmlBodyContent += "<br/>Logs scanned: " + (GetLatestLogFileFullName $logPaths[$i] $logFileMasks[$i]) + "<br/><br/>" + $tailIssueTable
SendIssueEmail $smtpServer $emailSubject $toEmailAddress $ccEmailAddress $fromEmailAddress $htmlBodyContent
}
}
}
# Loads in configuration data for the utility to use
function LoadLogTailerConfig($logConfigPath, [ref]$logPaths, [ref]$logFileMasks, [ref]$keywords, [ref]$maxCounts)
{
Write-Debug "Loading config file data from $logConfigPath"
[xml]$configData = Get-Content $logConfigPath
foreach ($log in $configData.Logs.Log) {
$logPaths.Value += $log.FilePath
$logFileMasks.Value += $log.FileMask
$kwp = #()
$kwc = #()
foreach ($keywordSet in $log.Keywords.Keyword)
{
$kwp += $keywordSet.Pattern
$kwc += $keywordSet.MaxMatches
}
$keywords.Value += #(,$kwp)
$maxCounts.Value += #(,$kwc)
}
}
# Gets a string containing the last X lines of the most recent log file
function GetLogTail($numLinesToTail, $logPath, $logFileMask)
{
$logFile = GetLatestLogFileFullName $logPath $logFileMask #Get-ChildItem $logPath -Filter $logFileMask | sort LastWriteTime | select -Last 1
Write-Debug "Getting $numLinesToTail line tail of $logFile"
$tail = Get-Content "$logFile" | select -Last $numLinesToTail
return $tail
}
function GetLatestLogFileFullName($logPath, $logFileMask)
{
$logFile = Get-ChildItem $logPath -Filter $logFileMask | sort LastWriteTime | select -Last 1
return "$logPath$logFile"
}
# Returns body text for email containing details on keywords in the log file and their frequency
function CheckForKeywords($tail, $keywords, $maxCounts)
{
$issuesFound = 0
$htmlBodyContent += "<table><tr><th style=""text-align : left;"">Keyword</th><th>Max Count Value</th><th>Count Total<th></tr>"
for ($i = 0; $i -lt $keywords.Count; $i++)
{
$keywordCount = ($tail | Select-String $keywords[$i] -AllMatches).Matches.Count
Write-Debug (("Match count for {0} : {1}" -f $keywords[$i], $keywordCount))
if ($keywordCount -gt $maxCounts[$i])
{
# style red if the count threshold has been exceeded
$htmlBodyContent += "<tr style=""color : red;""><td>" + $keywords[$i] + "</td><td>" + $maxCounts[$i] + "</td><td>" + $keywordCount + "</td></tr>"
$issuesFound = 1
}
else
{
# style green if the count threshold has not been exceeded
$htmlBodyContent += "<tr style=""color : green;""><td>" + $keywords[$i] + "</td><td>" + $maxCounts[$i] + "</td><td>" + $keywordCount + "</td></tr>"
}
}
$htmlBodyContent += "</table>"
if ($issuesFound -eq 1)
{
return $htmlBodyContent
}
return ""
}
# Sends out an email to the specified email address
function SendIssueEmail($smtpServer, $subject, $toAddress, $ccAddress, $fromAddress, $bodyContent)
{
Write-Debug "Sending email with subject: $subject, To: $toAddress, via SMTP ($smtpServer)"
Send-MailMessage -SmtpServer $smtpServer -Subject $subject -To $toAddress -From $fromAddress -BodyAsHtml $bodyContent
}
cls
Main
And a XML config example:
<Logs>
<Log>
<FilePath>C:/Some/Path</FilePath>
<FileMask>log.*</FileMask>
<Keywords>
<Keyword>
<Pattern>NullReferenceException</Pattern>
<MaxMatches>10</MaxMatches>
</Keyword>
<Keyword>
<Pattern>Exception</Pattern>
<MaxMatches>10</MaxMatches>
</Keyword>
</Keywords>
</Log>
<Log>
<FilePath>C:/Some/Path</FilePath>
<FileMask>test.*</FileMask>
<Keywords>
<Keyword>
<Pattern>NullReferenceException</Pattern>
<MaxMatches>100</MaxMatches>
</Keyword>
</Keywords>
</Log>
</Logs>
EDIT : The server that is having the issues is running Powershell V 1.0, however the test servers are also running the same version perfectly fine...
Your function GetLatestLogFileFullName is one problem. It can and will generate invalid paths.
function GetLatestLogFileFullName($logPath, $logFileMask)
{
$logFile = Get-ChildItem $logPath -Filter $logFileMask | sort LastWriteTime | select -Last 1
return "$logPath$logFile"
}
Use this instead:
return $logfile.FullName
And you should also check for cases where there is no valid log file:
if ($logfile) {
return $logfile.FullName
} else {
return $null
}
The second problem will be your Select-String usage.
$keywordCount = ($tail | Select-String $keywords[$i] -AllMatches).Matches.Count
In PowerShell v1 Select-String does not have -AllMatches parameter.
PS> Get-Help Select-String
NAME
Select-String
SYNOPSIS
Identifies patterns in strings.
SYNTAX
Select-String [-pattern] <string[]> -inputObject <psobject>[-include <string[]>] [-exclude <string[]>] [-simpleMatch] [-caseSensitive] [-quiet] [-list] [<CommonParameters>]
Select-String [-pattern] <string[]> [-path] <string[]> [-include<string[]>] [-exclude <string[]>] [-simpleMatch] [-caseSensitive] [-quiet] [-list] [<CommonParameters>]
Check the PowerShell versions on your servers using the $PSVersionTable variable. Do not rely on the version displayed in the title bar!
If the variable does not exist you have Version 1.