Powershell 'where' statement -notcontains - regex

I have a simple excerpt form a larger script, basically I'm trying to do a recursive file search, including sub-directories (and any child of the exclude).
clear
$Exclude = "T:\temp\Archive\cst"
$list = Get-ChildItem -Path T:\temp\Archive -Recurse -Directory
$list | where {$_.fullname -notlike $Exclude} | ForEach-Object {
Write-Host "--------------------------------------"
$_.fullname
Write-Host "--------------------------------------"
$files = Get-ChildItem -Path $_.fullname -File
$files.count
}
At the moment this script will exclude the T:\temp\Archive\cst directory, but not the T:\temp\Archive\cst\artwork directory. I'm struggling to overcome this simple thing.
I've tried the -notlike (which I didn't really expect to work) but also the -notcontains which I was hopeful of.
Can anyone offer any advice, I'm thinking it would require a regex match which I'm reading up on now, but not very familiar with.
In the future the $exclude variable will be an array of strings (directories) but at the moment just trying to get it to work with a straight string.

Try:
where {$_.fullname -notlike "$Exclude*"}
You could also try
where {$_.fullname -notmatch [regex]::Escape($Exclude) }
but the notlike apporach is easier.

When used without wildcards the -like operator does the same as the -eq operator. If you want to exclude a folder T:\temp\Archive\cst and everything below it, you need something like this:
$Exclude = 'T:\temp\Archive\cst'
Get-ChildItem -Path T:\temp\Archive -Recurse -Directory | ? {
$_.FullName -ne $Exclude -and
$_.FullName -notlike "$Exclude\*"
} | ...
-notlike "$Exclude\*" would only exclude subfolders of $Exclude, not the folder itself, and -notlike "$Exclude*" would also exclude folders like T:\temp\Archive\cstring, which may be undesired.
The -contains operator is used to check if a list of values contains a particular value. It doesn't check if a string contains a particular substring.
See Get-Help about_Comparison_Operators for further information.

Try changing
$Exclude = "T:\temp\Archive\cst"
To:
$Exclude = "T:\temp\Archive\cst\*"
This will still return the folder CST as it is a child item of Archive, but will exclude anything under cst.
Or:
$Exclude = "T:\temp\Archive\cst*
But that will also exclude anyfiles that start with "cst" under Archive. Same goes for Graimer's answer, jsut be aware of the trailing \ and if it's important to what you are doing

For those looking for a similar answer, what I ended up going with (to parse an array paths for a wildcard match):
# Declare variables
[string]$rootdir = "T:\temp\Archive"
[String[]]$Exclude = "T:\temp\Archive\cst", "T:\temp\archive\as"
[int]$days = 90
# Create Directory list minus excluded directories and their children
$list = Get-ChildItem -Path $rootdir -Recurse -Directory | where {$path = $_.fullname; -not #($exclude | ? {$path -like $_ -or $path -like "$_\*" }) }
Provides what I needed.

Thought I would add to this as I recently had a similar problem answered. You can use the -notcontains condition, but the thing that is counter intuitive is that the $exclude array needs to be at the start of the expression.
Here is an example.
If I perform the following no items are excluded and it returns "a","b","c","d"
$result = #()
$ItemArray = #("a","b","c","d")
$exclusionArray = #("b","c")
$ItemArray | Where-Object { $_ -notcontains $exclusionArray }
If I switch the variables around in the expression then it works and returns "a","d".
$result = #()
$ItemArray = #("a","b","c","d")
$exclusionArray = #("b","c")
$ItemArray | Where-Object { $exclusionArray -notcontains $_ }
I am not sure why the arrays have to be this way around to work. If anyone else can explain that would be great.
EDITED 12/12/20 - I now know that the other operation to use is "-in" as in
$_ -notin $exclusionArray

Related

Renaming multiple files in directory using a specific string in each file

I have a folder with multiple files and need to rename them to a string inside of the folder. The string is the date of interaction.
Currently the files are named as
AUDIT-1.log
AUDIT-2.log
AUDIT-3.log
ect..
I need to have them as
AUDIT-11-08-22-1.log
AUDIT-11-07-22-2.log
AUDIT-11-08-22-3.log
The issue I am having with the current iteration of the code, the dates of all files are collected and it attempts to rename the file with all dates
EXAMPLE:
NewName: 11-08-22 11-07-22 11-06-22 11-09-22 11-08-22 11-07-22 11-06-22 11-09-22-1.LOG
OldName: C:\TestTemp\AUDIT-2.LOG
There is only one date in each file.
The following is my current code:
$dir ="C:\TestTemp"
$files = Get-ChildItem -Path "$dir\*.log"
$RegexDate = '\d\d\/\d\d\/\d\d'
Measure-Command{
$file_map = #()
foreach ($file in $files) {
$DateName= Get-Content $files |
Select-String $RegexDate |
foreach-object { $_.Matches.Value } |
Select-Object
$NewDateName= $DateName.replace('/','-')
$b = 1
$file_map += #{
OldName = $file.Fullname
NewName = "$NewDateName-$b.LOG" -f $(Get-Content $file.Fullname | Select-Object $NewDateName.Fullname)
}
}
$file_map | ForEach-Object { Rename-Item -Path $_.OldName -NewName $_.NewName }
}
As pointed out in the comments by Santiago Squarzon, the immediate fix is to swap $files, for $file. For code brevity, here's a single pipeline solution you can implement to attain the same results:
Select-String -Path "$dir\*.log" -Pattern '(\d+\/){2}\d+' |
Rename-Item -NewName {
$_.FileName -replace '-', "-$($_.Matches.Value.Replace('/','-'))-"
} -WhatIf
Again, as mentioned in the comments, the use of Select-String allows the reading of file(s) presenting the opportunity to pipe directly into Rename-Item via parameter binding through its Path property. So, using a scriptblock for the new name replacement we're essentially inserting the value found from it's pattern matched into the file name where - would have been.
The -WhatIf safety/common parameter can be removed when you've dictated those are the results you are after.
This will rename the files using their last write time.
If the files were already in that format, they will not be renamed.
There is a hashtable to track the increment of the suffix for the date of the file. This way the files can be organized by date.
$dir = "C:\TestTemp"
$files = Get-ChildItem -Path "$dir\*.log"
#Hashtable to track the suffix for the files
[hashtable]$dateTracking = #{}
#Using padding to format the suffix with two digits, in case there more then 9 files
#incrase it if you have more then 99 files per day increase padding
$suffixPadding = '{0:d2}'
foreach ($file in $files) {
#Don't rename files that were already renamed
if ($file.Name -notmatch "AUDIT-\d{2}-\d{2}-\d{2}-\d{2}\.log") {
$date = $file.LastWriteTime.ToString("MM-yy-dd")
#If the date is not entered in the hashtable add it with suffix 01
if (-not $dateTracking.ContainsKey($date)) {
$dateTracking.Add($date, $suffixPadding -f 1)
}
#Else increment suffix
else {
$dateTracking[$date] = $suffixPadding -f ([int]$dateTracking[$date] + 1)
}
#Here we use the date in the name of the file and getting the suffix from the hashtable
Write-Host "Renaming $($file.Name) to AUDIT-$date-$($dateTracking[$date]).log"
Rename-Item -Path $file -NewName "AUDIT-$date-$($dateTracking[$date]).log"
}
}

PowerShell to match multiple lines with regex pattern

I write a Powershell script and regex to search two configs text files to find matches for Management Vlan. For example, each text file has two Management vlan configured as below:
Config1.txt
123 MGMT_123_VLAN
234 MGMT_VLAN_234
Config2.txt
890 MGMT_VLAN_890
125 MGMT_VLAN_USERS
Below is my script. It has several problems.
First, if I ran the script with the $Mgmt_vlan = Select-String -Path $File -Pattern $String -AllMatches then the screen output shows the expected four (4) Mgmt vlan, but in the CSV file output shows as follow
Filename Mgmt_vlan
Config1.txt System.Object[]
Config2.txt System.Object[]
I ran the script the output on the console screen shows exactly four (4) Management vlans that I expected, but in the CSV file it did not. It shows only these vlans
Second, if I ran the script with $Mgmt_vlan = Select-String -Path $File -Pattern $String | Select -First 1
Then the CSV shows as follows:
Filename Mgmt_vlan
Config1.txt 123 MGMT_123_VLAN
Config2.txt 890 MGMT_VLAN_890
The second method Select -First 1 appears to select only the first match in the file. I tried to change it to Select -First 2 and then CSV shows column Mgmt_Vlan as System.Object[].
The result output to the screen shows exactly four(4) Mgmt Vlans as expected.
$folder = "c:\config_folder"
$files = Get-childitem $folder\*.txt
Function find_management_vlan($Text)
{
$Vlan = #()
foreach($file in files) {
Mgmt_Vlan = Select-String -Path $File -Pattern $Text -AllMatches
if($Mgmt_Vlan) # if there is a match
{
$Vlan += New-Object -PSObject -Property #{'Filename' = $File; 'Mgmt_vlan' = $Mgmt_vlan}
$Vlan | Select 'Filename', 'Mgmt_vlan' | export-csv C:\documents\Mgmt_vlan.csv
$Mgmt_Vlan # test to see if it shows correct matches on screen and yes it did
}
else
{
$Vlan += New-Object -PSObject -Property #{'Filename' = $File; 'Mgmt_vlan' = "Mgmt Vlan Not Found"}
$Vlan | Select 'Filename', 'Mgmt_vlan' | Export-CSV C:\Documents\Mgmt_vlan.csv
}
}
}
find_management_vlan "^\d{1,3}\s.MGMT_"
Regex correction
First of all, there are a lot of mistakes in this code.
So this is probably not code that you actually used.
Secondly, that pattern will not match your strings, because if you use "^\d{1,3}\s.MGMT_" you will match 1-3 numbers, any whitespace character (equal to [\r\n\t\f\v ]), any character (except for line terminators) and MGMT_ chars and anything after that. So not really what you want. So in your case you can use for example this: ^\d{1,3}\sMGMT_ or with \s+ for more than one match.
Code Correction
Now back to your code... You create array $Vlan, that's ok.
After that, you tried to get all strings (in your case 2 strings from every file in your directory) and you create PSObject with two complex objects. One is FileInfo from System.IO and second one is an array of strings (String[]) from System. Inside the Export-Csv function .ToString() is called on every property of the object being processed. If you call .ToString() on an array (i.e. Mgmt_vlan) you will get "System.Object[]", as per default implementation. So you must have a collection of "flat" objects if you want to make a csv from it.
Second big mistake is creating a function with more than one responsibility. In your case your function is responsible for gathering data and after that for exporting data. That's a big no no. So repair your code and move that Export somewhere else. You can use for example something like this (i used get-content, because I like it more, but you can use whatever you want to get your string collection.
function Get-ManagementVlans($pattern, $files)
{
$Vlans = #()
foreach ($file in $files)
{
$matches = (Get-Content $file.FullName -Encoding UTF8).Where({$_ -imatch $pattern})
if ($matches)
{
$Vlans += $matches | % { New-Object -TypeName PSObject -Property #{'Filename' = $File; 'Mgmt_vlan' = $_.Trim()} }
}
else
{
$Vlans += New-Object -TypeName PSObject -Property #{'Filename' = $File; 'Mgmt_vlan' = "Mgmt Vlan Not Found"}
}
}
return $Vlans
}
function Export-ManagementVlans($path, $data)
{
#do something...
$data | Select Filename,Mgmt_vlan | Export-Csv "$path\Mgmt_vlan.csv" -Encoding UTF8 -NoTypeInformation
}
$folder = "C:\temp\soHelp"
$files = dir "$folder\*.txt"
$Vlans = Get-ManagementVlans -pattern "^\d{1,3}\sMGMT_" -files $files
$Vlans
Export-ManagementVlans -path $folder -data $Vlans```
Summary
But in my opinion in this case is overprogramming to create something like you did. You can easily do it in oneliner (but you didn't have information if the file doesn't include anything). The power of powershell is this:
$pattern = "^\d{1,3}\s+MGMT_"
$path = "C:\temp\soHelp\"
dir $path -Filter *.txt -File | Get-Content -Encoding UTF8 | ? {$_ -imatch $pattern} | select #{l="FileName";e={$_.PSChildName}},#{l="Mgmt_vlan";e={$_}} | Export-Csv -Path "$path\Report.csv" -Encoding UTF8 -NoTypeInformation
or with Select-String:
dir $path -Filter *.txt -File | Select-String -Pattern $pattern -AllMatches | select FileName,#{l="Mgmt_vlan";e={$_.Line}} | Export-Csv -Path "$path\Report.csv" -Encoding UTF8 -NoTypeInformation

Using "notin" with matching groups

Using powershell, I am trying to determine which perl scripts in a directory are not called from any other script. In my Select-String I am grouping the matches because there is some other logic I use to filter out results where the line is commented, and a bunch of other scenarios I want to exclude(for simplicity I excluded that from the code posted below). My main problem is in the "-notin" part.
I can get this to work if I remove the grouping from Select-string and only match the filename itself. So this works.
$searchlocation = "C:\Temp\"
$allresults = Select-String -Path "$searchlocation*.pl" -Pattern '\w+\.pl'
$allperlfiles = Get-Childitem -Path "$searchlocation*.pl"
$allperlfiles | foreach-object -process{
$_ | where {$_.name -notin $allresults.matches.value} | Select -expandproperty name | Write-Host
}
However I cannot get the following to work. The only difference between this and above is the value for the "-Pattern" and the value after "-notin". I'm not sure how to use "notin" along with matching groups.
$searchlocation = "C:\Temp\"
$allresults = Select-String -Path "$searchlocation*.pl" -Pattern '(.*?)(\w+\.pl)'
$allperlfiles = Get-Childitem -Path "$searchlocation*.pl"
$allperlfiles | foreach-object -process{
$_ | where {$_.name -notin $allresults.matches.groups[2].value} | Select -expandproperty name | Write-Host}
At a high level the code should search all perl scripts in a directory for any lines that execute any other perl script. With that I now have $allresults which basically gives me a list of all perl scripts called from other files. To get the inverse of that(files that are NOT called from any other file) I get a list of all perl scripts in the directory, cycle through those and list out the ones that DONT show up in $allresults.
When you select a grouping you need to do so using a Select statement, or iteratively in a loop, otherwise you are only going to select the value from the Nth match.
IE if your $Allresults object contains
File.pl, File 2.pl, File 3.pl
Then $allresults.Matches.Groups[2].value Only Returns File2.pl
Instead, you need to select those values!
$allresults | select #{N="Match";E={ $($_.Matches.Groups[2].value) } }
Which will return:
Match
-----
File1.pl
File2.pl
File3.pl
In your specific example, each match has three sub-items, the results will be completely sequential, so what you would term "match 1, group 1" is groups[0] while "match 2, group 1" is groups[3]
This means the matches you care about (those with grouping 2) are in the array values contained in the set {2,5,8,11,...,etc.} or can be described as (N*3-1) Where N is the number of the match. So For Match 1 = (1*3)-1 = [2]; while For Match 13 = (13*3)-1 = [38]
You can iterate through them using a loop to check:
for($i=0; $i -le ($allresults.Matches.groups.count-1); $i++){
"Group[$i] = ""$($allresults.Matches.Groups[$i].value)"""
}
I noticed that you took the time to avoid loops in collecting your data, but then accidentally seem to have fallen prey to using one in matching your data.
Not-In and other compares when used by the select and where clauses don't need a loop structure and are faster if not looped, so you can forego the Foreach-object loop and have a better process just by using a simple Where (?).
$SearchLocation = "C:\Temp\"
$FileGlob = "*.pl"
$allresults = Select-String -Path "$SearchLocation$FileGlob" -Pattern '(.*?)([\w\.]+\.bat)'
$allperlfiles = Get-Childitem -Path "$SearchLocation$FileGlob"
$allperlfiles | ? {
$_.name -notin $(
$allresults | select #{N="Match";E={ $($_.Matches.Groups[2].value) } }
)
} | Select -expandproperty name | Write-Host
Now, that should be faster and simpler code to maintain, but, as you may have noticed, it still has some redundancies now that you are not looping.
As you are piping it all into a Select which can do the work of the where, and what's more you only are looking to match the NAME property here so you can either for-go the last select by only piping the name of the file in the first place, or you can forgo the where and select exactly what you want.
I think the former is far simpler, and the latter is useful if you are going to actually do something with those other values inside the loop that we don't know yet.
Finally, Write-host is likely redundant as any object output will echo to the console.
Here is that version which incorporates the removal of the unnecessary loops and removes redundancies related to the output of the info you wanted, all together.
$SearchLocation = "C:\Temp\"
$FileGlob = "*.pl"
$allresults = Select-String -Path "$SearchLocation$FileGlob" -Pattern ('(.*?)([\w\.]+\'+$FileGlob+')')
$allperlfiles = Get-Childitem -Path "$SearchLocation$FileGlob"
$allperlfiles.name | ? {
$_ -notin $(
$allresults | select #{
N="Match";E={
$($_.Matches.Groups[2].value)
}
}
)
}

Powershell: unexpected behavior of negated -like and -match conditionals

I have 2 folders in my windows folder, software, and softwaretest.
So I have the main folder "software" if statement, then jump to the elseif - here I have the backup folder, so jump to the else...
my problem is that I'm getting the write-host from the elseif, and I have a backup folder that I'm calling softwaretest, so can't see why it give me that output and not the else.
hope someone can guide/help me :-)
If ($SoftwarePathBackup = Get-ChildItem -Path "$Env:SystemRoot" | Where-Object { (!$_.Name -like 'software') }) {
Write-Host ( 'There are no folder named \software\ on this machine - You cant clean/clear/empty the folder!' ) -ForegroundColor Red;
} elseif ($SoftwarePathBackup = Get-ChildItem -Path "$Env:SystemRoot" | Where-Object { ($_.Name -match '.+software$|^software.+') } | Sort-Object) {
Write-Host ( 'There are none folder-backups of \software\ on this machine - You need to make a folder-backup of \software\ before you can clean/clear/empty the folder!' ) -ForegroundColor Red;
} else {
Remove-Item
}
I find it very confusing, to have the negation on the right or even in the RegEx. I think it would be more obvious, to negate in the beginning with a ! or -not.
To test, if a folder exist, you can use Test-Path. Test-Path also has a -Filter parameter, which you can use instead of Where-Object. But I think you don't even have to filter.
$SoftwarePath = "$($Env:SystemRoot)\Software", "$($Env:SystemRoot)\SoftwareBackup"
foreach ($Path in $SoftwarePath) {
if (Test-Path -Path $Path) {
Remove-Item -Path $Path -Force -Verbose
}
else {
Write-Output "$Path not found."
}
}
Would that work for you?
Your primary problem is one of operator precedence:
!$_.Name -like 'software' should be ! ($_.Name -like 'software') or, preferably,
$_.Name -notlike 'software' - using PowerShell's not-prefixed operators for negation.
Similarly, you probably meant to negate $_.Name -match '.+software$|^software.+' which is most easily achieved with $_.Name -notmatch '.+software$|^software.+'
As stated in Get-Help about_Operator_Precedence, ! (a.k.a. -not) has higher precedence than -like, so !$_.Name -like 'software' is evaluated as (!$_.Name) -like 'software', which means that the result of !$_.Name - a Boolean - is (string-)compared to wildcard pattern 'software', which always returns $False, so the If branch is never entered.
That said, you can make do without -like and -match altogether and use the implicit wildcard matching supported by Get-Item's -Include parameter (snippet requires PSv3+):
# Get folders whose name either starts with or ends with 'software', including
# just 'software' itself.
$folders = Get-Item -Path $env:SystemRoot\* -Include 'software*', '*software' |
Where-Object PSIsContainer
# See if a folder named exactly 'software' is among the matches.
$haveOriginal = $folders.Name -contains 'software'
# See if there are backup folders among the matches (too).
# Note that [int] $haveOriginal evaluates to 1 if $haveOriginal is $True,
# and to 0 otherwise.
$haveBackups = ($folders.Count - [int] $haveOriginal) -gt 0
# Now act on $folders as desired, based on flags $haveOriginal and $haveBackups.
Note how Get-Item -Path $env:SystemRoot\* is used to explicitly preselect all items (add -Force if hidden items should be included too), which are then filtered down via -Include.
Since Get-Item - unlike Get-ChildItem- doesn't support -Directory, | Where-Object PSIsContainer is used to further limit the matches to directories (folders).
Note: Get-ChildItem was not used, because -Include only takes effect on child (descendant) items (too) when -Recurse is also specified; while -Recurse can be combined with -Depth 0 (PSv3+) in order to limit matching to immediate child directories, Get-ChildItem apparently still tries to read the entries of all child directories as well, which can result in unwanted access-denied errors from directories that aren't even of interest.
In other words: Get-ChildItem -Recurse -Depth 0 -Directory $env:SystemRoot -include 'software*', '*software' is only equivalent if you have (at least) read access to all child directories of $env:SystemRoot.

Powershell Exclusion of Folders with matching filenames

I'm using something like the below script to exclude Folders.
The problem is that there is a file name as of a folder name and i only want to exclude the folder and not the file. For example in the below i want to exclude "B" Folder only and not "B.txt" file whereas the current code as shown below excludes both file and folder.
$exclude = #("*.cer")
$excludeMatch = #("Member", "A", "B" , "C" , "D")
[regex] $excludeMatchRegEx = ‘(?i)‘ + (($excludeMatch |foreach {[regex]::escape($_)}) –join “|”) + ‘’
Get-ChildItem -Path $source -Recurse -Exclude $exclude |
where { $excludeMatch -eq $null -or `enter code here`$_.FullName.Replace($Source, "") -notmatch $excludeMatchRegEx} |
Copy-Item -Destination {
if ($_.PSIsContainer) {
Join-Path $Dest `enter code here`$_.Parent.FullName.Substring($source.length)
} else {
Join-Path $Dest $_.FullName.Substring($source.length)
}
} -Force -Exclude $exclude
If all you want to do is exclude folders, it's pretty easy to do:
Get-ChildItem -Path \Temp -Recurse | Where-Object {$_.PSIsContainer -eq $false}
If you want to compound this using a regex, you could also add that to your Where-Object statement:
Get-ChildItem -Path \Temp -Recurse | Where-Object {$_.PSIsContainer -eq $false -and $_.Name -match $regexPattern }
You also throw a -notmatch in there if you feel you DON'T want certain things.
One thing to keep in mind with PowerShell... In my experience, the pipeline is GREAT for executing commands from the shell, but that's generally when you're doing things that are very well-defined in your personal command dictionary (things like Get-ADUser -Filter {GivenName -eq "Sam"}, but it isn't as good when you're trying to do script-y sort of things -- in that case, you're really best doing filtering with Where-Object, and selecting down to the items that you need. PowerShell has a pretty reasonably featured debugger for if you want a good debugging experience, and you can't really step into the pipeline as it stands today to evaluate how something works. Especially if you're having problems with a script or a series of commands, the debugger can be your absolute best friend, and I'd highly recommend breaking things out into individual statements to do some analyses there.
Also, the Get-Member cmdlet is quite arguably one of the top-ten most useful cmdlets in Windows PowerShell (along with Get-Help, Where-Object, Select-Object, Get-Command, and a few others). It really helps when you're starting to evaluate how a script is going to function to analyze the properties of the objects you're working with (in your case, System.IO.FileInfo and System.IO.DirectoryInfo), to help reduce the amount of scratching your head later.
I hope this helps!
Edit:
After reading the comments, I am amending my answer to better fit the problem description.
If you do not wish to preserve the source-folder hierarchy, then you can just run Get-ChildItem -Recurse | Where-Object { $_.PSIsContainer -eq $false -and $_.Name -match $regexPattern } | Copy-Item -Destination **target_folder**. Copy-Item should simply treat the output as all belonging to one location. Note that you can also run Copy-Item (and most other cmdlets that modify something support the -WhatIf parameter. You'll want to verify the output before running it.