Im having some trouble counting the lines of code in all my powershell projects.
I want to ignore comment sections in my count, unfortunately I am not that good with regular expressions.
So what I want to achieve is to exclude all the "Synopsis help code in functions"
<#
.SYNOPSIS
#>
And my own comment blocks
<#
Get-ADUser -Identity ThisUserDoesNotExist
ThisCodeIsCommentedOut
#>
What I have so far is
Get-Content Script.ps1 | ?{$_ -ne "" -and $_ -notlike "#*"}
If you are in v3.0 I suggest to use this script: http://poshcode.org/4789
Here the relevant part modified just to count lines of code of a script file:
$file = ".\My_Script_File.ps1"
$fileContentsArray = Get-Content -Path $file
if ($fileContentsArray)
{
$codeLines = $null
$tokenAst = $null
$parseErrorsAst = $null
# Use the PowerShell 3 file parser to create the scriptblock AST, tokens and error collections
$scriptBlockAst = [System.Management.Automation.Language.Parser]::ParseFile($file, [ref]$tokenAst, [ref]$parseErrorsAst)
# Calculate the 'lines of code': any line not containing comment or commentblock and not an empty or whitespace line.
# Remove comment tokens from the tokenAst, remove all double newlines and count all the newlines (minus 1)
$prevTokenIsNewline = $false
$codeLines = #($tokenAst | select -ExpandProperty Kind | where { $_ -ne "comment" } | where {
if ($_ -ne "NewLine" -or (!$prevTokenIsNewline))
{
$_
}
$prevTokenIsNewline = ($_ -eq "NewLine")
} | where { $_ -eq "NewLine" }).Length-1
$codeLines
}
Related
I write a Powershell script and regex to search two configs text files to find matches for Management Vlan. For example, each text file has two Management vlan configured as below:
Config1.txt
123 MGMT_123_VLAN
234 MGMT_VLAN_234
Config2.txt
890 MGMT_VLAN_890
125 MGMT_VLAN_USERS
Below is my script. It has several problems.
First, if I ran the script with the $Mgmt_vlan = Select-String -Path $File -Pattern $String -AllMatches then the screen output shows the expected four (4) Mgmt vlan, but in the CSV file output shows as follow
Filename Mgmt_vlan
Config1.txt System.Object[]
Config2.txt System.Object[]
I ran the script the output on the console screen shows exactly four (4) Management vlans that I expected, but in the CSV file it did not. It shows only these vlans
Second, if I ran the script with $Mgmt_vlan = Select-String -Path $File -Pattern $String | Select -First 1
Then the CSV shows as follows:
Filename Mgmt_vlan
Config1.txt 123 MGMT_123_VLAN
Config2.txt 890 MGMT_VLAN_890
The second method Select -First 1 appears to select only the first match in the file. I tried to change it to Select -First 2 and then CSV shows column Mgmt_Vlan as System.Object[].
The result output to the screen shows exactly four(4) Mgmt Vlans as expected.
$folder = "c:\config_folder"
$files = Get-childitem $folder\*.txt
Function find_management_vlan($Text)
{
$Vlan = #()
foreach($file in files) {
Mgmt_Vlan = Select-String -Path $File -Pattern $Text -AllMatches
if($Mgmt_Vlan) # if there is a match
{
$Vlan += New-Object -PSObject -Property #{'Filename' = $File; 'Mgmt_vlan' = $Mgmt_vlan}
$Vlan | Select 'Filename', 'Mgmt_vlan' | export-csv C:\documents\Mgmt_vlan.csv
$Mgmt_Vlan # test to see if it shows correct matches on screen and yes it did
}
else
{
$Vlan += New-Object -PSObject -Property #{'Filename' = $File; 'Mgmt_vlan' = "Mgmt Vlan Not Found"}
$Vlan | Select 'Filename', 'Mgmt_vlan' | Export-CSV C:\Documents\Mgmt_vlan.csv
}
}
}
find_management_vlan "^\d{1,3}\s.MGMT_"
Regex correction
First of all, there are a lot of mistakes in this code.
So this is probably not code that you actually used.
Secondly, that pattern will not match your strings, because if you use "^\d{1,3}\s.MGMT_" you will match 1-3 numbers, any whitespace character (equal to [\r\n\t\f\v ]), any character (except for line terminators) and MGMT_ chars and anything after that. So not really what you want. So in your case you can use for example this: ^\d{1,3}\sMGMT_ or with \s+ for more than one match.
Code Correction
Now back to your code... You create array $Vlan, that's ok.
After that, you tried to get all strings (in your case 2 strings from every file in your directory) and you create PSObject with two complex objects. One is FileInfo from System.IO and second one is an array of strings (String[]) from System. Inside the Export-Csv function .ToString() is called on every property of the object being processed. If you call .ToString() on an array (i.e. Mgmt_vlan) you will get "System.Object[]", as per default implementation. So you must have a collection of "flat" objects if you want to make a csv from it.
Second big mistake is creating a function with more than one responsibility. In your case your function is responsible for gathering data and after that for exporting data. That's a big no no. So repair your code and move that Export somewhere else. You can use for example something like this (i used get-content, because I like it more, but you can use whatever you want to get your string collection.
function Get-ManagementVlans($pattern, $files)
{
$Vlans = #()
foreach ($file in $files)
{
$matches = (Get-Content $file.FullName -Encoding UTF8).Where({$_ -imatch $pattern})
if ($matches)
{
$Vlans += $matches | % { New-Object -TypeName PSObject -Property #{'Filename' = $File; 'Mgmt_vlan' = $_.Trim()} }
}
else
{
$Vlans += New-Object -TypeName PSObject -Property #{'Filename' = $File; 'Mgmt_vlan' = "Mgmt Vlan Not Found"}
}
}
return $Vlans
}
function Export-ManagementVlans($path, $data)
{
#do something...
$data | Select Filename,Mgmt_vlan | Export-Csv "$path\Mgmt_vlan.csv" -Encoding UTF8 -NoTypeInformation
}
$folder = "C:\temp\soHelp"
$files = dir "$folder\*.txt"
$Vlans = Get-ManagementVlans -pattern "^\d{1,3}\sMGMT_" -files $files
$Vlans
Export-ManagementVlans -path $folder -data $Vlans```
Summary
But in my opinion in this case is overprogramming to create something like you did. You can easily do it in oneliner (but you didn't have information if the file doesn't include anything). The power of powershell is this:
$pattern = "^\d{1,3}\s+MGMT_"
$path = "C:\temp\soHelp\"
dir $path -Filter *.txt -File | Get-Content -Encoding UTF8 | ? {$_ -imatch $pattern} | select #{l="FileName";e={$_.PSChildName}},#{l="Mgmt_vlan";e={$_}} | Export-Csv -Path "$path\Report.csv" -Encoding UTF8 -NoTypeInformation
or with Select-String:
dir $path -Filter *.txt -File | Select-String -Pattern $pattern -AllMatches | select FileName,#{l="Mgmt_vlan";e={$_.Line}} | Export-Csv -Path "$path\Report.csv" -Encoding UTF8 -NoTypeInformation
I am simply trying to create a powershell script that will change number values in a set of text files. The data in the text files are separated by semi-colons. The values I want to change are always the 2nd and 3rd tokens on each line of the text file.
An example of a line in one of the files:
"Bridge_Asphalt_F";202498.396728;1104.362183;9.721280;0.000000;0.000000;1.000000;-1.299559;
I want to allow the user of the script to enter values to be added to(or subtracted from) the 2nd and 3rd values in all the lines of all the text files in the current directory.
I have a very basic understanding of scripting, but I've been searching around for hours trying to wrap my head around how this would be accomplished.
This is what I have so far but I'm sure I'm getting a few things wrong:
$east = Read-Host 'Easting?'
$north = Read-Host 'Northing?'
Get-ChildItem *.txt |
Foreach-Object {
$c = ($_ | Get-Content)
$c = $c -replace $regexB,$regexB+$east
$c = $c -replace $regexC,$regexC+$north
[IO.File]::WriteAllText($_.FullName, ($c -join "`r`n"))
}
The values determine an object's location on a map (for a game) and I want to be able to move all objects on the entire map by a certain distance on both x and y axis.
Assuming that each line in the file has the same format as your example, then you can treat the file as a CSV and update it like this:
$offset2 = 100
$offset3 = 100
Import-Csv .\data.txt -Delimiter ';' -Header (1 .. 9) |
ForEach-Object {
$_.2 = ([double]$_.2) + $offset2
$_.3 = ([double]$_.3) + $offset3
$_
} | ConvertTo-Csv -NoTypeInformation -Delimiter ';' |
Select-Object -Skip 1 |
Add-Content .\updated.txt
Note:
ConvertTo-Csv surrounds each item with quotes, so you end up with something like this:
"Bridge_Asphalt_F";"202198.396728";"1104.362183";"9.721280";"0.000000";"0.000000";"1.000000";"-1.299559"
This may cause problems if this isn't expected by your game. If so, then some more processing on the pipeline could be done to strip it out.
Also, I've had issues in the past with trying to import and export to the same CSV file, hence my code outputs to a different file. Test it yourself and if it works with the same file, great, otherwise, copy my example, then add a line to replace the existing file with the new one (e.g. using Move-Item).
I guess that's what you need:
cls
cd C:\Users\dandraka\Desktop\test #or whereever
$eastStr = Read-Host 'Easting?'
$northStr = Read-Host 'Northing?'
# convert input to number
$east = [decimal]::Parse($eastStr)
$north = [decimal]::Parse($northStr)
# loop through files
$files = Get-ChildItem *.txt
$files | Foreach-Object {
$fileName = $_.FullName # just for clarity
Write-Host $fileName
$newLines = New-Object System.Collections.ArrayList
# loop through lines of each file
$lines = Get-Content -Path $fileName
$lines | ForEach-Object {
$line = $_.ToString() # just for clarity
$lineItems = $line -split ';'
$pointName = $lineItems[0]
$latitudeStr = $lineItems[1]
$longitudeStr = $lineItems[2]
# convert to number
$latitude = [decimal]::Parse($latitudeStr)
$longitude = [decimal]::Parse($longitudeStr)
Write-Host "$pointName latitude $latitude , longitude $longitude"
# do the math
$newLatitude = $latitude + $north
$newLongitude = $longitude + $east
Write-Host "$pointName new latitude $newLatitude , new longitude $newLongitude"
# recontruct the line
$newLine = ""
for($i=0; $i -lt $lineItems.Count; $i++) {
if ($i -eq 1) {
$newLine += "$newLatitude;"
continue
}
if ($i -eq 2) {
$newLine += "$newLongitude;"
continue
}
# this if fixes a small bug, without it there are two ; at the end of each line
if ($lineItems[$i].Length -gt 0) {
$newLine += "$($lineItems[$i]);"
}
}
Write-Host "Old line $line"
Write-Host "New line $newLine"
$newLines.Add($newLine) | Out-Null
}
# write file
$newFilename = $fileName.Replace(".txt", ".dat")
[System.IO.File]::WriteAllLines($newFilename, $newLines)
Write-Host "File $newFilename written"
}
A few things to note here:
As you mention that you're starting with powershell, I've written the code more verbose than I would for, say, a seasoned developer. But that actually doesn't hurt.
For the same reason, the code is sub-optimal on purpose (makes for easier to read code). But for better performance and large files (say, a few 10s of MB or more) you need to do things differently, e.g. avoid strings and use string builder instead.
Obviously you can comment out all the Write-Host statements, they're there just to help you make sure the code is working properly.
Hope that helps!
Jim
If your game cannot handle the quoted coordinate values you get when using ConvertTo-Csv or Export-Csv, this should update the values while leaving the quotes off:
$eastOffset = 100
$northOffset = -200
(Get-Content 'D:\coordinates.txt') | ForEach-Object {
$fields = $_ -split ';'
[double]$fields[1] += $eastOffset
[double]$fields[2] += $northOffset
# write the updated stuff to file
Add-Content -Path 'D:\newcoordinates.txt' -Value ($fields -join ';')
}
this content
"Bridge_Asphalt_F";202498.396728;1104.362183;9.721280;0.000000;0.000000;1.000000;-1.299559;
"Road_Asphalt_F";202123.396728;1104.362456;9.721280;0.000000;0.000000;1.000000;-1.299559;
would become
"Bridge_Asphalt_F";202598.396728;904.362183;9.721280;0.000000;0.000000;1.000000;-1.299559;
"Road_Asphalt_F";202223.396728;904.362456;9.721280;0.000000;0.000000;1.000000;-1.299559;
I am working on a powershell script and I've got several text files where I need to replace backslashes in lines which matches this pattern: .. >\\%name% .. < .. (.. could be anything)
Example string from one of the files where the backslashes should match:
<Tag>\\%name%\TST$\Program\1.0\000\Program.msi</Tag>
Example string from one of the files where the backslashes should not match:
<Tag>/i /L*V "%TST%\filename.log" /quiet /norestart</Tag>
So far I've managed to select every char between >\\%name% and < with this expression (Regex101):
(?<=>\\\\%name%)(.*)(?=<)
but I failed to select only the backslashes.
Is there a solution which I could not yet find?
I'd recommend selecting the relevant tags with an XPath expression and then do the replacement on the text body of the selected nodes.
$xml.SelectNodes('//Tag[substring(., 1, 8) = "\\%name%"]' | ForEach-Object {
$_.'#text' = $_.'#text' -replace '\\', '\\'
}
So here's my solution:
$original_file = $Filepath
$destination_file = $Filepath + ".new"
Get-Content -Path $original_file | ForEach-Object {
$line = $_
if ($line -match '(?<=>\\\\%name%)(.*)(?=<)'){
$line = $line -replace '\\','/'
}
$line
} | Set-Content -Path $destination_file
Remove-Item $original_file
Rename-Item $destination_file.ToString() $original_file.ToString()
So this will replace every \ with an / in the given pattern but not in the way which my question was about.
Is there any easy way to do this?
input: 123215-85_01_test
expected output: 01_test
Another example
input: 12154_02_test
expected output: 02_test
There will be always string "test", but different numbering before
for example this code..
$path = "c:\tmp\*.sql"
get-childitem $path | forEach-object {
$name = $_.Name
$result = $name -replace "","" # I don't know how write this regex..
$extension = $_.Extension
$newName = $prefix+"_"+ $result -f, $extension
Rename-Item -Path $_.FullName -NewName $newName
}
There are two ways you go go at this. Simple split and join or you can use one of many regexes....
Split on underscore and rejoin last 2 elements
$split = "123215-85_01_test" -split "_"
$split[-2..-1] -join "_" # $split[-2,-1] would also work.
Regex to locate the data between the last underscores
"123215-85_01_test" -replace "^.*_(\d+)_(.*)$", '$1_$2'
Note this fails if there is more than 2 underscores.
I am having some issues trying to match a certain config block (multiple ones) from a file. Below is the block that I'm trying to extract from the config file:
ap71xx 00-01-23-45-67-89
use profile PROFILE
use rf-domain DOMAIN
hostname ACCESSPOINT
area inside
!
There are multiple ones just like this, each with a different MAC address. How do I match a config block across multiple lines?
The first problem you may run into is that in order to match across multiple lines, you need to process the file's contents as a single string rather than by individual line. For example, if you use Get-Content to read the contents of the file then by default it will give you an array of strings - one element for each line. To match across lines you want the file in a single string (and hope the file isn't too huge). You can do this like so:
$fileContent = [io.file]::ReadAllText("C:\file.txt")
Or in PowerShell 3.0 you can use Get-Content with the -Raw parameter:
$fileContent = Get-Content c:\file.txt -Raw
Then you need to specify a regex option to match across line terminators i.e.
SingleLine mode (. matches any char including line feed), as well as
Multiline mode (^ and $ match embedded line terminators), e.g.
(?smi) - note the "i" is to ignore case
e.g.:
C:\> $fileContent | Select-String '(?smi)([0-9a-f]{2}(-|\s*$)){6}.*?!' -AllMatches |
Foreach {$_.Matches} | Foreach {$_.Value}
00-01-23-45-67-89
use profile PROFILE
use rf-domain DOMAIN
hostname ACCESSPOINT
area inside
!
00-01-23-45-67-89
use profile PROFILE
use rf-domain DOMAIN
hostname ACCESSPOINT
area inside
!
Use the Select-String cmdlet to do the search because you can specify -AllMatches and it will output all matches whereas the -match operator stops after the first match. Makes sense because it is a Boolean operator that just needs to determine if there is a match.
In case this may still be of value to someone and depending on the actual requirement, the regex in Keith's answer doesn't need to be that complicated. If the user simply wants to output each block the following will suffice:
$fileContent = [io.file]::ReadAllText("c:\file.txt")
$fileContent |
Select-String '(?smi)ap71xx[^!]+!' -AllMatches |
%{ $_.Matches } |
%{ $_.Value }
The regex ap71xx[^!]*! will perform better and the use of .* in a regular expression is not recommended because it can generate unexpected results. The pattern [^!]+! will match any character except the exclamation mark, followed by the exclamation mark.
If the start of the block isn't required in the output, the updated script is:
$fileContent |
Select-String '(?smi)ap71xx([^!]+!)' -AllMatches |
%{ $_.Matches } |
%{ $_.Groups[1] } |
%{ $_.Value }
Groups[0] contains the whole matched string, Groups[1] will contain the string match within the parentheses in the regex.
If $fileContent isn't required for any further processing, the variable can be eliminated:
[io.file]::ReadAllText("c:\file.txt") |
Select-String '(?smi)ap71xx([^!]+!)' -AllMatches |
%{ $_.Matches } |
%{ $_.Groups[1] } |
%{ $_.Value }
This regex will search for the text ap followed by any number of characters and new lines ending with a !:
(?si)(a).+?\!{1}
So I was a little bored. I wrote a script that will break up the text file as you described (as long as it only contains the lines you displayed). It might work with other random lines, as long as they don't contain the key words: ap, profile, domain, hostname, or area. It will import them, and check line by line for each of the properties (MAC, Profile, domain, hostname, area) and place them into an object that can be used later. I know this isn't what you asked for, but since I spent time working on it, hopefully it can be used for some good. Here is the script if anyone is interested. It will need to be tweaked to your specific needs:
$Lines = Get-Content "c:\test\test.txt"
$varObjs = #()
for ($num = 0; $num -lt $lines.Count; $num =$varLast ) {
#Checks to make sure the line isn't blank or a !. If it is, it skips to next line
if ($Lines[$num] -match "!") {
$varLast++
continue
}
if (([regex]::Match($Lines[$num],"^\s.*$")).success) {
$varLast++
continue
}
$Index = [array]::IndexOf($lines, $lines[$num])
$b=0
$varObj = New-Object System.Object
while ($Lines[$num + $b] -notmatch "!" ) {
#Checks line by line to see what it matches, adds to the $varObj when it finds what it wants.
if ($Lines[$num + $b] -match "ap") { $varObj | Add-Member -MemberType NoteProperty -Name Mac -Value $([regex]::Split($lines[$num + $b],"\s"))[1] }
if ($lines[$num + $b] -match "profile") { $varObj | Add-Member -MemberType NoteProperty -Name Profile -Value $([regex]::Split($lines[$num + $b],"\s"))[3] }
if ($Lines[$num + $b] -match "domain") { $varObj | Add-Member -MemberType NoteProperty -Name rf-domain -Value $([regex]::Split($lines[$num + $b],"\s"))[3] }
if ($Lines[$num + $b] -match "hostname") { $varObj | Add-Member -MemberType NoteProperty -Name hostname -Value $([regex]::Split($lines[$num + $b],"\s"))[2] }
if ($Lines[$num + $b] -match "area") { $varObj | Add-Member -MemberType NoteProperty -Name area -Value $([regex]::Split($lines[$num + $b],"\s"))[2] }
$b ++
} #end While
#Adds the $varObj to $varObjs for future use
$varObjs += $varObj
$varLast = ($b + $Index) + 2
}#End for ($num = 0; $num -lt $lines.Count; $num = $varLast)
#displays the $varObjs
$varObjs
To me, a very clean and simple approach is to use a multiline bloc regex, with named captures, like this:
# Based on this text configuration:
$configurationText = #"
ap71xx 00-01-23-45-67-89
use profile PROFILE
use rf-domain DOMAIN
hostname ACCESSPOINT
area inside
!
"#
# We can build a multiline regex bloc with the strings to be captured.
# Here, i am using the regex '.*?' than roughly means 'capture anything, as less as possible'
# A more specific regex can be defined for each field to capture.
# ( ) in the regex if for defining a group
# ?<> is for naming a group
$regex = #"
(?<userId>.*?) (?<userCode>.*?)
use profile (?<userProfile>.*?)
use rf-domain (?<userDomain>.*?)
hostname (?<hostname>.*?)
area (?<area>.*?)
!
"#
# Lets see if this matches !
if($configurationText -match $regex)
{
# it does !
Write-Host "Config text is successfully matched, here are the matches:"
$Matches
}
else
{
Write-Host "Config text could not be matched."
}
This script outputs the following:
PS C:\Users\xdelecroix> C:\FusionInvest\powershell\regex-capture-multiline-stackoverflow.ps1
Config text is successfully matched, here are the matches:
Name Value
---- -----
hostname ACCESSPOINT
userProfile PROFILE
userCode 00-01-23-45-67-89
area inside
userId ap71xx
userDomain DOMAIN
0 ap71xx 00-01-23-45-67-89...
For more flexibility, you can use Select-String instead of -match, but this is not really important here, in the context of this sample.
Here's my take. If you don't need the regex, you can use -like or .contains(). The question never says what the search pattern is. Here's an example with a windows text file.
$file = (get-content -raw file.txt) -replace "`r" # avoid the line ending issue
$pattern = 'two
three
f.*' -replace "`r"
# just showing what they really are
$file -replace "`r",'\r' -replace "`n",'\n'
$pattern -replace "`r",'\r' -replace "`n",'\n'
$file -match $pattern
$file | select-string $pattern -quiet