Powershell sed-like opertaion - regex

Consider the following use case:
I have a file in this format:
some_content
major_version=1
minor_version=33
some_other_content
combined_version=1.33
more_content
I want to write a Powershell script that takes Major/Minor as parameter, and increments the corresponding number in the file. So I tried using the -replace operator. This is what I came up with:
# Let's say $Major and $Minor contains the updated numbers (I mean, I incremented one of them by 1, according to the user selection)
$SearchExpr = '(?s)(?<First>.*major\D+)\d+(?<Second>.*minor\D+)\d+(?<Third>.*combined\D+)\d+\.\d+'
$ReplaceExp = "`${First}${Major}`${Second}${Minor}`${Third}${Major}.${Minor}"
$VersionFileContent -replace $SearchExp, $ReplaceExp | Out-File $VersionFile
But it's pretty nasty. The probelm is, that while replacing a string in a text is easy if you know the text, like:
"Girrafe, Zebra, Dog" -replace 'Dog', 'Cat'
Replacing "Whatever is after 'Zebra' " is less..
Ideas?

You could use switch -Regex -File for this quite easily:
$content = switch -Regex -File 'D:\Test\MyFile.txt' {
'^(major|minor)_version=(\d+)' {
'{0}_version={1}' -f $matches[1], ([int]$matches[2] + 1)
}
default { $_ }
}
# for safety, save to a new file
$content | Set-Content -Path 'D:\Test\MyUpdatedFile.txt' -Force
Result:
some_content
major_version=2
minor_version=34
some_other_content
combined_version=1.33
more_content
If you also need to update the combined_version, extend to:
$content = switch -Regex -File 'D:\Test\MyFile.txt' {
'^(major_version)=(\d+)' {
$major = [int]$matches[2] + 1
'{0}={1}' -f $matches[1], $major
}
'^(minor_version)=(\d+)' {
$minor = [int]$matches[2] + 1
'{0}={1}' -f $matches[1], $minor
}
'^(combined_version)=(\d+)' {
'{0}={1}.{2}' -f $matches[1], $major, $minor
}
default { $_ }
}
$content | Set-Content -Path 'D:\Test\MyUpdatedFile.txt' -Force

Related

Powershell use ForEach to match and replace string with regex and replace with incremental value

I have to replace multiple strings with the same pattern, and several strings are on the same line. The replacement value should be incremental. I need to match and replace only the pattern as in the example, not requesId, nor messageId.
Input:
<requestId>qwerty-qwer12-qwer56</requestId>Ace of Base Order: Q2we45-Uj87f6-gh65De<something else...
<requestId>zxcvbn-zxcv4d-zxcv56</requestId>
<requestId>1234qw-12qw9x-123456</requestId> Stevie Wonder <messageId>1234qw-12qw9x-123456</msg
reportId>plmkjh8765FGH4rt6As</msg:reportId> something <keyID>qwer1234asdf5678zxcv0987bnml65gh</msgdc
The desired output should be:
<requestId>Request-1</requestId>Ace of Base Order: Request-2<something else...
<requestId>Request-3</requestId>
<requestId>Request-4</requestId> Stevie Wonder <messageId>Request-4</msg
reportId>ReportId-1</msg:reportId> something <keyId>KeyId-1</msg
The regex finds all matching values but I cannot make the loop and replace these values. The code I am trying to make work is:
#'
<requestId>qwerty-qwer12-qwer56</requestId>Ace of Base Order: Q2we45-Uj87f6-gh65De<something else...
<requestId>zxcvbn-zxcv12-zxcv56</requestId>
<requestId>1234qw-12qw12-123456</requestId> Stevie Wonder <messageId>1234qw-12qw12-123456</msg
reportId>plmkjh8765FGH4rt6As</msg:reportId> something <keyID>qwer1234asdf5678zxcv0987bnml65gh</msgdc
'# | Set-Content $log -Encoding UTF8
$requestId = #{
Count = 1
Matches = #()
}
$tmp = Get-Content $log | foreach { $n = [regex]::matches((Get-Content $log),'\w{6}-\w{6}-\w{6}').value
if ($n)
{
$_ -replace "$n", "Request-$($requestId.count)"
$requestId.count++
} $_ }
$tmp | Set-Content $log
You want Regex.Replace():
$requestId = 1
$tmp = Get-Content $log |ForEach-Object {
[regex]::Replace($_, '\w{6}-\w{6}-\w{6}', { 'Request-{0}' -f ($script:requestId++) })
}
$tmp |Set-Content $log
The script block will run once per match to calculate the substitue value, allowing us to resolve and increment the $requestId variable, resulting in the consecutive numbering you need.
You can do this for multiple patterns in succession if necessary, although you may want to use an array or hashtable for the individual counters:
$counters = { requestId = 1; keyId = 1 }
$tmp = Get-Content $log |ForEach-Object {
$_ = [regex]::Replace($_, '\w{6}-\w{6}-\w{6}', { 'Request-{0}' -f ($counters['requestId']++) })
[regex]::Replace($_, '\b\w{32}\b', { 'Key-{0}' -f ($counters['keyId']++) })
}
$tmp |Set-Content $log
If you want to capture and the mapping between the original and the new value, do that inside the substitution block:
$translations = #{}
# ...
[regex]::Replace($_, '\w{6}-\w{6}-\w{6}', {
# capture value we matched
$original = $args[0].Value
# generate new value
$substitute = 'Request-{0}' -f ($counters['requestId']++)
# remember it
$translations[$substitute] = $original
return $substitute
})
In PowerShell 6.1 and newer versions, you can also do this directly with the -replace operator:
$requestId = 0
$tmp = Get-Content $log |ForEach-Object {
$_ -replace '\w{6}-\w{6}-\w{6}', { 'Request-{0}' -f ($requestId++) }
}
$tmp |Set-Content $log

Regular expression in power shell to convert character between double quotes to upper case

I want to write a powershell script which will convert a string which is present between double quotes in a file, and convert it into upper case.
The files are placed in different folders.
I am able to extract the string between the double quotes and convert it to upper case, but not able to replace it in the correct position.
Ex : This is the input string.
"e" //&&'i&&
The output should be
"E" //&&'i&&
This is what i have tried. Also this even i not replacing the content of the file.
$items = Get-ChildItem * -recurse
# enumerate the items array
foreach ($item in $items)
{
# if the item is a directory, then process it.
if ($item.Attributes -ne "Directory")
{
(Get-Content $item.FullName ) |
Foreach-Object {
if (($_ -match '\"'))
{
$str = $_
$ext = [regex]::Matches($str, '".*?"').Value -replace '"'
$ext = $ext.ToUpper()
Write-Host $ext
$_ = $ext
}
else { }
} |
Set-Content $item.FullName
}
}
This can do it. Really I wasn't following your code so I stripped it and modified the regex.
$items = Get-ChildItem "C:\Users\UsernameHere\Desktop\Folder123\*.txt"
# enumerate the items array
foreach ($item in $items){
# if the item is a directory, then process it.
if ($item.Attributes -ne "Directory"){
$content = (gc $item.FullName )
$content = $content.replace('"\w.*"',$matches[0].ToUpper)
$content | sc $item
}
}
If you had powershell 6 or 7:
'"hi"' -replace '".*"', { $_.value.toupper() }
"HI"
'"e" //&&''i&&' -replace '".*"', { $_.value.toupper() }
"E" //&&'i&&
I am able to print the upper case characters with the below code, but the file is not getting updated. It still has the old characters, How to update the fie with new contents.
$items = Get-ChildItem *.txt -recurse
# enumerate the items array
foreach ($item in $items)
{
# if the item is a directory, then process it.
if ($item.Attributes -ne "Directory")
{
(Get-Content $item.FullName ) |
Foreach-Object {
$str = $_
$_ = [regex]::Replace($_, '"[^"]*"', { param($m) $m.Value.ToUpper() })
Write-Host $_
} |
Set-Content $item.FullName
}
}

How do I use the values (read from a .ps1 file) to update the values of another .ps1 file

I have a 4.ps1 file that looks like this
#akabradabra
$one = 'o'
#bibi
$two = 't'
$three = 't' #ok thr
#four
$four = 'four'
And a 3.ps1 file that looks like this
#akabradabra
$one = 'one'
#biblibablibo
$two = 'two'
$three = 'three' #ok threer
My goal is to read the key-value pair from 4.ps1 and update the values in 3.ps1 and if new key-value pairs are introduced in 4.ps1, simply append them to the end of 3.ps1.
My idea is to use string functions such as .Split('=') and .Replace(' ', '') to extract the keys and if the keys match, replace the entire line in 3.ps1 with the one found in 4.ps1
I know that using Get-Variable might does the trick and also it will be a lot easier to work with the data if I convert all the key-value pairs into a .xml or a .json file but can anyone please show me how can I make it work in my own silly way?
Here is my code to do so
# Ignore this function, this is used to skip certain key-value pairs
#----------------------------------------------------------------------------
Function NoChange($something) {
switch ($something) {
'$CurrentPath' {return $true}
'$pathToAdmin' {return $true}
'$hostsPathTocompare' {return $true}
'$logs' {return $true}
'$LogFile' {return $true}
default {return $false}
}
}
#----------------------------------------------------------------------------
$ReadFromVARS = Get-Content $PSScriptRoot\4.ps1
$WriteToVARS = Get-Content $PSScriptRoot\3.ps1
foreach ($oldVar in $ReadFromVARS) {
if (('' -eq $oldVar) -or ($oldVar -match '\s*#+\w*')) {
continue
} elseif ((NoChange ($oldVar.Split('=').Replace(' ', '')[0]))) {
continue
} else {
$var = 0
#$flag = $false
while ($var -ne $WriteToVARS.Length) {
if ($WriteToVARS[$var] -eq '') {
$var += 1
continue
} elseif ($WriteToVARS[$var] -match '\s*#+\w*') {
$var += 1
continue
} elseif ($oldVar.Split('=').Replace(' ', '')[0] -eq $WriteToVARS[$var].Split('=').Replace(' ', '')[0]<# -and !$flag#>) {
$oldVar
$WriteToVARS.replace($WriteToVARS[$var], $oldVar) | Set-Content -Path $PSScriptRoot\3.ps1 -Force
break
#$var += 1
#$flag = $true
} elseif (<#!$flag -and #>($var -eq $WriteToVARS.Length)) {
Add-Content -Path $PSScriptRoot\3.ps1 -Value $oldVar -Force
$var += 1
} else {
$var += 1
}
}
}
}
I did not ran into any errors but it only updated one key-value pair ($two = t) and it did not append new key-value pairs at the end. Here is the result I got
#akabradabra
$one = 'one'
#biblibablibo
$two = 't'
$three = 'three' #ok threer
If I understand your question correctly, I think Dot-Sourcing is what you're after.
The PowerShell dot-source operator brings script files into the current session scope. It is a way to reuse script. All script functions and variables defined in the script file become part of the script it is dot sourced into. It is like copying and pasting text from the script file directly into your script.
To make it visible, use Dot-Sourcing to read in the variables from file 3.ps1, show the variables and their values. Next dot-source file 4.ps1 and show the variables again:
. 'D:\3.ps1'
Write-Host "Values taken from file 3.ps1" -ForegroundColor Yellow
"`$one : $one"
"`$two : $two"
"`$three : $three"
"`$four : $four" # does not exist yet
. 'D:\4.ps1'
Write-Host "Values after dot-sourcing file 4.ps1" -ForegroundColor Yellow
"`$one : $one"
"`$two : $two"
"`$three : $three"
"`$four : $four"
The result is
Values taken from file 3.ps1
$one : one
$two : two
$three : three
$four :
Values after dot-sourcing file 4.ps1
$one : o
$two : t
$three : t
$four : four
If you want to write these variables back to a ps1 script file you can:
'one','two','three','four' | Get-Variable | ForEach-Object {
'${0} = "{1}"' -f $_.Name,$_.Value
} | Set-Content 'D:\5.ps1' -Force
Theo's answer provides a easier way to do the same thing
Also, converting your Config files to JSON or XML will make the job lot more easier too
My original idea was to read both 4.ps1 and 3.ps1 ( these are my config files, I only store variables inside and switch statement to help choosing the correct variables ) then overwrite 3.ps1 with all the difference found but I could not get it working so I created a new 5.ps1 and just simply write everything I need to 5.ps1.
Here is my code if you would like to use it for your own project :-)
The obstacles for me were that I had switch statements and certain $variables that I wanted to ignore (in my actual project) so I used some Regex to avoided it.
$ReadFromVARS = Get-Content $PSScriptRoot\4.ps1
$WriteToVARS = Get-Content $PSScriptRoot\3.ps1
New-Item -ItemType File -Path $PSScriptRoot\5.ps1 -Force
Function NoChange($something) {
switch ($something) {
'$CurrentPath' {return $true}
'$pathToAdmin' {return $true}
'$hostsPathTocompare' {return $true}
'$logs' {return $true}
'$LogFile' {return $true}
default {return $false}
}
}
$listOfOldVars = #()
$switchStatementStart = "^switch(\s)*\(\`$(\w)+\)(\s)*(\n)*\{"
$switchStatementContent = "(\s)*(\n)*(\t)*\'\w+(\.\w+)+\'(\s)*\{(\s)*\`$\w+(\s)*=(\s)*\#\((\s)*\'\w+(\.\w+)+\'(\s)*(,(\s)*\'\w+(\.\w+)+\'(\s)*)*\)\}"
$switchStatementDefault = "(\s)*(\n)*(\t)*Default(\s)*\{\`$\w+(\s)*=(\s)*\#\((\s)*\'\w+(\.\w+)+\'(\s)*(,(\s)*\'\w+(\.\w+)+\'(\s)*)*\)\}\}"
$switchStatementEnd = "(\s)*(\n)*(\t)*\}"
foreach ($oldVar in $ReadFromVARS) {
if (('' -eq $oldVar) -or ($oldVar -match '^#+\w*')) {
continue
} elseif ((NoChange $oldVar.Split('=').Replace(' ', '')[0])) {
continue
} else {
$var = 0
while ($var -ne $WriteToVARS.Length) {
if ($WriteToVARS[$var] -eq '') {
$var += 1
continue
} elseif ($WriteToVARS[$var] -match '^#+\w*') {
$var += 1
continue
} elseif ($oldVar -match $switchStatementStart -or $oldVar -match $switchStatementContent -or $oldVar -match $switchStatementDefault -or $oldVar -match $switchStatementEnd) {
Add-Content -Path "$PSScriptRoot\5.ps1" -Value $oldVar -Force
$listOfOldVars += ($oldVar)
break
} elseif ($oldVar.Split('=').Replace(' ', '')[0] -eq $WriteToVARS[$var].Split('=').Replace(' ', '')[0]) {
Add-Content -Path "$PSScriptRoot\5.ps1" -Value $oldVar -Force
$listOfOldVars += ($oldVar.Remove(0,1).Split('=').Replace(' ', '')[0])
break
} else {
$var += 1
}
}
}
}
foreach ($newVar in $WriteToVARS) {
if ($newVar.StartsWith('#') -or $newVar -eq '') {
continue
} elseif ($newVar -match $switchStatementStart -or $newVar -match $switchStatementContent -or $newVar -match $switchStatementDefault -or $newVar -match $switchStatementEnd) {
} elseif (($newVar.Remove(0,1).Split('=').Replace(' ', '')[0]) -in $listOfOldVars) {
continue
} else {
Add-Content -Path "$PSScriptRoot\5.ps1" -Value $newVar -Force
}
}

How to stream text with powershell and regex match on multiline

I have a text file that an application constantly errors to. I want to monitor this file with Powershell and log every error to another source.
Problem to solve: how do i pass multiline text when we are in -wait? Get-Content is passing arrays of strings.
$File = 'C:\Windows\Temp\test.txt'
$content = Get-Content -Path $file
# get stream of text
Get-Content $file -wait -Tail 0 | ForEach-Object {
if ($_ -match '(<ACVS_T>)((.|\n)*)(<\/ACVS_T>)+'){
write-host 'match found!'
}
}
Example of text junks that get drop:
<ACVS_T>
<ACVS_D>03/01/2017 17:24:03.602</ACVS_D>
<ACVS_TI>bf37ba1c9,iSTAR Server Compone</ACVS_TI>
<ACVS_C>ClusterPort</ACVS_C>
<ACVS_S>SoftwareHouse.NextGen.HardwareInterface.Nantucket.Framework.ClusterPort.HandleErrorState( )
</ACVS_S>
<ACVS_M>
ERROR MESSAGE FROM APP
</ACVS_M>
<ACVS_ST>
</ACVS_ST>
</ACVS_T>
solved it!
$File = 'D:\Program Files (x86)\Tyco\CrossFire\Logging\SystemTrace.Log'
$content = Get-Content -Path $file
# get stream of text
$text = ''
Get-Content $file -wait -Tail 0 | ForEach-Object {
$text +=$_
if ($text -match '(<ACVS_T>)((.|\n)*)(<\/ACVS_T>)+'){
[xml]$XML = "<Root>" + $text + "</Root>"
$text='' #clear it for next one
$XML.Root.ACVS_T | ForEach-Object {
$Obj = '' | Select-Object -Property ACVS_D, ACVS_TI, ACVS_C, ACVS_S, ACVS_M, ACVS_ST
$Obj.ACVS_D = $_.ACVS_D
$Obj.ACVS_ST = $_.ACVS_ST
$Obj.ACVS_C = $_.ACVS_C
$Obj.ACVS_S = $_.ACVS_S
$Obj.ACVS_M = $_.ACVS_M
$Obj.ACVS_ST = $_.ACVS_ST
write-host "`n`n$($Obj.ACVS_M)"
}
}
}

Use Powershell to print out line number of code matching a RegEx

I think we have a bunch of commented out code in our source, and rather than delete it immediately, we've just left it. Now I would like to do some cleanup.
So assuming that I have a good enough RegEx to find comments (the RegEx below is simple and I could expand on it based on our coding standards), how do I take the results of the file that I read up and output the following:
Filename
Line Number
The actual line of code
I think I have the basis of an answer here, but I don't know how to take the file that I've read up and parsed with RegEx and spit it out in this format.
I'm not looking for the perfect solution - I just want to find big blocks of commented out code. By looking at the result and seeing a bunch of files with the same name and sequential line numbers, I should be able to do this.
$Location = "c:\codeishere"
[regex]$Regex = "//.*;" #simple example - Will expand on this...
$Files = get-ChildItem $Location -include *cs -recurse
foreach ($File in $Files) {
$contents = get-Content $File
$Regex.Matches($contents) | WHAT GOES HERE?
}
You could do:
dir c:\codeishere -filter *.cs -recurse | select-string -Pattern '//.*;' | select Line,LineNumber,Filename
gci c:\codeishere *.cs -r | select-string "//.*;"
The select-string cmdlet already does exactly what you're asking for, though the filename displayed is a relative path.
I would go personally even further. I would like to compute number of consecutive following lines. Then print the file name, count of lines and the lines itself. You may sort the result by count of lines (candidates for delete?).
Note that my code doesn't count with empty lines between commented lines, so this part is considered as two blocks of commented code:
// int a = 10;
// int b = 20;
// DoSomething()
// SomethingAgain()
Here is my code.
$Location = "c:\codeishere"
$occurences = get-ChildItem $Location *cs -recurse | select-string '//.*;'
$grouped = $occurences | group FileName
function Compute([Microsoft.PowerShell.Commands.MatchInfo[]]$lines) {
$local:lastLineNum = $null
$local:lastLine = $null
$local:blocks = #()
$local:newBlock = $null
$lines |
% {
if (!$lastLineNum) { # first line
$lastLineNum = -2 # some number so that the following if is $true (-2 and lower)
}
if ($_.LineNumber - $lastLineNum -gt 1) { #new block of commented code
if ($newBlock) { $blocks += $newBlock }
$newBlock = $null
}
else { # two consecutive lines of commented code
if (!$newBlock) {
$newBlock = '' | select File,StartLine,CountOfLines,Lines
$newBlock.File, $newBlock.StartLine, $newBlock.CountOfLines, $newBlock.Lines = $_.Filename,($_.LineNumber-1),2, #($lastLine,$_.Line)
}
else {
$newBlock.CountOfLines += 1
$newBlock.Lines += $_.Line
}
}
$lastLineNum=$_.LineNumber
$lastLine = $_.Line
}
if ($newBlock) { $blocks += $newBlock }
$blocks
}
# foreach GroupInfo objects from group cmdlet
# get Group collection and compute
$result = $grouped | % { Compute $_.Group }
#how to print
$result | % {
write-host "`nFile $($_.File), line $($_.StartLine), count of lines: $($_.CountOfLines)" -foreground Green
$_.Lines | % { write-host $_ }
}
# you may sort it by count of lines:
$result2 = $result | sort CountOfLines -desc
$result2 | % {
write-host "`nFile $($_.File), line $($_.StartLine), count of lines: $($_.CountOfLines)" -foreground Green
$_.Lines | % { write-host $_ }
}
If you have any idea how to improve the code, post it! I have a feeling that I could do it using some standard cmdlets and the code could be shorter..
I would look at doing something like:
dir $location -inc *.cs -rec | `
%{ $file = $_; $n = 0; get-content $_ } | `
%{ $_.FileName = $file; $_.Line = ++$n; $_ } | `
?{ $_ -match $regex } | `
%{ "{0}:{1}: {2}" -f ($_.FileName, $_.Line, $_)}
I.e. add extra properties to the string to specify the filename and line number, which can be carried through the pipeline after the regex match.
(Using ForEach-Object's -begin/-end script blocks should be able to simplify this.)