Issues finding and replacing strings in PowerShell - regex
I'm rather new to PowerShell and I'm trying to write a PowerShell script to convert some statements in VBScript to Microsoft JScript. Here is my code:
$vbs = 'C:\infile.vbs'
$js = 'C:\outfile.js'
(Get-Content $vbs | Set-Content $js)
(Get-Content $js) |
Foreach-Object { $_ -match "Sub " } | Foreach-Object { "$_()`n`{" } | Foreach-Object { $_ -replace "Sub", "function" } | Out-File $js
Foreach-Object { $_ -match "End Sub" } | Foreach-Object { $_ -replace "End Sub", "`}" } | Out-File $js
Foreach-Object { $_ -match "Function " } | Foreach-Object { "$_()`n`{" } | Foreach-Object { $_ -replace "Function", "function" } | Out-File $js
Foreach-Object { $_ -match "End Function" } | Foreach-Object { $_ -replace "End Function", "`}" } | Out-File $js
What I want is for my PowerShell program to take the code from the VBScript input file infile.vbs, convert it, and output it to the JScript output file outfile.js. Here is an example of what I want it to do:
Input file:
Sub HelloWorld
(Code Here)
End Sub
Output File:
function HelloWorld()
{
(Code Here)
}
Something similar would happen with regard to functions. From there, I would tweak the code manually to convert it. When I run my program in PowerShell v5.1, it does not show any errors. However, when I open outfile.js, I see only one line:
False
So really, I have two questions. 1. Why is this happening?2. How can I fix this program so that it behaves how I want it to (as detailed above)?
Thanks,
Gabe
You could also do this with the switch statement. Like so:
$vbs = 'C:\infile.vbs'
$js = 'C:\outfile.js'
Get-Content $vbs | ForEach-Object {
switch -Regex ($_) {
'Sub '{
'function {0}(){1}{2}' -f $_.Remove($_.IndexOf('Sub '),4).Trim(),[Environment]::NewLine,'{'
}
'End Sub'{
'}'
}
'Function ' {
'function {0}(){1}{2}' -f $_.Remove($_.IndexOf('Function '),9).Trim(),[Environment]::NewLine,'{'
}
'End Function' {
'}'
}
default {
$_
}
}
} | Out-File $js
As for question #2 (How can I fix this program [...]?):
Kirill Pashkov's helpful answer offers an elegant solution based on the switch statement.
Note, however, that his solution:
is predicated on Sub <name> / Function <name> statement parts not being on the same line as the matching End Sub / End Function parts - while this is typically the case, it isn't a syntactical requirement; e.g., Sub Foo() WScript.Echo("hi") End Sub - on a single line - works too.
in line with your own solution attempt, blindly appends () to Sub / Function definitions, which won't work with input procedures / functions that already have parameter declarations (e.g., Sub Foo (bar, baz)).
The following solution:
also works with single-line Sub / Function definition
correctly preserves parameter declarations
Get-Content $vbs | ForEach-Object {
$_ -replace '\b(?:sub|function)\s+(\w+)\s*(\(.*?\))', 'function $1$2 {' `
-replace '\bend\s+(?:sub|function)\b', '}'
} | Out-File $js
The above relies heavily on regexes (regular expressions) to transform the input; for specifics on how regex matching results can be referred to in the -replace operator's replacement-string operand, see this answer.
Caveat: There are many other syntax differences between VBScript and JScript that your approach doesn't cover, notably that VBScript has no return statement and instead uses <funcName> = ... to return values from functions.
As for question #1:
However, when I open outfile.js, I see only one line:
False
[...]
1. Why is this happening?
All but the first ForEach-Object cmdlet call run in separate statements, because the initial pipeline ends with the first call to Out-File $js.
The subsequent ForEach-Object calls each start a new pipeline, and since each pipeline ends with Out-File $js, each such pipeline writes to file $js - and thereby overwrites whatever the previous one wrote.
Therefore, it is the last pipeline that determines the ultimate contents of file $js.
A ForEach-Object that starts a pipeline receives no input. However, its associated script block ({...}) is still entered once in this case, with $_ being $null[1]:
The last pipeline starts with Foreach-Object { $_ -match "End Function" }, so its output is the equivalent of $null -match "End Function", which yields $False, because -match with a scalar LHS (a single input object) outputs a Boolean value that indicates whether a match was found or not.
Therefore, given that the middle pipeline segment (Foreach-Object { $_ -replace "End Function", "}" }) is an effective no-op ($False is stringified to 'False', and the -replace operator therefore finds no match to replace and passes the stringified input out unmodified), Out-File $js receives string 'False' and writes just that to output file $js.
Even if you transformed your separate commands into a single pipeline with a single Out-File $js segment at the very end, your command wouldn't work, however:
Given that Get-Content sends the input file's lines through the pipeline one by one, something like $_ -match "Sub " will again produce a Boolean result - indicating whether the line at hand ($_) matched string "Sub " - and pass that on.
While you could turn -match into a filter by making the LHS an array - by enclosing it in the array-subexpression operator #(...); e.g., #($_) -match "Sub " - that would:
pass line that contain substring Sub through as a whole, and
omit lines that don't.
In other words: This wouldn't work as intended, because:
lines that do not contain a matching substring would be omitted from the output, and
the lines that do match are reflected in full in $_ in the next pipeline segment - not just the matched part.
[1] Strictly speaking, $_ will retain whatever value it had in the current scope, but that will only be non-$null if you explicitly assigned a value to $_ - given that $_ is an automatic variable that is normally controlled by PowerShell itself, however, doing so is ill-advised - see this GitHub discussion.
OK there is a few things wrong with this script.
Foreach-Object otherwise known as % is to iterate every item in a pipe.
Example is
#(1..10) | %{ "This is Array Item $_"}
This will out put 10 lines counting the array items. In you current script you are using this where a Where-Object also known as ? should be.
#(1..10) | ?{ $_ -gt 5 }
This will output all numbers greater then 5.
A example of what you are kind of trying to go for is something like
function ConvertTo-JS([string]$InputFilePath,[string]$SaveAs){
Get-Content $InputFilePath |
%{$_ -replace "Sub", "function"} |
%{$_ -replace "End Function", "}"} |
%{$_ -replace "Function", "function"} |
%{$_ -replace "End Function", "}" } |
Out-File $SaveAs
}
ConvertTo-JS -InputFilePath "C:\TEST\TEST.vbs" -SaveAs "C:\TEST\TEST.JS"
This doesnt take into account adding a { at the beginning of a function or adding the () ether. But with the information provided hopefully that puts you on the right track.
Related
How can i replace all lines in a file with a pattern using Powershell?
I have a file with lines that i wish to remove like the following: key="Id" value=123" key="FirstName" value=Name1" key="LastName" value=Name2" <!--key="FirstName" value=Name3" key="LastName" value=Name4"--> key="Address" value=Address1" <!--key="Address" value=Address2" key="FirstName" value=Name1" key="LastName" value=Name2"--> key="ReferenceNo" value=765 have tried the following: ` $values = #('key="FirstName"','key="Lastname"', 'add key="Address"'); $regexValues = [string]::Join('|',$values) $lineprod = Get-Content "D:\test\testfile.txt" | Select-String $regexValues|Select-Object - ExpandProperty Line if ($null -ne $lineprod) { foreach ($value in $lineprod) { $prod = $value.Trim() $contentProd | ForEach-Object {$_ -replace $prod,""} |Set-Content "D:\test\testfile.txt" } } The issue is that only some of the lines get replaced and or removed and some remain. The output should be key="Id" value=123" key="ReferenceNo" value=765 But i seem to get key="Id" value=123" key="ReferenceNo" value=765 <!--key="Address" value=Address2" key="FirstName" value=Name1" key="LastName" value=Name2"--> Any ideas as to why this is happening or changes to the code above ?
Based on your comment, the token 'add key="Address"' should be changed for just 'key="Address"' then the concatenating logic to build your regex looks good. You need to use the -NotMatch switch so it matches anything but those values. Also, Select-String can read files, so, Get-Content can be removed. Note, the use of (...) in this case is important because you're reading and writing to the same file in the same pipeline. Wrapping the statement in parentheses ensure that all output from Select-String is consumed before passing it through the pipeline. Otherwise, you would end up with an empty file. $values = 'key="FirstName"', 'key="Lastname"', 'key="Address"' $regexValues = [string]::Join('|', $values) (Select-String D:\test\testfile.txt -Pattern $regexValues -NotMatch) | ForEach-Object Line | Set-Content D:\test\testfile.txt Outputs: key="Id" value=123" key="ReferenceNo" value=765
Extracting string with regex retrieves 'True 111499'
I have a function, which should retrieve the revision number in a text file. The text file looks like this: componentname1:123456 componentname2:234567 The second part is a number, which represents the subversion revision number. The powershell script reads this file line by line and every line is then processed with this function: function getRevision($line) { $line -match ":(?<revision>[0-9]*)" $result = $Matches['revision'] Write-Host "Found component revision '$($result)'" return $result } When the function getRevision is processed, then it gives the output Found component revision '111499' This function is called by another function like this: $rev = getRevision($Line) # .... someOtherFunction -Revision "$($rev)" In this someOtherFunction, I now get this output: Handle component with revision 'True 111499' Edit I add the basic code of the function someOtherFunction: function someOtherFunction { param( [Parameter(Mandatory=$True)] [String]$Revision ) Write-Host "Handle component with revision '$($Revision)'" } Now, the question occurs: Why is there this True stuff? Where is my mistake?
If that is how your source data is formed i.e colon delimited key value pairs. I think ConvertFrom-StringData removes all of the "complexity" from this issue. It however wants "key=value" so we need to make a small change and, in the end, you get the whole file as a hashtable. If that sample data was in a file called c:\temp\file.txt $revisions = (Get-Content -Raw C:\temp\file.txt) -replace ":","=" | ConvertFrom-StringData $revisions['componentname1'] If there was a risk of bad data you could do some basic filtering by only working with lines that have colons: $data = (Get-Content C:\temp\file.txt | Where-Object{$_ -match ":"}) -replace ":","=" | Out-String | ConvertFrom-StringData $data['componentname1']
The keyword True is from -match operator in first line of your function $line -match ":(?<revision>[0-9]*)" If -match is executed inside the function, its output (binary value) is added automatically to the output of your function and then $result is being added as a second element. Therefore if you check the value of $rev you'll see: PowerShell> $rev True 123456 If you want to get rid of the first element you just need to pipe it to Out-Null: $line -match ":(?<revision>[0-9]*)" | Out-Null Edit: as pointed out by Matt, if $line doesn't match with the pattern, it may return unexpected results (last successful match). To prevent this you could use: if ($line -match ":(?<revision>[0-9]*)") { $result = $Matches['revision'] Write-Host "Found component revision '$($result)'" return $result} else {return $false} In this case Out-Null is not needed as -match inside if condition is not being added to the output.
Regular expression seems not to work in Where-Object cmdlet
I am trying to add quote characters around two fields in a file of comma separated lines. Here is one line of data: 1/22/2018 0:00:00,0000000,001B9706BE,1,21,0,1,0,0,0,0,0,0,0,0,0,0,13,0,1,0,0,0,0,0,0,0,0,0,0 which I would like to become this: 1/22/2018 0:00:00,"0000000","001B9706BE",1,21,0,1,0,0,0,0,0,0,0,0,0,0,13,0,1,0,0,0,0,0,0,0,0,0,0 I began developing my regular expression in a simple PowerShell script, and soon I have the following: $strData = '1/29/2018 0:00:00,0000000,001B9706BE,1,21,0,1,0,0,0,0,0,0,0,0,0,0,13,0,1,0,0,0,0,0,0,0,0,0,0' $strNew = $strData -replace "([^,]*),([^,]*),([^,]*),(.*)",'$1,"$2","$3",$4' $strNew which gives me this output: 1/29/2018 0:00:00,"0000000","001B9706BE",1,21,0,1,0,0,0,0,0,0,0,0,0,0,13,0,1,0,0,0,0,0,0,0,0,0,0 Great! I'm all set. Extend this example to the general case of a file of similar lines of data: Get-Content test_data.csv | Where-Object -FilterScript { $_ -replace "([^,]*),([^,]*),([^,]*),(.*)", '$1,"$2","$3",$4' } This is a listing of test_data.csv: 1/29/2018 0:00:00,0000000,001B9706BE,1,21,0,1,0,0,0,0,0,0,0,0,0,0,13,0,1,0,0,0,0,0,0,0,0,0,0 1/29/2018 0:00:00,104938428,0016C4C483,1,45,0,1,0,0,0,0,0,0,0,0,0,0,35,0,1,0,0,0,0,0,0,0,0,0,0 1/29/2018 0:00:00,104943875,0016C4B0BC,1,31,0,1,0,0,0,0,0,0,0,0,0,0,25,0,1,0,0,0,0,0,0,0,0,0,0 1/29/2018 0:00:00,104948067,0016C4834D,1,33,0,1,0,0,0,0,0,0,0,0,0,0,23,0,1,0,0,0,0,0,0,0,0,0,0 This is the output of my script: 1/29/2018 0:00:00,0000000,001B9706BE,1,21,0,1,0,0,0,0,0,0,0,0,0,0,13,0,1,0,0,0,0,0,0,0,0,0,0 1/29/2018 0:00:00,104938428,0016C4C483,1,45,0,1,0,0,0,0,0,0,0,0,0,0,35,0,1,0,0,0,0,0,0,0,0,0,0 1/29/2018 0:00:00,104943875,0016C4B0BC,1,31,0,1,0,0,0,0,0,0,0,0,0,0,25,0,1,0,0,0,0,0,0,0,0,0,0 1/29/2018 0:00:00,104948067,0016C4834D,1,33,0,1,0,0,0,0,0,0,0,0,0,0,23,0,1,0,0,0,0,0,0,0,0,0,0 I have also tried this version of the script: Get-Content test_data.csv | Where-Object -FilterScript { $_ -replace "([^,]*),([^,]*),([^,]*),(.*)", "`$1,`"`$2`",`"`$3`",$4" } and obtained the same results. My simple test script has convinced me that the regex is correct, but something happens when I use that regex inside a filter script in the Where-Object cmdlet. What simple, yet critical, detail am I overlooking here? Here is my PSVerion: Major Minor Build Revision ----- ----- ----- -------- 5 0 10586 117
You're misunderstanding how Where-Object works. The cmdlet outputs those input lines for which the -FilterScript expression evaluates to $true. It does NOT output whatever you do inside that scriptblock (you'd use ForEach-Object for that). You don't need either Where-Object or ForEach-Object, though. Just put Get-Content in parentheses and use that as the first operand for the -replace operator. You also don't need the 4th capturing group. I would recommend anchoring the expression at the beginning of the string, though. (Get-Content test_data.csv) -replace '^([^,]*),([^,]*),([^,]*)', '$1,"$2","$3"'
This seems to work here. I used ForEach-Object to process each record. Get-Content test_data.csv | ForEach-Object { $_ -replace "([^,]*),([^,]*),([^,]*),(.*)", '$1,"$2","$3",$4' } This also seems to work. Uses the ? to create a reluctant (lazy) capture. Get-Content test_data.csv | ForEach-Object { $_ -replace '(.*?),(.*?),(.*?),(.*)', '$1,"$2","$3",$4' }
I would just make a small change to what you have in order for this to work. Simply change the script to the following, noting that I changed the -FilterScript to a ForEach-Object and fixed a minor typo that you had on the last item in the regular expression with the quotes: Get-Content c:\temp\test_data.csv | ForEach-Object { $_ -replace "([^,]*),([^,]*),([^,]*),(.*)", "`$1,`"`$2`",`"`$3`",`"`$4" } I tested this with the data you provided and it adds the quotes to the correct columns.
Regex replace contents of file and delete lines that don't match
I have a large log file where I want to extract certain types of lines. I have created a working regex to match these lines. How can I now use this regex to extract the lines and nothing else? I have tried cat .\file | %{ if($_ -match "..."){ $_ -replace "...", '...' } else{ $_ -replace ".*", "" } } Which almost works, but the lines that are not of interest still remain as blank lines (meaning the lines of interested are spaced VERY far apart).
The best way is to remove the else clause altogether. If you do that, then no object will be returned from that iteration of the ForEach-Object block. cat .\file | %{ if($_ -match "..."){ $_ -replace "...", '...' } }
Just to append to briantist's answer you don't even need the loop structure. -match and -replace will function as array operators. Removing the need for the if and ForEach-Object. (Get-Content .\file) -match "..." -replace "...","..." Get-Content being the target of the alias cat
Powershell RegEx not being invoked on piped output
Developed this statement on my primary workstation where it (correctly) outputs a delimited textfile: type output.tmp | -match /r /v "^-[-|]*-.$" > output.csv Now, working on my laptop (same win8.1) where supposedly all the same PS modules and snapins are loaded, it tosses an error: -match : The term '-match' is not recognized as the name of a cmdlet, Yet: "Software" –match "soft" works. 1) Why? 2) Is there a PS commandlet I should invoke to be able to get a more verbose/helpful error output? thx
-match is an operator on two arguments (one placed before one after the -match). But at the beginning of each pipeline segment you need a command (including cmdlets)1. There are two approaches: Wrap the -match into a cmdlet like foreach-object (or its % alias): ... | %{ $_ -match $regex } | ... remembering that -match returns a boolean, not the matched text. Use Select-String which is a cmdlet explicitly included for searching text. This does return the matched text (along with some other information), and can read a file itself: Select-String -Path $inputFile $regex 1 Strictly speaking: except the first, which can be any expression.
The reason for the error is that match is a comparison operator, not a cmdlet: Comparison operators let you specify conditions for comparing values and finding values that match specified patterns. To use a comparison operator, specify the values that you want to compare together with an operator that separates these values. Also: The match operators (-Match and -NotMatch) find elements that match or do not match a specified pattern using regular expressions. The syntax is: <string[]> -Match <regular-expression> <string[]> -NotMatch <regular-expression> The following examples show some uses of the -Match operator: PS C:\> "Windows", "PowerShell" -Match ".shell" PowerShell PS C:\> (Get-Command Get-Member -Syntax) -Match "-view" True PS C:\> (Get-Command Get-Member -Syntax) -NotMatch "-path" True PS C:\> (Get-Content Servers.txt) -Match "^Server\d\d" Server01 Server02 The match operators search only in strings. They cannot search in arrays of integers or other objects. So, the correct syntax is: #(type output.tmp) -match "^-[-|]*-.$" > output.csv Note: Just as #mjolinor suggested, the # prefix forces the (type output.tmp) into an array, just in case that the input file contains only one line.
To get obnoxious amounts of debug output, get-Help Set-PSDebug You simply need to add the following: type output.tmp | ? { $_ -match /r /v "^-[-|]*-.$" } > output.csv Or the more powershell-y way: Get-Content -Path:"Output.Tmp" | Where { $_ -match "^-[-|]*-.$" } | Out-File -FilePath:"output.csv"