Powershell -replace regex not working on connection strings - regex

I'm attempting to use the Powershell -replace command to update the data source in my config file. However, the -replace regex below will not remove the $oldServer value.
I've place a string directly in to the $_.connectionString variable in the loop and it saved properly, so I know that is not the issue. Seems to just be the regex.
#environment variables
$env = "DEV"
$oldServer = "quasq10"
$newValue = "$env-AR-SQL.CORP.COM"
$doc = [xml](Get-Content "D:\AMS\app.config")
$doc.configuration.connectionStrings.add|%{
$_.connectionString = $_.connectionString -replace $oldServer, $newValue;
}
$doc.Save($file.FullName)
EDIT
Per the comment below I added a Write-host $_.connectionString statement as the first line in the loop. Below is the console output
metadata=res:///MonetDb.csdl|res:///MonetDb.ssdl|res://*/MonetDb.msl;provider=System.Data.SqlClient;provider connection string="data source=quasq10\sql08a;initial catalog=MyDB
;integrated security=True;multipleactiveresultsets=True;App=EntityFramework"

I just put this right into ISE, I copied your connection string into a variable and was able to do this replace as a one off.
$connectionString = 'metadata=res:///MonetDb.csdl|res:///MonetDb.ssdl|res://*/MonetDb.msl;provider=System.Data.SqlClient;provider connection string="data source=quasq10\sql08a;initial catalog=MyDB ;integrated security=True;multipleactiveresultsets=True;App=EntityFramework"'
$env = "DEV"
$oldServer = "quasq10"
$newValue = "$env-AR-SQL.CORP.COM"
$connectionString -replace $oldServer, $newValue
res:///MonetDb.csdl|res:///MonetDb.ssdl|res://*/MonetDb.msl;provider=System.Data.SqlClient;provider connection string="data source=DEV-AR-SQL.CORP.COM\sql08a;initial catalog=MyDB ;integrated security=True;multipleactiveresultsets=True;App=EntityFramework"
I think your foreach loop might not be getting the info you want, because it looks like your replace is fine.
$doc.configuration.connectionStrings.add
I haven't done much with XML, does the XML data type have an ADD member function? You aren't really adding anything, right?
As a test, what do you get from this:
$doc.configuration.connectionStrings | % {
$_.connectionString -replace $oldServer, $newValue;
}
Run that against a dummy file and see what happens.
For a sanity check on the replace operator:
$string = "The quick brown fox jumped over the lazy dog"
$oldColor = "brown"
$newColor = "orange"
$string -replace $oldColor, $newColor
To avoid digging through comments, this method worked
$string.Replace($oldColor,$newColor)

Related

Regular expression only finding first match

I'm working on something that is similar to other designs I've done, but for some reason, it's only finding the first key/value pair, whereas other ones found all of them. It looks good in regex101.com, which is where I typically test these.
I'm parsing c++ code to get what I need for a reference spreadsheet for error tracking across a system, and results go into a spreadsheet, or is used as a key to lookup info in another file. I do something similar for about 20 files, plus there's other data coming from a sql query, or access/mdb file. The data for this file looks like this:
m_ErrorMap.insert(make_pair(
MAKEWORD(scError,seFatal),
HOP_FATAL_ERROR ));
m_ErrorMap.insert(make_pair(
MAKEWORD(scError,seNotSelected),
HOP_NOT_SELECTED));
m_ErrorMap.insert(make_pair(
MAKEWORD(scError,seCoverOpen),
HOP_COVER_OPEN ));
m_ErrorMap.insert(make_pair(
MAKEWORD(scError,seLeverPosition),
HOP_LEVER_POSITION ));
m_ErrorMap.insert(make_pair(
MAKEWORD(scError,seJam),
HOP_JAM ));
I read this as a string from the file (looks good), and feed it into this Function as $fileContent:
Function Get-Contents60{
[cmdletbinding()]
Param ([string]$fileContent)
Process
{
#m_ErrorMap.insert(make_pair(
#MAKEWORD(scError,seJam),
#HOP_JAM ));
# construct regex
switch -Regex ($fileContent -split '\r?\n') { #this is splitting on each line test regex with https://regex101.com/
'MAKEWORD["("][\w]+,(\w+)[")"],' { #seJam
# add relevant key to key collection
$keys = $Matches[1] } #only match once
',(HOP.*?)[\s]' { # HOP_JAM
# we've reached the relevant error, set it for all relevant keys
foreach($key in $keys){
Write-Host "60 key: $key"
Write-Host "Matches[0]: $($Matches[0]) Matches[1]: $($Matches[1])"
$errorMap[$key] = $($Matches[1])
Write-Host "60 key: $key ... value: $($errorMap[$key])"
}
}
'break' {
# reset/clear key collection
$keys = #()
}
}#switch
#Write-Host "result:" $result -ForegroundColor Green
#$result;
return $errorMap
}#End of Process
}#End of Function
I stepped through it in VSCode, and its finding the first key/value pair, and after that it's not finding anything. I looked at it in regex101.com, and it's finding line endings/breaks, and the MAKEWORD regex and HOP regex are finding what they should on each line it should.
I'm not sure if the issue is that they aren't all in the same line, and maybe I need to change it so it doesn't break on newline and breaks on something else for each key/value pair? I'm a little fuzzy on this.
I'm using powershell 5.1, and VSCode.
Update:
I modified Theo's answer and it worked great. I had simplified the class name from m_HopErrorMap to m_ErrorMap for this question, and the regular expression was grabbing that for each one. I modified that slightly, and Theo's works.
function Get-Contents60{
[cmdletbinding()]
Param ([string]$fileContent)
# create an ordered hashtable to store the results
$errorMap = [ordered]#{}
# process the lines one-by-one
switch -Regex ($fileContent -split '\r?\n') {
'MAKEWORD\([^,]+,([^)]+)\),' { # seJam, seFatal etc.
$key = $matches[1]
}
'(HOP_[^)]+)' {
$errorMap[$key] = $matches[1].Trim()
}
}
# output the completed data as object
[PsCustomObject]$errorMap
return $errorMap
}
I would simplify your function to
function Get-Contents60{
[cmdletbinding()]
Param ([string]$fileContent)
# create an ordered hashtable to store the results
$errorMap = [ordered]#{}
# process the lines one-by-one
switch -Regex ($fileContent -split '\r?\n') {
'MAKEWORD\([^,]+,([^)]+)\),' { # seJam, seFatal etc.
$key = $matches[1]
}
'(HOP[^)]+)' {
$errorMap[$key] = $matches[1].Trim()
}
}
# output the completed data as object
[PsCustomObject]$errorMap
}
Then, using your example text, for which I'm using a Here-string, but in real life you would load the file content with $c = Get-Content -Path 'X:\TheErrors.txt' -Raw you do
$result = Get-Contents60 -fileContent $c
To display on screen
$result | Format-Table -AutoSize
giving you
seFatal seNotSelected seCoverOpen seLeverPosition seJam
------- ------------- ----------- --------------- -----
HOP_FATAL_ERROR HOP_NOT_SELECTED HOP_COVER_OPEN HOP_LEVER_POSITION HOP_JAM

RegEx Finding multiple matches with a quantifier

I have this PowerShell code:
$uri = "http://charts.spotify.com/api/tracks/most_streamed/au/daily/latest"
$ContentType = "application/json"
$postblog = (Invoke-WebRequest -Uri $uri).Content -match 'track_name\S:\S(.*?)",'
$matches[1]
When I run this, I get this result:
FourFiveSeconds
Problem is, I know there are more songs than just this one song. And I know that the match I am using, the string of text "track_name" exists more than once. How can I change my RegEx so that it matches every match it can find? In other words, the expected output would be multiple matches, allowing me to list all the songs, e.g. $matches[1], $matches[2], $matches[3], $matches[4], etc.
Since you are using Invoke-WebRequest, I assume you are using Powershell v4.0. Therefore, you can use ConvertFrom-Json on the data received and iterate over it, instead of using a regex solution:
$uri = "http://charts.spotify.com/api/tracks/most_streamed/au/daily/latest"
$ContentType = "application/json"
$postblog = (Invoke-WebRequest -Uri $uri).Content | ConvertFrom-Json
Now, the entire tracks data is available inside $postblog.tracks array.
Iterate over them to get the track_urls:
Foreach( $track in $postblog.tracks ) {
Write-Output $track.track_url
}
EDIT
Apparently, you can simply use:
Write-Output $postblog.tracks.track_url
instead of the Foreach code-block. Thanks to #PetSerAl for that :)
Thanks to all, yes you are right, converting it to JSON gives me heaps more options and it is cleaner than using RegEx.
I have this script now, which should definitely do the trick.
$spotifytopsongs = #()
$uri = "http://charts.spotify.com/api/tracks/most_streamed/au/daily/latest"
$ContentType = "application/json"
$spotifyjson = (Invoke-WebRequest -Uri $uri).Content | ConvertFrom-Json
$spotifyjson.tracks | select -First 50 | % {$spotifytopsongs += #($_.artist_name + " - " + $_.track_name)}
cls;$spotifytopsongs

Using regex to validate email address in powershell

So i made a script that parses a .msg file in outlook and ommits the result. The whole scripts works except for when I receive an email from inside the network (we use Active Directory) is when i get a result similar to this: /O=Business/OU=FIRST ADMINISTRATIVE GROUP/CN=RECIPIENTS/CN=MIKEF
otherwise for emails outside the network I get email#email.com. I would like to to validate this this with regex that way it would take the name in the CN=" " and adds it to my #email.com
$MSGFILEPATH = '\\srv01\FTP\EmailtoSupportPortal\Testing'
$MSGCOMPLETED= '\\srv01\FTP\EmailtoSupportPortal\Testing\Completed'
Function MSGFiles {
Get-ChildItem $MSGFILEPATH -Filter *.msg|`
ForEach-Object{
$outlook = New-Object -comobject outlook.application
$msg = $outlook.CreateItemFromTemplate($_.FullName)
$body = $msg.Body
$SEM = $msg.SenderEmailAddress
$Subject = $msg.Subject
$SEM
}
}
MSGFiles
Check the MailItem.SenderEmailType property. If it is "EX", use MailItem.Sender.GetExchangeUser.PrimarySmtpAddress (be prepared to handle nulls/errors).
Otherwise use the SenderEmailAddress property the way you do that now.

Perl taint mode with domain name input for CGI resulting in “Insecure dependency in eval”

Given the following in a CGI script with Perl and taint mode I have not been able to get past the following.
tail /etc/httpd/logs/error_log
/usr/local/share/perl5/Net/DNS/Dig.pm line 906 (#1)
(F) You tried to do something that the tainting mechanism didn't like.
The tainting mechanism is turned on when you're running setuid or
setgid, or when you specify -T to turn it on explicitly. The
tainting mechanism labels all data that's derived directly or indirectly
from the user, who is considered to be unworthy of your trust. If any
such data is used in a "dangerous" operation, you get this error. See
perlsec for more information.
[Mon Jan 6 16:24:21 2014] dig.cgi: Insecure dependency in eval while running with -T switch at /usr/local/share/perl5/Net/DNS/Dig.pm line 906.
Code:
#!/usr/bin/perl -wT
use warnings;
use strict;
use IO::Socket::INET;
use Net::DNS::Dig;
use CGI;
$ENV{"PATH"} = ""; # Latest attempted fix
my $q = CGI->new;
my $domain = $q->param('domain');
if ( $domain =~ /(^\w+)\.(\w+\.?\w+\.?\w+)$/ ) {
$domain = "$1\.$2";
}
else {
warn("TAINTED DATA SENT BY $ENV{'REMOTE_ADDR'}: $domain: $!");
$domain = ""; # successful match did not occur
}
my $dig = new Net::DNS::Dig(
Timeout => 15, # default
Class => 'IN', # default
PeerAddr => $domain,
PeerPort => 53, # default
Proto => 'UDP', # default
Recursion => 1, # default
);
my #result = $dig->for( $domain, 'NS' )->to_text->rdata();
#result = sort #result;
print #result;
I normally use Data::Validate::Domain to do checking for a “valid” domain name, but could not deploy it in a way in which the tainted variable error would not occur.
I read that in order to untaint a variable you have to pass it through a regex with capture groups and then join the capture groups to sanitize it. So I deployed $domain =~ /(^\w+)\.(\w+\.?\w+\.?\w+)$/. As shown here it is not the best regex for the purpose of untainting a domain name and covering all possible domains but it meets my needs. Unfortunately my script is still producing tainted failures and I can not figure out how.
Regexp-Common does not provide a domain regex and modules don’t seem to work with untainting variable so I am at a loss now.
How to get this thing to pass taint checking?
$domain is not tainted
I verified that your $domain is not tainted. This is the only variable you use that could be tainted, in my opinion.
perl -T <(cat <<'EOF'
use Scalar::Util qw(tainted);
sub p_t($) {
if (tainted $_[0]) {
print "Tainted\n";
} else {
print "Not tainted\n";
}
}
my $domain = shift;
p_t($domain);
if ($domain =~ /(^\w+)\.(\w+\.?\w+\.?\w+)$/) {
$domain = "$1\.$2";
} else {
warn("$domain\n");
$domain = "";
}
p_t($domain);
EOF
) abc.def
It prints
Tainted
Not tainted
What Net::DNS::Dig does
See Net::DNS::Dig line 906. It is the beginning of to_text method.
sub to_text {
my $self = shift;
my $d = Data::Dumper->new([$self],['tobj']);
$d->Purity(1)->Deepcopy(1)->Indent(1);
my $tobj;
eval $d->Dump; # line 906
…
From new definition I know that $self is just hashref containing values from new parameters and several other filled in the constructor. The evaled code produced by $d->Dump is setting $tobj to a deep copy of $self (Deepcopy(1)), with correctly set self-references (Purity(1)) and basic pretty-printing (Indent(1)).
Where is the problem, how to debug
From what I found out about &Net::DNS::Dig::to_text, it is clear that the problem is at least one tainted item inside $self. So you have a straightforward way to debug your problem further: after constructing the $dig object in your script, check which of its items is tainted. You can dump the whole structure to stdout using print Data::Dumper::Dump($dig);, which is roughly the same as the evaled code, and check suspicious items using &Scalar::Util::tainted.
I have no idea how far this is from making Net::DNS::Dig work in taint mode. I do not use it, I was just curious and wanted to find out, where the problem is. As you managed to solve your problem otherwise, I leave it at this stage, allowing others to continue debugging the issue.
As resolution to this question if anyone comes across it in the future it was indeed the module I was using which caused the taint checks to fail. Teaching me an important lesson on trusting modules in a CGI environment. I switched to Net::DNS as I figured it would not encounter this issue and sure enough it does not. My code is provided below for reference in case anyone wants to accomplish the same thing I set out to do which is: locate the nameservers defined for a domain within its own zone file.
#!/usr/bin/perl -wT
use warnings;
use strict;
use IO::Socket::INET;
use Net::DNS;
use CGI;
$ENV{"PATH"} = ""; // Latest attempted fix
my $q = CGI->new;
my $domain = $q->param('domain');
my #result;
if ( $domain =~ /(^\w+)\.(\w+\.?\w+\.?\w+)$/ ) {
$domain = "$1\.$2";
}
else {
warn("TAINTED DATA SENT BY $ENV{'REMOTE_ADDR'}: $domain: $!");
$domain = ""; # successful match did not occur
}
my $ip = inet_ntoa(inet_aton($domain));
my $res = Net::DNS::Resolver->new(
nameservers => [($ip)],
);
my $query = $res->query($domain, "NS");
if ($query) {
foreach my $rr (grep { $_->type eq 'NS' } $query->answer) {
push(#result, $rr->nsdname);
}
}
else {
warn "query failed: ", $res->errorstring, "\n";
}
#result = sort #result;
print #result;
Thanks for the comments assisting me in this matter, and SO for teaching more then any other resource I have come across.

Regexp to find youtube url, strip off parameters and return clean video url?

imagine this url:
http://www.youtube.com/watch?v=6n8PGnc_cV4&feature=rec-LGOUT-real_rn-2r-13-HM
what is the cleanest and best regexp to do the following:
1.) i want to strip off every thing after the video URL. so that only http://www.youtube.com/watch?v=6n8PGnc_cV4 remains.
2.) i want to convert this url into http://www.youtube.com/v/6n8PGnc_cV4
Since i'm not much of a regexp-ert i need your help:
$content = preg_replace('http://.*?\?v=[^&]*', '', $content);
return $content;
edit: check this out! I want to create a really simple WordPress plugin that just recognizes every normal youtube URL in my $content and replaces it with the embed code:
<?php
function videoplayer($content) {
$embedcode = '<object class="video" width="308" height="100"><embed src="' . . '" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="308" height="100" wmode="opaque"></embed></object>';
//filter normal youtube url like http://www.youtube.com/watch?v=6n8PGnc_cV4&feature=rec-LGOUT-real_rn-2r-13-HM
//convert it to http://www.youtube.com/v/6n8PGnc_cV4
//use embedcode and pass along the new youtube url
$content = preg_replace('', '', $content);
//return embedcode
return $content;
}
add_filter('the_content', 'videoplayer');
?>
I use this search criteria in my script:
/((http|ftp)\:\/\/)?([w]{3}\.)?(youtube\.)([a-z]{2,4})(\/watch\?v=)([a-zA-Z0-9_-]+)(\&feature=)?([a-zA-Z0-9_-]+)?/
You could just split it on the first ampersand.
$content = explode('&', $content);
$content = $content[0];
Edit: Simplest regexp: /http:\/\/www\.youtube\.com\/watch\?v=.*/
Youtube links are all the same. To get the video id from them, first you slice off the extra parameters from the end and then slice off everything but the last 11 characters. See it in action:
$url = "http://www.youtube.com/watch?v=1rnfE4eo1bY&feature=...";
$url = $url.left(42); // "http://www.youtube.com/watch?v=1rnfE4eo1bY"
$url = $url.right(11); // "1rnfE4eo1bY"
$result = "http://www.youtube.com/v/" + $url; // "http://www.youtube.com/v/1rnfE4eo1bY"
You can uniformize all your youtube links (by removing useless parameters) with a Greasemonkey script: http://userscripts.org/scripts/show/86758. Greasemonkey scripts are natively supported as addons in Google Chrome.
And as a bonus, here is a one (okay, actually two) liner:
$url = "http://www.youtube.com/watch?v=1rnfE4eo1bY&feature=...";
$result = "http://www.youtube.com/v/" + $url.left(42).right(11);
--3ICE
$url = "http://www.youtube.com/v/6n8PGnc_cV4";
$start = strpos($url,"v=");
echo 'http://www.youtube.com/v/'.substr($url,$start+2);