I'm working on something that is similar to other designs I've done, but for some reason, it's only finding the first key/value pair, whereas other ones found all of them. It looks good in regex101.com, which is where I typically test these.
I'm parsing c++ code to get what I need for a reference spreadsheet for error tracking across a system, and results go into a spreadsheet, or is used as a key to lookup info in another file. I do something similar for about 20 files, plus there's other data coming from a sql query, or access/mdb file. The data for this file looks like this:
m_ErrorMap.insert(make_pair(
MAKEWORD(scError,seFatal),
HOP_FATAL_ERROR ));
m_ErrorMap.insert(make_pair(
MAKEWORD(scError,seNotSelected),
HOP_NOT_SELECTED));
m_ErrorMap.insert(make_pair(
MAKEWORD(scError,seCoverOpen),
HOP_COVER_OPEN ));
m_ErrorMap.insert(make_pair(
MAKEWORD(scError,seLeverPosition),
HOP_LEVER_POSITION ));
m_ErrorMap.insert(make_pair(
MAKEWORD(scError,seJam),
HOP_JAM ));
I read this as a string from the file (looks good), and feed it into this Function as $fileContent:
Function Get-Contents60{
[cmdletbinding()]
Param ([string]$fileContent)
Process
{
#m_ErrorMap.insert(make_pair(
#MAKEWORD(scError,seJam),
#HOP_JAM ));
# construct regex
switch -Regex ($fileContent -split '\r?\n') { #this is splitting on each line test regex with https://regex101.com/
'MAKEWORD["("][\w]+,(\w+)[")"],' { #seJam
# add relevant key to key collection
$keys = $Matches[1] } #only match once
',(HOP.*?)[\s]' { # HOP_JAM
# we've reached the relevant error, set it for all relevant keys
foreach($key in $keys){
Write-Host "60 key: $key"
Write-Host "Matches[0]: $($Matches[0]) Matches[1]: $($Matches[1])"
$errorMap[$key] = $($Matches[1])
Write-Host "60 key: $key ... value: $($errorMap[$key])"
}
}
'break' {
# reset/clear key collection
$keys = #()
}
}#switch
#Write-Host "result:" $result -ForegroundColor Green
#$result;
return $errorMap
}#End of Process
}#End of Function
I stepped through it in VSCode, and its finding the first key/value pair, and after that it's not finding anything. I looked at it in regex101.com, and it's finding line endings/breaks, and the MAKEWORD regex and HOP regex are finding what they should on each line it should.
I'm not sure if the issue is that they aren't all in the same line, and maybe I need to change it so it doesn't break on newline and breaks on something else for each key/value pair? I'm a little fuzzy on this.
I'm using powershell 5.1, and VSCode.
Update:
I modified Theo's answer and it worked great. I had simplified the class name from m_HopErrorMap to m_ErrorMap for this question, and the regular expression was grabbing that for each one. I modified that slightly, and Theo's works.
function Get-Contents60{
[cmdletbinding()]
Param ([string]$fileContent)
# create an ordered hashtable to store the results
$errorMap = [ordered]#{}
# process the lines one-by-one
switch -Regex ($fileContent -split '\r?\n') {
'MAKEWORD\([^,]+,([^)]+)\),' { # seJam, seFatal etc.
$key = $matches[1]
}
'(HOP_[^)]+)' {
$errorMap[$key] = $matches[1].Trim()
}
}
# output the completed data as object
[PsCustomObject]$errorMap
return $errorMap
}
I would simplify your function to
function Get-Contents60{
[cmdletbinding()]
Param ([string]$fileContent)
# create an ordered hashtable to store the results
$errorMap = [ordered]#{}
# process the lines one-by-one
switch -Regex ($fileContent -split '\r?\n') {
'MAKEWORD\([^,]+,([^)]+)\),' { # seJam, seFatal etc.
$key = $matches[1]
}
'(HOP[^)]+)' {
$errorMap[$key] = $matches[1].Trim()
}
}
# output the completed data as object
[PsCustomObject]$errorMap
}
Then, using your example text, for which I'm using a Here-string, but in real life you would load the file content with $c = Get-Content -Path 'X:\TheErrors.txt' -Raw you do
$result = Get-Contents60 -fileContent $c
To display on screen
$result | Format-Table -AutoSize
giving you
seFatal seNotSelected seCoverOpen seLeverPosition seJam
------- ------------- ----------- --------------- -----
HOP_FATAL_ERROR HOP_NOT_SELECTED HOP_COVER_OPEN HOP_LEVER_POSITION HOP_JAM
I'm pretty new to Pester so please bear with me. We're trying to make tests for a very large script that does some active directory queries and validates data. I've simplified it a bit to the following example:
Dummy.ps1
Param (
[String]$OU,
[String[]]$Groups
)
$AllOUusers = Get-ADUser -Filter * -SearchBase $OU
$GroupMemberUsers = foreach ($G in $Groups) {
Get-ADGroupMember $G -Recursive | Get-ADUser -Properties whenCreated
}
Dummy.Tests.ps1
$here = Split-Path -Parent $MyInvocation.MyCommand.Path
$sut = (Split-Path -Leaf $MyInvocation.MyCommand.Path) -replace '\.Tests\.', '.'
$Params = #{
OU = 'OU=users,DC=contoso,DC=com'
Groups = 'Group1', 'Group2'
}
Describe 'test' {
Mock Get-ADGroupMember {
[PSCustomObject]#{SamAccountName = 'Member1'}
[PSCustomObject]#{SamAccountName = 'Member2'}
}
Mock Get-ADUser {
[PSCustomObject]#{SamAccountName = 'User1'}
[PSCustomObject]#{SamAccountName = 'User2'}
[PSCustomObject]#{SamAccountName = 'User3'}
}
Mock Get-ADUser {
[PSCustomObject]#{SamAccountName = 'User4'}
[PSCustomObject]#{SamAccountName = 'User5'}
} -ParameterFilter {$identity -eq 'User1'}
."$here\$sut" #Params
it 'test 1' {
($AllOUusers | Measure-Object).Count | Should -BeExactly 3
}
it 'test 2' {
($GroupMemberUsers | Measure-Object).Count | Should -BeExactly 2
'User4', 'User5' | Should -BeIn $GroupMemberUsers.SamAccountName
}
}
Error:
Cannot validate argument on parameter 'Identity'.
In the case above we're trying to collect in the first place all user accounts in a specific OU and afterwards all the members of a specific security group. When we have the members we use the Get-ADUser CmdLet again to get more details for these specific members.
The end goal is to have a Mock for the first Get-ADUser that returns all the user accounts, and a second Mock of Get-ADUser that only returns a specific set of user accounts based on group membership (or simulated group membership).
System details
PowerShell 5.1
Pester 4.1.1.
Windows server 2012 R1
AD Module 1.0.0.0.
It seems like the error you're seeing is occurring because the users you create in your Mock of Get-ADGroupMember aren't being mapped to/accepted by the Identity parameter because it accepts pipeline input ByValue and expects an ADUser type object.
You can work around that by using New-MockObject to create the specific object type you need. Here's an alternative option for your Mock of Get-ADGroupMember:
Mock Get-ADGroupMember {
1..2 | ForEach-Object {
$User = New-MockObject -Type Microsoft.ActiveDirectory.Management.ADUser
$User.SamAccountName = "Member$_"
$User
}
}
I've used a ForEach as a quick way to return the 2 users you were returning before, with the value you were setting but as actual ADUser objects.
This doesn't actually make your -ParameterFilter on the second Get-ADUser work however. That seems to be again because the User is an object but you're comparing it to a string, it doesn't evaluate as true.
One possible workaround for this (that I think works somewhat for your specific use case) is not to check for a specific value of $Identity but just to check if it has any value:
Mock Get-ADUser {
[PSCustomObject]#{SamAccountName = 'User4'}
[PSCustomObject]#{SamAccountName = 'User5'}
} -ParameterFilter { $identity }
This causes your 2nd Mock to always be called for the 2nd Get-ADUser in your script as that's the only one that gets piped an Identity value. This isn't exactly what you were trying to achieve, but I figure might help a little.
So i made a script that parses a .msg file in outlook and ommits the result. The whole scripts works except for when I receive an email from inside the network (we use Active Directory) is when i get a result similar to this: /O=Business/OU=FIRST ADMINISTRATIVE GROUP/CN=RECIPIENTS/CN=MIKEF
otherwise for emails outside the network I get email#email.com. I would like to to validate this this with regex that way it would take the name in the CN=" " and adds it to my #email.com
$MSGFILEPATH = '\\srv01\FTP\EmailtoSupportPortal\Testing'
$MSGCOMPLETED= '\\srv01\FTP\EmailtoSupportPortal\Testing\Completed'
Function MSGFiles {
Get-ChildItem $MSGFILEPATH -Filter *.msg|`
ForEach-Object{
$outlook = New-Object -comobject outlook.application
$msg = $outlook.CreateItemFromTemplate($_.FullName)
$body = $msg.Body
$SEM = $msg.SenderEmailAddress
$Subject = $msg.Subject
$SEM
}
}
MSGFiles
Check the MailItem.SenderEmailType property. If it is "EX", use MailItem.Sender.GetExchangeUser.PrimarySmtpAddress (be prepared to handle nulls/errors).
Otherwise use the SenderEmailAddress property the way you do that now.
I have a small script that I can search people by their different properties and pull certain info. I pull the Memberof property, but it is kind of ugly with the full path. All I want is the canonical name.
I thought that I could pipe Memberof to Where-Object and match from cn to the first ,:
select Name, EmailAddress, SAMAccountName, SN, Memberof |
Where-Object -FilterScript:{$_.Memberof -match "cn*,"} | fl
Is this more complicated than I am thinking, or am I overlooking something? Any help would be nice.
Assuming what you're really after is common name (cn), I use this:
$regex = '^CN=(.+?),(?:CN|OU)=.+'
$cn = $dn -replace $regex,'$1'
Common names can have embedded commas. They have to be escaped with backslashes, but they can be in there, so using the comma as an anchor is unreliable. If it's a leaf object, the next AD reference in the DN will have to be either an OU, or a container, so it will be either OU= or CN=. I've always found this to be reliable.
The -match operator expects a regular expression. Your second operand is cn*,, which means "a 'c', followed by zero or more times 'n', followed by a comma". To match text from cn to the next comma use cn.*?, instead.
In regular expressions the special character * has the meaning "zero or more times the preceeding expression", and a dot matches any character except newlines. The ? makes the match non-greedy, so you get the shortest match instead of the longest match.
However, the -match operator is just matching an input string to an expression. It doesn't extract or remove anything from the input. Use the -replace operator for that:
select Name, EmailAddress, SAMAccountName, SN,
#{n='MemberOf';e={$_.MemberOf -replace '^(cn.*?),.*','$1'}}
Long function incoming. Source. Don't recall if I changed anything.
#Paste the function below into your session. Here's an example pulling all my groups:
Get-ADUser cookiemonster -Properties memberof | select -ExpandProperty memberof | Translate-ADName -InputType dn -OutputType canonical
#Looking at your question more closely, you aren't looking for the canonical name. There are several output types to the function below, maybe NT4 is what you are looking for:
Get-ADUser cookiemonster -Properties memberof | select -ExpandProperty memberof | Translate-ADName -InputType dn -OutputType NT4
#Add this function to your session before running the command above...
function Translate-ADName {
<#
.SYNOPSIS
Translates Active Directory names between various formats.
.DESCRIPTION
Translates Active Directory names between various formats using the NameTranslate COM object. Before names can be translated, the NameTranslate object must first be initialized. The default initialization type is 'GC' (see the -InitType parameter). You can use the -Credential parameter to initialize the NameTranslate object using specific credentials.
.PARAMETER OutputType
The output name type, which must be one of the following:
1779 RFC 1779; e.g., 'CN=Phineas Flynn,OU=Engineers,DC=fabrikam,DC=com'
DN short for 'distinguished name'; same as 1779
canonical canonical name; e.g., 'fabrikam.com/Engineers/Phineas Flynn'
NT4 domain\username; e.g., 'fabrikam\pflynn'
display display name
domainSimple simple domain name format
enterpriseSimple simple enterprise name format
GUID GUID; e.g., '{95ee9fff-3436-11d1-b2b0-d15ae3ac8436}'
UPN user principal name; e.g., 'pflynn#fabrikam.com'
canonicalEx extended canonical name format
SPN service principal name format
.PARAMETER Name
The name to translate. This parameter does not support wildcards.
.PARAMETER InputType
The input name type. Possible values are the same as -OutputType, with the following additions:
unknown unknown name format; the system will estimate the format
SIDorSIDhistory SDDL string for the SID or one from the object's SID history
The default value for this parameter is 'unknown'.
.PARAMETER InitType
The type of initialization to be performed, which must be one of the following:
domain Bind to the domain specified by the -InitName parameter
server Bind to the server specified by the -InitName parameter
GC Locate and bind to a global catalog
The default value for this parameter is 'GC'. When -InitType is not 'GC', you must also specify the -InitName parameter.
.PARAMETER InitName
When -InitType is 'domain' or 'server', this parameter specifies which domain or server to bind to. This parameter is ignored if -InitType is 'GC'.
.PARAMETER ChaseReferrals
This parameter specifies whether to chase referrals. (When a server determines that other servers hold relevant data, in part or as a whole, it may refer the client to another server to obtain the result. Referral chasing is the action taken by a client to contact the referred-to server to continue the directory search.)
.PARAMETER Credential
Uses the specified credentials when initializing the NameTranslate object.
.FUNCTIONALITY
Active Directory
.EXAMPLE
PS C:\> Translate-ADName -OutputType dn -Name fabrikam\pflynn
This command outputs the specified domain\username as a distinguished name.
PS C:\> Translate-ADName canonical 'CN=Phineas Flynn,OU=Engineers,DC=fabrikam,DC=com'
This command outputs the specified DN as a canonical name.
PS C:\> Translate-ADName dn fabrikam\pflynn -InitType server -InitName dc1
This command uses the server dc1 to translate the specified name.
PS C:\> Translate-ADName display fabrikam\pflynn -InitType domain -InitName fabrikam
This command uses the fabrikam domain to translate the specified name.
PS C:\> Translate-ADName dn 'fabrikam.com/Engineers/Phineas Flynn' -Credential (Get-Credential)
Prompts for credentials, then uses those credentials to translate the specified name.
PS C:\> Get-Content DNs.txt | Translate-ADName -OutputType display -InputType dn
Outputs the display names for each of the distinguished names in the file DNs.txt.
.NOTES
http://windowsitpro.com/active-directory/translating-active-directory-object-names-between-formats
#>
[CmdletBinding()]
param(
[parameter(Mandatory=$TRUE,Position=0)]
[validateset("NT4","1779","SPN","canonical","GUID","DN","UPN","display","domainSimple","enterpriseSimple","canonicalEx")]
[String] $OutputType,
[parameter(Mandatory=$TRUE,Position=1,ValueFromPipeline=$TRUE)]
[String[]] $Name,
[validateset("NT4","1779","SPN","canonical","GUID","DN","UPN","display","domainSimple","enterpriseSimple","canonicalEx","SIDorSidHistory","unknown")]
[String] $InputType="unknown",
[validateset("domain","server","GC")]
[String] $InitType="GC",
[String] $InitName="",
[Switch] $ChaseReferrals,
[System.Management.Automation.PSCredential] $Credential
)
begin {
# Hash table to simplify output type names and values
$OutputNameTypes = #{
"1779" = 1;
"DN" = 1;
"canonical" = 2;
"NT4" = 3;
"display" = 4;
"domainSimple" = 5;
"enterpriseSimple" = 6;
"GUID" = 7;
"UPN" = 9;
"canonicalEx" = 10;
"SPN" = 11;
}
# Copy output type hash table and add two additional types
$InputNameTypes = $OutputNameTypes.Clone()
$InputNameTypes.Add("unknown", 8)
$InputNameTypes.Add("SIDorSidHistory", 12)
# Same as with previous hash tables...
$InitNameTypes = #{
"domain" = 1;
"server" = 2;
"GC" = 3;
}
# Accessor functions to simplify calls to NameTranslate
function invoke-method([__ComObject] $object, [String] $method, $parameters) {
$output = $object.GetType().InvokeMember($method, "InvokeMethod", $NULL, $object, $parameters)
if ( $output ) { $output }
}
function get-property([__ComObject] $object, [String] $property) {
$object.GetType().InvokeMember($property, "GetProperty", $NULL, $object, $NULL)
}
function set-property([__ComObject] $object, [String] $property, $parameters) {
[Void] $object.GetType().InvokeMember($property, "SetProperty", $NULL, $object, $parameters)
}
# Create the NameTranslate COM object
$NameTranslate = new-object -comobject NameTranslate
# If -Credential, use InitEx to initialize it; otherwise, use Init
if ( $Credential ) {
$networkCredential = $Credential.GetNetworkCredential()
try {
invoke-method $NameTranslate "InitEx" (
$InitNameTypes[$InitType],
$InitName,
$networkCredential.UserName,
$networkCredential.Domain,
$networkCredential.Password
)
}
catch [System.Management.Automation.MethodInvocationException] {
write-error $_
exit
}
finally {
remove-variable networkCredential
}
}
else {
try {
invoke-method $NameTranslate "Init" (
$InitNameTypes[$InitType],
$InitName
)
}
catch [System.Management.Automation.MethodInvocationException] {
write-error $_
exit
}
}
# If -ChaseReferrals, set the object's ChaseReferral property to 0x60
if ( $ChaseReferrals ) {
set-property $NameTranslate "ChaseReferral" (0x60)
}
# The NameTranslate object's Set method specifies the name to translate and
# its input format, and the Get method returns the name in the output format
function translate-adname2([String] $name, [Int] $inputType, [Int] $outputType) {
try {
invoke-method $NameTranslate "Set" ($inputType, $name)
invoke-method $NameTranslate "Get" ($outputType)
}
catch [System.Management.Automation.MethodInvocationException] {
write-error "'$name' - $($_.Exception.InnerException.Message)"
}
}
}
process {
Foreach($item in $name){
translate-adname2 $name $InputNameTypes[$InputType] $OutputNameTypes[$OutputType]
}
}
}
Good luck!
If your regex supports negative look-behind assertions, you can use ^CN=(.+?)(?:(?<!\\),|$) (usage example here).
Here's my stab at this task:
function ConvertFrom-DistinguishedName ($DistinguishedName) {
$Domain = ($DistinguishedName -split ',DC=' | Where-Object { $_ -notmatch '^CN=' }) -join '.'
$CNPath = ($DistinguishedName -split ',DC=' | Where-Object { $_ -match '^CN=' }) -split ',\w\w='
[array]::Reverse($CNPath)
($Domain + '/' + ($CNPath -join '/')) -replace 'CN=' -replace '\\'
}
ConvertFrom-DistinguishedName "CN=Bradshaw\, Jeremy,OU=PowerShell,DC=stackoverflow,DC=com"
...and of course the output is below:
stackoverflow.com/PowerShell/Bradshaw, Jeremy
Given the following in a CGI script with Perl and taint mode I have not been able to get past the following.
tail /etc/httpd/logs/error_log
/usr/local/share/perl5/Net/DNS/Dig.pm line 906 (#1)
(F) You tried to do something that the tainting mechanism didn't like.
The tainting mechanism is turned on when you're running setuid or
setgid, or when you specify -T to turn it on explicitly. The
tainting mechanism labels all data that's derived directly or indirectly
from the user, who is considered to be unworthy of your trust. If any
such data is used in a "dangerous" operation, you get this error. See
perlsec for more information.
[Mon Jan 6 16:24:21 2014] dig.cgi: Insecure dependency in eval while running with -T switch at /usr/local/share/perl5/Net/DNS/Dig.pm line 906.
Code:
#!/usr/bin/perl -wT
use warnings;
use strict;
use IO::Socket::INET;
use Net::DNS::Dig;
use CGI;
$ENV{"PATH"} = ""; # Latest attempted fix
my $q = CGI->new;
my $domain = $q->param('domain');
if ( $domain =~ /(^\w+)\.(\w+\.?\w+\.?\w+)$/ ) {
$domain = "$1\.$2";
}
else {
warn("TAINTED DATA SENT BY $ENV{'REMOTE_ADDR'}: $domain: $!");
$domain = ""; # successful match did not occur
}
my $dig = new Net::DNS::Dig(
Timeout => 15, # default
Class => 'IN', # default
PeerAddr => $domain,
PeerPort => 53, # default
Proto => 'UDP', # default
Recursion => 1, # default
);
my #result = $dig->for( $domain, 'NS' )->to_text->rdata();
#result = sort #result;
print #result;
I normally use Data::Validate::Domain to do checking for a “valid” domain name, but could not deploy it in a way in which the tainted variable error would not occur.
I read that in order to untaint a variable you have to pass it through a regex with capture groups and then join the capture groups to sanitize it. So I deployed $domain =~ /(^\w+)\.(\w+\.?\w+\.?\w+)$/. As shown here it is not the best regex for the purpose of untainting a domain name and covering all possible domains but it meets my needs. Unfortunately my script is still producing tainted failures and I can not figure out how.
Regexp-Common does not provide a domain regex and modules don’t seem to work with untainting variable so I am at a loss now.
How to get this thing to pass taint checking?
$domain is not tainted
I verified that your $domain is not tainted. This is the only variable you use that could be tainted, in my opinion.
perl -T <(cat <<'EOF'
use Scalar::Util qw(tainted);
sub p_t($) {
if (tainted $_[0]) {
print "Tainted\n";
} else {
print "Not tainted\n";
}
}
my $domain = shift;
p_t($domain);
if ($domain =~ /(^\w+)\.(\w+\.?\w+\.?\w+)$/) {
$domain = "$1\.$2";
} else {
warn("$domain\n");
$domain = "";
}
p_t($domain);
EOF
) abc.def
It prints
Tainted
Not tainted
What Net::DNS::Dig does
See Net::DNS::Dig line 906. It is the beginning of to_text method.
sub to_text {
my $self = shift;
my $d = Data::Dumper->new([$self],['tobj']);
$d->Purity(1)->Deepcopy(1)->Indent(1);
my $tobj;
eval $d->Dump; # line 906
…
From new definition I know that $self is just hashref containing values from new parameters and several other filled in the constructor. The evaled code produced by $d->Dump is setting $tobj to a deep copy of $self (Deepcopy(1)), with correctly set self-references (Purity(1)) and basic pretty-printing (Indent(1)).
Where is the problem, how to debug
From what I found out about &Net::DNS::Dig::to_text, it is clear that the problem is at least one tainted item inside $self. So you have a straightforward way to debug your problem further: after constructing the $dig object in your script, check which of its items is tainted. You can dump the whole structure to stdout using print Data::Dumper::Dump($dig);, which is roughly the same as the evaled code, and check suspicious items using &Scalar::Util::tainted.
I have no idea how far this is from making Net::DNS::Dig work in taint mode. I do not use it, I was just curious and wanted to find out, where the problem is. As you managed to solve your problem otherwise, I leave it at this stage, allowing others to continue debugging the issue.
As resolution to this question if anyone comes across it in the future it was indeed the module I was using which caused the taint checks to fail. Teaching me an important lesson on trusting modules in a CGI environment. I switched to Net::DNS as I figured it would not encounter this issue and sure enough it does not. My code is provided below for reference in case anyone wants to accomplish the same thing I set out to do which is: locate the nameservers defined for a domain within its own zone file.
#!/usr/bin/perl -wT
use warnings;
use strict;
use IO::Socket::INET;
use Net::DNS;
use CGI;
$ENV{"PATH"} = ""; // Latest attempted fix
my $q = CGI->new;
my $domain = $q->param('domain');
my #result;
if ( $domain =~ /(^\w+)\.(\w+\.?\w+\.?\w+)$/ ) {
$domain = "$1\.$2";
}
else {
warn("TAINTED DATA SENT BY $ENV{'REMOTE_ADDR'}: $domain: $!");
$domain = ""; # successful match did not occur
}
my $ip = inet_ntoa(inet_aton($domain));
my $res = Net::DNS::Resolver->new(
nameservers => [($ip)],
);
my $query = $res->query($domain, "NS");
if ($query) {
foreach my $rr (grep { $_->type eq 'NS' } $query->answer) {
push(#result, $rr->nsdname);
}
}
else {
warn "query failed: ", $res->errorstring, "\n";
}
#result = sort #result;
print #result;
Thanks for the comments assisting me in this matter, and SO for teaching more then any other resource I have come across.