Regular expressions for long phrases in perl - regex

I'm looking to extract the "Account Name" and "Source Network Address" from the following text using regular expressions in a perl script. Adding a regular expression for such a long phrase, seems to take a lot of effort.
I need your help with finding the best regex for this, or any ideas would help. Keep in mind that this are just 3 examples out of possible 50? phrases similar to this (different lengths).
Example phrase 1:
WinEvtLog: Security: AUDIT_SUCCESS(4624): Microsoft-Windows-Security-Auditing: admin: DOMAIN: hostname.domain.com: An account was successfully logged on. Subject: Security ID: S-1-0-0 Account Name: - Account Domain: - Logon ID: 0x0 Logon Type: 3 New Logon: Security ID: S-1-5-21-1130994204-1932287720-1813960501-1239 Account Name: admin Account Domain: DOMAIN Logon ID: 0x1d12cfff5 Logon GUID: {AF5E2CF5-1A54-2121-D281-13381F397F41} Process Information: Process ID: 0x0 Process Name: - Network Information: Workstation Name: Source Network Address: 101.101.101.101 Source Port: 52616 Detailed Authentication Information: Logon Process: Kerberos Authentication Package: Kerberos Transited Services: - Package Name (NTLM only): - Key Length: 0 This event is generated when a logon session is created. It is generated on the computer that was accessed.
Example phrase 2:
WinEvtLog: Security: AUDIT_SUCCESS(4634): Microsoft-Windows-Security-Auditing: admin: DOMAIN: hostname.domain.com: An account was logged off. Subject: Security ID: S-1-5-21-1130554204-1932287720-1813960501-4444 Account Name: admin Account Domain: DOMAIN Logon ID: 0x1d12d000a Logon Type: 3 This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer." 4646,1
Example phrase 3:
WinEvtLog: Security: AUDIT_SUCCESS(540): Security: Administrator: HOST88: HOST88: Successful Network Logon: User Name: Administrator Domain: HOST88 Logon ID: (0x14,0x6E6FB948) Logon Type: 3 Logon Process: NtLmSsp Authentication Package: NTLM Workstation Name: DESKHOST88 Logon GUID: - Caller User Name: - Caller Domain: - Caller Logon ID: - Caller Process ID: - Transited Services: - Source Network Address: 10.10.10.10 Source Port: 43221

The following regex will handle your posted cases:
if ( $string =~ /(?<=Account Name:)\s+([^-\s]+).+(?:Source Network Address:)\s+([\d.]+)\s+/ ) {
$account_name = $1;
$source_addr = $2;
}

How rigorous do you want to be with your solution?
If you have log lines and want to extract the word that follows "Account Name:" and the address that follows "Source Network Address:" then you can do it with a very naive regex like this:
my ($account_name) = /Account Name:\s+(\S+)/;
my ($source_network_addr) = /Source Network Address:\s+(\S+)/;
That doesn't attempt to validate that anything else in the line is as you expect it to be, but if the application is only parsing lines that are generated by IIS or whatever, it may not need to be really precise.

Related

Logback.xml maskRegex to to mask a user's ID *if* the user ID is an E-mail Address

I have a requirement to mask user IDs in log files if the user ID is an email address. I can't access the code to the software we are using, so logback.xml is the only option I have to try and mask the e-mail address -- they want user1234#mycompany.org to show up as u******4#mycompany.org, so they can still search the logs if they need to find entries for someone. To complicate matters, there are other outputs to the logs that have the pattern something#something.com that aren't user IDs, so I need to find some keywords ahead of the e-mail address. Here are a couple of examples:
Initiating save of object; type: [User], id: [steven#verizon.net]
Cleaning user record; user: [steven#verizon.net]
And to further complicate things.... they also want to mask the E-mail in rest calls:
Request ended; method: [POST], uri: [/api/v2/token/validate?aid=web_demo&uid=steven%40verizon.net]
I've been trying to get something like this to work, capturing the strings that should prefix an email address or user ID, but I'm banging my head against the wall...
<maskRegex>(?:"user: ["|"[User], id: ["|"uid ["|"savedUser: "|"email_address"|"user"|"uid")\([a-zA-Z0-9]{1}([a-zA-Z0-9\+\.\_\-]{1,128}).#.*)"</maskRegex>
<maskRegex>(?:user: [|[User], id: [|uid [|savedUser: |email_address|user:|uid:)([a-zA-Z0-9]{1}([a-zA-Z0-9\+\.\_\-]{1,128}).#.*)</maskRegex>

Regex Pattern in Powershell doesn't work as expected

I have a string in Powershell which contains the following data
An account failed to log on.
Subject:
Security ID: S-1-5-18
Account Name: TEX
Account Domain: TD2
Logon ID: 0x3E7
Logon Type: 8
Account For Which Logon Failed:
Security ID: S-1-0-0
Account Name: test_mathysf
Account Domain: tdlz2
Failure Information:
Failure Reason: Unknown user name or bad password.
Status: 0xC000006D
Sub Status: 0xC000006A
Process Information:
Caller Process ID: 0x4f80
Caller Process Name: C:\Windows\System32\inetsrv\w3wp.exe
Network Information:
Workstation Name: T22
Source Network Address: 192.168.10.28
Source Port: 45221
Detailed Authentication Information:
Logon Process: Advapi
Authentication Package: Negotiate
Transited Services: -
Package Name (NTLM only): -
Key Length: 0
This event is generated when a logon request fails. It is generated on the computer where access was attempted.
The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Ser
ver service, or a local process such as Winlogon.exe or Services.exe.
The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).
The Process Information fields indicate which account and process on the system requested the logon.
The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be l
eft blank in some cases.
The authentication information fields provide detailed information about this specific logon request.
- Transited services indicate which intermediate services have participated in this logon request.
- Package name indicates which sub-protocol was used among the NTLM protocols.
- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.
I wanted to escape everything between Subject and Key Lenght with the following regex Pattern.
$pattern = "(?<=.*Subject:)\w+?(?=Length:*)"
The result of the regex should be like that
Subject:
Security ID: S-1-5-18
Account Name: TEX
Account Domain: TD2
Logon ID: 0x3E7
Logon Type: 8
Account For Which Logon Failed:
Security ID: S-1-0-0
Account Name: test_mathysf
Account Domain: tdlz2
Failure Information:
Failure Reason: Unknown user name or bad password.
Status: 0xC000006D
Sub Status: 0xC000006A
Process Information:
Caller Process ID: 0x4f80
Caller Process Name: C:\Windows\System32\inetsrv\w3wp.exe
Network Information:
Workstation Name: T22
Source Network Address: 192.168.10.28
Source Port: 45221
Detailed Authentication Information:
Logon Process: Advapi
Authentication Package: Negotiate
Transited Services: -
Package Name (NTLM only): -
Key Length: 0
But in my project, the regex doesn't work. (it doesn't get a match)
After that I would use the ConvertFrom-StringData cmdlet to create a hash table with the entries (For examle Security ID --> Key = S-1-5-18)
Have somebody an idea where the problem could be?
You seem to want to extract the structured information from the event text.
Here is what I would do.
function ExtractEventData {
param(
[string]$EventText
)
$pattern = [regex]"(?m)^(.*):\s*(^ \S.*:.*\n)+"
$result = #{}
foreach ($match in $pattern.Matches($EventText)) {
$section_name = $match.Groups[1].Value
$result[$section_name] = #{}
foreach ($line in $match.Groups[2].Captures) {
$key, $value = $line.Value.Split(':'.ToCharArray(), 2)
$result[$section_name][$key.Trim()] = $value.Trim()
}
$result[$section_name] = [pscustomobject]$result[$section_name]
}
[pscustomobject]$result
}
When you call it like this with your sample string above
$result = ExtractEventData -EventText $sampleEvent
it produces this data structure:
Process Information : #{Caller Process ID=0x4f80; Caller Process Name=C:\Windows\System32\inetsrv\w3wp.exe}
Detailed Authentication Information : #{Key Length=0; Logon Process=Advapi; Package Name (NTLM only)=-; Authentication Package=Negotiate; Transited Services=-}
Subject : #{Account Domain=TD2; Security ID=S-1-5-18; Account Name=TEX; Logon ID=0x3E7}
Account For Which Logon Failed : #{Account Domain=tdlz2; Security ID=S-1-0-0; Account Name=test_mathysf}
Failure Information : #{Status=0xC000006D; Failure Reason=Unknown user name or bad password.; Sub Status=0xC000006A}
Network Information : #{Workstation Name=T22; Source Network Address=192.168.10.28; Source Port=45221}
Which you can access directly, e.g.
$result.'Network Information'.'Workstation Name' # => T22
The regex is
(?m) # multiline mode
^ # start of line
(.*):\s* # anything (e.g. 'Network Information' into group 1), a colon, whitespace
( # group 2 (e.g. ' Workstation Name: T22')
^ \S.*:.*\n # start of line, 4 spaces, a non-space, a colon, anything, newline
)+ # end group 2, repeat
This way, each section gets handled individually, and within each section, each line.
This is the Message property you got from Get-WinEvent, which is a multiline string with LOCALIZED properties.
Instead of converting that string into an array of objects, better use the event XML notation where you can get universally named properties.
Something like this:
$filter = #{LogName='Security';ProviderName='Microsoft-Windows-Security-Auditing';ID=4625 }
$result = Get-WinEvent -FilterHashtable $filter -ComputerName SECRETSERVER | ForEach-Object {
# convert the event to XML and grab the Event node
$eventXml = ([xml]$_.ToXml()).Event
# output the properties you need in your output
[PSCustomObject]#{
Time = [DateTime]$eventXml.System.TimeCreated.SystemTime
UserName = ($eventXml.EventData.Data | Where-Object { $_.Name -eq 'TargetUserName' }).'#text'
UserSID = ($eventXml.EventData.Data | Where-Object { $_.Name -eq 'TargetUserSid' }).'#text'
Computer = ($eventXml.EventData.Data | Where-Object { $_.Name -eq 'WorkstationName' }).'#text'
}
}
# output on screen
$result | Format-Table -AutoSize
# output to CSV file
$result | Export-Csv -Path 'X:\FailedLogons.csv' -NoTypeInformation
Here you can see all properties the XML would contain for event 4625. The example above just lists the time the logon failure happened, the computer on which that attempt was made and the user name that failed to log in.
You can use
(?ms)^Subject:.*?Length:(?-s).*
See the regex demo.
Regex details:
(?ms) - inline modifiers, m makes ^ match start of any line and $ match any line end position, and s makes . match any chars including LF chars
^ - start of a line
Subject: - a literal text
.*? - any zero or more chars as few as possible
Length: - a literal text
(?-s) - now, . cannot match LF chars any longer
.* - any zero or more chars other than a newline (LF) char, as many as possible.

What is parastate test network chain ID?

I want to add Parastate tesnet to my metamask. I input this:
Network name: Parastate testnet
New RPC URL: https://rpc.parastate.io:8545/
But it also requires "Chain Id". Where can I find it or which one is correct?
Network Name: ParaState
New RPC URL: https://rpc.parastate.io:8545
Chain ID: 123
Currency Symbol (Optional): STATE
RPC URL: https://rpc.parastate.io:8545
Chain ID: 123
Currency Symbol: STATE
try it

Regex for event log 4624 logon type and account name

I need some help with regex which I am still trying to teach myself. I have beaten my head on this for hours. I am trying to filter logon events in my event log via a syslog server. I would like to only capture actual user logons not all the other stuff coming through.
Computer and user accounts are on the line that says "ACCOUNT NAME:". The computer accounts contain a $ appended to the end of their name and I need to filter those out while accepting the ones without the $ on the end.
The log comes through like this:
Logon Type: 3
Impersonation Level: Impersonation
New Logon:
Security ID: S-1-5-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Account Name: COMPUTERNAME$
Account Domain:(random whitespace) DomainName
Logon ID: 0xXXXXXXXXX
Logon GUID: {XXXXXXXXXXXXX}
Process Information:
Process ID: 0x0
Process Name: -
Network Information:
Workstation Name: -
Source Network Address: 192.168.0.x
Source Port: 000000
Any regex pro that can assist? Thanks
I'm not a regex pro, but I can help you with this specific problem. Use a look ahead to filter the dollar sign off, like this:
(?!.*\$)
This is called a negative look ahead '(?!' and it looks for any char any number of times '.*' UNTIL there's a '$' sign. If a dollar sign is found, no match.
Note that the dollar sign is escaped in the regex.
Edit:
To match account name and not dollar sign, it becomes (assuming account names doesn't contain Spaces):
(?!.*\$)Account Name:\s*(\w+)
Now grap the account in Group 1.
try this:
Account Name:\s*([^\s\$]+[^\s\$]$)
https://regex101.com/r/0W8xVg/3

ADFS 3.0 IP Filter

I'm working on a custom claim to match one of two specific IP addresses, and then deny the claim if it's either of them.
Here's the regex (IP changed for fun to 1.2.3.4 and 5.6.7.8)
c:[Type == "(notabletoposturl)ttp://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-forwarded-client-ip", Value =~ "(^1\.2\.3\.4|^5\.6\.7\.8)"] => issue(Type = "(notabletoposturl)ttp://schemas.microsoft.com/authorization/claims/deny", Value = "DenyUsersWithClaim");
It doesn't seem to be working as expected. I'm still seeing bad username and password attempts from those IPs in the event logs like:
Token validation failed. See inner exception for more details.
Additional Data
Activity ID: 00000000-0000-0000-0000-000000000000
Token Type:
(notabletopostlink)ttp://schemas.microsoft.com/ws/2006/05/identitymodel/tokens/UserName 
Client IP:
1.2.3.4,{Exchange Online Server IP} 
Error message:
username#contoso.com-The user name or password is incorrect
Is my syntax or regex off for ADFS 3?