I was following string manipulation docs from splunk itself
SPL2 example Returns the "body" field with phone numbers redacted.
...| eval body=replace(cast(body, "string"), /[0-9]{3}[-.][0-9]{3}[-.][0-9]{4}/, "<redacted>");
But when I tried to do query
... | eval hostname=replace(cast(hostname, "string"), /cron*/, ""); | ..
I got error
Error in 'eval' command: The expression is malformed. An unexpected character is reached at '/cron*/, "a");'.
I got confused, what did I do wrong?
Update:
String example:
pods-name-cron-3829hr832
pods-name-cron-8923eh32b
My goal was to remove the cron-<random_id>
You're looking at the documentation for Splunk Data Stream Processor (DSP), which is not Splunk Enterprise. DSP is an advanced method for bringing data into Splunk Enterprise (amongst other things). You are most likely doing a search which is within Splunk Enterprise, and the docs for that are at https://docs.splunk.com/Documentation/Splunk
If you are trying to rename a portion of a field and replace it with nothing, you need to use the replace command
... | eval hostname=replace(hostname, "cron*", "") | ..
For example, | makeresults | eval hostname="cronmaster.acme.com" | eval hostname=replace(hostname, "cron", "") will remove cron from cronmaster.acme.com
Post an example of the string and what you want it converted to, and we can confirm if the replace is sufficient, or if a regular expression is required.
You can use the same command, with a different regular expression. The following looks for -cron- followed by any non-whitespace characters, which is represented by \S+.
| makeresults | eval hostname="pods-name-cron-3829hr832" | eval hostname=replace(hostname, "-cron-\S+", "")
Every different version of Splunk might have different functions available for use. Please refer to the documentation according to the Splunk version you are using.
Try this:
... | eval hostname=replace(toString(hostname), "/cron*/", "") | ..
Here are some links you may find helpful if you are using Splunk 7.3.1:
Conversion Functions
Replace function
Related
I am looking for a regular expression in my jq query (using. It's a pretty simple one, I want to match entries starting with a number \d (or [0-9]) and ending in linux. What I've tried so far:
versions=`echo $allversions| jq '.tags[] | select(startswith("\d")) | select(endswith("linux"))'`
but I don't think startwith doesn't support regular expression. I'm reading that match supports regular expression, but I cannot find proper documentation or examples about it. A simple 'jq '.tags[]| match("\d.*linux")' doesn't work and gives a syntax error message:
syntax error near unexpected token `"\d*linux"'
How can I accomplish this? Or should I combine jq with sed instead?
FYI:
$ jq --version
jq-1.6
Ok I found how to do it!
jq -r '.tags[] | select(test("^[0-9].*linux"))'
The arguments to regex functions must be JSON strings, so any regex backslash must be escaped. Thus, instead of match("\d.*linux")
you'd write:
match("\\d.*linux")
I am just into learning of Splunk queries, I'm trying to grab a data from myfile.csv file based on the regex expression.
In particular, I'm looking forward, print only the rows where column fqdn not endswith udc.net and htc.com.
Below is my query which is working but i'm writing it twice.
| inputlookup myfile.csv
| regex support_group="^mygroup-Linux$"
| regex u_sec_dom="^Normal Secure$"
| regex fqdn!=".*?udc.net$"
| regex fqdn!=".*?htc.com$"
| where match(fqdn,".")
I am trying them to combine with | separeted but not working though...
| regex fqdn!="(.*?udc.net | ".*?htc.com)$"
You can do this with a search and where clause:
| inputlookup myfile.csv
| search support_group="mygroup-Linux" u_sec_dom="Normal Secure"
| where !match(fqdn,"udc.net$") AND !match(fqdn,"htc.com$")
Or just a single search clause:
| inputlookup myfile.csv
| search support_group="mygroup-Linux" u_sec_dom="Normal Secure" NOT (fqdn IN("*udc.net","*htc.com")
You can also rewrite the IN() thusly:
(fqdn="*udc.net" OR fqdn="*htc.com")
The combined regex will work if you omit the spaces on either side of the |. The extra spaces become part of the regex and prevent matches.
There's no need for the final where command. Splunk by default will display all events that match ..
I am trying to extract the Get-Help comment headers from a PowerShell script...using PowerShell. The file I'm reading looks something like this:
<#
.SYNOPSIS
Synopsis goes here.
It could span multiple lines.
Like this.
.DESCRIPTION
A description.
It could also span multiple lines.
.PARAMETER MyParam
Purpose of MyParam
.PARAMETER MySecondParam
Purpose of MySecondParam.
Notice that this section also starts with '.PARAMETER'.
This one should not be captured.
...and many many more lines like this...
#>
# Rest of the script...
I would like to get all the text below .DESCRIPTION, up to the first instance of .PARAMETER. So the desired output would be:
A description.
It could also span multiple lines.
Here's what I've tried:
$script = Get-Content -Path "C:\path\to\the\script.ps1" -Raw
$pattern = '\.DESCRIPTION(.*?)\.PARAMETER'
$description = $script | Select-String -Pattern $pattern
Write-Host $description
When I run that, $description is empty. If I change $pattern to .*, I get the entire contents of the file, as expected; So there must be something wrong with my RegEx pattern, but I can't seem to figure it out.
Any ideas?
(get-help get-date).description
The `Get-Date` cmdlet gets a DateTime object that represents the current date
or a date that you specify. It can format the date and time in several Windows
and UNIX formats. You can use `Get-Date` to generate a date or time character
string, and then send the string to other cmdlets or programs.
(get-help .\script.ps1).description
the Select-String cmdlet works on entire strings and you have given it ONE string. [grin]
so, instead of fighting with that, i went with the -match operator. the following presumes you have loaded the entire file into $InStuff as one multiline string with -Raw.
the (?ms) stuff is two regex flags - multiline & singleline.
$InStuff -match '(?ms)(DESCRIPTION.*?)\.PARAMETER'
$Matches.1
output ...
DESCRIPTION
A description.
It could also span multiple lines.
note that there is a blank line at the end. you likely will want to trim that away.
In the words of #Mathias R. Jessen:
Don't use regex to parse PowerShell code in PowerShell
Use the PowerShell parser instead!
So, let's use PowerShell to parse PowerShell:
$ScriptFile = "C:\path\to\the\script.ps1"
$ScriptAST = [System.Management.Automation.Language.Parser]::ParseFile($ScriptFile, [ref]$null, [ref]$null)
$ScriptAST.GetHelpContent().Description
We use the [System.Management.Automation.Language.Parser]::ParseFile() to parse our file and ouput an Abstract Syntax Tree (AST).
Once we have the Abstract Syntax Tree, we can then use the GetHelpContent() method (exactly what Get-Help uses) to get our parsed help content.
Since we are only interested in the Description portion, we can simply access it directly with .GetHelpContent().Description
I'm trying to write a bash script that would be able to grep table names from across files (within a directory) that partially match a string.
For my case, I'd like to return all table references following a certain convention (case insensitive):
tblpl
tbljoin
tbldim
This would ideally return a list like this:
product.dbo.tblplColors
product..tblplMonograms
solr.dbo.tbljoinSkuCategory
Matching one table name format at a time would also be alright if that helped reduce some of the complexity. To clarify, this would return just the table names- not the file name/all of the file contents. It's safe to say the end of the table name would be delimited by a space since it's SQL.
Where I've started:
grep -rio 'tblpl*[^ ]' d:/sqldirectoryhere > c:/Users/foo/Desktop/tables.txt
Any help/pointers are appreciated here- thanks!
Edit: Both of these answers nailed my use case. I ended up adding the extended regex (so huge thanks for that recommendation) but I have to give credit to the person who wrote the bulk of it. Thanks all!
My extended use case ended up being a way to return this list of tables and then script it to a query-friendly format so I could throw these into a WHERE IN clause. In case anyone ever needs it:
grep -rioE --no-filename '[a-zA-Z_.]+\.tbl(pl|join|dim)[a-zA-Z_]+' {DIRECTORY_HERE} | sed -n 's/.*/\x27&\x27/; $! s/$/,/; 1 h; 1 ! H; $ { x; s/\n/ /g; p; }'
Returns formatted as: 'db.tblplColorSwatches', 'db.tbljoinCustomerSegment'...
It finds any mixed sequence of letters and periods followed by .tblpl or .tbljoin or .tbldim followed by one or more letters (see regex101 link)
try this regular expression:
[a-zA-Z.]+\.tbl(pl|join|dim)[a-zA-Z]+
I would use the -E flag to use extended regular expression:
grep -rioE '[a-z]*\.[a-z]*\.tbl(pl|join|dim)[a-z]*' d:/sqldirectoryhere
How do I use Select-String cmdlet to search a text file for a string which starts with a specific string, then contains random text and has another specific string towards the end of the line? I'm only interested in matches across a single line in the text file, not across the entire file.
For example I am searching to match both 'Set-QADUser' and 'WhatIf' on the same line in the file. And my example file contains the following line:
Set-QADUser -Identity $($c.ObjectGUID) -ObjectAttributes #{extensionattribute7=$ekdvalue} -WhatIf | Out-Null
How do I use Select-String along with a Regular Expression to locate the pattern in question? I tried using the following and it does work but it also matches other instances of either 'Set-QADUser' or 'WhatIf' found elsewhere in the text file and I only want to match instances when both search strings are found on the same line.
Select-String -path "test.ps1" -Pattern "Set-QADUser.*WhatIf" | Select Matches,LineNumber
To make this more complicated I actually want to perform this search from within the script file that is being searched. Effectively this is used to warn the user that the script being run is currently set to 'WhatIf' mode for testing. But of course the regEx matches the text from the actual Select-String cmd within the script when it's run - so it finds multiple matches and I can't figure out a very good way to overcome that issue. So far this is what I've got:
#Warn user about 'WhatIf' if detected
$line=Select-String -path $myinvocation.mycommand.name -Pattern "Set-QADUser.*WhatIf" | Select Matches,LineNumber
If ($line.Count -gt 1)
{
Write-Host "******* Warning ******"
Write-Host "Script is currently in 'WhatIf' mode; to make changes please remove '-WhatIf' parameter at line no. $($line[1].LineNumber)"
}
I'm sure there must be a better way to do this. Hope somebody can help.
Thanks
If you use the -Quiet switch on Select-String it will just return a boolean True/False, depending on whether it found a match or not.
-Quiet <SwitchParameter>
Returns a Boolean value (true or false), instead of a MatchInfo object. The value is "true" if the pattern is found; otherwise, the value is "false".
Required? false
Position? named
Default value Returns matches
Accept pipeline input? false
Accept wildcard characters? false