Logback replace regex to detect empty values - regex

I am trying to use logback's replace feature to not have empty values printed in my MDC log pattern.
http://logback.qos.ch/manual/layouts.html#replace
I am trying to follow an example from here
http://logogin.blogspot.com/2013/04/logback-mdc-and-empty-values.html
Some background
For 90% of the time my log pattern prints
2014-08-28 11:30:27,014 emp:Peter org:IT Expense submitted
For 5% of the time it prints
2014-08-28 11:30:27,014 emp: org: Cleanup jobs.
This is because the emp and org do not need to be supplied on MDC in the latter case.
For these cases, I want the emp: and org: to not be present at all in the log line.
Desired
2014-08-28 11:30:27,014 Cleanup jobs.
Possible solution with replace
Here is my variable and the appender I am using. The idea is that the mdcPattern will resolve to an empty string for no emp and org values.
<variable scope="context" name="mdcPattern" value="%replace( emp:%X{empName} org:%X{orgName} ) {'[a-z]+:( |$)', ''}"/>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d ${mdcPattern} %thread %-5level %logger{25} - %msg%n</pattern>
</encoder>
</appender>
However the replace regex does not work. I see log lines as:
2014-08-28 11:30:27,014 emp: org: {'[a-z]+:( |$
My regex is a bit weak. I can't seem to understand why the replace pattern is appearing as is in my log line. Any help is greatly appreciated.

With help from the author of the original blog post, I've managed to get this to work.
He has provided another example on GitHub.
http://logogin.blogspot.com/2013/04/logback-mdc-and-empty-values.html
https://gist.github.com/logogin/ff44c254f655340b653c
I had extra spaces in the replacement pattern which I've removed.
Eg: {orgName} ) {'[a-z]+
To: {orgName}){'[a-z]+

Related

Find and Replace only portion of Regex Expression in IDE

I'm trying to fix malformed XML with Regex in VS Code.
Problem code is self-closing tags. On build of the project, these get nested as children. So I just need a simple regex pattern to match certain elements only (polygon, here), and ignore others.
Using Regex pattern:
(?:\.[0-9]+\"\s)\/\>
I want to search: tags containing a decimal point, number(s), and end double quote.
I want to replace: everything after that.
Problem -- it replaces the entire part, instead of the chunk in the second expression ()
Example:
...
<stop offset="1" stop-color="#7ccb13" />
<stop offset="1" stop-color="#7ccb55" />
...
<polygon points="393.38 285.04 394.44 284.63 393.67 283.79 394.61 284.38 394.59 283.31 394.94 284.34 395.79 283.54 395.21 284.5 396.3 284.49 395.28 284.83 396.16 285.64 395.05 285.05 395.12 286.27 394.73 285.1 393.89 285.88 394.43 284.99 393.38 285.04" />
<polygon points="159.41 439.9 160.88 439.35 159.82 438.19 161.12 439.01 161.08 437.52 161.57 438.95 162.74 437.84 161.94 439.16 163.46 439.15 162.04 439.63 163.25 440.75 161.72 439.92 161.82 441.61 161.28 440 160.12 441.07 160.87 439.84 159.41 439.9" />
<polygon points="180.71 444.75 182.18 444.2 181.11 443.04 182.41 443.86 182.38 442.37 182.87 443.8 184.04 442.69 183.24 444.01 184.75 444 183.33 444.48 184.55 445.59 183.02 444.77 183.12 446.46 182.57 444.85 181.42 445.92 182.16 444.69 180.71 444.75" />
...
Desired output -- closing tags:
I think (<polygon.+)(\/>) should work, and replace with $1></polygon>.
example

regex ansible lineinfile subsection

I am fairly new to ansible and have been solving the following problem with a shell script, but the proper way I believe is to use the lineinfile module, just not sure how to accomplish this.
Let's say I have a file with the following text in it.
<cpu>
<alarm>
active = yes
</alarm>
</cpu>
<disk>
<alarm>
active = yes
<fixed>
<#>
active = yes
description = File system /
<inode_error>
active = yes
threshold = 2
message = InodeError
</inode_error>
</#>
<#boot>
active = yes
description = File system /boot
percent = yes
<error>
active = yes
threshold = 5
message = DiskError
</error>
</#boot>
</fixed>
</alarm>
</disk>
I want to make sure the following section is set correctly.
<disk><alarm><fixed><#boot><error>"threshold = 2"</error></#boot></fixed></alarm></disk>
is there a way to only (modify/make sure exists) that line, normally this file is much larger with many more sections, but I erased some so the question is readable.
Update: Modifying this as it is not valid XML and the XML module will not parse the file correctly.
Thanks!
lineinfile scans file per line, so you can't define complex multiline regexp for context definition.
replace module support multiline regexp.
If you have threshold = X in the file and want to be sure it is set to specific value, you can use this regexp:
- replace:
path: ./test.txt
regexp: '(<disk>[\s\S]*<alarm>[\s\S]*<#boot>[\s\S]*<error>[\s\S]*)(threshold\s*=\s*\d+)([\s\S]*?<\/error>)'
replace: '\1threshold = 2\3'
It searches for line threshold\s*=\s*\d+ inside <disk>...<alarm>...<#boot>...<error>....
This code is idempotent – so if threshold = 2, then nothing is done.
But if there is no threshold = X string, it will fail. You should construct more complex regular expression for that case.
you could use the lineinfile module (http://docs.ansible.com/ansible/latest/lineinfile_module.html) where you could write a regex to modify/add the line and use the validate function to run a command to ensure that the xml file has the proper syntax.
If you are on Ansible 2.4 you can use the xml module (https://docs.ansible.com/ansible/2.4/xml_module.html) and use the attribute parameter to check if the xpath in xml file is set, like that:
- name: Read attribute value
xml:
path: /foo/bar.xml
xpath: /business/website/validxhtml
content: attribute
attribute: validatedon
register: xmlresp
regards

Extract username from forward slash separated text

I need to extract a username from the log below via regex for a log collector.
Due to the nature of the logs we're getting its not possible to define exactly how many forward slashes are going to be available and I need to select a specific piece of data, as there are multiple occurances of similar formatted data.
Required data:
name="performedby" label="Performed By" value="blah.com/blah/blah blah/blah/**USERNAME**"|
<46>Jun 23 10:38:49 10.51.200.76 25113 LOGbinder EX|3.1|success|2016-06-23T10:38:49.0000000-05:00|Add-MailboxPermission Exchange cmdlet issued|name="occurred" label="Occurred" value="6/23/2016 10:38:49 AM"|name="cmdlet" label="Cmdlet" value="Add-MailboxPermission"|name="performedby" label="Performed By" value="blah.com/blah/blah blah/blah/USERNAME"|name="succeeded" label="Succeeded" value="Yes"|name="error" label="Error" value="None"|name="originatingserver label="Originating Server" value="black"|name="objectmodified" label="Object Modified" value="blah/blah/USERNAME"|name="parameters" label="Parameters" value="Name: Identity, Value: [blah]Name: User, Value: [blah/blah]Name AccessRights, Value: [FullAccess]Name: InheritanceType, Value: [All]"|name="properties" label="Modified Properties" value="n/a"|name="additionalinfo" label="Additional Information"
I've tried a few different regex commands but I'm not able to extract the necessary information without exactly stating how many / there will be.
blah\.com[.*\/](.*?)"\|name
Try this :
blah\.com.*\/(.*?)"\|
Check here
If your username format is this :
value="abc.xyz/something/something/..../USERNAME"
then use this :
\..*\/(.*?)"
check here
Possible solution:
value="[a-z\.\/]*\/(.*)"
(The first capture group is the username)
Working example:
https://regex101.com/r/qZ0zC8/2
Mayby like this?
blah.(\w+\/)+\K([\w]+)
It's catch Username but since it's between ** so I also match them
tested in notepad++

Regex JSON response Gatling stress tool

Wanting to capture a variable called scanNumber in the http response loking like this:
{"resultCode":"SUCCESS","errorCode":null,"errorMessage":null,"profile":{"fullName":"TestFirstName TestMiddleName TestLastName","memberships":[{"name":"UA Gold Partner","number":"123-456-123-123","scanNumber":"123-456-123-123"}]}}
How can I do this with a regular experssion?
The tool I am using is Gatling stress tool (with the Scala DSL)
I have tried to do it like this:
.check(jsonPath("""${scanNumber}""").saveAs("scanNr")))
But I get the error:
---- Errors --------------------------------------------------------------------
> Check extractor resolution crashed: No attribute named 'scanNu 5 (100,0%)
mber' is defined
You were close first time.
What you actually want is:
.check(jsonPath("""$..scanNumber""").saveAs("scanNr")))
or possibly:
.check(jsonPath("""$.profile.memberships[0].scanNumber""").saveAs("scanNr")))
Note that this uses jsonPath, not regular expressions. JsonPath should more reliable than regex for this.
Check out the JsonPath spec for more advanced usage.
use this regex to match this in anywhere in json:
/"scanNumber":"[^"]+"/
and if you want to match just happens in structure you said use:
/\{[^{[]+\{[^{[]+\[\{[^{[]*("scanNumber":"[^"]+")/
Since json fields may change its order you should make your regex more tolerant for those changes:
val j = """{"resultCode":"SUCCESS","errorCode":null,"errorMessage":null,"profile":{"fullName":"TestFirstName TestMiddleName TestLastName","memberships":[{"name":"UA Gold Partner","number":"123-456-123-123","scanNumber":"123-456-123-123"}]}}"""
val scanNumberRegx = """\{.*"memberships":\[\{.*"scanNumber":"([^"]*)".*""".r
val scanNumberRegx(scanNumber) = j
scanNumber //String = 123-456-123-123
This will work even if the json fields will be in different order (but of course keep the structure)

JMeter Response Assertion: incorporating property into pattern test

I'd like to setup a JMeter test plan to suggest whether a web site (URL) is Drupal-based (based completely on the HTTP response from the site) and compare it with existing data that I have on the environment. (I realize that using an HTTP approach, as opposed to say examining the site's file system, is "iffy" but I'm curious how useful the approach is)
The JMeter command line might look like this:
JMeter -t "DrupalAssertions.jmx" -Jurl=http://my.dot.com -Jdrupal=true
where I provide the URL to test and an additional property "drupal" indicating my best guess on whether the site is Drupal-based.
In my test plan, I add an HTTP Request to return the HTML content of the page for the URL. I'm then able to successfully add a Response Assertion that tests a pattern (say (?i)(drupal) for a sadly lacking pattern) to see if it's contained in the response.
That much works fine, or as expected, but what I'd like to do is to compare the value of the "drupal" property against the result of that pattern test in that same Response Assertion. I know I'm missing something simple here, but I'm not seeing how to do that.
I want to try to use an expression like this:
(?i)(drupal) == ${__P(drupal)}
in a pattern, but that doesn't work. The name of the Compare Assertion looks promising, but I don't see how to incorporate the property into a comparison.
Update: The approach suggested by PMD UBIK-INGENIERIE does work. I used a Regular Expression Extractor like this:
<RegexExtractor guiclass="RegexExtractorGui" testclass="RegexExtractor" testname="Extract Drupal in Response" enabled="true">
<stringProp name="RegexExtractor.useHeaders">false</stringProp>
<stringProp name="RegexExtractor.refname">drupalInResponse</stringProp>
<stringProp name="RegexExtractor.regex">(.*drupal.*)</stringProp>
<stringProp name="RegexExtractor.template">$0$</stringProp>
<stringProp name="RegexExtractor.default">__false__</stringProp>
<stringProp name="RegexExtractor.match_number">1</stringProp>
</RegexExtractor>
followed by this BeanShell Assertion:
// Variable "drupalInResponse" is "__false__" by default
if ( !(vars.get("drupalInResponse").equals("__false__") ) ) {
vars.put("drupalInResponse","true");
}
else {
vars.put("drupalInResponse","false");
}
print("\n\nThe value of property 'drupal' is: " + props.get("drupal") + "\n");
print("\n\nThe value of variable 'drupalInResponse' is: " + vars.get("drupalInResponse") + "\n");
if (vars.get("drupalInResponse").equals( props.get("drupal") ) ) {
print("Site Drupalness is consistent with your beliefs");
}
else {
print("You're wrong about the site's Drupalness");
Failure = true;
FailureMessage = "Incorrect Drupal assumption";
}
In the Regular Expression Extractor, I'd set a default value that I felt wouldn't be matched by my pattern of interest, then did an ugly verbose Java comparison with the "drupal" property in the BeanShell Assertion.
Wish somehow that the assertion could be made in a single component rather than it having two parts, but you can't argue with "working" :)
You can use a regexp extractir with your first pattern
Then use a Beanshell assertion which will use your variable and compare it to drupal property.