I have a request please, I am usng regular expression on notepad++ and i have my database and it contains lines like
test1 - Level : 12 - Role : Healer
test2 - Level : 30 - Role : Healer
test3 - Level : 35 - Role : Healer
test3 - Level : 162 - Role : Healer
I want it to remove any lines whose level <30 so the output should be
test2 - Level : 30 - Role : Healer
test3 - Level : 35 - Role : Healer
test3 - Level : 162 - Role : Healer
Thanks in advance
You may try the following find and replace in regex mode:
Find: ^.*Level : [12]?[0-9]\b.*\R?
Replace: (empty)
Here is a demo showing that the logic is working.
Related
I have been experimenting a lot with writing unit tests for alerts as per this: https://prometheus.io/docs/prometheus/latest/configuration/unit_testing_rules/#alerts-yml
I have some simple cases out, but now I am tackling rules that are less trivial. For example this:
abs(
avg_over_time(my_metrics{service_name="aService"}[1m])
-
avg_over_time(my_metrics{service_name="aService"}[3m])
)
/ stddev_over_time(my_metrics{service_name="aService"}[3m])
> 3
I have one file with the above rule and then this is in my test:
- interval: 1m
# Series data.
input_series:
- series: 'my_metrics{service_name="aService"}'
values: '0 0 0 0 1 0 0 0 0 '
alert_rule_test:
- eval_time: 3m
alertname: myalert
exp_alerts:
- exp_labels:
severity: warning
service_name: aService
exp_annotations:
summary: "some text"
description: "some other text"
I am not sure what my series should look like in order to test deviation from the mean. Is it even possible to test such rule?
Thank you
EDIT
I can have a succesful test if I set it > 0 as opposed to >3 I have tried to set a series of this sort:
'10+10x2 30+1000x1000'
but I cannot understand what would be the correct setup to have it triggered
This isn't a direct answer, rather a tip from someone who spent quite some time on these tests. Did you know that apart from testing alert expressions, you can unittest PromQL expressions as well? See how it can be useful:
evaluation_interval: 1m
tests:
- interval: 1m
input_series:
- series: test_metric
values: 1 1 1 10 1 1 1
promql_expr_test:
- expr: avg_over_time(test_metric[1m])
eval_time: 4m
exp_samples:
- value: #5.5
- expr: avg_over_time(test_metric[3m])
eval_time: 4m
exp_samples:
- value: #3.25
- expr: stddev_over_time(test_metric[3m])
eval_time: 4m
exp_samples:
- value: #3.897114317029974
I've split your alert expression into three separate, simple parts. If you run this unittest, you will see the commented-out values in the error message. From here it is not difficult to join pieces together and see why the alert is not happening. You can use that to build a working sequence of values.
Below the text want to extract timestamp align with UserId from the below line and group it
2020-10-12 12:30:22.540 INFO 1 --- [enerContainer-4] c.t.t.o.s.s.UserPrepaidService : Validating the user with UserID:1111 systemID:sys111
From below whole logs
2020-10-12 12:30:22.538 INFO 1 --- [ener-4] c.t.t.o.s.service.UserService : AccountDetails":[{"snumber":"2222","sdetails":[{"sId":"0474889018","sType":"Java","plan":[{"snumber":"sdds22"}]}]}]}
2020-10-12 12:30:22.538 INFO 1 --- [ener-4] c.t.t.o.s.service.ReceiverService : Received userType is:Normal
2020-10-12 12:30:22.540 INFO 1 --- [enerContainer-4] c.t.t.o.s.s.UserPrepaidService : Validating the user with UserID:1111 systemID:sys111
2020-10-12 12:30:22.540 INFO 1 --- [enerContainer-4] c.t.t.o.s.util.CommonUtil : The Code is valid for userId: 1111 systemId: sys111
2020-10-12 12:30:22.577 INFO 1 --- [enerContainer-4] c.t.t.o.s.r.Dao : Saving user into dB ..... with User-ID:1111
....
same repetitive line
Below is my SPL search commands it returns only userid group by from that specific line.
But I want the time stamp as well from that line and group by it with time chart
index="tis" logGroup="/ecs/logsmy" "logEvents{}.message"="*Validating the user with UserID*" | spath output=myfield path=logEvents{}.message | rex field=myfield "(?<=Validating the user with UserID:)(?<userId>[0-9]+)(?= systemID:)" | table userId | dedup userId | stats count values(userId) by userId
Basically I tired the below
(^(?<dtime>\d{4}-\d{1,2}-\d{1,2}\s+\d{1,2}:\d{1,2}:\d{1,2}\.\d+) )(?<=Validating the user with UserID:)(?<userId>[0-9]+)(?= systemID:)
but it gave all the time stamp not specifically the line I mentioned above
You placed the lookaround right after matching the timestamp pattern, but you have to first move to the postition where the lookbehind is true.
If you want both values, you can match Validating the user with UserID: and systemID: instead of using a lookaround.
If there are leading whitspace chars, you could match them with \s or [^\S\r\n]*
^\s*(?<dtime>\d{4}-\d{1,2}-\d{1,2}\s+\d{1,2}:\d{1,2}:\d{1,2}\.\d+).*\bValidating the user with UserID:(?<userId>[0-9]+) systemID:
Regex demo
I want to transform unstructured data into structured form. The data is of the following form - (showing 1 row of data)
Agra - Ahmedabad### Sat, 24 Jan### http://www.cleartrip.com/m/flights/results?from=AGR&to=AMD&depart_date=24/01/2015&adults=1&childs=0&infants=0&class=Economy&airline=&carrier=&intl=n&page=loaded Air India### 15:30 -
14:35### 47h 5m, 3 stops , AI 406### Rs. 30,336###
and I want to extract the data in the following format using APACHE PIG
(Agra - Ahmedabad,Sat, 24 Jan,http://www.cleartrip.com/m/flights/results?from=AGR&to=AMD&depart_date=24/01/2015&adults=1&childs=0&infants=0&class=Economy&airline=&carrier=&intl=n&page=loaded,Air India,15:30 - 14:35,47h 5m, 3 , AI 406 , 30,336)
I am using the following lines in APACHE PIG :
A = LOAD '/prodqueue_cleartrip_23rdJan15.txt' using PigStorage as (value: chararray);
B = foreach A generate REGEX_EXTRACT_ALL('value', '([^#]+)#+\\s+([^#]+)#+\\s+([^\\s]+)\\s+([^#]+)#+\\s+([0-9]{1,2}:[0-9]{1,2}\\s-\\n[0-9]{1,2}:[0-9]{1,2})#+\\s+([^,]+),\\s([0-9]+)\\sstops\\s,\\s([^#]+)#+\\s+Rs.\\s([^#]+)#+
');
C = LIMIT B 5;
The output I am getting is this :
()
()
()
()
()
What is the mistake?
This may be just a typo for your question but
REGEX_EXTRACT_ALL('value', '([^#]+)#+\\s...
would just search a literal "value". You probably want to take out the single quote so that it matches the field value.
REGEX_EXTRACT_ALL(value, '([^#]+)#+\\s...
I am trying to parse a custom input file for a simulation code I am writting. It consist of nested "objects" with properties, values (see the link).
Here is an example file and the regex I am using currently.
([^:#\n]*):?([^#\n]*)#?.*\n
It is made such that each match is a line, with two capture group, one for the property and one for its value. It also excludes "#" and ":" from the character set, as they correspond to a comment delimiter and a property:value delimiter respectively.
How can I modify my regex so as to match the structure recursively? That is if line n+1 has an identation level higher than line n, it should be matched as a subgroup of line n's match.
I am working on Octave, which uses PCRE regex format.
I asked if you have control over the data format because as it is, the data is very easy to parse with YAML instead of regex.
The only problem is that the object is not well formed:
1) Take the regions object for example, it has many attributes called layer all of them. I think your intention is to build a list of layers instead of a lot of properties with the same name.
2) Consider now each layer property that has a corresponding value. Following each layer are orphan attributes that I presume belong to each layer.
With these ideas in mind. If you form your object following YAML rules, it would be a breeze to parse it.
I know that you are working in Octave, but consider the modifications I made to your data, and how easy it is to parse it, in this case with python.
DATA AS YOU HAVE IT NOW
case :
name : tandem solar cell
options :
verbose : true
t_stamp : system
units :
energy : eV
length : nm
time : s
tension : V
temperature: K
mqty : mole
light : cd
regions :
layer : Glass
geometry:
thick : 80 nm
npoints : 10
optical :
nk_file : vacuum.txt
layer : FTO
geometry:
thick : 10 nm
npoints : 10
optical :
nk_file : vacuum.txt
MODIFIED DATA TO COMPLY WITH YAML SYNTAX
case :
name : tandem solar cell
options :
verbose : true
t_stamp : system # a sample comment
units :
energy : eV
length : nm
time : s
tension : V
temperature: K
mqty : mole
light : cd
regions :
- layer : Glass # ADDED THE - TO MAKE IT A LIST OF LAYERS
geometry : # AND KEEP INDENTATION PROPERLY
thick : 80 nm
npoints : 10
optical :
nk_file : vacuum.txt
- layer : FTO
geometry:
thick : 10 nm
npoints : 10
optical :
nk_file : vacuum.txt
With only these instruction you get your object parsed:
import yaml
data = yaml.load(text)
""" your data would be parsed as:
{'case': {'name': 'tandem solar cell',
'options': {'t_stamp': 'system', 'verbose': True},
'regions': [{'geometry': {'npoints': 10, 'thick': '80 nm'},
'layer': 'Glass',
'optical': {'nk_file': 'vacuum.txt'}},
{'geometry': {'npoints': 10, 'thick': '10 nm'},
'layer': 'FTO',
'optical': {'nk_file': 'vacuum.txt'}}],
'units': {'energy': 'eV',
'length': 'nm',
'light': 'cd',
'mqty': 'mole',
'temperature': 'K',
'tension': 'V',
'time': 's'}}}
"""
I am trying to select everything after CORP ACT OPTION NO. however my regular expression stops after it meets a /
My reg expression atm to extract the info is CORP ACT OPTION NO.([^/]*)
CORP ACT REFERENCE : 007XS0212069115
SENDER'S REFERENCE : 1212070800330001
FUNCTION OF MESSAGE : NEW MESSAGE
CORP ACT EVENT : INTEREST PAYMENT
PLACE OF SAFEKEEPING : US
ISIN : XS0212069115
ISIN/DESCRIPTION : KFW 4.750 071212 GBP
METHOD OF INTEREST COMPUTATION : A006
EX-DATE : 20121207
RECORD DATE : 20121206
CORP ACT OPTION NO. : 001
CORPORATE ACTION OPTION CODE : CASH
CURRENCY OPTION : GBP
RESULTING AMT : GBP617,5
PAYMENT DATE : 20121207
EXCHANGE RATE : GBP/GBP/1,
INTEREST RATE : 4,75
SAFEKEEPING ACCOUNT : 000000000000
CONFIRMED BALANCE : FAMT/13000,
CREDIT/DEBIT IND : CREDIT
How can I select everything? Many thanks.
Just use \s\S in the character group:
CORP ACT OPTION NO\.([\s\S]*)
See it here in action: http://regexr.com?33vd0