Make sure the presence of a particular character in a string - regex

I'm using the following regex to parse my application log file to search for particular string
\[\s*\b(?:[0-9A-Za-z][0-9A-Za-z-_.#]{0,200})(?:\.(?:[0-9*A-Za-z][0-9A-Za-z-_.#]{0,200}))*(\.?|\b)\s*]
This works fine but now we need to make sure that the string "must" contain "-" character to match. I'm confused to add this condition to the original regx.
Any pointers would be helpful.
Thanks and Regards,
Santhosh

The regex matches a string inside square brackets, [ and ], and may only consist of non-[ and non-] symbols.
You can easily add a positive lookahead restriction after the opening [ like check if the next characters other than ] and [ are followed with -:
\[ # opening [
(?=[^\]\[]*-) # There must be a hyphen in [...]
\s* # 0+ whitespaces
\b(?:[0-9A-Za-z][0-9A-Za-z-_.#]{0,200}) # Part 1 (with obligatory subpattern)
(?:\. # Part 2, optional
(?:[0-9*A-Za-z][0-9A-Za-z-_.#]{0,200})
)*
(\.?|\b) # optional . or word boundary
\s* # 0+ whitespaces
] # closing ]
See the regex demo
And a one-liner:
\[(?=[^\]\[]*-)\s*\b(?:[0-9A-Za-z][0-9A-Za-z-_.#]{0,200})(?:\.(?:[0-9*A-Za-z][0-9A-Za-z-_.#]{0,200}))*(\.?|\b)\s*]
Tip: use the verbose /x modifier to split the pattern into separate multiline blocks for analysis, it will help you in the future when you need to modify the pattern again.
If you need to match only if - or # is present inside [...], modify the lookahead as (?=[^\]\[]*[-#]). For a more general case, use (?=[^\]\[]*(?:one|another|must-be-present)) alternatives inside an additional group inside the lookahead.

Updated answer - Assertion
In this case, the better way to do it is to use an assertion consisting of checking only the position's expected to match the character in question.
I know it's simple, but using the outter pseudo-anchor text \[ ... \] as
a delimiter that cannot exist in the body is a rarity.
You should always try to avoid doing it like this.
Things change, your input could change.
The rule to follow in validation of known characters that are Mid-String is to use only them
when using an assertion validator.
This avoids the necessity of relying on what is not there at the moment ie, not a ],
but should rely on what is there.
Again, this pertains to mid-string matching.
BOL/EOL is a different thing entirely ^$, and is a more permanent construct
with which to leverage.
It's always better to code smarter.
\[\s*\b(?=[0-9A-Za-z][0-9A-Za-z_.#]{0,199}-|[0-9A-Za-z][0-9A-Za-z_.#]{0,200}(?:\.[0-9*A-Za-z][0-9A-Za-z_.#]{0,200})*\.[0-9*A-Za-z][0-9A-Za-z_.#]{0,199}-)(?:[0-9A-Za-z][0-9A-Za-z_.#-]{0,200})(?:\.(?:[0-9*A-Za-z][0-9A-Za-z_.#-]{0,200}))*(\.?|\b)\s*\]
Using Conditionals
If your engine supports conditionals, the easy way is to not rely on a fluke
of pseudo anchor text, ie. [..].
\[\s*\b[0-9A-Za-z](?:[0-9A-Za-z_.#]|(-)){0,200}(?:\.(?:[0-9*A-Za-z](?:[0-9A-Za-z_.#]|(-)){0,200}))*(\.?|\b)\s*\](?(1)|(?(2)|(?!)))
Expanded
\[ \s* \b
[0-9A-Za-z]
(?:
[0-9A-Za-z_.#]
| ( - ) # (1)
){0,200}
(?:
\.
(?:
[0-9*A-Za-z]
(?:
[0-9A-Za-z_.#]
| ( - ) # (2)
){0,200}
)
)*
( \.? | \b ) # (3)
\s* \]
(?(1) # Fail if no dash found
| (?(2)
| (?!)
)
)

This conditional would work if just want to make sure that - occurs within your string before running your block of code.
if (myString.indexOf('-') >= 0) {
//your code
}

If you have to have a single hyphen, you'll have to either repeat most of the pattern, or check for it in a second phase:
if re.match(pattern, line):
if not '-' in line:
raise MissingDash('No dash in line: {}'.format(line))
I'd suggest adding the second check, since adding the requirement to the regex would make it even more horrible to read.

Related

Regex word can be optional but only if it matches the characters

Following pattern: (v[0-9]{1,2}\.[0-9]{1,2}\.[0-9]{1,2})(-[0-9]{1,2})?((-schema)?(-dev)?)((-schema)?(-dev)?) from http://regexr.com/ is meant to be used in a shell script with grep and does match the following strings (working example):
Hello I am a text and this is my v1.12.33-32 version
Hello I am a text and this is my v1.12.33-dev version
Hello I am a text and this is my v1.12.33-dev-schema version
Hello I am a text and this is my v1.12.33-schema version
Hello I am a text and this is my v1.12.33-3-schema version
and so forth
So I made the words schema and dev optional. They can be ommitted or used in a arbitrary order. What I don't what is this:
Hello I am a text and this is my v1.12.33-foo version
or Hello I am a text and this is my v1.12.33-asfs version
to match.
I want the option to be a bit more constrained. At the moment the Regex is still matching the stuff that...well actually matches.
This for example:
Hello I am a text and this is my v1.123.33
results in an empty string while this:
`Hello I am a text and this is my v1.12.33-bla"
still results in v.1.12.33
Is this because of the grouping I made? So at least the fully matching groups will be taken for the returned match-string?
To match only the version string, disallow extra trailing tags, yet allow trailing unmatched text, you need a regex language that supports lookahead. Standard grep / egrep regexes do not support lookahead.
You have two options:
Since you seem to be relying on GNU grep anyway, you could use a Perl regex, such as
v[0-9]{1,2}(\.[0-9]{1,2}){2}(-[0-9]{1,2})?((-schema(-dev)?)?|(-dev(-schema)?)?)?(?!\S)
The negative lookahead at the end allows the match to appear at the end of the line, but also requires that if it does not end the line then the next character following the match must be whitespace (which is not itself included in the match).
You could give up on completely isolating the target text via -o, and instead allow the pattern to match the trailing context, too:
v[0-9]{1,2}(\.[0-9]{1,2}){2}(-[0-9]{1,2})?((-schema(-dev)?)?|(-dev(-schema)?)?)?(\s.*)?$
In this case, you could isolate the target text in a second step, by stripping off any tail beginning with whitespace.
Note that neither of these pays attention to text preceeding the match. You have similar options for handling that portion as you do for handling the trailing portion.
The problem seems to be all the optional expressions lurking at the
edge (end).
You can solve that a few ways, but none are %100 because you'd need
more rules to control what matches.
It's not like you can say no - is allowed afterword, the engine will
backtrack to one of the range digits {1,2} to make a match.
What seems to work for now is passing on a whitespace end edge
or matching the dev/schema items.
(v[0-9]{1,2}\.[0-9]{1,2}\.[0-9]{1,2})(-[0-9]{1,2})?(?:(?!\S)|(-(schema|dev)(?:-(schema|dev))?))
Expanded
( # (1 start)
v [0-9]{1,2}
\. [0-9]{1,2}
\. [0-9]{1,2}
) # (1 end)
( - [0-9]{1,2} )? # (2)
(?:
(?! \S ) # Whitespace boundary
| # or,
( # (3 start)
-
( schema | dev ) # (4)
(?:
-
( schema | dev ) # (5)
)?
) # (3 end)
)
edit
If you want to avoid matching the same schema|dev word twice, just add
a negative assertion of group 4, before capture group 5 above.
(v[0-9]{1,2}\.[0-9]{1,2}\.[0-9]{1,2})(-[0-9]{1,2})?(?:(?!\S)|(-(schema|dev)(?:-(?!\4)(schema|dev))?))
Expanded
( # (1 start)
v [0-9]{1,2}
\. [0-9]{1,2}
\. [0-9]{1,2}
) # (1 end)
( - [0-9]{1,2} )? # (2)
(?:
(?! \S ) # Whitespace boundary
| # or,
( # (3 start)
-
( schema | dev ) # (4)
(?:
-
(?! \4 ) # Not same word twice
( schema | dev ) # (5)
)?
) # (3 end)
)
Since regular expressions are open-ended, you need to specify with $ where you want the match to end, so you don't let the regex engine silently ignore trailing junk.
With only two tags in the optional set, I would just enumerate the 4 possibilities:
(v[0-9]{1,2}\.[0-9]{1,2}\.[0-9]{1,2})(-[0-9]{1,2})?(-schema|-dev|-dev-schema|-schema-dev)?$
My version:
grep --perl-regexp \
'\bv(?:\d{1,2}\.){2}\d{1,2}(?:\-\d{1,2})?(?:\-(?:schema|dev))?(?:\s|$)' \
path/to/file
Where
the first \b is a word boundary(you might want to make it stricter);
(?: ... ) expressions are non-capturing groups;
\s|$ is either a space character, or the end of line
The rest is just refactored for simplicity.
The expression allows only schema, or dev at the "end".

Remove the text outside the first brackets in R

I know that it was asked a lot of times, but I've tried to adapt the other answers to my need and I was not able to make it work using SKIP and FAIL (I'm a bit confused, I've to admit)
I'm using R actually.
The url I need to clean is:
url <- "posts.fields(id,from.fields(id,name),message,comments.summary(true).limit(0),likes.summary(true).limit(0))"
and I need to retain only the content inside the first brackets that are always prefixed by the word "fields" (while "posts" may vary). In other words something like
id,from.fields(id,name),message,comments.summary(true).limit(0),likes.summary(true).limit(0)
As you may see there're some nesting inside. But I eventually could change my source code to accept this string too (removing every parhentesis by every prefix)
id,from,message,comments,likes
I don't know on how to remove the trailing parhentesis which balances the first.
If it's good enough to just remove everything up to and including the first open parenthesis and also remove the last close parenthesis and thereafter then:
sub("^.*?\\((.*)\\)[^)]*$", "\\1", url)
Note:
If it's good enough to just remove the first open parenthesis and last close parenthesis then try this:
sub("\\((.*)\\)", "\\1", url)
Using lazy .* instead of greedy:
sub(".*?fields\\((.*)\\)", "\\1", url)
[1] "id,from.fields(id,name),message,comments.summary(true).limit(0),likes.summary(true).limit(0)"
You need to use a recursive pattern:
sub("[^.]*+(?:\\.(?!fields\\()[^.]*)*+\\.fields\\(([^()]*+(?:\\((?1)\\)[^()]*)*+)\\)(?s:.*)", "\\1", url, perl=T)
demo
details:
# reach the dot before "fields("
[^.]*+ # all except a dot (possessive)
(?: # open a non-capturing group
\\. # a literal dot
(?!fields\\() # not followed by "fields("
[^.]* # all except a dot
)*+ # repeat the group zero or more times
\\.fields\\(
# match a content between parenthesis with any level of nesting
( # open the capture group 1
[^()]*+ # 0 or more character that are not brackets (possessive)
(?: # open a non capturing group
\\(
(?1) # recursion in group 1
\\) #
[^()]* # all that is not a bracket
)*+ # close the non capturing group and repeat 0 or more time (possessive)
) # close the capture group 1
\\)
(?s:.*) # end of the string
Possessive quantifiers are used here to limit the backtracking when for any reason a part of the pattern fails.

Proper validation of the Name entered by a user

I am trying to validate the Name entered by a user. I need to make sure the Name entered is 'literally' valid. I tried many regular expressions within my limited knowledge, none of them seem to work fully. For example /^[^.'-]+([a-zA-Z]+|[ .'-]{1})$/
Since the PHP website I'm working on, is fully in English, only English names are allowed.
The rules applicable to filtering the name are:
A name may contain any of these characters: [a-zA-Z .'-]
The name may start only with an Alphabet or an Apostrophe
Any of the characters in [ .'-] may not occur more than once in a stretch, ie., no '---' or '--'
A space should not follow - or ' nor come immediately before . or -
Can anyone please provide the proper regular expression to implement these?
Here's a regex to solve your problem (demo at regex101):
/^(?!.*(''| |--|- |' | \.| -|\.\.|\n))['a-z][- '.a-z]*$/gi
Breakdown:
(?!.*(''| |--|- |' | \.| -|\.\.|\n)) negative lookahead to ensure that no doubled characters are found
['a-z] start with one of these characters
[- '.a-z]* the rest can also include spaces and dashes, and are not required (* instead of +)
This should work for you:
/^'?([a-z]+((\. )|( ')|['\- \.])?)+$/i
Demo at regex101.com
Explanation:
'? Optionally start with an apostrophe
([a-z]+((\. )|( ')|['\- \.])?)+ Afterwards allow 1-n groups of at least 1 alphabetic character, followed by certain (both) valid combinations of special characters. These are ". ", " '" (quotes to see the spaces) or any single special character.
/i Match case insensitive, otherwise you'd just specify [a-zA-Z] instead of [a-z].
I am not sure if John,.Doe or ' should be considered valid names. With my expression they would not.
You could probably get better performance by limiting all the backtracking a giant assertion would have to do.
Something like this..
# (?i)^(?=[a-z'])(?:[a-z]+|([ ](?!-)(?<!'[ ])|[.'-])(?!\1))*$
(?i) # Modifier, case insensitive
^ # BOS
(?= [a-z'] ) # Starts with an alpha or apos '
(?: # Cluster
[a-z]+ # 1 to many alpha's
| # or
( # (1 start), Capture group, one of [ .'-]
[ ] # Single space
(?! - ) # not a dash - ahead
(?<! ' [ ] ) # nor apos space '[ ] behind
| # or,
[.'-] # Single dot . or apos ' or dash -
) # (1 end)
(?! \1 ) # Backref capt grp 1,
# not adjacent duplicate of one of [ .'-]
)* # Cluster end, do 0 to many times
$ # EOS

Regex Match Multiline Chat Messages

I am attemping to use regex to parse chat logs (namely Skype messages). So the regex I am currently using matches Skype logs correctly....as long as they don't have new lines.
So I tried adding the s modifier to the end, but this now makes it match everything (because its now multiline). So I was wondering if there was a way to both allow multiline, but stop before the [ at the beginning of Skype messages.
My regex is here: https://regex101.com/r/nL0vO9/1
You can use a tempered greedy Token:
\[(?:(?!\n\[).)*
Note you also need to include g modifier so you don't stop on first match
See Demo
Like #sln pointed out, if you want to keep new lines use this instead:
\[(?:.(?<!\n\[))*
Set the flags to //mg multi-line and global. Don't use the s Dot all flag.
edit: I guess you could use something simpler if you don't care about validating/parsing out the time/name/message parts. Either #CrayonViolent or #RodrigoLópez should work for that.
# ^\[([^\r\n\]]*)\]([^:\r\n]*):((?:(?!^\[).*(?:\r?\n)*)*)
^ # BOL
\[
( [^\r\n\]]* ) # (1), Time
\]
( [^:\r\n]* ) # (2), Person
:
( # (3 start), Message
(?: # Cluster group
(?! ^ \[ ) # Assert, not BOL and [
.* # Get all to end of line
(?: \r? \n )* # Optional, Get 1 to many line-breaks
)* # End cluster, do 1 to many times
) # (3 end)

Match double hyphens in comments of malformed XML

I'm to parse XML files that do not conform to the "no double hyphens in comments" -standard, which makes MSXML complain. I am looking for a way of deleting offending hyphens.
I am using StringRegExpReplace(). I attempted following regular expressions:
<!--(.*)--> : correctly gets comments
<!--(-*)--> : fails to be a correct regex (also tried escaping and using \x2D)
Given the right pattern, I would call:
StringRegExpReplace($xml_string,$correct_pattern,"") ;replace with nothing
How to match remaining extra hyphens within an XML comment, while leaving the remaining text alone?
You can use this pattern:
(?|\G(?!\A)(?|-{2,}+([^->][^-]*)|(-[^-]+)|-+(?=-->)|-->[^<]*(*SKIP)(*FAIL))|[^<]*<+(?>[^<]+<+)*?(?:!--\K|[^<]*\z\K(*ACCEPT))(?|-*+([^->][^-]*)|-+(?=-->)|-?+([^-]+)|-->[^<]*(*SKIP)(*FAIL)()))
details:
(?|
\G(?!\A) # contiguous to the precedent match (inside a comment)
(?|
-{2,}+([^->][^-]*) # duplicate hyphens, not part of the closing sequence
|
(-[^-]+) # preserve isolated hyphens
|
-+ (?=-->) # hyphens before closing sequence, break contiguity
|
-->[^<]* # closing sequence, go to next <
(*SKIP)(*FAIL) # break contiguity
)
|
[^<]*<+ # reach the next < (outside comment)
(?> [^<]+ <+ )*? # next < until !-- or the end of the string
(?: !-- \K | [^<]*\z\K (*ACCEPT) ) # new comment or end of the string
(?|
-*+ ([^->][^-]*) # possible hyphens not followed by >
|
-+ (?=-->) # hyphens before closing sequence, break contiguity
|
-?+ ([^-]+) # one hyphen followed by >
|
-->[^<]* # closing sequence, go to next <
(*SKIP)(*FAIL) () # break contiguity (note: "()" avoids a mysterious bug
) # in regex101, you can remove it)
)
With this replacement: \1
online demo
The \G feature ensures that matches are consecutive.
Two ways are used to break the contiguity:
a lookahead (?=-->)
the backtracking control verbs (*SKIP)(*FAIL) that forces the pattern to fail and all characters matched before to not be retried.
So when contiguity is broken or at the begining the first main branch will fail (cause of the \G anchor) and the second branch will be used.
\K removes all on the left from the match result.
(*ACCEPT) makes the pattern succeed unconditionnaly.
This pattern uses massively the branch reset feature (?|...(..)...|...(..)...|...), so all capturing groups have the same number (in other words there is only one group, the group 1.)
Note: even this pattern is long, it needs few steps to obtain a match. The impact of non-greedy quantifiers is reduced as much as possible, and each alternatives are sorted and as efficient as possible. One of the goals is to reduce the total number of matches needed to treat a string.
(?<!<!)--+(?!-?>)(?=(?:(?!-->).)*-->)
matches -- (or ---- etc.) only between <!-- and -->. You need to set the /s parameter to allow the dot to match newlines.
Explanation:
(?<!<!) # Assert that we're not right at the start of a comment
--+ # Match two or more dashes --
(?= # only if the following can be matched further onwards:
(?!-?>) # First, make sure we're not at the end of the comment.
(?: # Then match the following group
(?!-->) # which must not contain -->
. # but may contain any character
)* # any number of times
--> # as long as --> follows.
) # End of lookahead assertion.
Test it live on regex101.com.
I suppose the correct AutoIt syntax would be
StringRegExpReplace($xml_string, "(?s)(?<!<!)--+(?!-?>)(?=(?:(?!-->).)*-->)", "")