This question comes in an attempt to understand one of the answer in : How to check that a string is a palindrome using regular expressions?
Answer given by Markus Jarderot is :
/^((.)(?1)\2|.?)$/
Can someone please explain, whats exactly happening here....i need to do similar in Perl, but not able to understand this solution!!!
PS : I am not very good in perl so please go easy ....and also "this can't be considered a regular expression if you want to be strict" - i read this line, so i am aware that this not regex strictly
^ - matches beginning of string
( - starts capture group #1
(.) - matches any single character except a newline, save it in capture group #2
(?1) - recurse = replace this group with the entire regexp capture group #1
\2 - matches the same thing as capture group #2. This requires the first and last characters of the string to match each other
| - creates an alternative
.? - optionally matches any one character that isn't a newline - This handles the end of the recursion, by matching an empty string (when the whole string is an even length) or a single character (when it's an odd length)
) - ends capture group #1
$ - matches end of string or before a newline at the end of the string.
The recursion (?1) is the key. A palindrome is an empty string, a 1-character string, or a string whose first and last characters are the same and the substring between them is also a palindrome.
It might be easier to understand with this analogous function, that does the same thing for arrays:
sub palindrome {
if (scalar(#_) >= 2) {
my $first_dot = shift;
my $slash_two = pop;
return $first_dot eq $slash_two && palindrome(#_);
} else {
# zero or one items
return 1;
}
}
print "yes!\n" if palindrome(qw(one two three two one));
print "really?\n" if palindrome(qw(one two three two two one));
The (?1) notation is a recursive reference to the start of the first parenthesis in the regex, the \2 is a backreference in the current recursion to the (.). Those two are anchored at the start and end of 'whatever is matching at the current recursion depth', so everything else is matched at the next depth down.
ikegami suspects this is faster:
sub palindrome {
my $next = 0;
my %symbols;
my $s = join '', map chr( $symbols{$_} ||= $next++ ), #_;
return $s =~ /^((.)(?1)\2|.?)\z/s;
}
I made this regEx few days ago.
If you use it like this it will give you an array of all palindromes in a certain text.
The example is for #JavaScript but you can use the regEx itself in any language to do the job.
Works perfect for words to 21 chars or numbers to 21 digits. You can make it more accurate if you need to.
const palindromeFinder = /\b(\w?)(\w?)(\w?)(\w?)(\w?)(\w?)(\w?)(\w?)(\w?)(\w)\S?\10\9\8\7\6\5\4\3\2\1\b/g;
console.log(inputString.match(palindromeFinder));
Related
i searching to find some Perl Regular Expression Syntax about some requirements i have in a project.
First i want to exclude strings from a txt file (dictionary).
For example if my file have this strings:
path.../Document.txt |
tree
car
ship
i using Regular Expression
a1testtre -- match
orangesh1 -- match
apleship3 -- not match [contains word from file ]
Also i have one more requirement that i couldnt solve. I have to create a Regex that not allow a String to have over 3 times a char repeat (two chars).
For example :
adminnisstrator21 -- match (have 2 times a repetition of chars)
kkeeykloakk -- not match have over 3 times repetition
stack22ooverflow -- match (have 2 times a repetition of chars)
for this i have try
\b(?:([a-z])(?!\1))+\b
but it works only for the first char-reppeat
Any idea how to solve these two?
To not match a word from a file you might check whether a string contains a substring or use a negative lookahead and an alternation:
^(?!.*(?:tree|car|ship)).*$
^ Assert start of string
(?! negative lookahead, assert what is on the right is not
.*(?:tree|car|ship) Match 0+ times any char except a newline and match either tree car or ship
) Close negative lookahead
.* Match any char except a newline
$ Assert end of string
Regex demo
To not allow a string to have over 3 times a char repeat you could use:
\b(?!(?:\w*(\w)\1){3})\w+\b
\b Word boundary
(?! Negative lookahead, assert what is on the right is not
(?: NOn capturing group
\w*(\w)\1 Match 0+ times a word character followed by capturing a word char in a group followed by a backreference using \1 to that group
){3} Close non capturing group and repeat 3 times
) close negative lookahead
\w+ Match 1+ word characters
\b word boundary
Regex demo
Update
According to this posted answer (which you might add to the question instead) you have 2 patterns that you want to combine but it does not work:
(?=^(?!(?:\w*(.)\1){3}).+$)(?=^(?:(.)(?!(?:.*?\1){4}))*$)
In those 2 patterns you use 2 capturing groups, so the second pattern has to point to the second capturing group \2.
(?=^(?!(?:\w*(.)\1){3}).+$)(?=^(?:(.)(?!(?:.*?\2){4}))*$)
^
Pattern demo
One way to exclude strings that contain words from a given list is to form a pattern with an alternation of the words and use that in a regex, and exclude strings for which it matches.
use warnings;
use strict;
use feature qw(say);
use Path::Tiny;
my $file = shift // die "Usage: $0 file\n"; #/
my #words = split ' ', path($file)->slurp;
my $exclude = join '|', map { quotemeta } #words;
foreach my $string (qw(a1testtre orangesh1 apleship3))
{
if ($string !~ /$exclude/) {
say "OK: $string";
}
}
I use Path::Tiny to read the file into a a string ("slurp"), which is then split by whitespace into words to use for exclusion. The quotemeta escapes non-"word" characters, should any happen in your words, which are then joined by | to form a string with a regex pattern. (With complex patterns use qr.)
This may be possible to tweak and improve, depending on your use cases, for one in regards to the order of of patterns with common parts in alternation.†
The check that successive duplicate characters do not occur more than three times
foreach my $string (qw(adminnisstrator21 kkeeykloakk stack22ooverflow))
{
my #chars_that_repeat = $string =~ /(.)\1+/g;
if (#chars_that_repeat < 3) {
say "OK: $string";
}
}
A long string of repeated chars (aaaa) counts as one instance, due to the + quantifier in regex; if you'd rather count all pairs remove the + and four as will count as two pairs. The same char repeated at various places in the string counts every time, so aaXaa counts as two pairs.
This snippet can be just added to the above program, which is invoked with the name of the file with words to use for exclusion. They both print what is expected from provided samples.
† Consider an example with exclusion-words: so, sole, and solely. If you only need to check whether any one of these matches then you'd want shorter ones first in the alternation
my $exclude = join '|', map { quotemeta } sort { length $a <=> length $b } #words;
#==> so|sole|solely
for a quicker match (so matches all three). This, by all means, appears to be the case here.
But, if you wanted to correctly identify which word matched then you must have longer words first,
solely|sole|so
so that a string solely is correctly matched by its word before it can be "stolen" by so. Then in this case you'd want it the other way round,
sort { length $b <=> length $a }
I hope someone else will come with a better solution, but this seems to do what you want:
\b Match word boundary
(?: Start capture group
(?:([a-z0-9])(?!\1))* Match all characters until it encounters a double
(?:([a-z0-9])\2)+ Match all repeated characters until a different one is reached
){0,2} Match capture group 0 or 2 times
(?:([a-z0-9])(?!\3))+ Match all characters until it encounters a double
\b Match end of word
I changed the [a-z] to also match numbers, since the examples you gave seem to also include numbers. Perl regex also has the \w shorthand, which is equivalent to [A-Za-z0-9_], which could be handy if you want to match any character in a word.
My problem is that i have 2 regex that working:
Not allow over 3 pairs of chars:
(?=^(?!(?:\w*(.)\1){3}).+$)
Not allow over 4 times a char to repeat:
(?=^(?:(.)(?!(?:.*?\1){4}))*$)
Now i want to combine them into one row like:
(?=^(?!(?:\w*(.)\1){3}).+$)(?=^(?:(.)(?!(?:.*?\1){4}))*$)
but its working only the regex that is first and not both of them
As mentioned in comment to #zdim's answer, take it a bit further by making sure that the order in which your words are assembled into the match pattern doesn't trip you. If the words in the file are not very carefully ordered to start, I use a subroutine like this when building the match string:
# Returns a list of alternative match patterns in tight matching order.
# E.g., TRUSTEES before TRUSTEE before TRUST
# TRUSTEES|TRUSTEE|TRUST
sub tight_match_order {
return #_ unless #_ > 1;
my (#alts, #ordered_alts, %alts_seen);
#alts = map { $alts_seen{$_}++ ? () : $_ } #_;
TEST: {
my $alt = shift #alts;
if (grep m#$alt#, #alts) {
push #alts => $alt;
} else {
push #ordered_alts => $alt;
}
redo TEST if #alts;
}
#ordered_alts
}
So following #zdim's answer:
...
my #words = split ' ', path($file)->slurp;
#words = tight_match_order(#words); # add this line
my $exclude = join '|', map { quotemeta } #words;
...
HTH
I have a string test_demo_0.1.1.
I want in PowerShell script to add before the 0.1.1 some text, for example: test_demo_shay_0.1.1.
I succeeded to detect the first number with RegEx and add the text:
$str = "test_demo_0.1.1"
if ($str - match "(?<number>\d)")
{
$newStr = $str.Insert($str.IndexOf($Matches.number) - 1, "_shay")-
}
# $newStr = test_demo_shay_0.1.1
The problem is, sometimes my string includes a number in another location, for example: test_demo2_0.1.1 (and then the insert is not good).
So I want to detect the first number which the character before is _, how can I do it?
I tried "(_<number>\d)" and "([_]<number>\d)" but it doesn't work.
What you ask for is called a positive lookbehind (a construct that checks for the presence of some pattern immediately to the left of thew current location):
"(?<=_)(?<number>\d)"
^^^^^^
However, it seems all you want is to insert _shay before the first digit preceded with _. A replace operation will suit here best:
$str -replace '_(\d.*)', '_shay_$1'
Result: test_demo_shay_0.1.1.
Details
_ - an underscore
(\d.*) - Capturing group #1: a digit and then any 0+ chars to the end of the line.
The $1 in the replacement pattern is the contents matched by the capturing group #1.
I have the following code:
my $sDatabase = "abc_def:xyz_comp.";
if ($sDatabase =~ m/^(\w)*\:(\w*)\_em\.$/)
{
print "$1\:$2\.\n";
}
else
{
print "$1\:$2\_em\.\n";
}
but I am getting empty $1 and $2. The output is:
Use of uninitialized value in concatenation (.) or string at new_mscn_iden_parse.pl line 187.
Use of uninitialized value in concatenation (.) or string at new_mscn_iden_parse.pl line 187.
:_em.
This code will do what you want
my $sDatabase = "abc_def:xyz_comp.";
$sDatabase =~ m/^(\w+):(\w+?)(_em)?\.$/ or die "Invalid data";
if ($3) {
print "$1:$2.\n";
}
else {
print "$1:$2_em.\n";
}
What do you expect $1 and $2 to contain when you fail to match?!
It contains whatever it contains before you attempted the match.
Possible solution:
$sDatabase =~ s/(?<!_em)(?=\.\z)/_em/;
You have:
my $sDatabase = "abc_def:xyz_comp.";
if ($sDatabase =~ m/^(\w)*\:(\w*)\_em\.$/);
Let's see if this matches:
You're regular expression says:
Anchor at the start of a line.
You are looking for zero or more word characters . Word characters (in the ASCII alphabet) includes lowercase letters, uppercase letters numbers and underscores.
Thus /\w*/ will match all the following:
Computer
computer
computer23
computer_32
an empty string
You're next looking for a colon
Then, more word characters
Followed by a _em string
Followed by a period
And that should be the end of the string (if there's no NL and you're not doing multi-line string searches. Looks like you're safe there).
Now, let's look at your string: abc_def:xyz_comp.
\w* will match up to abc_def. Regular expressions are greedy and will try to match the biggest portion of the string as possible.
The : will match the colon. So far, you're matching abc_def:.
That \w* will match on xyz_comp.
Now, you're trying to match a _em. Oops! No good. There is no _em in your string. Your regular expression match will fail.
Since your regular expression match fails, the $1 and $2 variables simply are not set and have no value.
That's why you're getting Use of uninitialized value. What you can do is make the later half of your expression optional:
my $sDatabase = "abc_def:xyz_comp.";
if ($sDatabase =~ /^(\w)+:(\w*)(_em)?\.$/) {
if ( $3 ) {
print "$1:${2}${3}.\n";
else {
print "$1:${2}_em.";
}
}
else {
die qq(String doesn't match regular expression at all\n);
}
}
First of all, I think you want to match at least one character (I could be wrong), so I switched the asterisk which matches zero or more to a + which matches one or more.
Note I have a third set of parentheses followed by a ?. This means match this zero or one times. Thus, you will have a match, and $1 and $2 will be set as long as your string starts with one or more word characters, followed by a colon, followed by one or more word characters.
What won't necessarily happen is that $3 will be set. This will only be set if your string also ends with _em.. If your string doesn't include the _em, but ends with a period, $1 and $2 will still match.
In your case, we could simplify it by doing this:
my $sDatabase = "abc_def:xyz_comp.";
if ($sDatabase =~ /^(\w)+:(\w*)(?:_em)?\.$/) {
print "$1:${2}_em.";
else {
die qq(String doesn't match regular expression at all\n);
}
The (?:...) means don't set a match, just group. Thus, $3 will never be set. That's okay, either $3 is _em. or we add _em. to the end of the match anyway.
For example, string is "staaaae", If searchValue is "aa", n=2, then 3 will return.
This is not a feature of regex itself, but depending on your language you can get this information.
For one thing I have to correct you. The start index for the second match is 4 and not 3 (Standard regex matching behaviour)
staaaae
^^ first match
^^ second match
The pattern continues to search after the first match, so the matches do not overlap.
Perl
In Perl for example its no problem.
In addition to what was matched, Perl (since 5.6.0) also provides the
positions of what was matched as contents of the #- and #+ arrays.
$-[0] is the position of the start of the entire match and $+[0] is
the position of the end. Similarly, $-[n] is the position of the start
of the $n match and $+[n] is the position of the end. If $n is
undefined, so are $-[n] and $+[n] .
cited from Perlretut
so in Perl the information you want would be in $-[n].
c#/.net
In .net its also no problem. Only a bit more effort than in Perl.
The class Match returns you also the index of the match, but it matches only one at a time, so you have to go through a loop to get all matches.
string input = "staaaae";
string pattern = #"aa";
Match match = Regex.Match(input, pattern);
int count = 0;
while (match.Success)
{
count++;
Console.WriteLine("'{0}' was found the {1} timein the source code at position {2}.",
match.Value, count, match.Index);
match = match.NextMatch();
}
Console.ReadLine();
I'm a regular expression newbie and I can't quite figure out how to write a single regular expression that would "match" any duplicate consecutive words such as:
Paris in the the spring.
Not that that is related.
Why are you laughing? Are my my regular expressions THAT bad??
Is there a single regular expression that will match ALL of the bold strings above?
Try this regular expression:
\b(\w+)\s+\1\b
Here \b is a word boundary and \1 references the captured match of the first group.
Regex101 example here
I believe this regex handles more situations:
/(\b\S+\b)\s+\b\1\b/
A good selection of test strings can be found here: http://callumacrae.github.com/regex-tuesday/challenge1.html
The below expression should work correctly to find any number of duplicated words. The matching can be case insensitive.
String regex = "\\b(\\w+)(\\s+\\1\\b)+";
Pattern p = Pattern.compile(regex, Pattern.CASE_INSENSITIVE);
Matcher m = p.matcher(input);
// Check for subsequences of input that match the compiled pattern
while (m.find()) {
input = input.replaceAll(m.group(0), m.group(1));
}
Sample Input : Goodbye goodbye GooDbYe
Sample Output : Goodbye
Explanation:
The regex expression:
\b : Start of a word boundary
\w+ : Any number of word characters
(\s+\1\b)* : Any number of space followed by word which matches the previous word and ends the word boundary. Whole thing wrapped in * helps to find more than one repetitions.
Grouping :
m.group(0) : Shall contain the matched group in above case Goodbye goodbye GooDbYe
m.group(1) : Shall contain the first word of the matched pattern in above case Goodbye
Replace method shall replace all consecutive matched words with the first instance of the word.
Try this with below RE
\b start of word word boundary
\W+ any word character
\1 same word matched already
\b end of word
()* Repeating again
public static void main(String[] args) {
String regex = "\\b(\\w+)(\\b\\W+\\b\\1\\b)*";// "/* Write a RegEx matching repeated words here. */";
Pattern p = Pattern.compile(regex, Pattern.CASE_INSENSITIVE/* Insert the correct Pattern flag here.*/);
Scanner in = new Scanner(System.in);
int numSentences = Integer.parseInt(in.nextLine());
while (numSentences-- > 0) {
String input = in.nextLine();
Matcher m = p.matcher(input);
// Check for subsequences of input that match the compiled pattern
while (m.find()) {
input = input.replaceAll(m.group(0),m.group(1));
}
// Prints the modified sentence.
System.out.println(input);
}
in.close();
}
Regex to Strip 2+ duplicate words (consecutive/non-consecutive words)
Try this regex that can catch 2 or more duplicate words and only leave behind one single word. And the duplicate words need not even be consecutive.
/\b(\w+)\b(?=.*?\b\1\b)/ig
Here, \b is used for Word Boundary, ?= is used for positive lookahead, and \1 is used for back-referencing.
Example
Source
The widely-used PCRE library can handle such situations (you won't achieve the the same with POSIX-compliant regex engines, though):
(\b\w+\b)\W+\1
Here is one that catches multiple words multiple times:
(\b\w+\b)(\s+\1)+
No. That is an irregular grammar. There may be engine-/language-specific regular expressions that you can use, but there is no universal regular expression that can do that.
This is the regex I use to remove duplicate phrases in my twitch bot:
(\S+\s*)\1{2,}
(\S+\s*) looks for any string of characters that isn't whitespace, followed whitespace.
\1{2,} then looks for more than 2 instances of that phrase in the string to match. If there are 3 phrases that are identical, it matches.
Since some developers are coming to this page in search of a solution which not only eliminates duplicate consecutive non-whitespace substrings, but triplicates and beyond, I'll show the adapted pattern.
Pattern: /(\b\S+)(?:\s+\1\b)+/ (Pattern Demo)
Replace: $1 (replaces the fullstring match with capture group #1)
This pattern greedily matches a "whole" non-whitespace substring, then requires one or more copies of the matched substring which may be delimited by one or more whitespace characters (space, tab, newline, etc).
Specifically:
\b (word boundary) characters are vital to ensure partial words are not matched.
The second parenthetical is a non-capturing group, because this variable width substring does not need to be captured -- only matched/absorbed.
the + (one or more quantifier) on the non-capturing group is more appropriate than * because * will "bother" the regex engine to capture and replace singleton occurrences -- this is wasteful pattern design.
*note if you are dealing with sentences or input strings with punctuation, then the pattern will need to be further refined.
The example in Javascript: The Good Parts can be adapted to do this:
var doubled_words = /([A-Za-z\u00C0-\u1FFF\u2800-\uFFFD]+)\s+\1(?:\s|$)/gi;
\b uses \w for word boundaries, where \w is equivalent to [0-9A-Z_a-z]. If you don't mind that limitation, the accepted answer is fine.
This expression (inspired from Mike, above) seems to catch all duplicates, triplicates, etc, including the ones at the end of the string, which most of the others don't:
/(^|\s+)(\S+)(($|\s+)\2)+/g, "$1$2")
I know the question asked to match duplicates only, but a triplicate is just 2 duplicates next to each other :)
First, I put (^|\s+) to make sure it starts with a full word, otherwise "child's steak" would go to "child'steak" (the "s"'s would match). Then, it matches all full words ((\b\S+\b)), followed by an end of string ($) or a number of spaces (\s+), the whole repeated more than once.
I tried it like this and it worked well:
var s = "here here here here is ahi-ahi ahi-ahi ahi-ahi joe's joe's joe's joe's joe's the result result result";
print( s.replace( /(\b\S+\b)(($|\s+)\1)+/g, "$1"))
--> here is ahi-ahi joe's the result
Try this regular expression it fits for all repeated words cases:
\b(\w+)\s+\1(?:\s+\1)*\b
I think another solution would be to use named capture groups and backreferences like this:
.* (?<mytoken>\w+)\s+\k<mytoken> .*/
OR
.*(?<mytoken>\w{3,}).+\k<mytoken>.*/
Kotlin:
val regex = Regex(""".* (?<myToken>\w+)\s+\k<myToken> .*""")
val input = "This is a test test data"
val result = regex.find(input)
println(result!!.groups["myToken"]!!.value)
Java:
var pattern = Pattern.compile(".* (?<myToken>\\w+)\\s+\\k<myToken> .*");
var matcher = pattern.matcher("This is a test test data");
var isFound = matcher.find();
var result = matcher.group("myToken");
System.out.println(result);
JavaScript:
const regex = /.* (?<myToken>\w+)\s+\k<myToken> .*/;
const input = "This is a test test data";
const result = regex.exec(input);
console.log(result.groups.myToken);
// OR
const regex = /.* (?<myToken>\w+)\s+\k<myToken> .*/g;
const input = "This is a test test data";
const result = [...input.matchAll(regex)];
console.log(result[0].groups.myToken);
All the above detect the test as the duplicate word.
Tested with Kotlin 1.7.0-Beta, Java 11, Chrome and Firefox 100.
You can use this pattern:
\b(\w+)(?:\W+\1\b)+
This pattern can be used to match all duplicated word groups in sentences. :)
Here is a sample util function written in java 17, which replaces all duplications with the first occurrence:
public String removeDuplicates(String input) {
var regex = "\\b(\\w+)(?:\\W+\\1\\b)+";
var pattern = Pattern.compile(regex, Pattern.CASE_INSENSITIVE);
var matcher = pattern.matcher(input);
while (matcher.find()) {
input = input.replaceAll(matcher.group(), matcher.group(1));
}
return input;
}
As far as I can see, none of these would match:
London in the
the winter (with the winter on a new line )
Although matching duplicates on the same line is fairly straightforward,
I haven't been able to come up with a solution for the situation in which they
stretch over two lines. ( with Perl )
To find duplicate words that have no leading or trailing non whitespace character(s) other than a word character(s), you can use whitespace boundaries on the left and on the right making use of lookarounds.
The pattern will have a match in:
Paris in the the spring.
Not that that is related.
The pattern will not have a match in:
This is $word word
(?<!\S)(\w+)\s+\1(?!\S)
Explanation
(?<!\S) Negative lookbehind, assert not a non whitespace char to the left of the current location
(\w+) Capture group 1, match 1 or more word characters
\s+ Match 1 or more whitespace characters (note that this can also match a newline)
\1 Backreference to match the same as in group 1
(?!\S) Negative lookahead, assert not a non whitespace char to the right of the current location
See a regex101 demo.
To find 2 or more duplicate words:
(?<!\S)(\w+)(?:\s+\1)+(?!\S)
This part of the pattern (?:\s+\1)+ uses a non capture group to repeat 1 or more times matching 1 or more whitespace characters followed by the backreference to match the same as in group 1.
See a regex101 demo.
Alternatives without using lookarounds
You could also make use of a leading and trailing alternation matching either a whitespace char or assert the start/end of the string.
Then use a capture group 1 for the value that you want to get, and use a second capture group with a backreference \2 to match the repeated word.
Matching 2 duplicate words:
(?:\s|^)((\w+)\s+\2)(?:\s|$)
See a regex101 demo.
Matching 2 or more duplicate words:
(?:\s|^)((\w+)(?:\s+\2)+)(?:\s|$)
See a regex101 demo.
Use this in case you want case-insensitive checking for duplicate words.
(?i)\\b(\\w+)\\s+\\1\\b