Regex - should match newlines as well as should end at the first occurrence of a particular format
In reference to Regex - should match newlines as well as should end at the first occurence of a particular format
I am trying to read body of the mail from logs (some of them are more than 500 lines).
Sample data looks like: BodyOftheMail_Script = [ BEGIN 500 lines END ]
I've tried following regular expressions:
+-----------------------------------------------------------------------+----------+--------+
| Regexp | Steps | Time |
+-----------------------------------------------------------------------+----------+--------+
| BodyOftheMail_Script\s=\s[\sBEGIN\s{0,}((?s)[\s\S]*?)(?=\s{1,}END\s]) | 1015862 | ~474ms |
| BodyOftheMail_Script\s=\s[\sBEGIN\s{0,}((?s)[\w\W]*?)(?=\s{1,}END\s]) | 1015862 | ~480ms |
| BodyOftheMail_Script\s=\s[\sBEGIN\s{0,}((?s).*?)(?=\s{1,}END\s]) | 1015862 | ~577ms |
| BodyOftheMail_Script\s=\s\[\sBEGIN\s{0,}((.|\n)*?)(?=\s{1,}END\s\]) | 1681711 | ~829ms |
+-----------------------------------------------------------------------+----------+--------+
Is there a faster way (more optimal regexp) to match this?
Enhancing the pattern
The most efficient from 5 expressions turned out to be
BodyOftheMail_Script\s=\s\[\sBEGIN\s*(\S*(?:\s++(?!END\s])\S*)*)\s+END\s]
See the regex demo
The part I modified is \S*(?:\s++(?!END\s])\S*)*:
\S* - 0 or more non-whitespace characters
(?:\s++(?!END\s])\S*)* - 0 or more occurrences of
\s++(?!END\s]) - 1+ whitespace characters (matched possessively so that the lookahead check could only be performed once after all the 1+ whitespaces are matched) not followed with END, 1 whitespace and ] char
\S* - 0 or more non-whitespace characters
Why not a mere BodyOftheMail_Script\s=\s\[\sBEGIN\s*(.*?)\s+END\s] with re.DOTALL? The \s*(.*?)\s+END\s] will work as follows: 0+ whitespaces will be matched at once, then (.*?) will be skipped the first time, then \s+END\s] pattern will be tried. If \s+END\s] is not matched, .*? will grab one char and again let the subsequent patterns try to match the string. And so on. It might take a lot of backtracking steps to reach the end of a match (if it is there, else, it might end in a timeout sooner than later).
Performance comparison
Since the number of steps at regex101.com is not a direct proof a certain pattern is more efficient than another, I decided to run performance tests using Python PyPi regex library. See the code below.
The results obtained on a PC with 16GB RAM, Intel Core i5-9400F CPU, consistent results are obtained using PyPi regex versions 2.5.77 and 2.5.82:
┌──────────┬─────────────────────────────────────────────────────────────────┐
│ Regex │ Time taken │
├──────────┼─────────────────────────────────────────────────────────────────┤
│ OP 1 │ 0.5606743000000001 │
│ OP 2 │ 0.5524994999999999 │
│ OP 3 │ 0.5026944 │
│ OP 4 │ 0.7502984000000001 │
│ WS_1 │ 0.25729479999999993 │
│ WS_2 │ 0.3680949 │
└──────────┴─────────────────────────────────────────────────────────────────┘
Conclusions:
The worst OP regex is the one that contains a notorious (.|\n)*? pattern, it is one of the most inefficient patterns I have seen in my regex life, it always causes issues across all languages. Please never use it in your patterns
The first three OP patterns are comparable, but it is clear than the common workarounds for a . to match any char, [\w\W] and [\s\S], should be avoided if there is a way to make . match any char with a modifier, such as (?s) or regex.DOTALL. The (?s). native solution is a tiny bit more efficient.
My suggestion appears to be twice as fast comapring to the best OP pattern due to the fact it matches strings from left-hand delimiter to the right-hand delimiter in chunks, only stopping to check for the right-hand delimiter after grabbing whitespace chunks of text and the whitespaces that follow them.
The .*? construct is expanding each time a char is not the start of the right-hand delimiter, with longer strings, its efficiency will be decreasing.
The Python testing code:
import regex, timeit
text = 'BodyOftheMail_Script = [ BEGIN some text\nhere and\nhere, too \nEND ]'
regex_pattern_1=regex.compile(r'BodyOftheMail_Script\s=\s\[\sBEGIN\s{0,}((?s)[\s\S]*?)(?=\s{1,}END\s])')
regex_pattern_2=regex.compile(r'BodyOftheMail_Script\s=\s\[\sBEGIN\s{0,}((?s)[\w\W]*?)(?=\s{1,}END\s])')
regex_pattern_3=regex.compile(r'BodyOftheMail_Script\s=\s\[\sBEGIN\s{0,}((?s).*?)(?=\s{1,}END\s])')
regex_pattern_4=regex.compile(r'BodyOftheMail_Script\s=\s\[\sBEGIN\s{0,}((.|\n)*?)(?=\s{1,}END\s\])')
regex_pattern_WS_1=regex.compile(r'BodyOftheMail_Script\s=\s\[\sBEGIN\s*(\S*(?:\s++(?!END\s])\S*)*)\s+END\s]')
regexp_patternWS_2 = regex.compile(r'BodyOftheMail_Script\s=\s\[\sBEGIN\s*(.*?)\s+END\s]', regex.DOTALL)
print(timeit.timeit("p.findall(text)", 'from __main__ import text, regex_pattern_1 as p', number=100000))
# => 0.5606743000000001
print(timeit.timeit("p.findall(text)", 'from __main__ import text, regex_pattern_2 as p', number=100000))
# => 0.5524994999999999
print(timeit.timeit("p.findall(text)", 'from __main__ import text, regex_pattern_3 as p', number=100000))
# => 0.5026944
print(timeit.timeit("p.findall(text)", 'from __main__ import text, regex_pattern_4 as p', number=100000))
# => 0.7502984000000001
print(timeit.timeit("p.findall(text)", 'from __main__ import text, regex_pattern_WS_1 as p', number=100000))
# => 0.25729479999999993
print(timeit.timeit("p.findall(text)", 'from __main__ import text, regexp_patternWS_2 as p', number=100000))
# => 0.3680949
Unless you missed some important details in your question, I don't see any reason to overcomplicate the things. Why not use simple BodyOftheMail_Script = \[ BEGIN.*?END \]? So you have your start indicator BodyOftheMail_Script = [ BEGIN, you have end indicator END ], and you want to match everything in between in non-greedy way .*?. Of course it requires flags like re.MULTILINE and re.DOTALL (if we're talking about Python):
import re
regexp = re.compile(r'BodyOftheMail_Script = \[ BEGIN.*?END \]', re.DOTALL | re.MULTILINE)
The first rule of regexps - do not overcomplicate ;) Someone will read it after you.
Using the same comparison script as in #Wictor's answer, I got following results:
OP 1 0.24152620000000002
OP 2 0.28501820000000005
OP 3 0.20582650000000002
OP 4 0.3379188999999999
WS 0.16937669999999994
Subj 0.10387990000000014
Replacing to \s is possible and it does not really change the speed (but if you have only space in the actual file, then just use space, do not overcomplicate)
Also if you want, you can add the group to directly get the content, it adds ~0.02s for me, most probably it will be faster to trim each result afterwards instead of using regexp group.
So I'm trying to practice regex with sed on linux. I've got these lines:
│ .--. 3..6 °C │ _ /"".-. 6..8 °C │ _ /"".-. 2..6 °C │ _ /"".-. 0..4 °C │
│ _ /"".-. -2..+3 °C │ _ /"".-. 1..5 °C │ ,\_( ). 1..4 °C │ ,\_( ). -1..+2 °C │
│ ( ). -1..+1 °C │ ( ). -2..+2 °C │ ( ). -4..+2 °C │ ( ). -4..+2 °C │
How can I extract only numbers and their sign? But every single on of them? I'm very sloppy with using groups and regexes, and any tip and explanation would help. I got this regex which can shorten the problem, but I still can't extract every single match. I hope it can help.
sed -n -E "/[+-]?[0-9]+\.{2}[+-]?[0-9]+/p"
I want my output to be 3,6 6,8 2,6 0,4 if done on first line, for second and third:
-2,+3 1,5 1,4 -1,+2
-1,+1 -2,+2 -4,+2 -4,+2
This might work for you (GNU sed):
sed -E 's/([+-]?[0-9]+)\.\.([+-]?[0-9]+)/\n\1,\2\n/g;s/^[^0-9+-].*$/ /mg;s/^ |\n| $//g' file
Surround valid strings by newlines (convert .. to , at the same time).
Replace all non-valid strings by a space.
Remove the spaces at the front and end of the line and any introduced newlines.
N.B. The use of the m flag on the substitution command.
An alternative, is to work through the line from beginning to end removing non-valid characters:
sed -E 's/^/\n/;:a;s/\n([+-]?[0-9]+)\.\.([+-]?[0-9]+)/\1,\2 \n/;ta;s/\n./\n/;ta;s/ ?\n//' file
This is the second part of a series of educational regex articles. It shows how lookaheads and nested references can be used to match the non-regular languge anbn. Nested references are first introduced in: How does this regex find triangular numbers?
One of the archetypal non-regular languages is:
L = { anbn: n > 0 }
This is the language of all non-empty strings consisting of some number of a's followed by an equal number of b's. Examples of strings in this language are ab, aabb, aaabbb.
This language can be show to be non-regular by the pumping lemma. It is in fact an archetypal context-free language, which can be generated by the context-free grammar S → aSb | ab.
Nonetheless, modern day regex implementations clearly recognize more than just regular languages. That is, they are not "regular" by formal language theory definition. PCRE and Perl supports recursive regex, and .NET supports balancing groups definition. Even less "fancy" features, e.g. backreference matching, means that regex is not regular.
But just how powerful is this "basic" features? Can we recognize L with Java regex, for example? Can we perhaps combine lookarounds and nested references and have a pattern that works with e.g. String.matches to match strings like ab, aabb, aaabbb, etc?
References
perlfaq6: Can I use Perl regular expressions to match balanced text?
MSDN - Regular Expression Language Elements - Balancing Group Definitions
pcre.org - PCRE man page
regular-expressions.info - Lookarounds and Grouping and Backreferences
java.util.regex.Pattern
Linked questions
Does lookaround affect which languages can be matched by regular expressions?
.NET Regex Balancing Groups vs PCRE Recursive Patterns
The answer is, needless to say, YES! You can most certainly write a Java regex pattern to match anbn. It uses a positive lookahead for assertion, and one nested reference for "counting".
Rather than immediately giving out the pattern, this answer will guide readers through the process of deriving it. Various hints are given as the solution is slowly constructed. In this aspect, hopefully this answer will contain much more than just another neat regex pattern. Hopefully readers will also learn how to "think in regex", and how to put various constructs harmoniously together, so they can derive more patterns on their own in the future.
The language used to develop the solution will be PHP for its conciseness. The final test once the pattern is finalized will be done in Java.
Step 1: Lookahead for assertion
Let's start with a simpler problem: we want to match a+ at the beginning of a string, but only if it's followed immediately by b+. We can use ^ to anchor our match, and since we only want to match the a+ without the b+, we can use lookahead assertion (?=…).
Here is our pattern with a simple test harness:
function testAll($r, $tests) {
foreach ($tests as $test) {
$isMatch = preg_match($r, $test, $groups);
$groupsJoined = join('|', $groups);
print("$test $isMatch $groupsJoined\n");
}
}
$tests = array('aaa', 'aaab', 'aaaxb', 'xaaab', 'b', 'abbb');
$r1 = '/^a+(?=b+)/';
# └────┘
# lookahead
testAll($r1, $tests);
The output is (as seen on ideone.com):
aaa 0
aaab 1 aaa
aaaxb 0
xaaab 0
b 0
abbb 1 a
This is exactly the output we want: we match a+, only if it's at the beginning of the string, and only if it's immediately followed by b+.
Lesson: You can use patterns in lookarounds to make assertions.
Step 2: Capturing in a lookahead (and f r e e - s p a c i n g mode)
Now let's say that even though we don't want the b+ to be part of the match, we do want to capture it anyway into group 1. Also, as we anticipate having a more complicated pattern, let's use x modifier for free-spacing so we can make our regex more readable.
Building on our previous PHP snippet, we now have the following pattern:
$r2 = '/ ^ a+ (?= (b+) ) /x';
# │ └──┘ │
# │ 1 │
# └────────┘
# lookahead
testAll($r2, $tests);
The output is now (as seen on ideone.com):
aaa 0
aaab 1 aaa|b
aaaxb 0
xaaab 0
b 0
abbb 1 a|bbb
Note that e.g. aaa|b is the result of join-ing what each group captured with '|'. In this case, group 0 (i.e. what the pattern matched) captured aaa, and group 1 captured b.
Lesson: You can capture inside a lookaround. You can use free-spacing to enhance readability.
Step 3: Refactoring the lookahead into the "loop"
Before we can introduce our counting mechanism, we need to do one modification to our pattern. Currently, the lookahead is outside of the + repetition "loop". This is fine so far because we just wanted to assert that there's a b+ following our a+, but what we really want to do eventually is assert that for each a that we match inside the "loop", there's a corresponding b to go with it.
Let's not worry about the counting mechanism for now and just do the refactoring as follows:
First refactor a+ to (?: a )+ (note that (?:…) is a non-capturing group)
Then move the lookahead inside this non-capturing group
Note that we must now "skip" a* before we can "see" the b+, so modify the pattern accordingly
So we now have the following:
$r3 = '/ ^ (?: a (?= a* (b+) ) )+ /x';
# │ │ └──┘ │ │
# │ │ 1 │ │
# │ └───────────┘ │
# │ lookahead │
# └───────────────────┘
# non-capturing group
The output is the same as before (as seen on ideone.com), so there's no change in that regard. The important thing is that now we are making the assertion at every iteration of the + "loop". With our current pattern, this is not necessary, but next we'll make group 1 "count" for us using self-reference.
Lesson: You can capture inside a non-capturing group. Lookarounds can be repeated.
Step 4: This is the step where we start counting
Here's what we're going to do: we'll rewrite group 1 such that:
At the end of the first iteration of the +, when the first a is matched, it should capture b
At the end of the second iteration, when another a is matched, it should capture bb
At the end of the third iteration, it should capture bbb
...
At the end of the n-th iteration, group 1 should capture bn
If there aren't enough b to capture into group 1 then the assertion simply fails
So group 1, which is now (b+), will have to be rewritten to something like (\1 b). That is, we try to "add" a b to what group 1 captured in the previous iteration.
There's a slight problem here in that this pattern is missing the "base case", i.e. the case where it can match without the self-reference. A base case is required because group 1 starts "uninitialized"; it hasn't captured anything yet (not even an empty string), so a self-reference attempt will always fail.
There are many ways around this, but for now let's just make the self-reference matching optional, i.e. \1?. This may or may not work perfectly, but let's just see what that does, and if there's any problem then we'll cross that bridge when we come to it. Also, we'll add some more test cases while we're at it.
$tests = array(
'aaa', 'aaab', 'aaaxb', 'xaaab', 'b', 'abbb', 'aabb', 'aaabbbbb', 'aaaaabbb'
);
$r4 = '/ ^ (?: a (?= a* (\1? b) ) )+ /x';
# │ │ └─────┘ | │
# │ │ 1 | │
# │ └──────────────┘ │
# │ lookahead │
# └──────────────────────┘
# non-capturing group
The output is now (as seen on ideone.com):
aaa 0
aaab 1 aaa|b # (*gasp!*)
aaaxb 0
xaaab 0
b 0
abbb 1 a|b # yes!
aabb 1 aa|bb # YES!!
aaabbbbb 1 aaa|bbb # YESS!!!
aaaaabbb 1 aaaaa|bb # NOOOOOoooooo....
A-ha! It looks like we're really close to the solution now! We managed to get group 1 to "count" using self-reference! But wait... something is wrong with the second and the last test cases!! There aren't enough bs, and somehow it counted wrong! We'll examine why this happened in the next step.
Lesson: One way to "initialize" a self-referencing group is to make the self-reference matching optional.
Step 4½: Understanding what went wrong
The problem is that since we made the self-reference matching optional, the "counter" can "reset" back to 0 when there aren't enough b's. Let's closely examine what happens at every iteration of our pattern with aaaaabbb as input.
a a a a a b b b
↑
# Initial state: Group 1 is "uninitialized".
_
a a a a a b b b
↑
# 1st iteration: Group 1 couldn't match \1 since it was "uninitialized",
# so it matched and captured just b
___
a a a a a b b b
↑
# 2nd iteration: Group 1 matched \1b and captured bb
_____
a a a a a b b b
↑
# 3rd iteration: Group 1 matched \1b and captured bbb
_
a a a a a b b b
↑
# 4th iteration: Group 1 could still match \1, but not \1b,
# (!!!) so it matched and captured just b
___
a a a a a b b b
↑
# 5th iteration: Group 1 matched \1b and captured bb
#
# No more a, + "loop" terminates
A-ha! On our 4th iteration, we could still match \1, but we couldn't match \1b! Since we allow the self-reference matching to be optional with \1?, the engine backtracks and took the "no thanks" option, which then allows us to match and capture just b!
Do note, however, that except on the very first iteration, you could always match just the self-reference \1. This is obvious, of course, since it's what we just captured on our previous iteration, and in our setup we can always match it again (e.g. if we captured bbb last time, we're guaranteed that there will still be bbb, but there may or may not be bbbb this time).
Lesson: Beware of backtracking. The regex engine will do as much backtracking as you allow until the given pattern matches. This may impact performance (i.e. catastrophic backtracking) and/or correctness.
Step 5: Self-possession to the rescue!
The "fix" should now be obvious: combine optional repetition with possessive quantifier. That is, instead of simply ?, use ?+ instead (remember that a repetition that is quantified as possessive does not backtrack, even if such "cooperation" may result in a match of the overall pattern).
In very informal terms, this is what ?+, ? and ?? says:
?+
(optional) "It doesn't have to be there,"
(possessive) "but if it is there, you must take it and not let go!"
?
(optional) "It doesn't have to be there,"
(greedy) "but if it is you can take it for now,"
(backtracking) "but you may be asked to let it go later!"
??
(optional) "It doesn't have to be there,"
(reluctant) "and even if it is you don't have to take it just yet,"
(backtracking) "but you may be asked to take it later!"
In our setup, \1 will not be there the very first time, but it will always be there any time after that, and we always want to match it then. Thus, \1?+ would accomplish exactly what we want.
$r5 = '/ ^ (?: a (?= a* (\1?+ b) ) )+ /x';
# │ │ └──────┘ │ │
# │ │ 1 │ │
# │ └───────────────┘ │
# │ lookahead │
# └───────────────────────┘
# non-capturing group
Now the output is (as seen on ideone.com):
aaa 0
aaab 1 a|b # Yay! Fixed!
aaaxb 0
xaaab 0
b 0
abbb 1 a|b
aabb 1 aa|bb
aaabbbbb 1 aaa|bbb
aaaaabbb 1 aaa|bbb # Hurrahh!!!
Voilà!!! Problem solved!!! We are now counting properly, exactly the way we want it to!
Lesson: Learn the difference between greedy, reluctant, and possessive repetition. Optional-possessive can be a powerful combination.
Step 6: Finishing touches
So what we have right now is a pattern that matches a repeatedly, and for every a that was matched, there is a corresponding b captured in group 1. The + terminates when there are no more a, or if the assertion failed because there isn't a corresponding b for an a.
To finish the job, we simply need to append to our pattern \1 $. This is now a back reference to what group 1 matched, followed by the end of the line anchor. The anchor ensures that there aren't any extra b's in the string; in other words, that in fact we have anbn.
Here's the finalized pattern, with additional test cases, including one that's 10,000 characters long:
$tests = array(
'aaa', 'aaab', 'aaaxb', 'xaaab', 'b', 'abbb', 'aabb', 'aaabbbbb', 'aaaaabbb',
'', 'ab', 'abb', 'aab', 'aaaabb', 'aaabbb', 'bbbaaa', 'ababab', 'abc',
str_repeat('a', 5000).str_repeat('b', 5000)
);
$r6 = '/ ^ (?: a (?= a* (\1?+ b) ) )+ \1 $ /x';
# │ │ └──────┘ │ │
# │ │ 1 │ │
# │ └───────────────┘ │
# │ lookahead │
# └───────────────────────┘
# non-capturing group
It finds 4 matches: ab, aabb, aaabbb, and the a5000b5000. It takes only 0.06s to run on ideone.com.
Step 7: The Java test
So the pattern works in PHP, but the ultimate goal is to write a pattern that works in Java.
public static void main(String[] args) {
String aNbN = "(?x) (?: a (?= a* (\\1?+ b)) )+ \\1";
String[] tests = {
"", // false
"ab", // true
"abb", // false
"aab", // false
"aabb", // true
"abab", // false
"abc", // false
repeat('a', 5000) + repeat('b', 4999), // false
repeat('a', 5000) + repeat('b', 5000), // true
repeat('a', 5000) + repeat('b', 5001), // false
};
for (String test : tests) {
System.out.printf("[%s]%n %s%n%n", test, test.matches(aNbN));
}
}
static String repeat(char ch, int n) {
return new String(new char[n]).replace('\0', ch);
}
The pattern works as expected (as seen on ideone.com).
And now we come to the conclusion...
It needs to be said that the a* in the lookahead, and indeed the "main + loop", both permit backtracking. Readers are encouraged to confirm why this is not a problem in terms of correctness, and why at the same time making both possessive would also work (though perhaps mixing mandatory and non-mandatory possessive quantifier in the same pattern may lead to misperceptions).
It should also be said that while it's neat that there's a regex pattern that will match anbn, this is in not always the "best" solution in practice. A much better solution is to simply match ^(a+)(b+)$, and then compare the length of the strings captured by groups 1 and 2 in the hosting programming language.
In PHP, it may look something like this (as seen in ideone.com):
function is_anbn($s) {
return (preg_match('/^(a+)(b+)$/', $s, $groups)) &&
(strlen($groups[1]) == strlen($groups[2]));
}
The purpose of this article is NOT to convince readers that regex can do almost anything; it clearly can't, and even for the things it can do, at least partial delegation to the hosting language should be considered if it leads to a simpler solution.
As mentioned at the top, while this article is necessarily tagged [regex] for stackoverflow, it is perhaps about more than that. While certainly there's value in learning about assertions, nested references, possessive quantifier, etc, perhaps the bigger lesson here is the creative process by which one can try to solve problems, the determination and hard work that it often requires when you're subjected to various constraints, the systematic composition from various parts to build a working solution, etc.
Bonus material! PCRE recursive pattern!
Since we did bring up PHP, it needs to be said that PCRE supports recursive pattern and subroutines. Thus, following pattern works for preg_match (as seen on ideone.com):
$rRecursive = '/ ^ (a (?1)? b) $ /x';
Currently Java's regex does not support recursive pattern.
Even more bonus material! Matching anbncn !!
So we've seen how to match anbn which is non-regular, but still context-free, but can we also match anbncn, which isn't even context-free?
The answer is, of course, YES! Readers are encouraged to try to solve this on their own, but the solution is provided below (with implementation in Java on ideone.com).
^ (?: a (?= a* (\1?+ b) b* (\2?+ c) ) )+ \1 \2 $
Given that no mention has been made of PCRE supporting recursive patterns, I'd just like to point out the simplest and most efficient example of PCRE that describes the language in question:
/^(a(?1)?b)$/
As mentioned in the question — with .NET balancing group, the patterns of the type anbncndn…zn can be matched easily as
^
(?<A>a)+
(?<B-A>b)+ (?(A)(?!))
(?<C-B>c)+ (?(B)(?!))
...
(?<Z-Y>z)+ (?(Y)(?!))
$
For example: http://www.ideone.com/usuOE
Edit:
There is also a PCRE pattern for the generalized language with recursive pattern, but a lookahead is needed. I don't think this is a direct translation of the above.
^
(?=(a(?-1)?b)) a+
(?=(b(?-1)?c)) b+
...
(?=(x(?-1)?y)) x+
(y(?-1)?z)
$
For example: http://www.ideone.com/9gUwF
I have a property whose value may contain following characters: ~!##$%^&*() and the space character.
I want to replace all of them with an empty string.
Please suggest a suitable regular expression to do this.
You already have the regular expression, it's the class of all the characters you listed:
[~!##$%^&*() ]
You just have to replace all occurrences by an empty string, using the regex/string API of your language.
For example, in Java:
// The pattern can be declared as a constant, computed only once.
Pattern p = Pattern.compile("[~!##$%^&*() ]");
String newPropName = p.matcher(propName).replaceAll("");
There is a thus-far undocumented APOC function, apoc.text.replace, that you can use from your Cypher code. It accepts a regular expression as its second parameter. (Since it is a function, it is not invoked in a CALL clause.)
For example:
RETURN apoc.text.replace('~!#1~!#', '[~!##$%^&*() ]', '') AS res;
returns:
╒═══╕
│res│
╞═══╡
│1 │
└───┘