I just want to create a regular expression out of any possible string.
var usersString = "Hello?!*`~World()[]";
var expression = new RegExp(RegExp.escape(usersString))
var matches = "Hello".match(expression);
Is there a built-in method for that? If not, what do people use? Ruby has RegExp.escape. I don't feel like I'd need to write my own, there have got to be something standard out there.
The function linked in another answer is insufficient. It fails to escape ^ or $ (start and end of string), or -, which in a character group is used for ranges.
Use this function:
function escapeRegex(string) {
return string.replace(/[/\-\\^$*+?.()|[\]{}]/g, '\\$&');
}
While it may seem unnecessary at first glance, escaping - (as well as ^) makes the function suitable for escaping characters to be inserted into a character class as well as the body of the regex.
Escaping / makes the function suitable for escaping characters to be used in a JavaScript regex literal for later evaluation.
As there is no downside to escaping either of them, it makes sense to escape to cover wider use cases.
And yes, it is a disappointing failing that this is not part of standard JavaScript.
For anyone using Lodash, since v3.0.0 a _.escapeRegExp function is built-in:
_.escapeRegExp('[lodash](https://lodash.com/)');
// → '\[lodash\]\(https:\/\/lodash\.com\/\)'
And, in the event that you don't want to require the full Lodash library, you may require just that function!
Most of the expressions here solve single specific use cases.
That's okay, but I prefer an "always works" approach.
function regExpEscape(literal_string) {
return literal_string.replace(/[-[\]{}()*+!<=:?.\/\\^$|#\s,]/g, '\\$&');
}
This will "fully escape" a literal string for any of the following uses in regular expressions:
Insertion in a regular expression. E.g. new RegExp(regExpEscape(str))
Insertion in a character class. E.g. new RegExp('[' + regExpEscape(str) + ']')
Insertion in integer count specifier. E.g. new RegExp('x{1,' + regExpEscape(str) + '}')
Execution in non-JavaScript regular expression engines.
Special Characters Covered:
-: Creates a character range in a character class.
[ / ]: Starts / ends a character class.
{ / }: Starts / ends a numeration specifier.
( / ): Starts / ends a group.
* / + / ?: Specifies repetition type.
.: Matches any character.
\: Escapes characters, and starts entities.
^: Specifies start of matching zone, and negates matching in a character class.
$: Specifies end of matching zone.
|: Specifies alternation.
#: Specifies comment in free spacing mode.
\s: Ignored in free spacing mode.
,: Separates values in numeration specifier.
/: Starts or ends expression.
:: Completes special group types, and part of Perl-style character classes.
!: Negates zero-width group.
< / =: Part of zero-width group specifications.
Notes:
/ is not strictly necessary in any flavor of regular expression. However, it protects in case someone (shudder) does eval("/" + pattern + "/");.
, ensures that if the string is meant to be an integer in the numerical specifier, it will properly cause a RegExp compiling error instead of silently compiling wrong.
#, and \s do not need to be escaped in JavaScript, but do in many other flavors. They are escaped here in case the regular expression will later be passed to another program.
If you also need to future-proof the regular expression against potential additions to the JavaScript regex engine capabilities, I recommend using the more paranoid:
function regExpEscapeFuture(literal_string) {
return literal_string.replace(/[^A-Za-z0-9_]/g, '\\$&');
}
This function escapes every character except those explicitly guaranteed not be used for syntax in future regular expression flavors.
For the truly sanitation-keen, consider this edge case:
var s = '';
new RegExp('(choice1|choice2|' + regExpEscape(s) + ')');
This should compile fine in JavaScript, but will not in some other flavors. If intending to pass to another flavor, the null case of s === '' should be independently checked, like so:
var s = '';
new RegExp('(choice1|choice2' + (s ? '|' + regExpEscape(s) : '') + ')');
Mozilla Developer Network's Guide to Regular Expressions provides this escaping function:
function escapeRegExp(string) {
return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string
}
In jQuery UI's autocomplete widget (version 1.9.1) they use a slightly different regular expression (line 6753), here's the regular expression combined with bobince's approach.
RegExp.escape = function( value ) {
return value.replace(/[\-\[\]{}()*+?.,\\\^$|#\s]/g, "\\$&");
}
There is an ES7 proposal for RegExp.escape at https://github.com/benjamingr/RexExp.escape/, with a polyfill available at https://github.com/ljharb/regexp.escape.
Nothing should prevent you from just escaping every non-alphanumeric character:
usersString.replace(/(?=\W)/g, '\\');
You lose a certain degree of readability when doing re.toString() but you win a great deal of simplicity (and security).
According to ECMA-262, on the one hand, regular expression "syntax characters" are always non-alphanumeric, such that the result is secure, and special escape sequences (\d, \w, \n) are always alphanumeric such that no false control escapes will be produced.
There is an ES7 proposal for RegExp.escape at https://github.com/benjamingr/RexExp.escape/, with a polyfill available at https://github.com/ljharb/regexp.escape.
An example based on the rejected ES proposal, includes checks if the property already exists, in the case that TC39 backtracks on their decision.
Code:
if (!Object.prototype.hasOwnProperty.call(RegExp, 'escape')) {
RegExp.escape = function(string) {
// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions#Escaping
// https://github.com/benjamingr/RegExp.escape/issues/37
return string.replace(/[.*+\-?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string
};
}
Code Minified:
Object.prototype.hasOwnProperty.call(RegExp,"escape")||(RegExp.escape=function(e){return e.replace(/[.*+\-?^${}()|[\]\\]/g,"\\$&")});
// ...
var assert = require('assert');
var str = 'hello. how are you?';
var regex = new RegExp(RegExp.escape(str), 'g');
assert.equal(String(regex), '/hello\. how are you\?/g');
There is also an npm module at:
https://www.npmjs.com/package/regexp.escape
One can install this and use it as so:
npm install regexp.escape
or
yarn add regexp.escape
var escape = require('regexp.escape');
var assert = require('assert');
var str = 'hello. how are you?';
var regex = new RegExp(escape(str), 'g');
assert.equal(String(regex), '/hello\. how are you\?/g');
In the GitHub && NPM page are descriptions of how to use the shim/polyfill for this option, as well. That logic is based on return RegExp.escape || implementation;, where implementation contains the regexp used above.
The NPM module is an extra dependency, but it also make it easier for an external contributor to identify logical parts added to the code. ¯\(ツ)/¯
Another (much safer) approach is to escape all the characters (and not just a few special ones that we currently know) using the unicode escape format \u{code}:
function escapeRegExp(text) {
return Array.from(text)
.map(char => `\\u{${char.charCodeAt(0).toString(16)}}`)
.join('');
}
console.log(escapeRegExp('a.b')); // '\u{61}\u{2e}\u{62}'
Please note that you need to pass the u flag for this method to work:
var expression = new RegExp(escapeRegExp(usersString), 'u');
This is a shorter version.
RegExp.escape = function(s) {
return s.replace(/[$-\/?[-^{|}]/g, '\\$&');
}
This includes the non-meta characters of %, &, ', and ,, but the JavaScript RegExp specification allows this.
XRegExp has an escape function:
XRegExp.escape('Escaped? <.>');
// -> 'Escaped\?\ <\.>'
More on: http://xregexp.com/api/#escape
escapeRegExp = function(str) {
if (str == null) return '';
return String(str).replace(/([.*+?^=!:${}()|[\]\/\\])/g, '\\$1');
};
Rather than only escaping characters which will cause issues in your regular expression (e.g.: a blacklist), consider using a whitelist instead. This way each character is considered tainted unless it matches.
For this example, assume the following expression:
RegExp.escape('be || ! be');
This whitelists letters, number and spaces:
RegExp.escape = function (string) {
return string.replace(/([^\w\d\s])/gi, '\\$1');
}
Returns:
"be \|\| \! be"
This may escape characters which do not need to be escaped, but this doesn't hinder your expression (maybe some minor time penalties - but it's worth it for safety).
The functions in the other answers are overkill for escaping entire regular expressions (they may be useful for escaping parts of regular expressions that will later be concatenated into bigger regexps).
If you escape an entire regexp and are done with it, quoting the metacharacters that are either standalone (., ?, +, *, ^, $, |, \) or start something ((, [, {) is all you need:
String.prototype.regexEscape = function regexEscape() {
return this.replace(/[.?+*^$|({[\\]/g, '\\$&');
};
And yes, it's disappointing that JavaScript doesn't have a function like this built-in.
I borrowed bobince's answer above and created a tagged template function for creating a RegExp where part of the value is escaped and part isn't.
regex-escaped.js
RegExp.escape = text => text.replace(/[\-\[\]{}()*+?.,\\\^$|#\s]/g, '\\$&');
RegExp.escaped = flags =>
function (regexStrings, ...escaped) {
const source = regexStrings
.map((s, i) =>
// escaped[i] will be undefined for the last value of s
escaped[i] === undefined
? s
: s + RegExp.escape(escaped[i].toString())
)
.join('');
return new RegExp(source, flags);
};
function capitalizeFirstUserInputCaseInsensitiveMatch(text, userInput) {
const [, before, match, after ] =
RegExp.escaped('i')`^((?:(?!${userInput}).)*)(${userInput})?(.*)$`.exec(text);
return `${before}${match.toUpperCase()}${after}`;
}
const text = 'hello (world)';
const userInput = 'lo (wor';
console.log(capitalizeFirstUserInputCaseInsensitiveMatch(text, userInput));
For our TypeScript fans...
global.d.ts
interface RegExpConstructor {
/** Escapes a string so that it can be used as a literal within a `RegExp`. */
escape(text: string): string;
/**
* Returns a tagged template function that creates `RegExp` with its template values escaped.
*
* This can be useful when using a `RegExp` to search with user input.
*
* #param flags The flags to apply to the `RegExp`.
*
* #example
*
* function capitalizeFirstUserInputCaseInsensitiveMatch(text: string, userInput: string) {
* const [, before, match, after ] =
* RegExp.escaped('i')`^((?:(?!${userInput}).)*)(${userInput})?(.*)$`.exec(text);
*
* return `${before}${match.toUpperCase()}${after}`;
* }
*/
escaped(flags?: string): (regexStrings: TemplateStringsArray, ...escapedVals: Array<string | number>) => RegExp;
}
There has only ever been and ever will be 12 meta characters that need to be escaped
to be considered a literal.
It doesn't matter what is done with the escaped string, inserted into a balanced regex wrapper or appended. It doesn't matter.
Do a string replace using this
var escaped_string = oldstring.replace(/[\\^$.|?*+()[{]/g, '\\$&');
This one is the permanent solution.
function regExpEscapeFuture(literal_string) {
return literal_string.replace(/[^A-Za-z0-9_]/g, '\\$&');
}
Just published a regex escape gist based on the RegExp.escape shim which was in turn based on the rejected RegExp.escape proposal. Looks roughly equivalent to the accepted answer except it doesn't escape - characters, which seems to be actually fine according to my manual testing.
Current gist at the time of writing this:
const syntaxChars = /[\^$\\.*+?()[\]{}|]/g
/**
* Escapes all special special regex characters in a given string
* so that it can be passed to `new RegExp(escaped, ...)` to match all given
* characters literally.
*
* inspired by https://github.com/es-shims/regexp.escape/blob/master/implementation.js
*
* #param {string} s
*/
export function escape(s) {
return s.replace(syntaxChars, '\\$&')
}
I'm having trouble escaping double dollars from a string to be used with regex functions pattern/matcher.
This is part of the String:
WHERE oid_2 = $$test$$ || oid_2 = $$test2$$
and this is the closest code I've tried to get near the solution:
List<String> strList = new ArrayList<String>();
Pattern pattern = Pattern.compile("\$\$.*?\$\$");
log.debug("PATTERN: "+pattern)
Matcher matcher = pattern.matcher(queryText);
while (matcher.find()) {
strList.add(matcher.group());
}
log.debug(strList)
This is the debug output i get
- PATTERN: $$.*?$$
- []
So the pattern is actually right, but the placeholders are not found in the string.
As a test I've tried to replace "$$test$$" with "XXtestXX" and everything works perfectly. What am I missing? I've tried "/$" strings, "\\" but still have no solution.
Note that a $ in regex matches the end of the string. To use it as a literal $ symbol, you need to escape it with a literal backslash.
You used "\$\$.*?\$\$" that got translated into a literal string like $$.*?$$ that matches 2 end of string positions, any 0+ chars as few as possible and then again 2 end of strings, which has little sense. You actually would need a backslash to first escape the $ that is used in Groovy to inject variables into a double quoted string literal, and then use 2 backslashes to define a literal backslash - "\\\$\\\$.*?\\\$\\\$".
However, when you work with regex, slashy strings are quite helpful since all you need to escape a special char is a single backslash.
Here is a sample code extracting all matches from the string you have in Groovy:
def regex = /\$\$.*?\$\$/;
def s = 'WHERE oid_2 = $$test$$ || oid_2 = $$test2$$'
def m = s =~ regex
(0..<m.count).each { print m[it] + '\n' }
See the online demo.
Anyone who gets here might like to know another answer to this, if you want to use Groovy slashy strings:
myComparisonString ==~ /.*something costs [$]stuff.*/
I couldn't find another way of putting a $ in a slashy string, at least if the $ is to be followed by text. If, conversely, it is followed by a number (or presumably any non-letter), this will work:
myComparisonString ==~ /.*something costs \$100.*/
... the trouble being, of course, that the GString "compiler" (if that's its name) would recognise "$stuff" as an interpolated variable.
When I run the code below I get the unexpected result where \* also captures É. Is there a way to make it only capture * like I wanted?
let s =
"* A
ÉR
* B"
let result = System.Text.RegularExpressions.Regex.Replace(s, "\n(?!\*)", "", Text.RegularExpressions.RegexOptions.Multiline)
printfn "%s" result
Result After Running Replace
* AÉR
* B
Expected Result
"* A
ÉR
* B"
UPDATE
This seems to be working, when I use a pattern like so \n(?=\*). I guess I needed a positive lookahead.
You're right that you need to use positive lookahead instead of negative lookahead to get the result you want. However, to clarify an issue that came up in the comments, in F# a string delimited by just "" is not quite like either a plain C# string delimited by "" or a C# string delimited by #"" - if you want the latter you should also use #"" in F#. The difference is that in a normal F# string, backslashes will be treated as escape sequences only when used in front of a valid character to escape (see the table towards the top of Strings (F#)). Otherwise, it is treated as a literal backslash character. So, since '*' is not a valid character to escape, you luckily see the behavior you expected (in C#, by contrast, this would be a syntax error because it's an unrecognized escape). I would recommend that you not rely on this and should use a verbatim #"" string instead.
In other words, in F# the following three strings are all equivalent:
let s1 = "\n\*"
let s2 = "\n\\*"
let s3 = #"
\*"
I think that the C# design is more sensible because it prevents confusion on what exactly is being escaped.
I wrote a basic mp3 organizing script for myself. I have the line: $outname = "/home/jebsky/safehouse/music/mp3/" . $inital . "/" . $artist . "/" . $year ." - ". $album . "/" . $track ." - ". $artist ." - ". $title . ".mp3";
I want a regex to change $outname so that any non safe for filename characters get replaced by an underscore
If any of your components include "/", you really want to do the substitution on them before assembling them into $outname.
Which characters are safe can vary from one operating system and/or filesystem to another.
Many filesystems have no problem with any characters other than "/" and nul. You're probably better off deciding which characters you want to keep, for other reasons than what your filesystem allows.
The following keeps only letters and digits, replacing sequences of other characters with _:
for ( $initial, $artist, $year, $album, $track, $title ) {
s/[^A-Za-z0-9]+/_/g;
}
One quick way to escape all non-alphabetic characters in a string is to use the \Q and \U operators, as in:
# assuming $outname already contains the required path and
# globally "unescaping" file chars / and .
($outname = "\Q$outname\U") =~ s/\\([\/\.])/$1/g;
One thing to consider is that long run-on string cats like you have tend to both be hard to read and maintain. A better way of representing this operation might be to break it up into logical units, like:
my $basename = '/home/jebsky/safehouse/music/mp3';
my $dirpath = "${basename}/${initial}/${artist}/${year}-${album}/";
my $filename = "${track}-${artist}-${title}.mp3";
$outname = "${dirpath}/${filename}";
Within strings, representing a variable as "${varname}" assures that the character that follows the varname cannot interfere with it and is usually a good idea even if the next character after the var isn't alphanumeric because it clearly marks variables within the string.
Finally, I think it's a good idea to get away from using '"' and '\'' as string delimiters since they require quoting if the string contains the delimiter.
Use the qq// and q// delimiters (replacing the / with a char not appearing in the string if required) instead, as in:
my $basename = q!/home/jebsky/safehouse/music/mp3!;
my $dirpath = qq!${basename}/${initial}/${artist}!;
my $filename = qq!${year}-${album}/${track}-${artist}-${title}.mp3!;
$outname = qq!${dirpath}/${filename}!;
This way, you'll rarely have to quote any char in the string.