I need a regex to obfuscate emails in a database dump file I have. I'd like to replace all domains with a set domain like #fake.com so I don't risk sending out emails to real people during development. The emails do have to be unique to match database constraints, so I only want to replace the domain and keep the usernames.
I current have this regex for finding emails
\b[A-Z0-9._%-]+#[A-Z0-9.-]+\.[A-Z]{2,4}\b
How do I convert this search regex into a regex I can use in a find and replace operation in either Sublime Text or SED or Vim?
EDIT:
Just a note, I just realized I could replace all strings found by #[A-Z0-9.-]+\.[A-Z]{2,4}\b in this case, but academically I am still interested in how you could treat each section of the email regex as a token and replace the username / domain independently.
SublimeText
SublimeText uses Boost syntax, which supports quite a large subset of features in Perl regex. But for this task, you don't need all those advanced constructs.
Below are 2 possible approaches:
If you can assume that # doesn't appear in any other context (which is quite a fair assumption for normal text), then you can just search for the domain part #[A-Z0-9.-]+\.[A-Z]{2,4}\b and replace it.
If you use capturing groups (pattern) and backreference in replacement string.
Find what
\b([A-Z0-9._%-]+)#[A-Z0-9.-]+\.[A-Z]{2,4}\b
([A-Z0-9._%-]+) is the first (and only) capturing group in the regex.
Replace with
$1#fake.com
$1 refers to the text captured by the first capturing group.
Note that for both methods above, you need to turn off case-sensitivity (indicated as the 2nd button on the lower left corner), unless you specifically want to remove only emails written in ALL CAPS.
You may use the following command for Vim:
:%s/\(\<[A-Za-z0-9._%-]\+#\)[A-Za-z0-9.-]\+\.[A-Za-z]\{2,4}\>/\1fake.com/g
Everything between \( and \) will become a group that will be replaced by an escaped number of the group (\1 in this case). I've also modified the regexp to match the small letters and to have Vim-compatible syntax.
Also you may turn off the case sensitivity by putting \c anywhere in your regexp like this:
:%s/\c\(\<[A-Z0-9._%-]\+#\)[A-Z0-9.-]\+\.[A-Z]\{2,4}\>/\1fake.com/g
Please also note that % in the beginning of the line asks Vim to do the replacement in a whole file and g at the end to do multiple replacements in the same line.
One more approach is using the zero-width matching (\#<=):
:%s/\c\(\<[A-Z0-9._%-]\+#\)\#<=[A-Z0-9.-]\+\.[A-Z]\{2,4}\>/fake.com/g
Related
I need to find all email addresses with an arbitrary number of alphanumeric words, separated through a period. To test the regex, I'm using the website https://regex101.com/.
The structure of a valid email addresses is word1.word2.wordN#word1.word2.wordN.word.
The regex /[a-zA-Z0-9.]+#[a-zA-Z0-9.]+.[a-zA-Z0-9]+/gm finds all email addresses included in the document string, but also includes invalid addresses like ........#....com, if present.
I tried to group the repeating parts by using round brackets and a Kleene star, but that causes the regex engine to collapse.
Invalid regex:
/([a-zA-Z0-9]+.?)*[a-zA-Z0-9]+#([a-zA-Z0-9]+.?)*[a-zA-Z0-9]+.[a-zA-Z0-9]+/gm
Although there are many posts concerning regex groups, I was unable to find an explanation, why the regex engine fails. It seems that the engine gets stuck, while trying to find a match.
How can I avoid this problem, and what is the correct solution?
I think the main issue that caused you troubles is:
. (outside of []) matches any character,you probably meant to specify \. instead (only matches literal dot character).
Also there is no need to make it optional with ?, because the non-dot part of your regex will just match with the alphanumerical characters anyway.
I also reduced the right part (x*x is the same as x+), added a case-insensitive flag and ended up with this:
/([a-z0-9]+\.)*[a-z0-9]+#([a-z0-9]+\.)+[a-z0-9]+/gmi
I have a need to replace this:
fixed variable 123
with this:
fixed variable 234
In VSCode this matches fine:
fixed(.*)123
I can't find any way to make it put the capture in the output if a number follows:
fixed$1234
fixed${1}234
But the find replace window just looks like this:
I read that VSCode uses rust flavoured rexes.. Here indicates ${1}234 should work, but VSCode just puts it in the output..
Tried named capture in a style according to here
fixed(?P<n>.*)123 //"invalid regular expression" error
VSCode doesn't seem to understand ${1}:
ps; I appreciate I could hack it in the contrived example with
FIND: fixed (.*) 123
REPL: fixed $1 234
And this does work in vscode:
but not all my data consistently has the same character before the number
After a lot of investigation by myself and #Wiktor we discovered a workaround for this apparent bug in vscode's search (aka find across files) and replace functionality in the specific case where the replace would have a single capture group followed by digits, like
$1234 where the intent is to replace with capture group 1 $1 followed by 234 or any digits. But $1234 is the actual undesired replaced output.
[This works fine in the find/replace widget for the current file but not in the find/search across files.]
There are (at least) two workarounds. Using two consecutive groups, like $1$2234 works properly as does $1$`234 (or precede with the $backtick).
So you could create a sham capture group as in (.*?)()(\d{3}) where capture group 2 has nothing in it just to get 2 consecutive capture groups in the replace or
use your intial search regex (.*?)(\d{3}) and then use $` just before or after your "real" capture group $1.
OP has filed an issue https://github.com/microsoft/vscode/issues/102221
Oddly, I just discovered that replacing with a single digit like $11 works fine but as soon as you add two or more it fails, so $112 fails.
I'd like to share some more insights and my reasoning when I searched for a workaround.
Main workaround idea is using two consecutive backreferences in the replacement.
I tried all backreference syntax described at Replacement Strings Reference: Matched Text and Backreferences. It appeared that none of \g<1>, \g{1}, ${1}, $<1>, $+{1}, etc. work. However, there are some other backreferences, like $' (inserts the portion of the string that follows the matched substring) or $` (inserts the portion of the string that precedes the matched substring). However, these two backreferences do not work in VS Code file search and replace feature, they do not insert any text when used in the replacement pattern.
So, we may use $` or $' as empty placeholders in the replacement pattern.
Find What: fix(.*?)123
Replace With:
fix$'$1234
fix$`$1234
Or, as in my preliminary test, already provided in Mark's answer, a "technical" capturing group matching an empty string, (), can be introduced into the pattern so that a backreference to that group can be used as a "guard" before the subsequent "meaningful" backreference:
Find What: fixed()(.*)123 (see () in the pattern that can be referred to using $1)
Replace With: fixed$1$2234
Here, $1 is a "guard" placeholder allowing correct parsing of $2 backreference.
Side note about named capturing groups
Named capturing groups are supported, but you should use .NET/PCRE/Java named capturing group syntax, (?<name>...). Unfortunately, the none of the known named backreferences work replacement pattern. I tried $+{name} Boost/Perl syntax, $<name>, ${name}, none work.
Conclusion
So, there are several issues here that need to be addressed:
We need an unambiguous numbered backerence syntax (\g<1>, ${1}, or $<1>)
We need to make sure $' or $` work as expected or are parsed as literal text (same as $_ (used to include the entire input string in the replacement string) or $+ (used to insert the text matched by the highest-numbered capturing group that actually participated in the match) backreferences that are not recognized by Visual Studio Code file search and replace feature), current behavior when they do not insert any text is rather undefined
We need to introduce named backreference syntax (like \g<name> or ${name}).
I am looking for alternate methods to get john from the provided example.
My expression works as is but was hoping for some examples of better methods.
Example: john&home
my regexp: [a-z]{3,6}[^&home]
Im matching any character of length 3-6 upto but not including &home
Every item i run the regexp on is in the same format. 3-6 characters followed by &home
I have looked at other posts but was hoping for a reply specific to my regexp.
Most regex engines allow you to capture parts of a regex with capture groups. For instance:
^([A-Za-z]{3,6})&home$
The brackets here mean that you are interested in the part before the &home. The ^ and $ mean that you want to match the entire string. Without it, averylongname&homeofsomeone will be matched as well.
Since you use rubular, I assume you use the Ruby regex engine. In that case you can for instance use:
full = "john&home"
name = full.match(/^([A-Za-z]{3,6})&home$/).captures
And name will in this case contain john.
I am having an issue with trying to figure out how to insert some text after I perform a regex search. I know there is a replace function, but I am not looking for that option, just inserting. The text editor I am using is Notepad2, but I am willing to try this in other text editors.
Here is the example that I have.
TEST|Test2|Test3|Test4
This is what I am looking for
Test|Test2|PrefixTest3|Test4
Notice that I am trying to insert the the phrase "Prefix" after the 2nd pipe and leave everything else alone.
I can successfully query the result by using this regex:
^[^|]*\|[^|]*|
But then I do not know how I can retain everything prior and after the search point. Any ideas?
You could simply use \K inorder to discard the previously matched characters.
^[^|]*\|[^|]*\|\K
Then replace the match with the string prefix.
DEMO
You may easily do that in Notepad2 using the regex-based Replace feature.
Find: ^\([^|]*|[^|]*|\)
Replace: \1Prefix
Details:
^ - start of a line (Notepad2 never overflows line boundaries!)
\([^|]*|[^|]*|\) - Capturing group 1 matching a sequence of:
[^|]* - zero or more chars other than |
| - a literal (yes, no escaping is necessary, both escaped and unescaped | match a literal |) pipe symbol
[^|]*| - see above, gets to the second |.
The replacement contains a \1 backreference that inserts what was captured with the capturing group 1.
NOTE that Notepad2 regex engine is very limited. Here is what the Notepad2 documentation says:
Notepad2 supports only a limited subset of regular expressions, as provided by built-in engine of the Scintilla source code editing component. The advantage is that it has a very small footprint. There's currently no plans to integrate a more advanced regular expressions engine, but this may be an option for future development.
Note: Regular expression search is limited to single lines, only.
Also, you may refer to the inline comments inside Scintilla RESearch.cxx file describing the supported syntax. Bear in mind that the regex type used in the Notepad2 S&R tool is that of POSIX and not all of the described Scintilla regex features will work in the tool.
Note that Notepad2 does not seem to support alternation and limiting quantifiers (similar to Lua patterns), but \w matches Unicode letters together with ASCII ones. Sadly, I could not make ? quantifier work.
^([^|]*\|[^|]*\|)
Try this.Replace by $1prefix.See demo.Just capture the first group and then use it for replace.The first group can be accessed by $1.
http://regex101.com/r/pQ9bV3/11
I'm struggling to understand regex's in powershell. On Friday user mike-z helped me out with a script to extract the number from a group of folders with a naming convention like this -
Core_1.1.2
Core_1.3.4
The following regex;
-replace '.*_(\d+(\.\d+){1,3})', '$1')
works perfectly to extract only the numbers (eg "1.1.2").
Unfortunately, I later realized that a couple of the folder names had some other junk text trailing the version numbers (eg. Core_1.2.4_Prod). I tried on my own to tweak the above regex to make it also ignore the trailing text but I didn't get too far. I used various online regex generators as well as my own limited regex experience but I didn't get anywhere; I was able to generate regexes which should capture the text I need but they didn't work in powershell. Converesely, the working regex above (as in it works in powershell) fails in any regex tool I used.
Basically, given a list of folder names like this
Core_1.1.2
Core_1.2.4_Prod
Core_1.2.6
Core_1.3.1_Prod
Core_1.4.4
I need to capture just the version number. Also, it would be greatly appreciated if you could explain why the regex works as I'm extremely confused by PS regexes at this point.
You just need to add .* at the last in your pattern,
.*_(\d+(\.\d+){1,3}).*
So your code would be,
-replace '.*_(\d+(\.\d+){1,3}).*', '$1')
DEMO
By default replace function in all the languages should replace the matched characters only. Your regex .*_(\d+(\.\d+){1,3}) matches upto the last digit in the version number. It won't match the remaining part. So whenever you replace the matched characters with $1, the trailing part _Prod should be printed along with the characters inside the first capturing group , because the trailing part is not matched. Just match also the trailing part , inorder to replace the whole line with $1(ie; version number).