How to use Postgres Regex Replace with a capture group - regex

As the title presents above I am trying to reference a capture groups for a regex replace in a postgres query. I have read that the regex_replace does not support using regex capture groups. The regex I am using is
r"(?:[\s\(\)\=\)\,])(username)(?:[\s\(\)\=\)\,])?"gm
The above regex almost does what I need it to but I need to find out how to only allow a match if the capture groups also capture something. There is no situation where a "username" should be matched if it just so happens to be a substring of a word. By ensuring its surrounded by one of the above I can much more confidently ensure its a username.
An example application of the regex would be something like this in postgres (of course I would be doing an update vs a select):
select *, REGEXP_REPLACE(reqcontent,'(?:[\s\(\)\=\)\,])(username)(?:[\s\(\)\=\)\,])?' ,'NEW-VALUE', 'gm') from table where column like '%username%' limit 100;
If there is any more context that can be provided please let me know. I have also found similar posts (postgresql regexp_replace: how to replace captured group with evaluated expression (adding an integer value to capture group)) but that talks more about splicing in values back in and I don't think quite answers my question.
More context and example value(s) for regex work against. The below text may look familiar these are JQL filters in Jira. We are looking to update our usernames and all their occurrences in the table that contains the filter. Below is a few examples of filters. We originally were just doing a find a replace but that doesn't work because we have some usernames that are only two characters and it was matching on non usernames (e.g je (username) would place a new value in where the word project is found which completely malforms the JQL/String resulting in something like proNEW-VALUEct = balh blah)
type = bug AND status not in (Closed, Executed) AND assignee in (test, username)
assignee=username
assignee = username
Definition of Answered:
Regex that will only match on a 'username' if its surrounded by one of the specials
A way to regex/replace that username in a postgres query.

Capturing groups are used to keep the important bits of information matched with a regex.
Use either capturing groups around the string parts you want to stay in the result and use their placeholders in the replacement:
REGEXP_REPLACE(reqcontent,'([\s\(\)\=\)\,])username([\s\(\)\=\)\,])?' ,'\1NEW-VALUE\2', 'gm')
Or use lookarounds:
REGEXP_REPLACE(reqcontent,'(?<=[\s\(\)\=\)\,])(username)(?=[\s\(\)\=\)\,])?' ,'NEW-VALUE', 'gm')
Or, in this case, use word boundaries to ensure you only replace a word when inside special characters:
REGEXP_REPLACE(reqcontent,'\yusername\y' ,'NEW-VALUE', 'g')

Related

Perform Regex on value returned by Regex

This is probably straightforward but I'm not even sure which phrase I should google to find the answer. Forgive my noobiness.
I've got strings (filenames) that look like this:
site12345678_date20160912_23001_to_23100_of_25871.txt
What this naming convention means is "Records 23001 through 23100 out of 25871 for site 12345678 for September 12th 2016 (20160912)"
What I want to do is extract the date part (those digits between _date and the following _)
The Regex: .*(_date[0-9]{8}).* will return the string _date20160912. But what I'm actually looking for is just 20160912. Obviously, [0-8]{8} doesn't give me what I want in this case because that could be confused with the site, or potentially record counts
How can I responsibly accomplish this sort of 'substringing' with a single regular expression?
You just need to shift you parentheses so as to change the capture group from including '_date' in it. Then you would want to look for your capture group #1:
If done in python, for example, it would look something like:
import re
regex = '.*_date([0-9]{8}).*'
str = 'site12345678_date20160912_23001_to_23100_of_25871.txt'
m = re.match(regex, str)
print m.group(0) # the whole string
print m.group(1) # the string you are looking for '20160912'
See it in action here: https://eval.in/641446
The Regex: .*(_date[0-9]{8}).* will return the string _date20160912.
That means you are using the regex in a method that requires a full string match, and you can access Group 1 value. The only thing you need to change in the regex is the capturing group placement:
.*_date([0-9]{8}).*
^^^^^^^^^^
See the regex demo.

Google Analytics - Content grouping - Regex fix

This is our URL structure:
http://www.disabledgo.com/access-guide/the-university-of-manchester/176-waterloo-place-2
http://www.disabledgo.com/access-guide/kingston-university/coombehurst-court-2
http://www.disabledgo.com/access-guide/kings-college-london/franklin-wilkins-building-2
http://www.disabledgo.com/access-guide/redbridge-college/brook-centre-learning-resource-centre
I am trying to create a list of groups based on the client names
/access-guide/[this bit]/...
So I can have a performance list of all our clients.
This is my regex:
/access-guide/(.*universit(y|ies)|.*colleg(e|es))/
I want it to group anything that has university/ies or college/es in it, at any point within that client name section of the URL.
At the moment, my current regex will only return groups that are X-University:
Durham-University
Plymouth-University
Cardiff-University
etc.
What does the regex need to be to have the list I'm looking for?
Do I need to have something at the end to stop it matching things after the client name? E.g. ([^/]+$)?
Thanks for your help in advance!
Depending upon your needs you may want to do:
/access-guide/([^/]*(?:university|universities|college|colleges)[^/]*)/
This will match names even if "university" or "college" is not at the end of the string. For example "college-of-the-ozarks" Note the non-capturing internal parenthesis, that should probably be used no matter what solution you go with, as you don't want to just match the word "university" or "college"
Live Example
Additionally, I don't know what may be in your but if you may have compound words you want to eliminate using a \b may be advisable. For instance if you don't want to match "miskatonic-postcollege" you may want to do something like this:
/access-guide/([^/]*\b(?:university|universities|college|colleges)\b[^/]*)/
If the client name section of the URL is after the access-guid/ and before the next /:
http://www.disabledgo.com/access-guide/the-university-of-manchester/176-waterloo-place-2
|----------------------------|
you need to use a negated character class to only match university before the regex reaches that rightmost / boundary.
As per the Reference:
You can extract pages by Page URL, Page Title, or Screen Name. Identify each one with a regex capture group (Analytics uses the first capture group for each expression)
Thus, you can use
/access-guide/([^/]*(universit(y|ies)|colleges?))
^^^^^
See demo.
The regex matches
/access-guide/ - leftmost boundary, matches /access-guide/ literally
[^/]* - any character other than / (so we still remain in that customer section)
(universit(y|ies)|colleges?) - university, or universities, orcollegeorcolleges` literally. Add more if needed.

Oracle - Search for text - Retrieve snippet of result

I'm currently building a simple search page in Node JS Express and Oracle.
I'd like to show the user a snippet of the matching text (first instance would do) to add a bit context of what the SQL found.
Example:
Search term: 'fish'
Results: Henry really likes going fishing, and once he caug ...
I'm not sure the best way to approach this - I could retrieve the whole block of text and do it in Node JS, but I don't really like the idea of dragging the whole text across to the app, just to get a snippet.
I've been thinking that REGEXP_SUBSTR could be way to do it... But I'm not sure whether I could use a regular expression to retrieve x amount of characters before and after the matching word.
Have I got the right idea or am I going about it in the wrong way?
Thanks
SELECT text
, REGEXP_SUBSTR(LOWER(text), LOWER('fish')) AS potential_snippet
FROM table
WHERE LOWER(text) LIKE LOWER('%fish%');
Try this:
select text
, SUBSTR( TEXT, INSTR(LOWER(TEXT),'fish', 1)-50,100 )
FROM test
WHERE INSTR(LOWER(text),'fish', 1)<>0;
Play with the position and length numbers(50 and 100 in my example) to limit the length of the string.
If you need to extract some context with the help of JavaScript, you can use limiting quantifiers in a regex:
/\b.{0,15}fish.{0,15}\b/i
See demo
Here,
\b - matches at the word boundary (so that the context contains only whole words)
.{0,15} - any characters other than a newline (replace with [\s\S] or [^] if you need to include newlines)
fish - the keyword
The /i modifier enables case-insensitive search.
If you need a dynamic regex creation, use a constructor notation:
RegExp("\\b.{0,15}" + keyword + ".{0,15}\\b", "i");
Also, if you need to find multiple matches, use g modifier alongside the i.

Regex pattern for containing string as well as not ending with pattern

I have been asked to make 2 regex to determine by the URL if a page is a product page or a category page.
These are the URLs:
Product page: www.domain.com/art/something/someotherthing/article(X123456.123)/
Category page: www.domain.com/art/something/someotherthing
I created this regex which works fine for the product page:
^.*\/art.*\/[xX]?[0-9]{6,7}\.[0-9]+\/$
Now I have problems with the category page. The only thing I see that is possible is to make sure it does not end with the pattern that check the ending numbers "[xX]?[0-9]{6,7}.[0-9]+". But I also need to make sure that it starts with /art/ after the domain.
My first try was this for the category page:
.*\/art.*\/(?!([xX]?[0-9]{6,7}\.[0-9]+\(\/)?))$
This doesn't work since negative lookup is positive since it does not find the pattern after the 2nd any characters matching (.*).
Looks like a differencing factor is the number of slashes, possibly excluded by an optional end-slash that is often ignored.
^[^\/]*(\/[^\/]*){3}\/?$ would match the category, and
^[^\/]*(\/[^\/]*){4}\/?$ would match the product.
I think you don't have to use any lookarounds here.
Since the domain is permanent and the art is permanent and the last part of the product like article+something is permanent you can use them explicitly in the regex making it faster.
For product:
^www\.domain\.com\/art\/[^\/]+\/[^\/]+\/article\([^\/]+\)\/$
For category:
^www\.domain\.com\/art\/[^\/]+\/[^\/]+\/$
From the question description and the URL data given...
Product URLs
matched by ^([^\/\r\n]+?)\/(art)\/(.*)\/.*?\(([xX]?[0-9]{6,7}\.[0-9]+)\).*?\/?$
1st capture == domain
2nd capture == art (main category?)
3rd capture == category
4th capture == Product ID
Category URLs
matched by ^([^\/\r\n]+?)\/(art)\/((?!.*[xX]?[0-9]{6,7}\.[0-9]+).*?)\/?$
1st capture == domain
2nd capture == art (main category?)
3rd capture == category
I did infer that the trailing / was optional for both URLs, but that may be an incorrect assumption.
The above regex's link to live regex101 fiddlers with the given regex plus test data.
Do note that the \r\n inclusion within the character class for the domain match is only needed because the regex101 fiddler match is done globally on combined test data. You can remove that character sequence if you are only matching against a single URL at a time.

regex using positive lookahead

My source data text looks something like this:
a1,a2,a3
a4,a5,a6
a7,a8,a9
test="1"
b1,b2,b3
b4,b5,b6
b7,b8,b9
test="2"
c1,c2,c3
c4,c5,c6
c7,c8,c9
test="3"
I need to parse this so the end result looks like this (appropriate “test” field included in each row):
a1,a2,a3,1
a4,a5,a6,1
a7,a8,a9,1
b1,b2,b3,2
b4,b5,b6,2
b7,b8,b9,2
c1,c2,c3,3
c4,c5,c6,3
c7,c8,c9,3
...etc
this what I started with and captures the fields correctly:
(?<f1>.*?),(?<f2>.*?),(?<f3>.*?)\s+
I understand I need to use lookarounds to capture and include the “test” field on each line.
So something like this added (using a positive lookahead)…
(?<f1>.*?),(?<f2>.*?),(?<f3>.*?)\s+(?=test="(?<test>.*?)")
This seems close but is not yielding all rows of data, but instead only the last row of data with the included test value as if it is consuming the look ahead row.
This expression with its captured groups are input into a .NET application that inserts these captured groups as fields within a database table. Number of fields is always static (4 in the example above; field1=f1, field2=f2, field3=f3, field4=test), but the number of records will be variable.
Any guidance would be appreciated.
Parsing your data to extract the relevant values
You are almost there, but need to allow the look ahead to skip the rows between the current one and the test line:
(?ms)(?<f1>\w+),(?<f2>\w+),(?<f3>\w+)\R(?=.*?^test="(?<test>\d+)")
\R matches all sort of newlines, (?ms) is the inline way of turning on the multiline and dot match all modifiers, so that .*?^test matches every line up to the test one, see demo here.
Again, your issue was that \s+ forced the lookahead to be on the line right after the one your were matching.