I'm working on an Excel 2007 VBA project that my client wants done yesterday and I need to use RegEx to locate strings within some pretty challenging data. This is my first exposure to RegEx so I'm stuck doing something I think is simple (maybe not) and I'm clueless.
I've added the reference to the VBScript RegEx engine (5.5) and RegEx is working O.K. in Excel - I just don't know how to construct the pattern statement. I need to locate occurrences of the word "trust" in a range of cells on a worksheet. In some of my data this word has been abbreviated "Tr". I have constructed the following RegEx statement to locate the word "trust" and all words that start with a space and contain "tr".
"trust| tr"
Unfortuantely, this matches any word that contains "tr", like "trail", "tree", and so on. What I want to match is " tr" - meaning it has a leading space, the "tr", and nothing else in the word. Can somebody tell me what I need to do to make this happen?
I'm also going to need RegEx patterns for street addresses, city, state, and zip plus last name and first name. If there's a resource someone can point me to for these expressions I'd appreciate the help. I'm sorry to ask the group this question without spending the proper amount of time educating myself, by this is a time-sensitive project for which I need your expertise.
Thanks In Advance -
PS - Here a sample of data that I'm working with. I have this type of data present in 5 columns over 4,000 rows.
Jones Family **Trust**
3420 E Ave of the Ftns
3420 E Avenue of the Fountain
320 E ARROWHEAD **TRAILHEAD**
501 S 29TH ST
PO BOX 13422
71343 W Paradise Dr
152035 S 29TH ST
124 Owl Grove Pl
Johnson **Tr**
1900 E Arrowhead **Trl**
1900 E ARROWHEAD **TRL**
This is a sample from a column that predominantly contains street addresses. Other columns contain client names without addresses. So not every cell contains data that starts with a number.
I would rewrite your expression that finds trust and tr where they not preceded or followed by a other letters by using the \b is a word boundary assertion. \b matches at a position that is aptly called a "word boundary".
There are three different positions that qualify as word boundaries:
Before the first character in the string, if the first character is a
word character.
After the last character in the string, if the last
character is a word character.
Between two characters in the string,
where one is a word character and the other is not a word character.
For more information on word boundaries then see also regular-expressions.info. I'm not affiliated with that site.
\b(?:trust|tr)\b
After viewing the above, if you're still set on requiring the tr preceded by a space, then use this \b(?:trust|\str)\b
Examples
Live Demo
https://regex101.com/r/xM4fR9/1
Note: I am assuming you're using the case insensitive flag for this
Explanation
NODE EXPLANATION
----------------------------------------------------------------------
\b the boundary between a word char (\w) and
something that is not a word char
----------------------------------------------------------------------
(?: group, but do not capture:
----------------------------------------------------------------------
trust 'trust'
----------------------------------------------------------------------
| OR
----------------------------------------------------------------------
tr 'tr'
----------------------------------------------------------------------
) end of grouping
----------------------------------------------------------------------
\b the boundary between a word char (\w) and
something that is not a word char
----------------------------------------------------------------------
Or
The \b(?:trust|tr)\b expression isn't the most efficient, but it is readable.
A functionally identical, but more efficient regular expression would be:
\btr(?:ust)?\b
Here we're still using the \b word boundary, but we've just made the ust part of the word trust optional with the (?: ... )? construct.
Related
I'm trying to extract all words with Uppercase initial letter from a text, with the REGEXEXTRACT formula in google sheets.
Ideally the first word of sentences should be ignored and only all subsequent words with first Uppercase letter should be extracted.
Other Close Questions and Formulas:
I've found those other two questions and answers:
How to extract multiple names with capital letters in Google Sheets?
=ARRAYFORMULA(TRIM(IFERROR(REGEXREPLACE(IFERROR(REGEXEXTRACT(IFERROR(SPLIT(A2:A, CHAR(10))), "(.*) .*#")), "Mr. |Mrs. ", ""))))
Extract only ALLCAPS words with regex
=REGEXEXTRACT(A2, REPT(".* ([A-Z]{2,})", COUNTA(SPLIT(REGEXREPLACE(A2,"([A-Z]{2,})","$"),"$"))-1))
They are close but I can't apply them successfully to my project.
The Regex Pattern I Use:
I also found this regex [A-ZÖ][a-zö]+ pattern that works well to get all the Uppercase first letter words.
The problem is that it's not ignoring the first words of sentences.
Other Python Solution Vs Google Sheets Formula:
I've also found this python tutorial and script to do it:
Proper Noun Extraction in Python using NLP in Python
# Importing the required libraries
import nltk
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize, sent_tokenize
# Function to extract the proper nouns
def ProperNounExtractor(text):
print('PROPER NOUNS EXTRACTED :')
sentences = nltk.sent_tokenize(text)
for sentence in sentences:
words = nltk.word_tokenize(sentence)
words = [word for word in words if word not in set(stopwords.words('english'))]
tagged = nltk.pos_tag(words)
for (word, tag) in tagged:
if tag == 'NNP': # If the word is a proper noun
print(word)
text = """Down the Rabbit-Hole
Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do: once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, `and what is the use of a book,' thought Alice `without pictures or conversation?'
So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her.
There was nothing so very remarkable in that; nor did Alice think it so very much out of the way to hear the Rabbit say to itself, `Oh dear! Oh dear! I shall be late!' (when she thought it over afterwards, it occurred to her that she ought to have wondered at this, but at the time it all seemed quite natural); but when the Rabbit actually took a watch out of its waistcoat-pocket, and looked at it, and then hurried on, Alice started to her feet, for it flashed across her mind that she had never before seen a rabbit with either a waistcoat-pocket, or a watch to take out of it, and burning with curiosity, she ran across the field after it, and fortunately was just in time to see it pop down a large rabbit-hole under the hedge."""
# Calling the ProperNounExtractor function to extract all the proper nouns from the given text.
ProperNounExtractor(text)
It works well, but I the idea in doing it in Google Sheets is to have the Uppercase first letter Words adjacent to the text in a table format for more convenient reference.
Question Summary:
How would you adjust my formula in the sample sheet below
=ARRAYFORMULA(IF(A1:A="","",REGEXEXTRACT(A1:A,"[A-ZÖ][a-zö]+")))
to add those functions:
Extract all the first Uppercase letter Words from each cell with text
Ignore the first words of sentences
return all the first Uppercase letter Words save the first words from sentences into the adjacent cells, one word per cell (similar to this example (from the 2nd Close question above): )
Sample Sheet:
Here's my testing Sample Sheet
Many thanks for your help!
You can use
=ARRAYFORMULA(SPLIT(REGEXREPLACE(REGEXREPLACE(A111:A, "(?:[?!]|\.(?:\.\.+)?)\s+", CHAR(10)), "(?m)^\s*[[:upper:]][[:alpha:]]*|.*?([[:upper:]][[:alpha:]]*|$)", "$1" & char(10)), CHAR(10)))
Or, to make sure the ?, ! or . / ... that are matched as sentence boundaries are followed with an uppercase letter:
=ARRAYFORMULA(SPLIT(REGEXREPLACE(REGEXREPLACE(A111:A, "(?:[?!]|\.(?:\.\.+)?)\s+([[:upper:]])", CHAR(10) & "$1"), "(?m)^\s*[[:upper:]][[:alpha:]]*|.*?([[:upper:]][[:alpha:]]*|$)", "$1" & char(10)), CHAR(10)))
See the demo screenshot:
See the regex demo.
First, we split the text into sentences in a cell with REGEXREPLACE(A111:A, "(?:[?!]|\.(?:\.\.+)?)\s+", CHAR(10)). Actually, this just replaces final sentence punctuation with a newline.
The second REGEXREPLACE is used with another regex that matches
(?m)^\s*[[:upper:]][[:alpha:]]* - a capitalized word ([[:upper:]][[:alpha:]]*) at the start of string/line (^) together with optional whitespace (\s*)
| - or
.*? - any zero or more chars other than line break chars, as few as possible
([[:upper:]][[:alpha:]]*|$) - Group 1 ($1): an uppercase letter ([[:upper:]]) and then any zero or more letters ([[:alpha:]]*), or end of string ($)
and replaces the match with Group 1 value and a newline, LF char. Then, the result is SPLIT with a newline char.
My two cents:
Formula in B1:
=INDEX(IF(A1:A<>"",SPLIT(REGEXREPLACE(A1:A,"(?:(?:^|[.?!]+)\s*\S+|\b([A-ZÖ][a-zö]+(?:-[A-ZÖ][a-zö]+)*)\b|.+?)","$1|"),"|",1),""))
The pattern: (?:(?:^|[.?!]+)\s*\S+|\b([A-ZÖ][a-zö]+(?:-[A-ZÖ][a-zö]+)*)\b|.+?) means:
(?: - Open non-capture group to allow for alternations:
(?:^|[.?!]+)\s*\S+ - A nested non-capture group to allow for the start-line anchor or 1+ literal dots or question/exclamation marks, followed by 0+ whitespace chars and 1+ non-whitespace chars;
| - Or;
\b([A-ZÖ][a-zö]+(?:-[A-ZÖ][a-zö]+)*)\b - A 1st capture-group to catch camel-case strings (with optional hyphen) between word-boundaries;
| - Or;
.+? - Any 1+ characters (Lazy);
) - Close non-capture group.
The idea is here to use REGEXREPLACE() to substitute any match with the backreference to the 1st capture group and a pipe-symbol (or any symbol for that matter that won't be in your input) and use SPLIT() to get all words seperated. Note that it is important to use the 3rd parameter of the function to ignore empty strings.
INDEX() will trigger the array-functionality and spill the results. I used an nested IF() statement to check for empty cells to skip.
Since you have already found a Python solution for your use case, you may try utilizing that directly as a custom function in your Google Sheet by hosting the Python code as an API and using Google App script to call it and extract natively in the Google Sheet.
For reference, you can check this repository and this YouTube video.
I am creating regexes that get the whole sentence if a piece of specific information exists. Right now I am working on my name regex, so if there is any composed name (example: "Jorge Martel", "Jorge Martel del Arnold Albuquerque") the regex should get the whole sentence that has the name.
If I have these two sentences:
(1) - "A hardworking guy is working at the supermarket. They call him Jorge Horizon, but that's not his real name."
(2) - "He has an identity document that contains the name, Jorge Martel Arnold."
The regex should return these two results from the sentences above:
(1) - "They call him Jorge Horizon, but that's not his real name."
(2) - "He has an identity document that contains the name, Jorge Martel Arnold."
This is my regex:
(?:(?(?<=[\.!?]\s([A-Z]))(.+?[^.])|))?((?:(?:[A-Z][A-zÀ-ÿ']+\s(?:(?:(?:[A-zÀ-ÿ']{1,3}\s)?(?:[A-ZÀ-Ÿ][A-zÀ-ÿ']*\s?))+))\b)(.+?[\.!?](?:\s|\n|\Z)))
Basically, it verifies if there is a dot, exclamation, or interrogation symbol with a blank space and an upper case character and tells the regex that everything must be select, else it should get all the sentence.
My else case (|) right now is empty, because using (.+?) avoids my first condition...
Regex without the else case:
Validates until the dot, but doesn't get the second sentence.
Regex with the else case:
Validates the second sentence, but overrides the first condition that appears in the first sentence.
I expect my regex to return correctly the sentences:
"They call him Jorge Horizon, but that's not his real name."
"He has an identity document that contains the name, Jorge Martel Arnold."
I have also created a text to validate the regex operations as I will be using it a lot in texts. I added a lot of conditions in this text, which will probably appear in my daily work.
Check my regex, sentence, and text here:
Does anyone know what should I change in my regex? I have tried many variations and still cannot find the solution.
P.S.: I intend to use it in my python code, but I need to fix it with the regex and not with the python code.
you can try this.
[\w\ \,\']+\.\ ?([\w\ \,\']+\.)|^([\w\ \,\']+\.)$
prints $1$2. I.e if group one is empty it prints blank since there is no match, then will print group 2. Visa versa, it prints group 1 when group 2 is not there.
[\w\ ,']+.\ ?([\w\ ,']+.) - as matching anything with XXX. XXX.
then
^([\w\ ,']+.)$ - must start end with only 1 sentence.
Though honestly this can easily be done with a Tokenizer of (.) that check length of 1 or 2. It' really like using a sledgehammer to hammer a nail.
Matching names can be a very hard job using a regex, but if you want to match at least 2 consecutive uppercase words using the specified ranges.
Assuming the names start with an uppercase char A-Z (else you can extend that character class as well with the allowed chars or if supported use \p{Lu} to match an uppercase char that has a lowercase variant):
(?<!\S)[A-Z][A-Za-zÀ-ÿ]*(?:\s+[a-zÀ-ÿ,]+)*\s+[A-Z][a-zÀ-ÿ]*\s+[A-Z][a-zÀ-ÿ,]*.*?[.!?](?!\S)
(?<!\S) Assert a whitespace boundary to the left
[A-Z][A-Za-zÀ-ÿ]* Match an uppercase char A-Z optionally followed by matching the defined ranges
(?:\s+[a-zÀ-ÿ,]*)* Optionally repeat matching 1+ whitespace chars and 1 or more of the ranges
\s+[A-Z][a-zÀ-ÿ]*\s+[A-Z][a-zÀ-ÿ,]* Match 2 times whitespace chars followed by an uppercase A-Z and optional chars defined in the character class
.*?[.!?] Match as least as possible chars followed by one of . ! or ?
(?!\S) Assert a whitspace boundary to the right
Regex demo
Try this:
((?:^|(?:[^\.!?]*))[^\.!?\n]*(?:(?:[A-ZÀ-Ÿ][A-zÀ-ÿ']+\s?){2,}[^\.!?]*[\.!?]))
It will capture sentences where name has at least two words, e.g. His name is John Smith.
It won't capture sentences like: John went to a concert.
I'd like to keep together initials (max two letters) when there are punctuation or spaces in between.
I have the following snippet to tackle almost everything, but I am having issues in keeping together initials that are separated by punctuation and space. For instance, this is working on regular regex, but not in postgresql:
SELECT regexp_replace('R Z ELEMENTARY SCHOOL',
'(\b[A-Za-z]{1,2}\b)\s+\W*(?=[a-zA-Z]{1,2}\b)',
'\1')
The outcome should be "RZ ELEMENTARY SCHOOL". Other examples will include:
A & D ALTERNATIVE EDUCATION
J. & H. KNOWLEDGE DEVELOPMENT
A. - Z. EVOLUTION IN EDUCATION
The transformation should be as follows:
AD ALTERNATIVE EDUCATION
JH KNOWLEDGE DEVELOPMENT
AZ EVOLUTION IN EDUCATION
How to achieve this in Postgresql?
Thanks
Building on your current regex, I can recommend
SELECT REGEXP_REPLACE(
REGEXP_REPLACE('J. & H. KNOWLEDGE DEVELOPMENT', '\m([[:alpha:]]{1,2})\M\s*\W*(?=[[:alpha:]]{1,2}\M)', '\1'),
'^([[:alpha:]]+)\W+',
'\1 '
)
See the online demo, yielding
regexp_replace
1 JH KNOWLEDGE DEVELOPMENT
It is a two step solution. The first regex matches
\m([[:alpha:]]{1,2})\M - a whole one or two letter words captured into Group 1 (\m is a leading word boundar, and \M is a trailing word boundary)
\s* - zero or more whitespaces
\W* - zero or more non-word chars
(?=[[:alpha:]]{1,2}\M) - a positive lookahead that requires a whole one or two letter word immediately to the right of the current position.
The match is replaced with the contents of Group 1 (\1).
The second regex matches a letter word at the start of the string and replaces all non-word chars after it with a space.
I wonder if anyone can help.
I need to write a regular expression that throws away everything apart from the last word if that last word is an alphanumeric (numbers and letters) or a single number or a single letter.
For example
Ground floor Apartment 2
Garden Apartment 1A
Block 2D
Suite 12
Unit C
Basement Flat
General Office
I would like to remove all words and characters that are not part of the actual number i.e.
Ground Floor Apartment 2 should become 2
Garden Apartment 1A should become 1A
Block 2D should become 2D
Suite 12 should become 12
Unit C should become C
Basement Flat should become Blank as there is no numbers involved
General Office should become blank
Many Thanks in advance
You could try using a positive lookahead which asserts your requirements at the end of the string.
(?:\b[A-Za-z]{1}|\d+|(?=.*\d)[a-zA-Z0-9]+)$
Explanation
A non capturing group (?:
A word boundary \b
Match a single letter [A-Za-z]{1}
Or |
One or more digits \d+
Or |
A positive lookahead which asserts that the last word contains a digit (?=.*\d)
Match one or more lower/upper case characters or digits [a-zA-Z0-9]+
Close non capturing group )
The end of the string $
What language are you using? You should be able to get the last word by splitting/exploding the string using spaces, then apply the regex to the last word.
You may want to just handle if the length of the word is 1 to make your regex simpler to understand and troubleshoot. This regex works for any word that is 2 letters or longer.
Here's a regex that should work for that last word. It uses a positive lookahead to ensure one letter and one number are present. https://regex101.com/r/i5R9bq/1/
(?=.*[0-9])(?=.*[A-z])[0-9A-z]+
I am trying to match only the street name from a series of addresses. The addresses might look like:
23 Barrel Rd.
14 Old Mill Dr.
65-345 Howard's Bluff
I want to use a regex to match "Barrel", "Old Mill", and "Howard's". I need to figure out how to exclude the last word. So far I have a lookbehind to exclude the digits, and I can include the words and spaces and "'" by using this:
(?<=\d\s)(\w|\s|\')+
How can I exclude the final word (which may or may not end in a period)? I figure I should be using a lookahead, but I can't figure out how to formulate it.
You don't need a look-behind for this:
/^[-\d]+ ([\w ']+) \w+\.?$/
Match one or more digits and hyphens
space
match letters, digits, spaces, apostrophes into capture group 1
space
match a final word and an optional period
An example Ruby implementation:
regex = /^[-\d]+ ([\w ']+) \w+\.?$/
tests = [ "23 Barrel Rd.", "14 Old Mill Dr.", "65-345 Howard's Bluff" ]
tests.each do |test|
p test.match(regex)[1]
end
Output:
"Barrel"
"Old Mill"
"Howard's"
I believe the lookahead you want is (?=\s\w+\.?$).
\s: you don't want to include the last space
\w: at least one word-character (A-Z, a-z, 0-9, or '_')
\.?: optional period (for abbreviations such as "St.")
$: make sure this is the last word
If there's a possibility that there might be additional whitespace before the newline, just change this to (?=\s\w+\.?\s*$).
Why not just match what you want? If I have understood well you need to get all the words after the numbers excluding the last word. Words are separated by space so just get everything between numbers and the last space.
Example
\d+(?:-\d+)? ((?:.)+) Note: there's a space at the end.
Tha will end up with what you want in \1 N times.
If you just want to match the exact text you may use \K (not supported by every regex engine) but: Example
With the regex \d+(?:-\d+)? \K.+(?= )
Another option is to use the split() function provided in most scripting languages. Here's the Python version of what you want:
stname = address.split()[1:-1]
(Here address is the original address line, and stname is the name of the street, i.e., what you're trying to extract.)