So I have this text that I am trying to parse with Regex:
Name: Test Data 1
Description: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec feugiat nulla id nisi venenatis blandit.
Donec blandit egestas orci, at tristique dui vehicula in. Maecenas fringilla fringilla enim, in pulvinar ex gravida
in. Nam cursus facilisis ante, sed tristique nisl sagittis sed. In auctor felis id neque suscipit ullamcorper. Nunc
faucibus elit sed metus vestibulum, ullamcorper pulvinar nisi auctor. Praesent sodales orci mauris, eget dapibus
mauris sodales in. Ut iaculis, ante vitae ullamcorper semper, metus tortor auctor purus, eu convallis nulla lacus
in tellus. Phasellus feugiat tempus neque, in fringilla nisi scelerisque sed. Donec elementum diam nec mattis dignissim.
I am trying to parse it to load it into a database.
With this expression, I am trying to get a match on the "Name" and "Description" parameters but also trying to get a match on the parameter value as well (which can sometimes be multi-line).
(.*):\s(.*)
I have been searching for a while now and I cannot seem to be able to make it match the whole paragraph but stop when it hits a blank line.
I would like the result to be as follows:
1st Match
Group 1: Name
Group 2: Test Data 1
2nd Match
Group 1: Description
Group 2: Description value with multi-line
https://regex101.com/r/mG2ms9/3
Thanks
You can use the following:
(.*?):\s([\s\S]*?)(?=\n(?:\n|\w|$))
Here it is on regex101.
[\s\S] matches any character, even a new line (whereas '.' does not, by default).
Then we're matching as few characters as possible (*?) up until the point where the next line is either blank (\n), starts with a word character (\w), or is the end of the string ($).
We can get away with the \w option since all of the new lines in the description parameter are followed by a space. If this isn't always the case, you could replace \w with something like .*: to check instead if the next line contains ':' and stop if so.
Note that I disabled multi-line mode; it's not suitable here.
Related
I have an OCR text document where paragraphs have been broken into individual lines. I'd like to make them whole paragraphs on a single line again (as per the original PDF).
How can I use regex, or find and replace, to remove the line breaks between two lines of text and replace them with a space?
Eg:
Every line of text is on a newline. I'd like them to be whole paragraphs on a single line.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam vehicula tellus faucibus metus consequat
scelerisque. Maecenas sit amet urna quis ipsum interdum consequat. Praesent elementum libero nec
velit suscipit placerat accumsan vitae lacus. Aliquam erat volutpat. Etiam egestas lectus sed orci
venenatis, ullamcorper gravida elit pulvinar. Pellentesque imperdiet, augue pulvinar sodales dapibus,
tortor magna rutrum nulla, vel ullamcorper mi purus a diam. Ut id odio sed arcu aliquet lobortis.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Donec quam arcu, egestas feugiat eleifend blandit, vulputate non elit. Nulla a erat vel leo maximus
viverra at ac lorem. Nam non imperdiet lorem. Fusce tempor arcu massa, non commodo ligula lobortis
nec. Aliquam sit amet fringilla sapien, non euismod metus. Donec orci mi, sagittis vitae lobortis eu,
aliquet nec libero. Sed sodales magna lacus, pretium lobortis magna varius nec. Pellentesque quis
ipsum viverra orci lobortis egestas. Aliquam porttitor tincidunt ipsum, egestas placerat ante
consectetur in. Morbi porttitor lacus eu augue tincidunt, at aliquet lorem consectetur.
You might be looking for a programatic/dynamic approach for every new scan generated so I'm not sure if this answers your question, but since you have visual studio code in your tags I will answer how to do this in vscode.
Open keyboard shortcuts from File > Preferences > Keyboard shortcuts, and bind editor.action.joinLines to a shortcut of your choice like for example Ctrl + J.
Then go ahead and open the text you are looking to fix in vscode, select it and press that keybinding. You will notice everything will be in 1 line. I hope I helped!
I am using two regular expressions when removing linebreaks from OCR texts.
They can be used in the Find&Replace dialog from VS Code.
Remove linebreaks at lines ending with a hyphen: (?<=\w)- *\n *
Replace remaining linebreaks with whitespace, but keeping blank lines: (?<!\n) *\n *(?!\n).
Note that the * in the regular expression trims whitespace at the end and beginning of the lines.
There is also a Python tool based on Flair called dehyphen that does the job.
In my experience it produces useful results but may take quite long compared to replacing linebreaks with regular expressions.
I need to remove all anchors (anchor text remains) from the string except those anchors that have href="/"
This is example text:
Fusce imperdiet nulla ut sapien aliquet, congue varius dui consectetur. This link remains et blandit nisl. Curabitur euismod volutpat urna, eget dignissim libero cursus rhoncus. Nulla ac test sollicitudin link from this text should be removed. Maecenas sodales vel lorem eu placerat.
Here is regex that I think should work (using negative lookahead):
/<a.*?(?!href=["']\/["'])>(.*?)</a>/gi
Yet it selects both anchors.
try regex <a(?!.*href=["']\/["']).*?>(.*?)<\/a>
The negative lookahead (?!.*href=["']\/["']) won't capture the tag with href="/"
Regex
I’m having a hard time figuring out the regex code in Google Sheets to check a cell then return everything including new lines \n and returns \r before a certain pattern \*+.
A little more background: I'm using REGEXEXTRACT(A:A,"...") format inside a bigger ArrayFormula so that it automatically updates when a new row is added. This one’s working properly. It’s only the regex part I’m having trouble with.
So, for the purpose of this question, let's say I'm only worried about extracting the data from the A1 cell before a certain pattern and return that value in cell B1. Which brings us to this code in cell B1:
REGEXEXTRACT(A1,"...")
For example, this is how my A1 cell looks like:
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Phasellus accumsan risus id ex dapibus sodales.
Curabitur dui lacus, tincidunt vel ligula quis, volutpat mattis eros.
In quis metus at ex auctor lobortis. Aliquam sed nisi purus. Sed cursus odio erat, ut tristique sapien interdum interdum. Morbi vel sollicitudin ante, non pellentesque libero.
***********
Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Aenean egestas urna facilisis massa posuere, quis accumsan erat ornare.
Curabitur at dapibus nibh. Nam nec vestibulum ligula. Phasellus bibendum mi urna, ac hendrerit libero interdum non. Suspendisse semper non elit aliquam auctor.
Morbi vel sem tortor. Donec a sapien quis erat condimentum consequat in ut sem. Quisque in tellus sed est lobortis ultricies sed vitae enim.
I want to return this value in B1:
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Phasellus accumsan risus id ex dapibus sodales.
Curabitur dui lacus, tincidunt vel ligula quis, volutpat mattis eros.
In quis metus at ex auctor lobortis. Aliquam sed nisi purus. Sed cursus odio erat, ut tristique sapien interdum interdum. Morbi vel sollicitudin ante, non pellentesque libero.
Which is basically anything before the pattern *******. In Python, I can add the re.DOTALL to the .* but I can't get this to work in Google Sheets.
To make a dot match line breaks, you need to add (?s) to the pattern. To match any char, you may use a .. To match up to the leftmost occurrence, use lazy quantifier, *?. To actually extract a substring you need, wrap the part of the pattern you are interested in getting with capturing parentheses.
So, to match up to the first ******* substring, you may use
(?s)^(.*?)\*\*\*\*\*\*\*
or (?s)^(.*?)\*{7}. See the regex demo (note that Go regex engine is also RE2, so you may test your patterns there, at regex101.com).
(?s) - a DOTALL modifier
^ - start of string
(.*?) - Group 1: any 0+ chars as few as possible
\*\*\*\*\*\*\* - 7 literal asterisk symbols.
Note you cannot rely on a negated character class (that matches line breaks) if your substring may contain * chars, that is, ^([^*]*)\*\*\*\*\*\*\* won't work in those cases.
If you just want to match any chars up to the first * in the string, your regex will simplify greatly to
^([^*]+)
It matches
^ - start of string
([^*]+) - Capturing group 1: one or more chars other than *.
re.DOTALL flag in python corresponds to (?s) single line mode flag in re2.
Python:
(Dot.) In the default mode, this matches any character except a newline. If the DOTALL flag has been specified, this matches any character including a newline.
re2:
Flags: s let . match \n (default false)
So,
=REGEXEXTRACT(A1,"(?s)(.*?)\*")
This corresponds to re.findall()
Not regex though might suit someone wanting the same result but less particular about the method:
=ArrayFormula(LEFT(A1:A,Find("***********",A1:A)-3))
If you really only want to match everything before the first *:
=REGEXEXTRACT(A1;"[^*]*")
If you want to allow a single star in the text and only stop at multiple (2 or more) stars (possibly divided by newlines) at the beginning of a line, you could try:
=REGEXEXTRACT(A1;"(?s)^(.*)\n(\*\n?){2,}")
But you would have to strip the stars. E.g.
=REGEXREPLACE(REGEXEXTRACT(A1;"(?s)^(.*)\n(\*\n?){2,}"); "\n(\*\n?){2,}"; "")
A lookahead does not seem to work in Google Sheets.
I am trying to write an expression to take a block of text an return up until a full-stop before an ellipsis or three full-stops (... or …). So the idea is that the example text test string:
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam felis nisi, eleifend ut quam eget, venenatis vestibulum turpis. Nam dignissim laoreet iaculis. Etiam sit amet rhoncus sem. Duis laoreet justo tellus, at volutpat risus molestie sed. Etiam posuere, arcu vitae faucibus hendrerit, lorem elit consequat urna, id congue eros felis in mauris. Donec non fermentum ipsum. Curabitur nec...
Would become:
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam felis nisi, eleifend ut quam eget, venenatis vestibulum turpis. Nam dignissim laoreet iaculis. Etiam sit amet rhoncus sem. Duis laoreet justo tellus, at volutpat risus molestie sed. Etiam posuere, arcu vitae faucibus hendrerit, lorem elit consequat urna, id congue eros felis in mauris. Donec non fermentum ipsum.
So far I have come up with this pathetic attempt. I keep getting right up until the last full-stop (because the quantifier consumes the previous two full-stops so there is nothing for the look ahead to fail on). I just can't seem to wrap my head around it:
Dim testText As String = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam felis nisi, eleifend ut quam eget, venenatis vestibulum turpis. Nam dignissim laoreet iaculis. Etiam sit amet rhoncus sem. Duis laoreet justo tellus, at volutpat risus molestie sed. Etiam posuere, arcu vitae faucibus hendrerit, lorem elit consequat urna, id congue eros felis in mauris. Donec non fermentum ipsum. Curabitur nec..."
Dim ellipsisExpression As String = "(.*\.(?!\.\.))"
Dim ellipsisMatch As Match
ellipsisMatch = Regex.Match(testText, ellipsisExpression)
If ellipsisMatch.Success Then
testText = ellipsisMatch.Groups(1).Value
End If
edit: I also need this expression to take any ... character in the text into account. for example the string:
`begin. this is a test... test complete. beginning shutdown... shutting down... `
should return
`begin. this is a test... test complete.`
The aim of this expression is to find the most flowing text before any truncation has occurred. A block of text with closure so it doesn't confuse readers expecting to be able to 'get more'.
You could replace [^.]*(?:\.{3}|…).* with an empty string to get the desired result.
For example:
result = Regex.Replace(input, "[^.]*(?:\\.{3}|…).*", "")
Use this:
result = Regex.Replace(input, "(.+\.).+(?:\.{3}|…)\s*", "$1")
Edit:
Use this regex instead:
(.+[^.]\.)(?:(?:[^.]{2})|$)
You could match that with:
.*(?<!\.)\.(?!\.)(?=(?:[^.]+|\.{3})*(?:\.{3}|…)$)
Or replace
(?<!\.)\.(?!\.)(?:[^.]+|\.{3})*(?:\.{3}|…)$
with a ..
I think I have come up with a solution that works for me. Thank you to everyone who answered previously but this expression seems to do what I need and doesn't execute as slowly as some of the other answers. It also takes other sentence terminating punctuation into account such as ! or ? and not just ..
(.*([^\.](?=\.|\?|!)(?!\.\.\.)).)
This get's the last sentence terminating character (defined with the lookahead). In this case they are ?, ! and . that isn't followed by .... This also solves the ellipsis character issue since it is effectively a sentence terminating white list. This expression succeeds in finding the largest block of text with closure.
Is there a regex to match "all characters including newlines"?
For example, in the regex below, there is no output from $2 because (.+?) doesn't include new lines when matching.
$string = "START Curabitur mollis, dolor ut rutrum consequat, arcu nisl ultrices diam, adipiscing aliquam ipsum metus id velit. Aenean vestibulum gravida felis, quis bibendum nisl euismod ut.
Nunc at orci sed quam pharetra congue. Nulla a justo vitae diam eleifend dictum. Maecenas egestas ipsum elementum dui sollicitudin tempus. Donec bibendum cursus nisi, vitae convallis ante ornare a. Curabitur libero lorem, semper sit amet cursus at, cursus id purus. Cras varius metus eu diam vulputate vel elementum mauris tempor.
Morbi tristique interdum libero, eu pulvinar elit fringilla vel. Curabitur fringilla bibendum urna, ullamcorper placerat quam fermentum id. Nunc aliquam, nunc sit amet bibendum lacinia, magna massa auctor enim, nec dictum sapien eros in arcu.
Pellentesque viverra ullamcorper lectus, a facilisis ipsum tempus et. Nulla mi enim, interdum at imperdiet eget, bibendum nec END";
$string =~ /(START)(.+?)(END)/;
print $2;
If you don't want add the /s regex modifier (perhaps you still want . to retain its original meaning elsewhere in the regex), you may also use a character class. One possibility:
[\S\s]
a character which is not a space or is a space. In other words, any character.
You can also change modifiers locally in a small part of the regex, like so:
(?s:.)
Add the s modifier to your regex to cause . to match newlines:
$string =~ /(START)(.+?)(END)/s;
Yeap, you just need to make . match newline :
$string =~ /(START)(.+?)(END)/s;
You want to use "multiline".
$string =~ /(START)(.+?)(END)/m;