Regular expression to get HTML table contents - c++

I've stumbled on a bit of challenge here: how to get the contents of a table in HTML with the help of a regular expression. Let's say this is our table:
<table someprop=2 id="the_table" otherprop="val">
<tr>
<td>First row, first cell</td>
<td>Second cell</td>
</tr>
<tr>
<td>Second row</td>
<td>...</td>
</tr>
<tr>
<td>Another row, first cell</td>
<td>Last cell</td>
</tr>
</table>
I already found a method that works, but involves multiple regular expression to be executed in steps:
Get the right table and put it's rows in back-reference 1 (there may be more than one in the document):
<table[^>]*?id="the_table"[^>]*?>(.*?)</table>
Get the rows of the table and put the cells in back-reference 1:
<tr.*?>(.*?)</tr>
And lastly fetch the cell contents in back-reference 1:
<td.*?>(.*?)</td>
Now this is all good, but it would be infinitely more awesome to do this all using one fancy regular expression... Does someone know if this is possible?

There really isn’t a possible regex solution that works for an arbitrary number of table data and puts each cell into a separate back reference. That’s because with backreferences, you need to have a distinct open paren for each backref you want to create, and you don’t know how many cells you have.
There’s nothing wrong with using looping of one or another sort to pull out the data. For example, on the last one, in Perl it would be this, given that $tr already contains the row you need:
#td = ( $tr =~ m{<td.*?>(.*?)</td>}sg );
Now $td[0] will contain the first <td>, $td[1] will contain the second one, etc. If you wanted a two-dimensional array, you might wrap that in a loop like this to populate a new #cells variable:
our $table; # assume has full table in it
my #cells;
while(my($tr) =~ $table = m{<tr.*?>(.*?)</tr>}sg) {
push #cells, [ $tr =~ m{<td.*?>(.*?)</td>}sg ];
}
Now you can do two-dimensional addressing, allowing for $cells[0][0], etc. The outer explicit loop processes the table a row at a time, and the inner implicit loop pulls out all the cells.
That will work on the canned sample data you showed. If that’s good enough for you, then great. Use it and move on.
What Could Ever Be Wrong With That?
However, there are actually quite a few assumptions in your patterns about the contents of your data, ones I don’t know that you’re aware of. For one thing, notice how I’ve used /s so that it doesn’t get stuck on newlines.
But the main problem is that minimal matches aren’t always quite what you want here. At least, not in the general case. Sometimes they aren’t as minimal as you think, matching more than you want, and sometimes they just don’t match enough.
For example, a pattern like <i>(.*?)</i> will get more than you want if the string is:
<i>foo<i>bar</i>ness</i>
Because you will end up matching the string <i>foo<i>bar</i>.
The other common problem (and not counting the uncommon ones) is that a pattern like <tag.*?> may match too little, such as with
<img alt=">more" src="somewhere">
Now if you use a simplistic <img.*?> on that, you would only capture <img alt=">, which is of course wrong.
I think the last major remaining problem is that you have to altogether ignore certain things in parsing. The simplest demo of this embedded comments (also <script>, <style>, andCDATA`), since you could have something like
<i> some <!-- secret</i> --> stuff </i>
which will throw off something like <i>(.*?)</i>.
There are ways around all these, of course. Once you’ve done so, and it is really quite a bit of effort, you’ll find that you have built yourself a real parser, completely with a lot of auxiliary logic, not just one pattern.
Even then you are only processing well-formed input strings. Error recovery and failing softly is an entirely different art.

This answer was added before it was known the the OP needed a solution for c++...
Since using regex to parse html is technically wrong, I'll offer a better solution. You could use js to get the data and put it into a two dimensional array. I use jQuery in the example.
var data = [];
$('table tr').each(function(i, n){
var $tr = $(n);
data[i] = [];
$tr.find('td').text(function(j, text){
data[i].push(text);
});
});
jsfiddle of the example: http://jsfiddle.net/gislikonrad/twzM7/
EDIT
If you want a plain javascript way of doing this (not using jQuery), then this might be more for you:
var data = [];
var rows = document.getElementById('the_table').getElementsByTagName('tr');
for(var i = 0; i < rows.length; i++){
var d = rows[i].getElementsByTagName('td');
data[i] = [];
for(var j = 0; j < d.length; j++){
data[i].push(d[j].innerText);
}
}
Both these functions return the data the same way.

Related

How can I identify tags that do not have CSS selectors with Html.Table?

I am writing a query for work for a report that has some poorly written HTML code where only some of the tags have a CSS selector I can use for creating my columns but others don't. (Guess who wrote it?) I wrote some workarounds using nth-child and describing each step to the desired tab but, as expected, when scaled up to all the other reports it causes issues because not every table is alike with what it contains.
My working code:
{"Rule ID", "td[class='rule-id']"}
However the next column I used this:
{"Time", "div[class='panel-body'] > TABLE > TBODY > TR:nth-child(6) > TD:nth-child(2)"}
Which in some of the tables does grab the time, but in others it grabs the wrong tag. I'm wondering if there's a way I can refer to the information in-between the tags. The time code looks like this:
<table>
...
<tr>
<td>Time</td>
<td>2022-04-28T10:01:15+00:00</td>
</tr>
...
</table>
I tried this but it results in a 100% empty column:
{"Time", "td[contains='Time']"}
This is my first project in Power BI so I'm learning as I go and have found answers to most of my questions through Google but I couldn't quite phrase this question properly to find a good result. All of the code is on my GFE so I can't get it over here to share, hopefully the bits I could share are enough.
Thank you in advance for your help!
If you link the site I can give a better answer.
There's an optional 3rd parameter, that gives you more access to attributes.
{"Name", "Selector"} would become {"Name", "Selector", each _ }, ex:
= Html.Table(Source,
{
{"Name", "Selector", each _ }
}
)
selector
td[contains='Time']
This isn't looking for what your html, it would match this. Here's a good reference/cheatsheet
https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors
https://developer.mozilla.org/en-US/docs/Web/CSS/Attribute_selectors
<td contains='Time'>
Testing Selectors in a browser
In the web console, you can use a CSS Selector query to preview what it will select.
Firefox has shorthand using the function $ and $$ . otherwise use document.querySelector() and document.querySelectorAll()
Your RowSelector
If you were to use $$("TABLE.table > * > TR") as your RowSelector
Your columnNameSelectorPairs
And then TABLE.table > * > TR > :nth-child(1)
From:
ninmonkeys.com/Select_Tables_using_your_own_CSS_Selectors
https://developer.mozilla.org/en-US/docs/Web/CSS/Attribute_selectors

Regex (re2 googlesheets) multiple values in multiline cell

Getting stuck on how to read and pretty up these values from a multiline cell via arrayformula.
Im using regex as preceding line can vary.
just formulas please, no custom code
The first column looks like a set of these:
```
[config]
name = the_name
texture = blah.dds
cost = 1000
[effect0]
value = 1000
type = ATTR_A
[effect1]
value = 8
type = ATTR_B
[feature0]
name = feature_blah
[components]
0 = comp_one,1
[resources]
res_one = 1
res_five = 1
res_four = 1
<br/>
Where to be useful elsewhere, at minimum it needs each [tag] set ([effect\d], [feature\d], ect) to be in one column each, for example the 'effects' column would look like:
ATTR_A:1000,ATTR_B:8
and so on.
Desired output can also be seen in the included spreadsheet
<br/>
<b>Here is the example spreadsheet:</b>
https://docs.google.com/spreadsheets/d/1arMaaT56S_STTvRr2OxCINTyF-VvZ95Pm3mljju8Cxw/edit?usp=sharing
**Current REGEXREPLACE**
Kinda works, finds each 'type' and 'value' great, just cant figure out how to extract just that from the rest, tried capture (and non-capturing) groups before and after but didnt work
=ARRAYFORMULA(REGEXREPLACE($A3:$A,"[\n.][effect\d][\n.](.)\n(.)","1:$1 2:$2"))
**Current SUBSTITUTE + REGEXEXTRACT + REGEXREPLACE**
A different approach entirely, also kinda works, longer form though and left with having to parse the values out of that string, where got stuck again. Idea was to use this to simplify, then regexreplace like above. Getting stuck removing content around the final matches though, and if can do that then above approach is fine too.
// First ran a substitute
=ARRAYFORMULA(SUBSTITUTE(SUBSTITUTE($A3:$A,char(10),";"),";;",char(10)))
// Then variation of this (gave up on single line 'effect/d' so broke it up to try and get it working)
=ARRAYFORMULA(IF(A3:A<>"",IFERROR(REGEXEXTRACT(A3:A,"(?m)^(?:[effect0]);(.)$")&";;")&""&IFERROR(REGEXEXTRACT(A3:A,"(?m)^(?:[effect1]);(.)$")&";;")&""&IFERROR(REGEXEXTRACT(A3:A,"(?m)^(?:[effect2]);(.)$")&";;"),""))
// Then use regexreplace like above
=ARRAYFORMULA(REGEXREPLACE($B3:$B,"value = (.);type = (.);;","1:$1 2:$2"))
**--EDIT--**
Also, as my updated 'Desired Output' sheet shows (see timestamped comment below), bonus kudos if you can also extract just the values of matching 'type's to those extra columns (see spreadsheet).
All good if you cant though, just realized would need that too for lookups.
**--END OF EDIT--**
<br/>
Ive tried dozens of things, discarding each in turn, had a quick look in version history to grab out two promising attempts and shared them in separate sheets.
One of these also used SUBSTITUTE to simplify input column, im happy for a solution using either RAW or the SUBSTITUTE results.
<br/>
**Potentially Useful links:**
https://github.com/google/re2/wiki/Syntax
<br/>
<b>Just some more words:</b>
I also have looked at dozens of stackoverflow and google support pages, so tried both REGEXEXTRACT and REGEXREPLACE, both promising but missing that final tweak. And i tried dozens of tweaks already on both.
Any help would be great, and hopefully help others in future since examples with spreadsheets are great since every new REGEX seems to be a new adventure ;)
<br/>
P.S. if we can think of better title for OP, please say in comment or your answer :)
paste in B3:
=ARRAYFORMULA(SUBSTITUTE(TRIM(TRANSPOSE(QUERY(TRANSPOSE(
IF(C3:E<>"", C2:E2&":"&C3:E, )),,999^99))), " ", ", "))
paste in C3:
=ARRAYFORMULA(IFNA(REGEXEXTRACT($A3:$A, "(\d+)\ntype = "&C2)))
paste in D3:
=ARRAYFORMULA(IFNA(REGEXEXTRACT($A3:$A, "(\d+)\ntype = "&D2)))
paste in E3:
=ARRAYFORMULA(IFNA(REGEXEXTRACT($A3:$A, "(\d+)\ntype = "&E2)))
paste in F3:
=ARRAYFORMULA(IFNA(REGEXEXTRACT(A3:A, "\[feature\d+\]\nname = (.*)")))
paste in G3:
=ARRAYFORMULA(IFNA(REGEXEXTRACT(A3:A, "\[components\]\n\d+ = (.*)")))
paste in H3:
=ARRAYFORMULA(IFNA(REGEXREPLACE(INDEX(SPLIT(REGEXEXTRACT(
REGEXREPLACE(A3:A, "\n", ", "), "\[resources\], (.*)"), "["),,1), ", , $", )))
spreadsheet demo
This was a fun exercise. :-)
Caveat first: I have added some "input data". Examples:
[feature1]
name = feature_active_spoiler2
[components]
0 = spoiler,1
1 = spoilerA, 2
So the output has "extra" output.
See the tab ADW's Solution.

How to remove tags using RegexTokenizer() in Spark/Scala ML?

I have a a feature column that has HTML tags in it. I would like to remove all tags.
An example of one row of data from column "body" is as follows:
"<p>Are questions related to and similar products on-topic?</p>"
I would like the output after using RegexTokenizer() to be as follows:
"are questions related to and similar products on-topic?"
Here is what I have started:
val regexTokenizer = new RegexTokenizer()
.setInputCol("body")
.setOutputCol("removedTags")
.setPattern("")
I think I need to fix the .setPattern() but unsure of how.
Assuming that you may not have any other < or > in your strings, maybe,
<[^>]+>
replaced with an empty string might be working OK to some extent, otherwise it'd fail.
If you wish to simplify/modify/explore the expression, it's been explained on the top right panel of regex101.com. If you'd like, you can also watch in this link, how it would match against some sample inputs.

web browser innertext data how received in textbox?

I have posted my HTML below. In which I want to get the name value from within my textbox area. I've tried several processes and I'm still not getting any valid solution. Please check my HTML and code snippet, and show me a possible solution.
The name prefix will always stay the same when I refresh the page. However, the last name within the "name" area will change, but will always contain the literal "mr." as the first 3 digits. regex as ([mM]r.\ ) - Four digits if you consider the literal space. Below is my table example.
<table>
<tr><td><b>Your Name is </b> mr. kamrul</td></tr>
<tr><td><b>your age </b> 12</td></tr>
<tr><td><b>Email:</b>kennethdasma30#gmail.com</td></tr>
<tr><td><b>job title</b> sales man</td></tr>
</table>
As shown below I am trying this process using listbox but I am not receiving anything.
HtmlElementCollection bColl =
webBrowser1.Document.GetElementsByTagName("table");
foreach (HtmlElement bEl in bColl)
{
if (bEl.GetAttribute("table") != null)
{
listBox1.Items.Add(bEl.GetAttribute("table"));
}
}
If anyone ca give me an idea of how I am able to receive all in the browser window as ("mr. " + text) within my list box I would appreciate it. Also, if you can explain the answer verbosely and with good comments I would appreciate it, as I'd like to understand the answer in greater detail as well.
Here is one simple way using Regex, assuming that the format of your html page doesn't change.
Regex re = new Regex(#"(?<=<tr><td><b>Your\sName\sis\s?</b>\s?)[mM]r\.\s.+?(?=</td></tr>)", RegexOptions.Singleline);
foreach (Match match in re.Matches(webBrowser1.DocumentText))
{
listBox1.Items.Add(match.Value);
}

Matching innermost braces with regex or strpos?

I have a sort of mini parsing syntax I made up to help me streamline my view code in cakephp. Basically I have created a table helper which, when given a dataset and (optionally) a set of options for how to format the data will render out a table, as opposed to me looping though the data and editing it manually.
It allows the user to be as complex or as simple as they like, it can get pretty powerful. However, In order to achieve this I had to make a simple parsing syntax. As a quick example the user would do something like so:
$this->Table->data = $userData;
$this->Table->elements['td']['data'] = array(
'{:User.username:}',
'{:User.created:}' => array('Time::nice')
);
echo $this->Table->render();
And when rendering the table would then generate:
<table>
<tbody>
<tr><td>rich97</td><td>Sun 21st 02:30pm</td></tr>
</tbody>
</table>
The problem occurs then I try to nest the braces like so:
{:User.levels.iconClasses.{:User.access:}:}
Is there anyway I can only get the inner most brackets on the first time round and loop though until there are no matches? Or even do it in one go? Or even better use strpos?
Here is my regex as it stands:
'/\{\:([^}]+)\:\}/'
Just add the opening brace to your negated character class:
'/\{:([^{}]+):\}/'
var $validate= array(
'name'=>array(
'notEmpty' =>array(
'rule'=>'notEmpty',
'message'=>'Please Enter The Name'
),
'isUnique' =>array(
'rule'=>'isUnique',
'message'=>'Name Already Exist'
)
),
'address'=>array(
'rule'=>'notEmpty',
'message'=>'Please Enter The Address')
);