BeautifulSoup extract data from certain columns from a table I am getting too much data out - python-2.7

Question

I am trying to extract some data out of my Selenium Test Report html file.
I am getting too much data out of the table of rows and columns.
The data I would like to extract is all columns which have the class value "testcase" and there is a column below with a class value "popup_link" and the text value will say Pass or Fail.
E.g.
<td class='none'><div class='testcase'>test_000001_login_valid_user</div></td>
<a class="popup_link" onfocus='this.blur();' href="javascript:showTestDetail('div_pt1.1')" >
pass</a>
I would like the text "test_000001_login_valid_user" and the text "pass"
There are lots of test cases in my report so I would like to iterate over the rows and get the test case name out and the pass or fail text.
My HTML snippet is:
<table id='result_table'>
<colgroup>
<col align='left' />
<col align='right' />
<col align='right' />
<col align='right' />
<col align='right' />
<col align='right' />
</colgroup>
<tr id='header_row'>
<td>Test Group/Test case</td>
<td>Count</td>
<td>Pass</td>
<td>Fail</td>
<td>Error</td>
<td>View</td>
</tr>
<tr class='passClass'>
<td>Regression_TestCase.RegressionProjectEdit_TestCase.RegressionProject_TestCase_Project_Edit</td>
<td>75</td>
<td>75</td>
<td>0</td>
<td>0</td>
<td>Detail</td>
</tr>
<tr id='pt1.1' class='hiddenRow'>
<td class='none'><div class='testcase'>test_000001_login_valid_user</div></td>
<td colspan='5' align='center'>
<!--css div popup start-->
<a class="popup_link" onfocus='this.blur();' href="javascript:showTestDetail('div_pt1.1')" >
pass</a>
<div id='div_pt1.1' class="popup_window">
<div style='text-align: right; color:red;cursor:pointer'>
<a onfocus='this.blur();' onclick="document.getElementById('div_pt1.1').style.display = 'none' " >
[x]</a>
</div>
<pre>
pt1.1: *** test_login_valid_user ***
test login with a valid user - Passed
</pre>
</div>
<!--css div popup end-->
</td>
</tr>
<tr id='pt1.2' class='hiddenRow'>
<td class='none'><div class='testcase'>test_000002_select_a_project</div></td>
<td colspan='5' align='center'>
<!--css div popup start-->
<a class="popup_link" onfocus='this.blur();' href="javascript:showTestDetail('div_pt1.2')" >
pass</a>
<div id='div_pt1.2' class="popup_window">
<div style='text-align: right; color:red;cursor:pointer'>
<a onfocus='this.blur();' onclick="document.getElementById('div_pt1.2').style.display = 'none' " >
[x]</a>
</div>
<pre>
pt1.2: *** test_login_valid_user ***
test login with a valid user - Passed
*** test_select_a_project ***
08_12_1612_08_03
Selenium_Regression_Edit_Project_Test
</pre>
</div>
<!--css div popup end-->
</td>
</tr>
<tr id='pt1.3' class='hiddenRow'>
<td class='none'><div class='testcase'>test_000003_verify_Lademo_CRM_DataPreview_is_present</div></td>
<td colspan='5' align='center'>
<!--css div popup start-->
<a class="popup_link" onfocus='this.blur();' href="javascript:showTestDetail('div_pt1.3')" >
pass</a>
<div id='div_pt1.3' class="popup_window">
<div style='text-align: right; color:red;cursor:pointer'>
<a onfocus='this.blur();' onclick="document.getElementById('div_pt1.3').style.display = 'none' " >
[x]</a>
</div>
<pre>
pt1.3: *** test_login_valid_user ***
test login with a valid user - Passed
*** test_select_a_project ***
08_12_1612_08_03
Selenium_Regression_Edit_Project_Test
*** Test verify_Lademo_CRM_DataPreview_is_present ***
aSelenium_LADEMO_CRM_DONOTCHANGE
File
498
</pre>
</div>
<!--css div popup end-->
</td>
</tr>
My code is:
from bs4 import BeautifulSoup
table = soup.select_one("#result_table")
for row in table.select("tr.hiddenRow"):
print(" ".join([td.text for td in row.find_all("td")]))
How can i achieve this please?
Thanks, Riaz

Answer 1

Check each row for both, if both exist then extract the text:
from bs4 import BeautifulSoup
soup = BeautifulSoup(html)
for row in soup.select("#result_table tr"):
div, a = row.select_one("div.testcase"), row.select_one("a.popup_link")
if div and a:
print(div.text.strip(), a.text.strip())
which gives you:
(u'test_000001_login_valid_user', u'pass')
(u'test_000002_select_a_project', u'pass')
(u'test_000003_verify_Lademo_CRM_DataPreview_is_present', u'pass')
Of course if they always go together we can simplify to:
from bs4 import BeautifulSoup
soup = BeautifulSoup(html)
for div in soup.select("#result_table tr div.testcase"):
print(div.text.strip(), div.find_next("a", class_="popup_link").text.strip())