Hive regex serde for multi level YAML data? - regex

Question

I have a custom export of several tables and their columns in multilevel yaml format.
example extract modified with dummy values:
schemas:
- name: exports
tables:
- name: sugar
description: makes stuff sweet
active_date: 2019-01-07 00:00:00
columns:
- name: color
type: abcd
- name: taste
type: abcd
description: xyz
example: 21352352
- name: structure
type: abcd
description: xyzasaa
example: 10001
- name: salt
description: not that sweet.
makes it salty.
active_date: 2018-12-18 00:00:00
columns:
- name: strength
type: abcdef
description: easy to find
example: 2018-03-03 12:30:00
- name: color
type: abcdeffa
description: not sweet
example: 21352352
- name: quality
type: abcd
description: how much is needed
example: 10001
I need to ingest this data in hive table using some serde. I am familiar with jsonSerde but unfortunately this format is not supported and so looking for an alternate way. Can someone suggest an optimum approach? Can a regexSerde help at all with what I am trying to achieve?
Hive table Data can be represented maybe in one of the below ways:
<style type="text/css">
table.tableizer-table {
font-size: 12px;
border: 1px solid #CCC;
font-family: Arial, Helvetica, sans-serif;
}
.tableizer-table td {
padding: 4px;
margin: 3px;
border: 1px solid #CCC;
}
.tableizer-table th {
background-color: #104E8B;
color: #FFF;
font-weight: bold;
}
</style>
<table class="tableizer-table">
<thead><tr class="tableizer-firstrow"><th>Level 1(name)</th><th>Level 2(name)</th><th>Level 2 (type)</th><th>Level 2 (description)</th></tr></thead><tbody>
<tr><td>sugar</td><td>color</td><td>abcd</td><td> </td></tr>
<tr><td>sugar</td><td>taste</td><td>abcd</td><td>xyz</td></tr>
<tr><td>sugar</td><td>structure</td><td>abcd</td><td>xyzasaa</td></tr>
<tr><td>salt</td><td>strength</td><td>abcdef</td><td>easy to find</td></tr>
<tr><td>salt</td><td>color</td><td>abcdeffa</td><td>not sweet</td></tr>
<tr><td>salt</td><td>quality</td><td>abcd</td><td>how much is needed</td></tr>
</tbody></table>
--- or ---
<style type="text/css">
table.tableizer-table {
font-size: 12px;
border: 1px solid #CCC;
font-family: Arial, Helvetica, sans-serif;
}
.tableizer-table td {
padding: 4px;
margin: 3px;
border: 1px solid #CCC;
}
.tableizer-table th {
background-color: #104E8B;
color: #FFF;
font-weight: bold;
}
</style>
<table class="tableizer-table">
<thead><tr class="tableizer-firstrow"><th>Level 1(name.colum)</th><th>Level 2 (type)</th><th>Level 2 (description)</th></tr></thead><tbody>
<tr><td>sugar.color</td><td>abcd</td><td> </td></tr>
<tr><td>sugar.taste</td><td>abcd</td><td>xyz</td></tr>
<tr><td>sugar.structure</td><td>abcd</td><td>xyzasaa</td></tr>
<tr><td>salt.strength</td><td>abcdef</td><td>easy to find</td></tr>
<tr><td>salt.color</td><td>abcdeffa</td><td>not sweet</td></tr>
<tr><td>salt.quality</td><td>abcd</td><td>how much is needed</td></tr>
</tbody></table>
EDIT: Using simplest approach I was able to extract below:
$ grep -P '(?<=- name: ).*' export.yaml
- name: exports
- name: sugar
- name: color
- name: taste
- name: structure
- name: salt
- name: strength
- name: color
- name: quality
but how do I establish the indentation relationship so that result is like :
sugar.color,sugar.taste,sugar.structure
salt.strength,salt.color,salt.quality