I have a table in BigQuery with a RECORD type column. This RECORD column is just a RECORD, it's not a REPEATED column. I would like to change the type of one of the elements in the record. My record is defined as:
product_action RECORD NULLABLE
product_action.action_type INTEGER NULLABLE
product_action.checkout_step INTEGER NULLABLE
I would like to change the type of ACTION_TYPE from INTEGER to STRING, but I don't know how to do it. I know it involves selecting and casting to a new table, but I don't know the right syntax. There is a very similar question here with an answer
How to change the col type of a BigQuery repeated record
but it's only for REPEATED records and was not able to change it to RECORD only (not REPEATED).
Thanks!
Potential problem with OP's answer is that it does not preserve original order of fields in record, so it ends up with below schema
product_action RECORD NULLABLE
product_action.checkout_step INTEGER NULLABLE
product_action.action_type STRING NULLABLE <-- Type and position changed
instead of
product_action RECORD NULLABLE
product_action.action_type STRING NULLABLE <-- Type changed while position preserved
product_action.checkout_step INTEGER NULLABLE
Below handles this issue
#standardSQL
SELECT * REPLACE((
SELECT AS STRUCT product_action.* REPLACE(
CAST(product_action.action_type AS STRING) AS action_type
)) AS product_action)
FROM `mytable`
Got it working finally
SELECT
* REPLACE
(
(
SELECT
AS STRUCT product_action.* EXCEPT (action_type),
CAST(product_action.action_type AS STRING) AS action_type
) AS product_action
),
FROM
`mytable`
Related
For BigQuery, we can extract the special repeated structs using UNNEST. This works very well when you have data in the struct key and value (one of string, int or double). However, occasionally I have a struct with a key ("foo") and often with data value.string_value but not always. The following will NOT include the row when there is no string_value:
SELECT
event_timestamp,
event_params_custom.value.string_value
FROM
`my_database.events_*` T,
UNNEST(T.event_params) event_params_custom
WHERE
event_name = "some_custom_event"
AND event_params_custom.key = "foo"
Is it possible to do something akin to "outer join" and include all rows even when there is no value for the key "foo"?
#standardSQL
SELECT
event_timestamp,
(SELECT value.string_value FROM UNNEST(event_params) WHERE key = "foo") AS string_value
FROM
`my_database.events_*`
WHERE
event_name = "some_custom_event"
I am importing data into Power BI desktop from an XML file. If the field value is empty, e.g.,
<Address></Address> then PBI assigns that column type "Table" and it is not possible to change the type into text. I am getting an error:
Expression.Error: We cannot convert a value of type Table to type Text.
Details:
Value=Table
Type=Type
It looks like if the first encountered value is empty, then all following rows of that column in the same "page" (data is being read in chunks a.k.a "pages" by the data connector) fetch are assigned type "Table", though not always.
For some columns, it even not possible to change the column data type (they have "table" icon)
Any ideas on how to change the column type to text?
Edit1: I have noticed that clicking "Table" actually shows there is a text value in that column. Maybe there is some invisible setting for Power BI to overcome this blank string value problem?
Edit2: it looks like I have the same problem as described here https://community.powerbi.com/t5/Desktop/XML-Import-Table-Type-in-Column/td-p/97512. There is no solution to this simple problem?
I found this https://community.powerbi.com/t5/Issues/Power-Query-should-not-wrap-simple-xml-elements-in-a-table/idi-p/125525. It looks it is "by design" and no solution exists. It's a pitty for wasting time...
I have found someone's solution:
let
ConvertTableField = (aTable as table, fieldName as text) =>
let
//alternative: https://community.powerbi.com/t5/Desktop/Expand-value-from-table/td-p/214838
//#"Expanded g_v" = Table.TransformColumns(#"Expanded ts_info", {{"g_v", each if _ is table then Table.FirstValue(_, "") else _}})
#"Lists from table" =
Table.TransformColumns(
aTable,
{
{
fieldName,
each
if _ is table then
Table.ToList(_)
else {_}
}
}
),
#"Expanded List" =
Table.ExpandListColumn(
#"Lists from table",
fieldName
)
in
#"Expanded List"
in
ConvertTableField
I haven't still figured out how to run this function for all the columns of the table. M language is the most difficult of the programming languages I know more or less...
I am not sure about the XML usage, but as far as table type to text conversion goes - this worked for me. Maybe it's an overkill.
let
Source = (Table1 as table) =>
let
Section = Record.ToTable(#sections[Section1]),
Compare =
Table.SelectRows(
Section,
each [Value] = Table1
){0},
GetName = Record.Field(Compare, "Name")
in
GetName
in
Source
in sqlite it is possible to have string by which the table was created:
select sql from sqlite_master where type='table' and tbl_name='MyTable'
this could give:
CREATE TABLE "MyTable" (`id` PRIMARY KEY NOT NULL, [col1] NOT NULL,
"another_col" UNIQUE, '`and`,''another'',"one"' INTEGER, and_so_on);
Now I need to extract with this string any additional parameters that given column name has been set with.
But this is very difficult since the column name could be enclosed with special characters, or put plain, column name may have some special characters that are used as encapsulation etc.
I don't know how to approach it. The result should be having a column name the function should return anything that is after this name and before , so giving it id it should return PRIMARY KEY NOT NULL.
Use the pragma table_info:
http://www.sqlite.org/pragma.html#pragma_table_info
sqlite> pragma table_info(MyTable);
cid|name|type|notnull|dflt_value|pk
0|id||1||1
1|col1||1||0
2|another_col||0||0
3|`and`,'another',"one"|INTEGER|0||0
4|and_so_on||0||0
I use HSQLDB2.0 and JPA2.0 for my current project and have few date columns in DB.
I would like to run wildcard queries on the date columns. How could I do that?
Ex : If my DB contains two rows with date values as : 10-01-2011 and 15-02-2011
and my search criteria will be "%10-01%", then result should be 10-01-2011.
Else if search criteria is "%2011%" then both rows need to be fetched with the select query.
Thanks in advance,
Satya
You can define an autogenerated column of type VARCHAR containing a copy of the date. You can then perform queries with both LIKE predicates and REGEXP_MATCHES() function. An example of the column definition is below:
DATEGEN VARCHAR(10) GENERATED ALWAYS AS (CAST(DATECOL AS VARCHAR(10))
Note the string representation of DATE is in the form '2011-02-26' and your query strings should follow this pattern.
This can be achieved in this format :
select date_birth from member where to_char(date_birth,'MM-yyyy') like '%02-2011%'
select date_birth from member where to_char(date_birth,'MM-dd') like '%02-15%'
select date_birth from member where to_char(date_birth,'dd-yyyy') like '%30-2011%'
Regards,
Satya
There is a problem when the int is nullable on GetHashCode
At the point of GetHashCode on ActiveRecord.tt there is a need for a nullable check.
Something like this.
<#
if(tbl.PK.SysType=="int" && !tbl.PK.Nullable ){
#>
public override int GetHashCode() {
return this.<#=tbl.PK.CleanName #>;
}
<# }#>
(update)
This value can be come null on the Views.
I have include the views using this code I found on inet.
const string TABLE_SQL=#"SELECT *
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE='BASE TABLE'
union
select Table_catalog, table_schema, table_name, 'View' table_type
from information_schema.views";
After that, this error appears.
SubSonic requires a primary key that is not null. You need to modify your view so it returns an Id column that is a non null Integer or Guid.
EDIT: You can add a non null primary key that'll work for SubSonic by changing your view as follows:
CREATE VIEW MyView AS
SELECT NewID() AS Id, *
FROM MyTable
The one drawback is that you won't get a consistent Id. If that's a problem you could replace NewID() with a value computed from the column but the drawback of that is the performance will be slower.