How to use string as column name in Bigquery - google-cloud-platform

There is a scenario where I receive a string to the bigquery function and need to use it as a column name.
here is the function
CREATE OR REPLACE FUNCTION METADATA.GET_VALUE(column STRING, row_number int64) AS (
(SELECT column from WORK.temp WHERE rownumber = row_number)
);
When I call this function as select METADATA.GET_VALUE("TXCAMP10",149); I get the value as TXCAMP10 so we can say that it is processed as SELECT "TXCAMP10" from WORK.temp WHERE rownumber = 149 but I need it as SELECT TXCAMP10 from WORK.temp WHERE rownumber = 149 which will return some value from temp table lets suppose the value as A
so ultimately I need value A instead of column name i.e. TXCAMP10.
I tried using execute immediate like execute immediate("SELECT" || column || "from WORK.temp WHERE rownumber =" ||row_number) from this stack overflow post to resolve this issue but turns out I can't use it in a function.
How do I achieve required result?

I don't think you can achieve this result with the help of UDF in standard SQL in BigQuery.
But it is possible to do this with stored procedures in BigQuery and EXECUTE IMMEDIATE statement. Consider this code, which simulates the situation you have:
create or replace table d1.temp(
c1 int64,
c2 int64
);
insert into d1.temp values (1, 1), (2, 2);
create or replace procedure d1.GET_VALUE(column STRING, row_number int64, out result int64)
BEGIN
EXECUTE IMMEDIATE 'SELECT ' || column || ' from d1.temp where c2 = ?' into result using row_number;
END;
BEGIN
DECLARE result_c1 INT64;
call d1.GET_VALUE("c1", 1, result_c1);
select result_c1;
END;

After some research and trial-error methods, I used this workaround to solve this issue. It may not be the best solution when you have too many columns but it surely works.
CREATE OR REPLACE FUNCTION METADATA.GET_VALUE(column STRING, row_number int64) AS (
(SELECT case
when column_name = 'a' then a
when column_name = 'b' then b
when column_name = 'c' then c
when column_name = 'd' then d
when column_name = 'e' then e
end from WORK.temp WHERE rownumber = row_number)
);
And this gives the required results.
Point to note: the number of columns you use in the case statement should be of the same datatype else it won't work

Related

combine column in json format in big query

I have columns in bigquery like this:
expected output:
I am trying to merge columns into json using bigquery.
I am taking letter before underscore(common name ) as column then converting.
I am trying this query:
with selectdata as (
SELECT a_firstname, a_middlename,a_lastname FROM `account_id.Dataset.Table_name`
)
select TO_JSON_STRING(t) AS json_data FROM selectdata AS t;
How can I join columns with condition or with case to achieve this output in bigquery
Consider below approach
create temp function extract_keys(input string) returns array<string> language js as """
return Object.keys(JSON.parse(input));
""";
create temp function extract_values(input string) returns array<string> language js as """
return Object.values(JSON.parse(input));
""";
select * except(row_id) from (
select format('%t',t) row_id,
split(key, '_')[offset(0)] as col,
'{' || string_agg(format('"%s":"%s"', split(key, '_')[safe_offset(1)], value)) || '}' as value
from your_table t, unnest(extract_keys(to_json_string(t))) key with offset
join unnest(extract_values(to_json_string(t))) value with offset
using(offset)
group by row_id, col
)
pivot (any_value(value) for col in ('a','b','c'))
if applied to sample data in your question - output is

JOINING Two Tables together in SAS

I am working on the this SAS code and would need assistance with joining the two tables below. I am getting errors while trying to join the two tables.
Requirement: i. Left Join Table B to Table A
Table A:
PROC SQL;
create table stand as select distinct
put(datepart(Max(a.REPORT_DATE)),Date9.) as M_Date
, a.BUSINESS_GROUP as PORTF_LEVEL1
, A.SPLIT as PORTF_LEv2
, Count(distinct a.Report_Date) as Number_of_Days
, (B.TOTAL_BREACH/Count(distinct a.Report_Date))*100 as FREQ
, A.MINIMUM_ACCEPTABLE_COUNT
, A.MAX_COUNT
, (case WHEN (B.TOTAL_BREACH/Count(distinct a.Report_Date)) * 100 LT MIN_COUNT
THEN 'TRUE' ELSE 'FALSE' END) as NUMBER__UNDER
, (case WHEN (B.TOTAL_BREACH/Count(distinct a.Report_Date)) * 100 GT MAX_COUNT THEN 'TRUE' ELSE 'FALSE' END) as NUMBER__OVER
from temp a
INNER join
( select BUSINESS_GROUP as PORTF_LEVEL1
,SPLIT AS PORTF_LEv2
,Count(distinct c.Report_Date) as Number_of_Days
from temp c
Inner join temp2 d
on c.Report_Date=d.Report_Date
WHERE &Alert and TENOR = '+'
and datepart(c.REPORT_DATE) ge '31-APR-21'd
and datepart(c.REPORT_DATE) le '31-APR-22'd
Group by BUSINESS_GROUP, SPLIT
)B
on a.BUSINESS_GROUP = b.PORTF_LEVEL1
AND a.SPLIT = b.PORTF_LEVEL2
INNER JOIN temp2 e
on a.REPORT_DATE = e.REPORT_DATE
where &Alert and TENOR = '+'
and datepart(a.REPORT_DATE) ge '31-APR-21'd
and datepart(a.REPORT_DATE) le '31-APR-22'd
Group by Business_GROUP, SPLIT
;
QUIT;
Table B:
In the table B, i am trying to find the median of the variable Data_M. The code seems to be okay. I only need assistance joining the Table B to table A above.
Proc sql outobs=1; create table median_dt1 as select distinct put(datepart(max(REPORT_DATE)), date9.) as M_Date , median(Data_M) as median_data from transp
WHERE datepart(REPORT_DATE) ge '01-APR-22'd and datepart(REPORT_DATE) le '31-APR-22'd group by BUSINESS_GROUP order by Report_Date Desc; quit;
Thank you in advance!
sas
from temp a
INNER join
( select BUSINESS_GROUP as PORTF_LEVEL1
,SPLIT AS PORTF_LEv2
,Count(distinct c.Report_Date) as Number_of_Days
from temp c
Inner join temp2 d
on c.Report_Date=d.Report_Date
WHERE &Alert and TENOR = '+'
and datepart(c.REPORT_DATE) ge '31-APR-21'd
and datepart(c.REPORT_DATE) le '31-APR-22'd
Group by BUSINESS_GROUP, SPLIT
)B
on a.BUSINESS_GROUP = b.PORTF_LEVEL1
AND a.SPLIT = b.PORTF_LEVEL2
You're trying to join on b.PORTF_LEVEL2. However, that column doesn't exist in B. The column "PORTF_LEV2" exists, though. Try that?
If that doesn't resolve the issue, please paste the complete error message that you're receiving.

Custom Function Power Query (M) - Return Table

I need a custom function that takes two parameters, Column1 and Column2, so:
For each Row, return the value of Column1 but only if exists a Value in the Column2 else return null
I have tried this:
let ColumnsFilter = (Tabla,C1,C2)=>
Table.AddColumn(Tabla, "Custom", each if [C2] <> null then [C1] else null)
in
ColumnsFilter
And calling the function:
#"Previous Step" = .....
#"P" = ColumnsFilter(#"Previous Step","Column1","Column2")
in
P
And is not working. clearly I am not using the syntax properly.
In summary I need a table as input and a table as output adding custom columns.
How can I write this?
(Please don't tell me to use the assisted of Power Query, I need to write similar functions manually)
Since you're passing column names as text and individual rows are a record type, you have to use Record.Field to pull the right column (field) from the current row (record).
let
ColumnsFilter = (Tabla as table, C1 as text, C2 as text) as table =>
Table.AddColumn(Tabla, "Custom",
each if Record.Field(_, C2) <> null then Record.Field(_, C1) else null
)
in
ColumnsFilter

How to return multiple values through function

I have created the below function that will return workspace details which the loggedin user has access to.
But this function is returning only the first record from the select list.
I need all the records to be displayed as output.
Please modify it and let me know.
CREATE OR REPLACE FUNCTION "F_WORKSPACE_LOGIN_USERS" (
p_email VARCHAR2
) RETURN VARCHAR2 IS
l_error VARCHAR2(1000);
l_workspace VARCHAR2(1000);
l_teams VARCHAR2(1000);
l_team VARCHAR2(1000);
BEGIN
FOR i IN ( SELECT a.name workspace,
a.team_id id
FROM slackdatawarehouse.teams a,
( SELECT TRIM(workspaces) workspaces
FROM alluser_workspaces_fact
WHERE lower(email) = lower(p_email)
) b
WHERE a.team_id IN ( SELECT c.team_id
FROM slackdatawarehouse.team_tokens c
)
OR instr(', '
|| lower(b.workspaces),', '
|| lower(a.name) ) > 0
ORDER BY 1 ) LOOP
l_teams := l_team
|| ','
|| i.id;
l_teams := ltrim(rtrim(l_teams,','),',');
RETURN l_teams;
END LOOP;
END;
Current output is :
T6HPQ5LF7,T6XBXVAA1,T905JLZ62,T7CN08JPQ,T9MV4732M,T5PGS72NA,T5A4YHMUH,TAAFTFS0P,T69BE9T2A,T85D2D8MT,T858U7SF4,T9D16DF5X,T9DHDV61G,T9D17RDT3,T5Y03HDQ8,T5F5QPRK7
Required output is :
T6HPQ5LF7
T6XBXVAA1
T905JLZ62
i need output like above as one by one
I don't know what that code really does (can't test it), but this might be the culprit:
...
RETURN l_teams;
END LOOP;
As soon as code reaches the RETURN statement, it exits the loop and ... well, returns what's currently in L_TEAMS variable. Therefore, move RETURN out of the loop:
...
END LOOP;
RETURN l_teams;
If it still doesn't work as expected (which might be the case), have a look at pipelined functions (for example, on Oracle-base site) as they are designed to return values you seem to be looking for.
A simple example:
SQL> create or replace type t_dp_row as object
2 (deptno number,
3 dname varchar2(20));
4 /
Type created.
SQL> create or replace type t_dp_tab is table of t_dp_row;
2 /
Type created.
SQL> create or replace function f_depts
2 return t_dp_tab pipelined
3 is
4 begin
5 for cur_r in (select deptno, dname from dept)
6 loop
7 pipe row(t_dp_row(cur_r.deptno, cur_r.dname));
8 end loop;
9 return;
10 end;
11 /
Function created.
SQL> select * from table(f_depts);
DEPTNO DNAME
---------- --------------------
10 ACCOUNTING
20 RESEARCH
30 SALES
40 OPERATIONS
SQL>

SELECT Statement within IF statement

I would like to get a different result to my select statement when a parameter is 0, 1 or 2. I am not very skilled in PLSQL so I am not sure if my code would give the expected result. If i run this code i get a "SQL statement ignored" on line 3.
BEGIN
IF (:PARTYPE = 1) THEN
SELECT * FROM x
WHERE to_date(date) >= (Select to_date(sysdate)from DNV.dual)
ELSE
SELECT * FROM x
WHERE to_date(date) <= (Select to_date(sysdate)from DNV.dual)
END IF;
END;
This is just a example of my SELECT statement. Later this statement will become longer and more complex but I think this shows which results I am trying to get.
Below is a copy of my entire code but because I am not allowed to show this it has become very unreadable:
BEGIN
IF (:PARTYPE = 1) THEN
Select table1.Column1
, table1.Column2
, table1.Column3
, table1.Column4
, table1.Column5
, table1.Column6
, table1.Column7
, table1.Column8
, table1.Column9
, table1.Column10
, table1.Column11
, table1.Column12
, (Select table2.ColumnX From x2 table2 Where somthing) as "something" From x1 table1
WHERE to_date(date) >= (Select to_date(sysdate)from DNV.dual)
Order by columnX
ELSE
Select table1.Column1
, table1.Column2
, table1.Column3
, table1.Column4
, table1.Column5
, table1.Column6
, table1.Column7
, table1.Column8
, table1.Column9
, table1.Column10
, table1.Column11
, table1.Column12
, (Select table2.ColumnX From x2 table2 Where somthing) as "something" From x1 table1
WHERE to_date(date) <= (Select to_date(sysdate)from DNV.dual)
Order by columnX
END IF;
END;
I have created some new code with which i am trying to learn how a case statement works. This might help me with the code above. Unfortunately this code also doesn't work but I think it explanes my situation better. In this excample i use a separate table with data i made up. In some cases user2 is null but user1 is always filled. I want to get all items where user2 equals the parameter but if user2 is null and user1 does equal the paramter i still need that item to apear.
Select t1.user1,
t1.user2
From table t1
Where (Case
When t1.user2 IS NULL Then t1.user1 in (:PARUSER)
ELSE t1.user2 in (:PARUSER)
End Case)
Since the relational operator of the where clause depends on the partype, you cannot do the traditional CASE statement charm here. I'll have to resort with this one:
SELECT * FROM x
WHERE (to_date(date) >= (Select to_date(sysdate)from DNV.dual) AND :PARTYPE = 1)
OR (to_date(date) <= (Select to_date(sysdate)from DNV.dual) AND :PARTYPE != 1)