I would like to know how to use the IFNULL() BigQuery Standard SQL function properly. This is my current data structure. The columns named "key" and "stringColumn" store strings. Meanwhile, the column named "integerColumn" stores integers:
I would like to create a new column named "singleValueColumn" that takes the value of the "stringColumn" or "integerColumn" that is not null:
This is my BigQuery Standard SQL query:
SELECT key,
value.string_value as stringColumn,
value.int_value as integerColumn,
IFNULL(value.string_value, value.int_value) as singleValueColumn
FROM `com_skytracking_ANDROID.app_events_*`,
UNNEST(event_dim) as event,
UNNEST(event.params) as event_param
WHERE event.name = "order_event"
However, when I run the query I am getting this error:
Error: No matching signature for function IFNULL for argument types: STRING, INT64. Supported signature: IFNULL(ANY, ANY) at [4:9]
Thanks for your help.
Check this doc. I think you need to cast the int_value as a string:
IFNULL(value.string_value, CAST(value.int_value AS STRING)) AS singleValueColumn
Related
I have a C++Builder SQL Statement with a parameter like
UnicodeString SQLStatement = "INSERT INTO TABLENAME (DATETIME) VALUES (:dateTime)"
Can I add the parameter without quotes?
Usually I'd use
TADOQuery *query = new TADOQuery(NULL);
query->Parameters->CreateParameter("dateTime", ftString, pdInput, 255, DateTimeToStr(Now()));
which will eventually produce the SQL String
INSERT INTO TABLENAME (DATETIME) VALUES ('2022-01-14 14:33:00.000')
but because this is a legacy project (of course, it always is) and I have to maintain different database technologies, I need to be able to inject database specific date time conversion methods, so that the endresult would look like
INSERT INTO TABLENAME (DATETIME) VALUES (to_date('2022-01-14 14:33:00.000', 'dd.mm.yyyy hh24:mi:ss')))
If I try injecting this via my 'usual' method (because I don't think I can inject a second parameter into this one) it'd look like:
TADOQuery *query = new TADOQuery(NULL);
query->Parameters->CreateParameter("dateTime", ftInteger, pdInput, 255, "to_date('" + DateTimeToStr(Now()) + "', 'dd.mm.yyyy hh24:mi:ss')");
but of course the result would look like:
INSERT INTO TABLENAME (DATETIME) VALUES ('to_date('2022-01-14 14:33:00.000', 'dd.mm.yyyy hh24:mi:ss')'))
and therefore be invalid
Or is there another way to do this more cleanly and elegantly? Although I'd settle with 'working'.
I can work around this by preparing two SQL Statements and switch the statement when another database technology is but I just wanted to check if there is another way.
Why are you defining the parameter's DataType as ftInteger when your input value is clearly NOT an integer? You should be defining the DataType as ftDateTime instead, and then assigning Now() as-is to the parameter's Value. Let the database engine decide how it wants to format the date/time value in the final SQL per its own rules.
query->Parameters->CreateParameter("dateTime", ftDateTime, pdInput, 0, Now());
I am using REGEXP_EXTRACT function in Google BigQuery to extract a specific word from a string. While regexp works good when tested, function REGEXP_EXTRACT returns null in Google BigQuery.
For example, there is string "RR_SM_Brand_A_Additive_Clean_jun2020", and I want to extract a value from the list (Brand_A, Brand_B, Brand_C, etc.)
When I test RegExp, I receive correct value Brand_A: https://regexr.com/5tecm
RegExp Code: Brand_A|Brand_B (thanks to #Barmar)
But when I run it in Google BigQuery:
SELECT distinct utm_campaign, -- REGEXP_EXTRACT(utm_campaign, r"(?:Brand_A|Brand_B)") REGEXP_EXTRACT(utm_campaign, r"Brand_A|Brand_B") FROM project.dataset.table WHERE utm_campaign = "RB_Display_Brand_A_Botanica_2020"
I receive "This query returned no results.", and not expected Brand_A value.
Note: BigQuery does not return "Cannot parse regular expression: invalid perl operator: (?<"." like in question "duplicate"
I'd suggest that your WHERE clause may be at issue. Both forms you use extract the brand string you appear to be asking for. If the REGEXP_EXTRACT was not matching you'd still get rows, but the value would be NULL.
Converting this to just use the literal from your existing where clause:
SELECT
val,
REGEXP_EXTRACT(val, r"(?:Brand_A|Brand_B)"),
REGEXP_EXTRACT(val, r"Brand_A|Brand_B")
FROM
(
SELECT "RB_Display_Brand_A_Botanica_2020" as val
)
I generate a list of ID numbers. I want to execute an insert statement that grabs all records from one table where the ID value is in my list and insert those records into another table.
Instead of running through multiple execute statements (as I know is possible), I found this cx_Oracle function, that supposedly can execute everything with a single statement and list parameter. (It also avoids the clunky formatting of the SQL statement before passing in the parameters) But I think I need to alter my list before passing it in as a parameter. Just not sure how.
I referenced this web page:
https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-executemany.html
ids = getIDs()
print(ids)
[('12345',),('24567',),('78945',),('65423',)]
sql = """insert into scheme.newtable
select id, data1, data2, data3
from scheme.oldtable
where id in (%s)"""
cursor.prepare(sql)
cursor.executemany(None, ids)
I expected the SQL statement to execute as follows:
Insert into scheme.newtable
select id, data1, data2, data3 from scheme.oldtable where id in ('12345','24567','78945','65423')
Instead I get the following error:
ORA-01036: illegal variable name/number
Edit:
I found this StackOverflow: How can I do a batch insert into an Oracle database using Python?
I updated my code to prepare the statement before hand and updated the list items to tuples and I'm still getting the same error.
You use executemany() for batch DML, e.g. when you want to insert a large number of values into a table as an efficient equivalent of running multiple insert statements. There are cx_Oracle examples discussed in https://blogs.oracle.com/opal/efficient-and-scalable-batch-statement-execution-in-python-cx_oracle
However what you are doing with
insert into scheme.newtable
select id, data1, data2, data3
from scheme.oldtable
where id in (%s)
is a different thing - you are trying to execute one INSERT statement using multiple values in an IN clause. You would use a normal execute() for this.
Since Oracle keeps bind data distinct from SQL, you can't pass in multiple values to a single bind parameter because the data is treated as a single SQL entity, not a list of values. You could use %s string substitution syntax you have, but this is open to SQL Injection attacks.
There are various generic techniques that are common to Oracle language interfaces, see https://oracle.github.io/node-oracledb/doc/api.html#sqlwherein for solutions that you can rewrite to Python syntax.
using temporary table to save ids (batch insert)
cursor.prepare('insert into temp_table values (:1)')
dictList = [{'1': x} for x in ids]
cursor.executemany(None, dictList)
then insert selected value into newtable
sql="insert into scheme.newtable (selectid, data1, data2, data3 from scheme.oldtable inner join temp_table on scheme.oldtable.id = temp_table.id)"
cursor.execut(sql,connection)
the script of create temporary table in oracle
CREATE GLOBAL TEMPORARY TABLE temp_table
(
ID number
);
commit
I hope this useful.
I have a table
CREATE TABLE TEST
(
test_column CLOB
)
I want to change the datatype of test_column to LVARCHAR. How can I achieve this? I tried several things until now:
alter table test modify test_column LVARCHAR(2500)
This works, but the content of test_column gets converted from 'test' to '01000000d9c8b7a61400000017000000ae000000fb391956000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000'.
alter table test add tmp_column LVARCHAR(2500);
update test set tmp_column = DBMS_LOB.SUBSTR(test_column,2500,1);
This does not work and I get the following exception:
[Error Code: -674, SQL State: IX000] Method (substr) not found.
Do you have any further ideas?
Using a 12.10.xC5DE instance to do some tests.
From what i could find in the manuals, there isn't a cast from CLOB to other data types.
CLOB data type
No casts exist for CLOB data. Therefore, the database server cannot convert data of the CLOB type to any other data type, except by using these encryption and decryption functions to return a BLOB. Within SQL, you are limited to the equality ( = ) comparison operation for CLOB data. To perform additional operations, you must use one of the application programming interfaces from within your client application.
The encryption/decryption functions mentioned still return CLOB type objects, so they do not do what you want.
Despite the manual saying that there is no cast for CLOB, there is a registered cast in the SYSCASTS table. Using dbaccess , i tried an explicit cast on some test data and got return values similar to the ones you are seeing. The text in the CLOB column is 'teste 01', terminated with a line break.
CREATE TABLE myclob
(
id SERIAL NOT NULL
, doc CLOB
);
INSERT INTO myclob ( id , doc ) VALUES ( 0, FILETOCLOB('file1.txt', 'client'));
SELECT
id
, doc
, doc::LVARCHAR AS conversion
FROM
myclob;
id 1
doc
teste 01
conversion 01000000d9c8b7a6080000000800000007000000a6cdc0550000000001000000000
0000000000000000000000000000000000000000000000000000000000000000000
0000000000
So, there is a cast from CLOB, but it does not seem to be useful for what you want.
So back to the SQL Packages Extension . You need to register this datablade on the database. The files required are located in the $INFORMIXDIR/extend and you want the excompat.* module. Using the admin API, you can register the module by executing the following:
EXECUTE FUNCTION sysbldprepare('excompat.*', 'create');
If the return value is 0 (zero) then the module should now be registered.
SELECT
id
, DBMS_LOB_SUBSTR(doc, DBMS_LOB_GETLENGTH(doc) - 1, 1) as conversion
FROM
myclob;
id 1
conversion teste 01
Another way would be to register your own cast from CLOB to LVARCHAR, but you would have to code an UDR to implement it.
P.S:
Subtracting 1 from the CLOB length to remove the line break.
I have a table with one column having a large json object in the format below. The column datatype is VARCHAR
column1
--------
{"key":"value",....}
I'm interested in the first value of the column data
in regex I can do it by .*?:(.*),.* with group(1) giving me the value
How can i use it in the select query
Don't do that, it's bad database design. Shred the keys and values to their own table as columns, or use the XML data type. XML would work fine because you can index the structure well, and you can use XPATH queries on the data. XPATH supports regexp natively.
You can use regular expression with xQuery, you just need to call the function matches from a SQL query or a FLORW query.
This is an example of how to use regular expressions from SQL:
db2 "with val as (
select t.text
from texts t
where xmlcast(xmlquery('fn:matches(\$TEXT,''^[A-Za-z 0-9]*$'')') as integer) = 0
)
select * from val"
For more information:
http://pic.dhe.ibm.com/infocenter/db2luw/v10r5/topic/com.ibm.db2.luw.xml.doc/doc/xqrfnmat.html
http://angocadb2.blogspot.fr/2014/04/regular-expressions-in-db2.html
DB2 doesn't have any built in regex functionality, unfortunately. I did find an article about how to add this with libraries:
http://www.ibm.com/developerworks/data/library/techarticle/0301stolze/0301stolze.html
Without regexes, this operation would be a mess. You could make a function that goes through the string character by character to find the first value. Or, if you will need to do more than this one operation, you could make a procedure that parses the json and throws it into a table of keys/values. Neither one sounds fun, though.
In DB2 for z/OS you will have to pass the variable to XMLQUERY with the PASSING option
db2 "with val as (
select t.text
from texts t
where xmlcast(xmlquery('fn:matches($TEXT,''^[A-Za-z 0-9]*$'')'
PASSING t.text as "TEXT") as integer) = 0
)
select * from val"