How do I insert a time value in Redshift in a time column? - amazon-web-services

Given the following query
ALTER TABLE public.alldatatypes ADD x_time time ;,
how do I insert a value into x_time?
Time appears to be a valid column type according to the documentation.
https://docs.aws.amazon.com/redshift/latest/dg/r_Datetime_types.html#r_Datetime_types-time
https://docs.aws.amazon.com/redshift/latest/dg/c_Supported_data_types.html
However, when I try to do an insert, I always get an error.
Insert query: insert into public.alldatatypes(x_time) values('08:00:00');
Error:
SQL Error [500310] [0A000]: Amazon Invalid operation:
Specified types or functions (one per INFO message) not supported on
Redshift tables.;
I do not want to use another column type.
I am testing all the column types defined in the documentation.

That cryptic error message is the one Redshift gives when you try to use a leader-only function as source data for a compute node. So expect you aren't showing the exact code you ran to generated this error. I know that it can seem like you didn't change anything important to the issue but you likely have.
You see select now(); works just fine but insert into <table> select now(); will give the error you are showing. This is because now() is a leader only function. However insert into <table> select getdate(); works great - this is because getdate() is a function that runs on compute nodes.
Now the following SQL runs just fine for me:
create table fred (ttt time);
insert into public.fred(ttt) values('01:23:00'); -- this is more correctly written values('01:23:00':time)
insert into public.fred(ttt) select getdate()::time;
select * from fred;
While this throws the error you are getting:
insert into public.fred(ttt) select now()::time;
So if this doesn't help clear things up please post a complete test case that demonstrates the error.

Related

HIVE_PARTITION_SCHEMA_MISMATCH in Athena due to different order in struct?

Full error
HIVE_PARTITION_SCHEMA_MISMATCH: There is a mismatch between the table and partition schemas.
The types are incompatible and cannot be coerced. The column 'ein_verification' in table 'dynamodb_etl_dev.widget_user_snapshots' is declared as type
'struct<status:string,unlocktimestamp:bigint,message:string,lastverifiedtimestamp:bigint,datelastverified:bigint>', but partition 'snapshot_time=2022-08-03T18%3A41' declared column 'ein_verification' as type
'struct<status:string,unlocktimestamp:bigint,lastverifiedtimestamp:bigint,message:string,datelastverified:bigint>'.
It looks like the only difference is the order message:string,lastverifiedtimestamp:bigint is reversed but they are otherwise the same.
I know there are settings for updating the table definition and updating existing partitions with metadata from the table, but I'd like to understand why this is happening and possibly prevent it from happening at all.
Also it appears Athena is not trying to query the latest partition, as there is a new partitition with a more recent timestamp in this s3 bucket. I'm stuck on how to proceeed as I can run this job once and get a single partition and it works fine. But everytime so far that I run it a second time I get the error with struct out of order.
Found the answer to the partion error here . I n particular, theres a comment on how to do it in terraform that helped me get it running

PowerBI subscription error : there is no data for the field at position x

We have run a PowerBI subscription to generate visualisations report in PDF format we have get many errors like this
There is no data for the field at position x
The problem is we searched many times about it we found that it may occurred due to missing data in dataset.
But we have about 30 datasets with a query to oracle database we cannot figure out which is the missing data and the log does not mention which report causes the error.
Is there a way to figure out which field is missing?
Or is there a way to enrich the reports error log to give us which report failed?
A sample of exact error is repeated with different positions :
processing!ReportServer_0-8!1e18!02/07/2022-09:56:36:: e
ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: , Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 29.;
Dears
I found a solution help me. I will share it.
The error is due to missing data, not missing values, which means the column name defined in the data set field has been changed in the database.
note:
When make the value equals null it will not give the same error; even if it is used in the report it will give a different error.
about how to detect ?
Simply install report builder in machine has connection to this database and open this report with report builder and make verify fields, it will give detailed error with the name of dataset fields not found so we tracked it in database we found it has been changed so fix it in either dataset or column name in database it fix the issue.
New challenge we are going to handle it either column name exist or changed to e, never get error and give empty report better as there is some database the report will connect may not have the same column names so it should give empty part of report instead of error.
thanks BR,

How to see 'full' SQL Error Messages in BigQuery?

I am writing a large MERGE statement in BigQuery.
When I attempt to run this query the validator gives me an error involving a lot of ...'s that hides the useful information as shown below:
Value has type ARRAY<STRUCT<eventName STRING, eventUUID STRING, eventDate DATE, ...>> which cannot be inserted into column Events, which has type ARRAY<STRUCT<eventName STRING, eventUUID STRING, eventDate DATE, ...>> at [535:1]
I am extremely confident these two array objects match exactly, however since I am struggling to get around this I would love to see the full error message.
Is there any way to see the full error?
I have looked into the Google Logging tool and cannot see any additional information.
I have also tried the following Cloud Shell command:
bq --format=prettyjson show -j [Job Id Goes Here]
Again, this seems to provide no additional information.
This approach feels pretty silly but it could be the last resort for really long nest type.
Use INFORMATION_SCHEMA.COLUMNS to get a full string of the target type, in your case, type of column Events.
Use CREATE TABLE <yourDataset>.<yourTempTable> AS SELECT ... to dump one row of the Value into a table. Use 1) again to see its full type string.

Google Bigquery: Join of two external tables fails if one of them is empty

I have 2 external tables in BiqQuery, created on top of JSON files on Google Cloud Storage. The first one is a fact table, the second is errors data - and it might or might not be empty.
I can query each table separately just fine, even an empty one - here is an
empty table query result example
I'm also able to left join them if both of them are not empty.
However, if errors table is empty, my query fails with the following error:
The query specified one or more federated data sources but not all of them were scanned. It usually indicates incorrect uri specification or a 'limit' clause over a union of federated data sources that was satisfied without having to read all sources.
This situation isn't covered anywhere in the docs, and it's not related to this versioning issue - Reading BigQuery federated table as source in Dataflow throws an error
I'd rather avoid converting either of this tables to native, since they are used in just one step of the ETL process, and this data is dropped afterwards. One of them being empty doesn't look like an exceptional situation, since plain select works just fine.
Is some workaround possible?
UPD: raised an issue with Google, waiting for response - https://issuetracker.google.com/issues/145230326
It feels like a bug. One workaround is to use scripting to avoid querying the empty table:
DECLARE is_external_table_empty BOOL DEFAULT
(SELECT 0 = (SELECT COUNT(*) FROM your_external_table));
-- do things differently when is_external_table_empty is true
IF is_external_table_empty = true
THEN ...
ELSE ...
END IF

JPA generating an invalid column

Helo there.
I am attempting to a execute a many-to-many get all query. To be clear, I am attepmting to get a collection within a collection to be pulled back. Ie, we will get a result set, but in that result set, there will be a collection of all objects linked to it via a foreign key. Now, to do this, I have a collection which I annotate thusly...
#ManyToMany
#JoinTable(name="QUICK_LAUNCH_DISTLIST",
joinColumns=#JoinColumn(name="QUICK_LAUNCH_ID"),
inverseJoinColumns=#JoinColumn(name="LIST_ID"))
private Collection<QuickLaunchDistlist> distributionLists;
Which seems to be just about text book...
I call a named query which looks like this...
#NamedQuery(name="getQuickLaunch", query = "SELECT q FROM QuickLaunch q")
Which is executed like so...
qlList = emf.createNamedQuery("getQuickLaunch").getResultList();
Every time I make this call, I get back the expected data in the first collection. But none of the collections seem to populate with it. To find out why, I looked at the sql being generated by the call... This is what I find...
I get this exception...
This is a FFDC log generated for the Default Resource Adapter from source:com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.executeQuery
The exception caught:java.sql.SQLSyntaxErrorException: ORA-00904: "T1"."QL_DISTLIST_ID": invalid identifier
SQL Error Code is 904 SQL State is :42000
Along with this query...
SELECT t1.QL_DISTLIST_ID, t2.LIST_ID, t2.CREATE_DATE, t2.CREATE_USERID, t2.description, t2.flag, t2.MOD_DATE, t2.MOD_USERID, t2.ORGANIZATION_ID, t2.owner, t2.STATUS_ID, t1.MESSAGE_TYPE_ID, t1.MOD_DATE, t1.MOD_USERID, t1.QUICK_LAUNCH_ID FROM EPCD13.QUICK_LAUNCH_DISTLIST t0, EPCD13.QUICK_LAUNCH_DISTLIST t1, EPCD13.DISTRIBUTION_LIST t2 WHERE t0.QUICK_LAUNCH_ID = ? AND t0.LIST_ID = t1.QL_DISTLIST_ID AND t1.LIST_ID = t2.LIST_ID(+)
If you look at the first column it request's to pull back you will notice that it selects t1.QL_DISTLIST_ID... Problem is, I have no such named column any where in my db!?!?!? Why on earth is that column being called? How does JPA generate the queries that it calls? If I knew that, I might be a little closer to figuring out what went wrong here or what I did wrong. Any help would be greatly appreciated.