I have a text field "completed_on" with text values "Thu Jan 27 2022 11:55:12 GMT+0530 (India Standard Time)".
I need to convert this into timestamp.
I tried , cast(completed_on as timestamp) which should give me the timestamp but I am getting the following error in REDSHIFT
ERROR: Char/varchar value length exceeds limit for date/timestamp conversions
Since timestamps can be in many different formats, you need to tell Amazon Redshift how to interpret the string.
From TO_TIMESTAMP function - Amazon Redshift:
TO_TIMESTAMP converts a TIMESTAMP string to TIMESTAMPTZ.
select sysdate, to_timestamp(sysdate, 'YYYY-MM-DD HH24:MI:SS') as seconds;
timestamp | seconds
-------------------------- | ----------------------
2021-04-05 19:27:53.281812 | 2021-04-05 19:27:53+00
For formatting, see: Datetime format strings - Amazon Redshift.
Related
I have date in the following format in a file:
January 2, 2020 8:15:32 AM UTC
How to query it using Athena? I am trying to query all the rows where date is greater than a specific date.
I don't see above mentioned date format in the documentation.
As #John Rotenstein said you will have problens if your file row has columns split by , because your date columns also uses ,.
Nevertheless here is how you can cast your string January 2, 2020 8:15:32 AM UTC to date in Athena.
Athena uses presto in the backend and you can use the parse_datetime with the corresponding format to convert your string to a date.
parse_datetime(string, format) → timestamp with time zone
WITH t(x) AS (VALUES 'January 2, 2020 8:15:32 AM UTC')
SELECT parse_datetime(x, 'MMM d, YYYY hh:mm:ss a z')
FROM t;
produces the output
"_col0"
"2020-01-02 08:15:32.000 UTC"
I am trying to create an athena table with a timestamp column that has time zone information. The create sql looks something like this:
CREATE EXTERNAL TABLE `tmp_123` (
`event_datehour_tz` timestamp with time zone
)
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
's3://...'
TBLPROPERTIES (
'Classification'='parquet'
)
When I run this, I get the error:
line 1:8: mismatched input 'external'. expecting: 'or', 'schema', 'table', 'view' (service: amazonathena; status code: 400; error code: invalidrequestexception; request id: b7fa4045-a77e-4151-84d7-1b43db2b68f2; proxy: null)
If I remove the with time zone it will create the table. I've tried this and timestamptz. Is it not possible to create a table in athena that has a timestamp with time zone column?
Unfortunately Athena does not support timestamp with time zone.
What you may do is use the CAST() function around that function call, which will change the type from timestamp with time zone into timestamp.
Or, you can maybe save it as timestamp and use AT TIME STAMP operator as given below:
SELECT event_datehour_tz AT TIME ZONE 'America/Los_Angeles' AS la_time;
Just to give a complete solution after #AswinRajaram answered that Athena does not support timestampo with timezone. Here is how one can CAST the timestamp from a string and use it with timezone.
select
parse_datetime('2022-09-10_00', 'yyyy-MM-dd_H'),
parse_datetime('2022-09-10_00', 'yyyy-MM-dd_H') AT TIME ZONE 'Europe/Berlin',
at_timezone(CAST(parse_datetime('2022-09-10_00', 'yyyy-MM-dd_HH') AS timestamp), 'Europe/Berlin') AS date_partition_berlin,
CAST(parse_datetime('2022-09-10_00', 'yyyy-MM-dd_HH') AT TIME ZONE 'Europe/Berlin' AS timestamp) AS date_partition_timestamp;
2022-09-10 00:00:00.000 UTC
2022-09-10 02:00:00.000 Europe/Berlin // time zone conversion + 2 hours
2022-09-10 02:00:00.000 Europe/Berlin // time zone conversion + 2 hours
2022-09-10 00:00:00.000
My data has the date in the "02JAN2020" format and I want to load the data using the COPY Command
copy test.Demographics from 's3://xyz-us-east-1/Blu/'
access_key_id ,’Access_Key_ID>’
secret_access_key ’<Secret_Access_Key>’
delimiter ',' dateformat 'auto'
GZIP;
The column data type is a date but it's still failing. I checked the stl error logs and it's the date formate issue.
I want the value in the column as 2020-01-02 and not 02Jan2020
Specify the date format with
DATEFORMAT 'DDMONYYYY'
I am looking to convert the following string: mmm-dd-yyyy to a date: yyyy-mm-dd
e.g
Nov-06-2015 to 2015-11-06
within Amazon Athena
I would do date_parse. Adjust your regex accordingly.
select date_parse('Nov-06-2015','%b-%d-%Y')
2015-11-06 00:00:00.000
refd:https://prestodb.io/docs/current/functions/datetime.html
You can also use cast function to get desire output as date type.
select cast(date_parse('Nov-06-2015','%M-%d-%Y') as date);
output--2015-11-06
in amazon athena
https://prestodb.io/docs/current/functions/datetime.html
used date parse to parse string and cast to convert 2015-11-06 00:00:00.000
into 2015-11-06
My current iot design is iot > rule > kinesis firehose > redshift
I have iot rule as
SELECT *, timestamp() AS timestamp FROM 'topic/#
I get json message something like below
{
"deviceID": "device6",
"timestamp": 1480926222159
}
In my redshift table I have a column eventtime as Timestamp
Now i want to store the json timestamp value to eventtime column, but it gives me error as it needs
TIMEFORMAT AS 'MM.DD.YYYY HH:MI:SS
for timestamp. So how to covert the iot rules timestamp to redshift timestamp?
There is no direct way to converting epoch date value while inserting it to Redshift table Timestamp datatype column.
I have created a column with Bigint datatype and inserting epoch value directly to this column.
After that I am using Quicksight for analytics so I can edit my dataset and create New calculated field for this column and use Qucksight function as below
epochDate(epoch_date)
which converts the epoch value to timestamp field.
One can use similar functions like
SELECT
(TIMESTAMP 'epoch' + myunixtimeclm * INTERVAL '1 Second ')
AS mytimestamp
FROM
example_table