I'm trying to set up Redshift scheduling using CRON for running it 5 days a week (MON-FRI) between 7-14 ETC. The format is described as such:
Cron format: minute hour day-of-month month day-of-week year
Use , (comma) to include additional values, - (hyphen) to specify a range, * (asterisk) to include all values, and / (forward slash) to specify increments.
So I use these values:
Pause - 0 16 * * 1-5 *
Start - 0 9 * * 1-5 *
AWS shows me an error Invalid cron schedule expression: cron(0 9 * * 1-5 *). If I substitute 1-5 by ?, it works, but that's not what I need.
Is the range indeed supported by this scheduler and I'm doing something wrong?
The third asterisk has to be a question mark when specifying the day of the week like this: 0 16 ? * 1-5 *
The doc says:
The * (asterisk) wildcard includes all values in the field. In the
Hours field, * would include every hour. You cannot use * in both the
Day-of-month and Day-of-week fields. If you use it in one, you must
use ? in the other.
Related
I need to schedule a query on BigQuery, that will retrieve some data from 24h ago. The problem is that for example if its scheduled for 5am, the query needs time to be executed (let's say 2.248 seconds), and so the yesterday's data from 5:00:00.000 to 5:00:02.248 will not be retrieved. I'm using a timestamp field, and i do something like this :
SELECT *
FROM my_table
WHERE my_timestamp >= TIMESTAMP_SUB(current_timestamp(), INTERVAL 24 hour)
I would like to know if there is a way to get the yesterday's date, at a specific hour, so even if there is a little gap due to the execution, it will still retrieve data from a specific hour.
edit:
I found something :
SELECT *
FROM my_table
WHERE my_timestamp >= TIMESTAMP(DATE_SUB(current_date(), INTERVAL 1 DAY) )
AND my_timestamp < TIMESTAMP(current_date)
But this retrieves yesterday between 00:00:00.000 to 23:59:59.999
It is okay but is there a way to choose the hour ?
Consider below least verbose approach
select *
from my_table
where my_timestamp >= timestamp(current_date - 1 + interval 5 hour)
and my_timestamp < timestamp(current_date + interval 5 hour)
Okay I found on my own lol, maybe it will help someone else looking for this.
SELECT *
FROM my_table
WHERE my_timestamp >= TIMESTAMP_ADD(TIMESTAMP(DATE_SUB(current_date, INTERVAL 1 DAY)), INTERVAL 5 HOUR)
AND my_timestamp < TIMESTAMP_ADD(TIMESTAMP(current_date), INTERVAL 5 HOUR)
I am looking into creating a list, where I see how many IDs were generated in a given hour. Because I want to display the hours from 8 to 21 in advance independent of the hour of the date, I used generate series and now need to find the equivalent for Snowfake. This is my query:
series as (
SELECT seq4() as Hour
FROM TABLE(GENERATOR(rowcount => 21))
where Hour between 7 and 20
ORDER BY Hour),
ID_table as (
select extract(hour from date) as "Hour",
count(ID) as "Count"
from ID_table
group by 1)
select (Hour.Hour) + 1 AS "Hour",
id."Count",
from series as Hour
left join ID_table as id on id."Hour" = Hour.Hour
order by Hour.Hour;
For some reason I only get the Hours 8 to 16, however, I want it to display the hours 8-21, what could be the issue?
You should always consider that SEQ() functions do not guarantee gaps, so for generating the range, I suggest you to use ROW_NUMBER() function:
https://community.snowflake.com/s/article/Generate-gap-free-sequences-of-numbers-and-dates
Anyway, when I test it, I see it returns expected numbers:
SELECT seq4() as Hour FROM TABLE(GENERATOR(rowcount => 21));
-- returnns numbers from 0 to 20
SELECT seq4() as Hour FROM TABLE(GENERATOR(rowcount => 21)) where Hour between 7 and 20 order by hour;
-- returnns numbers from 7 to 20
with series as (SELECT seq4() as Hour FROM TABLE(GENERATOR(rowcount => 21)) where Hour between 7 and 20 ORDER BY Hour)
select (Hour.Hour) + 1 AS "Hour"
from series as Hour;
-- returnns numbers from 8 to 21
Could it be something with the browser/UI?
I want to create a scheduled expression, which will start js script every week at 5:30PM PT using CloudWatch Events. I tried to insert:
- 0 30 3 * * THU *
- 0 30 3 THU
But got
Error
There was an error while saving rule getEvery1min.
Details: Parameter ScheduleExpression is not valid..
Can you help me pls?
According the docs - Schedule Expressions for Rules
Only this is work - 30 17 ? * THU *
You can't specify the Day-of-month and Day-of-week fields in the same cron expression. If you specify a value (or a *) in one of the fields, you must use a ? (question mark) in the other.
The cron expression should be in the below format
"Minutes" "Hours" "Day of month" "Month" "Day of week" "Year"
Based on the above cron it appears you're trying to run the cron expression only on Thursdays on the third day of the month. However if you want it to run once a week at 5:30pm you'd want something like the below.
30 17 * * THU *
Also take care to remember this expression is in UTC, so you would need to adjust it for your timezone.
For more information take a look at the Scheduled Events documentation.
I need to obtain the time difference in minutes between a row and the one that follows. I would like to use the field DATE to calculate and maybe create an additional field with the new measurement.
This is the structure of my fields in my database.
You should be able to use the TRANSACTION number to look up the prior row.
Something like this as a calculated column:
Diff = 24 * 60 *
(Table1[DATE] -
LOOKUPVALUE(
Table1[DATE],
Table1[TRANSACTION],
Table1[TRANSACTION] - 1
)
)
Note: The difference has units of days, so the 24 (hours/day) * 60 (minutes/hour) is the conversion you need.
I have values in my db as follows
1
2
2A
.......
10
10A
.......
250A
......
300
and i need to find the records that are within range 1-10,11-20,....291-300
can't figure out how to do this as there are characters after number
I'm using postgres db so i can apply regex query on db
You can use:
SUBSTRING(mycolumn FROM '^[0-9]+')
in order to extract the number that is at the start of your column.
To perform the filtering you want, you can use:
SELECT *
FROM mytable
WHERE CAST(SUBSTRING(mycolumn FROM '^[0-9]+') AS INT) BETWEEN 1 AND 10
Demo here