I need a scheduled query only between Monday to Friday between 9 and 7 o'clock:
Scheduled queries is currently: every hour from 9:00 to 19:00
But how to modify for Mo-Fr ?
every monday to friday from 9:00 to 19:00 not working
every monday from 9:00 to 19:00 working (so day of the week is in general not working ?)
Thanks
UPDATE: The question at hand is much more complex than the Custom setting in BigQuery Scheduled Queries allows. For this purpose, #guillaume blaquiere has the best suggestion: use Cloud Scheduler to run a cron job. Tools like Crontab Guru can be helpful in creating a statement such as 00 9-19 * * 1-5.
For simpler Scheduled Queries, please review the following from the official documentation: Set up scheduled queries.
Specifically,
To specify a custom frequency, select Custom, then enter a Cron-like
time specification in the Custom schedule field; for example every 3
hours.
There is excellent documentation in the Custom Interval tab here on the many options you have available in this field.
thanks for the Feedback. So like this one ? But this is not working
Related
I have a query and I wanted to schedule it as quarterly basis.
For example, first quarter of calendar is Jan 2022 - Mar 2022 and so I wanted to run this query on 01 Mar 2022. Is there anyway we can do that ?
Any help will be highly appreciated.
Many thanks.
BigQuery uses the cron sintax of AppEngine (see docs) for custom schedules, so if want run a query every "1 Mar", you can set the "Repeats" option to "Custom" and the "Custom Schedule" as below:
1 of mar 00:00
To schedule multiple months, you can use a comma-separated list:
1 of mar,jun,sep,dec 00:00
Option1: Using some programmatic api like python/go/java for bigquery to call your query and schedule through some cron on backend/GCE box.
Option2: If you do not want to use some programatic api to do it on some cron job: Bigquery native UI now does support scheduled queries. You can schedule your query through it. Screenshot below:
Option3: If you have many of such queries which you want to run on some cadence you can either go: Airflow OR even better option: https://potens.io/products/#magnus
I am working on a NIFI project which I want to get the next business day.
For example if I input Jan-14-2022, I can get next businessday is Jan-18-2022 since 14th, 15th is weekends and 17th is holiday.
I currently have
:toDate('MMM-dd-yyyy'):toNumber():plus(86400000):format('MMM-dd-yyyy')
which will add 24 hours and return me the date one day after.
But how Can I get the next business day?
This sounds like something that a NiFi Processor could be configured to do using something such as a script or another option inside a processor.
However, this is something that would better be done with code, not NiFI as this isnt something that NiFi can natively do. I am not aware if this is something natively something NiFi should and can do.
I want to configure a job that runs at dawn on the first Saturday of every month through Cloud Scheduler.
Considering the Scheduler Job Frequency setting to be the first Saturday of every month, I have designated it as follows.
ex) 45 2 1-7 * 6
However, it was confirmed that the above scheduler was running on the 23rd, last Saturday.
Is it not possible to configure a monthly schedule in Cloud Scheduler?
If you could give me an answer, I would be very grateful.
I have checked these links in relation to the above.
Your current schedule, 45 2 1-7 * 6, reads as At 02:45 on every day-of-month from 1 through 7 and on Saturday. You can check it on Crontab guru.
In order to set a custom interval, you will need to use the App Engine Cron format.
In this case, try first saturday of month 02:45.
Currently 24 hours is the limit of time a Cloud Monitoring (erstwhile Stackdriver) alert policy can be set.
However, if you have a daily activity, like a database backup, it might take a little more or less time each day (e.g. run in 1 hour 10min one day, 1 hour 12min the next day). In this case, you might not see your completion indicator until 24 hours and 2 minutes since the prior indicator. This will cause Cloud Monitoring to issue an alert (because you are +2min over the alerting window limit).
Is there a way to better handle the variance in these alerts, like a 25 hour look back period?
Currently, there is no way to increase the period time over 24 hours.
However, there is a Feature Request already opened for that.
You can follow it in this public link [1].
Cheers,
[1] https://issuetracker.google.com/175703606
I found a work around to this problem.
Create a metric for when your job starts (e.g. started_metric)
Create a metric for when your job finishes (e.g. completed_metric)
Now create a two part Alert Policy
Require that started_metric occurs once per 24 hours
Require that completed_metric occurs once per 24 hours
Trigger if (1) and (2) above are met (e.g. both > 24 hours)
This works around the 24 hour job jitter issue, as the job might take > 24 hours to complete, but it should always start (e.g. cron job) within 24 hours.
I would like to order a job today in Control-M but have it run 8 days later, how would one go about doing this? Tomorrow the job should be ordered again for tomorrow's date and ran 8 days from tomorrow, so on and so forth.
If you want run a job after 8days if it is not schedule on that date then you can order out that job on 7thdays depending up on time . Or if the request is that it should run on permantely on calendar days then you can add schedule and move job to production so that it get automatically order out and run.