I am trying to deploy a triggered webjob and schedule it with settings.json file. My webapp runs in 3 instances so I assume the webjob will be deployed in 3 instances.
So if I schedule the webjob with settings.json file with cron expression will the job run in all the instances? I am assuming that setting the "is_singleton" to true will make it run only on one instance? But if the instance in which it is running fails, will Azure failover the webjob in spite of the is_singleton setting?
You shouldn't need to use is_singleton in this case. For each schedule occurence, the runtime will pick an instance at random and run your job. Each schedule occurrence will run on one instance only, as you would expect.
If a schedule occurrence is missed or fails, the runtime will try to get back on schedule by running the function again. You can see that logic in the source here.
Related
So i have two steps in Octopus Deploy. first step will deploy the cluster and task definition and the other will update the task definition and deploys a new task. The First step is only needed for the first time.. how do i skip the step with run conditions.
EDIT: Updated to better reflect the way the Octopus Deploy an ECS Service and Update an ECS Service steps work.
Typically, I wouldn't expect to see both the Deploy an ECS service step and the Update an ECS Service step in the same Octopus deployment or runbook process.
The reason being is that the Deploy an ECS service step creates a service and task for you from scratch in a cloud formation stack that Octopus then manages - it helps you get started with ECS if you haven't got something running in there already.
Conversely, you'd tend to use the update an ECS service step when the service and task already exist, and you want to deploy new versions of the task (new container images) over time. The cluster may be managed with a tool like terraform or by hand.
You'd, therefore, only need to use one or the other step, not both, as the deploy an ECS service step will update a previously deployed (by Octopus) ECS service if it exists.
If you really need to use both, then one option is to use a new step, that would be placed to run before the "first" and "second" step from your scenario.
This new step would likely be a run an AWS CLI script step. With it, you can use one of the ecs commands, probably list-services, to see if the service with the specified name already exists in the cluster.
When you know if the service exists or not, the step would then make use of an octopus output variable. You set it (assuming PowerShell script step) like so:
# Set to true if it exists
Set-OctopusVariable -name "ECSServiceExists" -value "True"
# OR False if it doesnt
Set-OctopusVariable -name "ECSServiceExists" -value "False"
Then for your "first" step that creates the service, you'd set a variable run condition to something like this:
#{unless Octopus.Deployment.Error}#{RunIfServiceDoesntExist}#{/unless}
The condition is doing two things:
Only running when there isn't a general error in the deployment (indicated by Octopus.Deployment.Error
Testing a new project variable called RunIfServiceDoesntExist to see if that evaluates to true.
For the condition to work, you need to create a new RunIfServiceDoesntExist project variable with value:
#{if Octopus.Action[Check for ECS Service].Output.ECSServiceExists == "False"}true#{/if}
Note: Replace Check for ECS Service with the name you give the new step to test for the ecs service's existence.
You can have the if condition above directly in the run condition itself, but I find the project variable hides away the complexity a little and makes it easier to read.
Hope that helps!
turns out i can update the existing service using Deploy AWS ECS Service steps itself.Just takes a really long time to deploy.
cheers....!
I want to run ECS task only once. I have created a ECS service which continuously runs a task.
Like if I check in the ec2 instance after first container gets executed it automatically starts another container. I don't want another container to be executed after first container execution I will close the task.
Note: I haven't created any scheduler.
Is there any way to handle this?
ECS services are for constantly running tasks that need to be replaced if they fail or exit for some reason. ECS scheduled tasks are much more like what you need.
If you want to run a task once in ECS, you can use the AWS CLI. Or check out this tool that simplifies the process a bit.
I have scheduled 2 cronjobs for my application.
My Application server is in an autoscaling group and I kept a minimum of 2 instances because of High availability. Everything working is fine but cron job is running multiple times because of 2 instances in autoscaling.
I could not limit the instance size to 1 because already my application in the production environment I prefer to have HA.
How should I have to limit execute cron job on a single instance? or should i have to use other services like AWS Lamda or AWS ELasticBeanstalk
Firstly you should consider whether running the crons on these instances is suitable. If you're trying to keep this highly available and it is directly interacted via customers what will the impact of the crons performance be?
Perhaps consider using a separate autoscaling group or instance with a total of 1 instances to run these crons? You could launch the instance or update the autoscaling group just before the cron needs to run and then automate the shutdown after it has completed.
Otherwise you would need to consider using a locking mechanism for your script. By using this your script write a lock to confirm that it is in process, at the beginning of the script run it would check whether there was any script lock in progress. To further prevent the chance of a collision between multiple servers consider adding jitter (random seconds of sleep) to the start of your script.
Suitable technologies for writing a lock are below:
DynamoDB using strongly consistent reads.
EFS for a Linux application, or FSX for a Windows application.
S3 using strong consistency.
Solutions suggested by Chris Williams sound reasonable if using lambda function is not an option.
One way to simulate cron job is by using CloudWatch Events (now known as EventBridge) in conjunction with AWS Lambda.
First you need to write a Lambda function with the code that needs to be executed on a schedule. Lambda supports cron expressions.
You can then use Schedule Expressions with EventBridge/CloudWatch Event in the same way as a cron tab and mention the Lambda function as target.
you can enable termination protection on of the instance. Attach necessary role & permission for system manager. once the instance is available under managed instance under system manager you can create a schedule event in cloudwatch to run ssm documents. if you are running a bash script convert that to ssm document and set this doc as targate. or you can use shellscript document for running commands
I am having a webjob which needs to run at regular intervals but having it as continuous with cron expression rather than triggered type.
Will it make any difference especially in terms of any performance?
Any help in understanding this very much appreciated
Continuous webjob for a schedule run: It will starts immediately when the WebJob is created and run on the schedule.
Triggered webjob for a scheduled run: Starts only when triggered on a schedule.
If your app runs continuous or scheduled WebJobs, enable Always On to ensure that the WebJobs run reliably. This feature is available only in the Basic, Standard, and Premium pricing tiers.
I want to schedule my docker image to run on a specific time on every day or if the size of a particular folder in my Amazon S3 reached a threshold size? Is it possible? (Any of these case)
There's no build in scheduler from AWS to do this.
Anyway,
you can either run a cronjob on a different machine and use the API to start a task on ECS (http://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_RunTask.html)
or
you can create a Lambda function that runs your task (http://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html)