I recently tried to set up a warm pool for an ASG to reduce the launch time of scaling out. But it turns out the instance launched from the warm pool got stuck at the 'pending:wait' status and did not finish the deployment which should be done by CodeDeploy. At the same time, if an instance is launched from the ASG directly, it works well.
After contacting AWS support, I added a aws autoscaling complete-lifecycle-action command in the last line of the userdata to manually complete the CodeDeploy managed lifecycle hook. Also, I creted a Lambda Function triggered by a EventBridge rule that send aws autoscaling complete-lifecycle-action request to forcely make instances go into "inservice" status after launched from the warm pool. However, this method only makes the instance be in the "inservice" status but the deploymemt actually still not finished. The CodeDeploy activity history shows that the deployment is stuck and no Event is processed. When I remoted into the instance to have a look, I found the codedeploy agent is not even installed.
AWS support admits that the warm pool does not offically support CodeDeploy. However, he also confirms that these two services should work well with some extra works. Does any have a thought about this? Thanks in advance.
Related
I have AWS CodeDeploy deploying to a Deployment Group that targets an AutoScalingGroup of EC2 instances that can have between min and max number of instances.
CodeDeploy hooks can be specified on individual instances to launch scripts on those instances at various stages of the deployment process.
Is there a way to launch a script, Lambda function, etc... after CodeDeploy successfully finishes deploying to the final instance in the ASG? In other words, is there an "All Done With Everything" hook that I can use? How are others tackling and solving this problem?
If you're using codepipeline, how about adding another stage after code deploy?
Or you can also trigger SNS topic with AWS CodeDeploy about deployment status as well.
Here: https://docs.aws.amazon.com/codedeploy/latest/userguide/monitoring-cloudwatch-events.html
I need to set up a monitoring into an aws account to ping certain servers from outside the account, create a custom cloudwatch metric with the package loss and i need to deploy the solution without any EC2 instance.
My first choice was lambda, but it seems that lambda does not allow pinging from it.
Second choice was a container, as FARGATE has the ability to execute containers without any EC2 instance. The thing is im able to run the task definition and i see the task in RUNNNING state in the cluster, but the cloudwatch metric is never received.
If I use the normal EC2 cluster, the container works perfectly, so i assume I have some error within the configuration, but I'm lost why. I have added admin rights to the ECS Task Execution Role and opened all ports in the sec group.
I have tried public/private subnets with no success.
Anyone could please help me?
Here you can find that the task is certainly RUNNING, however the app dont generate any further action
So i solved the problem. There was problem inside the container. It seems Fargate doesn't like cron, so i removed my cron schedule from the container and used a cloudwatch event rule instead and it works perfectly
I have created a pipeline using AWS Codepipeline, Github, Jenkins and AWS Elastic Beanstalk (Docker) running a nodejs application. Everytime a build is triggered in AWS Codepipeline and deployment done on the Elastic Beanstalk instance, it's corresponding EC2 instance is terminated and another one created afresh and we only want the app to be deployed without termination of EC2 instance. What could be the cause for termination on every build/deployed?
how many instances do you have in your beanstalk and what deployment method are you using: All at Once, Rolling, Rolling with an Additional Batch or Immutable?
With these responses, we can continue the research.
I switched to Immutable deployment and stopped experiencing the issue as explained here: Difference between rolling, rolling with additional batch and immutable deployments in AWS?
Turns out that Rolling deployments can cause the timeouts especially that I had a single instance needed
I'm new at AWS.
Before 30 minutes, I launch ecs to deploy my docker container.
Everything looks fine.
After finishing my work, I deleted cluster, task definition.
But in my ec2 console, ec2 launch every 2 minutes inifinitly.
I deleted every resource about it.
Why it launch automatically?
Is there any solution about cleaning aws ecs configuration?
Thanks.
As per your confirmation, Recreation of the associated autoscaling group which was responsible to spin up instances solved your problem.
I have a Docker image containing Python code and third-party binary executables. There are only outbound network requests. The image must run hourly and each execution lasts ~3 minutes.
I can:
Use an EC2 instance and schedule hourly execution via cron
Create a CloudWatch Event/Rule to run an ECS Task Defintion hourly
Setup an Elastic Beanstalk environment and schedule hourly deployment of the image
In all of these scenarios, an EC2 instance is running 24/7 and I am being charged for extended periods of no usage.
How do I accomplish scheduling the starting of an existing EC2 instance hourly and the stopping of said instance after the completion of my docker image?
Here's one approach I can think of. It's very high-level, and omits some details, but conceptually it would work just fine. You'll also need to consider the Identity & Access Management (IAM) Roles used:
CloudWatch Event Rule to trigger the Step Function
AWS Step Function to trigger the Lambda function
AWS Lambda function to start up EC2 instances
EC2 instance polling the Step Functions service for Activity Tasks
Create a CloudWatch Event Rule to schedule a periodic task, using a cron expression
The Target of the CloudWatch Event Rule is an AWS Step Function
The AWS Step Function State Machine starts by triggering an AWS Lambda function, which starts the EC2 instance
The next step in the Step Functions State Machine invokes an Activity Task, representing the Docker container that needs to execute
The EC2 instance has a script running on it, which polls the Activity Task for work
The EC2 instance executes the Docker container, waits for it to finish, and sends a completion message to the Step Functions Activity Task
The script running on the EC2 instance shuts itself down
The AWS Step Function ends
Keep in mind that a potentially better option would be to spin up a new EC2 instance every hour, instead of simply starting and stopping the same instance. Although you might get better startup performance by starting an existing instance vs. launching a new instance, you'll also have to spend time to maintain the EC2 instance like a pet: fix issues if they crop up, or patch the operating system periodically. In today's world, it's a commonly accepted practice that infrastructure should be disposable. After all, you've already packaged up your application into a Docker container, so you most likely don't have overly specific expectations around which host that container is actually being executed on.
Another option would be to use AWS Fargate, which is designed to run Docker containers, without worrying about spinning up and managing container infrastructure.
AWS Step Functions
AWS Fargate
Blog: AWS Fargate: An Overview
Creating a CloudWatch Event Rule that triggers on a schedule