I have configured an aws asg using ansible to provision new instances and then install the codedeploy agent via "user_data" script in a similar fashion as suggested in this question:
Can I use AWS code Deploy for pulling application code while autoscaling?
CodeDeploy works fine and I can install my application onto the asg once it has been created. When new instances are triggered in the ASG via one of my rules (e.g. high cpu usage), the codedeploy agent is installed correctly. The problem is, CodeDeploy does not install the application on these new instances. I suspect it is trying to run before the user_data script has finished. Has anyone else encountered this problem? Or know how to get CodeDeploy to automatically deploy the application to new instances which are spawned as part of the ASG?
AutoScaling tells CodeDeploy to start the deployment before the user data is started. To get around this CodeDeploy gives the instance up to an hour to start polling for commands for the first lifecycle event instead of 5 minutes.
Since you are having problems with automatic deployments but not manual ones and assuming that you didn't make any manual changes to your instances you forgot about, there is most likely a dependency specific to your deployment that's not available yet at the time the instance launches.
Try listing out all the things that your deployment needs to succeed and make sure that each of those is available before you install the host agent. If you can log onto the instance fast enough (before AutoScaling terminates the instance), you can try and grab the host agent logs and your application's logs to find out where the deployment is failing.
If you think the host agent is failing to install entirely, make sure you have Ruby2.0 installed. It should be there by default on AmazonLinux, but Ubuntu and RHEL need to have it installed as part of the user data before you can install the host agent. There is an installer log in /tmp that you can check for problems in the initial install (again you have to be quick to grab the log before the instance terminates).
Related
I want to be able to use AWS CodeCommit as a repo for my scripts, and then have AWS automatically deploy any new commits to a bunch of Raspberry Pi systems (on-premise instances which I've already set up in Systems Manager). Preferably, it would take a commit and install it on a single staging RPi first, test it, and if the tests go well, then install it on the rest of the fleet of RPi systems.
(The Raspberry Pi systems are running Ubuntu Server 20.04 LTS, so are all compatible as per the requirements of Systems Manager)
Is this possible with AWS? Are there any clear guides on how to do this?
The closest I've come to success was following this: https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-simple-codecommit.html, but that tutorial explains how to deploy from CodeCommit to an EC2 instance rather than to an on-premises instance. I tried switching to an on-premise instance instead of EC2 (in step 5), and specified the tags I've already assigned to my on-premises instance (in Systems Manager > Fleet Manager), but when I try to run the deployment, I get an error: "The deployment failed because no instances were found for your deployment group. Check your deployment group settings to make sure the tags for your Amazon EC2 instances or Auto Scaling groups correctly identify the instances you want to deploy to, and then try again." The tags are definitely correct so I don't know why that's failing.
Thanks in advance for any help.
Essentially, I had skipped a bunch of steps in the user guide without realising it. Going back to the start after a decent night's sleep helped.
PEBCAK is a thing.
So we are launching an ecommerce store built on magento. We are looking to deploy it on Amazon EC2 instance using RDS as database service and using amazon auto-scaling and elastic load balancer to scale the application when needed.
What I don't understand is this:
I have installed and configured my production magento enviorment on an EC2 instance (database is in RDS). This much is working fine. But now when I want to dynamically scale the number of instances
how will I deploy the code on the dynamically generated instances each time?
Will aws copy the whole instance assign it a new ip and spawn it as a
new instance or will I have to write some code to automate this
process?
Plus will it not be an overhead to pull code from git and deploy every time a new instance is spawned?
A detailed explanation or direction towards some resources on the topic will be greatly appreciated.
You do this in the AutoScalingGroup Launch Configuration. There is a UserData section in the LaunchConfiguration in CloudFormation where you would write a script that is ran when ever the ASG scales up and deploys a new instance.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-launchconfig.html#cfn-as-launchconfig-userdata
This is the same as the UserData section in an EC2 Instance. You can use LifeCycle hooks that will tell the ASG not to put the EC2 instance into load until everything you want to have configured it set up.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-as-lifecyclehook.html
I linked all CloudFormation pages, but you may be using some other CI/CD tool for deploying your infrastructure, but hopefully that gets you started.
To start, do check AWS CloudFormation. You will be creating templates to design how the infrastructure of your application works ~ infrastructure as code. With these templates in place, you can rollout an update to your infrastructure by pushing changes to your templates and/or to your application code.
In my current project, we have a github repository dedicated for these infrastructure templates and a separate repository for our application code. Create a pipeline for creating AWS resources that would rollout an updated to AWS every time you push to the repository on a specific branch.
Create an infrastructure pipeline
have your first stage of the pipeline to trigger build whenever there's code changes to your infrastructure templates. See AWS CodePipeline and also see AWS CodeBuild. These aren't the only AWS resources you'll be needing but those are probably the main ones, of course aside from this being done in cloudformation template as mentioned earlier.
how will I deploy the code on the dynamically generated instances each time?
Check how containers work, it would be better and will greatly supplement on your learning on how launching new version of application work. To begin, see docker, but feel free to check any resources at your disposal
Continuation with my current project: We do have a separate pipeline dedicated for our application, but will also get triggered after our infrastructure pipeline update. Our application pipeline is designed to build a new version of our application via AWS Codebuild, this will create an image that will become a container ~ from the docker documentation.
we have two triggers or two sources that will trigger an update rollout to our application pipeline, one is when there's changes to infrastructure pipeline and it successfully built and second when there's code changes on our github repository connected via AWS CodeBuild.
Check AWS AutoScaling , this areas covers the dynamic launching of new instances, shutting down instances when needed, replacing unhealthy instances when needed. See also AWS CloudWatch, you can design criteria with it to trigger scaling down/up and/or in/out.
Will aws copy the whole instance assign it a new ip and spawn it as a new instance or will I have to write some code to automate this process?
See AWS ElasticLoadBalancing and also check out more on AWS AutoScaling. On the automation process, if ever you'll push through with CloudFormation, instance and/or containers(depending on your design) will be managed gracefully.
Plus will it not be an overhead to pull code from git and deploy every time a new instance is spawned?
As mentioned, earlier having a pipeline for rolling out new versions of your application via CodeBuild, this will create an image with the new code changes and when everything is ready, it will be deployed ~ becomes a container. The old EC2 instance or the old container( depending on how you want your application be deployed) will be gracefully shut down after a new version of your application is up and running. This will give you zero downtime.
I have an environment in AWS where EC2 instances are in autoscaling mode, i.e. new instances spin up as per the load on deployed instances.
Now, if I want to integrate this environment with Jenkins, how can I push my codes from Github to these EC2 instances, where my application is deployed. And with every change in my code version, Github should invoke EC2 instances to have the same versions deployed, and also every new instances should be created with this updated version of code, i.e. every autoscaled instances must have the same code version running. Please help.
I assume you have an executable version of your latest code on a deploy server. You can do this by forcing Jenkins to deploy your code when a new commit is made on a specific branch in GitHub. Then all you need is an AMI for your Auto Scaling Group that has a job/task which runs let's say every 5 minutes (based on how long one single task takes). This job/task fetches (copies) the code from the deploy server and then starts the application. As an example, in Windows Task Scheduler you can add two actions to a task: one for updating (e.g. a simple robocopy) the code and one for running the app.
I have custom chef 12.2 script to run the deployment on my opsworks, deployment and running recipe used work great on the instances (with Windows Custom AMI - OS server 2012).
Post migration of instances to a Domain. nothing seems to work.
opsworks agent is running on the instances. not sure what else to look at to solve the issue.
any suggestions how i can investigate the issue and solve it?
**note Reboot from opsworks reboots the instance
whenever opsworks runs for a long time use
opsworks-agent-cli show_log
command to get the current activity logs of opsworks agent. if its stuck inside some loop forever, you can get its pid and kill it abruptly.
I'm trying to setup an Auto Scaling Group in combination with CodeDeploy. Everything works fine except for the fact that when a new instance is created CodeDeploy starts before the user data script (defined in the Launch Configuration) finishes.
The default value of this user data script downloads and install the code deploy agent and i've extended it with installation of a couple of windows features, IIS rewrite module and msdeploy.
In my appspec.yml i'm using the hook AfterInstall to deploy my IIS website and this obviously fails when msdeploy is not installed (yet).
Am i going about this the wrong way or is there a way to make CodeDeploy wait for the user data script to finish?
Unfortunately, there's no was for CodeDeploy to know anything more than the instance has loaded it's OS. The good thing is that CodeDeploy give the host agent 1 hour to start polling for commands with automatic deployments. The easiest thing to do is install the host agent after all the required dependencies are installed. The automatic deployment will be created, but can't proceed until after the host agent is started.
This is explained in detail here - https://aws.amazon.com/blogs/devops/under-the-hood-aws-codedeploy-and-auto-scaling-integration/
Ordering execution of launch scripts – The CodeDeploy agent looks for and executes deployments as soon as it starts. There is no ordering between the deployment execution and launch scripts such as user data, cfn-init, etc. We recommend you install the host agent as part of (and maybe as the last step in) the launch scripts so that you can be sure the deployment won’t be executed until the instance has installed dependencies that are not part of your CodeDeploy deployment. If you prefer baking the agent into the base AMI, we recommend that you keep the agent service in a stopped state and use the launch scripts to start the agent service.