I have an Autoscaling Group that gets scaled/deployed on using CodeDeploy, I had created a working instance and got it all up and running and took an image of it for a launch template, this instance was attached to the Autoscaling group and when I deployed it worked perfectly fine.
However, over the weekend, AWS decided to tear that instance down and create a whole new one, and when it did, the deployment failed because of the error File already exists. I understand what the error is, but I don't understand why it occurred? That deployment had already occurred on that image, so it shouldn't run into that issue....
Anyways, I know when you manually kick off a deployment, you can opt to Overwrite existing files to avoid this error. But I haven't been able to find this setting for when Autoscaling automatically kicks off deployments. How can you allow duplicate files to be overwritten by CodeDeploy when the deployment is kicked off automatically by your Auto Scaling group?
Make sure you have specified overwrite in your appspec.yml file
file_exists_behavior: DISALLOW|OVERWRITE|RETAIN
AWS Docs
Related
When I try to run codedeploy, I get the following error :
The overall deployment failed because too many individual instances failed deployment, too few healthy instances are available for deployment, or some instances in your deployment group are experiencing problems.
When I look at the ec2 instances created by the auto scalable group, they are both running, and status is passed 2/2 checks passed in green. I am wondering if this is just a catch all error, because its supposed to throw this error if one or both or all are not running.
It seems it wanted an appspec.yml. I am not sure why a appspec.yml is needed, If you create the autodeploy group, and create the codedeploy application, why is there a need for a appspec.yml. I am using jenkins to run the code deploy. I gave it a appspec.yml that is empty, and seems to work.
I have used Cloudformation template to create my EC2 instance. In this instance there is a file
in home directory that I copy from S3 while creating stack.
I have that file stored locally as well. Now, I modify that file locally and want to copy it to S3
and from S3 to EC2 instance.
I want to automate this process through Cloudformation. So that, whenever I modify this file locally,
after doing update stack, it uploads the modified file to S3 and from S3 to my EC2 instance.
Can anyone please help how this can be achieved?
One thing that comes to mind (bearing in mind the application specific nature of what you are trying to do) is using ECS instead of just EC2.
Note: This may be overkill, but it would work. Also if updates were extremely frequent this would be a major pain so just uploading the file to S3 with a script alongside update-stack (if the update-stack is even necessary) and then polling for changes to that S3 file in your EC2 application would be fine.
Anyway, this is a pattern we use when we are doing something like training a model with new data, which we then wish to deploy to AWS, replacing an application with an older version of the model.
You build the Docker image locally and your special file gets included inside the container. You push the Docker image to DockerHub or AWS ECS Registry or wherever. You update the cloudformation template ECS configuration to use the tag of this new Docker image and update the stack. Then ECS pulls this new image and the new Docker container(s) take the place of the old one(s) and it will have your special file inside it.
I know of at least one way: set up your EC2 in an AutoScaling Group (ASG). Then, on creation, use cfn-init on your UserData and have it fetch the file under sources. Use both Creation and Update policies. On update, have the WillReplace attribute set to true under AutoScalingReplacingUpdate. When you update, CloudFormation will create a new ASG with your fresh new file copy. If you signal success on your update, it will remove the previous instance and ASG, giving you an immutable infrastructure setup. Put everything behind a load balancer, and you got yourself a highly available blue/green deployment too.
I have a CD work flow using CodePipeline + CodeCommit + CodeDeploy to EC2 instances.
I have everything else working except for this --file-exists-behavior option. If I create a deployment from the Aws CLI or the console. The deployment works fine.
Is there anyway I can use this option with CodePipeline?
I am still getting an error without this option.
The deployment failed because a specified file already exists at this location
So, when you deploy, CodeDeploy will look for a file (on the instance) with information on previously deployed files for the application/deployment-group in question, if it finds the file it then it uses it to cleanup (delete) the existing files, preparing for copying of the new revision files.
Refer This AWS Documentation
But if the cleanup file doesn't exist, either because the existing code was initially deployed in some other way (eg cloudformation instance userdata/metadata sources) or the CodeDeploy deployment-group has been recreated for some reason then the cleanup file will either not exist or not match the deployment-group id. Therefore it wont be run. Therefore the files will not be cleaned up, and the deployment will fail since CodeDeploy does not overwrite files.
What a Solution could be in this scenario is to have a hook (eg BeforeInstall) that calls a script to delete the applications files (if they exist) on the instance.
Hope this helps
I am trying to set AWS Code deploy with Github to automate my deloyment. the problem i am having is the ec2 instance is already active and has the web app code. the problem is there is also other content on the instance that I don't want under source control.
Code deploy is failing with the following error:
The deployment failed because a specified file already exists at this location
this is because the source code is on the instance and was not added as a revision.
So my question is there any way to make the code deploy recognise the current files on the instance as an inital revision?
This is possible! This is an option during a deployment: "Content options:
Choose the action for AWS CodeDeploy to take during a deployment when a file on a target instance has the same name as a file in the application revision, for the same target location."
You can choose fail, overwrite and retain. Retain is probably the best choice in your case.
You can find more information
docs.aws.amazon.com (strangely under 'rollback')
cli docs.aws.amazon.com (see --file-exists-behavior (string))
This is my sample appspec.yml
version: 0.0
os: linux
files:
- source: /
destination: /var/www/wordpress/
overwrite: true
file_exists_behavior: OVERWRITE
And this is Updated AWS User Guide for AppSpec 'files' section (EC2/On-Premises deployments only). In there you will find three options, DISALLOW,OVERWRITE and RETAIN.
You can simply delete the entire contents of your web root and allow CodeDeploy to update it. Any type of deployment will replace the entire contents.
If you're feeling unsure then copy the webroot to another folder, and in the very unlikely event of a failure you can copy it all back.
Deployments often fail when a file has been modified on the disk, I assume CodeDeploy is checking the modified time of the file.
If you just want the error to go away, you can use one of the earlier Lifecycle Hook scripts (BeforeInstall maybe) to delete that folder before the source gets re-added in a later Hook. If the source doesn't get re-added in a later Hook, add it yourself.
I was getting this error even on new instances, so CodeDeploy must be caching that folder somehow, then getting upset that it did so.
The issue is that CodeDeploy maintains a cleanup file that it updates after every deployment. These are the files it will delete before installing the new ones. You could of course create this yourself, but that's hard. The point of CodeDeploy is to make things easy. This is what I did (Note: My app has a lot of files and I was about to find something other than CodeDeploy that actually made transitioning to it easy instead of painful and stupid, and then I formulated this approach; if you have a small number of files, it might be easier to create a BeforeInstall script or delete them manually):
create a free-tier EC2 instance (t2.micro). If your app is giant, you may need more disk space, so adjust as necessary. THIS WON'T BE RUNNING YOUR APP!
Create a new deployment group pointing to that new instance.
Create a new YAML that doesn't actually do any BeforeInstall or AfterInstall tasks. You are not trying to run your app. You are literally moving files over. This is so that you can grab that cleanup file.
After deployment, ssh into the ec2 instance (remember to allow SSH in the Security Group!) and go to /opt/codedeploy-agent/deployment-root/deployment-instructions/
You will see {deployment-group-identifier}-cleanup. scp this to the ec2 instance on which you actually want to run your app. Or, FileZilla it out onto your main machine and then onto the main ec2 instance. That was the easiest choice for me, because my local machine already had all the right keys.
SSH into the main ec2 instance and move the cleanup file into the /opt/codedeploy-agent/deployment-root/deployment-instructions/ if it isn't already there. Note the failed {deployment-group-identifier} and replace the cleanup file's {deployment-group-identifier} with the failed identifier.
Rerun the deployment.
8.???
Profit.
I will provide more in-depth instructions for each of these steps (except for 8 and 9... which will go away) when I have time.
I had same issue when working in CodeDeploy with EC2 Instances.
I removed EC2 Instances from AWS CodeDeploy Deployment Group. I added new EC2 instances to the deployment group.
Otherwise, Replace new deployment group with an existing deployment group.
I have configured an aws asg using ansible to provision new instances and then install the codedeploy agent via "user_data" script in a similar fashion as suggested in this question:
Can I use AWS code Deploy for pulling application code while autoscaling?
CodeDeploy works fine and I can install my application onto the asg once it has been created. When new instances are triggered in the ASG via one of my rules (e.g. high cpu usage), the codedeploy agent is installed correctly. The problem is, CodeDeploy does not install the application on these new instances. I suspect it is trying to run before the user_data script has finished. Has anyone else encountered this problem? Or know how to get CodeDeploy to automatically deploy the application to new instances which are spawned as part of the ASG?
AutoScaling tells CodeDeploy to start the deployment before the user data is started. To get around this CodeDeploy gives the instance up to an hour to start polling for commands for the first lifecycle event instead of 5 minutes.
Since you are having problems with automatic deployments but not manual ones and assuming that you didn't make any manual changes to your instances you forgot about, there is most likely a dependency specific to your deployment that's not available yet at the time the instance launches.
Try listing out all the things that your deployment needs to succeed and make sure that each of those is available before you install the host agent. If you can log onto the instance fast enough (before AutoScaling terminates the instance), you can try and grab the host agent logs and your application's logs to find out where the deployment is failing.
If you think the host agent is failing to install entirely, make sure you have Ruby2.0 installed. It should be there by default on AmazonLinux, but Ubuntu and RHEL need to have it installed as part of the user data before you can install the host agent. There is an installer log in /tmp that you can check for problems in the initial install (again you have to be quick to grab the log before the instance terminates).