I am deploying via CodeDeploy a logstash configuration in a target instance in /etc/logstash/.
This is the last step in a 3-staged CodePipeline.
However it fails with message
The deployment failed because a specified file already exists at this location: /etc/logstash/pipelines.yml
How can I instruct CodeDeploy to overwrite files?
March 2022 Update: Here are the docs on how to handle the file-exists-behavior: AppSpec 'files' Section.
July 2021 Update: Looks like AWS has finally acknowledged this defect -- look here for updates on all that.
August 2020 answer:
The only way I have seen that you can overwrite files is selecting to overwrite or retain, as seen here, when you're creating a new deployment. In no other place will you see those settings. And if I'm not mistaken, this is the only way to do it because it is not supported on the CLI.
An issue to add an overwrite option is open on GitHub and it has been there for quite a while. I solved this by adding a custom script in the BeforeInstallation hook which will erase all existing files before copying over my new deployment.
This is how my appspec.yml file looks like
version: 0.0
os: linux
files:
- source: /
destination: /path/to/destination
hooks:
BeforeInstall:
- location: DeploymentScripts/CleanupScript.sh
AfterInstall:
- location: DeploymentScripts/InstallScript.sh
timeout: 600
ApplicationStart:
- location: DeploymentScripts/RestartScript.sh
timeout: 3600
You can pass this attribute in the appspec.yml under
files :
overwrite: true
Related
I am trying to deploy an application onto EC2 instances with AWS CodeDeploy. I am getting up to the point where CodeDeploy errors out at the BeforeInstall hook. The error message I am getting is related to a ScriptMissing. The exact error is:
Script does not exist at specified location: /opt/codedeploy-agent/deployment-root/1bfe51a8-151a-4366-8a3f-c061adb4bb90/d-X8N0B5IOE/deployment-archive/scripts\codedeploy\install_dependencies.sh
The odd thing is the file exists in my version control repository. So far I have tried changing the permissions of the script, and have also tried different method in the appspec.yml file in order to get CodeDeploy to recognize this file. All methods have failed. I tried the other solutions posted relating to this scenario but they do not seem to help me.
Do these script files int he appspec.yml also need to part of the zip file that CodeDeploy uses? Any advice would be appreciated. Below is the appspec.yml file.
Appspec
version: 0.0
os: linux
files:
- source: /
destination: /var/www/html
overwrite: true
hooks:
BeforeInstall:
- location: scripts/codedeploy/install_dependencies.sh
timeout: 300
runas: root
AfterInstall:
- location: scripts/codedeploy/install_composer_dependencies.sh
timeout: 300
runas: root
- location: scripts/codedeploy/start_server.sh
timeout: 30
runas: root
ApplicationStop:
- location: scripts/codedeploy/stop_server.sh
timeout: 30
runas: root
I had a similar problem. My paths was scripts/file-name.sh. I just removed 'scripts/' from location path and leave only .sh file names and it works for me.
i am trying to deploy the python application through code-deploy to the ec2 instances
but during deployment i am facing this error
The deployment failed because a specified file already exists at this
location: /home/ubuntu/yello/manage.py
attaching my appsecfile also tried with overwrite but no luck
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu/yello
overwrite: true
hooks:
AfterInstall:
- location: script/services.sh
timeout: 300
runas: ubuntu
can anyone help ?
This is a very common error so AWS public docs has a section for the same.
https://docs.aws.amazon.com/codedeploy/latest/userguide/troubleshooting-deployments.html#troubleshooting-same-files-different-app-name
Basically when creating the deployment, you need to select Overwrite the content option so the deployment does not fail (default behaviour). Details here:
https://docs.aws.amazon.com/codedeploy/latest/userguide/deployments-rollback-and-redeploy.html#deployments-rollback-and-redeploy-content-options
The default behaviour prevents two Deployments from two different CodeDeploy "Applications" writing the same file and possibly causing some sort of conflict.
cd /var/www/your-directory/public/funnel_webhooks/test/
sudo rm -rf clickfunnel.txt
After removing this file please release the change in the code pipeline.
Now you can specify file_exists_behavior in your appspec file. Allowed values are DISALLOW, OVERWRITE, or RETAIN.
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu/yello
file_exists_behavior: OVERWRITE
I have two instances in AWS. One for production and one for homologation. I deploy automatically with CodeDeploy. I have two branches on BitBucket, the master and homolog. When I commit in the homolog deploy must go to the instance of homologation, and if I make a merge in the master deploy must be in the production stage.
To do the automatic deploy of Bitbucket to AWS there is a series of files that configure the deploy details. One of these files is the appspec.yml. According to AWS it is only possible to have an appspec.yml file.
This basic form file has the following structure:
version: 0.0
os: linux
files:
- source: /
destination: /var/www/html
hooks:
AfterInstall:
- location: deploy-scripts/install_dependencies.py
timeout: 300
runas: root
The problem is that for each instance I have a destination folder.
If I do the deploy on the homolog instance the destination folder should be var/www/html and for the production instance it should be var/www/html/test/
I tried to do it as below:
version: 0.0
os: linux
files:
- source: /
destination: deploy-scripts/destination.py
hooks:
AfterInstall:
- location: deploy-scripts/install_dependencies.py
timeout: 300
runas: root
That's the destination.py:
if os.environ['APPLICATION_NAME'] == 'ahimsa-store-homolog':
return '/var/www/html/'
elif os.environ['APPLICATION_NAME'] == 'ahimsa-store':
return '/var/www/html/teste/'
The above option does not work. How can I do this?
The files section of appspec.yml doesn't run scripts.
Move the required files to a temporary folder using the files section
Create a script that will move these files from the temporary location to the required destination depending on your requirements. As you suggested, using os.environ['APPLICATION_NAME'] for example.
Make sure your script sets correct file permissions after moving the files.
Include that script in the AfterInstall section, so the script can find the new files "installed" in the temporary location you choose. Also make sure it is before installing the dependencies!
Another option is to have a different appspec in each branch. It would make merging more difficult, but it could help.
I have installed the Bitbucket addon to deploy with AWS CodeDeploy but for an unknown reason, I get this error "Unable to remove top level folder" when I try to deploy from the bitbucket view.
This is my appspec.yml
version: 0.0
os: linux
files:
- source: /
destination: /var/www/citytwig
hooks:
AfterInstall:
- location: scripts/configure.sh
timeout: 300
runas: root
I have already deployed other bitbucket repositories successfully, I'm wondering why this one doesn't work.
Aws codedeploy tries to rollback previous deployment before it applies a new one. It can happen that you have deleted some files manually on the instance and now rollback script is failing.
Solution
Delete rollback scripts of codedeploy-agent on the instance.
if you are using ubuntu, ssh into instance and go to the directory
cd /opt/codedeploy-agent/deployment-root/deployment-instructions/
and look for a file that ends with -cleanup
delete this file.
Now try you deployment again.
Check out documentation, or read it in Rollback and Redeployment Workflow section of pdf
Depending on the rollback script the Deployment may fail with different messages, for me it was Directory not empty # dir_s_rmdir
But it always fails on Install Event.
After a long searching, I realized that a file had Russian characters on his filename.
It seems that the Bitbucket CodeDeploy Addon have issues with this kind of characters.
I had got the exact error, but in my case, it was a typo in the file name appspec.yml
While trying to deploy a Django project with CodeDeploy for the first time, I keep getting the following error in the AfterInstall phase:
Error Code: ScriptFailed
Script Name: /setup.sh
Message: Script at specified location: /setup.sh failed with exit code 2
Log Tail: LifecycleEvent - AfterInstall
Script - /setup.sh
[stderr]python: can't open file 'setup_start.py': [Errno 2] No such file or directory
This is probably because I'm misunderstanding the files section of the AppSpec. Below is a snippet of what I'm doing for that section:
files:
- source: ./BlackBoxes
destination: project/BlackBoxes
- source: ./Documentation
destination: project/Documentation
- source: ./manage.py
destination: project
- source: ./setup_start.py
destination: project
...
I did make the project folder manually on the S3 bucket but none of the subfolders.
And the AfterInstall section:
hooks:
...
AfterInstall:
- location: /setup.sh
timeout: 180
What I originally thought source was supposed to mean was the relative path of the file/directory with respect to the root of the project directory on my local development machine. I also assumed that any folder needed that didn't exist on the S3 bucket would be created automatically. Clearly, I am misunderstanding something about CodeDeploy, most likely pertaining to the AppSpec file. What exactly am I doing wrong with the deployment and what am I supposed to be doing instead?
You're going wrong in the files section. This section is where you tell Code Deploy: "place this directory/file from my repo to this location on my EC2 instance". You only need to do this for stuff your deploying, you don't need to do this for deployment hook scripts. Refer to the docs for the fine details on this section.
In a hook, the location is the relative path to your hook script from the root of your repo. So /setup.sh isn't correct -> you need to give it the relative path. Again, the docs is the place to read more on this.
What I usually do is in the root of my repo, I create a folder called eg. scripts and I store the hook scripts there.
Say my repo directory structure looks like this:
application_code/
scripts/
appspec.yml
I can then set up my appspec.yml like this:
version: 0.0
os: linux
files:
- source: application_code
destination: /desired/code/location #path to where the code should be put on the instance
hooks:
ApplicationStop:
- location: scripts/some_script.sh
timeout: 300
runas: root
Code Deploy is simple to use once you've read the documentation comprehensively. Until then, you're going to keep encountering problems like this.
Best of luck! :)