My files are properly copied to an EC2 instance for deployments, but they are not copied to the destination. I've got at appspec.yml file like this:
ubuntu#ip:~$ cat /opt/coded*/deployment-root/*/*/de*/appspec.yml
version: 0.0
os: linux
# I also tried with source: /
files:
- source: .
destination: /home/ubuntu/
file_exists_behavior: OVERWRITE
# notes
# ------------------------------------------------------------------------------------
# To learn more about hooks, visit the docs here:
# https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-structure-hooks.html#appspec-hooks-server
hooks:
BeforeInstall:
- location: scripts/install_dependencies
timeout: 300
runas: root
ApplicationStart:
- location: scripts/start_server
timeout: 300
runas: root
ApplicationStop:
- location: scripts/stop_server
timeout: 300
runas: root
I've also tried setting source to source: / with the same result; nothing's copied
And I see my files are all in that /opt/coded* etc. path. But why isn't the CodeDeploy agent copying my files to the destination? The Download bundle hook is good, and the only error I get is in the ApplicationStart hook when I need to actually access my files.
Note: I'm using GitHub as my revision storage, not S3, if that helps
Edit: something that might be useful to know is in my install_dependencies script, I run something along the lines of:
cd /home/ubuntu/
pip install -r requirements.txt
where the pip installation is the line where it can't find the files
Reasoning: https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-example.html#appspec-file-example-server
The order of the hooks. The files are copied before the AfterInstall hook.
Related
I am trying to deploy an application onto EC2 instances with AWS CodeDeploy. I am getting up to the point where CodeDeploy errors out at the BeforeInstall hook. The error message I am getting is related to a ScriptMissing. The exact error is:
Script does not exist at specified location: /opt/codedeploy-agent/deployment-root/1bfe51a8-151a-4366-8a3f-c061adb4bb90/d-X8N0B5IOE/deployment-archive/scripts\codedeploy\install_dependencies.sh
The odd thing is the file exists in my version control repository. So far I have tried changing the permissions of the script, and have also tried different method in the appspec.yml file in order to get CodeDeploy to recognize this file. All methods have failed. I tried the other solutions posted relating to this scenario but they do not seem to help me.
Do these script files int he appspec.yml also need to part of the zip file that CodeDeploy uses? Any advice would be appreciated. Below is the appspec.yml file.
Appspec
version: 0.0
os: linux
files:
- source: /
destination: /var/www/html
overwrite: true
hooks:
BeforeInstall:
- location: scripts/codedeploy/install_dependencies.sh
timeout: 300
runas: root
AfterInstall:
- location: scripts/codedeploy/install_composer_dependencies.sh
timeout: 300
runas: root
- location: scripts/codedeploy/start_server.sh
timeout: 30
runas: root
ApplicationStop:
- location: scripts/codedeploy/stop_server.sh
timeout: 30
runas: root
I had a similar problem. My paths was scripts/file-name.sh. I just removed 'scripts/' from location path and leave only .sh file names and it works for me.
I have a scenario where I want to create a new directory every time I push the code. The pipeline will run and it updates my existing directory pulling my all latest changes.
I want to keep the previous directory as it is and clone the complete directory every time with new name when pushed with the help of CodeDeploy.
My appspec.yml is as follows
# Definition file for AWS CodeDeploy
version: 0.0
os: linux
files:
- source: /
destination: /var/www/sample-app/
permissions:
- object: /var/www/sample-app
owner: www-data
group: www-data
type:
- file
- directory
hooks:
BeforeInstall:
- location: codedeploy-prepare.sh
AfterInstall:
- location: codedeploy-setup-app.sh
I have two instances in AWS. One for production and one for homologation. I deploy automatically with CodeDeploy. I have two branches on BitBucket, the master and homolog. When I commit in the homolog deploy must go to the instance of homologation, and if I make a merge in the master deploy must be in the production stage.
To do the automatic deploy of Bitbucket to AWS there is a series of files that configure the deploy details. One of these files is the appspec.yml. According to AWS it is only possible to have an appspec.yml file.
This basic form file has the following structure:
version: 0.0
os: linux
files:
- source: /
destination: /var/www/html
hooks:
AfterInstall:
- location: deploy-scripts/install_dependencies.py
timeout: 300
runas: root
The problem is that for each instance I have a destination folder.
If I do the deploy on the homolog instance the destination folder should be var/www/html and for the production instance it should be var/www/html/test/
I tried to do it as below:
version: 0.0
os: linux
files:
- source: /
destination: deploy-scripts/destination.py
hooks:
AfterInstall:
- location: deploy-scripts/install_dependencies.py
timeout: 300
runas: root
That's the destination.py:
if os.environ['APPLICATION_NAME'] == 'ahimsa-store-homolog':
return '/var/www/html/'
elif os.environ['APPLICATION_NAME'] == 'ahimsa-store':
return '/var/www/html/teste/'
The above option does not work. How can I do this?
The files section of appspec.yml doesn't run scripts.
Move the required files to a temporary folder using the files section
Create a script that will move these files from the temporary location to the required destination depending on your requirements. As you suggested, using os.environ['APPLICATION_NAME'] for example.
Make sure your script sets correct file permissions after moving the files.
Include that script in the AfterInstall section, so the script can find the new files "installed" in the temporary location you choose. Also make sure it is before installing the dependencies!
Another option is to have a different appspec in each branch. It would make merging more difficult, but it could help.
I have website, which stored on AWS EC2 servers.
We have 2 servers, one for production environment and another one for development and staging environments.
Development and staging environments located in different folders. For example development env stored in /var/www/development, while staging stored in /var/www/staging.
I'd like to use AWS CodeDeploy to upload files directly from bitbucket. I put AppSpec file, which copy source code to /var/www/html folder and install all dependencies and configurations. But I want my AppSpec file to copy source code to /var/www/development or to /var/www/staging depending on Development group, that was selected.
Is there is any way to do it or, maybe, there are some better approach in my situation?
The appspec.yml is a bit inflexible so use the following to deploy code in to different folders on the same instance.
version: 0.0
os: linux
files:
- source: /
destination: /var/www/my-temp-dir
permissions:
- object: /var/www/my-temp-dir
owner: ec2-user
group: ec2-user
hooks:
BeforeInstall:
- location: ci/integrations-deploy-pre.sh
runas: root
AfterInstall:
- location: ci/integrations-deploy-post.sh
runas: root
Inside of my integrations-deploy-post.sh file, I then use the CodeDeploy environment variables to move the files in to the place I need them to be;
#!/bin/bash
if [ "$DEPLOYMENT_GROUP_NAME" == "Staging" ]
then
cp -R /var/www/my-temp-dir /var/www/my-staging-dir
chown -R ec2-user:ec2-user /var/www/my-staging-dir
# Insert other commands that need to run...
fi
if [ "$DEPLOYMENT_GROUP_NAME" == "UAT" ]
then
cp -R /var/www/my-temp-dir /var/www/my-uat-dir
chown -R ec2-user:ec2-user /var/www/my-uat-dir
# Insert other commands that need to run...
fi
NOTE: In my integrations-deploy-post.sh You'll also need the commands you want to run on production. Removed for simplicity.
The recommended way to change AppSpec or custom scripts behavior is to utilize environment variables provided by the CodeDeploy agent. You have access to the deployment group name and the application name.
if [ "$DEPLOYMENT_GROUP_NAME" == "Staging" ]; then
# Copy to /var/www/staging
elif [ "$DEPLOYMENT_GROUP_NAME" == "Development" ]; then
# Copy to /var/www/development
elif [ "$DEPLOYMENT_GROUP_NAME" == "Production" ]; then
# Copy to /var/www/html
else
# Fail the deployment
fi
I had the same problem, but I used source control as a solution to this.
My workflow is using Gitlab CI > AWS Code Pipeline (S3 Source and CodeDeploy).
So in my development branch, my AppSpec file would look like this:-
version: 0.0
os: linux
files:
- source: /
destination: /var/www/html/my-project-dev
hooks:
AfterInstall:
- location: scripts/after_install.sh
timeout: 400
runas: root
in my staging branch:-
version: 0.0
os: linux
files:
- source: /
destination: /var/www/html/my-project-staging
hooks:
AfterInstall:
- location: scripts/after_install.sh
timeout: 400
runas: root
My Gitlab-CI just uses a shell executor to my EC2 instance and it basically compresses my project folder and uploads to S3.
.gitlab-ci.yml
stages:
- deploy
setup dependencies:
stage: .pre
script:
- echo "Setup Dependencies"
- pip install awscli
deploy to s3:
stage: deploy
script:
- tar -cvzf /tmp/artifact_$CI_COMMIT_REF_NAME.tar ./*
- echo "Copy artifact to S3"
- aws s3 cp /tmp/artifact_$CI_COMMIT_REF_NAME.tar s3://project-artifacts/
clean up:
stage: .post
script:
- echo "Removing generated artifact"
- rm /tmp/artifact_$CI_COMMIT_REF_NAME.tar
Note that $CI_COMMIT_REF_NAME is used to differentiate the artifact file being generated. In development branch it would be artifact_development.tar, in staging branch artifact_staging.tar.
Then, I have 2 pipelines listening to the two respective artifacts which deploys to 2 different CodeDeploy Application.
Not sure if this is the best way, surely welcome any suggestions that is better
While trying to deploy a Django project with CodeDeploy for the first time, I keep getting the following error in the AfterInstall phase:
Error Code: ScriptFailed
Script Name: /setup.sh
Message: Script at specified location: /setup.sh failed with exit code 2
Log Tail: LifecycleEvent - AfterInstall
Script - /setup.sh
[stderr]python: can't open file 'setup_start.py': [Errno 2] No such file or directory
This is probably because I'm misunderstanding the files section of the AppSpec. Below is a snippet of what I'm doing for that section:
files:
- source: ./BlackBoxes
destination: project/BlackBoxes
- source: ./Documentation
destination: project/Documentation
- source: ./manage.py
destination: project
- source: ./setup_start.py
destination: project
...
I did make the project folder manually on the S3 bucket but none of the subfolders.
And the AfterInstall section:
hooks:
...
AfterInstall:
- location: /setup.sh
timeout: 180
What I originally thought source was supposed to mean was the relative path of the file/directory with respect to the root of the project directory on my local development machine. I also assumed that any folder needed that didn't exist on the S3 bucket would be created automatically. Clearly, I am misunderstanding something about CodeDeploy, most likely pertaining to the AppSpec file. What exactly am I doing wrong with the deployment and what am I supposed to be doing instead?
You're going wrong in the files section. This section is where you tell Code Deploy: "place this directory/file from my repo to this location on my EC2 instance". You only need to do this for stuff your deploying, you don't need to do this for deployment hook scripts. Refer to the docs for the fine details on this section.
In a hook, the location is the relative path to your hook script from the root of your repo. So /setup.sh isn't correct -> you need to give it the relative path. Again, the docs is the place to read more on this.
What I usually do is in the root of my repo, I create a folder called eg. scripts and I store the hook scripts there.
Say my repo directory structure looks like this:
application_code/
scripts/
appspec.yml
I can then set up my appspec.yml like this:
version: 0.0
os: linux
files:
- source: application_code
destination: /desired/code/location #path to where the code should be put on the instance
hooks:
ApplicationStop:
- location: scripts/some_script.sh
timeout: 300
runas: root
Code Deploy is simple to use once you've read the documentation comprehensively. Until then, you're going to keep encountering problems like this.
Best of luck! :)