So, I'm setting up my first application that uses CodeDeploy (EC2 + S3) and I'm having a very hard time to figure out how to run the scripts after instalation.
So I defined an AfterInstall hook in the AppSpec file refering to my bash script file in the project diretory. When the commands in the script run I get the error stating the files could not be found. So I put an ls command before it all and checked the logs.
My script file is running in the CodeDeploy agent folder. There are many files there that I accidentally created when I was testing but I was expecting them to be in my project root folder.
--Root
----init.sh
----requirements.txt
----server.py
appspec.yml
version: 0.0
os: linux
files:
- source: ./
destination: /home/ubuntu/myapp
runas: ubuntu
hooks:
AfterInstall:
- location: init.sh
timeout: 600
init.sh
#!/bin/bash
ls
sudo apt install python3-pip
pip3 install -r ./requirements.txt
python3 ./server.py
So when ls is executed, it doesn't list the files in my project root directory. I also tried ${PWD} instead of ./ and it didn't work. It is copying the script file to the agent folder and running it.
Refer to this https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-structure-hooks.html
This is written at the end of the above document
The location of scripts you specify in the 'hooks' section is relative
to the root of the application revision bundle. In the preceding
example, a file named RunResourceTests.sh is in a directory named
Scripts. The Scripts directory is at the root level of the bundle.
But apparently it refers to the paths in the appspec file only.
Could someone help? Is it correct? I MUST use absolute paths hard-coded in the script file?
Yes correct. The script doesn't execute in the destination folder, as you might expect. You need to hard code a reference the destination directory /home/ubuntu/myapp to resolve file paths in life cycle scripts.
Use cd to change the directory first:
cd /home/ubuntu/myapp
ls
sudo apt install python3-pip
pip3 install -r ./requirements.txt
python3 ./server.py
Related
I want to copy a file from AWS S3 to a local directory through a docker container.
This copying command is easy without docker, I can see the file downloaded in the current directory.
But the problem is with docker that I don’t even know how to access the file.
Here is my Dockerfile:
FROM ubuntu
WORKDIR "/Users/ezzeldin/s3docker-test"
RUN apt-get update
RUN apt-get install -y awscli
ENV AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
ENV AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
CMD [ "aws", "s3", "cp", "s3://ezz-test/s3-test.py", "." ]
The current working folder that I should see the file downloaded to is s3docker-test/. This is what I'm doing after building the Dockerfile to mount a volume myvol to the local directory
docker run -d --name devtest3 -v $PWD:/var/lib/docker/volumes/myvol/_data ubuntu
So after running the image I get this:
download: s3://ezz-test/s3-test.py to ./s3-test.py
which shows that the file s3-test.py is already downloaded, but when I run ls in the interactive terminal I can't see it. So how can I access that file?
Looks like you are overriding containers folder with your empty folder, when you run -v $PWD:/var/lib/docker/volumes/myvol/_data.
Try to simply copy the files from container to host fs by running:
docker cp \
<containerId>:/Users/ezzeldin/s3docker-test/s3-test.py \
/host/path/target/s3-test.py
You could perform this command even on downed container. But first you will have to run it without folder override:
docker run -d --name devtest3 ubuntu
I have two problems deploying via AWS CodeDeploy.
I'm trying to deploy CodeCommit's code to an EC2 ubuntu instance.
At appspec.yml
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu
hooks:
ApplicationStart:
- location: scripts/ApplicationStart.sh
timeout: 300
runas: ubuntu
There are several config files that I need to place at the right place in the application before starting pm2. I also assume since I set runas in appspec.yml as ubuntu, the bash script will at /home/ubuntu.
The my /home/ubuntu has
config/ backend/ frontend/
Looks like Code deploy won't overwrite the previous deployment so if I have backend/ and frontend/ folder at the directory, it will fail at Install stage.
In the ApplicationStart.sh
#!bin/bash
sudo cp config/config1.json backend/config/config1.json
sudo cp config/config2.json backend/config/environments/development/config2.json
sudo cp config/config3.json frontend/config3.json
sudo pm2 kill
cd backend
sudo npm install
sudo pm2 start "strapi start" --name backend
cd ../frontend
sudo npm install
sudo pm2 start "npm start" --name frontend
While the ApplicationStart stage, it gives me the following error.
LifecycleEvent - ApplicationStart
Script - scripts/ApplicationStart.sh
[stderr]bash: /opt/codedeploy-agent/path/to/deployment/scripts/ApplicationStart.sh: bin/bash:
bad interpreter: No such file or directory
I run the same bash file at the /home/ubuntu. It works fine.
Question 1.
- how to run BeforeInstall.sh without the error? Is there the path problems or something else I try to do but I am not supposed to do?
Question 2.
- How can I let code deploy to overwrite the previous deployment when there are already application folders in the directory (/home/ubuntu)?
- Do I manually delete the directory at BeforeInstall stage?
You're missing a slash before bin/bash in #!bin/bash.
It should be #!/bin/bash.
I'm attempting to run awsebcli from inside a docker image based on amazonlinux
The docker file is like this..
FROM amazonlinux:latest
ENV PATH "$PATH:/root/.local/bin"
ADD . /myfiles
WORKDIR /myfiles
#copy credentials
RUN cp -R .aws ~
RUN curl -O https://bootstrap.pypa.io/get-pip.py
RUN python get-pip.py --user --no-warn-script-location
RUN pip install awsebcli --upgrade --user
CMD eb --version
This just returns:
ERROR: OSError - [Errno 2] No such file or directory
What have I missed?
This was a dumb issue.
I has named the elastic beanstalk config file just "config" (like the .aws/config)
It was supposed to be called .elasticbeanstalk/config.yml
I had the same problem and it turned out that in my configuration file under .elasticbeanstalk/config.yml I had the following line
sc: git
This makes the eb tool search for a .git which was not present in my case since I only wanted to deploy a zip file.
The error message is far from clear !
I have a note app that I am building with a Dockerfile in the maven app.
I want to copy the artifact note-1.0.war to local linked volume to folder like webapps. So far I have the following in a Dockerfile:
FROM maven:latest
MAINTAINER Sonam <emailme#gmail.com>
RUN apt-get update
WORKDIR /code
#Prepare by downloading dependencies
ADD pom.xml /code/pom.xml
RUN ["mvn", "dependency:resolve"]
RUN ["mvn", "verify"]
#Adding source, compile and package into a fat jar
ADD src /code/src
RUN ["mvn", "clean"]
#RUN ["mvn", "install"]
RUN ["mvn", "install", "-Dmaven.test.skip=true"]
RUN mkdir webapps
COPY note-1.0.war webapps
#COPY code/target/note-1.0.war webapps
Unfortunately, I keep seeing the "no such file or directory" at the COPY statement. The following is the error from build on Docker hub:
...
---> bd555aecadbd
Removing intermediate container 69c09945f954
Step 11 : RUN mkdir webapps
---> Running in 3d114c40caee
---> 184903fa1041
Removing intermediate container 3d114c40caee
Step 12 : COPY note-1.0.war webapps
lstat note-1.0.war: no such file or directory
How can I copy the war file to a "webapps" folder that I executed in
RUN mkdir webapps
thanks
The COPY instruction copies new files or directories from <src> and adds them to the filesystem of the container at the path <dest>.
In your example the docker build is looking for note-1.0.war in the same directory than Dockerfile.
If I understand your intention, you want to copy a file inside the image that is build from previous RUN in Dockerfile.
So you should use something like
RUN cp /code/target/note-1.0.war /code/webapps
I am trying to setup CI using AWS CodeDeploy and CircleCI. Right now I am stuck at the step where AWS CodeDeploy should copy stuff into EC2 and run scripts. But somehow CircleCI tells me something is wrong. Does anyone know what might be happening? Thanks.
the appspec.yml is:
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu
hooks:
BeforeInstall:
- location: scripts/setup.sh
timeout: 3800
runas: root
ApplicationStart:
- location: scripts/start.sh
timeout: 3800
runas: root
and setup.sh is:
#!/bin/bash
sudo apt-get install nodejs npm
npm install
in the above code I also tried only apt-get install nodejs npm but it's still nor working.
the error message in /var/log/aws/codedeploy-agent/codedeploy-agent.log is as follows:
2015-10-22 08:02:54 ERROR [codedeploy-agent(1314)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Error during
perform: InstanceAgent::Plugins::CodeDeployPlugin::ScriptError - Script at specified location:
./scripts/setup.sh run as user root failed with exit code 127 - /opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/hook_executor.rb:150:in `execute_script'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/hook_executor.rb:107:in `block (2 levels) in execute'
......
Exit code 127 generally means that the OS couldn't find something required to execute the command. In this case it could be either the script wasn't at the expected path or /bin/bash doesn't exist (unlikely).
Check that the archive being produced by your build process is actually putting your scripts in the archive where your appspec expects them. scripts/setup.sh needs to be in that exact path within your archive.
You can also look at what the agent actually got by checking the deployment archive for your deployment: /opt/codedeploy-agent/deployment-root/deployment-group-id/deployment-id/deployment-archive to make sure the archive is being extracted correctly.
Do the following steps for the debugging:
in the CodeDeploy error log /var/log/aws/codedeploy-agent/codedeploy-agent.log there is a line that says Error during perform: InstanceAgent::Plugins::CodeDeployPlugin::ScriptError - Script at specified location: scripts/setup.sh failed with exit code 1. So from the error log I know the problems might be from this script.
In the above mentioned script setup.sh, put something like this at the beginning of the script:
exec 3>&1 4>&2
trap 'exec 2>&4 1>&3' 0 1 2 3
exec 1>/home/ubuntu/out.log 2>&1
This logs the entire error outputs for you.
Permission issues
It's also possible that EC2 failed to execute those scripts, you need to make sure those files have at least 755 permissions when copied to your instance. So you need to specify 755 file mode for your scripts.
How to change the File Mode on GitHub?
Also in appspec.yml you need can specify a runas directive. Could be ubuntu or root or whatever that gives you the correct permission.
miscellaneous
Some pitfalls on deploying like when you do sudo apt-get install nodejs there will be intermediate steps that ask if you want to install packages and used disk spaces and you have to type Y or N to proceed installation. those scripts would hang there and timeout resulting in failed deployment. So instead you do
sudo apt-get -y install nodejs npm
Or in your setup.sh script maybe you have
chmod -R 777 public
but it's possible CodeDeploy is executing this code in a folder that's different than your project root. So make sure all the paths are valid.