Previously I had CodeDeploy running properly as user www-data and everything was fine but I think CodeDeploy got corrupted somehow because it always runs as the root user now. Then, it breaks mid-deploy because of the "wrong permissions" because it shouldn't try to run as the root user, but the www-data user.
I checked another working server with a working CodeDeploy setup and the settings match EXCEPT there are no root user entries after running "ps aux | grep codedeploy-agent" seen here:
CodeDeploy says here that it's running as PID 3914 - so why is there a root entry with PID 3650? I think it is also using the root entry PID 3650 when running CodeDeploy because it is creating new directories with "root:root" permissions. However on the working servers, everything is created with "www-data:wheel" as I configured it.
So, how can I remove the root user here from executing everything without breaking anything else? (I prefer not to delete stuff without knowing if it's safe and this is also a server that another team works on so I am trying to avoid a full rebuild or uninstall - as well as breaking any of their stuff.)
FYI to get this working in the first place as www-data running CodeDeploy, I changed the user using this article and it's worked great on other servers for months now: https://aws.amazon.com/premiumsupport/knowledge-center/codedeploy-agent-non-root-profile/
*Note: Tried deleting the .pid and .pid.lock files recommended in another post and printed in screenshot, but that didn't do anything.
This would come down to an update to the CodeDeploy agent. Possibly either an accidental replacement in the update process or incompatible changes which required settings to be reset to the default settings.
I would suggest leaving these default settings for CodeDeploy as it can be reconfigured during patching or future updates (which leaves it out of your control). Instead set permissions via the appspec file.
You can define the users that actions within CodeDeploy run as via the appspec file that is required for each deployed.
By specifying runas the hook will be run as that user, as seen in this example below.
version: 0.0
os: linux
files:
- source: Config/config.txt
destination: /webapps/Config
- source: source
destination: /webapps/myApp
hooks:
BeforeInstall:
- location: Scripts/UnzipResourceBundle.sh
- location: Scripts/UnzipDataBundle.sh
AfterInstall:
- location: Scripts/RunResourceTests.sh
timeout: 180
ApplicationStart:
- location: Scripts/RunFunctionalTests.sh
timeout: 3600
ValidateService:
- location: Scripts/MonitorService.sh
timeout: 3600
runas: codedeployuser
You can also set the permissions that particular directories or files have (from the files hierarchy.
Related
I need to change this setting on my AutoScaling group in my Beanstalk environment:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environmentconfig-autoscaling-healthchecktype.html
I'm doing exactly like the example shows, but it just doesn't work. Nothing happens.
The file content:
Resources:
AWSEBAutoScalingGroup:
Type: "AWS::AutoScaling::AutoScalingGroup"
Properties:
HealthCheckType: ELB
HealthCheckGracePeriod: 300
The structure:
my_project/
.ebextensions/
autoscaling.config
src/
...
Is there any log where I can check if the file is even being read or not?
If you use eb deploy to deploy your changed code, it will not deploy your local code, but your last git commit.
You can try to find something in the logs in /var/log, for example in eb-engine.log or messages, or even in the system logs ($ journalctl), if you configured to be able to ssh into the machine.
You can also use the web console to download the logs (like this demo link).
If you don't find something, can you give some more information about the platform (java, node.js, python) and the way you are deploying?
Bye,
Dirk
Okay I found the problem. I'm using Green/Blue environments to deploy my application, so in the process of uploading new changes to the environment they were being made to the Green Env that was being terminated after.
In order for the new config to work I had to rebuild the Blue env.
From now on all the new deploys will have the new config.
I am trying to deploy some new code using aws codepipeline. The first time it works no problem, the second time the deployment fails because of existing files. How can I instruct my flow to overwrite existing files?
Error Message:
The deployment failed because a specified file already exists at this location:
I think the best way to deploy is to delete the project directory and kill the process before deploying. This let the state of the project directory keep pure and in sync with the original repository.
As you can see from this link(appspec.yml hooks for deploying to EC2), CodeDeploy download artifacts on Install phase and we cannot access to that step. Install phase comes after BeforeInstall hook.
So you should delete the directory and kill the processes before
Install phase to be executed.
hooks:
BeforeInstall:
- location: codedeploy-scripts/deleteAndKill.sh
# runas: root # this might be needed depending on your setting.
Define codedeploy-scripts/deleteAndKill.sh properly and try running CodePipeline and CodeDeploy again.
P.S. Deleting the project directory and killing the processes are somewhat bothering. so once you use docker, what you have to do is only docker stop {container name} and docker run {image name}.
I have two instances in AWS. One for production and one for homologation. I deploy automatically with CodeDeploy. I have two branches on BitBucket, the master and homolog. When I commit in the homolog deploy must go to the instance of homologation, and if I make a merge in the master deploy must be in the production stage.
To do the automatic deploy of Bitbucket to AWS there is a series of files that configure the deploy details. One of these files is the appspec.yml. According to AWS it is only possible to have an appspec.yml file.
This basic form file has the following structure:
version: 0.0
os: linux
files:
- source: /
destination: /var/www/html
hooks:
AfterInstall:
- location: deploy-scripts/install_dependencies.py
timeout: 300
runas: root
The problem is that for each instance I have a destination folder.
If I do the deploy on the homolog instance the destination folder should be var/www/html and for the production instance it should be var/www/html/test/
I tried to do it as below:
version: 0.0
os: linux
files:
- source: /
destination: deploy-scripts/destination.py
hooks:
AfterInstall:
- location: deploy-scripts/install_dependencies.py
timeout: 300
runas: root
That's the destination.py:
if os.environ['APPLICATION_NAME'] == 'ahimsa-store-homolog':
return '/var/www/html/'
elif os.environ['APPLICATION_NAME'] == 'ahimsa-store':
return '/var/www/html/teste/'
The above option does not work. How can I do this?
The files section of appspec.yml doesn't run scripts.
Move the required files to a temporary folder using the files section
Create a script that will move these files from the temporary location to the required destination depending on your requirements. As you suggested, using os.environ['APPLICATION_NAME'] for example.
Make sure your script sets correct file permissions after moving the files.
Include that script in the AfterInstall section, so the script can find the new files "installed" in the temporary location you choose. Also make sure it is before installing the dependencies!
Another option is to have a different appspec in each branch. It would make merging more difficult, but it could help.
I have installed the Bitbucket addon to deploy with AWS CodeDeploy but for an unknown reason, I get this error "Unable to remove top level folder" when I try to deploy from the bitbucket view.
This is my appspec.yml
version: 0.0
os: linux
files:
- source: /
destination: /var/www/citytwig
hooks:
AfterInstall:
- location: scripts/configure.sh
timeout: 300
runas: root
I have already deployed other bitbucket repositories successfully, I'm wondering why this one doesn't work.
Aws codedeploy tries to rollback previous deployment before it applies a new one. It can happen that you have deleted some files manually on the instance and now rollback script is failing.
Solution
Delete rollback scripts of codedeploy-agent on the instance.
if you are using ubuntu, ssh into instance and go to the directory
cd /opt/codedeploy-agent/deployment-root/deployment-instructions/
and look for a file that ends with -cleanup
delete this file.
Now try you deployment again.
Check out documentation, or read it in Rollback and Redeployment Workflow section of pdf
Depending on the rollback script the Deployment may fail with different messages, for me it was Directory not empty # dir_s_rmdir
But it always fails on Install Event.
After a long searching, I realized that a file had Russian characters on his filename.
It seems that the Bitbucket CodeDeploy Addon have issues with this kind of characters.
I had got the exact error, but in my case, it was a typo in the file name appspec.yml
While trying to deploy a Django project with CodeDeploy for the first time, I keep getting the following error in the AfterInstall phase:
Error Code: ScriptFailed
Script Name: /setup.sh
Message: Script at specified location: /setup.sh failed with exit code 2
Log Tail: LifecycleEvent - AfterInstall
Script - /setup.sh
[stderr]python: can't open file 'setup_start.py': [Errno 2] No such file or directory
This is probably because I'm misunderstanding the files section of the AppSpec. Below is a snippet of what I'm doing for that section:
files:
- source: ./BlackBoxes
destination: project/BlackBoxes
- source: ./Documentation
destination: project/Documentation
- source: ./manage.py
destination: project
- source: ./setup_start.py
destination: project
...
I did make the project folder manually on the S3 bucket but none of the subfolders.
And the AfterInstall section:
hooks:
...
AfterInstall:
- location: /setup.sh
timeout: 180
What I originally thought source was supposed to mean was the relative path of the file/directory with respect to the root of the project directory on my local development machine. I also assumed that any folder needed that didn't exist on the S3 bucket would be created automatically. Clearly, I am misunderstanding something about CodeDeploy, most likely pertaining to the AppSpec file. What exactly am I doing wrong with the deployment and what am I supposed to be doing instead?
You're going wrong in the files section. This section is where you tell Code Deploy: "place this directory/file from my repo to this location on my EC2 instance". You only need to do this for stuff your deploying, you don't need to do this for deployment hook scripts. Refer to the docs for the fine details on this section.
In a hook, the location is the relative path to your hook script from the root of your repo. So /setup.sh isn't correct -> you need to give it the relative path. Again, the docs is the place to read more on this.
What I usually do is in the root of my repo, I create a folder called eg. scripts and I store the hook scripts there.
Say my repo directory structure looks like this:
application_code/
scripts/
appspec.yml
I can then set up my appspec.yml like this:
version: 0.0
os: linux
files:
- source: application_code
destination: /desired/code/location #path to where the code should be put on the instance
hooks:
ApplicationStop:
- location: scripts/some_script.sh
timeout: 300
runas: root
Code Deploy is simple to use once you've read the documentation comprehensively. Until then, you're going to keep encountering problems like this.
Best of luck! :)