I have a Wordpress repository that I'm deploying to a Beanstalk application through the CodePipeline + Github.
When the CodePipeline releases any changes on the master branch, the deployment process starts, but it replaces the whole project and I'd like to keep some directories in the instance such as uploads/ and plugins/.
Is there a way to keep them when the Deployment stage starts?
I solved the issue using the S3 service instead.
Setup a S3 bucket
Install this plugin https://github.com/humanmade/S3-Uploads
The plugin contains instructions to perform the migration
Done, all files will be uploaded to S3 instead and the Pipeline will never replace it.
Related
I've set up my codedeploy appspec.yml file currently located in my root folder. I have a CI/CD that runs 2 pipelines depending on which branch the new code is pushed/merged into. I have one for production and one for staging. Previously i've used the s3 artifacts from the staging to build the production but i want to change it so it runs the production pipeline and deploys when i push to master. The issue is that in order to build it i have some hooks running bash scripts where the content is different from the staging appspec file.
Any ideas on how it is possible to have multiple appspec files for aws codedeploy?
BR Lukas
You can't rename artifacts on the fly. You can do it in an intermediate step before invoking CodeDeploy. BTW, you say you are building different versions for different environments, a better approach would be to user either instance / user metadata or tags.
I created a repo in Code Commit for a static s3 website
Then I created a CodePipeline and configured the code build part.
There I set the Build Spec file with the some basic commands:
build and then copy the files in the s3 bucket.
The third step the Code Deploy I'm not sure why it's needed.
When I run it it gets stuck for an hour.
I did disable it and the site was deployed just fine.
Am I missing something?
You can disable the CodeDeploy part if it is working fine for you. Or you can skip the CodeBuild step and use appspec.yml to deploy the static website onto S3.
You have to use either of the steps to make it work, you can't skip both the steps.
CodeDeploy part is present in the CodePipeline in case you need to deploy it on your EC2 fleet or Autoscaling Group after you have built the artifacts. If not needed, just skip it.
Codepipeline has three stages source->codeBuild->codeDeploy. According to Amazon you must use atleast two stages of the Codepipeline, You cannot skip the first stage (i.e source) but you choose any one or both from the remaining. For your use case source and CodeBuild stages are enough you don't need codedeploy. Just remove the codeDeploy stage.
Trying to create a simple deployment job on jenkins with the plugin post-build aws codedeploy.
The issue i'm facing is not able to target an s3 zip file as a deployment target. I don't want to upload nothing from codedeploy, just simply trigger a deployment from jenkins with a proper configuration ( bucket, region and of course the package.zip, which is already exists in the bucket )
Is there any "easy" way i can do that?
https://aws.amazon.com/blogs/devops/setting-up-the-jenkins-plugin-for-aws-codedeploy/
Current setup works as charmed, the deployment is triggered on aws but with wrong target file so the deployment fails at the moment. There is no chance to merge the build(and upload to s3) and deploy job together.
Switched to aws-cli for properly target a bucket for deployment. There is no way to use the plugin for situation like this
Instead of having build and deploy as two different stages you can have both in same stage where the jenkins job will checkout from the pipeline and codedeploy post job will automatically zip and store the revision in S3 actually this is the way I achieved it. But the best way is to use AWS cli.
I have a site in a S3 bucket, configured for web access, for which I run an aws s3 sync command every time I push on a specific git repository (I'm using Gitlab at the moment).
So if I push to stable branch, a Gitlab runner performs the npm start build command for building the site, and then aws s3 sync to synchronize to a specific bucket.
I want to migrate to CodeCommit and use pure AWS tools to do the same.
So far I was able to successfully setup the repository, create a CodeBuild for building the artifact, and the artifact is being stored (not deployed) to a S3 bucket. Difference is that I can't get it to deploy to the root folder of the bucket instead of a subfolder, seems like the process is not made for that. I need it to be on a root folder because of how the web access is configured.
For the deployment process, I was taking a look at CodeDeploy but it doesn't actually let me deploy to S3 bucket, it only uses the bucket as an intermediary for deployment to a EC2 instance. So far I get the feeling CodeDeploy is useful only for deployments involving EC2.
This tutorial with a similar requirement to mine, uses CodePipeline and CodeBuild, but the deployment step is actually a aws s3 sync command (same as I was doing on Gitlab), and the actual deployment step on CodePipeline is disabled.
I was looking into a solution which involves using AWS features made for this specific purpose, but I can't find any.
I'm also aware of LambCI, but to me looks like what CodePipeline / CodeBuild is doing, storing artifacts (not deploying to the root folder of the bucket). Plus, I'm looking for an option which doesn't require me to learn or deploy new configuration files (outside AWS config files).
Is this possible with the current state of AWS features?
Today AWS has announced as a new feature the ability to target S3 in the deployment stage of CodePipeline. The announcement is here, and the documentation contains a tutorial available here.
Using your CodeBuild/CodePipeline approach, you should now be able to choose S3 as the deployment provider in the deployment stage rather than performing the sync in your build script. To configure the phase, you provide an S3 bucket name, specify whether to extract the contents of the artifact zip, and if so provide an optional path for the extraction. This should allow you to deploy your content directly to the root of a bucket by omitting the path.
I was dealing with similar issue and as far as I was able to find out, there is no service which is suitable for deploying app to S3.
AWS CodeDeploy is indeed for deploying code running as server.
My solution was to use CodePipeline with three stages:
Source which takes source code from AWS CodeCommit
Build with AWS CodeBuild
Custom lambda function which after successful build takes artifact from S3 artifact storage, unzip it and copies files to my S3 website host.
I used this AWS lambda function from SeamusJ https://github.com/SeamusJ/deploy-build-to-s3
Several changes had to be made, I used node-unzip-2 instead of unzip-stream for unziping artifict from s3.
Also I had to change ACLs in website.ts file
Uploading from CodeBuild is currently the best solution available.
There's some suggestions on how to orchestrate this deployment via CodePipeline in this answer.
When using AWS, it seems a nice way to deploy an application to a newly created instance is via AWS CodeDeploy. This works as follows:
Set up an auto-scaling group for the application
Write a user-data bash script for the auto-scaling group which pulls the CodeDeploy agent from S3, installs it and starts it
Set up a CodeDeploy deployment group which deploys to the auto-scaling group
Now, when an application bundle (e.g. jar or debian package) is deployed to the deployment group, it will be deployed automatically to new instances launched in the auto-scaling group.
My question is: how can this deployment strategy fit with a CI tool like Travis CI?
Specifically:
How can CodeDeploy pick up a package built by a CI tool like Travis CI? Does the build job need to upload the package to S3?
How can CodeDeploy be used to deploy the application gradually (e.g. one instance at a time)?
Does this deployment strategy require each running instance to be shut down and replaced, or is the new version of the application deployed on the existing instances? If it is the former, machine IP addresses would change during deployment, so how can other services discover the newly deployed application (i.e. without hardcoded IP addresses)?
tl;dr version:
The build job needs to upload the package to S3.
Use the one at a time deployment config.
The new version of the application is deployed on the existing instances.
Ok, here's the long version:
I recommend you try the Deployment Walkthrough or take a looks at Concepts in the documentation. It should help you get familiar with CodeDeploy faster.
You don't have to use an AutoScaling group with CodeDeploy if you don't want to. CodeDeploy with AutoScaling integration allows you to manage fleets that need to change in size dynamically separately from the code that is deployed to them, but that is not a requirement to use CodeDeploy. You can also launch some EC2 instances manually, install the host agent, and then tag them into a deployment group - but they won't get deployed to automatically on launch like the AutoScaling instances would. In either case, you can always create fleet wide deployments.
You'll have to do some work to integrate it with your CI tool. CodeDeploy doesn't directly manage your build artifacts, so your build process will need to do that. To have automatic deployments, your will need to:
Create a archive bundle with an appspec.yml, any scripts you need to handle the install/upgrade, and your build artifacts.
Upload the bundle to S3.
Create a deployment in CodeDeploy.
You might want to look at CodePipeline as an example of a continuous delivery system that's integrated with CodeDeploy.
CodeDeploy uses deployment configs to control how aggressively it deploys to the instances in your fleet. (This config gets ignored for automatic deployments, since each instance is handled separately.) CodeDeploy will fail your deployment and stop deploying to new instances if it cannot potentially fail another instance without violating the constraints in the deployment config.
There are three built in deployment configs, and you can create your own via the CLI or API if you need a different one. To deploy to only one instance at a time, you can use the CodeDeployDefault.OneAtATime deployment config, which allows at most one unhealthy host at any given time.
For anyone else (like me) looking for an example on how to actually integrate Travis-CI with CodeDeploy:
Configure the Application, DeploymentGroups and instances, as explained in the CodeDeploy walkthrough.
Use aws-cli commands to deploy your first revision successfully to the CodeDeploy target instance.
After you have the application deployed and running, configure Travis to trigger the deployments.
The CodeDeploy appspec.yml file and any scripts used for the deployment should be packaged inside your application bundle (latest.zip in the below example).
The following .travis.yml config worked for me:
script: npm run build
before_deploy:
- zip -r latest dist/*
- mkdir -p dpl_cd_upload
- mv latest.zip dpl_cd_upload/latest.zip
deploy:
- provider: s3
access_key_id: "XXXX"
secret_access_key: "YYYYY"
bucket: "deployments-bucket-name"
local_dir: dpl_cd_upload
skip_cleanup: true
- provider: codedeploy
access_key_id: "ZZZZZ"
secret_access_key: "WWWW"
bucket: "deployments-bucket-name"
key: latest.zip
bundle_type: zip
application: CodeDeployAppName
deployment_group: CodeDeployDeploymentGroupName
This examples were really useful:
https://github.com/travis-ci/cat-party/blob/master/.travis.yml