I want to automate the process by which when I push my code to my private repository on Bitbucket, it will directly update on my EC2 Instances. For that I gained some knowledge which was like:
AWS CodeDeploy is used for that. What I have learned is:
Go to IAM Role and create the role and attach the policy i.e AmazonEC2RoleForCodeDeploy & AWSCodeDeployRole
Edit Trust Relationship -> "service": "codedeploy.amazonaws.com"
At the time of the creation of EC2 Instance use the IAM Role to the role which we just created.
But I am confused about the YML code which will be paste on Advance Details (at the time of Instance Creation).
I attended many Blogs and youtube tutorials but getting confused about the right implementation.
On some tutorial, used S3 for code deploy. I am a little confused about this problem regarding the automation process.
My simple requirement is When I update/push the code on my personal branch, it will update on my directory which is under my EC2 instance.
Any AWS expert or DevOps will help me with that, Kindly share the best efficient manner to achieve this task. I am Mentioning "expert" because I need the best and optimum solution not the solution to achieve the goal.
Any help or suggestion is really appreciated for that. Pardon me if there is some typo error or any grammatical error on mine query.
The basic steps would be:
Add the CodeDeploy addon from from Bitbucket's App Marketplace and
configure it.
To add an IAM role that allows Bitbucket to first push code to an
intermediate S3 bucket in your account.
Once you've configured the CodeDeploy settings from the Settings
page of the Bitbucket console, you can perform a deployment to the
Deployment group from your branch and monitor this in the Codedeploy
console.
The blog below has clear and detailed steps for this:
https://aws.amazon.com/blogs/apn/announcing-atlassian-bitbucket-support-for-aws-codedeploy/
The Yaml file you are confused about is the Appspec file that tells CodeDeploy what actions to perform during a Deployment.
The bare minimum appspec.yml file you can have to deploy all the files from the source package:
version: 0.0
os: linux
files:
- source: /
destination: /var/www/html
This file is placed in the root of your source code.
Appspec file example:
https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-example.html#appspec-file-example-server
Related
I have created a simple deploy pipeline using jenkins, I have created the codedeploy, the S3 Bucket, the Autoscalable group, the ami. Everything listed in the docs. But it needs a appspec.yml. I have looked at the documentation for appspec.yml. And it’s very confusing.
Is there any way to generate a appspec.yml. I am not even sure what its role is. I thought the code deploy would take the zip file out of the S3 Bucket and deploy it to the scaleble group.
Any help?
appspec.yml is the file that tells codedeploy service about what tasks it should do with the code on your EC2 servers. So it needs to be built according to your workflow. This documentation and the examples will help you what you want to achieve.
Is there any way to generate a appspec.yml
You can't auto-generate the file. It must be custom designed for your specific application, and only you know what your application is, how it works, how it is configured, what are its dependencies, and so on.
I want to achieve a Continuous delivery for provisioning AWS resources using Bitbucket & AWS. My use case is to create a kinesis Firehose Delivery stream with destination as Elastic Search. I want this to be achieved by using the AWS cloudformation templates (keeping in mind the different stages for dev, uat, prod). Whenever I update my bitbucket repo the build should get created and the stack will get updated in AWS. Any help will be highly appreciated.
I have searched a lot over the internet but could not find any relevant examples which clearly describes my use case.
Cloudformatiom template committed in bitbucket will provision AWS resources in the cloud
You can use AWS CodePipeline for this purpose. The only issue is that Code Pipeline does not work directly with Bitbucket, only works with AWS CodeCommit or GitHub as the triggering repo.
But there is a workaround for that. You can set up syncing from Bitbucket to GitHub and then set the GitHub repository as the source repository for the pipeline you will create on AWS CodePipeline. You can find many guides for syncing so I won't explain here. The pipeline itself can be defined as a template.
I have already explained Setting up AWS CodePipeline in another answer here that you can follow for this purpose. Hope this helps!
I have a site in a S3 bucket, configured for web access, for which I run an aws s3 sync command every time I push on a specific git repository (I'm using Gitlab at the moment).
So if I push to stable branch, a Gitlab runner performs the npm start build command for building the site, and then aws s3 sync to synchronize to a specific bucket.
I want to migrate to CodeCommit and use pure AWS tools to do the same.
So far I was able to successfully setup the repository, create a CodeBuild for building the artifact, and the artifact is being stored (not deployed) to a S3 bucket. Difference is that I can't get it to deploy to the root folder of the bucket instead of a subfolder, seems like the process is not made for that. I need it to be on a root folder because of how the web access is configured.
For the deployment process, I was taking a look at CodeDeploy but it doesn't actually let me deploy to S3 bucket, it only uses the bucket as an intermediary for deployment to a EC2 instance. So far I get the feeling CodeDeploy is useful only for deployments involving EC2.
This tutorial with a similar requirement to mine, uses CodePipeline and CodeBuild, but the deployment step is actually a aws s3 sync command (same as I was doing on Gitlab), and the actual deployment step on CodePipeline is disabled.
I was looking into a solution which involves using AWS features made for this specific purpose, but I can't find any.
I'm also aware of LambCI, but to me looks like what CodePipeline / CodeBuild is doing, storing artifacts (not deploying to the root folder of the bucket). Plus, I'm looking for an option which doesn't require me to learn or deploy new configuration files (outside AWS config files).
Is this possible with the current state of AWS features?
Today AWS has announced as a new feature the ability to target S3 in the deployment stage of CodePipeline. The announcement is here, and the documentation contains a tutorial available here.
Using your CodeBuild/CodePipeline approach, you should now be able to choose S3 as the deployment provider in the deployment stage rather than performing the sync in your build script. To configure the phase, you provide an S3 bucket name, specify whether to extract the contents of the artifact zip, and if so provide an optional path for the extraction. This should allow you to deploy your content directly to the root of a bucket by omitting the path.
I was dealing with similar issue and as far as I was able to find out, there is no service which is suitable for deploying app to S3.
AWS CodeDeploy is indeed for deploying code running as server.
My solution was to use CodePipeline with three stages:
Source which takes source code from AWS CodeCommit
Build with AWS CodeBuild
Custom lambda function which after successful build takes artifact from S3 artifact storage, unzip it and copies files to my S3 website host.
I used this AWS lambda function from SeamusJ https://github.com/SeamusJ/deploy-build-to-s3
Several changes had to be made, I used node-unzip-2 instead of unzip-stream for unziping artifict from s3.
Also I had to change ACLs in website.ts file
Uploading from CodeBuild is currently the best solution available.
There's some suggestions on how to orchestrate this deployment via CodePipeline in this answer.
I'm working all the day and couldn't find the answer. So I'm asking you guys: is it possible to use AWS Pipeline with AWS Lightsail?
My objective is to store the code inside CodeCommit and use CodeBuild, CodeDeploy, CodePipeline and S3 to create a Continuous Deployment inside a Lightsail instance.
Those are the steps I think I have to follow to accomplish the task:
[x] setup a Lightsail instance
[x] create an IAM user and set permissions
[x] transfer my repository to CodeCommit
[x] create an S3 bucket to hold the build artifacts
[x] create a CodeBuild project to build the artifacts
[x] create a buildspec.yml file with my build steps
[ ] create a CodeDeploy project to deploy my application
[ ] create a CodePipeline project to trigger the build when I commit to certain branch
As you can see, I'm almost there. But I couldn't find any way to use my Lightsail instance with CodeDeploy. So, my question is: is it possible? Is there some limitation? Did I miss something really basic? Is there any other way to make the CD with Lighsail? Sorry, I'm getting a little crazy right here ahhaha.
Today, 08/16/2017, it's not possible to integrate them.
I asked the same question on AWS forums and they replied that those technologies are not integrated yet since they are separated from each other.
Well I guess I'll have to find another way.
i’m not a total expert here, but I think the way to do it would be with a custom script in CodeBuild, rather than with CodeDeploy.
CodeDeploy has a lot of custom stuff going on to support rollbacks and that sorr of advanced stuff (means you have to install the agent on your target server etc).
CodeBuild is just made for running scripts, so I think it’d be reasonable to add a deploy script (that runs after your tests) that connects up to yor Lightsail instance via SSH and deploy any changed files (similar to how you’d do it in open source using Travis CI etc).
Specifically I’ve used the dploy package on npm to do the actual SFTP upload before. It’s Git-aware so it only uploads changes since the last revision (but you could just rsync if you didn’t care about that).
I recently had the same challenge and got it working.
It is necessary to register the Lightsail Instance as an on-premise instance with CodeDeploy. On the instance itself the CodeDeploy agent needs to be installed and configured.
I have written a post about how to set this up on my blog.
https://scratchpad.blog/howto/how-to-use-codedeploy-with-aws-lightsail/
Following those steps can help you deploy lightsail as an onpremises instance and you configure codedeploy to deploy to the onpremises instance
I have followed this guide:
https://blogs.aws.amazon.com/application-management/post/Tx33XKAKURCCW83/Automatically-Deploy-from-GitHub-Using-AWS-CodeDeploy
It mentions that it will push the default branch from GitHub.
What about all the other branches one might have in the same repo?
Can I somehow specify which branch to deploy?
Here is how you can accomplish branch-specific deploy scenarios using AWS Code Deploy and AWS CodePipeline:
Assuming you've already set up an application and deploy group with Code Deploy, create one group for your "Dev" branch, and another deploy group for "qa" or "stage".
Enable CodePipeline in your AWS Console.
Create a new pipeline by authorizing your Github account and providing access to the repository and branches you desire.
In the BETA section of your new pipeline, edit it, authorize github again, and choose the specific branch you wish to deploy when changes are made.
Now your system will automatically deploy based on a specific branch.
After scratching and cursing and researching and out of the box thinking...I managed to do it like this.
As long as CodeDeploy plays nicely only with the default branch, let's manipulate that one from the GitHUb API [you can do it also from the settings of the GH UI].
This is the code to change/update the default branch from your repo.
I have confirmed that CodeDeploy had no problem deploying the new branch! :]