I have followed this guide:
https://blogs.aws.amazon.com/application-management/post/Tx33XKAKURCCW83/Automatically-Deploy-from-GitHub-Using-AWS-CodeDeploy
It mentions that it will push the default branch from GitHub.
What about all the other branches one might have in the same repo?
Can I somehow specify which branch to deploy?
Here is how you can accomplish branch-specific deploy scenarios using AWS Code Deploy and AWS CodePipeline:
Assuming you've already set up an application and deploy group with Code Deploy, create one group for your "Dev" branch, and another deploy group for "qa" or "stage".
Enable CodePipeline in your AWS Console.
Create a new pipeline by authorizing your Github account and providing access to the repository and branches you desire.
In the BETA section of your new pipeline, edit it, authorize github again, and choose the specific branch you wish to deploy when changes are made.
Now your system will automatically deploy based on a specific branch.
After scratching and cursing and researching and out of the box thinking...I managed to do it like this.
As long as CodeDeploy plays nicely only with the default branch, let's manipulate that one from the GitHUb API [you can do it also from the settings of the GH UI].
This is the code to change/update the default branch from your repo.
I have confirmed that CodeDeploy had no problem deploying the new branch! :]
Related
I am setting up a CI&CD environment for a GCP project involves Cloud Run. While setting up everything via Terraform is pretty much straightforward, I cannot figure out how to update the environment when the code changes.
The documentation says:
Make a change to the configuration file.
But that couples the application deployment to terraform configuration, which should be responsible only for infrastructure deployment.
Ideally, I use terraform to provision the infrastructure, and another CI step to build and deploy the container.
Is there a best-practice here?
Relevant sources: 1.
I ended up separating Cloud Run service creation (which is still done in Terraform) and deployment to two different workflows.
The key component was to make terraform ignore the actual deployed image so that when the code deployment workflow is done, terraform won't complained that the Cloud Run image is different from the one it manages. I achieved this by setting ignore_changes = [template[0].spec[0].containers[0].image] on the google_cloud_run_service resource.
I am new to Terraform and building a CI setup. When I want to create a CodePipeline that is going to be connected to a GitHub repo, do I run specific commands inside my Terraform codebase that will reach out to AWS and create the CodePipeline config/instance for me? Or would I set this CodePipeline up manually inside AWS console and hook it up to Terraform after the fact?
do I run specific commands inside my Terraform codebase that will reach out to AWS and create the CodePipeline config/instance for me?
Yes, you use aws_codepipeline which will create new pipeline in AWS.
Or would I set this CodePipeline up manually inside AWS console and hook it up to Terraform after the fact?
You can also import existing resources to terraform.
I see you submitted this eight months ago, so I am pretty sure you have your answer, but for those searching that comes across this question, here are my thoughts on it.
As most of you have researched, terraform is infrastructure as code (IaC). As IaC it needs to be executed somewhere. This means that you either execute locally or inside a pipeline. A pipeline consists of docker containers that emulate a local environment and run commands for you to deploy your code. There is more to that, but the premise of understanding how terraform runs remains the same.
So to the magic question, Terraform is Code, and if you intend to use a pipeline, Jenkins, AWS, GitLab, and more, then you need a code repository to put all your code into. In this case, a code repository where you can store your terraform code so a pipeline can consume it when deploying your code. There are other reasons why you should use a code repository, but your question is directed to terraform and its usage with the pipeline.
Now the magnificent argument, the chicken or the egg, when to create your pipeline and how to do it. To your original question, you could do both. You could store all your terraform code in a repository (i recommend), clone it down, and locally run terraform to create your pipeline. This would be ideal for you to save time and leverage automation. Newbies, you will have to research terraform state files which is an element you need to backup in some form or shape once the pipeline is deployed for you.
If you are not so comfortable with Terraform, the GUI in AWS is also fine, and you can configure it easily to hook your pipeline into Github to run jobs.
You must set up Terraform and AWS locally on your machine or within the pipeline to deploy your code in both scenarios. This article is pretty good and will give you the basic understanding of setting up terraform
Don't forget to configure AWS on your local machine. For you Newbies using pipeline, you can leverage some of the pipeline links to get you started. Remember one thing, within AWS Codepipeine; you have to use IAM roles and not access keys. That will make more sense once you have gone through the first link. Please also go to youtube and search Terraform for beginners in AWS. Various videos can provide a lot more substance to help you get started.
Source code in my organization is managed in a GitHub repository. For now, our CI process uses AWS CodePipeline as follows:
Webhooks detect code changes in a specific git branch
The updated branch is then used the input for AWS CodeBuild
The finished build is deployed onto one of our staging environments using Elastic Beanstalk
Tests are run on the Elastic Beanstalk environment.
We want to add detection of new pull requests in our git repository. Whenever a new PR is created in our repo, we'd like to automatically trigger a build to an EB environment, through CodePipeline as above.
Our roadblocks:
Looking at the available settings for GitHub Webhooks in CodePipeline, we cannot find a way to specify that the pipeline's trigger should be a new PR.
In any case, the GitHub source for a CodePipeline must be a specific branch. We'd like PRs to be detected in any branch.
What would be the best approach here? I've seen some methods being discussed, but most of them appear to be on the cumbersome/high-maintenance side. If there's anything new in the AWS toolchain that makes this easy, it'd be cool to know.
Thanks!
The best approach to solving this problem seems to be creating a CodePipeline for each PR using a parameterized CloudFormation stack.
Essentially the steps are:
Define your CodePipeline using CloudFormation and have a parameter that identifies the environment - Prod, QA, PR_xyz etc.
Set up CodeBuild to trigger on any changes to your GitHub repository. When a new PR is created, have CodeBuild construct a new CodePipeline based on your CloudFormation template. Supply the name of the PR as the environment name when creating the CloudFormation stack.
Detailed steps are described here: https://moduscreate.com/blog/track-git-branches-aws-codepipeline/
I created a repo in Code Commit for a static s3 website
Then I created a CodePipeline and configured the code build part.
There I set the Build Spec file with the some basic commands:
build and then copy the files in the s3 bucket.
The third step the Code Deploy I'm not sure why it's needed.
When I run it it gets stuck for an hour.
I did disable it and the site was deployed just fine.
Am I missing something?
You can disable the CodeDeploy part if it is working fine for you. Or you can skip the CodeBuild step and use appspec.yml to deploy the static website onto S3.
You have to use either of the steps to make it work, you can't skip both the steps.
CodeDeploy part is present in the CodePipeline in case you need to deploy it on your EC2 fleet or Autoscaling Group after you have built the artifacts. If not needed, just skip it.
Codepipeline has three stages source->codeBuild->codeDeploy. According to Amazon you must use atleast two stages of the Codepipeline, You cannot skip the first stage (i.e source) but you choose any one or both from the remaining. For your use case source and CodeBuild stages are enough you don't need codedeploy. Just remove the codeDeploy stage.
I'm working all the day and couldn't find the answer. So I'm asking you guys: is it possible to use AWS Pipeline with AWS Lightsail?
My objective is to store the code inside CodeCommit and use CodeBuild, CodeDeploy, CodePipeline and S3 to create a Continuous Deployment inside a Lightsail instance.
Those are the steps I think I have to follow to accomplish the task:
[x] setup a Lightsail instance
[x] create an IAM user and set permissions
[x] transfer my repository to CodeCommit
[x] create an S3 bucket to hold the build artifacts
[x] create a CodeBuild project to build the artifacts
[x] create a buildspec.yml file with my build steps
[ ] create a CodeDeploy project to deploy my application
[ ] create a CodePipeline project to trigger the build when I commit to certain branch
As you can see, I'm almost there. But I couldn't find any way to use my Lightsail instance with CodeDeploy. So, my question is: is it possible? Is there some limitation? Did I miss something really basic? Is there any other way to make the CD with Lighsail? Sorry, I'm getting a little crazy right here ahhaha.
Today, 08/16/2017, it's not possible to integrate them.
I asked the same question on AWS forums and they replied that those technologies are not integrated yet since they are separated from each other.
Well I guess I'll have to find another way.
i’m not a total expert here, but I think the way to do it would be with a custom script in CodeBuild, rather than with CodeDeploy.
CodeDeploy has a lot of custom stuff going on to support rollbacks and that sorr of advanced stuff (means you have to install the agent on your target server etc).
CodeBuild is just made for running scripts, so I think it’d be reasonable to add a deploy script (that runs after your tests) that connects up to yor Lightsail instance via SSH and deploy any changed files (similar to how you’d do it in open source using Travis CI etc).
Specifically I’ve used the dploy package on npm to do the actual SFTP upload before. It’s Git-aware so it only uploads changes since the last revision (but you could just rsync if you didn’t care about that).
I recently had the same challenge and got it working.
It is necessary to register the Lightsail Instance as an on-premise instance with CodeDeploy. On the instance itself the CodeDeploy agent needs to be installed and configured.
I have written a post about how to set this up on my blog.
https://scratchpad.blog/howto/how-to-use-codedeploy-with-aws-lightsail/
Following those steps can help you deploy lightsail as an onpremises instance and you configure codedeploy to deploy to the onpremises instance