I am new to AWS and trying to create a pipeline for CICD. Stages involved in my pipeline are:
Source -> Codecommit
Build -> Codebuild project
Deploy using Cloudformation
I am able to complete the pipeline and deployment is successful. But I am struggling to implement a rollback procedure with this. How to deploy previous version without making a code revert in the repository? Any help regarding this?
I changed the pipeline configuration a bit and now I am able to deploy any version from history. Below is the solution:
Source -> Codecommit
Build -> Codebuild project
Deploy using CodeDeploy instead of Cloudformation
Now, deployment history can be triggered at any time. Pick the version from the history in the deployments under code deploy and retry deployment.
Unfortunately there is currently no rollback step for CodePipeline, traditionally people would rollback by reverting the change from their master branch (which is meant to represent the state of live).
If you're unable to do this revert, then you will need to manage the rollback either from a different service or different pipeline.
As you're using CloudFormation you could take a look at implementing Rollback Triggers which would monitor the status of an alarm. If the alarm fails then it could rollback and fail the pipeline.
Related
I have AWS Glue jobs setup to upload test data to our database. Uploading takes place only 1 time and no additional runs are required unless additions or changes are required on the test data. However, we have multiple environments where the upload needs to happen. One way is to deploy the jobs using CDK and manually run the jobs in each environment. Looking for pointers to automatically trigger a run when the jobs is either updated or created.
Use a CustomResource to invoke it via command line or the Glue SDK.
It is important to remember that CDK is not a deployment solution - it is an infrastructure as code solution. CDK does not actually do any deployment - the cdk deploy command is just a shortcut for sending the template to CloudFormation.
CDK is just a way to lay code over top the creation of the CloudFormation templates and give developers far more options. All it really does is generate a CloudFormation Template - everything else is window dressing.
As such, anything that has to happen after the CloudFormation template is synthed and deployed is not possible for CDK to interact with. You need to make a custom resource that can watch for Stack Updates and when the stack is done deploying, trigger whatever else you want.
Alternatively, this is a perfect use of CodePipeline - run your cdk in the the pipeline (either with a Synth and CodeDeploy stages or a single codeBuild that just runs cdk deploy) and then in a stage after it have a lambda that triggers your jobs.
I'm trying to figure out a way to come up with a CI/CD pipeline for CloudFormation. We use Cloudformation Console directly to deploy our infrastructure and app to the cloud.
Does anyone have any examples of how they have created a CI/CD pipeline using Jenkins or other types of CI tools to do some type of linting, CI, version control, and artifact deployment to Artifactory (or similar toolset)? I'd like to execute a pipeline once a new version of the cloud formation templates is uploaded to Artifactory.
You can always use CodePipeline.
see docs:
CodePipeline
CI:
I am using GitHub, so before i can merge a pull request, my code must pass 3 tests.
Those tests are 3 Codebuilds containers that run tests.
CD:
After my code merged it invoke a CodePipeline that use mainly CodeDeploy and CodeBuild.
About your goal:
I'd like to execute a pipeline once a new version of the cloud formation templates is uploaded to Artifactory.
I don't really think you need a pipeline for this.
Let assume your artifacts uploaded to s3 bucket called artifact-bucket.
You can create a CloudWatch rule that will execute StepFunctions state machine when file added to
artifact-bucket.
see docs:
Trigger StepFunctions from S3
You can easily deploy stack with StepFunctions.
Background: I am using github actions for CI and aws codepipeline for CD. Github actions pushes a set of versioned artifacts to S3 and ECR. I setup my AWS codepipeline using CDK.
Question: how do I get the codepipeline to pick up those artifacts and deploy them?
opt 1: Just tag your images and everything else with "latest"
answer: no, having a pipeline that always deploys the latest is not the same as a pipeline that deploys version X.
opt 2: Just send the version number (version X) to codepipeline so that codepipeline knows which artifacts to fetch
answer: no, codepipeline seems to support passing variables between actions (actions generate output variables, other action can pick them up), but I have found no documentation stating that a codepipeline can be triggered with input parameters.
opt 3: tag your commit in github and use a webhook to pass that information along to codepipeline.
answer: no, codepipeline can filter webhooks so that you can trigger the pipeline for certain events, but it does not support parsing the webhook body, picking out stuff you want to use.
opt 4: resolve the version number in cdk synth before that pesky critter tries to update itself.
answer: yeah, that kinda works, I can query an ecr repo, find that actual version number of the release and regenerate the pipeline so that it points to the resolved version. Its not the same as passing a version number from github to codepipeline, but at least my pipeline is versioned and all my deployment units (like ECS services, batch jobs, etc) are pointing to an explicit version after deployment. Unfortunately, this has several drawbacks, like making the deployment pipeline (even) slow(er) and if the pipeline fails I will have update the pipeline by running cdk deploy from my machine.
opt 5: you come in to save the day :-)
Source code in my organization is managed in a GitHub repository. For now, our CI process uses AWS CodePipeline as follows:
Webhooks detect code changes in a specific git branch
The updated branch is then used the input for AWS CodeBuild
The finished build is deployed onto one of our staging environments using Elastic Beanstalk
Tests are run on the Elastic Beanstalk environment.
We want to add detection of new pull requests in our git repository. Whenever a new PR is created in our repo, we'd like to automatically trigger a build to an EB environment, through CodePipeline as above.
Our roadblocks:
Looking at the available settings for GitHub Webhooks in CodePipeline, we cannot find a way to specify that the pipeline's trigger should be a new PR.
In any case, the GitHub source for a CodePipeline must be a specific branch. We'd like PRs to be detected in any branch.
What would be the best approach here? I've seen some methods being discussed, but most of them appear to be on the cumbersome/high-maintenance side. If there's anything new in the AWS toolchain that makes this easy, it'd be cool to know.
Thanks!
The best approach to solving this problem seems to be creating a CodePipeline for each PR using a parameterized CloudFormation stack.
Essentially the steps are:
Define your CodePipeline using CloudFormation and have a parameter that identifies the environment - Prod, QA, PR_xyz etc.
Set up CodeBuild to trigger on any changes to your GitHub repository. When a new PR is created, have CodeBuild construct a new CodePipeline based on your CloudFormation template. Supply the name of the PR as the environment name when creating the CloudFormation stack.
Detailed steps are described here: https://moduscreate.com/blog/track-git-branches-aws-codepipeline/
Is there any way to automate cloudformation templates deployment? I mean it would be awesome if I just push the changes in the code and somebody looking for those changes in the code and once they appear - deploy the updated template.
Yes, with AWS Code Pipeline !
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline.html
With AWS CloudFormation and AWS CodePipeline, you can use continuous delivery to automatically build and test changes to your AWS CloudFormation templates [...]
AWS CodePipeline has built-in integration with AWS CloudFormation, so you can specify AWS CloudFormation-specific actions, such as creating, updating, or deleting a stack, within a pipeline.
CodePipeline is a great way to do what you're looking for.
At Giftbit we do this is by having a Github Repo that has our CloudFormation template in it.
When we want to make a change, we make the changes on a branch in the repo, and create a pull request into the staging branch. CodePipeline monitors the staging branch then automates a CodeBuild to validate the templates, package any SubStacks, then creates a Change Set and Executes it.
Below are have some examples to help Quick Start anyone interested:
Continuous Integration CloudFormation Template Example
Serverless Application Model (SAM) that gets deployed