Is it possible to create a new REST api gateway using codepipeline? I already have a terraform script to create the pipeline, but I want to know if there is a way to create a pipeline that will take my script and propogate it from a dev environment api gateway to a test environment? I am trying to automate the pipeline to possibly run the script for me once the code is updated in a code commit stash
Any suggestions would be greatly appreciated.
To run a script from AWS CodePipeline you can use AWS CodeBuild action in one of your CodePipeline stages.
With CodeBuild you can specify the list of commands you want to run, like installing and running terraform.
Related
I have AWS Glue jobs setup to upload test data to our database. Uploading takes place only 1 time and no additional runs are required unless additions or changes are required on the test data. However, we have multiple environments where the upload needs to happen. One way is to deploy the jobs using CDK and manually run the jobs in each environment. Looking for pointers to automatically trigger a run when the jobs is either updated or created.
Use a CustomResource to invoke it via command line or the Glue SDK.
It is important to remember that CDK is not a deployment solution - it is an infrastructure as code solution. CDK does not actually do any deployment - the cdk deploy command is just a shortcut for sending the template to CloudFormation.
CDK is just a way to lay code over top the creation of the CloudFormation templates and give developers far more options. All it really does is generate a CloudFormation Template - everything else is window dressing.
As such, anything that has to happen after the CloudFormation template is synthed and deployed is not possible for CDK to interact with. You need to make a custom resource that can watch for Stack Updates and when the stack is done deploying, trigger whatever else you want.
Alternatively, this is a perfect use of CodePipeline - run your cdk in the the pipeline (either with a Synth and CodeDeploy stages or a single codeBuild that just runs cdk deploy) and then in a stage after it have a lambda that triggers your jobs.
I'm trying to figure out a way to come up with a CI/CD pipeline for CloudFormation. We use Cloudformation Console directly to deploy our infrastructure and app to the cloud.
Does anyone have any examples of how they have created a CI/CD pipeline using Jenkins or other types of CI tools to do some type of linting, CI, version control, and artifact deployment to Artifactory (or similar toolset)? I'd like to execute a pipeline once a new version of the cloud formation templates is uploaded to Artifactory.
You can always use CodePipeline.
see docs:
CodePipeline
CI:
I am using GitHub, so before i can merge a pull request, my code must pass 3 tests.
Those tests are 3 Codebuilds containers that run tests.
CD:
After my code merged it invoke a CodePipeline that use mainly CodeDeploy and CodeBuild.
About your goal:
I'd like to execute a pipeline once a new version of the cloud formation templates is uploaded to Artifactory.
I don't really think you need a pipeline for this.
Let assume your artifacts uploaded to s3 bucket called artifact-bucket.
You can create a CloudWatch rule that will execute StepFunctions state machine when file added to
artifact-bucket.
see docs:
Trigger StepFunctions from S3
You can easily deploy stack with StepFunctions.
Trying to create a simple deployment job on jenkins with the plugin post-build aws codedeploy.
The issue i'm facing is not able to target an s3 zip file as a deployment target. I don't want to upload nothing from codedeploy, just simply trigger a deployment from jenkins with a proper configuration ( bucket, region and of course the package.zip, which is already exists in the bucket )
Is there any "easy" way i can do that?
https://aws.amazon.com/blogs/devops/setting-up-the-jenkins-plugin-for-aws-codedeploy/
Current setup works as charmed, the deployment is triggered on aws but with wrong target file so the deployment fails at the moment. There is no chance to merge the build(and upload to s3) and deploy job together.
Switched to aws-cli for properly target a bucket for deployment. There is no way to use the plugin for situation like this
Instead of having build and deploy as two different stages you can have both in same stage where the jenkins job will checkout from the pipeline and codedeploy post job will automatically zip and store the revision in S3 actually this is the way I achieved it. But the best way is to use AWS cli.
I have jenkins installed on an AWS EC2 Instance. My end state is whenever I commit cloudformation templates to my bitbucket repo, jenkins will automatically create/update/delete cf stack.
My thoughts on it was via aws cf cli commands in the jenkinsfile after installing aws cli on the server. Is there a better way of approaching this? I am new to devops
You could try AWS Cloudformation Plugin, but it's up for adoption and wasn't updated in 3 years.
I would say your approach with using the AWS cf cli commands looks safer.
I would say using CLI commands in your Jenkins pipelines is a good practice.
I am a fan of setting up Jenkins pipelines using the S3 artifact manager so your pipeline artifacts like CF templates are automatically available from S3. From there just execute the CloudFormation stack in a Jenkins task.
If your hosting Jenkins in AWS it's also nice to just add an IAM role to the instance to control what API actions Jenkins is allowed to run and use a plugin like CloudBees AWS CLI for your pipeline tasks.
Is there any way to automate cloudformation templates deployment? I mean it would be awesome if I just push the changes in the code and somebody looking for those changes in the code and once they appear - deploy the updated template.
Yes, with AWS Code Pipeline !
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline.html
With AWS CloudFormation and AWS CodePipeline, you can use continuous delivery to automatically build and test changes to your AWS CloudFormation templates [...]
AWS CodePipeline has built-in integration with AWS CloudFormation, so you can specify AWS CloudFormation-specific actions, such as creating, updating, or deleting a stack, within a pipeline.
CodePipeline is a great way to do what you're looking for.
At Giftbit we do this is by having a Github Repo that has our CloudFormation template in it.
When we want to make a change, we make the changes on a branch in the repo, and create a pull request into the staging branch. CodePipeline monitors the staging branch then automates a CodeBuild to validate the templates, package any SubStacks, then creates a Change Set and Executes it.
Below are have some examples to help Quick Start anyone interested:
Continuous Integration CloudFormation Template Example
Serverless Application Model (SAM) that gets deployed