I'm using CodePipeline to deploy my CloudFormation templates that contain Lambda functions as AWS::SAM::Functions.
The CodePipeline is triggered by a commit in my main branch on GitHub.
The Source Stage in the CodePipeline retrieves the source files from GitHub. Zero or more Lambda functions could change in a commit. There are several Lambda Functions in this repository.
I intend on running through taskcat for CloudFormation Templates and Unit Tests for Lambda Python code during a test stage and then deploy the CloudFormation templates and Lambda Functions to production. The problem is, I can't figure out how to differentiate between changed and unchanged Lambda functions or automate the deployment of these Lambda functions.
I would like to only test and deploy new or update changed Lambda functions along with my CloudFormation templates - what is the best practice for this (ideally without Terraform or hacks)?
Regarding testing: Best practice is actually to simply test all lambda code in the repo on push before deploying. You might skip some work for example with github actions that you only test the files that have changed, but it definitely takes some scripting and it hardly ever adds much value. Each testing tool has its own way of dealing with that (sometimes you can simply pass the files you want to test as an argument and then its easy, but sometimes test tools are more of a all-or-nothing approach and it gets quite complicatedreal fast).
Also, personally I'm not a big fan of taskcat since it doesn't really add a lot of value and it's not a very intuitive tool (also relatively outdated IMO). Is there a reason you need to do these types of testing?
Regarding deployment: There are a few considerations when trying to only update lambdas that have changed.
Firstly, cloudformation already does this automatically: as long as the cloudformation resource for the lambda doesn't change, the lambda will not be updated.
However, SAM has a small problem there, since it will re-package the lambda code on every pipeline run and update the CodeUri property of the lambda. And thus the lambda gets updated (even though the code might stay the same).
To work around this, you have several options:
Simply accept that SAM updates your function even though the code might not have changed.
Build SAM locally, and use the --cached and --cache-dir option when deploying in your pipeline. Make sure to push the folder that you set as cache-dir.
Use a different file packaging tool than SAM. Either some custom script that or something else that only pushes your code to s3 when the files have changed.
If you're into programming I'd suggest you take a look into CDK. It's a major upgrade from cloudformation/SAM, and it handles code bundling better (only updates when files have changed). Also the testing options are much wider for CDK.
Related
tldr; I can deploy a single CFN stack as part of my pipeline, but how do I deploy multiple dynamic stacks?
An even better tldr; How would you do this? forEach BuildStage.artifact invoke CloudFormation.build
I am using CodePipeline in a pretty conventional way. The goal is to source control my CloudFormation templates, push them through the pipeline when a template changes, and then automatically deploy the stack.
Source Stage (CodeCommit commit my CFN yaml templates)
Build Stage (CodeBuild finds the new files from the commit, and pushes them to S3)
Deploy Stage (CloudFormation deploys my templates as CFN stacks)
Almost everything is working great. I commit my template changes to CodeCommit, the build stage runs my codeBuild gatekeeper, which gathers only the files that have changed, and uploads them to S3. So far so good.
The challenge is that sometimes I have one template change, and sometimes I have multiple(n). I can detect changed files and get them up to S3 no problem in my build stage. If I commit a change for one template, everything works fine. I can create an exported variable with my template location on S3, pass that to my deploy stage, and have the CloudFormation deploy action use that file as the template source. But how would I handle this if I have 2 templates?
I can't just create endless exported variables in my build stage.
And if I can, AFAIK there is no way to iterate over each entry for the deploy stage.
My thought is I would need to do one of the following:
Inside of my current buildspec (after I upload the files to S3), use the AWS CLI to invoke a CFN stack build. I can add this as part of a loop, so it iterates on each file to be uploaded. OR
After my build stage, use a Lambda to perform the same as #1. Loop through each file, and then use the CLI or SDK to invoke a CFN stack build.
Both of these options seem to defeat the purpose of the deploy stage altogether, which seems clunky to me.
Are there other options I am missing? What would you do?
Just want to answer my own question, in case anyone else is trying to figure out how to do this.
I ended up going with option 1...just doing a cli CFN deployment directly from within CodeBuild. I was really trying to shoehorn the idea of using a CodePipeline deploy stage, but this works just fine.
If anyone else ends up coming along with a better solution, I am all ears.
so we have a pipeline that zips code functions for our Lambda, uploads it to S3 and builds
the every lambda we have again with new version of zipped codes.
Now, the problem is, every single Lambda is being Built every pipeline run. even if there are no changes to other lambda code function. ex. (only 1 of 10 lambda has code change)
What would be the best approach or checking that we need to add in our pipeline in order to build the only Lambda that has code change? open for any suggestions even creating new pipeline and breaking this lambdas into pieces
I think the best way is to add a stage before the zipfile to see with files changes in the code in the last merge.
Simply take those names and check which lambda was affected.
Then it will pass the list of lambdas need to redeploy.
What we do at our company is have a single pipeline per lambda / repo. We have a few mono repos that when they deploy they deploy all the lambdas in that repo at once but still through a single pipeline. If you concerned about cost in the pipeline sticking around you could always delete them and then have another job to recreate them when you need to feploy a new change.
We've got everything done through cloud formation scripts so it's all simple scripts running here and there to create pipelines.
Curious what is the reason to have one pipeline deploy all 10 lambdas?
I am new to serverless framework, I used to create and write lambda codes through aws console. But I am little bit confused about the serverless framework structure. Should I need to create a serverless yml for each lambda function, Or I can use a single yml file for my whole aws lambda functions? I don't know which is the best way to start, because each api gateway end point will point to different lambda functions. Please suggest the best way to start.
My few cents here, based on some experience in the last 6 months. I started using a single Git repo with all lambda functions each in a folder. During the first 2 months, I had a single YML file to define all functions.
Then after some issues, I moved to separate YML files inside each folder for lambda function.
You can use plugins specific to each functions. In my use case, I had a NodeJS and GraphQL function for which I use Webpack for compressing the packages.
The Webpack didn't work for my other function which uses some pre-built binary for running specifically in AWS.
Not all functions need to be deployed every time. So managing in separate folders gave lot of flexibility in CI/CD and deployment version management. Otherwise, you will have releases for all functions.
I am trying to make a code pipeline which will build my branch when I make a pull request to the master branch in AWS. I have many developers working in my organisation and all the developers work on their own branch. I am not very familiar with ccreating lambda function. Hoping for a solution
You can dynamically create pipelines everytime a new pull-request has been created. Look for the CodeCommit Triggers (in the old CodePipeline UI), you need lambda for this.
Basically it works like this: Copy existing pipeline and update the the source branch.
It is not the best, but afaik the only way to do what you want.
I was there and would not recommend it for the following reasons:
I hit this limit of 20 in my region: "Maximum number of pipelines with change detection set to periodically checking for source changes" - but, you definitely want this feature ( https://docs.aws.amazon.com/codepipeline/latest/userguide/limits.html )
The branch-deleted trigger does not work correctly, so you can not delete the created pipeline, when the branch has been merged into master.
I would recommend you to use Github.com if you need a workflow as you described. Sorry for this.
I have recently implemented an approach that uses CodeBuild GitHub webhook support to run initial unit tests and build, and then publish the source repository and built artefacts as a zipped archive to S3.
You can then use the S3 archive as a source in CodePipeline, where you can then transition your PR artefacts and code through Integration testing, Staging deployments etc...
This is quite a powerful pattern, although one trap here is that if you have a lot of pull requests being created at a single time, you can get CodePipeline executions being superseded given only one execution can proceed through a given stage at a time (this is actually a really important property, especially if your integration tests run against shared resources and you don't want multiple instances of your application running data setup/teardown tasks at the same time). To overcome this, I publish an S3 notification to an SQS FIFO queue when CodeBuild publishes the S3 artifact, and then poll the queue, copying each artifact to a different S3 location that triggers CodePipeline, but only if there are are currently no executions waiting to execute after the first CodePipeline source stage.
We can very well have dynamic branching support with the following approach.
One of the limitations in AWS code-pipeline is that we have to specify branch names while creating the pipeline. We can however overcome this issue using the architecture shown below.
flow diagram
Create a Lambda function which takes the GitHub web-hook data as input, using boto3 integrate it with AWS pipeline(pull the pipeline and update), have an API gateway to make the call to the Lambda function as a rest call and at last create a web-hook to the GitHub repository.
External links:
https://aws.amazon.com/quickstart/architecture/git-to-s3-using-webhooks/
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codepipeline.html
Related thread: Dynamically change branches on AWS CodePipeline
I have a serverless framework service with (say)five aws lambda functions using python. By using github I have created a CodePipeline for CI/CD.
When I push the code changes, it deploys all the functions even only function is changed.
I want to avoid the deployment of all functions and the CI/CD should determine the changed function and deploy it. Rest of functions should not be deployed again.
Moreover, is there anyway to deal with such problems using AWS SAM, as at this stage I have an option to switch towards SAM by quitting serverless framework
Unfortunately there is no "native" way to do it. You would need to write a bash that will loop through the changed files and call sls deploy -s production -f for each one of them.
I was also faced this issue, and eventually it drove me to create an alternative.
Rocketsam takes advantage of sam local to allow deploying only changed functions instead of the entire microservice.
It also supports other cool features such as:
Fetching live logs for each function
Sharing code between functions
Template per function instead of one big template file
Hope it solves your issue :)