Here is the thing, I have a serverless project that creates many AWS resources (Lambdas, API Gateway, etc), now I need to change the tags I used a couple of months ago, but when I try to run the serverless I see this message: " A version for this Lambda function exists ( 6 ). Modify the function to create a new version..". I have been reading and applying a couple of different workarounds but same issue.
Does any body have seen this behavior? Is there a way to retag all resources withouth delete the whole stack or doing that manually?
Thanks for your recommendations.
You can use a serverless plugin (serverless-plugin-resource-tagging). it will tag your Lambda function, Dynamo Tables, Bucket, Stream, API Gateway, and CloudFront resources. The way it works is you have to provide stacksTags having your tags inside under the Provider section of serverless.
provider:
stackTags:
STACK: "${self:service}"
PRODUCT: "Product Name"
COPYRIGHT: "Copyright"
You can also update tags value using this plugin.
Related
I am trying to understand the correct way to setup my project on AWS so that I ultimately get the possibility to have CI/CD on the lambda functions. And also to ingrain good practices.
My application is quite simple : an API that calls lambda functions based on users' requests.
I have deployed the application using AWS SAM. For that, I used a SAM template that was using local paths to the lambda functions' code and that created the necessary AWS ressources (API Gateway and Lambda). It was necessary to use local paths for the lambda functions because the way SAM works does not allow using existing S3 buckets for S3 events trigger (see here) and I deploy a Lambda function that is watching the S3 bucket to see any updated code to trigger lambda updates.
Now what I have to do is to push my Lambda code on Github. And have a way that Github pushes the lambda functions' code from github to the created S3 bucket during the SAM deploy and the correct prefix. Now what I would like is a way to automatically to that upon Github push.
What is the preferred way to achieve that ? I could not find clear information in AWS documentation. Also, if you see a clear flaw in my process don't hesitate to point it out.
What you're looking to do is a standard CI/CD pipeline.
The steps of your pipeline will be (more or less): Pull code from GitHub -> Build/Package -> Deploy
You want this pipeline to be triggered upon a push to GitHub, this can be done by setting up a Webhook which will then trigger the pipeline.
Last two steps are supported by SAM which I think you have already implemented before, so will be a matter of triggering the same from the pipeline.
These capabilities are supported by most CI/CD tools, if you want to keep everything in AWS you could use CodePipeline which also supports GitHub integration. Nevertheless, Jenkins is perfectly fine and suitable for your use case as well.
There are a lot of ways you can do it. So would depend eventually on how you decide to do it and what tools you are comfortable with. If you want to use native AWS tools, then Codepipeline is what might be useful.
You can use CDK for that
https://aws.amazon.com/blogs/developer/cdk-pipelines-continuous-delivery-for-aws-cdk-applications/
If you are not familiar with CDK and would prefer cloudformation, then this can get you started.
https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-github-gitclone.html
We are migrating REST API service from EC2 to Lambda/API Gateway (to lower billing) using the AWS - SAM feature. This service is consumed only by internal application(INTRANET). We don't have VPN connectivity between on-premise and AWS. Each function is housed in a separate folder which includes a YAML file template. When deployed using the same stack name it deletes the previous function. We tried to use,
DeletionPolicy: Retain
which errored out ,
'property DeletionPolicy not defined for resource of type
AWS::Serverless::Function'
Our requirement is, to have a common base URL without using R53 (if possible).
Is there a better way to do this?
CloudFormation attributes, such as DeletionPolicy, are not defined inside the Properties section. You may need to un-indent DeletionPolicy so it is not defined within the Properties section
I believe you don't need to retain the old lambda, it's fine to delete the old as you deploy new changes.
What needed is to tied the Lambda with API gateway.
In other words, your cloudformation template should have Lambda resources and for API Gateway point to the like
Uri: !Sub 'arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LambdaFunction.Arn}/invocations'
within AWS::ApiGateway::Method resources
I quoted my code block from AWS Samples on github
In case you have different cloud formation templates you might consider to Output section in cloudformation templates to export resources
I am trying to develop a serverless application which will use AWS SQS (Simple Queue Service), AWS SES (Simple Email Service) and AWS Lambda. The application will perform these steps:
get some messages in the SQS queue
trigger a Lambda function to handle all these messages
the Lambda will either send an email using AWS SES or an SMS using some 3rd party API, depending on the type of message
To test this out, I created the queue, lambda and configured SES, all manually using the web interface at https://aws.amazon.com . For the Lambda function, I simply typed my code in the web IDE provided at Lambda console. Since it was a very simple POC, it didnt need any testing and I got it to work.
Now, I want to turn this into a production ready application. My requirements:
code this entire application locally on my machine
test it locally in an environment similar to the one at AWS
publish the code to GitLab and then finally deploy it at AWS
all these resources (SQS queue, SES config, Lambda function) to be created automatically through code
Based on what I read online, I could find 3 different options for doing this:
AWS Cloudformation
AWS SAM
Serverless framework
My questions are:
For my use case, are serverless applications the correct technology or do I need something else like AWS SWF or AWS Step Functions? I also read about AWS Lambda applications. Are they something else?
Which is the best option among these in terms of cost, ease of setup and use? I checked that CloudFormation itself doesnt cost anything, you just have to pay for the services (SQS, SES, Lambda) being used but for Serverless, there are some costs involved for using the framework.
Are there some other options as well apart from these?
I will be using NodeJS for the code and only AWS as my cloud platform.
Short answer: yes, this is totally doable with Serverless functions and actually a typical Serverless use case.
Long answer:
It's not necessary to use AWS SWF or AWS Step Functions here. However, you could use Step Functions in case your process gets more complicated (e.g. more external services are involved and you need certain error handling, or you want to improve parallel processing powers).
First of all, CloudFront is not comparable to AWS SAM or Serverless Framework. Did you mean AWS CloudFormation instead? CloudFront is a CDN to serve (and cache) any kind of content whereas CloudFormation is a tool to describe your infrastructure as code.
CloudFormation is the "basis" for AWS SAM and Serverless Framework
because they both translate their template code to CloudFormation
code in the end. However, CloudFormation makes developing Serverless
Functions a bit complicated in my opinion. That's why tools like AWS
SAM or Serverless Framework popped up at some point. AWS SAM is
basically an extension of CloudFormation, i.e. it provides
additional resource types like AWS::Serverless::Function but
everything else is CloudFormation. Serverless Framework also lets
you add CloudFormation resources but has its own syntax for
specifying Serverless Functions.
In terms of costs, CloudFormation, AWS SAM, and Serverless Framework are all
free. However, you can use some premium features of Serverless Framework but you don't have to. However, CloudFront is not free to use - but I believe it wasn't the service you were looking for. Besides that, for SQS, SES and Lambda you only pay for what you use.
I personally prefer AWS SAM because you are closer to CloudFormation code and compared to Serverless Framework, you don't need a plugin for some things to circumvent the abstractions that the Serverless Framework does for you. You'll notice this for 'bigger' projects where you are leaving the standard hello world examples. On the other side, the Serverless Framework is quite popular and hence, there are many resources out there to help you. Up to you what you prefer :)
In terms of infrastructure tooling, you could have a look at AWS CDK (a good starting point is cdkworkshop.com) which is becoming more and more popular.
For local development, you can have a look at Localstack. The free version supports emulating SQS and SES locally, so that should be helpful.
I would like to automate setting up the collection of AWS Application Load Balancer logs using Sumo Logic as documented here:
https://help.sumologic.com/07Sumo-Logic-Apps/01Amazon_and_AWS/AWS_Elastic_Load_Balancer_-_Application/01_Collect_Logs_for_the_AWS_Elastic_Load_Balancer_Application_App
This involves creating a bucket, creating a Sumo Logic hosted collector with an S3 source, taking the URL of the collector source provided by Sumo Logic and then creating an SNS Topic with an HTTP subscription where the subscription URL is the one provided by the Sumo Logic source.
The issue with this is that the SumoLogic source URL is not known at synthesis time. The Bucket must be deployed, then the Sumlogic things created, then the SNS topic created.
As best I can figure, I will have to do this through separate invocations of CDK using separate stacks, which is slower. One stack to create the bucket. After deploying that stack, use the Sumo Logic api to create or affirm prior creation of the Sumo Logic hosted collector and source, another CDK deploy to create the SNS topic and HTTP subscription.
I was just wondering if anyone knew of a better way to do this, perhaps some sort of deploy time hook that could be used.
There are two ways(which I know of) in which you can automate the collection of AWS Application Load Balancer.
Using CloudFormation
Sumo Logic have a template that creates the Collection process for AWS Application Load Balancer which is part of the AWS Observability Solution. You can fork the repository and can create your own CloudFormation template after removing resources you do not require.
Sumo Logic also have a Serverless Application which auto enable Access logging for existing and new (which are created after application installation) load balancer. Example template which uses the application.
Using Terraform
As mentioned by Grzegorz, you can create a terraform script also.
Disclaimer: Currently employed by Sumo Logic.
You could try using a Custom Resource SDK Call to trigger a lambda that does what you want.
https://docs.aws.amazon.com/cdk/api/latest/docs/#aws-cdk_custom-resources.AwsSdkCall.html
(I know this is not a perfect answer as it suggests to use another tool, yet I believe it fulfills the needs expressed in the question)
How about using Terraform?
sumologic_s3_source in Terraform is able to create the source at Sumo AND output its URL for other uses within Terraform - e.g. to set up AWS resources.
The docs on this even mention URL being one of the returned values:
url - The HTTP endpoint to use with SNS to notify Sumo Logic of new
files.
Disclaimer: I am currently employed by Sumo Logic.
I've been working with CloudFormation YAML for awhile and have found it to be comprehensive - until now. I'm struggling in trying to use SAM/CloudFormation to create a Lambda function that is triggered whenever an object is added to an existing S3 bucket.
All of the examples I've seen thus far seem to require that you create the bucket in the same CloudFormation script as you create the Lambda function. This doesn't work for me, because we have a design goal to be able to use CloudFormation redeploy our entire stack to different regions or AWS accounts and quickly stand up our application. S3 bucket names must be globally unique, so if I create the bucket in CloudFormation, the script will break when I try to deploy it to a different region/account. I could probably get around this by creating buckets with the account name/region in the name, but that's just not desirable from a bucket sprawl perspective.
So, does anyone have a solution for creating a Lambda function in CloudFormation that is triggered by objects being written to an existing S3 bucket?
Thanks!
This is impossible, according to the SAM team. This is something which the underlying CloudFormation service can't do.
There is a possible workaround, if you implement a Custom resource which would trigger a separate Lambda function to modify the existing bucket and link it to the Lambda function that you want to deploy.
As "implement a Custom Resource" isn't very specific: Here is an AWS github repo with scaffold code to help write it, and then you declare something like the following in your template (where LambdaToBucket) is the custom function you wrote. I've found that you need to configure two things in that function: one is a bucket notification configuration on the bucket (saying tell Lambda about changes), the other is a Lambda Permission on the function (saying allow invocations from S3).
Resources:
JoinLambdaToBucket:
Type: Custom::JoinLambdaToExistingBucket
Properties:
ServiceToken: !GetAtt LambdaToBucket.Arn