I have set up a lambda function using AWS Sam CLI that I am using for local development. After development, I have deployed this function to AWS console for production from my IDE (visual studio code). After deployment, when I see It has created several other resources such as cloudformation, Api Gateway, and a few others.
What's problem?
I am seeking a way through which I can deploy only my Lambda code that doesn't create other resources like Api gateway, etc. Is there any way that allow me to only create lambda function on local environment and then I want to push my code to AWS console.
Moreover, when I use AWS Sam the size of my Lambda code also increased incredibly. When I created the same Lambda manually on AWS Console it consumes only a few kbs but when I created Lambda using AWS Sam it's size ramped up to 25MB.
If someone know a better way to do this please elaborate.
You can see my concerns the following:
Create Lambda function on local machine for development
I don't want to shift my Lambda function manually from local environment to AWS Console.
Also assign the Lambda function with specific permissions
What are the best practices for this? If someone is still confuse please ask anything in the comment section.
Use local devtools which simplify cloud infrastructure local development and automate the process of deployment.
Check for example Altostra local devtools:
VS Code extension to build AWS infrastructure in visual way, including configuration of each AWS resource (extension is available on official VS Code marketplace)
CLI (available on official npm marketplace) - automatically package all your code and cloud infrastructure and push it to your AWS account. After push you can also deploy your project directly from your local dev environment to AWS - Altostra automatically generates CloudFormation from your visual design and deploy the stack to your AWS account. All permissions are generated automatically as well.
Notice, you need to open an account on Altostra to be able to do all described in #2 (account is free).
You can upload your application to aws lambda in 3 way :
1- Create a zip file and upload via console (project files can not exceed 250mb)
2- Upload your files to s3 and reference it (doable)
3- Create docker images and upload it (the easiest way)
The best way is to upload as container images because you can upload files/dependencies up to 10gb inside the docker image.
After you create your docker images, you can test it locally too. Please check :
https://docs.aws.amazon.com/lambda/latest/dg/images-test.html
I'm learning SAM, and I created two projects.
The first one, example1, I created it from the AWS web console, by going to Lambda, Applications, and choosing this template:
After the wizard finishes creating the app, it looks like this:
I'm interested in the yellow-highlighted area because I don't understand it yet.
I tried to replicate this more or less manually by using sam init and created example2. It's easy to look at the template.yml it creates and see how the stuff in Resources are created, but how is the stuff in Infrastructure created.
When I deploy example2 with sam deploy --guided, indeed there's nothing in Infrastructure:
Given example2, how should I go about creating the same infrastructure as example1 had out of the box (and then changing it, for example, I want several environments, prod, staging, etc). Is this point and click in the AWS console or can it be done with CloudFormation?
I tried adding a permission boundary to example2, on of the things example1 has in Infrastructure, I created the policy in IAM (manually, in the console), added it to the template.yml, and deployed it but it didn't show up in "Infrastructure".
Part 1: In which I answer your question
Where are these infrastructure entries coming from in AWS SAM?
I replicated your steps in the Lambda console to create a "Serverless API Backend" called super-app. When you press create, AWS creates
two CloudFormation Stacks, each with a YAML template. You can view the stack resources and the YAML templates in the CloudFormation console under Stacks > Templates Tab.
super-app: the "Resources" stack with the lambda and dynamo resources you managed to replicate.
serverlessrepo-super-app-toolchain: the mystery stack with the "Infrastructure" CI/CD resources1.
Is this point and click in the AWS console or can it be done with CloudFormation?
Yes and Yes. You can use sam deploy (or aws cloudformation deploy) to update the stacks. Or point and click.
Example: update the serverlessrepo-super-app-toolchain template with the SAM CLI:
# compile
sam build -t cicd_template.yaml --region us-east-1 --profile sandbox
# send changes to the cloud
sam deploy --stack-name serverlessrepo-super-app-toolchain --capabilities CAPABILITY_NAMED_IAM --region us-east-1 --profile sandbox
You must pass in values for the template parameters at deploy-time. The current values for the parameters are in the console under CloudFormation > Stack > Parameters Tab. You can pass them using the --parameter-overrides param in the deploy command. If the
parameters are static, I find it easier to pass SAM parameter values in samconfig.toml, which sam deploy will use by default:
# samconfig.toml
version = 0.1
[default]
[default.deploy]
[default.deploy.parameters]
# template default parameters - fill in the template blanks
# Where do the values come from? the CloudFormation console, Parameters tab
AppId = "super-app"
AppResourceArns = "arn:aws:lambda:us-east-1:1xxxxxx:function..."
ConnectionArn = "arn:aws:codestar-connections:us-east-1:xxxxxx:connection/xxxx3c5c-f0fe-4eb9-8164-d3c2xxxxx6e2"
GitHubRepositoryOwner = "mygithuborg"
RepositoryName = "super-app"
SourceCodeBucketKey = "sample-apps/nodejs/14.x/javascript/sam/web-backend.zip"
SourceCodeBucketName = "prodiadstack-subsystemsn-apptemplatesbucket03axxx-96eem3xxxxxx"
UseCodeCommit = false
If there were changes made in the template, they will deploy. Success!
Part 2: In which I try to convince you to use the CDK instead
SAM and YAML templates are far from dead, but I think it's safe to say that for proficient developers starting out with AWS, the newer AWS Cloud Development Kit is a natural first choice for ambitious applications that need CI/CD and testing. For most of us, editing a 800-line YAML file is not a fun experience.
AWS Infrastructure-As-Code
There are lots AWS and 3rd Party IaaC tools to deploy infra on AWS. Each abstraction is best for somebody sometime. The important thing to remember is that no matter what higher-level IaaC toolset you use, it ends up being deployed as a CloudFormation template. Here are the AWS approaches, oldest to newest:
CloudFormation YAML2 templates
The OG, all-powerful, lowest-level approach is to hand-code YAML templates. The Cfn template reference
docs are indespensible no
matter what tool you use, because that's what gets deployed.
SAM YAML templates
With AWS SAM, you
still handcode YAML, but less3. A SAM template is a superset of CloudFormation with some higher-level abstractions for the main serverless components like Lambdas, DynamoDB tables and Queues. The SAM CLI compiles the SAM template to Cfn. It has nifty features like local testing and deploy conveniences.
Cloud Development Kit
The newest, shiniest IaaC approach is the CDK, now on V2. With the CDK, we write Typescript/Python/Java/etc. instead of YAML. The CDK CLI compiles your language code to Cfn and deploys with cdk deploy. It has a bigger set of high-level infra abstractions that goes beyond serverless, and escape hatches to expose low-level Cfn constructs for advanced use cases. It natively supports testing and CI/CD.
AWS CDK workshop including testing and pipelines. Lots of AWS CDK example apps.
Note that CloudFormation is the ultimate soure of this info. The lambda console makes a cloudformation.DescribeStack API call to fetch it.
YAML or JSON
SAM also has a marketplace-like repository with reusable AWS and 3rd party components
Edit :
If I understand correctly, you want to reproduce the deployment on the SAM app. If that's the case, there is an AWS sample that covers the same approach.
It seems you are using either CodeStar/CodeCommit/CodePipeline/CodeDeploy/Code... etc. from AWS to deploy your SAM application on example1.
At deploy time, these resources under infrastructure are created by the "Code" services family in order to authorize, instantiate, build, validate, store, and deploy your application to CloudFormation.
On the other hand, on example2, whenever you build your project in your local machine, both instantiation, build, validation, storage (of the upload-able built artifacts) are leveraged by your own device, hence not needed be provisioned by AWS.
To shortly answer your question: No. Your can't recreate these infrastructure resources on your own. But again, you wouldn't need to do so while deploying outside of AWS' code services.
Is this possible to deploy AWS Lambda Containers without using SAM.
Every article I found on internet is suggesting to use SAM to deploy.
As SAM is a wrapper on AWS cloud formation I want to use only cloud formation YAML to deploy lambda containers.
As you already know, you need to create a docker image of your lambda application and then push to AWS docker registry which is ECR, now there are several ways to deploy your lambda:
1- use AWS console, go into lambda in GUI choose "container image" in the options and provide ECR link (No cloudformation will be used this way)
2- create a SAM template and then use AWS CLI or AWS Console again to setup a cloudformation and SAM will be compiled to cloudformation later in the process.
3- directly create a cloudformation template and then use CLI or AWS console to deploy your lambda
4- use CDK to do your deployment
And maybe many other choices and methods. now depending on what exactly you want to do, ask for more specific detail.
I am new to Serverless and AWS SAM.
I have tried creating a template.yaml for my new lambda functions and api gateway's of my application and deployed to AWS using AWS SAM. It worked fine.
But my question is for every deploy, we got to create new cloud formation stack but is there a way we can update the individual lambda codes for a function without deploying whole stack using AWS SAM?
I did check Serverless framework as well in which we can use below command to individually deploy lambda function without creating new stack
serverless deploy function -f functionName
Is there any such flexibility using SAM ?I did check SAM CLI which provides local environment for testing the api's and lambda functions, but haven't seen any individual deploy function for lambda. Could someone help me out?
I am working to extend this solution https://github.com/adieuadieu/serverless-chrome to my needs.
I am using serverless (on my laptop with Debian 9) to deploy it to AWS Lambda. I would like to use AWS-Sam-local https://github.com/awslabs/aws-sam-local to run it locally for developing.
I would like to use AWS-Sam-local because I believe that there is difference between running this solution via serverless webpack serve --function run and sam local start-api. The difference I think, is event object which I want to make contain POST or binary data (multipart files transfer). For that I have to allow binary transfer via API Gateway.
But correct me if I am wrong because I am totally green in the AWS and Serverless field and this is my first time with this technologies.
The problem I get is aws-sam-local needs the CloudFormation template to know how to run serverless-chrome project. If I make deploy to AWS and go to CloudFormation Console I can copy that template after selecting it in "Stacks" table and clicking "Template" tab. Then I use cfn-flip to convert JSON into YAML. In the end I got template.yml, but running sam local start-api gives me error:
2017/10/06 11:03:23 Connected to Docker 1.32
ERROR: No Serverless functions were found in your SAM template.
Please tell me what to do to make serverless-chrome run locally as it would run on AWS Lambda.
The templates Serverless uses to deploy are available in two places:
Remotely, in the S3 deployment bucket
locally, in .serverless/