I'm learning SAM, and I created two projects.
The first one, example1, I created it from the AWS web console, by going to Lambda, Applications, and choosing this template:
After the wizard finishes creating the app, it looks like this:
I'm interested in the yellow-highlighted area because I don't understand it yet.
I tried to replicate this more or less manually by using sam init and created example2. It's easy to look at the template.yml it creates and see how the stuff in Resources are created, but how is the stuff in Infrastructure created.
When I deploy example2 with sam deploy --guided, indeed there's nothing in Infrastructure:
Given example2, how should I go about creating the same infrastructure as example1 had out of the box (and then changing it, for example, I want several environments, prod, staging, etc). Is this point and click in the AWS console or can it be done with CloudFormation?
I tried adding a permission boundary to example2, on of the things example1 has in Infrastructure, I created the policy in IAM (manually, in the console), added it to the template.yml, and deployed it but it didn't show up in "Infrastructure".
Part 1: In which I answer your question
Where are these infrastructure entries coming from in AWS SAM?
I replicated your steps in the Lambda console to create a "Serverless API Backend" called super-app. When you press create, AWS creates
two CloudFormation Stacks, each with a YAML template. You can view the stack resources and the YAML templates in the CloudFormation console under Stacks > Templates Tab.
super-app: the "Resources" stack with the lambda and dynamo resources you managed to replicate.
serverlessrepo-super-app-toolchain: the mystery stack with the "Infrastructure" CI/CD resources1.
Is this point and click in the AWS console or can it be done with CloudFormation?
Yes and Yes. You can use sam deploy (or aws cloudformation deploy) to update the stacks. Or point and click.
Example: update the serverlessrepo-super-app-toolchain template with the SAM CLI:
# compile
sam build -t cicd_template.yaml --region us-east-1 --profile sandbox
# send changes to the cloud
sam deploy --stack-name serverlessrepo-super-app-toolchain --capabilities CAPABILITY_NAMED_IAM --region us-east-1 --profile sandbox
You must pass in values for the template parameters at deploy-time. The current values for the parameters are in the console under CloudFormation > Stack > Parameters Tab. You can pass them using the --parameter-overrides param in the deploy command. If the
parameters are static, I find it easier to pass SAM parameter values in samconfig.toml, which sam deploy will use by default:
# samconfig.toml
version = 0.1
[default]
[default.deploy]
[default.deploy.parameters]
# template default parameters - fill in the template blanks
# Where do the values come from? the CloudFormation console, Parameters tab
AppId = "super-app"
AppResourceArns = "arn:aws:lambda:us-east-1:1xxxxxx:function..."
ConnectionArn = "arn:aws:codestar-connections:us-east-1:xxxxxx:connection/xxxx3c5c-f0fe-4eb9-8164-d3c2xxxxx6e2"
GitHubRepositoryOwner = "mygithuborg"
RepositoryName = "super-app"
SourceCodeBucketKey = "sample-apps/nodejs/14.x/javascript/sam/web-backend.zip"
SourceCodeBucketName = "prodiadstack-subsystemsn-apptemplatesbucket03axxx-96eem3xxxxxx"
UseCodeCommit = false
If there were changes made in the template, they will deploy. Success!
Part 2: In which I try to convince you to use the CDK instead
SAM and YAML templates are far from dead, but I think it's safe to say that for proficient developers starting out with AWS, the newer AWS Cloud Development Kit is a natural first choice for ambitious applications that need CI/CD and testing. For most of us, editing a 800-line YAML file is not a fun experience.
AWS Infrastructure-As-Code
There are lots AWS and 3rd Party IaaC tools to deploy infra on AWS. Each abstraction is best for somebody sometime. The important thing to remember is that no matter what higher-level IaaC toolset you use, it ends up being deployed as a CloudFormation template. Here are the AWS approaches, oldest to newest:
CloudFormation YAML2 templates
The OG, all-powerful, lowest-level approach is to hand-code YAML templates. The Cfn template reference
docs are indespensible no
matter what tool you use, because that's what gets deployed.
SAM YAML templates
With AWS SAM, you
still handcode YAML, but less3. A SAM template is a superset of CloudFormation with some higher-level abstractions for the main serverless components like Lambdas, DynamoDB tables and Queues. The SAM CLI compiles the SAM template to Cfn. It has nifty features like local testing and deploy conveniences.
Cloud Development Kit
The newest, shiniest IaaC approach is the CDK, now on V2. With the CDK, we write Typescript/Python/Java/etc. instead of YAML. The CDK CLI compiles your language code to Cfn and deploys with cdk deploy. It has a bigger set of high-level infra abstractions that goes beyond serverless, and escape hatches to expose low-level Cfn constructs for advanced use cases. It natively supports testing and CI/CD.
AWS CDK workshop including testing and pipelines. Lots of AWS CDK example apps.
Note that CloudFormation is the ultimate soure of this info. The lambda console makes a cloudformation.DescribeStack API call to fetch it.
YAML or JSON
SAM also has a marketplace-like repository with reusable AWS and 3rd party components
Edit :
If I understand correctly, you want to reproduce the deployment on the SAM app. If that's the case, there is an AWS sample that covers the same approach.
It seems you are using either CodeStar/CodeCommit/CodePipeline/CodeDeploy/Code... etc. from AWS to deploy your SAM application on example1.
At deploy time, these resources under infrastructure are created by the "Code" services family in order to authorize, instantiate, build, validate, store, and deploy your application to CloudFormation.
On the other hand, on example2, whenever you build your project in your local machine, both instantiation, build, validation, storage (of the upload-able built artifacts) are leveraged by your own device, hence not needed be provisioned by AWS.
To shortly answer your question: No. Your can't recreate these infrastructure resources on your own. But again, you wouldn't need to do so while deploying outside of AWS' code services.
So, first go at AWS LAmbda Containers (need to deploy a "big-ish" Lambda) and we use Serverless framework for all other Lambdas and I tried it for the container as well.
It all went fine and the Lambda was created with all the expected parameters following this blog/guide: https://www.serverless.com/blog/container-support-for-lambda
Of course I had messed up the code and forgotten a module so teh Lambda didn't run in AWS.
I added the module and did a re-deploy (sls deploy) from my laptop and it writes out everything as "success" like in half a second so it is clear it is not deploying anything (I am using the --force flag as well but no difference.
The only way to get it updated seems to be to alter some code and save it and Serverless will "detect" a change and redeploy (for real).
This will cause a problem in our DevOps deploy pipeline so any way of getting it redeploying through a parameter/command?
I am trying to deploy Lambda functions using AWS Cloud9. When I press deploy, all of my functions are deployed/synced at the same time rather than just the one I selected when deploying. Same thing when right clicking on the function and pressing deploy. I find this quite annoying and wondering if there is any work around?
When you click deploy Cloud9 runs aws cloudformation package and aws cloudformation deploy on your template.yaml file in the background. (source: I developed the Lambda integration for AWS Cloud9).
Because all your files are bundled into one serverless application and there is only one CloudFormation stack they can only be all deployed at once with CloudFormation.
If you're only making a code change for one function and are not modifying any configuration settings you could update that one function from the command line with the command:
zip -r - . | aws lambda update-function-code --function-name <function-name>`
Run this in the same folder as your template.yaml file, replacing <function-name> with it's full generated name like cloud9-myapp-myfunction-ABCD1234 (You can see the full name under the remote functions list in the AWS Resources panel).
In AWS Cloud9, Lambda functions are created within serverless applications and are therefore deployed via CloudFormation. With CloudFormation, the whole stack is deployed at once, so all functions are deployed together (see this discussion for more info).
I am working to extend this solution https://github.com/adieuadieu/serverless-chrome to my needs.
I am using serverless (on my laptop with Debian 9) to deploy it to AWS Lambda. I would like to use AWS-Sam-local https://github.com/awslabs/aws-sam-local to run it locally for developing.
I would like to use AWS-Sam-local because I believe that there is difference between running this solution via serverless webpack serve --function run and sam local start-api. The difference I think, is event object which I want to make contain POST or binary data (multipart files transfer). For that I have to allow binary transfer via API Gateway.
But correct me if I am wrong because I am totally green in the AWS and Serverless field and this is my first time with this technologies.
The problem I get is aws-sam-local needs the CloudFormation template to know how to run serverless-chrome project. If I make deploy to AWS and go to CloudFormation Console I can copy that template after selecting it in "Stacks" table and clicking "Template" tab. Then I use cfn-flip to convert JSON into YAML. In the end I got template.yml, but running sam local start-api gives me error:
2017/10/06 11:03:23 Connected to Docker 1.32
ERROR: No Serverless functions were found in your SAM template.
Please tell me what to do to make serverless-chrome run locally as it would run on AWS Lambda.
The templates Serverless uses to deploy are available in two places:
Remotely, in the S3 deployment bucket
locally, in .serverless/
Can anybody please guide me how to easily deploy code from github to aws using aws cpde deploy ? I gave tried my best to deploy my code and it is not working as it gives error every time it launch deploy revision.
You mentioned deployment is failing. So I'm assuming github automatically kicking off the deployment part is working. Now for the deployment failure, do you see the instance being deployed marked failed? Can you also look into the instance and see if the codedeploy agent is running fine and paste the log here if possible?
-Surya