I'm building an API using AWS API Gateway and AWS Lambda. I would like to achieve continuous delivery for this API. The path I've chosen to do it is to use CloudFormation through AWS CodePipeline. I've managed to to it for another project using Lambdas (without API Gateway), it works perfectly and it is really pleasant to use.
The issue I'm facing when deploying is that the Lambdas are properly updated but not the API definition. From what I understand, the AWS::ApiGateway::Deployment are immutable resources which means that for each deployment of the API I need to create a new AWS::ApiGateway::Deployment resource. This is not practical at all because for each of this AWS::ApiGateway::Deployment I have a new Invoke URL. This is not acceptable since I would have to either change my DNS record to the newly deployed API invoke URL or ask our API users to change the URL in their applications.
What I would like is to be able to change the API definition and the Lambdas implementations without my API users having to change anything in their applications.
How can I achieve this behavior?
I created a tutorial to highlight my issue. You can find it at: https://github.com/JonathanGailliez/aws-api-gateway-lambda-example
As per: https://forums.aws.amazon.com/thread.jspa?messageID=789869󀵭
joey-aws says:
We are currently in the process of rolling out a solution which
addresses this exact problem. In the meantime, a common workaround
would be to update something small, such as a "description" field
which could then be used to "trigger" an API Gateway deployment when
updating the CloudFormation stack.
I'll update this answer and the example repo once it's rolled out.
You could run a Cloudformation update from the command line or in the AWS console. This would change the API definitions and any lambda code without changing the unique id to access your gateway.
The other option is to put your API behind a custom domain name and then you could keep deploy a new API or stage and switch over the custom domain mapping when you are ready. The users wouldn't recognize any change.
A way to achieve that is by leveraging existing frameworks like
AWS SAM
Serverless
Claudia
I was able to achieve this by using CloudFormation template generated by troposphere and boto3 api in Python as follows:
Split the template into two parts
API definition, Method(s), IAM roles, ApiKey and Lambda (a)
Deployment, UsagePlan and UsagePlanKey (b)
Once changed Lambda code is zipped up and uploaded to S3 using boto3 api
Stack (b) is deleted
Stack (a) is updated with new resource id for the GET method connected to lambda
Stack (b) is created anew
Steps 3, 4, 5 are performed using CloudFormation boto3 api with blocking until completed.
Most importantly after all steps are done ApiKey value and stage Invoke URL remain the same, running updated Lambda code as tested with curl.
Note: it may take additional 30-60s for API to become fully functional after CloudFormation update is completed.
Related
I have a mobile back-end set up on as:
Apigateway-->[Cognitico authorizer]AWS Lambda --> DyanmoDb
I have one Environment setup for PROD
I want to create another environment for DEV and UAT. For this purposes I have created separate AWS account. My lambda code is easy to deploy and I can create DynamoDB tables manually.
API gateway was created via the web console. Now I could export the API via swagger but when I import into another account I get the error below:
Invalid authorizer ID specified. Setting the authorization type to CUSTOM or COGNITO_USER_POOLS requires a valid authorizer.
I could manually remove authorizers and import the swagger file but that seam like a hack.
So the questions:
1) Assuming having one account for PROD and another account for UAT is the right approach. What would proper/easier way to move API Gateway config from one AWS account to another?
2)Is there a better way to have multiple environments for kind of setup above? Any official recommendations?
I am playing around with the Cloudformation Serverless Transformation, and am trying to figure out how and when to use AWS::Serverless::Function or AWS::Lambda::Function. For some reason the Lambda version is used for SAM examples in AWS. This is very confusing. SAM github. And then there is also the servlerless framework itself which is really a nodejs framework. All this stuff is getting me extremely confused.
First of all, let me explain each of these.
AWS Serverless Stack - A combination of AWS offered serverless technologies including API Gateway, Lambda, S3 and etc.
When Lambda was initially introduced it was challenging to develop Serverless applications using these technologies using Cloudformation. The problem was due to the complexity of writing Cloudformation for API Gateway and Lambda.
This is where Serverless Framework came in.
Serverless Framework - An Open Source DevOps framework which simplifies defining API Gateway and Lambda using a simple file called serverless.yml. Since Serverless Framework uses conventions over configurations, it required only a few lines of code to define Lambda, API Gateway and etc. Underneath Serverless Framework generates Cloudformation based on whats defined in serverless.yml. In addition, Serverless Framework supported Multiple Cloud Providers.
Later AWS introduced their own simplified scripting language called AWS SAM(Note: AWS SAM is not a Framework like Serverless with plugins and extension support as of now) to reduce the complexity in defining CloudFormation as an alternative to Serverless Framework.
AWS::Lambda::Function is the Cloudformation syntax to define a Lambda function.
AWS::Serverless::Function is the AWS SAM syntax to define a Lambda function which internally creates a Lambda function in Cloudformation (AWS::Lambda::Function) and related resources by convention when executing AWS SAM.
AWS::Serverless::Function is for when it is used with AWS SAM, whereas AWS::Lambda::Function is for usage with plain Cloudformation. They are essentially both modelling Lambda functions, but it is just to different frameworks, that therefore have different names for more or less the same thing.
Conceptually there is no difference between a serverless or a Lambda function. Serverless is the generic term for what AWS calls Lambda (and API Gateway).
The serverless framework is then just a software project that builds upon serverless principles, and that can work with AWS Lambda (amongst others). It's really not much different from other frameworks such a Chalice or Zappa (both Python based serverless frameworks that run on AWS Lambda).
Framework wise there is not much difference between both however, there are minor difference technically as below
Serverless: It creates the IAM role, S3 Bucket, Cloudformation Template, create, update & deploy the code. Can Invoke the program from command.
One more important point, when we would like to delete the function and all associated services like S3, IAM role etc, that can be done using a single line command. In AWS Lambda, we need to manually remove each service one by one
AWS Lambda: It allows snippets of code to execute in response to triggers caused by activity from other AWS resources such as an AWS CloudWatch alarm, changes in a NoSQL table as DynamoDB, an upload event in an S3 bucket, etc.
AWS Lambda function has limits and it is recommended that you keep code size down and break up bigger function to smaller ones.
I am new to this whole serverless framework. I created my first serverless function as documented here https://www.npmjs.com/package/serverless#quick-start. Next when I do a "serverless invoke" it works. I am confused how this works, the questions I have around this are
There does not seem to be an API gateway created so how can it invoke?
There are also stages mentioned in the serverless.yml file, I'm not sure what these translate to
Any help on this is highly appreciated.
First of all the default code comes with the AWS template, is only having a Lambda function declared. Let me try to answer your questions inline.
There does not seem to be an API gateway created so how can it invoke?
Yes, since there is no API Gateway created, its not possible to invoke the Lambda through URLs. However, it is possible to invoke the Lambda using AWS CLI or SDKs which is what Serverless Framework is providing with "serverless invoke". To create a API Gateway, you need to add an event object to the function code as shown below.
functions:
hello:
handler: handler.hello
events:
- http:
method: get
path: hello
There are also stages mentioned in the serverless.yml file, I'm not
sure what these translate to
When you define a stage in serverless.yml file, after the deployment, it creates a stage in API Gateway including it in the API Gateway URL path as shown below.
https://your-api/<stage-you-defined>/resurce-methods
Note: that if you setup a custom certificate for API Gateway, then you have option to setup your own custom paths.
Also its important to note that, although API Gateway supported this feature to have different stages(e.g test, staging, production) of a Single API Gateway deployment, latest Serverless Framework doesn't use this feature. Instead when you define a new stages, it will deploy a whole new API Gateway with the new stage. Serverless Framework has the argument for separating the API Gateway and having a single stage to self-contain each stage for isolation.
You can attach an API gateway for invoking your lambda
Or
You can get event driven. Where your lambda gets invoked in response to some events like a new message in AWS SNS or when a new object gets created in S3
Or
You can have scheduled invocations using cloudwatch trigger events
For a comprehensive list of events that can invoke lambda see Invoking Lambda Functions
As documented in AWS Regions and Endpoints there are HTTPS endpoints for Lambda. For example, in the us-east-1 region the endpoint is https://lambda.us-east-1.amazonaws.com. This is how you're able to call a Lambda directly without the API gateway. API gateway can add additional functionality and puts a full HTTP protocol on top of a Lambda.
I would like to build API Gateway in automated way (using some kind of framework), but I cannot find ideal solution.
The problem is that every framework is using one lambda (proxy) for entire API Gateway. I'm intrested in one API and multiple lambdas (with different persmissions and node_modules).
I was trying:
- claudiajs
- severless
- dawson
Do we have other options or I need to create it by myself via CloudFormation?
Thanks in advance!
So all Lambda's on AWS have their own execution roles, regardless of framework. Under serverless (serverless docs) you can specify a role for each Lambda by following the example in that link.
You can also individually package functions see this forum thread for more info.
We are using Amazon Gateway API and currently we describe API endpoints manually through the web console.
Is it possible to create definition of API in some file(s) and import it?
Why do we need this:
We want every change in the API be reviewed (it's our development process) by other people.
In case if API is deleted or broken accidently, we want to be able to restore it easily.
By now the only solution I see is to write script, based on aws apigateway command line command that creates all resources and methods.
But probably there is a better way to do it?
Thanks!
P.S. It may partially overlaps with this question: exporting api definition from AWS api gateway.
I think the Swagger Importer feature of AWS API Gateway is what you are looking for: https://aws.amazon.com/about-aws/whats-new/2015/07/introducing-swagger-importer-easily-import-swagger-api-definitions-into-amazon-api-gateway/
You can export your API definition with API Gateway extensions into swagger format. Then you use the API Gateway API importer to import/update your API.
If you are backing your API Gateway endpoints with Lambda functions you may want to check out the serverless project https://github.com/serverless/serverless
Using this framework you have a JSON file that describes your endpoints and binds them to your Lambda code in the same project structure. The tool lets you deploy the endpoints or code from the command line. It also allows you to manage other AWS resources in a CloudFormation template in the same project structure and deploy it from the command line.