When I'm in AWS --> Lambda > Functions > myFunctionName, whenever I edit the Code and then run the Test (clicking on Test button to Left of Deploy), it is running the test on the old (deployed) version of the code. This happens whether or not I File > Save first.
This is easy to confirm by just adding a console.log("blah"); and seeing that it does not appear in the Test output.
The Test > Execution Results also confirm the test is running on $LATEST (see bolded section):
Response
null
Function Logs
START RequestId: <snip> Version: $LATEST
<snip>
Of course I can test my version by deploying it, but isn't there any way to test BEFORE I deploy? (Sorry if this is an ignorant question - I feel I'm missing something dead obvious...)
Assuming you are doing it via AWS Console.
In the console, only LATEST is available for editing.
And for testing the function you need to deploy a version.
Lambda function versions
You can change the function code and settings only on the unpublished version of a function. When you publish a version, the code and most of the settings are locked to maintain a consistent experience for users of that version
You can test separately by publishing versions Lambda function versions
You can Lambda function aliases
for better handling these versions.
Configuring functions in the console
Related
edit: Turns out the solution is in the docs. I had bog standard normal 'sam' installed but I needed what they call the 'public preview version' AKA 'sam-beta-cdk'. With this installed the API can be started locally with sam-betacdk start-api and works well. While I appreciate the answers which suggest that development should be done using purely TDD I feel there is also value in this more interactive, manual mode as it permits quicker exploration of the problem space.
I'm trying to build my first app with CDK + Typescript using API Gateway, Lambdas and DynamoDB. I have built a couple of Lambdas and deployed them and they work fine live on the web. However I don't want a minute long deploy cycle and various associated AWS costs as part of my workflow. What I want is to be able to test my API locally.
I have struggled to find docs on how to do this. Amazon seem to recommend using the SAM CLI here so that is what I've been trying.
The docs claim running sam local xyz runs cdk synth to make a "could assembly" in ./aws-sam/build but I see no evidence of this. Instead what I get is a complaint that sam could not find a 'template.yml'. So I manually run cdk synth > template.yml which creates one in the root folder. Then I run sam local start-api and it seems happy to start up.
Then I try and hit my test lambda using CURL: curl 'http://127.0.0.1:3000/test' I get {"message":"Internal server error"} and a huge ugly stack trace in the console that is running sam local start-api
The lambda is this...
exports.handler = async function() {
console.log("WooHoo! Test handler ran")
return {statusCode: 200, headers: {"Content-Type": "application/json"}, body: "Test handler ran!"}
}
Start of the huge ugly stack trace...
Mounting /home/user/code/image-cache/asset.beeaa749e012b5921018077f0a5e4fc3ab271ef1c191bd12a82aa9a92148782e as /var/task:ro,delegated inside runtime container
START RequestId: 99f53642-b294-4ce5-a1b4-8c967db80ce1 Version: $LATEST
2021-09-15T12:33:37.086Z undefined ERROR Uncaught Exception {"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module 'test'\nRequire stack:\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js","stack":["Runtime.ImportModuleError: Error: Cannot find module 'test'","Require stack:","- /var/runtime/UserFunction.js","- /var/runtime/index.js"," at _loadUserApp (/var/runtime/UserFunction.js:100:13)"," at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)",
The end of the huge ugly stack trace...
Invalid lambda response received: Lambda response must be valid json
So it would seem sam local start-api can't find test and throws and error which means the API gateway doesn't get a valid 'lambda response'. So far this has not helped me chase down the problem :/ It certainly seems aware that test is a route, as trying to hit other endpoints gives the classic {"message":"Missing Authentication Token"} but it chokes hard trying to fulfill it despite me having both functions/test.ts and the compiled functions/test.js present.
I have the test route and handler defined in my CDK stack definition like so...
const testLambda = new lambda.Function(this, "testLambdaHandler", {
runtime: lambda.Runtime.NODEJS_14_X,
code: lambda.Code.fromAsset("functions"),
handler: "test.handler"
})
api.root
.resourceForPath("test")
.addMethod("GET", new apigateway.LambdaIntegration(testLambda))
I considered posting my template.yml but that is even longer than the big ugly error message so I haven't.
So I have three questions (well actually a million but I don't want to be too cheeky!)
Is this actually the canonical way of locally testing apps made with CDK
If so, where am I going wrong?
If not, what is the better/proper way?
Lambda handlers are just functions. They do not need any special environment to function - they are called at a specific point in the Lambda Invocation process, and provided an event (a json object) and a context (another json object)
You can (and should!) unit test them just like any other individual function in your language/testing framework.
As #Lucasz mentioned, you should rely on the fact that, if set up properly, API gateway and Lambda will interact the same way every time. Once you have run one end to end test and you know that the basics work, any further implementation can be done trough unit testing
There are numerous libraries for mocking AWS service calls in unit testing, and there are plenty of good practice work arounds for the services that are more difficult to mock (ie: its difficult to mock a Lambda call from inside another lambda - but if you wrap that lambda call in its own function, you can mock the function itself to return whatever you want it to - and this is good practice for testing as well!)
using jest, in a coded unit test, you can call the lambda handler, give it stubbed or mocked event json, as well as a context json (probably just blank as youre not using it) and the lambda handler will act just like any other function with two parameters you've ever written, including returning what you want it to return.
You must be doing something wrong with your file directory. Where is your index.js located? If you generate the template.json, is the directory correct?
Also in what directory do you execute the Sam local command?
The thing with testing your serverless application is you don't have to test your full application. You need to count on AWS that API gateway, dynamodb and lambda is perfectly working.
The only thing you need to test is the logic you implemented.
In here you make sure your function prints out something and returns a 200. That's all you have to do.
Look into 'jest' for testing js.
If you want to test cdk you should into https://docs.aws.amazon.com/cdk/latest/guide/testing.html
Also "running Aws locally" is not good practice. it's never the same as how it's running in real life aka the cloud. You use plugins for this, tools for that... Local is not the same as in the cloud.
If you have any more questions, feel free to ask.
Please point me to the documentation where AWS defines $LATEST of the lambda version, or suggest that it is how AWS uses a terminology without defining it.
Clearly AWS lambda documents say $LATEST is a version.
AWS Lambda Function Versions
If you haven't published a version of the selected function, the Versions panel lists only the $LATEST version.
Initially I thought it is like HEAD tag pointing to the latest commit in a Git branch. Git commit is immutable, but lambda version is not.
AWS Lambda Function Versions
A version can be published one which is persistent.
The system creates a new version of your Lambda function each time that you publish the function. ... When you publish a version, the code and most of the settings are locked to ensure a consistent experience for users of that version.
But also it can be unpublished that can be changed and saved.
You can change the function code and settings only on the unpublished version of a function.
So I guess is, since I could not find the definition by AWS, that $LATEST is a temporary snapshot of code+runtime+setting which someone has saved at some point in an AWS account, and has not yet been updated and saved.
I suppose it is like work-in-progress transient artifact in a local git repository. We would not know what the code is because someone may be saving it, and we never know what we are executing if we do not use a published, hence immutable, version as the qualifier at invocation.
And personally believe a version should be immutable. It should not be mutable like Lambda $LATEST. Version 1 as of March 19 3PM or version 1 as of March 20 1PM do not make any sense to me. Version 1 must be unique. If change is made, the it must be version 1.1 or 2.0.
Invoke
POST /2015-03-31/functions/FunctionName/invocations?Qualifier=Qualifier HTTP/1.1
Please advise if this is correct understanding, and we should not use $LATEST for invocation unless only one person is working on and testing.
And if it is so, I suppose having $LATEST serves no practical purpose in a proper engineering where we should be able to consistently identify a revision of a code and reproduce the same result if we re-do the same procedure on the revision.
Update
Currently I believe my confusion was because of the inconsistent and possibly wrong naming by AWS.
Version is immutable if published but mutable if not published in which case called $LATEST.
Version can be a snapshot of a lambda function but it can be a integer number.
$LATEST is pointing to the last saved lambda function which can be changed anytime, hence we cannot know for certain what we are executing or looking at if we use $LATEST. $LATEST I see could be different from what someone else sees. It looks to me working on $LATEST is working on a moving target.
AWS has so many strange "gotchas" like ECS capacity provider is immutable and cannot delete once created and if the ECS cluster is deleted, that capacity provider remains there and invisible, causing errors when trying to create a new capacity provider for a new ECS cluster with the same name (the bug which called "as designed" finally fixed though). I believe $LATEST is one of them.
Based on the comments, the following helped to clarify the issues:
When you refer to "version" in lambda you usually mean "published version" (e.g., 1, 2, 3) which is immutable. $LATEST is simply most recent "unpublished version" which you can change at will. You keep working/testing on $LATEST till you are satisfied with it. Once you are happy, you can "freeze" it by publishing it. At the time it becomes immutable version no. 1, for instance.
Since versions are immutable, aliases are normally used in your clients code. The aliases can be changed to point to different versions whenever you publish new version of your function. If versions were mutable, or could somehow be "unpublished", aliases would be rather pointless.
I would like to debug an AWS Lambda function written in C++ locally. Ideally, I want to be able to step through my code (like you do with GDB or any decent IDE). The way to do this with AWS Lambdas would usually include AWS SAM.
Idea 1, debug using SAM:
Unfortunately, SAM does not enable you do debug provided runtimes, since it "only" supports Node.js, Python and Go (compare documentation).
Question 1:
Is there a way to somehow attach GDB to the my compiled lambda function?
Idea 2, run using docker-lambda:
You can also start your function using lambci/docker-lambda. If you would like to get some debugging output, you could do strace docker run --rm -v my/build/dir:/var/task lambci/lambda:provided handler '{"some": "event"}'. This works, but is not really the quality of debugging that I am looking for.
Question 2:
Can I somehow attach a to a lambda function that is running in docker-lambda?
Idea 3: remote:
Be desperate and go for anything you can get (this is NOT what I wanna do).
Question 3:
If there is really no way to do this locally, how should I do it on AWS?
I have been trying to look at the code that is deployed in an aws lambda.
There is an existing go function that is running in the go lambda.
However, I am not able to. AWS docs says we can look at the code through the visual config view, where is this view? This is the screen that I see, where is the view to see the code?
Please help.
Or is it because we are using a go server, only the executable which is a binary is running in the lambda and hence we are not able to see the code?
Code inline is supported only for interpreted languages (js for example) and not compiled languages.
Beside below lamda limitation, It seems console lamda editor does not support go.
However the documentation suggest to use code star.
You can also get started with AWS Lambda Go support through AWS
CodeStar. AWS CodeStar lets you quickly launch development projects
that include a sample application, source control and release
automation. With this announcement, AWS CodeStar introduced new
project templates for Go running on AWS Lambda. Select one of the
CodeStar Go project templates to get started. CodeStar makes it easy
to begin editing your Go project code in AWS Cloud9, an online IDE,
with just a few clicks.
announcing-go-support-for-aws-lambda
Q: How do I create an AWS Lambda function using the Lambda console?
If you are using Node.js or Python, you can author the code for your
function using code editor in the AWS Lambda console which lets you
author and test your functions, and view the results of function
executions in a robust, IDE-like environment
lambda-faqs
Deployment package size
50 MB (zipped, for direct upload)
250 MB (unzipped, including layers)
3 MB (console editor)
lambda limits
lambda-go-how-to-create-deployment-package
Based on the code size the AWS code editor will display the code. Since the size of the code/package is large AWS Code editor can't display the same.
But you can download the package from AWS Lambda function using export.
Kindly follow the below steps:
Go to the lambda
Select the lambda
Click on Actions
Select Export function
You will get few options. Select Download deployment package.
I have a Flask app running as an AWS Lambda Function deployed with Zappa and would like to activate X-Ray to get more information for the different functions.
Activating X-Ray with Zappa was easy enough - it only requires adding this line in the zappa-settings.json:
"xray_tracing": true
Further, I installed the AWS X-Ray Python SDK and added a few decorators to some functions, like this:
#xray_recorder.capture()
When I deploy this as a Lambda function, it all works well. The problem is using the system locally, both when running tests and when running the Flask in a local server instead of as a lambda function.
When I use any of the functions that are decorated either in a test or through the local server, the following exception is thrown:
aws_xray_sdk.core.exceptions.exceptions.SegmentNotFoundException: cannot find the current segment/subsegment, please make sure you have a segment open
Which of course makes sense, because AWS Lambda handles the creation of segments.
Are there any good ways to deactivate capturing locally? This would be useful e.g. for running unit tests locally on functions that I would like to watch in X-Ray.
One of the feature request of this SDK is to have a "disabled" global flag so everything becomes no-ops https://github.com/aws/aws-xray-sdk-python/issues/26.
However, it still depends on what you are testing against. It's good practice to test what actually will be run on Lambda. You can set some environment variables so the SDK thinks it is running on Lambda.
You can see the SDK is looking for two env vars https://github.com/aws/aws-xray-sdk-python/blob/master/aws_xray_sdk/core/lambda_launcher.py. One is LAMBDA_TASK_ROOT set to true so it knows to switch to lambda-mode. The other one is _X_AMZN_TRACE_ID which contains the tracing context normally passed by lambda container.
If you just want to test non-XRay code you can set AWS_XRAY_CONTEXT_MISSING to LOG_ERROR so the SDK doesn't complain on context missing and simply give up capturing wrapped functions. This will run much less code path than mimic lambda behaviors. Ideally it would be better for the lambda local testing tool to be X-Ray friendly. Are you using https://github.com/awslabs/aws-sam-cli? There is already an open issue for this feature https://github.com/awslabs/aws-sam-cli/issues/217