I need to be able to watch for my TypeScript lambda function changes within my CDK app. I'm using SAM to locally invoke the API and do not want to deploy to the cloud each time changes happen. So using something such as SAM Accelerate, for example, is not an option.
Currently, I must run cdk build and sam local start-api manually each time I change a single line in my function code, and it painfully takes a long time to start.
Any solutions or workarounds for this?
You need a Typescript watch feature with a hook to run arbitrary post-compile commands.* Typescript's tsc --watch can't do it (open issue), but the tsc-watch package can:
tsc-watch --onSuccess "./start-api.sh"
tsc-watch will call start-api.sh after each each successful compile, synthing a sam-friendly template version and starting the local testing api:
# start-api.sh
STACK_NAME=MyStack
npx cdk synth $STACK_NAME -a 'ts-node ./bin/app.ts' --no-staging --no-validation --quiet --output cdk.local
sam local start-api --template cdk.local/$STACK_NAME.template.json
* cdk watch (an alias of cdk deploy --watch) won't work in your case, because you don't want to deploy on each change.
Related
thanks in advance for any help/guidance you could provide.
Context
I am currently using CDK in a project to create AWS resources (a few Lambda functions) and SAM to test locally, this works wonderfully but I'm struggling with environment variables to be used locally with my setup of CDK + SAM.
I run and test the project locally via the command
$ cdk synth --no-staging > template.yaml && sam local start-api
Deployments are done via
$ cdk deploy testStack123 --context secretToken=123
The issue came when I had to include (locally) a sensitive token required for my project and I couldn't figure out how to differentiate like how you would in a project, for example, that only uses AWS SAM where you can define:
your local environment variables via a env.json file
and your environment variables you want to use for deployment that you'd pass in via
$ sam deploy --stack-name=testStack123 --secretToken=123
What I tried?
Sam's --env-vars command such as:
$ sam local start-api --env-vars env.json
but since I'm not managing the template.yaml myself instead I'm relaying on CDK's synth command to output the CloudFormation, there is no way I can reliably reference the Lambda function names in the env.json to pass local environment variables via --env-vars env.json.
// env.json example
{
"TestLambdaFunction": { // Will fail as its referenced in template.yaml as TestLambdaFunction67CA3BED for example
"SECRET_TOKEN": "123"
"ENVIRONMENT": "test"
},
}
I tried the runtime context via cdk.json that the AWS team suggests for CDK envrionments but I noticed that it is also required to push the cdk.json file to the repo, so you always ran the risk of the dev not noticing that they’re accidentally staging and pushing sensitive tokens to the repo. However, this solution would work for both CI and local dev, but comes with the risk mentioned before.
Any advice on how to best solve this so I can make it so local environments can safely be passed via an (git)ignored filed such as env.json but actually work by referencing the Lambdas correctly that are emitted by the synthesised CloudFormation template cia cdk synth.
The generated Logical IDs like TestLambdaFunction67CA3BED are stable (unless you change the Construct ID or construct tree), so are generally fine to use in env.json Alternatively, you can place all the env vars under a Parameters key:
// env.json
{
"Parameters": {
"TABLE_NAME": "localtable",
"BUCKET_NAME": "testBucket",
"STAGE": "dev"
}
}
I am building a typescript project with AWS CDK and SAM.
I am trying to setup a backend where I have several lambda functions which share a common lib (which will ultimately talk to dynamo-db).
I created a very simple demo with one hello-world lambda importing from one common my-lib package (using yarn workspaces).
https://github.com/ziggy6792/aws-cdk-lambda-shared-package
I am using yarn workspaces to share my common my-lib library with my hello-world lambda
If I deploy this stack to AWS and run my hello-world lambda (by testing from the AWS console) it works! (It successfully imports my-lib does not error).
However I can't invoke my lambda function locally.
I have tried to use this method (which I found here) to mock locally (this method works fine when I don't import my common my-lib).
cdk synth --no-staging > template.yml (to find the logical lambda function id = HelloWorldLambda5A02458E)
sam local invoke HelloWorldLambda5A02458E
But I get an error
{"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module 'my-lib/MyLib'\nRequire stack:\n- /var/task/index.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js"}
It seems sam local invoke does not like my local dependency which is created as a symlink by yarn workspaces. If I replace the synlink my-lib with a hard copy of the my-lib package then the code runs fine locally. (But I don't want to so this every-time I want to run locally)
My question is
How can I invoke my hello-world lambda function locally?
Thanks a lot
is anyone aware of a method to execute post-deploy functionality. Follwing is a sample of a casual CDK app.
app = core.App()
Stack(app, ...)
app.synth()
What I am looking for is a way to apply some logic after template is deployed. The thing is the app completes before cdk tool starts deploying template.
thanks
Edit: CDK now has https://github.com/cdklabs/cdk-triggers, which allows calling Lambda functions before/after resource/stack creation
You can't do that from CDK at the moment. See https://github.com/awslabs/aws-cdk/issues/2849. Maybe add your +1 there, let them know you'd like to see this feature.
What you can do is wrap cdk deploy in a shell script that will run whatever you need after the CDK is done. Something like:
#!/bin/sh
cdk deploy "$#"
success=$?
if [ $success != 0 ]; then
exit $success
fi
run_post_deploy_with_arguments.sh "$#"
will run deploy with the given arguments, then call a shell scripts passing it the same arguments if deployment was successful. This is a very crude example.
Instead of wrapping the cdk deploy command in a bash script I find it more convenient to add a pre and post deployment script under a cdk_hooks.sh file and call it before and after the CDK deployment command via the cdk.json file. In this way you can keep using the cdk deploy command without calling custom scripts manually.
cdk.json
{
"app": "sh cdk_hooks.sh pre && npx ts-node bin/stacks.ts && sh cdk_hooks.sh post"
,
"context": {
"#aws-cdk/core:enableStackNameDuplicates": "true",
"aws-cdk:enableDiffNoFail": "true"
}
}
and cdk_hooks.sh
#!/bin/bash
PHASE=$1
case "$PHASE" in
pre)
# Do something
;;
post)
# Do something
;;
*)
echo "Please provide a valid cdk_hooks phase"
exit 64
esac
You can use CustomResource to run some code in a lambda (which you will also need to deploy unfortunately). The lambda will get the event of the custom resource (create, update delete), so you will be able to handle different scenarios (let's say you want to seed some table after deploy, this way you will be able to clean the data at a destroy for instance).
Here is a pretty good post about it.
Personally I couldn't find a more elegant way to do this.
Short answer: you can't. I've been waiting for this feature as well.
What you can do is wrap your deployment in a custom script that performs all your other logic, which also makes sense given that what you want to do is probably not strictly a "deploy thing" but more like "configure this and that now that the deploy is finished".
Another solution would be to rely on codebuild to perform your deploys and define there all your steps and which custom scripts to run after a deploy (I personally use this solution, with a specific stack to deploy this particular codedeploy project).
I created and built a new CDK project:
mkdir myproj
cd myproj
cdk init --language typescript
npm run build
If I try to run the resulting javascript, I see the following:
PS C:\repos\myproj> node .\bin\myproj.js
CloudExecutable/1.0
Usage:
C:\repos\myproj\bin\myproj.js REQUEST
REQUEST is a JSON-encoded request object.
What is the right way to run my app?
You don't need to run your CDK programs directly, but rather use the CDK Toolkit instead.
To synthesize an AWS CloudFormation from your app:
cdk synth --app "node .\bin\myproj.js"
To avoid re-typing the --app switch every time, you can setup a cdk.json file with:
{ "app": "node .\app\myproj.js" }
Note: A default cdk.json is created by cdk init, so you should already see it under C:\repos\myproj.
You can also use the toolkit to deploy your app into an AWS environment:
cdk deploy
Or list all the stacks in your app:
cdk ls
The CDK application expects a request to be provided as a positional CLI argument when you're using the low-level API (aka running the app directly), for example:
node .\bin\myproj.js '{"type":"list"}'
It can also be passed as a Base64-encoded blob instead (that can make quoting the JSON less painful in a number of cases) - the Base64 needs to be prefixed with base64: in this case.
node .\bin\myproj.js base64:eyAidHlwZSI6ICJsaXN0IiB9Cg==
In order to determine what are the APIs that are available, and what arguments they expect, you can refer to the #aws-cdk/cx-api specification.
Is it a good approach to commit the .elasticbeanstalk/config.yml inside the git repo of a project which uses eb deploy?
We want to deploy using our CI and so we can not use the interactive eb init.
What we are thinking now is to define our dev, uat and prod inside that config.yml (if possible) and to point to that environment using eb deploy.
We saw that we could perform eb init with all necessary parameters in ebcli version2 but not in version 3 anymore? So it seems the approach is changed?
Can someone explain how to deploy EB for multiple environments, without interaction?
We want to deploy using our CI and so we can not use the interactive eb init
You can suppress the interactive mode as follows:
eb init --platform <platform-name> --region <region-name> <application-name>
Is it a good approach to commit the .elasticbeanstalk/config.yml inside the git repo of a project which uses eb deploy?
Can someone explain how to deploy EB for multiple environments, without interaction?
By design, the EBCLI avoids committing the .elasticbeanstalk/ directory since it can contain developer-specific information, which when committed to VC can cause confusion. So, it's best avoided from VC. You are free to commit it to version control. Ensure there's no sensitive information here. Logs, and saved configurations are usually stored in .elasticbeanstalk/.
You can copy pertinent portions of the .elasticbeanstalk/config.yml file into root-level file from which CI could read information such as the environment name to use.
Locally, you could create a pre-commit Git hook that would read the default environment name from the .elasticbeanstalk/config.yml file into the root-level file -- let's call it .environment_config.sh. It could be a statement as simple as export BEANSTALK_ENVIRONMENT_NAME=<environment name from .elasticbeanstalk/config.yml>
On the CI server:
3.1. Ensure PWD is git init-ed. Systems such as Jenkins usually are git init-ed with the necessary branch, so CI can simply source .environment_config.sh at this point and load the name of the environment to deploy.
3.2. eb init --platform <platform-name> --region <region-name> <application-name>
3.3. eb use $BEANSTALK_ENVIRONMENT_NAME
3.4. eb deploy
(You could combine 3.3. and 3.4. by performing eb deploy $BEANSTALK_ENVIRONMENT_NAME instead; I just wanted to demonstrate the use of eb use)
The EB CLI is really meant to be used from a workstation. I think you'd be better off scripting your CI with the AWS CLI.
A deployment with eb deploy will archive your code in S3 (or CodeCommit), create a new application version then update the environment with the new version label. All of those operations are supported with AWS CLI commands.
Or, you could write your own deployment script in Python with boto3. That's an easy option too. That's basically what the EB CLI is.