How to execute .jar file from AWS Lambda Serverless? - amazon-web-services

I have tried with following code.
var exec = require('child_process').execFile;
var runCmd = 'java -jar ' + process.env.LAMBDA_TASK_ROOT + '/src/' + 'myjar.jar'
exec(runCmd,
function (err, resp) {
if (err) {
cb(null, { err: err})
} else {
cb(null, { resp: resp})
}
)
Here, I have put my jar file in the root folder and src folder also.
but it is giving my following error. I have already added the.jar file with the code.but i got following error.
"err": {
"code": "ENOENT",
"errno": "ENOENT",
"syscall": "spawn java -jar /var/task/src/myjar.jar",
"path": "java -jar /var/task/src/myjar.jar",
"spawnargs": [],
"cmd": "java -jar /var/task/src/myjar.jar"
}
So How, Can I execute this .jar file in AWS Lambda environment?
Please help me.

With Lambda Layers you can now bring in multiple runtimes.
https://github.com/lambci/yumda and https://github.com/mthenw/awesome-layers both have a lot of prebuilt packages that you can use to create a layer so you have a second runtime available in your environment.
For instance, I'm currently working on a project that uses the Ruby 2.5 runtime on top of a custom layer built from lambci/yumbda to provide Java.
mkdir dependencies
docker run --rm -v "$PWD"/dependencies:/lambda/opt lambci/yumda:1 yum install -y java-1.8.0-openjdk-devel.x86_64
cd dependencies
zip -yr ../javaLayer .
upload javaLayer.zip to aws lambda as a layer
add layer to your function
within your function, java will be located at /opt/lib/jvm/{YOUR_SPECIFIC_JAVA_VERSION}/jre/bin/java

AWS Lambda lets you select a runtime at the time of creation of that lambda function, or later you can change it again.
So, as you are running the Lambda function with NodeJs runtime, the container will not have Java runtime available to it.
You can only have one type of runtime in one container in case of AWS Lambda.
So, Create a separate Lambda with the Jar file that you want to run having Java as the runtime and then you can trigger that lambda function from your current NodeJS lambda function if that's what you ultimately want.
Following is an example of how you can call another Lambda function using NodeJS
var aws = require('aws-sdk');
var lambda = new aws.Lambda({
region: 'put_your_region_here'
});
lambda.invoke({
FunctionName: 'lambda_function_name',
Payload: JSON.stringify(event, null, 2)
}, function(error, data) {
if (error) {
context.done('error', error);
}
if(data.Payload){
context.succeed(data.Payload)
}
});
You can refer to the official documentation for more details.

In addition to the other answers: Since 2020 December, Lambda supports container images: https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/
Ex.: I created a container image using AWS's open-source base image for python, adding a line to install java. One thing my python code did was execute a .jar file using a sys call.

Related

CDK Unit test using jest and NodeJsFunction

I faced issue when trying to writs units for my CDK project.
Stack is creating, pretty simple one:
-> APIGateway (Rest)
-> POST endpoint pointing to lambda
-> Lambda
I have very simple unit:
describe("Test WebhookProxyStack", () => {
it("template must be defined", () => {
const app = new cdk.App();
const processorStack = new WebhookProxyStack(app, "dev" as never);
const template = Template.fromStack(processorStack);
expect(template).toBeDefined();
});
});
Lambda code is
const lambda = new NodejsFunction(scope, name, {
runtime: Runtime.NODEJS_14_X,
handler: `handler`,
entry: require.resolve(
"#webhook-proxy/src/XXX.ts",
),
});
When I deploy CDK, everything is bundling (via local esbuild) fine, no errors, but when trying to run this unit I am getting error like:
Error: Failed to bundle asset WebhookProxy-dev/lambda-name/Code/Stage, bundle output is located at /private/var/folders/g_/7s34q40s3rg40280qhrvx5fm0000gn/T/cdk.outQhujdM/bundling-temp-1309d84e2e3633714893bccef1ab36748a1c6088468eb4607da3325bcd2d7058-error: Error: bash -c yarn run esbuild --bundle "{OUTPUT_PATH}" --target=node14 --platform=node --outfile="/private/var/folders/g_/7s34q40s3rg40280qhrvx5fm0000gn/T/cdk.outQhujdM/bundling-temp-1309d84e2e3633714893bccef1ab36748a1c6088468eb4607da3325bcd2d7058/index.js" --external:aws-sdk run in directory {PROJECT_PATH} exited with status 127
at AssetStaging.bundle (/Users/XXX/Sites/XXX/webhook-proxy/node_modules/aws-cdk-lib/core/lib/asset-staging.ts:395:13)
at AssetStaging.stageByBundling (/Users/XXX/Sites/XXX/webhook-proxy/node_modules/aws-cdk-lib/core/lib/asset-staging.ts:243:10)
at stageThisAsset (/Users/XXX/Sites/XXX/webhook-proxy/node_modules/aws-cdk-lib/core/lib/asset-staging.ts:134:35)
at Cache.obtain (/Users/XXX/Sites/XXX/webhook-proxy/node_modules/aws-cdk-lib/core/lib/private/cache.ts:24:13)
at new AssetStaging (/Users/XXX/Sites/XXX/webhook-proxy/node_modules/aws-cdk-lib/core/lib/asset-staging.ts:159:44)
at new Asset (/Users/XXX/Sites/XXX/webhook-proxy/node_modules/aws-cdk-lib/aws-s3-assets/lib/asset.ts:72:21)
at AssetCode.bind (/Users/XXX/Sites/XXX/webhook-proxy/node_modules/aws-cdk-lib/aws-lambda/lib/code.ts:180:20)
at new Function (/Users/XXX/Sites/XXX/webhook-proxy/node_modules/aws-cdk-lib/aws-lambda/lib/function.ts:348:29)
at new NodejsFunction (/Users/XXX/Sites/XXX/webhook-proxy/node_modules/aws-cdk-lib/aws-lambda-nodejs/lib/function.ts:50:5)
Any hint\help what could be wrong with bundling this lambda when units and how to pass it ?
Another question is why this lambda must be build when executing units?
Info:
AWS CDK v2
Esbuild error in test when building Lambda
Not a fix, but a diagnostic/workaround: pass bundling: {forceDockerBundling: true } in the lambda props to use Docker instead of esbuild for local bundling.
Why is the lambda built when executing units?
Stacks need to be synth-ed to be tested. CDK generates a hash for assets as part of the synth process. The source hash is "used at construction time to determine whether the contents of an asset have changed."

AWS lambda module not found error when debugging locally using SAM cli and AWS CDK?

I am trying to debug lambda function locally using SAM cli and AWS CDK. So I am getting error function module not found any idea why so? I have taken this project from github https://github.com/mavi888/cdk-serverless-get-started
function.js:
exports.handler = async function (event) {
console.log("request:", JSON.stringify(event));
// return response back to upstream caller
return sendRes(200, "HELLLOOO");
};
const sendRes = (status, body) => {
var response = {
statusCode: status,
headers: {
"Content-Type": "text/html",
},
body: body,
};
return response;
};
Inside lib folder
// lambda function
const dynamoLambda = new lambda.Function(this, "DynamoLambdaHandler", {
runtime: lambda.Runtime.NODEJS_12_X,
code: lambda.Code.asset("functions"),
handler: "function.handler",
environment: {
HELLO_TABLE_NAME: table.tableName,
},
});
I am using cdk synth > template.yaml command which generates cloud formation template.yaml file. Now I find function name with logicalID eg: myFunction12345678 and then trying to debug it locally using this command sam local invoke myFunction12345678 in my case it is DynamoLambdaHandler function. I get function module not found error. Any idea what I am missing?
Code is available on github: https://github.com/mavi888/cdk-serverless-get-started
The issue is that sam runs a Docker container with a Volume mount from the current directory. So, it's not finding the Lambda code because the path to the code from your CloudFormation template that CDK creates does not include the cdk.out directory in which cdk creates the assets.
You have two options:
Run your sam command with a defined volume mount sam local invoke -v cdk.out
Run the command from within the cdk.out directory and pass the JSON template as an argument since cdk writes a JSON template: sam local invoke -t <StackNameTemplate.json>
I'd recommend the latter because you're working within the framework that CDK creates and not creating additional files.

AWS CDK: run external build command in CDK sequence?

Is it possible to run an external build command as part of a CDK stack sequence? Intention: 1) create a rest API, 2) write rest URL to config file, 3) build and deploy a React app:
import apigateway = require('#aws-cdk/aws-apigateway');
import cdk = require('#aws-cdk/core');
import fs = require('fs')
import s3deployment = require('#aws-cdk/aws-s3-deployment');
export class MyStack extends cdk.Stack {
const restApi = new apigateway.RestApi(this, ..);
fs.writeFile('src/app-config.json',
JSON.stringify({ "api": restApi.deploymentStage.urlForPath('/myResource') }))
// TODO locally run 'npm run build', create 'build' folder incl rest api config
const websiteBucket = new s3.Bucket(this, ..)
new s3deployment.BucketDeployment(this, .. {
sources: [s3deployment.Source.asset('build')],
destinationBucket: websiteBucket
})
}
Unfortunately, it is not possible, as the necessary references are only available after deploy and therefore after you try to write the file (the file will contain cdk tokens).
I personally have solved this problem by telling cdk to output the apigateway URLs to a file and then parse it after the deploy to upload it so a S3 bucket, to do it you need:
deploy with the output file options, for example:
cdk deploy -O ./cdk.out/deploy-output.json
In ./cdk.out/deploy-output.json you will find a JSON object with a key for each stack that produced an output (e.g. your stack that contains an API gateway)
manually parse that JSON to get your apigateway url
create your configuration file and upload it to S3 (you can do it via aws-sdk)
Of course, you have the last steps in a custom script, which means that you have to wrap your cdk deploy. I suggest to do so with a nodejs script, so that you can leverage aws-sdk to upload your file to S3 easily.
Accepting that cdk doesn't support this, I split logic into two cdk scripts, accessed API gateway URL as cdk output via the cli, then wrapped everything in a bash script.
AWS CDK:
// API gateway
const api = new apigateway.RestApi(this, 'my-api', ..)
// output url
const myResourceURL = api.deploymentStage.urlForPath('/myResource');
new cdk.CfnOutput(this, 'MyRestURL', { value: myResourceURL });
Bash:
# deploy api gw
cdk deploy --app (..)
# read url via cli with --query
export rest_url=`aws cloudformation describe-stacks --stack-name (..) --query "Stacks[0].Outputs[?OutputKey=='MyRestURL'].OutputValue" --output text`
# configure React app
echo "{ \"api\" : { \"invokeUrl\" : \"$rest_url\" } }" > src/app-config.json
# build React app with url
npm run build
# run second cdk app to deploy React built output folder
cdk deploy --app (..)
Is there a better way?
I solved a similar issue:
Needed to build and upload react-app as well
Supported dynamic configuration reading from react-app - look here
Released my react-app with specific version (in a separate flow)
Then, during CDK deployment of my app, it took a specific version of my react-app (version retrieved from local configuration) and uploaded its zip file to S3 bucket using CDK BucketDeployment
Then, using AwsCustomResource I generated a configuration file with references to Cognito and API-GW and uploaded this file to S3 as well:
// create s3 bucket for react-app
const uiBucket = new Bucket(this, "ui", {
bucketName: this.stackName + "-s3-react-app",
blockPublicAccess: BlockPublicAccess.BLOCK_ALL
});
let confObj = {
"myjsonobj" : {
"region": `${this.region}`,
"identity_pool_id": `${props.CognitoIdentityPool.ref}`,
"myBackend": `${apiGw.deploymentStage.urlForPath("/")}`
}
};
const dataString = JSON.stringify(confObj, null, 4);
const bucketDeployment = new BucketDeployment(this, this.stackName + "-app", {
destinationBucket: uiBucket,
sources: [Source.asset(`reactapp-v1.zip`)]
});
bucketDeployment.node.addDependency(uiBucket)
const s3Upload = new custom.AwsCustomResource(this, 'config-json', {
policy: custom.AwsCustomResourcePolicy.fromSdkCalls({resources: custom.AwsCustomResourcePolicy.ANY_RESOURCE}),
onCreate: {
service: "S3",
action: "putObject",
parameters: {
Body: dataString,
Bucket: `${uiBucket.bucketName}`,
Key: "app-config.json",
},
physicalResourceId: PhysicalResourceId.of(`${uiBucket.bucketName}`)
}
});
s3Upload.node.addDependency(bucketDeployment);
As others have mentioned, this isn't supported within CDK. So this how we solved it in SST: https://github.com/serverless-stack/serverless-stack
On the CDK side, allow defining React environment variables using the outputs of other constructs.
// Create a React.js app
const site = new sst.ReactStaticSite(this, "Site", {
path: "frontend",
environment: {
// Pass in the API endpoint to our app
REACT_APP_API_URL: api.url,
},
});
Spit out a config file while starting the local environment for the backend.
Then start React using sst-env -- react-scripts start, where we have a simple CLI that reads from the config file and loads them as build-time environment variables in React.
While deploying, replace these environment variables inside a custom resource based on the outputs.
We wrote about it here: https://serverless-stack.com/chapters/setting-serverless-environments-variables-in-a-react-app.html
And here's the source for the ReactStaticSite and StaticSite constructs for reference.
In my case, I'm using the Python language for CDK. I have a Makefile which I invoke directly from my app.py like this:
os.system("make"). I use the make to build up a layer zip file per AWS Docs. Technically you can invoke whatever you'd like. You must import the os package of course. Hope this helps.

Amazon lambda nodejs elasticache store data in redis

I have a lambda function(runtime: Nodejs 4.3) and I need to store data in elasticache. The engine is redis. This is my function:
const redis = require("redis");
exports.handler = function (event, context, callback) {
callback(null, {});
};
Lambda returns "errorMessage": "Cannot find module 'redis'", error.
Should I add nom redis package in zip?
All npm modules needs to be installed in node_modules folder. You can use npm install --save package name command to install node_module and save its name in package.json. Refer this link for --save option explanation.
Zip the index.js and node_moule for upload. index.js file is your handler function, if your handler name is different, use that name for js file and zip file.
Note:- It is best to use node-lambda module for developing your code on local ec2 machine. Execute it with the node-lambda run and use node-lambda deploy for deploying the code over lambda. Zipping and all other things will be taken care by node-lambda.

Call aws-cli from AWS Lambda

is there ANY way to execute aws-cli inside AWS Lambda?
It doesn't seem to be pre-installed.
(I've checked with "which aws" via Node.js child-process, and it didn't exist.)
Now we can use Layers inside Lambda. Bash layer with aws-cli is available at https://github.com/gkrizek/bash-lambda-layer
handler () {
set -e
# Event Data is sent as the first parameter
EVENT_DATA=$1
# This is the Event Data
echo $EVENT_DATA
# Example of command usage
EVENT_JSON=$(echo $EVENT_DATA | jq .)
# Example of AWS command that's output will show up in CloudWatch Logs
aws s3 ls
# This is the return value because it's being sent to stderr (>&2)
echo "{\"success\": true}" >&2
}
Not unless you include it (and all of its dependencies) as part of your deployment package. Even then you would have to call it from within python since Lambda doesn't allow you to execute shell commands. Even if you get there, I would not recommend trying to do a sync in a Lambda function since you're limited to a maximum of 5 minutes of execution time. On top of that, the additional spin-up time just isn't worth it in many cases since you're paying for every 100ms chunk.
So you can, but you probably shouldn't.
EDIT: Lambda does allow you to execute shell commands
aws-cli is a python package. To make it available on a AWS Lambda function you need to pack it with your function zip file.
1) Start an EC2 instance with 64-bit Amazon Linux;
2) Create a python virtualenv:
mkdir ~/awscli_virtualenv
virtualenv ~/awscli_virtualenv
3) Activate virtualenv:
cd ~/awscli_virtualenv/bin
source activate
4) Install aws-cli and pyyaml:
pip install awscli
python -m easy_install pyyaml
5) Change the first line of the aws python script:
sed -i '1 s/^.*$/\#\!\/usr\/bin\/python/' aws
6) Deactivate virtualenv:
deactivate
7) Make a dir with all the files you need to run aws-cli on lambda:
cd ~
mkdir awscli_lambda
cd awscli_lambda
cp ~/awscli_virtualenv/bin/aws .
cp -r ~/awscli_virtualenv/lib/python2.7/dist-packages .
cp -r ~/awscli_virtualenv/lib64/python2.7/dist-packages .
8) Create a function (python or nodejs) that will call aws-cli:
For example (nodejs):
var Q = require('q');
var path = require('path');
var spawn = require('child-process-promise').spawn;
exports.handler = function(event, context) {
var folderpath = '/folder/to/sync';
var s3uel = 's3://name-of-your-bucket/path/to/folder';
var libpath = path.join(__dirname, 'lib');
var env = Object.create(process.env);
env.LD_LIBRARY_PATH = libpath;
var command = path.join(__dirname, 'aws');
var params = ['s3', 'sync', '.', s3url];
var options = { cwd: folderpath };
var spawnp = spawn(command, params, options);
spawnp.childProcess.stdout.on('data', function (data) {
console.log('[spawn] stdout: ', data.toString());
});
spawnp.childProcess.stderr.on('data', function (data) {
console.log('[spawn] stderr: ', data.toString());
});
return spawnp
.then(function(result) {
if (result['code'] != 0) throw new Error(["aws s3 sync exited with code", result['code']].join(''));
return result;
});
}
Create the index.js file (with the code above or your code) on ~/awscli_lambda/index.js
9) Zip everything (aws-cli files and dependencies and your function):
cd ~
zip -r awscli_lambda.zip awscli_lambda
Now you can simply run it as Docker container within lambda along with AWS CLI.
You can use the AWS node.js SDK which should be available in Lambda without installing it.
var AWS = require('aws-sdk');
var lambda = new AWS.Lambda();
lambda.invoke({
FunctionName: 'arn:aws:lambda:us-west-2:xxxx:function:FN_NAME',
Payload: {},
},
function(err, result) {
...
});
As far as I can tell you get most, if not all the cli functionality. See the full documentation here: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html
you can try this. I got this working for me.
1- add the AWS CLI layer
https://harishkm.in/2020/06/16/run-aws-cli-in-a-lambda-function/
2- add a lambda and run the following commands to run any AWS CLI command line.
https://harishkm.in/2020/06/16/run-bash-scripts-in-aws-lambda-functions/
function handler () {
EVENT_DATA=$1
DATA=`/opt/awscli/aws s3 ls `
RESPONSE="{\"statusCode\": 200, \"body\": \"$DATA\"}"
echo $RESPONSE
}
If you are provisioning your Lambda using code, then this is the most easiest way
lambda_function.add_layers(AwsCliLayer(scope, "AwsCliLayer"))
Ref: https://pypi.org/project/aws-cdk.lambda-layer-awscli/
I think that you should separate your trigger logic from the action.
Put a container with aws cli on another ec2 And use aws lambda to trigger that into an action.