I'm writing a lambda (in node.js 6.10) to update an endpoint SageMaker. To do so I have to create a new HyperParamterTuningJob (and then describe it).
I succeeded to call all functions of the service SageMaker from the sdk (like listModels, createTrainingJob, ...) (https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SageMaker.html) except some of them.
All the functions that are related to HyperParameterTuningJob
(createHyperParameterTuningJob, describeHyperParameterTuningJob, listHyperParameterTuningJobs and stopHyperParameterTuningJob)
are not known in the sdk by the lambda.
I have attached the policy 'AmazonSageMakerFullAccess' to the role IAM used (where all these functions are allowed). So the error can't come from a problem of authorization.
I have already created a HyperParameterTuningJob (by the interface of AWS) called 'myTuningJob'.
I have an error everytime I use the function describeHyperParamterTuningJob.
Here is my lambda code :
const AWS = require('aws-sdk');
const sagemaker = new AWS.SageMaker({region: 'eu-west-1', apiVersion: '2017-07-24'});
var role = 'arn:aws:iam::xxxxxxxxxxxx:role/service-role/AmazonSageMaker-ExecutionRole-xxxxxxxxxxxxxxx';
exports.handler = (event, context, callback) => {
var params = {
HyperParameterTuningJobName: 'myTuningJob'
};
sagemaker.describeHyperParameterTuningJob(params, function(err, data) {
if (err) console.log(err, err.stack);
else console.log(data);
});
};
When I try to test this code in AWS lambda, it returns this result in the console :
Function Logs:
START RequestId: 6e79aaa4-9a18-11e8-8dcd-d58423b413c1 Version: $LATEST
2018-08-07T08:03:56.336Z 6e79aaa4-9a18-11e8-8dcd-d58423b413c1 TypeError: sagemaker.describeHyperParameterTuningJob is not a function
at exports.handler (/var/task/index.js:10:15)
END RequestId: 6e79aaa4-9a18-11e8-8dcd-d58423b413c1
REPORT RequestId: 6e79aaa4-9a18-11e8-8dcd-d58423b413c1 Duration: 50.00 ms
Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 32 MB
RequestId: 6e79aaa4-9a18-11e8-8dcd-d58423b413c1 Process exited before completing request
When I call all other functions of the SageMaker service from the sdk, it runs correctly, whitout any error.
I don't find any explanation in the documentation of why these functions related to HyperParameterTuningJob are not recognized as functions in the sdk.
Does anyone have any idea of why it doesn't work ? Or any solutions to call theses functions ?
In AWS Lambda, only the sdk that have a sable version are available.
The sdk of the SageMaker service is not stable yet, so functions related to HyperParameterTuningJob are not in the version of the sdk included in AWS Lambda.
To use theses functions, you need to install the latest version of the sdk on local on your machine (with npm install aws-sdk).
Then zip the node_modules folder and your script (called index.js), then upload this zip folder into the AWS lambda.
Related
I am trying to deploy a simple containerized Express app using either GCE or Cloud Run.
It simply calls getSignedUrl with action set to 'read':
router.get('/', async function (req, res, next) {
const gs = new Storage();
const credentials = await gs.authClient.getCredentials();
log(`Client email: ${credentials.client_email}`);
const [url] = await gs.bucket(GS_BUCKET).file('0.txt').getSignedUrl({
version: 'v4',
action: 'read',
expires: Date.now() + 60_000,
contentType: 'text/plain',
});
res.render('index', { title: 'Express', ...credentials, url });
})
I set up my local development environment using the default service account for the project, as explained here.
Now, when I run it on my local machine, either directly (using Node.js) or in a container (using Docker), it works fine and generates a signed URL every time. When I try to build and deploy the container in the cloud (using Cloud Build + Cloud Run or GCE), however, I get the following error:
Error: The caller does not have permission
at Gaxios._request (/workspace/node_modules/gaxios/build/src/gaxios.js:130:23)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Compute.requestAsync (/workspace/node_modules/google-auth-library/build/src/auth/oauth2client.js:382:18)
at async GoogleAuth.signBlob (/workspace/node_modules/google-auth-library/build/src/auth/googleauth.js:721:21)
at async sign (/workspace/node_modules/#google-cloud/storage/build/src/signer.js:181:35) {
The client_email property is the same in both environments: 6***********-compute#developer.gserviceaccount.com i.e. the project's default service account, which seems to have the required permissions (as shown by the operation's succeeding on my local machine).
What could cause this error and how can I find out?
I use Visual Studio AWS extensions in order to create simple aws lambda function using docker image template.
Docker image has been built, tagged and finally pushed to the container registry. From there I create a new lambda function using container image. But when I test it (either locally using dotnet lambda invoke-function or using Test tool on AWS console I'm getting
Amazon Lambda Tools for .NET Core applications (5.3.0) Project Home:
https://github.com/aws/aws-extensions-for-dotnet-cli,
https://github.com/aws/aws-lambda-dotnet
Payload: {"Lower":"test","Upper":"TEST"}
Log Tail: START RequestId: 67ac52b9-8cd0-449b-9bc3-8584f31f9726
Version: $LATEST END RequestId: 67ac52b9-8cd0-449b-9bc3-8584f31f9726
REPORT RequestId: 67ac52b9-8cd0-449b-9bc3-8584f31f9726 Duration:
890.55 ms Billed Duration: 1224 ms Memory Size: 128 MB Max Memory Used: 61 MB Init Duration: 332.49 ms
This would be ok but I changed the internal implementation of returning my custom string instead of default one. It should return "my custom string test" instead of {"Lower":"test","Upper":"TEST"}
public async Task<string> FunctionHandler(string input, ILambdaContext context)
{
return "my custom string " + input;
}
Again, why the function returns default response over and over again.
Check whether your lambda points to the newly deployed image in AWS Console.
Depending on your deployment method, there is potential that your lambda was pointing to the first deployed image all the time. Your situation looks like you are using the old image all the time.
PS: both dotnet lambda invoke-function and using Test tool on AWS console are calling the deployed lambda, so it sounds reasonable that they give you the same
result.
I want to deploy AWS CDK stacks from with a Lambda function. The use case is to support similar functionality to that described in Trek10's Serverless CI/CD pipeline, where the relevant code is here.
In a nutshell, AWS CodePipelines only support listening to changes within a single Git branch. I want to listen to GitHub events relating to the creation of repos and branches and create CodePipeline instances in response to these events so that there is a Pipeline for each branch of each Git repository. I want a Lambda to listen to the GitHub events and create CDK stacks. The Trek10 example uses Python and calls CloudFormation directly. I'd like the Lambdas to be much simpler and use the CDK instead.
Here is a simple Lambda using Typescript that is derived from this AWS CDK issue:
// index.ts
import {ScheduledEvent} from 'aws-lambda';
import {CloudFormationDeploymentTarget, DEFAULT_TOOLKIT_STACK_NAME} from 'aws-cdk/lib/api/deployment-target';
import {CdkToolkit} from 'aws-cdk/lib/cdk-toolkit';
import {AppStacks} from 'aws-cdk/lib/api/cxapp/stacks';
import {Configuration} from 'aws-cdk/lib/settings';
import {execProgram} from "aws-cdk/lib/api/cxapp/exec";
import * as yargs from 'yargs';
import {SDK} from 'aws-cdk/lib/api/util/sdk';
export const handleCloudWatchEvent = async (event: ScheduledEvent): Promise<void> => {
try {
const aws = new SDK();
const argv = await yargs.parse(['deploy', '--app', 'bin/pipeline.js', '--staging', '/tmp', '--verbose', '--require-approval', 'never']);
const configuration = new Configuration(argv);
await configuration.load();
const appStacks = new AppStacks({
configuration,
aws,
synthesizer: execProgram,
});
const provisioner = new CloudFormationDeploymentTarget({ aws });
const cli = new CdkToolkit({ appStacks, provisioner });
const toolkitStackName = configuration.settings.get(['toolkitStackName']) || DEFAULT_TOOLKIT_STACK_NAME;
await cli.deploy({
stackNames: [],
exclusively: argv.exclusively as boolean,
toolkitStackName,
roleArn: argv.roleArn as string,
requireApproval: configuration.settings.get(['requireApproval']),
ci: true,
reuseAssets: argv['build-exclude'] as string[],
sdk: aws
});
return;
} catch (e) {
console.error(e);
return;
}
};
However, I get an error as described in the issue mentioned above:
ERROR { Error: Cannot find module '../package.json'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
at Function.Module._load (internal/modules/cjs/loader.js:562:25)
at Module.require (internal/modules/cjs/loader.js:692:17)
at new SDK (/var/task/node_modules/aws-cdk/lib/api/util/sdk.ts:92:39)
at Runtime.exports.handleCloudWatchEvent [as handler] (/resources/index.ts:204:21)
at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)
at process._tickCallback (internal/process/next_tick.js:68:7) code: 'MODULE_NOT_FOUND' }
I don't want to patch sdk.ts: https://github.com/aws/aws-cdk/blob/master/packages/aws-cdk/lib/api/util/sdk.ts#L92 as that seems to be a very dirty solution so I'm looking for another solution.
What is the simplest working example of calling the CDK from within a Lambda function?
Recently I had an ideal use case for a CDK deployer in Lambda but couldn't find any good and full examples of how to do this.
Using kadishmal example from CDK Github I managed to put CDK into Lambda layer, delete AWS SDK module from there (Lambda already has it), include it into a Lambda, and run CDK deploy/destroy from there. There is a size restriction for a Lambda and all its layers of 250mb so I also had to use Webpack to optimize the build size.
Here a Github repository with a basic example - a stack template that deploys S3 bucket and destroys it.
I created this repository which uses a 5 minute approach with Gradle & Docker to install CDK in a targeted location and mounts the Lambda Layer zipfile on your local system which you can use directly to manually upload in the AWS console or use i.e. with CDK.
An example Lambda handler function with NodeJS runtime and with the layer attached can look something like:
exports.handler = async (event) => {
const spawnSync = require('child_process').spawnSync;
const process = spawnSync('cdk', ['--version'], {
stdio: 'pipe',
stderr: 'pipe'
});
console.log(process.status);
console.log(process.stdout.toString());
};
I have deployed a gcloud background function (pubsub) in emulator.
It is getting succesfully invoked from command line
functions call helloPubSub --data='{"message":"Hello World"}'
How to invoke gcloud local function from local server code ?
= = =
Below is my code on server side to publish to topic
pubsub
.topic(topicName)
.publisher()
.publish(dataBuffer)
.then(results => {
const messageId = results[0];
console.log(`Message ${messageId} published.`);
res.status(200)
res.send({hello:'world'})
})
.catch(err => {
console.error('ERROR:', err);
res.status(200)
res.send({err:err})
});
I receive following error message
{"err":{"code":7,"metadata":{"_internal_repr":{}},"note":"Exception occurred in retry method that was not classified as transient"}}
In the official docs it states:
Note: Functions deployed in the Emulator with non-HTTP triggers like Cloud Storage or Cloud Pub/Sub will not be invoked by these services. The Emulator is intended as a development environment only, so you would need to call these functions manually to have them invoked.
So if you deployed a function locally with Cloud Pub/Sub trigger, the only way to invoke it is using the command line command:
functions call [your-function]
I have a pipeline stored as a template. I'm using the node.js client to run this pipeline from a cloud function. Everything works fine, but when I need to run this template from different regions I get errors.
According to the documentation, I can set it through the location parameter in the payload
{
projectId: 123,
resource: {
location: "europe-west1",
jobName: `xxx`,
gcsPath: 'gs://xxx'
}
}
That gives me the following error:
The workflow could not be created, since it was sent to an invalid regional endpoint (europe-west1).
Please resubmit to a valid Cloud Dataflow regional endpoint.
I get the same error if I move the location parameter out of resource node, such as:
{
projectId: 123,
location: "europe-west1",
resource: {
jobName: `xxx`,
gcsPath: 'gs://xxx'
}
}
If I set the zone in the environment and remove the location such as:
{
projectId: 123,
resource: {
jobName: `xxx`,
gcsPath: 'gs://xxx',
environment: {
zone: "europe-west1-b"
}
}
}
I do not get any errors anymore, but dataflow UI tells me the job is running in us-east1
How can I run this template and providind the region / zone I
As explained here, there are actually two endpoints:
dataflow.projects.locations.templates.launch (API Explorer)
dataflow.projects.templates.launch (API Explorer)
For Dataflow regional endpoints to work, the first one must be used (dataflow.projects.locations.templates.launch). This way, the location parameter in the request will be accepted. Code snippet:
var dataflow = google.dataflow({
version: "v1b3",
auth: authClient
});
var opts = {
projectId: project,
location: "europe-west1",
gcsPath: "gs://path/to/template",
resource: {
parameters: launchParams,
environment: env
}
};
dataflow.projects.locations.templates.launch(opts, (err, result) => {
if (err) {
throw err;
}
res.send(result.data);
});
I have been testing this though both the API explorer and the console using Google-provided templates. Using the wordcount example I get the same generic error than you do with the API explorer, which is the same if the location name is incorrect. However, the Console provides more information:
Templated Dataflow jobs using Java or Python SDK version prior to 2.0
are not supported outside of the us-central1 Dataflow Regional
Endpoint. The provided template uses Google Cloud Dataflow SDK for
Java 1.9.1.
Which is documented here as I previously commented. Running it confirms it's using a deprecated SDK version. I would recommend doing the same process to see if this is actually your case, too.
Choosing a different template, in my case the GCS Text to BigQuery option from the Console's drop-down menu (which uses Apache Beam SDK for Java 2.2.0) with location set to europe-west1 works fine for me (and the job actually runs in that region).
TL;DR: your request is correct in your first example but you'll need to update the template to a newer SDK if you want to use regional endpoints.