503: Service Unavailable AWS Lambda with Github Probot - amazon-web-services

I'm trying to receive webhooks from Github using a probot application, but every single time I try this, I get a {"message":"Service Unavailable"} error.
Github sends this payload to an AWS Lambda function, and by googling (i think) this is an issue with not having enough nodes to handle the number of requests.
Either something is wrong with my code or there is an error with the configuration.
I'm using the serverless framework to upload to AWS lambda.
Here is the part where the code fails (no error messages in logs, and the bot just quits):
const yamlFile = async (context) => {
try {
console.log("trying to get yaml")
var yamlfile = await context.octokit.repos.getContent({
owner: context.payload.repository.owner.login,
repo: context.payload.repository.name,
path: ".bit/config.yml",
});
console.log(yamlfile)
} catch (e) {
console.log("Error with getting content of yaml");
console.log(e)
return null
}
console.log("got yaml, but no content yet");
yamlfile = Buffer.from(yamlfile.data.content, 'base64').toString()
console.log(yamlfile)
try {
let fileContents = yamlfile
configyml = yaml.load(fileContents);
} catch (e) {
const issueBody = context.issue({
title: "[ERROR] Please read",
body: `There was an issue parsing the config file of this course. Please contact your counselor and send them the below error.\n${e}`,
});
context.octokit.issues.create(issueBody)
console.log("ERROR: " + e);
return null
}
console.log("returining configyml")
return configyml;
}
The function yamlFile() is being called in our main function here:
let currentStep = ""
let configData = await data.yamlFile(context);
console.log(configData)
if (configData == null) {
console.log("null config data");
return
}
AWS Config
Timeout: 60 seconds
serverless.yml for Serverless framework:
service: <SERVICE NAME>
# app and org for use with dashboard.serverless.com
app: <APP NAME>
org: <ORG NAME>
frameworkVersion: "2"
useDotenv: true
provider:
name: aws
runtime: nodejs12.x
lambdaHashingVersion: 20201221
environment:
APP_ID: ${param:APP_ID}
PRIVATE_KEY: ${param:PRIVATE_KEY}
WEBHOOK_SECRET: ${param:WEBHOOK_SECRET}
NODE_ENV: production
LOG_LEVEL: debug
memorySize: 2048
functions:
webhooks:
handler: handler.webhooks
memorySize: 2048
events:
- httpApi:
path: /api/github/webhooks
method: post

Related

User is not authorized to perform: dynamodb:Query on resource error when accessing db from local lambda function

I am trying to run a newly created lambda function using SAM template with run time node.js locally.
I have the following :
a) aws account with region, accessKeyId & secretAccessKey
b) aws-cli, aws-sam, docker
Running lambda function locally using sam local invoke is fine, but the problem is when i used to connect to dynamodb in my function, getting the below error.
{"message":"User: arn:aws:iam::*********:user/test.user is not authorized to perform: dynamodb:Query on resource: arn:aws:dynamodb:us-west-2:********:table/my_table with an explicit deny","code":"AccessDeniedException","time":"2020-08-30T07:45:51.678Z","requestId":"7SIBTHTKDSSDJNSTLNUSSBHDNDHBSMVJF66Q9ASUAAJG","statusCode":400,"retryable":false,"retryDelay":40.84164538820391}
I have access to aws account and has accessKeyId & secretAccessKey and able to query when trying from aws console. In vscode editor, installed aws-toolkit and added the credentials, but still i am getting the same error in local.
Can somebody help me with this issue as i am going through a difficult situation in finding the solution.
Here is my code snippet.
let response;
const AWS = require('aws-sdk');
AWS.config.update({
region: "us-west-2",
accessKeyId: '************',
secretAccessKey: '********************'
});
const dbClient = new AWS.DynamoDB.DocumentClient();
exports.lambdaHandler = async (event, context) => {
try {
let myTable = await getData(event.myId);
response = {
'statusCode': 200,
'body': JSON.stringify({
message: myTable
})
}
} catch (err) {
console.log(err);
return err;
}
return response
};
const MY_TABLE_NAME = "my_table";
const getData = async (myId) => {
const params = {
TableName: MY_TABLE_NAME ,
KeyConditionExpression: "#uid = :id",
ExpressionAttributeValues: {
':id': myId
},
ExpressionAttributeNames: {
'#uid': 'userID'
}
};
let { Count, Items } = await dbClient.query(params).promise();
if (Items.length == 0) {
return false;
} else {
var obj = {
Name: (Count > 0) ? Items[0].name : null,
MY_TABLE: MY_TABLE
};
return obj;
}
};
template.yaml
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function # More info
Properties:
CodeUri: hello-world/
Handler: app.lambdaHandler
Runtime: nodejs12.x
Policies: AmazonDynamoDBFullAccess
Events:
HelloWorld:
Type: Api # More info
Properties:
Path: /hello
Method: get
Any help would be really appreciated. Thanks in advance.
The reason why i am getting user not authorized error was beacuase i donot have programmatic access, and only authorized to access aws console access.
Thanks

I get internal server error when i try to hit api endpoint from postman

I am new to aws.I have this code.
When i try with serverless offline,or when i test the api on the aws console it works great,
but when i hit the endpoint after successfully deploy in postman,i get internal server error message.
Also there are no logs in the cloud.
Please help
module.exports.hello = function(event, context, callback) {
const response = {
statusCode: 200,
body: JSON.stringify({ message: 'Hello World!' }),
};
callback(null, response);
};
My yml file
service: serverless-test
plugins:
- serverless-offline
provider:
name: aws
runtime: nodejs12.x
include:
- utils/**
functions:
hello:
handler: handler.hello
events:
- http:
path: message
method: get

AWS SAM Deployment Configuration Issue

I have set up a simple serverless rest api using Node JS and AWS Lambda.
The deployment is done using AWS SAM
Below is the SAM Template :
AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: Serverless API With SAM
Resources:
createUser:
Type: AWS::Serverless::Function
Properties:
Handler: handler.create
MemorySize: 128
Runtime: nodejs12.x
Timeout: 3
Events:
createUserApi:
Type: Api
Properties :
Path : /users
Method : post
listUsers:
Type: AWS::Serverless::Function
Properties:
Handler: handler.list
MemorySize: 128
Runtime: nodejs12.x
Timeout: 3
Events:
listUserApi:
Type: Api
Properties :
Path : /users
Method : get
This works fine but below is my observation regarding stacks created.
It creates two AWS Lambda functions instead of one.
Both contain two APIs listed as -
createUser
listUsers
Can it not contain only on Lambda function with these two handlers inside ?
handler.js file :
const connectedToDb = require('./src/db/db.js');
const User = require('./src/model/user.js');
module.exports.create = async (event,context) =>{
console.log('create function called !!');
try{
console.log('before connecting to DB ');
await connectedToDb();
console.log('after connecting to DB ');
const usr = new User(JSON.parse(event.body));
console.log('saving a user now with ',event.body);
await usr.save();
return{
statusCode : 200,
body:JSON.stringify(usr)
}
}catch(err){
return{
statusCode : err.statusCode || 500,
body : 'Cannot Create Order'
}
}
}
module.exports.list = async (event,context) => {
console.log('listing all users');
try{
await connectedToDb();
const users = await User.find({});
console.log('users are == ',users);
return {
statusCode:200,
body:JSON.stringify(users)
}
}catch(err){
return {
statusCode:err || 500,
body:JSON.stringify(err)
}
}
}

Get File From S3 bucket using Serverless and Node js

I am new to Serverless and AWS also. So My requirement is like. I need to download a file from S3. I have tried in many ways. I read many articles but could not find a proper document for this purpose.
So What I did was, I created a boilerplate using Serverless and got all the files like handler.js, serverless.yml
I need to know the correct steps to download a file from S3.
What I tried is.
Handler.js
const AWS = require('aws-sdk');
const S3= new AWS.S3();
exports.hello = async (event, context) => {
console.log(`Hi from Node.js ${process.version} on Lambda!`)
S3.getObject({Bucket: '*******', Key: '******'}).promise().then( data =>{
return {
statusCode: 200,
body: data
})
}
Serveless.yml
service: node11
custom:
bucket: *********
provider:
name: aws
runtime: provided # set to provided
stage: dev
region: us-east-1
iamRoleStatements:
- Effect: Allow
Action:
- s3:*
Resource: "arn:aws:s3:::${self:custom.bucket}/*"
functions:
hello:
handler: handler.hello
events:
- http:
path: /
method: get
layers: # add layer
- arn:aws:lambda:us-east-1:553035198032:layer:nodejs11:3
Whatever I did, I am always getting an error like INTERNAL SERVER ERROR.
What is the proper way to get the file from S3?
Try this:
const AWS = require('aws-sdk');
const S3= new AWS.S3();
exports.hello = async (event, context) => {
try {
console.log(`Hi from Node.js ${process.version} on Lambda!`);
// Converted it to async/await syntax just to simplify.
const data = await S3.getObject({Bucket: '*******', Key: '******'}).promise();
return {
statusCode: 200,
body: JSON.stringify(data)
}
}
catch (err) {
return {
statusCode: err.statusCode || 400,
body: err.message || JSON.stringify(err.message)
}
}
}
and ensure that in serverless.yml you've set runtime: nodejs8.10 under provider.
Lambda response body must be a string as defined here.

How to obtain AWS IOT endpoint URL from within a CloudFormation template?

I want some of my Lambda resources to push to an AWS IOT endpoint using aws-sdk's AWS.IotData({ endpoint: url }) function - where endpoint is a required parameter.
Right now, I am passing the endpoint URL via an environment variable to my Lambda. However, when put into a SAM/CF template, I can't find a way to retrieve my IOT endpoint URL, so that I could simply !Ref it.
Browsing through the AWS resource type reference I did not find any resource that corresponds to an IOT endpoint.
It seems like IOT endpoint can only be provisioned manually, via AWS Console (enabled / disabled), as on the screenshot below:
Any advice on how to have control over provisioning an IOT endpoint or at least reading the IOT URL from within a SAM/CF template, without scripting this with aws-cli?
For anyone interested in the solution with CloudFormation Custom Resource, I wrote a simple Lambda and a CF template that provides an IOT endpoint address to other CF stacks.
template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Resources:
IotEndpointProvider:
Type: 'AWS::Serverless::Function'
Properties:
FunctionName: IotEndpointProvider
Handler: iotEndpointProvider.handler
Runtime: nodejs6.10
CodeUri: .
MemorySize: 128
Timeout: 3
Policies:
- Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- iot:DescribeEndpoint
Resource:
- '*'
IotEndpoint:
Type: 'Custom::IotEndpoint'
Properties:
ServiceToken: !GetAtt IotEndpointProvider.Arn
Outputs:
IotEndpointAddress:
Value: !GetAtt IotEndpoint.IotEndpointAddress
Export:
Name: IotEndpointAddress
iotEndpointProvider.js
var aws = require("aws-sdk");
exports.handler = function(event, context) {
console.log("REQUEST RECEIVED:\n" + JSON.stringify(event));
// For Delete requests, immediately send a SUCCESS response.
if (event.RequestType == "Delete") {
sendResponse(event, context, "SUCCESS");
return;
}
const iot = new aws.Iot();
iot.describeEndpoint({}, (err, data) => {
let responseData, responseStatus;
if (err) {
responseStatus = "FAILED";
responseData = { Error: "describeEndpoint call failed" };
console.log(responseData.Error + ":\n", err);
} else {
responseStatus = "SUCCESS";
responseData = { IotEndpointAddress: data.endpointAddress };
console.log('response data: ' + JSON.stringify(responseData));
}
sendResponse(event, context, responseStatus, responseData);
});
};
// Send response to the pre-signed S3 URL
function sendResponse(event, context, responseStatus, responseData) {
var responseBody = JSON.stringify({
Status: responseStatus,
Reason: "See the details in CloudWatch Log Stream: " + context.logStreamName,
PhysicalResourceId: context.logStreamName,
StackId: event.StackId,
RequestId: event.RequestId,
LogicalResourceId: event.LogicalResourceId,
Data: responseData
});
console.log("RESPONSE BODY:\n", responseBody);
var https = require("https");
var url = require("url");
var parsedUrl = url.parse(event.ResponseURL);
var options = {
hostname: parsedUrl.hostname,
port: 443,
path: parsedUrl.path,
method: "PUT",
headers: {
"content-type": "",
"content-length": responseBody.length
}
};
console.log("SENDING RESPONSE...\n");
var request = https.request(options, function(response) {
console.log("STATUS: " + response.statusCode);
console.log("HEADERS: " + JSON.stringify(response.headers));
// Tell AWS Lambda that the function execution is done
context.done();
});
request.on("error", function(error) {
console.log("sendResponse Error:" + error);
// Tell AWS Lambda that the function execution is done
context.done();
});
// write data to request body
request.write(responseBody);
request.end();
}
I'm afraid you cannot provision an IoT endpoint, as the only API call that is related to an IoT endpoint is DescribeEndpoint.
What you can do is create a Lambda-backed CloudFormation Custom Resource. The Lambda function will execute the DescribeEndpoint call (using the AWS SDK of your choice depending on the Lambda's runtime) and return the endpoint's URL so your other CloudFormation resources can consume it.
Here's a good example on Lambda-backed Custom Resources: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources-lambda.html.