AWS Lambda Cannot Access RDS via JDBC - amazon-web-services

I have a test Lambda function setup using Java 8
So far I have
given the Lambda function an execution role AWSLambdaVPCAccessExecutionRole
attached the Lambda function to the only VPC I have on my account and selected all subnets within the VPC so that it may access the
my RDS instance in this case is open to the public, and I am able to access it via my Laptop (i.e. the Lambda code actually runs on remote hosts not inside the VPC)
security group assigned to the Lambda is the most permissive possible (i.e all traffic on all CIDR block)
However, I am still unable to access my RDS instance when running the Lambda function on AWS (but the same code works on my laptop, when run from a main() function)
Sample Code
public class Application implements RequestHandler<Object, Boolean> {
private Logger logger = Logger.getLogger(Application.class);
public Boolean handleRequest(Object object, Context context) {
try {
Class.forName("com.mysql.jdbc.Driver");
logger.info("Calling DriverManager.getConnection()");
Connection con = DriverManager.getConnection(
"jdbc:mysql://...endpoint...defaultdb",
"awsops",
"..."
);
Statement stmt = con.createStatement();
logger.info("Test Started!");
ResultSet result = stmt.executeQuery("SELECT\n" +
" last_name, COUNT(*)\n" +
"FROM\n" +
" users\n" +
"GROUP BY\n" +
" last_name\n" +
"HAVING COUNT(*) > 1");
if (result.next()) {
logger.info(result.getString("last_name"));
}
return true;
} catch (Exception e) {
logger.info(e.getLocalizedMessage());
}
return false;
}
}
Can you please help me understand what I could be doing wrong?
The CloudWatch logs shows that the function hangs at DriverManager.getConnection()

It turns out my RDS was launched with an automatically created security group that literally just whitelisted my personal IP address ... so it feels like I was able to connect to it from 'anywhere'
I had to update the security group of my RDS instance to allow traffic from the subnets where the Lambda's virtual network interface could be coming from

Related

Calls to Square createPayment timing out in AWS lambda function

I'm creating a Square Client object like this:
const squareClient = new Client({
environment: Environment.Sandbox,
accessToken:
"The_correct_sandbox_token_goes_here",
});
const paymentsApi = squareClient.paymentsApi;
and calling the createPayment method from within a lambda function with a body like the one below:
{
"sourceId": "cnon:CBASEHY1uZmmlYRYagaqS7yd9Zo",
"amountMoney": {
"amount": "12500",
"currency": "USD"
},
"locationId": "Location_ID_here",
"idempotencyKey": "6a36e49c-914d-4934-bc34-c183ba0a08c5"
}
This works fine on my local machine (using serverless offline), but when deployed to AWS, the call to createPayment times out after six seconds. Is there something extra that needs to be done to call createPayment from a lambda function?
The timeout you experienced might be due to the function being attached to a VPC.
If an AWS Lambda function is configured to use a VPC, then the function only has access to the Internet if the following have been configured:
Lambda function is connected to a private subnet
A NAT Gateway or NAT Instance is running in a public subnet in the same VPC
A Route Table on the private subnet directs Internet-bound traffic to the NAT
If the Lambda function does not require access to any resources in the VPC, do not attach the function to a VPC. This will automatically grant direct access to the Internet.
Alternatively, you might try increasing the timeout of the Lambda function in case the external service simply needed more time.

Node Lambda AWS TimeoutError: Socket timed out without establishing a connection to cloudformation

I am running a Node(12.x) Lambda in AWS. The purpose of this lambda is to interact with Cloudformation stacks, and I'm doing that via the aws-sdk. When testing this lambda locally using lambda-local, it executes successfully and the stack can be seen in CREATING state in AWS console.
However, when I push and run this lambda in AWS, it fails after 15 seconds, and I get this error:
{"errorType":"TimeoutError","errorMessage":"Socket timed out without establishing a connection","code":"TimeoutError","message":"Socket timed out without establishing a connection","time":"2020-06-29T03:10:27.668Z","region":"us-east-1","hostname":"cloudformation.us-east-1.amazonaws.com","retryable":true,"stack":["TimeoutError: Socket timed out without establishing a connection"," at Timeout.connectTimeout [as _onTimeout] (/var/task/node_modules/aws-sdk/lib/http/node.js:69:15)"," at listOnTimeout (internal/timers.js:549:17)"," at processTimers (internal/timers.js:492:7)"]}
This lead me to investigate the lambda timeout and the possible configuration changes I could make found in https://aws.amazon.com/premiumsupport/knowledge-center/lambda-function-retry-timeout-sdk/ and https://aws.amazon.com/premiumsupport/knowledge-center/lambda-vpc-troubleshoot-timeout/ but nothing worked.
I found a couple of similar issues such as AWS Lambda: Task timed out which include possible suggestions such as lambda timeout and lambda memory issues, but Ive set mine to 30 seconds and the logs show max memory used is 88MB out of possible 128MB, but I tried with an increase anyway, and no luck.
The curious part is that it fails without establishing a connection to hostname cloudformation.us-east-1.amazonaws.com. How is that possible when the role assigned to the lambda has full Cloudformation privileges? I'm completely out of ideas so any help would be greatly appreciated. Heres my code:
TEST EVENT:
{
"stackName": "mySuccessfulStack",
"app": "test"
}
Function my handler calls (createStack):
const AWS = require('aws-sdk');
const templates = {
"test": {
TemplateURL: "https://<bucket>.s3.amazonaws.com/<path_to_file>/test.template",
Capabilities: ["CAPABILITY_IAM"],
Parameters: {
"HostingBucket": "test-hosting-bucket"
}
}
}
async function createStack(event) {
AWS.config.update({
maxRetries: 2,
httpOptions: {
timeout: 30000,
connectTimeout: 5000
}
});
const cloudformation = new AWS.CloudFormation();
const { app, stackName } = event;
let stackParams = templates[app];
stackParams['StackName'] = app + "-" + stackName;
let formattedTemplateParams = [];
for (let [key, value] of Object.entries(stackParams.Parameters)) {
formattedTemplateParams.push({"ParameterKey":key, "ParameterValue": value})
}
stackParams['Parameters'] = formattedTemplateParams;
const result = await cloudformation.createStack(stackParams).promise();
return result;
}
Lambda function in a VPC does not public IP address nor internet access. From docs:
Connect your function to private subnets to access private resources. If your function needs internet access, use NAT. Connecting a function to a public subnet does not give it internet access or a public IP address.
There are two common solutions for that:
place lambda function in a private subnet and setup NAT gateway in public subnet. Then set route table from private subnet to the NAT device. This will enable the lambda to access the internet and subsequently CloudFormation service.
setup a VPC interface endpoint for CloudFormation. This will allow your lambda function in private subnet to access CloudFormation without the internet.

Want to call AWS lambda in VPC to another AWS lambda without VPC in same region and account using JAVA 8

Want to connect one AWS lambda which is in VPC to another lambda which is not in VPC but in same region and account.
Code of caller lambda "lambda_1":
#Override
public String handleRequest(final Object input, final Context context) {
logger.log(context.getFunctionName() + " invoked");
final AWSLambda client = AWSLambdaClientBuilder.standard().withRegion(Regions.EU_WEST_2).build();
final InvokeRequest request = new InvokeRequest();
request.withFunctionName("lambda_2").withPayload("JSON data as String").withInvocationType(InvocationType.RequestResponse);
logger.log("Lambda is about to invoke");
final InvokeResult response = client.invoke(request);
logger.log(context.getFunctionName() + " returned");
return input.toString();
}
AWS java sdk for lambda is used through maven dependency
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-lambda</artifactId>
<version>1.11.681</version>
</dependency>
Allow: lambda:InvokeFunction is given to lambda_1 for lambda_2
Code of callee lambda "lambda_2":
#Override
public String handleRequest(final Object input, final Context context) {
logger.log(context.getFunctionName() + " invoked");
logger.log(context.getFunctionName() + " returned");
return input.toString();
}
Logs get created for lambda_1 till Lambda is about to invoke line, execution doesn't go further and times up.
Enough time and memory has been set for lambda_1, lambda_2 as 2 min 198 MB and 1 min 128 MB respectively.
But Unable to call lambda_2 and even not getting any kind of error at runtime, kindly help thanks
Timeout with no further log output sounds like a typical networking-issue, I'd start looking there.
To reach the Lambda service, the calling Lambda (and with that your VPC) needs access to the Internet, have you got an Internet Gateway or NAT Interface/Gateway in the VPC?
Alternatively, instead of directly invoking the Lambda, you could go over SNS, that would allow you to add an SNS Interface endpoint in your VPC, if connecting your VPC to the Internet is not an option.

AWS Java SDK - Get EC2 instance info

Given an instance id, I want to get an EC2 instance info (for example, its running status, private IP, public IP).
I have done some research (i.e. looking at the sample code posted here Managing Amazon EC2 Instances)
but there is only sample code of getting the Amazon EC2 instances for your account and region.
I tried to modify the sample and here is what I came up with:
private static AmazonEC2 getEc2StandardClient() {
// Using StaticCredentialsProvider
final String accessKey = "access_key";
final String secretKey = "secret_key";
BasicAWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);
return AmazonEC2ClientBuilder.standard()
.withRegion(Regions.AP_NORTHEAST_1)
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.build();
}
public static void getInstanceInfo(String instanceId) {
final AmazonEC2 ec2 = getEc2StandardClient();
DryRunSupportedRequest<DescribeInstancesRequest> dryRequest =
() -> {
DescribeInstancesRequest request = new DescribeInstancesRequest()
.withInstanceIds(instanceId);
return request.getDryRunRequest();
};
DryRunResult<DescribeInstancesRequest> dryResponse = ec2.dryRun(dryRequest);
if(!dryResponse.isSuccessful()) {
System.out.println("Failed to get information of instance " + instanceId);
}
DescribeInstancesRequest request = new DescribeInstancesRequest()
.withInstanceIds(instanceId);
DescribeInstancesResult response = ec2.describeInstances(request);
Reservation reservation = response.getReservations().get(0);
Instance instance = reservation.getInstances().get(0);
System.out.println("Instance id: " + instance.getInstanceId(), ", state: " + instance.getState().getName() +
", public ip: " + instance.getPublicIpAddress() + ", private ip: " + instance.getPrivateIpAddress());
}
It is working fine but I wonder if it's the best practice to get info from a single instance.
but there is only sample code of getting the Amazon EC2 instances for your account and region.
Yes, you may get only instance information you have permission to read.
It is working fine but I wonder if it's the best practice to get info from a single instance
You have multiple options.
For getting EC2 metadata from any client (e.g. from your on-premise network) your code seems ok.
If you are running the code in the AWS environment (on an EC2, lambda, docker, ..) you may specify a service role allowed calling the describeInstances operation from the service. Then you don't need to specify the AWS credentials explicitly (DefaultAWSCredentialsProviderChain will work).
If you are getting the EC2 metadata from the instance itself, you can use the EC2 metadata service.

Auto scale behavior of AWS ECS service

I am trying to test auto scale behavior in amazon ECS. Here is how i tested
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;
public class LoadHttp {
private static final String GET_URL = "http://tournament-ecs-weather-lb-1526102172.ap-south-1.elb.amazonaws.com/status";
public static void main(String[] args) throws IOException {
System.out.println("GET DONE");
while(true){
sendGET();
}
}
private static void sendGET() throws IOException {
Thread t = new Thread(() -> {
URL obj;
try {
obj = new URL(GET_URL);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod("GET");
con.setConnectTimeout(5000);
int responseCode = con.getResponseCode();
System.out.println(responseCode);
} catch (Exception e) {
e.printStackTrace();
}
});
t.start();
}
}
and configuration of auto scale of service that i am running
here is my auto scale group configuration
I am using ECS Service to create cluster and my service is running in docker container. Still after executing my test code i found my service is not getting autoscaled. Number of container instance is still one. If you need additional information I will be happy.
If I understood your question, the service is autoscaled when there are not enough resources available in the current instance to handle the requests.
In your particular case, you have set the Min value of the instances as 0 which tells that the system should react fine even there is no instance running. You need to have at least 1 for the min value to make the whole cluster run ok. I believe that for this reason your autoscaling is not launching any instance.
Also, I would suggest you to add the Elastic Load balancer to your autoscaling group which launch asks autoscaling group to launch more instances when the minimum number of instances specified in your autoscaling are not healthy.
Lastly, to test if the autoscaling group is working or not, you check the availability zone of your instance and note that down. Now terminate your instance. Since the running instance is terminated the autoscaling group should automatically launch an instance in different availability zone. If it launches new instance in different availability zone than the terminated instance then it works ok.
Hope this will help.