I have a NodeJS server running on an EC2 instance, orchestrated by Elastic Beanstalk.
I would like to create a Lambda function that is triggered upon certain events in AWS Cognito. I want the Lambda function to make a POST call to my NodeJS server.
How can this be done?
As far as I know, EC2 and Lambda are not integrated out of the box. You'll need to expose your EC2 endpoint through a web-server (e.g. Apache) to receive HTTP requests. Then you send a GET or POST request from Lambda to the EC2 server.
You could have both running in a VPC, so that the IP address you use to make requests to your EC2 web-server is only reachable from within your VPC. It won't make your EC2 callable only by the Lambda function but will prevent the external world from calling your EC2 server.
This tutorial might be useful to you: Configuring a Lambda Function to Access Resources in an Amazon VPC.
Related
I can't figure out how to make them talk using API calls. Previously I used API Gateways which would trigger lambdas and that lambdas would interact with dynamodb and other services and send me back json response. Now I want to shift to EC2 instances and totally skip API gateway usage. And let a server I run in ec2 do the computation for me. Do I need to deploy a web service(DJango RESTFUL) in EC2 instance and then use it to call in my frontend? If yes I need little guidance how
And Suppose I want to access s3 storage from my DJango restufl in EC2. Can I do it without having to enter the access key and ID and use roles instead just like how I would access s3 from the ec2 instance without access key and ID. Traditionally with SDK we have to use access key and secret keys to even get authorized to use services in SDK so I was wondering if there was a way to get over this since the program will be running in EC2 instance itself. One really inefficient way will be to run a batch command that makes the EC2 interact with services I need without SDK and with roles instead but It is really inefficient and too much work as far as I can see.
As you are familiar with API Gateway, you can use the same to connect to your EC2 instance, its private integration, with the use of VPC Links.
You can create an API Gateway API with private integration to provide your customers access to HTTP/HTTPS resources within your Amazon Virtual Private Cloud (Amazon VPC). Such VPC resources are HTTP/HTTPS endpoints on an EC2 instance behind a Network Load Balancer in the VPC.
You can go though this document for step by step integration.
If you do not want to use API gateway any more, then you can simply use Route53 to route traffic to EC2 instance, all you need is the IP address of the EC2 instance and a hosted zone created using Route53.
Here is a tutorial for your reference.
I created an ecs service running a task definition that creates a docker container running a flask app. The flask app is getting an image through the http request and returning a ML model inference.
Since I want this service to run in production env, I created an internal network load balancer, and attached it to a vpc endpoint service. The nlb is forwarding requests to a target group which registers all ECS cluster's instances.
I have an API gateway which integrates with a lambda function that is configured inside the same vpc, and from the lambda I need to access the ecs service(through the nlb or vpc endpoint)
*** However, I keep getting 'max retries exceeded' with python requests- as the endpoint is not reachable. Nor the nlb and neither the vpc endpoint service.
Please help me understand whats wrong, or suggest another infrastructure that handles these requests in a production env.
** The docker container is valid and responding using postman directly, but I need postman to send to API gateway and get an edited response from the lambda.
Attached are some of my configurations. Let me know if you think more configurations are required.
Thanks:)
Lambda connection error to endpoint
VPC endpoint service configuration
NLB configurtion and integration
Lambda vpc configuration
Rest of your configurations seem alright, and it is validated by accessing the service inside EC2.
There is only one piece to the puzzle I can point out, i.e. while attaching your Lambda Function with the VPC, only use the private subnets. Currently, I'm not sure if the attached subnets to your Lambda Function are private or public.
I have gone through the Amazon API gateway and lamda and its i understand the combination as lamda proving computation.
Is lamda is providing computation only or it can connect to EC2 linux instance and further to RDS?
Or Amazon API gateway direct connect to EC2 Linux instance?
I am confused?
Can any one help me on this?
Thanks
So in my opinion you are looking at serverless framework which is an API gateway which is configured with stages that send requests to your lambda function.
There is no need for Ec2 instance usage. Lambda function will establish connections with your database and make calls to it. Your database setup can be either private or public.
Additionally, on top of your api gateway you can implement something like cloudfront distribution and WAF which will provide further enhancements to your setup.
Assuming an web application is deployed in Amazon EC2 instance(say in 5 instances/servers) behind Elastic Load Balancer(ELB).
I want to call an API in deployed web application in each server to do some admin level operation in each instance.
eg : https://testapplicaiton/doadminupdate
How can i invoke this API in each server? Can i use AWS API to get list of server/ip address from ELB and invoke API in each instance by http://instance:8080/doadminupdate ? Or is there any provision in ELB or AWS itself?
How can i invoke this API in each server? Can i use AWS API to get
list of server/ip address from ELB and invoke API in each instance by
http://instance:8080/doadminupdate ?
Yes, that is a valid way to accomplish your requirement.
Or is there any provision in ELB or AWS itself?
No, the ELB will only send a request to a single instance. It won't fan-out the request.
Alternatively, you might look into configuring each server to subscribe to an SNS topic on startup, and send your admin requests to the SNS topic, which will fan out the message to all subscribers.
Is it possible to connect from an AWS Lambda function to a Redis ElastiCache cluster?
I can't figure out if it's a configuration problem or it's simply not possible.
PS: I made a test from an EC2 instance and I can connect to the Redis node. Also the Lambda function and the Redis node are in the same region.
UPDATE (09 Oct 2015):
Amazon announced VPC for AWS Lambda functions. Details here
This means we can now access any resource in AWS behind VPC security group, including ElastiCache and RDS machines.
UPDATE (11 Feb 2016):
Amazon launched VPC for AWS Lambda.
https://aws.amazon.com/about-aws/whats-new/2016/02/access-resources-within-a-vpc-using-aws-lambda/
As of Feb 2016, AWS allows using lambda functions to connect to Elasticache. Refer to Access Resources within a VPC using AWS Lambda. Here is a link how it works - Tutorial: Configuring a Lambda Function to Access Amazon ElastiCache in an Amazon VPC
Setting up an HTTP Proxy or iptables wouldn't work for the following reasons:
Redis calls are not HTTP and will not be handled by HTTP proxies. iptables (or any port forwarding for that matter) will either won't accept a domain name as destination or is highly inefficient due to DNS resolution required every time.
The best and convenient method is to install twemproxy in an EC2 machine and route your requests through it. As a bonus, you suddenly have deployed a fantastic sharding strategy as well.
I have tried connecting lambda to memcached elasticache and it works fine. Redis should also be doable.
Couple of things to keep in mind:
Lambda and Elasticache has to be in the same VPC.
When lambda is run in VPC, it won't have access to internet (so access to public APIs won't work). NATGateway is required for this.
I was experiencing the same issue. I did not find a direct solution but instead used a Lambda function to connect to an EC2 server using socket.io which was pretty easy and emit an event to that EC2 server.
When the EC2 server received the event it performed the necessary Redis task ( database cleanup after image thumbnail generation ).
Hope this helps! If anyone finds out how to connect to ElastiCache from Lambda directly I'd still love to know!