I'm trying to make beanstalk instances not connect outside the VPC to public internet.
The problem is that, when I clear the outbound security group and actually block the internet access, it makes the instances not start properly and throw error "502 Bad Gateway", also the health status in beanstalk change to "unknown".
When I checked the system log I saw that the instance is trying to connect outside the VPC in the user data script.
How can I solve this problem?
Using EB environment in a private subnet (no internet) is not that straight forward. The general consideration of how to do this are described in the AWS docs:
Using Elastic Beanstalk with Amazon VPC
For example, EB uses Network Time Protocol (NTP) thus you have to have internet connectivity. Without that, as docs describe, there are issues with Elastic Beanstalk health reporting.
You also need to setup VPC endpoints for Elastic Beanstalk as well as the endpoints for other services (e.g. S3):
Using Elastic Beanstalk with VPC endpoints
If you .ebextenstions install any packages from the internet, you need to ensure that you have properly setup NAT gateway or NAT instance.
The docs also provide example in CloudFormation vpc-private.yaml with exemplary setup for Internet gateway, NAT gateway, and route table.
Related
I have a GraphQL server Gateway app in NodeJS, which talks to a REST API server. I would like to deploy both using AWS Beanstalk, keeping the REST API server private (not exposed to the internet).
Gateway is talking to the REST API via HTTP.
How do I solve this communication between both and support future scaling?
ElasticBeanstalk (EB) can operate in a custom VPC. What's more it can operate in a private subnet.
Subsequently, you could create a VPC with public and private subnets, across 2 or 3 availability zones. You would place Gateway app in an EB environment in the public subnets, while the REST API EB environment in the private subnets. This way, the REST API wouldn't be accessible from the internet. Only the Gateway EB could communicate with it since they would be in the same VPC.
Regarding scaling. Each EB environment places your instances in an AutoScaling Group. For high-availability and scalability you would need to choose load-balanced EB environment type, rather then single-instance type.
The overall solution would be similar to the three-tear architecture , as shown for example here. The third-tear is database tier, so obviously you wouldn't have that. Also, instead of manually setting up the load balancers, autoscaling groups like in the example link, you would employ EB for that.
I have a Django web application, running on AWS Elastic Beanstalk, inside of an EC2 linux instance.
Only the load balancer is accessible to the internet, for security reasons. Deployment to the EC2 app server is facilitated by Elastic Beanstalk, but new packages added to the repository can't be installed without an internet connection.
What is the standard way to automatically update without an internet connection?
Should I just create another EC2 and use it as a proxy server? How can I configure this?
Are there any references on this topic?
Typically NAT gateway is used for that:
You can use a network address translation (NAT) gateway to enable instances in a private subnet to connect to the internet or other AWS services, but prevent the internet from initiating a connection with those instances.
There are also NAT instances which also can be used for the same purpose. However NAT gateway is a managed and scalable service and is usually preferred over NAT instances.
I have an Elastic Beanstalk web service running Tomcat + Java 8. I also have an API Gateway, which will contain an endpoint pointing to a VPC Link. I am having issues with deploying the Beanstalk with the settings I believe I should be using. My goal is to make the Beanstalk web service only accessible via the API Gateway endpoint, so no external user/service can access the xyz.us-east-2.elasticbeanstalk.com URL directly.
To accomplish this, I created a VPC through the console that uses the 10.0.0.0/16 IPv4 block.
I also created two sets of subnets, "EC2Subnets" which uses 10.0.0.0/24, and "ELBSubnets" which uses 10.0.1.0/24.
From the ebcli, I run:
$eb create MyApp --elb-type network --vpc
Which prompts for some info:
Enter the VPC ID: vpc-myvpcID
Do you want to associate a public IP address? (Y/n): y
Enter a comma-separated list of Amazon EC2 subnets: subnet-myec2subnetID
Enter a comma-separated list of Amazon ELB subnets: subnet-myelbsubnetID
Enter a comma-separated list of Amazon VPC security groups:
Do you want the load balancer to be public? (Select no for internal) (Y/n): n
Doing this causes a hang in adding the EC2 instance to Beanstalk, and eventually leads to an error: "The EC2 instances failed to communicate with AWS Elastic Beanstalk, either because of configuration problems with the VPC or a failed EC2 instance. Check your VPC configuration and try launching the environment again."
What am I doing wrong? Obviously something is wrong with my network, but I am struggling to see what it is exactly.
EDIT: Managed to figure it out; the VPC needed an Internet Gateway added to its Route Table. To do this, I created an Internet Gateway, attached it to the VPC, then went into that VPC's route table and added the newly created gateway to correspond to 0.0.0.0/0.
Managed to figure it out; the VPC needed an Internet Gateway added to its Route Table. To do this, I created an Internet Gateway, attached it to the VPC, then went into that VPC's route table and added the newly created gateway to correspond to 0.0.0.0/0.
I've spent the majority of today reading google results and documentation on how to connect AWS API Gateway to EC2 instances (created by Elastic Beanstalk) in a private subnet. I know that API Gateway requires targets to be publicly addressable, so...
I manually created an Application Elastic Load Balancer that listens for (and terminates) HTTPS at a public IP address;
I created a VPC with two subnets: one public (holds load balancer) and one private (holds EC2 instances); and,
I believe I have to create security groups that allow everyone/everywhere to connect to the load balancer, but only entities in my public subnet to connect to my EC2 instances.
Unfortunately I'm unable to view the sample Beanstalk application via the load balancer's DNS name. The connection just times-out.
Can someone please confirm I've identified all the steps? Is there any way I can trace my requests to see where they're failing? Or (even better) why they're failing? Thanks!
Check your security groups to make sure that HTTPS traffic is allowed
Make sure your Network ACLs are allowing traffic from your load balance to your EC2 instances
Check your VPC routes to ensure there is a route from your load balancer to your EC2 instance
I've created a new environment for my beanstalk app under a vpc.
I created a vpc with private and public subnets.
I've configured the security groups as outlined here:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo-vpc-basic.html
The environment seemed to setup correctly but the status is red.
Clicking on monitor reveals no data for latency or requests.
The website url returns error code 503.
Not sure what to do
This is almost certainly an issue with not being able to communicate with the NAT. The documentation is a little ambiguous when it comes to configuring your .ebextensions config files.
Don't mess with the Network ACLs for the VPC. The infrastructure needs to be able to talk to itself.
The NAT instance & Beanstalk ELB need to be in the Public subnet (10.0.0.0/24)
The Beanstalk instances themselves need to be in the Private subnet (10.0.1.0/24)
The Security Group that you apply to the instances needs to also apply to the NAT. That Security Group also needs to be on its own Inbound list so that everything inside that SG can talk to each other.
Using curl, try to make outbound calls. First on the Bastion Host, then on the NAT, then on a Beanstalk instance. Are any of them unable to make outbound calls?
I just recently spent a couple of weeks trying to develop a CloudFormation template that handles this stuff. I think that most of it is still in my brain. I think.
If these troubleshooting steps don't help, let me know and I'll dig a little deeper.
When you created the VPC, I assume you did it using the wizard. In this situation, the NAT is configured to accept calls from "default" security group.
When you created the Elastic beanstalk environment, did you specify that you want the new instances to be on "default" security group?
Check if the new instance belongs to the VPC's default security group in EC2 management area. If it is not a member of default Security group, add this security group to your NAT's inbound connections.
There is a good AWS CloudFormation template example to create VPC with public and private subnets and the NATs:
https://github.com/awsdocs/elastic-beanstalk-samples/blob/9720e38e9da155752dce132a31d8e13a27364b83/cfn-templates/vpc-privatepublic.yaml
May be very useful for those having issues with such VPC configuration.
To deploy it to your AWS using cli:
aws cloudformation deploy --stack-name mystackname --template-file path/to/vpc-privatepublic.yaml
Or you can deploy from CloudFormation UI dashboard.