I'm kind of confused on the concept of VPC.
I understand that when I create a EC2 instance(s), it automatically resides in a VPC, as EC2 is an virtual server instance. Actually, I understand how VPC settings work (I read all the official tutorial documents for VPC)!
But what about other services or resources? S3, Lambda, RDS, ... Don't they have to be in a VPC either?
Probably, I don't clearly understand what are VPCs for. Or, I have confusions about other managed services AWS provides. I want to get clear about what can be in VPC and what can't be.
Related
I made and deployed my Django application in AWS Elastic Beanstalk. It has a connection to a Postgres DB in RDS, through the EBS console.
When I click configuration -> network in EBS, I see: "This environment is not part of a VPC."
How can I make it a part of a VPC? Thanks!
NOTE: very new to this ;)
You have to recreate the Elastic Beanstalk environment and pick the VPC during the creation. It is not possible to move an existing environment into a VPC.
But, unless you have access to EC2-Classic, the EC2 servers that were launched are already be in a VPC. They are just in the default VPC. But as far as Elastic Beanstalk is concerned, it seems oblivious to this.
I am not sure if there are any features that are exclusively available to VPC environments. My suggestion is to try to use your current environment, and if you happen to recreate the environment later for some other reason, then you can try picking a VPC and see if it offers anything new.
As already explained by #stefansundin you can't move existing EB into a custom VPC. You have to create new one.
These are general steps to consider:
Create a custom VPC with public and private subnets as described in the docs: VPC with public and private subnets (NAT). NAT is needed for instances and rds in private subnet to communicate with internet, but no inbound internet traffic will be allowed. This ensures that your instances and rds are not accessible from the outside.
Create new RDS, external to EB. This is good practice as otherwise the lifetime of your RDS is coupled with EB environment. Starting point is the following AWS documentation: Launching and connecting to an external Amazon RDS instance in a default VPC
Create new EB environment and ensure to customize its settings to use the VPC. Pass the RDS endpoint to the EB instances using environmental variables. Depending on how you want to handle password to the RDS, there are few options, starting from using environmental variables (low security) through SSM Parameter Store (free) or AWS Secrets Manager (not free).
Setting this all up correctly can be difficult for someone new to AWS. But with patience and practice it can be done. Thus, I would recommend with starting with default VPC, as you have now. Then once you are comfortable with how to work with external RDS, think on creating custom VPC as described.
The latest DocumentDB documentation states that a jump host is necessary for accessing the database from outside its native VPC:
By design, you access Amazon DocumentDB (with MongoDB compatibility)
resources from an Amazon EC2 instance within the same Amazon VPC as
the Amazon DocumentDB resources. However, suppose that your use case
requires that you or your application access your Amazon DocumentDB
resources from outside the cluster's Amazon VPC. In that case, you can
use SSH tunneling (also known as "port forwarding") to access your
Amazon DocumentDB resources.
However, VPC sharing seems to allow multiple accounts/VPCs to share the same resources.
Is it possible to use VPC sharing to access a documentDB resource in another VPC without having to use jump hosts?
Thank you in advance for your consideration and response.
Yes.
https://aws.amazon.com/documentdb/faqs/
Amazon DocumentDB clusters deployed within a VPC can be accessed directly by EC2 instances or other AWS services that are deployed in the same VPC. Additionally, Amazon DocumentDB can be accessed by EC2 instances or other AWS services in different VPCs in the same region or other regions via VPC peering.
We will get the documentation updated.
Currently, building a serverless app that use DynamoDb and Elasticsearch is quite easy
Using serverless, you just declare everything in serverless.yml and you are good to go
Problems (quickly) arise when you need to use RDS or Elasticache because you have all kind of troubles with VPCs...which then simply defeats the serverless paradigm (developper should only focus on code)
The quickest solution is then to use a 3rd party solution (like RedisLabs or ClearDb)
My question is : why RDS and Elasticache require the VPC mode ? why aren't they usable directly like a 3rd party service ?
EDIT : as noted in the comments, you can place DynamoDB and Elasticsearch behind a VPC .
The problem then becomes : how to efficiently access them (RDS, elasticache, dynamodb, elasticsearch) from a lambda function ?
You need to configure the VPC of the lambda function to access all the other VPCs as described in https://docs.aws.amazon.com/lambda/latest/dg/vpc.html
Also, consider that the lamba also needs to access 3rd party services on the internet (ex : sendgrid, onesignal, ...) so I think that you still need a NAT somewhere
The distinction here is where the resources are actually running. Both Elasticsearch and DynamoDB are managed services running outside of your AWS account. RDS and Elasticache are different - they are launched into your AWS account, hence the need to tell AWS where you want to run them.
By the way, RDS doesn't require VPC. You can optionally run it in EC2-Classic or in EC2-VPC. And those are the only options to run compute on AWS (either in VPC, or not in VPC), so you are not actually being constrained here. You are simply being asked which you prefer.
The solution for access to private resources in your VPC (like RDS databases) is to configure the Lambda function to run in that VPC. Now the Lambda function is essentially inside the VPC, so it is constrained by the VPC's networking configuration. For the Lambda function to reach external websites, it needs a route to the public internet. Typically the way you do this in VPC is to configure an IGW and some form of NAT (roll-your-own, or managed NAT from AWS). This is all normal VPC behavior, and not specific to Lambda.
The best article I found on the subject : http://blog.brianz.bz/post/accessing-vpc-resources-with-lambda/
It explains nearly everything about VPC access from lambda (from what is a VPC, why you need it, how to access from lamdba and to configure from serverless)
I have one VPC with an RDS instance in it. They are both located in the same region.
I want to use the RDS instance in another VPC, that is in another region on another AWS account (we have multiple AWS accounts). If that's not complicated enough the 2nd VPC comes up via CloudFormation (i.e. dynamic). Whenever I am bringing up a CloudFormation stack I want to attach the RDS instance automatically.
I have looked at:
exposing the RDS instance on the public internet :(
an ELB w/ TCP transport to put the database instance behind
VPC peering but the different regions and the approval workflow in the AWS console make little sense in the case we are using CloudFormation
All of these seem suboptimal to me and was wondering if somebody already did this before. If yes, please share what you did and what the though process behind it was.
Use a VPN tunnel from one VPC to the other. You could build your own or look at Vyatta. Ideally the two VPCs do not have overlapping CIDRs. Note that you cannot use VPC peering inter-region.
For anyone who stumbles around here, it looks like AWS VPC Peering can now be done cross region: https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html
I'm creating a VPC to host a web app at AWS, and I want to use load balancers. Do I need to create a endpoint for elb like I have to s3?
Confusing AWS uses 'endpoint' to refer to a couple of different things. Judging by your question are you referring to this: https://aws.amazon.com/blogs/aws/new-vpc-endpoint-for-amazon-s3/
Essentially before VPC endpoints were introduced the only way to access certain AWS services was using a public URL, this is fine unless you are working in a locked down VPC where an instance might not have access to the public internet. With the introduction of VPC endpoints a few days ago you can now access AWS service(s) directly from a private instance.
As of right now S3 is the only one supported but no doubt it will be rolled out to similar services, e.g. DynamoDB, SQS, SNS, etc in the near future.
The exception to this is services that are able to live inside a VPC that you create, i.e. when creating them you tell them which VPC, and often subnet as well where they should be created. Examples of this are ELB, RDS, EC2, Redshift, etc. For these there is no need to create an endpoint, they already exist in your VPC and can be accessed directly.