Been on EC2 Classic for years and we're getting squeezed off. I'm having trouble planning the migration for the following reasons:
ec2 classic security groups don't see vpc security groups
routing only seems possible through public internet
I need to migrate master/slave db and a redis cluster into the vpc, but I can't see a clear path for the two bullet points above. Short of taking the site offline and importing all the data via dumps, I'm unsure how to proceed.
Any advice would be appreciated.
You cannot migrate anything "LIVE" from classic to VPC. You need to take Snapshot, create AMI.. etc and then re-launch the whole thing from the scratch inside the VPC. There is no other way out.
For the security group (SG), VPC SG and EC2-classic SG do not mingle. You will have to create separate SGs inside the VPC.
You need to figure out what all things you want to host in public subnet of VPC (OR Private subnet of VPC). Things only inside the Public Subnet could be accessed from Internet.
e.g. you can have your Webserver in the Public Subnet while you can have the back-end Application server in the private subnet. This was just an example.
To make the long story short, you are eventually going to launch everything new in the VPC (take help of EC2 AMI, snapshot…etc so that the things that you will launch in the VPC will have your data)
Related
I've spend about a full day trying to solve this, but have no luck so far. I'm also open to alternative suggestions than my current setup.
I have an RDS instance inside of a VPC. I am trying to make CodeBuild be able to access this RDS instance for a testing step.
Currently, I setup a VPC endpoint for the CodeBuild service, with all 3 subnets of the VPC. I know that if I allow all inbound traffic for the security group on the RDS, it works. I don't want to allow all inbound traffic though- and given this, have been unsuccessful.
I have tried the following to no avail:
Taking the private IPv4's of the ENI's created by the VPCE, adding them as inbound rules to the security group on the RDS
Creating a separate VPC for CodeBuild, and setup VPC peering (this seemed overly complex, and I'm not sure if the peering would even allow CodeBuild traffic to hit an RDS; it also makes things complicated down the road for CodeDeploy).
Putting CodeBuild inside the VPC of the RDS instance. When doing this, I created a new subnet in the VPC, assigned it to a NAT in the routes table (and this NAT was on the VPC of the RDS instance); put CodeBuild kept telling me it had no internet access.
setup a VPC endpoint for the CodeBuild service,
VPC endpoints are not used for inbound traffic from CB to VPC. They are used for your applications in VPC to interact with CB service without the internet.
Putting CodeBuild inside the VPC of the RDS instance.
This is the correct way. Sadly you haven't provided any details of your VPC, subents, NAT, route tables, security groups, NACLs setup, thus its difficult to speculate why it does not work.
Thanks Marcin for pointing me in the right direction to make CodeBuild in the same VPC. When I was able to focus on that, I saw this post again:
CodeBuild cannot find the 0.0.0.0/0 destination for the target internet gateway
which I had the same issue; my NAT was also on the private subnet. Now, it's on the public subnet, and it's working.
For example, when I deploy two AWS EC2 instances in same VPC and same subnet, that two instances connect through internet using public IP?
I already know that traffic flows referring to route table and my current route table sends all traffic except between private IP to internet gateway.
By the way, I wonder if those "all traffic except between private IP" includes AWS public IP.
In this situation, Does AWS automatically route traffic just inside VPC or not??
Yes, all traffic between the EC2 instances within the same VPC will be routed using internal, private IP obtained from the private DNS.
You can find more in the VPC international DNS here https://docs.aws.amazon.com/vpc/latest/userguide/vpc-dns.html#vpc-dns-viewing
Best, Stefan
The question is a little confusing, but I think I understand. Quick rephrasing:
How does traffic route for EC2 Instances.
If traffic has to go to the internet, then it will. There are techniques where this is done on purpose, such as hairpinning, and if this is your intention, there shouldn't be any problem with doing something like this.
AWS has a 'Backend' (Sometimes called a backbone) network. All communication to EC2 Instances, or Amazon Services in General, should be routed on the AWS Backbone.
I had a little trouble locating a doc with the exact wording, but this doc covers it a bit. Please remember that the AWS Backbone is fast, so it is a good thing to use it.
https://docs.aws.amazon.com/vpc/latest/userguide/how-it-works.html
I've been learning about Amazon AWS EC2/VPCs and after multiple videos and articles I've got a VPC up an running with private/public subnets, however, I find myself confused on one key point... how do I connect to new instances deployed in a private subnet, in order to set them up? If I deploy it to the private subnet straight away, I cannot connect to it and setup the instance.
There are multiple ways this can be achieved. A lot of people use a bastion host. This provides an instance you can SSH or RDP in to then SSH or RDP to your target instance in a private subnet from there. You could also set up a VPN server. This is going to be highly dependent on what type of setup works best for you.
This has probably been answered elsewhere but I can't seem to find it!
I have a number of AWS EC2 instances that I am using as part of a project being built and I am now looking into securing the setup a bit. I want to lock down access to some of the ports.
For example I want to have one of the instances act as a database server (hosting mysql). I want this to be closed to public access but open to access from my other EC2 instances on their private IP's.
I also use the AWS auto-scaler to add/remove instances as required and need these to be able to access the DB server without having to manually add its IP to a list.
Similarly if possible I want to lock down some instances so that they can only accept traffic from an AWS Load Balancer. So port 80 is open on the instance but only for traffic coming from the Load Balancer.
I've looked at specifying the IP's using CIDR notation but can't seem to get it working. From the look of the private IP's being assigned to my instances the first two octets remain the same and the last two vary. But opening it to all instances with the same first two octets doesn't seem that secure either?!
Thanks
What you want to do is all pretty standard stuff, and is extensively documented in the AWS VPC documentation for Virtual Private Clouds. If your EC2 instances are not running in a VPC, they should be.
The link below should help, it seems to be your scenario:
Scenario 2: VPC with Public and Private Subnets (NAT)
The configuration for this scenario includes a VPC with a public
subnet and private subnet, and a network address translation (NAT)
instance in the public subnet. A NAT instance enables instances in the
private subnet to initiate outbound traffic to the Internet. We
recommend this scenario if you want to run a public-facing web
application, while maintaining back-end servers that aren't publicly
accessible. A common example is a multi-tier website, with the web
servers in a public subnet and the database servers in a private
subnet. You can set up security and routing so that the web servers
can communicate with the database servers.
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Scenario2.html
I am faced with a chicken and egg problem. I currently have a server in EC2 classic, as well as an RDS instance -- in EC2 classic as well. The EC2 instances also interact with Cassandra cluster, which also resides in EC2 classic.
However, I need to move RDS into the VPC. Now, in an ideal world, I'd have all of my stuff in VPC at this point. However, that presents a major migration challenge and I'd like to minimize impact on users and keep steps to minimum -- this is mainly because of the Cassandra cluster.
It turns out that I cannot create security group rules between VPC and Non-VPC security groups.
So, how can I have RDS in VPC that my EC2 instances can access w/o having to open up my RDS to the entire world ?
Any help is greatly appreciated.
UPDATE: So, one idea I had is to assign elastic IPs to my EC2 instances and add IPs explicitly to the security group for RDS within VPC. Would that work ? (trying it now using https://github.com/skymill/aws-ec2-assign-elastic-ip)
Yes, unfortunately that's the only way to do it. You cannot use DNS in security groups, so you're stuck with IP address.
So, I ended up solving it exactly like I described -- assign elastic IPs to my EC2 instances and add IPs explicitly to the security group for RDS within VPC. It ended up working great.