How to access instance in a different project without using external IP
I have two projects, say A and B, and I want to ssh from a instance in project-A to a instance in project-B.
What I found is that I was able to ping the instance in B using its eternal IP from the instance in A, not its internal IP. After I add my public key to the instance in B, I was able to ssh to it using its eternal IP from the instance in A(I have my private key here).
I wonder if I can access project B instance from, project A instance without going through external IP as it will go out of the GCP and comes back. Is there a way that I could do this internally?
Both project A and B are under the same gcp account.
Google VPC's use RFC 1918 IP addresses. These addresses are not routable across the Internet. VPC's can use the same address range in more than one VPC.
If your VPCs are not using overlapping IP addresses, you can enable VPC Network Peering to connect the two VPCs together. You can then use private IP addresses to access resources in each VPC subnet.
Google VPC Network Peering
As I understand it, your could create a shared VPC and have your Compute Engines in your distinct projects have network interfaces to this shared VPC. They would then be able to access each other directly. For full details on shared VPC, see GCP Shared VPC.
Another solution would be to use GCP VPC Peering which allows two distinct but NOT overlapping networks to connect to each other using the GCP VPC Peering capabilities.
There is so much to say about these concepts it doesn't seem to make sense repeating that here. I encourage you to read the docs in the links above and post new specific questions as needed.
Related
I need to set up a connection between VPCs. My plan was VPC peering but customer asks for Private Link as they heard it is the secure way. But I am mostly concerned with performance overhead with the private link. What I understood (maybe wrong); in the Private Link architecture there is an extra NLB. Does not this introduce a latency because of extra network hop?
VPC peering and Private Link serve two different purposes.
VPC peering enables you to connect two VPC in a same way you would connect to local networks together, and remote networks using VPN. VPC peering allows network traffic from one VPC to the second VPC. For example, you can SSH from an instance in VPC A into an instance in VPC B.
Private Link is used to expose individual services of yours in VPC A to VPC B. But this does not allow for free flow of network traffic from VPC A to VPC B. For example, let's say you've developed very cool application for image segmentation. The application and all its databases and other resources that it requires are in VPC A. Now a friend comes and he/she would like to use your application. But the friend is in VPC B. Since your application is private, not exposed to the internet, a way for your friend to use the app would be to expose it through Private Link. For this you create NLB in-front of your application, and your friend will get network interface in his VPC B through which he can access your private application in VPC A.
Based on this and your question, there is no clear answer as the two options are used for different purposes. I would suggest to clarify exactly what are your or your customer requirements.
But generally, both will be equally fast. AWS docs write about VPC peering the following:
AWS uses the existing infrastructure of a VPC to create a VPC peering connection; it is neither a gateway nor a VPN connection, and does not rely on a separate piece of physical hardware. There is no single point of failure for communication or a bandwidth bottleneck.
Other examples from AWS docs is here:
Example: Services Using AWS PrivateLink and VPC Peering
Edit: Based on #Michael comment.
I am not an expert in networking, so I want to get a clearer image. I have an AWS running instance, and its local network is 172.31.16.0/20 with address. I know that Amazon uses 172.31.0.0/16 CIDR to manage private addresses.
If someone does a scan on 172.31.0.0/16, could he/she discover my instance?
I tried to do it with another instance of mine and it detects it, but I am not sure if it works, for instance, I don't own because of this notion of VPC that I don't really understand.
Simply no. This CIDR is for a VPC, and your VPC is different from another AWS user's VPC.
To allow another AWS user to access your VPC network, you need to share it manually, so if you do not share it, it is not possible for other users to detect your instance by a brute force query.
For public IP addresses, you definitely can be discovered.
For intern IP addresses, to the extent I know, it is a virtual network, and it is isolated from other VPCs.
Traffic for private RFC1918 addresses is not routable over the Internet. No one can hit your 172.31 address across the Internet. Not from outside AWS and not from another VPC (yours or anyone else's).
VPCs are per account and are isolated from each other. You can, however, share subnets of your VPC with another AWS account within the same AWS Organization, if you choose to. You can also peer VPCs, if you choose to.
Other instances within your VPC can reach an instance in the same VPC, of course, assuming the default routing and NACLs, as can anyone on your VPC's extended network, for example if you have a VPN connection into your VPC (but I assume that's not relevant here).
We are trying to use Elastic Load Balancing in AWS with auto-scaling so we can scale in and out as needed.
Our application consists of several smaller applications, they are all on the same subnet and the same VPC.
We want to put our ELB between one of our apps and the rest.
Problem is we want the load balancer to be working both internally between different apps using an API and also internet-facing because our application still has some usage that should be done externally and not through the API.
I've read this question but I could not figure out exactly how to do it from there, it does not really specify any steps or maybe I did understand it very well.
Can we have an ELB that is both internal and external?
For the record, I can only access this network through a VPN.
It is not possible to for an Elastic Load Balancer to have both a public IP address and a private IP address. It is one or the other, but not both.
If you want your ELB to have a private IP address, then it cannot listen to requests from the internet.
If your ELB is public-facing, you can still call to it from your internal EC2 instances using the public endpoint. However, there are some caveats that goes with this:
The traffic will exit your VPC and re-enter it. It will not be direct instance-to-ELB connection that a private IP address will afford you.
You also cannot use security groups in your security group rules.
There are 3 alternative scenarios:
Duplicate the ELB and EC2 instances, one dedicated to private traffic, one dedicated to public traffic.
Have 2 ELBs (one public, one private) that share the same back-end EC2 instances.
Don't use an ELB for either private or public traffic, and instead use an Elastic IP address (if public) or a private IP address (if private) on a single EC2 instance.
I disagree with #MattHouser answer. Actually, in a VPC, your ELB have all its internal interfaces listed in Network Interfaces with Public IP AND Primary private IP.
I've tested the private IP of my public ELB and it's working exactly like the external one.
The problem is : theses IPs are not listed anywhere in a up to date manner like on a private ELB DNS. So you have to do it by yourself.
I've made a little POC script on this, with an internal Route53 hosted zone : https://gist.github.com/darylounet/3c6253c60b7dc52da927b80a0ae8d428
I made a Lambda function that checks which private IPs are set to the loadbalancer and will update Route53 record when it changes: https://github.com/Bramzor/lambda-sync-private-elb-ips
Using this function, you can easily make use of the ELB for private traffic. I personally use it to connect multiple regions to each other over a VPC inter-region peering without needing an additional ELB.
The standard AWS solution would be to have an extra internal ELB for this.
Looks like #DaryL has an interesting workaround, but it could fail for 5 minutes if the DNS is not updated. Also there is no way to have a separate security group for the internal IPs since they share the ENI and security of the external IP of the ELB.
I faced the same challenge and I can confirm the best solution so far is to have two different ALBs, one internet-facing and the other internal. You can attach both ALBs to a single AutoScaling Group so you can access the same cluster.
Make sure the networking options (Subnets, security groups) of both ALBs are the same in order for both to access the same cluster instances. Autoscaling and Launch Configuration works seamlessly with both ALBs attached to the same AutoSacling Group. This is also working with ALBs created from ElasticBeanstalk environments.
This has probably been answered elsewhere but I can't seem to find it!
I have a number of AWS EC2 instances that I am using as part of a project being built and I am now looking into securing the setup a bit. I want to lock down access to some of the ports.
For example I want to have one of the instances act as a database server (hosting mysql). I want this to be closed to public access but open to access from my other EC2 instances on their private IP's.
I also use the AWS auto-scaler to add/remove instances as required and need these to be able to access the DB server without having to manually add its IP to a list.
Similarly if possible I want to lock down some instances so that they can only accept traffic from an AWS Load Balancer. So port 80 is open on the instance but only for traffic coming from the Load Balancer.
I've looked at specifying the IP's using CIDR notation but can't seem to get it working. From the look of the private IP's being assigned to my instances the first two octets remain the same and the last two vary. But opening it to all instances with the same first two octets doesn't seem that secure either?!
Thanks
What you want to do is all pretty standard stuff, and is extensively documented in the AWS VPC documentation for Virtual Private Clouds. If your EC2 instances are not running in a VPC, they should be.
The link below should help, it seems to be your scenario:
Scenario 2: VPC with Public and Private Subnets (NAT)
The configuration for this scenario includes a VPC with a public
subnet and private subnet, and a network address translation (NAT)
instance in the public subnet. A NAT instance enables instances in the
private subnet to initiate outbound traffic to the Internet. We
recommend this scenario if you want to run a public-facing web
application, while maintaining back-end servers that aren't publicly
accessible. A common example is a multi-tier website, with the web
servers in a public subnet and the database servers in a private
subnet. You can set up security and routing so that the web servers
can communicate with the database servers.
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Scenario2.html
We have two AWS instances (Instance A and Instance B) which are running in the same VPC. There is an internet facing service on Instance A which is restricted (via security group) to a subset of IP addresses. Instance A has a DNS entry so the service can be accessed via someservice.example.org.
When trying to access the service from Instance B it works correctly if we used the VPC internal IP address however we cannot seem to get the correct security group configuration to allow this instance access via the public DNS.
We have added the 'default' VPC security group to Instance A but we're still unable to access this service directly. We also have the same problem trying to configure access to Instance A from other VPCs.
I know that we can create a private DNS for the VPC which could solve the problem when we are in the same VPC but this doesn't get around the problem when running in another VPC.
This sounds like a DNS resolution issue. If you are using Route53 for DNS the easiest way to fix this is to create a private Route53 DNS zone for your VPC and add something like:
CNAME 'someservice.example.org' that points to the instance's internal IP address.
Note that you really want to use the internal private IP address whenever possible. It will keep the network traffic within your VPC, which will be much faster and more secure. It may also be cheaper for you, at least if the instances are also within the same availability zone. You can read more about that on the EC2 pricing page in the Data Transfer section.
Also note that you can't open up the security group to allow only instances from your VPC/security group to access something via the public IP. This is because the traffic hitting the public IP is seen as coming "from the internet", not from your VPC. You would have to grant access to the servers via their public IP addresses instead of their security groups.
You mention also using a second VPC, but that would be a separate problem that could be addressed via VPC Peering.