What is the standard way to block an external IP from accessing my GCP cluster? Happy for the answer to include another Google service.
Because your cluster is deployed on Compute Engine instance, you can simply set a firewall rule to discard connection from a specific IP.
If you use an HTTP load balancer, you can add Cloud Armor policy to exclude some IPs.
In both case, keep in mind that IP filtering isn't very efficient. A VPN or Proxy can be easily and freely used on the internet and change the IP source of the requester.
Related
Lets say I have a service running clustered on N ec2 instances. On top of that I have Amazon EKS and Elastic Loadbalancer. There is a service not managed by me running outside of AWS where I have an account that my services in AWS are using via HTTP requests. When I made an account to this external service I was asked for an IP (range) of services which will be using this external service. There is my problem. Currently lets say I have 3 EC2 instances with Elastic IP addresses (which are static), so I can just give those three IP addresses to this external service provider and everything works just fine. But in the future I might add more EC2 instances to scale out and whitelisting new IP addresses in the external service is a pain. In some cases those whitelist change requests may take for a week to approve by the external service provider and I dont have that time. Even further, accessing this external service is the only reason I go for static IPs for the EC2 instances. So if possible I would ditch the Elastic IPs.
So my question is how could I act so that if I make requests outside of the AWS in a random instance in my cluster, external service providers would always see the same IP address for me as a service consumer?
Disclaimer: I actually dont have that setup running yet, but I am in the middle of doing research if that would be a feasible option. So forgive me if my question sounds dumb for some obvious reason
Something like Network address translation (NAT) can solve your problem. A NAT gateway with Elastic IP, used for rerouting all traffic through it.
NAT gateway provided by AWS as service can be expensive if your data traffic is big, so you can make your own NAT instance, but that is bit complicated to set up and maintain.
The main difference between NAT gateway and NAT instance are listed here
The example bellow is assumed that EC2 instances are in private subnet, but it doesn't have to be a case.
I believe you need a proxy server in your environment with an Elastic IP. Basically you can use something like NGINX/Apache and configure it with an elastic IP. Configure the webserver to provide an endpoint to your EC2 instances, and doing a proxy pass to the external endpoint.
For high availability, you can manage a proxy in each availability zone, ideally configured using an auto scaling group to keep at leaset one instance alive in each AZ. Going through this approach, you will need to make sure that you assign the public IP from your elastic IP pool.
Generally, hostnames are better alternative to the IP addresses to avoid such situations as they can provide a static endpoint no matter what is the IP behind. Not sure whether you can explore that path with your external API provider. It can be challenging when there is static IP based routing/whitelisting rules in place.
This is what a NAT Gateway is for. NAT Gateways have an Elastic IP attached and allow the instances inside a VPC to make outbound connections, transparently, using the gateway's static address.
I have an instance running on premise and its IP address is changed regularly. My other services are running on AWS and they are using IP to connect to the premise's services. I have to update the IP address saved on AWS services whenever the IP is changed on premise network. I have a thought about using DNS but it is still a need to update A record.
I am looking for a way to do some auto-detect instead of manual updating. I wonder whether I can use load balancer to do the check. I know there will be a range of IP addresses on premise network. Can load balancer do a health check on these IP within the range? So my AWS service can send request to the load balancer. Is there any side-effect on this approach?
You need to use hostname instead of IP address as you mentoned the IP addresses keeps changing. AWS VPC can use a DNS forwarder like Unbound, which can forward the requests to your on premise DNS server when VPC resolution is unable to resolve the hostnames. This appraoch is quite effective as you send only those DN resolution to on-premise DNS that are missed by AWS VPC DNS.
Unbound allows resolution of requests originating from AWS by
forwarding them to your on-premises environment—and vice versa. For
the purposes of this post, I will focus on a basic installation of
Amazon Linux with the configuration necessary to direct traffic to
on-premises environments or to the Amazon VPC–provided DNS, as
appropriate. Review the Unbound documentation for details and other
configuration options.
Further reading : How to setup DNS resolution from AWS to on premise servers
Is there anyway to allow Google Memorystore to be accessible to the outside world and just whitelist connections by IP Address?
I have some off google services that need access to the redis but it doesnt seem possible to hit Memorystore with an IP.
You can follow the solution proposed in this other thread. This way, you would be using that Compute Engine instance as a proxy and you would just need to configure some firewall rules to only allow specific IP addresses to access that VM.
To put it in a few steps:
Create your VM in the same VPC your Google Memorystore instance is using
Add firewall rules to deny all traffic (ingress and egress) and other higher priority rules allowing all traffic (again, ingress and egress) from your desired IP addresses targeting your VPC
Use port forwarding as explained in the mentioned thread
We want to be able to connect to my on-premise database from our google cloud kubernetes.
We are currently attempting to do so by using "Create a VPN connection" from within the google console.
In the field IP address, I am forced to create (or pick from existing) "External IP Addresses".
I am able to link a single VM-instance to this External IP Address. But I want my VPN connection/tunnel to be between my on-premises network and EVERYTHING within my Google cloud network.
This IP should not just work as External IP Addr. for a single instance. I need to make it a gateway to the network as a whole. What am I missing?
Thanks in advance.
Another way to frame the question:
How do I find the IP Address of the gateway to my Google cloud network (VPC) and how do I supply that IP to the VPN Connection creation ?
The Cloud VPN connects your on-premises to the VPC, that means every Instance, Cluster or other products that use Google Cloud Engine (GCE).
As mentioned in a previous answer from avinoam-meir the VPN has at least two components: Gateway and Tunnel but I will add a third one: Type of routing.
a) Gateway: This is where you can add an existing or reserve any static IP address (from the Google Pool of External IP Addresses).
b) Tunnel: Where the encapsulated and encrypted traffic will flow to reach the Local IP ranges.
c) Type of routing: Cloud VPN has three possibilities:
Tunnel using Dynamic Routing
Route Based VPN
Policy based VPN
Depending on the type you choose, the routing happens in a different way but in general terms, it will propagate your subnetwork(s) to your on-premises network and receive the routes from it.
Important: Remember to open your firewall on your GCP VPC to receive traffic from your on-premises IP Ranges as the default and implied rule for Ingress will block it.
The implied allow egress rule: An egress rule whose action is allow, destination is 0.0.0.0/0, and priority is the lowest possible (65535) lets any instance send traffic to any destination.
The implied deny ingress rule: An ingress rule whose action is deny, source is 0.0.0.0/0, and priority is the lowest possible (65535) protects all instances by blocking incoming traffic to them.
The answer was simpler than I thought.
My question was:
How do I find the IP Address of the gateway to my Google cloud network
(VPC) and how do I supply that IP to the VPN Connection creation ?
The answer is simply to fill out the "Create a VPN connection" page. It automatically sets up whatever IP you get/choose in the "IP Address" field as the gateway. I did NOT need to configure this IP address to work as a gateway. Simply getting it assigned in this step is enough. Google does the rest behind the scenes.
You need to distinguish between gateway IP address and local IP range of the VPN tunnel
The gateway IP address is the IP of the gateway where all the packets from your on-premises arrive encapsulated and encrypted.
The local IP range of the VPN tunnel is the range of IPs that can be reached through the VPN tunnel. By default this is all the
private IP addresses of your GCP network
Create a NAT gateway [1] with Kubernetes Engine and Compute Engine Network Routes to route outbound traffic from an existing GKE cluster through the NAT Gateway instance.
Use that NAT gateway IP address to create a VPN connection to remote peer gateway.
[1] https://cloud.google.com/solutions/using-a-nat-gateway-with-kubernetes-engine
With a webserver (apache or nginx) I am able to find the x-forwarded-for header and find the client IP rather than the ELB's IP.
Can I do the same thing using IPTables so that I am able to block certain IP addresses?
I can do this at the webserver level. However, I think this is a bit inefficient and I am hoping I can achieve this with IP Tables or something similar?
Since July 30th ELB supports Proxy Protocol. As stated in the end of this thread in the AWS forum:
Elastic Load Balancing (ELB) now supports Proxy Protocol version 1.
This feature allows you to identify the client’s connection
information when using TCP load balancing, providing additional
insight into visitors to your applications. Having this information
can be useful for analyzing traffic logs, gathering connection
statistics, troubleshooting, or managing whitelists of IP addresses.
You must enable Proxy Protocol in the ELB.
The developers guide has more information about Proxy Protocol.
You can't do this with iptables, because iptables will only ever see the IP address of the elastic load balancer, since the ELB is what is establishing the connections to your instance.
Using the web server to block certain x-forwarded-for values isn't particularly inefficient, but if you want to control who can access your ELB by IP address, that can also be accomplished with the Security Group attached to the ELB.
Update: Your comment is partially correct, because, at least as of now, ELB on EC2 "classic" does not support an inbound security group or network access control list, but ELB on VPC does.
Q: Can I configure a security group for the front-end of the Elastic Load Balancer?
If you are using Amazon Virtual Private Cloud, you can configure security groups for the front-end of your Elastic Load Balancer. — http://aws.amazon.com/ec2/faqs/#ELB6
Security groups are easiest to use when you need to allow a relatively small set of specific IP address ranges. If you want to allow most but block a few, then a VPC Network Access Control List is the easier approach.