I need to add a route in GCP's VPC Network and when I do, I get an error (shown below) that appears to state there is an overlap/conflict with the 10.130.0.0/16 range. I unfortunately do not see this 10.130.0.0/16 in any route, in any region and I have no idea why this error is occurring.
Creating route "test" failed. Error: Invalid value for field 'resource.destRange': '10.130.90.82/32'. 10.130.90.82/32 hides the reserved address space for network (10.130.0.0/16).
I have tried adding this simple route in several gcp projects but they all fail and seem to imply there might be some hidden reserved address space. Could this be? What am I missing? This occurs with any route destination value that is in the 10.130.0.0/16 space eg; 10.130.90.82/32 10.130.90.0/24
For clarification here is an example Route that fails:
Additional Clarification, Here is the 'default' VPC Network:
Google Cloud does not allow you to create a new subnet or peering subnet route whose destination exactly matches or is broader than (would contain) an existing custom static route. For example, if your VPC network has a custom static route for the 10.70.1.128/25 destination, Google Cloud prohibits the creation of any subnet or peering subnet route with a primary or secondary subnet IP address range of 10.70.1.128/25, 10.70.1.0/24, or any other range that contains all the IP addresses in 10.70.1.128/25.
Kindly check Configuring private services access docs. Included on the docs are the considerations , Creating an IP allocation, Deleting an allocated IP address range etc.
There is in fact a hidden reserved address space in the default VPC network. I hesitate calling hidden as JaysonM mentioned it in his answer but it does not appear anywhere in the GPC console (to my knowledge).
The default VPC network is using 'auto' subnet creation mode. With this setting enabled the VPC network has a range of 10.128.0.0/9 (10.128.0.0 - 10.255.255.255) that cannot be overlapped. Simply setting 'auto' subnet creation of the VPC network to 'custom' will resolve this issue. Do also note this is a one-way change for your VPC network.
Cheers!
Related
I am getting below error while trying to create VPC Access connector in GCP region us-central1:
An internal error occurred: VPC Access connector failed to get healthy. Please check GCE quotas, logs and org policies and recreate.
I also tried to create the VPC access connector in region us-east1 but got the same issue.
I tried searching for existing bugs on gcp issues portal but could not find this issue.
I have tried to follow image access constraint but I don't have an organisation so I am unable to edit the required policy.
I am having the same issue. After reading this thread I checked different regions with exactly the same configuration:
Network: Default
Subnet: Custom IP range
IP range: 10.8.0.0/28
I can confirm that changing the area solves the issue. In my case, I proceeded successfully with australia-southeast2. Basically, when creating a VPC connector in Google Cloud, we have some regions working and some others are not.
It may be a capacity problem over some Google regions.
It can be an internal IP subnet assignment issue. This subnet must be used exclusively by the connector per the documentation
Every VPC connector requires its own /28 subnet to place connector instances on; this subnet must not have any other resources on it other than the VPC connector. If you don't use Shared VPC, you can either create a subnet for the connector to use, or specify an unused custom IP range for the connector to create a subnet for its use. If you choose the custom IP range, the subnet that is created is hidden and cannot be used in firewall rules and NAT configurations.
Or it can also be that you are missing the required image access constraint. In this case, you may follow this step by step guide in seting image access constraints
Go to the Organization policies page.
In the policies list, click Define trusted image projects.
Click Edit to customize your existing trusted image constraints.
On the Edit page, select Customize.
In the Policy values drop-down list, select Custom to set the constraint on specific image projects.
In the Policy type drop-down list, specify a value as follows:
-To restrict the specified image projects, select Deny.
-To remove restrictions for the specified image projects, select Allow.
In the Custom values field, enter the names of image projects using projects/IMAGE_PROJECT format. Replace IMAGE_PROJECT with the image project you want in this case “serverless-vpc-access-images“ to set constraints on. If you are setting project-level constraints, then they might conflict with the existing constraints set on your organization or folder.
Click New policy value to add multiple image projects.
Click Save to apply the constraint.
Additionally, please make sure that there are no firewall rules on your VPC network with a priority before 1000 that deny ingress from your connector's IP range.
I have created a VPN to our VPC but when I connect to it from my machine, nothing works - neither Internet, nor can I reach VPN-internal endpoints.
I added public dns-servers and split-tunnel=enabled to the configuration.
The VPN is set to
`Client IPv4 CIDR 10.10.0.0/16`
An association has been added (line from AWS console):
cvpn-assoc-<id> subnet-<id> cvpn-endpoint-<id>  Associated sg-<id>
There are two authorization rules (one allowing everything until I get it working)
The route table looks like this (was automatically added via the assoc):
cvpn-endpoint-<id> 10.1.0.0/16 subnet-<id> Nat associate  Active Default Route
This RT is the only thing which looks odd. In the VPC, the subnet has a definition of 10.1.0.0/24 - but the automatic association sets it to 10.1.0.0/16. But there's actually no way to set it to 10.1.0.0/24 in the routing table, doing so results in an error about the range being invalid.
I also tried creating the VPC with a client IP CIDR of 10.10.0.0/24 but then it errored saying it must at least be /22.
At our company we have three AWS accounts, the main one, used as "root" account for IAM and hosting an OpenVPN Access Server. The other two accounts are pro and stg. Each one has its own VPC, with different IP ranges, and we have a VPC peering between the root and pro accounts, and other one between root and stg. IP routing is already setup and everything is under control from this side.
(I'm sorry I can't upload images yet, so here you have the link)
VPN+VPC-Peering
The problem comes with DNS resolution. The setup is this one:
I've installed BIND9 in the OpenVPN server, to allow DNS forwarding for private hosted domains, using a configuration like this one in named.conf.local
zone "stg-my-internal-domain.com" IN {
type forward;
forward only;
forwarders { 10.229.1.100;10.229.2.100; };
};
zone "pro-my-internal-domain.com" IN {
type forward;
forward only;
forwarders { 10.228.1.100;10.228.2.100; };
};
And also two Route53 inbound resolvers (a simple BIND server running on each VPC also works) running in 10.229.1.100 and 10.229.2.100 for stg and 10.228.1.100 10.228.2.100 for pro account
VPN clients have OpenVPN profiles that use the Access Server as DNS resolver.
From my client, I can resolve both my-service-1.pro-my-internal-domain.com and my-service-2.stg-my-internal-domain.com perfectly, but the problem comes when I want to resolve internal domain names like the ones that AWS generates inside each VPC with my-service-2.eu-west-1.compute.internal
I know that this is an anti-pattern and I should always use the private domain as much as I can, but for some cases like EMR clusters, YARN and Hadoop managers use links that reference to the internal AWS names, making the resolution impossible.
So my question is: Is there any way to configure DNS to delegate resolution to a secondary address if primary fails?
I could set up a forwarder for the eu-west-1.compute.internal zone using all the accounts resolvers, but
DNS specification says that the secondary nameserver will only be used if the first one is unreachable, so as far as it answers an empty or "unknown" response, it's still a valid response and the second one will not be queried.
Any help is really appreciated!
Why not just change the internal host name to a public dns name? Those services are using the hostname assigned to them of course. You can change it.
See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-hostname.html
You may (or may not) need to assign fixed private ips to each. In any case publish this private IP in a public DNS zone. You should then be able to resolve these names properly. Note you can also have a script run on each instance on startup, to update the hostname and dns record.
For a good discussion on private ip addresses in public DNS, see https://serverfault.com/questions/4458/private-ip-address-in-public-dns
For reference, here is the best answer there:
Some people will say no public DNS records should ever disclose private IP addresses....with the thinking being that you are giving potential attackers a leg up on some information that might be required to exploit private systems. Personally, I think that obfuscation is a poor form of security, especially when we are talking about IP addresses because in general they are easy to guess anyway, so I don't see this as a realistic security compromise. The bigger consideration here is making sure your public users don't pickup this DNS record as part of the normal public services of your hosted application. ie: External DNS lookups somehow start resolving to an address they can't get to. Aside from that, I see no fundamental reason why putting private address A records into the public space is a problem....especially when you have no alternate DNS server to host them on. If you do decide to put this record into the public DNS space, you might consider creating a separate zone on the same server to hold all the "private" records. This will make it clearer that they are intended to be private....however for just one A record, I probably wouldn't bother.
AWS only supports DNS resolution of these internal ipv4 DNS hostnames if your VPN is in the same region as your EMR cluster (or any other compute resource). I have reached out to their Support and they have confirmed this.
For example, I have an AWS Client VPN endpoint setup in Frankfurt and an EMR cluster in Ireland. I am pushing to my host the private DNS server of the VPC (and all other related config is enabled in both VPCs) so that I can resolve private Route53 DNS zone records.
While I am connected to the VPN,
I can't resolve this:
$ dig +short ip-10-11-x-x.eu-west-1.compute.internal
$
But I can resolve the following, which is an instance that's in the same region as the VPN endpoint:
$ dig +short ip-10-10-x-y.eu-central-1.compute.internal
10.10.x.y
How to solve this:
Either move your EMR clusters in the same region as your VPN is, or the other way around.
But the simplest solution might be to just use a Chrome plugin (here's an example) that automatically redirects ip-x-y-z... URLS to x.y.z IPs.
We are trying to configure a VPC, which has a private subnet and a public subnet. In the private subnet there is an RDS which is not publicly accessible. We have test it and seems that works fine! The issue though its that when I ping the RDS endpoint from my computer it returns the Private IP of the RDS (its not returns any packets though).
We do not want to shows the Private IP.
Any help would be appreciated!
I went ahead and popped open a chat with our AWS support team to pick their brain. Basically, this boils down to how they host their DNS mappings for RDS endpoints; they're created in a public hosted zone by default (not modifiable). Hence, you can resolve your RDS endpoint over the internet (because the mapping is hosted publicly), but can't actually route any data to it.
If this is an issue, to get around it you can ... jump through some hoops:
An alternative will be to create a private hosted zone with a record
that points to the rds endpoint. (for example a private hosted zone
"xxxx.com" that has an alias record pointing to rds endpoint), in which case you will reach out to your rds instance
using xxxxx.com
However, this doesn't actually disable the original AWS created endpoint from returning the private IP, it just allows you to configure an endpoint that doesn't.
For what it's worth, revealing your private IP is pretty harmless; several thousand devices likely share your exact private IP. The only way this information would be concerning for you is if an attacker was actually in your network - and at that point... they could just do a lookup on the DNS from there to get the IP.
First question: why do you want to do this? Your 10.1.2.3 or 172.31.2.3 or whatever is a non-routable address. It really doesn't matter whether people know it if they can't get into your VPC.
As for actually preventing it, you can't: Amazon makes the endpoint available via DNS (you can use nslookup to find it). You could always try filing a support ticket, but I wouldn't expect any results.
Also, FYI the second component of the endpoint is related to your account. So in your image you redacted non-important information but left the (potentially) important information present.
In case it's not clear, the problem is in how Amazon resolves DNS requests, not in how the networks are connected. Here's an example of an nslookup call for one of our database instances that's running on a private subnet. This is from my PC, not connected to the VPC via VPN or any other means:
> nslookup REDACTED.REDACTED.us-east-1.rds.amazonaws.com
Server: 127.0.1.1
Address: 127.0.1.1#53
Non-authoritative answer:
Name: REDACTED.REDACTED.us-east-1.rds.amazonaws.com
Address: 10.1.56.119
Consider an server ec2-50-1-2-3.compute-1.amazonaws.com, which is not publicly available and which does not have an elastic IP address. I cannot assign it an Elastic IP address as I don't have any more addresses to assign (used all 5 already on publicly-available servers).
The publicly-available servers need to access a service on ec2-50-1-2-3.compute-1.amazonaws.com. However, if I restart that server then it may receive a different address and I'll have to update 20 websites across 5 webservers with the new address. Is there any way to refer to the ec2-50-1-2-3.compute-1.amazonaws.com server which will persist even if I restart that server, considering that I have no more Elastic IP addresses to assign to it?
Is there any way to refer to
Key word "refer to" -- indeed, there is... a DNS CNAME.
Whether your DNS is in Route 53 or elsewhere, a CNAME record refers a system asking for a particular host by name, to a different host -- also by name.
Let's say, for example, that the service this system provides is the generation of reports. In the "example.com" domain...
reports IN CNAME ec2-50-1-2-3.compute-1.amazonaws.com.
Any machine looking up "reports.example.com" from the DNS will be referred to the hostname ec2-50-1-2-3.compute-1.amazonaws.com which will of course resolve to the machine's IP address.
If the machine's IP address (and therefore, in AWS, its hostname) changes because the instance was terminated or failed or replaced, you only have to update the information in one place -- the DNS. The systems that need to access this system would be configured with "reports.example.com" instead of the other hostname, so they wouldn't have to be maintained individually.
If you are using Route 53, it's also possible to configure Route 53 to actually give out a different answer using failover routing with health checks and divert requests elsewhere when the instance isn't working properly.
Amazon will not give you any difficulty at all if you simply request more Elastic IPs. It's right here: Request to Increase Elastic IP Address Limit
It turns out the the best way to refer to other instances in AWS is to use Amazon Virtual Private Cloud (VPC). In VPC each machine gets a static internal IP address, which persists for the lifetime of the instance. In fact in VPC one can configure full networking!