I'm located 1000 miles from Singapore. I use S3 in Singapore region with CloudFront to serve data.
When I download contents, cloudfront is serving me from US Washington server. (checking IP addresses)
Why doesn't it serve from Singapore instead?
GeoIP lookups for IP addresses associated with CDN's is notoriously inaccurate.
Services that provide a GeoIP lookup gather information about the geographical assignment of each IP address from a wide range of sources and do their best to provide an accurate geographic assignment. In my experience, cheaper services are 80%-85% accurate, while the most expensive services are not more than around 90% accurate.
AWS does not publish the assignment of IP address to specific region. Instead, they designate the IP addresses merely as GLOBAL. As a result, the specific geography of each IP is likely unknown to the GeoIP service you are using. They make the best guess they can.
Additionally, a CDN will attempt to use the node with the least latency to you. Latency generally increases with geographic distance, but at times the longer route may offer lower latency due to a faster or less congested connection.
In your case, I suspect that you are receiving data from Singapore and your GeoIP provider is just getting the location of the IP wrong.
Related
I've used the traceroute command to see which network path packets are taking when reaching out to an EC2 instance with a public IP address.
I don't understand why at step 8 of the traceroute, packets are going to the USA (Seattle)
My EC2 is located in the AWS Paris region (eu-west-3) and my laptop making the request is located in Paris too.
Any idea why ?
IP addresses are not actually associated with geographic locations. However, various services over time have attempted to map each IP address with a location.
See: How does IP geolocation service providers collect data or how does IP geolocation databases are filled? - Quora
These activities are not always accurate and nobody is responsible for updating the information whenever devices are moved or addresses are reallocated.
I have a video streaming application and my client is from Maldives. The nearest AWS region for Maldives is Asia Pacific Mumbai. I had set up CDN and now most of the requests from Maldives are served from Singapore region even though Mumbai is the nearest region. Why is it so?
I presume that you are using Amazon CloudFront to serve the video.
CloudFront uses many methods, such as AnyCast, to determine from where to serve traffic. Typically, it will serve content from an edge location that provide better performance in terms of network latency. This will not necessarily be the closest physical location.
In fact, some cities have more than one edge location so that major ISPs can be better serviced via fast network connections.
One best way to start troubleshoot the problem is to:
Get one IP of India POP (BOM/DEL/HYD/BLR) and perform MTR to see if we see better latency compare to when you connect to Singapore.
If so:
Try 8.8.8.8 as a resolver on a test machine, this supports EDNS and check if you get any India pop.
At last, perform dig/nslookup resolver-identity.cloudfront.net and provide your resolver IP to AWS to see which edge location is best for you or if they can fix anything for you.
my website receives most of the traffic from a specific country. Though I am using Google cloud DNS, I am observing DNS lookup time to hover in the range of 200 to 300 ms which is quite slow I guess. Is there any extra optimization on Cloud DNS that will help to resolve DNS queries faster in my country of interest?
I am observing DNS lookup time to hover in the range of 200 to 300 ms
which is quite slow I guess.
This neither slow nor fast. However, the performance of your Internet connection and provider can have a big impact on this number. First remove your Internet connection from the equation to determine so that you can determine how much of the delay is caused by Google -> your provider and then your provider -> your computer.
Is there any extra optimization on Cloud DNS that will help to resolve
DNS queries faster in my country of interest?
You are leaving our details. What country? How are you testing queries?
Use an Internet DNS testing service that will report response time from points around the world. This will provide you with more realistic data. Testing from a single Internet connection is pointless unless that is the source of all your traffic.
However, there are no settings you can change in Google Cloud DNS that will affect geolocation response times.
The response time for DNS has very little to do with Internet performance of a website unless you set the TTL for DNS resource records to be very short. A client and the DNS servers around the world will look up your DNS resource records once, cache the result and use the resulting IP address for traffic until the TTL expires. This means that for most traffic, your DNS server is not involved. I am oversimplifying the process of resource record lookup and caching around the world.
In summary, you are focussing on something that really does not matter and that you cannot change.
I am wondering about Anycast. From the diagram it looks like you can map a single IP address to multiple IP addresses. This means you can have an interface of 1 IP address, but then under the hood redirect it to region a or region b, geographically distributed.
Not knowing much about IP addresses, I try this:
$ ping google.com
PING google.com (216.58.194.206) ....
....
To me that says that google.com is at 216.58.194.206 IP address. I have a few questions about this:
If that IP address will ever change.
If this is a region-specific IP address.
Or if this is probably (under the hood) redirecting to a region-specific IP address, if that's a thing.
I would like to do something like this:
mywebsite.com -> 123.123.123.123 -> 200.200.200.200 (region 1)
-> 200.200.200.201 (region 2)
-> 200.200.200.202 (region 3)
...
-> 200.... (region n)
Where only 123.123.123.123 can connect to the region specific IP addresses, and when you do ping mywebsite.com you always see 123.123.123.123 no matter where you are in the world.
Wondering if that's how it works with Anycast, or if not then how it does work. I am trying to architect a system where there are many regions for reliability and latency optimization (closeness to request source), while at the same time having it so there is only 1 domain for all regions. That is, there is not zh.mywebsite.com for china, uk.mywebsite.com for UK, etc, it is all just mywebsite.com and under the hood it redirects to the appropriate region. Also, the reason for having the intermediate 123.123.123.123 IP is so that there is a constant interface to the regions, so when you do ping mywebsite.com you will always see 123.123.123.123, not 200.200.200.200 for region 1, etc. Or perhaps there is no way around this, which would be good to know: that the IP will always resolve to the region IP.
Actually now that I try ping google.com again I get a different IP address, so there's at least 2 for some reason.
Suggestion: Ask a single question and not a broad question that covers the design of the global Internet. Ask about one cloud provider and not several. In my answer below I am limiting the amount of details that I am providing otherwise my answer would be the size of a book.
I am wondering about Anycast. From the diagram it looks like you can
map a single IP address to multiple IP addresses. This means you can
have an interface of 1 IP address, but then under the hood redirect it
to region a or region b, geographically distributed.
In Google Cloud, a global IP address is used to route the customer to the closest Google Edge location, where the traffic is then sent via the Google internal network to the final destination. The final destination depends on the type of resource. Let's assume Google Compute Engine or AWS EC2.
AWS offers AWS Global Accelerator for Anycast IP addresses which provides similar technologies.
Not knowing much about IP addresses, I try this: ping google.com
For enterprise systems, it is common practice to have many compute servers provide services for a single DNS endpoint (google.com). The servers can be load balanced by the DNS server using multiple A records. The load balancing strategy can be round robin, least busy, geo-location based, etc. I do not know Google's internet design for the DNS name google.com, but the design possibilities are large. The domain can also be served by a load balancer that hides the backend implementation. Using a load balancer is a typical design but older legacy designs use DNS servers.
If that IP address will ever change.
The answer is Yes, No and Maybe. The answer depends on the internal design of the systems. Fault tolerant, elastic designs do not require a fixed static IP address. The goal of the DNS server to translate a DNS name (google.com) into an IP address (216.58.194.206) at that point in time that the translation is required.
If this is a region-specific IP address.
Or if this is probably (under the hood) redirecting to a region-specific IP address, if that's a thing.
It depends on how Google designed its systems internally. Google offers both global and region IP addresses.
Where only 123.123.123.123 can connect to the region specific IP
addresses, and when you do ping mywebsite.com you always see
123.123.123.123 no matter where you are in the world.
You are imposing a design criteria that is not necessary. The DNS server can provide the IP address translation for the domain name. Your question indicates an lack of understanding how public and private IP addressing is managed, how addresses are translated and routed and the interplay with DNS servers, load balancers and auto-scaling. Hopefully my answers will help you know what to learn about.
I am trying to architect a system where there are many regions for
reliability and latency optimization (closeness to request source),
while at the same time having it so there is only 1 domain for all
regions.
Having one domain name (example.com) with servers located in Europe, US, South America and Asia is very easy to achieve with any cloud provider. Anycast is not required. You just need either a global load balancer (Google) or regional load balancers (AWS), and a modern DNS server, which both Google and AWS provide.
You have created a question covering both AWS and Google Cloud. The two providers are very similar at a high level but very different in implementation details. Pick one, learn how DNS, Load Balancers, and Auto-scaling works for either AWS or Google. Note: you can achieve your design goal mixing both AWS and Google Cloud, but why add that layer of complexity. The more complex your design, the more fragile it becomes. This means more management layers by humans or machines for logging, monitoring, metrics and alerting.
If I have servers placed across the globe through AWS, Rackspace, some other cloud, or even bare metal, how do I direct traffic from, say Singapore, to a server instance living in the Asia region?
Is it some kind of load balancing, or DNS type things I would have to configure?
Thanks!
Use Route 53 Latency Based Routing: http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/CreatingLatencyRRSets.html
UPDATE: Route 53 now support geolocation resource record sets:
http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resource-record-sets-values-geo.html
As Julio pointed out Latency Based Routing on Route 53 is a good option for this. One downside if you're hoping to say users in China go to this datacenter Route 53 won't do that for you. Instead they are constantly measuring the latencies to each of the AWS datacenters and will simply send people to the best AWS option (of the ones you have configured). To be honest this is probably good enough for you.
Lots of other DNS providers have similar offerings. I believe they however mainly focus on letting you decide where each region should go. I'm not a huge fan of this approach but it does give you a bit more flexibility at a cost of effort and potentially performance (if you're worried about that).