I am following the instructions at https://cloud.google.com/cdn/docs/setting-up-cdn-with-ex-backend-internet-neg and https://medium.com/the-innovation/how-to-enable-google-cdn-for-custom-origin-websites-google-cdn-for-external-websites-56e3fe66cca9 to setup Google Cloud CDN for my website www.datanumen.org.
For the "Fully qualified domain name and port" in "New network endpoint", I choose www.datanumen.org.
All others are same as the above two articles, I use HTTP protocol for all the communications. Finally I get a frontend IP address 34.96.69.82. So I try to visit http://34.96.69.82/, but get a default "SORRY" web page instead of the contents from www.datanumen.org. Why?
Also later I plan to update the DNS A Record for www.datanumen.org so that datanumen.org will points to 34.96.69.82 instead of its current IP address. I am just curious that if I do that, then since what I put in "Fully qualified domain name and port" in "New network endpoint" is www.datanumen.org, will it cause the following deadloop:
a user visit www.datanumen.org
Based on DNS A record, he will go to 34.96.69.82(frondend)
The frontend will request data from backend, and the endpoint is www.datanumen.org,
Based on DNS A record, the backend end point will also solved to 34.96.69.82.
Thus will cause a deadloop for ever?
Update:
For the 1st question, I find the solution. My website is hosted on a server with shared IP. In article https://cloud.google.com/cdn/docs/setting-up-cdn-with-ex-backend-internet-neg, it asks me to add "Host" to the request header, which is used to identify the actual site to be accessed when the request reaches the original server. In my previous configuration, I thought this step is useless so I just skip it. After adding the "Host" field, now I can visit my website properly with the IP address given by Google.
You already fixed your fist issue so you're at a point when you can successfully access your site using an IP address.
Right now all you need is a CDN enabled - if it's not you can do it in the "backend" section of the "load balancing" page.
Have a look also at Network Endpoint Groups documentation to see how to create a load balancer utilising them (the only way you can use GCP's CDN for external site).
To answer your second question - there will be no loop - your site works properly.
Since you're not using secure HTPPS (only HTTP) then you don't have to worry about SSL certificates and the only thing that remains for you to do is to direct your domain to your load-balancer's IP and you're done.
If you encounter any issues or just want to check if CDN is working correctly then have a look at CDN troubleshooting page.
Most simple way to verify if it's working is to use curl: curl -s -D - -o /dev/null http://example.com/style.css and see if you have Cache-Control line present in the output:
HTTP/1.1 200 OK
Date: Tue, 16 Feb 2016 12:00:31 GMT
Content-Type: text/css
Content-Length: 1977
Cache-Control: max-age=86400,public
Via: 1.1 google
However I recommend using HTTPS and SSL certificates for the security reasons - it's much harder to spoof the traffic/listen to the between the site and the client. It's not mandatory though.
Related
I am trying to create a GitHub page for a repository. But when I gave the custom domain name, it shows the following message "Domain's DNS record could not be retrieved. For more information"
As I am new to GitHub I am not getting the information what is documented in GitHub pages. Could anyone help me to resolve this problem?
If you've recently changed or removed your custom domain and can't access the new URL in your browser, you may need to clear your browser's cache to reach the new custom domain. For more information on clearing your cache, see your browser's help site.
In order to serve the Page, your DNS records must point to GitHub's server. To confirm that your custom domain points to GitHub's servers, use the dig command with your custom domain. The dig command shows you where your custom domain points. For example:
$ dig example.com +nostats +nocomments +nocmd
example.com. 3600 IN A 185.199.108.153
In the example above, example.com points to the IP address 185.199.108.153.
If you configured A records through your DNS provider, your A records must point your custom domain to the following IP addresses:
185.199.108.153
185.199.109.153
185.199.110.153
185.199.111.153
You may see a different IP address, since we serve Pages with a global Content Delivery Network. Use dig username.github.io to see the full resolution path. Note that DNS caching may cause a delay.
If you're using an A record that points to 192.30.252.153 or 192.30.252.154, you'll need to update your DNS settings for your site to be available over HTTPS or served with a Content Delivery Network. For more information, see "HTTPS errors."
If you're using an A record that points to 207.97.227.245 or 204.232.175.78, you'll need to update your DNS settings, as we no longer serve Pages directly from those servers.
Source: https://help.github.com/en/articles/troubleshooting-custom-domains
I created this question earlier but was told that it is a DNS issue as apposed to an issue with HSTS. Regardless, here is what I need help troubleshooting:
Issue:
A single site (one that I own), is showing server DNS address could not be found. DNS_PROBE_FINISHED_NXDOMAIN when I try to connect to it via chrome, firefox, or safari. I can however connect to it via Tor Browser. I can also verify that the address resolves correctly using mxtoolbox. I also am not able to connect via two other computers and two other phones. I also am not able to connect via a different WIFI connection or personal hotspot via my phone. Curl and Host via the command line are also not able to get a response.
What I've tried:
As I said above, I've tried different internet connections and computers. I've also tried flushing my DNS cache and pointing to another DNS server.
Having said that, I am not sure how else to trouble shoot this. The only change I made to the web app was to add HSTS headers, hence why I created the earlier posing. Please let me know what other information I can provide. Otherwise, here are some details about the site itself:
Other information about my stack:
Django web app
Gunicorn / WSGI server
Hosted on Heroku - Cedar-14 stack
DNS setup with AWS route53
domain name registered through AWS
EDIT:
Possibly related: https://serverfault.com/questions/606880/how-can-i-troubleshoot-a-route-53-hosted-zone
I had the similar issue and was not able to open Facebook. Rest all sites were working fine. Initially, I thought Facebook blocked me as I never faced this crappy issue earlier. Later when I searched in Google, I found an article which described the DNS_PROBE_FINISHED_NXDOMAIN issue on Chrome.
I just changed my DNS server address as 8.8.8.8 (preferred) and 8.8.4.4 (alternate) and I never faced that issue again.
Reference - https://www.mobipicker.com/dns_probe_finished_nxdomain/
So from our discussion regarding the NS server records always make sure that the local NS records matches the Parent NS records.
In your case there there were 2 extra NS records associated with your domain that was the reason why your domains and sub domains were acting unhealthy. once you deleted those records the domains and sub domains were back to normal.
you can also try to open an anon window
access the url
use it in anon mode
or
close it and it will load ok
I have used ShimmerCat with sc-tool to connect to my development sites as described here, and everything has worked always like a charm with it, but I also wanted to follow the "old way" configuring my /etc/hosts. In this case I had a small problem, the server ran ok, and I could access to my development site (let's say that I used https://www.example.com:4043/), but I'm also using a reverse proxy as described on this article, and on the config file reference. It redirects to a Django app I'm using. Let's say it is my devlove.yaml config file:
---
shimmercat-devlove:
domains:
www.example.com:
root-dir: site
consultant: 8080
cache-key: xxxxxxx
api.example.com:
port: 8080
The problem is that when I try to access to a URL that requests the API, a 404 response is sent from the API. Let me try to explain it through an example. I try to access to https://www.example.com:4043/country/, and on this page I do a request to the API: /api/<country>/towns/, then the API endpoint is returning a 404 response so it is not finding this URL, which does not happen when using Google Chrome with sc-tool. I had set both domains www.example.com, and api.example.com on my /etc/hosts. I have been trying to solve it, but without any luck, is there something I'm missing? Any help will be welcome. Thanks in advance.
With a bit more of data, we may be able to find the issue. In the meantime, here is a list of troubleshooting tips:
Possible issue: DNS is cached in browser, /etc/hosts is not being used (yet)
This can happen if somehow your browser has not done a DNS lookup since before you changed your /etc/hosts file. Then the connection is going to a domain in the Internet that may not have the API endpoint that you are calling.
Troubleshooting: Check ShimmerCat's log for the requests. If this is the issue, closing and opening the browser may solve the issue.
Possible issue: the host header is incorrect
ShimmerCat uses the Host header in HTTP/1.1 requests and the :authority header in HTTP/2 requests to distinguish the domains. It always discards any port number present in them. If these headers are not set or are set to a domain other than the ones ShimmerCat is configured to listen, the server will consider the situation so despicable that it will just close the connection.
Troubleshooting: This is not a 404 error, but a connection close (if trying to connect un-proxied, directly to the SSL port where ShimmerCat is listening), or a Socks Connection Failed (if trying to connect through ShimmerCat's built-in SOCKS5 proxy). In the former case, the server will print the message "Rejected request to Just https://some-domain-or-ip/some/path" in his log, using the actual value for the domain, or "Rejected request to Nothing", if no header was present. The second case is more complicated, because the SOCKS5 proxy is before the HTTP routing algorithm.
In any case, the browser will put a red line in the network panel of the developer tools. If you are accessing the server using curl, like this:
curl -k -H host:api.incorrect-domain.com https://127.0.0.1:4043/contents/blog/data-density/
or like
curl -k -H host:api.incorrect-domain.com
(notice the --http2 parameter in the second form), you will get a response:
curl: (56) Unexpected EOF
Extra-tip: There is a field for the network address in the browser's developer tools. Check it, it may tell you something!
Possible issue: something gets messed up when passing the request to the api back-end.
API backends are also sensitive to the host header, and to additional things like authentication cookies and request parameters.
Troubleshooting: A way to diagnose things is invoking ShimmerCat using the --show-proxied-headers command-line option. It makes ShimmerCat to report the proxied headers to the log:
Issuing request with headers :authority: api.example.com
:method: GET
:path: /my/api/endpoint/path/
:scheme: https
accept: */*
user-agent: curl/7.47.0
Possible issue: there are two instances or more of ShimmerCat running
...and they are using different configurations. ShimmerCat uses port sharing among several processes to increase availability. A downside of this is that is perfectly possible to mistakenly start ShimmerCat, forget about stopping it, and start it again after changing some configuration bit. The two instances will be running at the same time, and any of them will pick connections made to the listening port.
Troubleshooting: Shutdown all instances of ShimmerCat, then double-check there are none running by using the corresponding form of the ps command, and start the server with the configuration you want.
The problem:
My url xyz.co is getting resolved into an ugly AWS public DNS name such as ec2-11-22-33-44.ap-southeast-2.compute.amazonaws.com. It doesn't stick to xyz.co.
Here's what I did:
I have set up my Route 53 configuration according to http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/MigratingDNS.html, so I created an A record pointing to the IP address and a CNAME alias record to allow for www.xyz.co. The domain is sitting with godaddy and the name servers are configured to the AWS delegation set.
The instance itself sits in the default VPC. I double-checked and DNS resolution and DNS host names are both active.
I'm a bit stuck here with this. Any help would be highly appreciated!
Cheers,
Bruno
What you are seeing isn't actually related to name resolution.
It's impossible for DNS to change what appears in the address bar of the web browser -- DNS and web browsers simply do not interact in a way that makes such behavior possible. Your URL is not "getting resolved to" this new value via anything DNS-related, since DNS, configured correctly or incorrectly, can't impact what shows up there, on its own.
The fact that navigating to the IP address has the same impact backs up this assertion.
What you are seeing is not related in any way to DNS or Route 53 or even EC2 or VPC. Your web server is, for whatever reason, configured to redirect incoming requests with any other hostname... over to the hostname you are subsequently seeing in the address bar (which is the one you don't like).
You should notice this in your web server's log. It will be issuing a 301 or 302 redirect on the initial request.
You should also be able to verify this yourself with the curl command line utility. Here, a server accessed as "www.example.com" is redirecting the browser to use its preferred address, "example.com." (Hostnames and addresses are sanitized, but the output is otherwise unmodified.)
$ curl -v www.example.com
* Rebuilt URL to: www.example.com/
* Hostname was NOT found in DNS cache
* Trying 203.0.113.139...
* Connected to www.example.com (203.0.113.139) port 80 (#0)
The next block of output is the request sent to the web server.
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: www.example.com
> Accept: */*
>
The http response from the web server includes a redirect.
< HTTP/1.1 301 Moved Permanently
< Content-length: 0
< Location: http://example.com/
< Connection: close
<
* Closing connection 0
If we were using a browser instead of a command line tool, this would cause the address bar to change to the new value, and establish a new connection to the web server (which might actually be the same one, or a different one... in this case, it's the same).
In spite of the fact that I had typed http://www.example.com into my browser, it would now show only http://example.com/. The same thing would happen if I typed in the IP address if my server was configured to redirect everything to one hostname, as yours appears to be. In my case, it's deliberately configured to do something else.
The above should illustrate that you do not actually have a DNS issue, and explain the mechanism that's causing this to occur (because you may find this to be something useful to do deliberately in the future, as my web servers do -- any www.* request gets stripped and rewritten without the www).
The issue is with your web server, telling the browser to use a different hostname. How to fix that will depend on what web server you are running and why it thinks the redirect is necessary.
I have the following problem and I am struggling to find if a solution exists for it, or what the best practice is here.
I have a site example.com, and multiple servers with different IP addresses around the world. I am seeing the following behaviour in my browser (Chrome) - for simplicity lets say I only have 2 IP addresses for now.
I connect to example.com and data is served from IP address A.B.C.D (server 1). After 40 seconds or, any subsequent request (GET/POST) to example.com then resolves to W.X.Y.Z (server 2). My issue is that I have a cookie based web session on server 1, and server 2 knows nothing about that session. There is no kind of back-end replication I can do to sync state between both servers.
Is there any way I can force the browser to only connect to a single server once a server has served the first page? I am using RR DNS with multiple A records at the moment. Would switching to CNAME solve this problem?
One solution I was thinking of was having each server reply with a configured domain in the http headers (e.g. server1 would reply with X-HEADER: server1.example.com, server2 would reply with X-HEADER: server2.example.com) and then force the browser to make requests to these. I would then have a single IP address for server1.example.com, and another for server2.example.com. Does this break same-origin policy though? If I am on example.com can I send GET/POST/PUT etc. to server1.example.com?
I'd really appreciate any advice on this - I'm so confused!