I have the following problem and I am struggling to find if a solution exists for it, or what the best practice is here.
I have a site example.com, and multiple servers with different IP addresses around the world. I am seeing the following behaviour in my browser (Chrome) - for simplicity lets say I only have 2 IP addresses for now.
I connect to example.com and data is served from IP address A.B.C.D (server 1). After 40 seconds or, any subsequent request (GET/POST) to example.com then resolves to W.X.Y.Z (server 2). My issue is that I have a cookie based web session on server 1, and server 2 knows nothing about that session. There is no kind of back-end replication I can do to sync state between both servers.
Is there any way I can force the browser to only connect to a single server once a server has served the first page? I am using RR DNS with multiple A records at the moment. Would switching to CNAME solve this problem?
One solution I was thinking of was having each server reply with a configured domain in the http headers (e.g. server1 would reply with X-HEADER: server1.example.com, server2 would reply with X-HEADER: server2.example.com) and then force the browser to make requests to these. I would then have a single IP address for server1.example.com, and another for server2.example.com. Does this break same-origin policy though? If I am on example.com can I send GET/POST/PUT etc. to server1.example.com?
I'd really appreciate any advice on this - I'm so confused!
Related
I am following the instructions at https://cloud.google.com/cdn/docs/setting-up-cdn-with-ex-backend-internet-neg and https://medium.com/the-innovation/how-to-enable-google-cdn-for-custom-origin-websites-google-cdn-for-external-websites-56e3fe66cca9 to setup Google Cloud CDN for my website www.datanumen.org.
For the "Fully qualified domain name and port" in "New network endpoint", I choose www.datanumen.org.
All others are same as the above two articles, I use HTTP protocol for all the communications. Finally I get a frontend IP address 34.96.69.82. So I try to visit http://34.96.69.82/, but get a default "SORRY" web page instead of the contents from www.datanumen.org. Why?
Also later I plan to update the DNS A Record for www.datanumen.org so that datanumen.org will points to 34.96.69.82 instead of its current IP address. I am just curious that if I do that, then since what I put in "Fully qualified domain name and port" in "New network endpoint" is www.datanumen.org, will it cause the following deadloop:
a user visit www.datanumen.org
Based on DNS A record, he will go to 34.96.69.82(frondend)
The frontend will request data from backend, and the endpoint is www.datanumen.org,
Based on DNS A record, the backend end point will also solved to 34.96.69.82.
Thus will cause a deadloop for ever?
Update:
For the 1st question, I find the solution. My website is hosted on a server with shared IP. In article https://cloud.google.com/cdn/docs/setting-up-cdn-with-ex-backend-internet-neg, it asks me to add "Host" to the request header, which is used to identify the actual site to be accessed when the request reaches the original server. In my previous configuration, I thought this step is useless so I just skip it. After adding the "Host" field, now I can visit my website properly with the IP address given by Google.
You already fixed your fist issue so you're at a point when you can successfully access your site using an IP address.
Right now all you need is a CDN enabled - if it's not you can do it in the "backend" section of the "load balancing" page.
Have a look also at Network Endpoint Groups documentation to see how to create a load balancer utilising them (the only way you can use GCP's CDN for external site).
To answer your second question - there will be no loop - your site works properly.
Since you're not using secure HTPPS (only HTTP) then you don't have to worry about SSL certificates and the only thing that remains for you to do is to direct your domain to your load-balancer's IP and you're done.
If you encounter any issues or just want to check if CDN is working correctly then have a look at CDN troubleshooting page.
Most simple way to verify if it's working is to use curl: curl -s -D - -o /dev/null http://example.com/style.css and see if you have Cache-Control line present in the output:
HTTP/1.1 200 OK
Date: Tue, 16 Feb 2016 12:00:31 GMT
Content-Type: text/css
Content-Length: 1977
Cache-Control: max-age=86400,public
Via: 1.1 google
However I recommend using HTTPS and SSL certificates for the security reasons - it's much harder to spoof the traffic/listen to the between the site and the client. It's not mandatory though.
I have a bit problem with my site.
So setup is ElasticBeanstalk(NGINX) + Cloudflare
But each day around 4AM I have direct IP attack to my server.
Around 300 requests in 1-2 minutes.
Bot try to access some resources like
GET /phpMyadmi/index.php HTTP/1.1
GET /shaAdmin/index.php HTTP/1.1
POST /htfr.php HTTP/1.1
For now all of them going to 80 or 8080 ports.
And successfully handled by Nginx configuration that redirect it to example:443
server {
listen 80 default_server;
listen 8080 default_server;
server_name _;
return 301 https://example.com$request_uri;
}
server {
listen 443 ssl;
server_name example.com;
ssl on;
...
So questions are,
have many site owners/devOps face the same attack. What is your action to prevent such attacks.
For now it is handled very well and did not affect on server work, should I worry about it? Or just filter out logs with /phpmy/ pattern and forgot about it.
Before this attacks I have request with method PROPFIND, should I blocked it for security reasons? It is handled by default server for now.
I know that I can use Cloudflare Argotunel or ELB + WAF. But I am not really want to do it for now.
I have found one solution on stackoverflow. Is whitelist of all cloudflare ips. But i think it is not a good one.
Also another solution that should work I guess it is to check Host header, and compare it with 'example.com'.
To answer your specific questions:
Every public IP receives unwanted traffic like you describe, sadly its pretty normal. This isnt really an attack as such, its just a bot looking for signs of specific weaknesses, or otherwise trying to provoke a response that contains useful data. This data is no doubt later used in actual attacks, but its basically automated recognisance on a potentially massive scale.
This kind of script likely isnt trying to do any damage, so as long your server is well configured & fully patched its not a big concern. However these kinds of scans are first step towards launching an attack - by identifying services & application versions with known vulnerabilities - so its wise to keep your logs for analysis.
You should follow the principle of least privilege. PROPFIND is related to WebDAV - if you dont use it, disable it (or better white list the verbs you do support and ignore the rest).
If your site is already behind CloudFlare then you really should firewall access to your IP so only Cloudflares IPs can talk to your server. Those IPs do change, so I would suggest a script to download the latest from https://www.cloudflare.com/ips-v4 and have it periodically update your firewall. Theres a slightly vuage help article from CloudFlare on the subject here: https://support.cloudflare.com/hc/en-us/articles/200169166-How-do-I-whitelist-Cloudflare-s-IP-addresses-in-iptables-
If for whatever reason you cant firewall the IP, your next best option is something like fail2ban (www.fail2ban.org) - its a log parser that can manipulate the firewall to temporarily or permanently block an IP address based on patterns found in your log files.
A final thought - id advise against redirecting from your IP to your domain name - your telling the bot/hackers your URL - which they can then use to bypass the CDN and attack your server directly. Unless you have some reason to allow HTTP/HTTPS traffic to your IP address, return a 4XX (maybe 444 a " Connection Closed Without Response") instead of redirecting when requests hit your IP. You should then create a separate server block to handle your redirects, but only have it respond to genuine named URLs.
I have following suspicious logs in my django output logs. Somebody is doing vulnerability check or what?
Invalid HTTP_HOST header: '47.95.231.250:58204'. You may need to add '47.95.231.250' to ALLOWED_HOSTS.
[03/Dec/2017 20:09:28] "GET http://47.95.231.250:58204/ip_js.php?IP=my_ip&DK=my_port&DD=FOQGCINPZHEHIIFR HTTP/1.0" 400 62446
How can I prevent it? Tried to block 47.95.231.250 IP, but didn't help. Request is coming from different IP address probably
Check your server - you will very likely find that 47.95.231.250 is your own server's IP address! This error indicates that someone is able to get to your server but that your Django application is not set to respond to the requests based on IP address. If it is working otherwise then you actually have ALLOWED_HOSTS set correctly based on domain name. Do NOT add the IP address to your ALLOWED_HOSTS unless you actually want to access it by IP address, which is usually not necessary in a production system.
So the IP address access is an indication of someone trying to get it that shouldn't be allowed. The port 58204 is also a clue. Regular ports for most web servers are 80 & 443. Occasionally, in order to have alternate ports for different applications, you will see 8000 or 8080 or other numbers. 58204 is not a typical web site port number. The third clue is that the requested file is ip_js.php which indicates a request for a PHP-based web site and not Django/Python.
Bottom line: See if you can configure your firewall to allow ONLY the necessary open ports from the outside world in to your server. Typically this will include:
80 - http
443 - https
22 - ssh
and possibly others depending on how your server is configured and what applications it runs. For example, if you host MySQL or another database on the same box then you will need to open additional ports if-and-only-if you require remote access to the database outside of the application.
Here is my setup.
Public site hosted by squarespace.com (www.example-domain.com)
Web application (AWS EC2/ELB), i would like to be available via the same domain. (my.example-domain.com)
Custom profile pages available as www.example-domain.com/username
My question is how can i setup the DNS to achieve this? If can't do it just through DNS, any suggestions? The problem i am facing is that if squarespace.com is handling the www.example-domain.com traffic how can i have it only partially handle it for certain urls. Maybe i am going about this in the wrong was all together though.
The two first are ok. As you mention, (1) is not compatible with (3) for a pure DNS config as www of example-domain.com has to be configured to a single end-point.
Some ideas of non-DNS workaround:
Having the squarespace.com domain on sqsp.example-domain.com and configure your www domain to a custom web server on which you configure the root (/) to redirect (HTTP 300) to sqsp.example-domain.com. It will be quite transparent for the user, except in his browser address.
The same but setting on / a full page HTML iframe containing sqsp.example-domain.com.
The iframe approach is a "less clean", Google the solutions to build your opinion.
EDIT:
As #mike-ryan mentioned, there is the proxy solution as well where you configure you web server to request another server to get the content to return to your user. If you are already using AWS, a smart way to do this is to use CloudFront: you can setup CloudFront to proxy one server on one URL and proxy another server on other URL. Actually, this is maybe the faster to way to implement you need. Of course, a proxy is one more "hop", so it may add more delay.
If you really want to have content served from different servers while only using a single domain name, you'll need to set up a proxy server to handle the request routing for you. I am assuming your custom profile pages must be served from your EC2 instance.
Nginx will receive all requests, and will then decide whether they should be sent to Square Space or your web app. Requests will be reverse proxied to Square Space or to your app, depending on the URL.
This is similar to #smad's answer, except it will all be invisible to the users which IMHO is better than redirecting the user to a new domain name.
Example steps:
Set up an Nginx server, create two virtual hosts - one for my.example.com, and one for www.example.com
Create two upstreams in your Nginx config - one for Square Space, and one for your app
Configure the www.example.com virtual host to reverse proxy connections to the Square Space upstream, if the URL is "/". Otherwise, traffic should be proxied to your app upstream [0]
Configure the my.example.com virtual host to proxy all traffic to your app upstream
[0] how to reverse proxy via nginx a specific url?
The problem:
My url xyz.co is getting resolved into an ugly AWS public DNS name such as ec2-11-22-33-44.ap-southeast-2.compute.amazonaws.com. It doesn't stick to xyz.co.
Here's what I did:
I have set up my Route 53 configuration according to http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/MigratingDNS.html, so I created an A record pointing to the IP address and a CNAME alias record to allow for www.xyz.co. The domain is sitting with godaddy and the name servers are configured to the AWS delegation set.
The instance itself sits in the default VPC. I double-checked and DNS resolution and DNS host names are both active.
I'm a bit stuck here with this. Any help would be highly appreciated!
Cheers,
Bruno
What you are seeing isn't actually related to name resolution.
It's impossible for DNS to change what appears in the address bar of the web browser -- DNS and web browsers simply do not interact in a way that makes such behavior possible. Your URL is not "getting resolved to" this new value via anything DNS-related, since DNS, configured correctly or incorrectly, can't impact what shows up there, on its own.
The fact that navigating to the IP address has the same impact backs up this assertion.
What you are seeing is not related in any way to DNS or Route 53 or even EC2 or VPC. Your web server is, for whatever reason, configured to redirect incoming requests with any other hostname... over to the hostname you are subsequently seeing in the address bar (which is the one you don't like).
You should notice this in your web server's log. It will be issuing a 301 or 302 redirect on the initial request.
You should also be able to verify this yourself with the curl command line utility. Here, a server accessed as "www.example.com" is redirecting the browser to use its preferred address, "example.com." (Hostnames and addresses are sanitized, but the output is otherwise unmodified.)
$ curl -v www.example.com
* Rebuilt URL to: www.example.com/
* Hostname was NOT found in DNS cache
* Trying 203.0.113.139...
* Connected to www.example.com (203.0.113.139) port 80 (#0)
The next block of output is the request sent to the web server.
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: www.example.com
> Accept: */*
>
The http response from the web server includes a redirect.
< HTTP/1.1 301 Moved Permanently
< Content-length: 0
< Location: http://example.com/
< Connection: close
<
* Closing connection 0
If we were using a browser instead of a command line tool, this would cause the address bar to change to the new value, and establish a new connection to the web server (which might actually be the same one, or a different one... in this case, it's the same).
In spite of the fact that I had typed http://www.example.com into my browser, it would now show only http://example.com/. The same thing would happen if I typed in the IP address if my server was configured to redirect everything to one hostname, as yours appears to be. In my case, it's deliberately configured to do something else.
The above should illustrate that you do not actually have a DNS issue, and explain the mechanism that's causing this to occur (because you may find this to be something useful to do deliberately in the future, as my web servers do -- any www.* request gets stripped and rewritten without the www).
The issue is with your web server, telling the browser to use a different hostname. How to fix that will depend on what web server you are running and why it thinks the redirect is necessary.