My requirement:
Prevent non Twilio access to my ALB managed Application using CloudFlare.
My restrictions:
Due to the nature of Twilio's cloud design, it is not possible to whitelist access down to a set of IPs due to the wide pool of IPs a request could come from.
Possible solution:
Twilio suggest a couple of options under https://www.twilio.com/docs/usage/security but I don't know how to use any of these methods as a means to only allow twilio Traffic. But any designed validation must only be applied to the dns record of /api in the url to my site.
Further Info:
The underlying application is written in php.
I would prefer a CloudFlare solution over changing code in the application.
A possible approach could be:
Use Cloudflare Firewall Rules to check for the presence of X-Twilio-Signature on your api path (as a first, basic check), block requests that do not have it.
Use a Cloudflare Worker, configured on your API path. The worker code can read X-Twilio-Signature and the request data, and use the procedure described in the Twilio documentation to validate it. If it matches, forward the request to your load balancer. If it doesn't, return an error to the caller.
Also, make sure your origin server only accepts traffic from Cloudflare to prevent direct tampering.
Related
I have a cloud run service behind an HTTPS external load balancer. I'm seeing a number of path scanning style requests (e.g. "/owa", "/admin") that are trying to find vulnerabilities. None of these are supplying a host header.
To reduce unnecessary startups of my cloud run container, and to minimise exposing information to random IP address scanners, I'm trying to find a way to reject requests to my load balancer that are missing a host header.
What I had hoped to do was to respond with a static 403 forbidden style response. However, from the documentation this isn't possible. The best I have come up with is specifying a "Default URL Redirect" in my url map to send all traffic to google.com that doesn't match any host rule in my url map. This works, but seems clunky and perhaps not appropriate.
Is it possible to send a static response - or is it only possible to redirect? Is there a better way to solve this?
The correct answer from a GCP product perspective:
Use a Cloud Armor rule to effect the desired behaviour
This allows you to control a range of aspects of the response.
Alternate options you could consider:
Create an empty bucket and have that as the default backend.
A barebones nginx service on cloud run as the default backend that returns your desired status code.
This is the first time that I am using load balancer... I have spent quite a bit of time going through documentation and I am still quite confused.
I want to host my website. My website supports HTTPS only. I want to put my backend servers behind an Application Load Balancer.
I am using AWS' default VPC, I have created an ALB (myALB) and installed my SSL certificate on it. I have also created 2 EC2 instances (myBackEndServer1 & myBackEndServer2).
Questions:
Should the communication between backend servers and myALB be
through HTTP or HTTPS?
I have created an HTTPS listener on myALB, do I also need an HTTP
listener on myALB? what I want is to redirect any HTTP request to
HTTPS (I believe this should happen on myALB)?
I want to use External ID login (using Facebook). I have set up Facebook
login to work with HTTPS only. Does the communication between
Facebook and my backend servers go through myALB? I mean, I either
need HTTPS on my backend servers, or the communication with facebook
should go through myALB.
I would appreciate any general advice.
You can use both HTTP and HTTPS listeners.
Yes, you can achieve that with ALB. You can add a rule to it that says that any request that is coming to port 80 will be redirected to port 443 on a permanent basis. Check out rules for ALB.
If you make a request from your instances to Facebook - it depends on Facebook, whether your communication will be encrypted, because in such case you are a client. However if you set up some webhook, Facebook is now a client and to communicate with you, you're gonna give your load balancer's DNS name. And due to the point 2 in this list, Facebook will be forced to use TLS.
I'm not sure I fully understood your question number three, but here's something you may also find useful. ALB has some features that allows to authenticate users with Cognito. It explicitly says that your EC2 instances can be abstracted away from any authentication, also if it makes use of Facebook ID or Google Id or whatever. Never tried though.
I am trying to access an AWS service directly from the browser- specifically the SNS service. I want to be able to post a message directly to an sns topic, but using a CNAME record so I can control which region the browser ultimately goes to (sns.mydomain.com -> sns.us-east-1.amazonaws.com | sns.eu-west-1.amazonaws.com depending on requesters region).
My issue is that if I make an HTTPS request to my aliased endpoint, the returned certificate will not be signed to my endpoint and the browser will refuse to work with it. And while I can get around this by making only HTTP requests, the browser will refuse to make an HTTP request from a secure origin (a site served on HTTPS).
Is it possible to have a CNAME point to an AWS service in the way that I'm trying to do it?
Ultimately, i'm trying to avoid locking the client application in the browser into an aws region.
Is it possible to have a CNAME point to an AWS service in the way that I'm trying to do it?
No. You're hitting up against a central feature of https verification, namely the Common Name of the cert or a SAN ( Subject Alternative Name) must match the certificate. If it weren't so, HTTPS would not be validating that the server is who they claim to be.
Ultimately, i'm trying to avoid locking the client application in the browser into an aws region.
That's a fine goal. Instead of doing so at the DNS layer, why not create an endpoint or configuration setting that supplies region or regions to use? A smart client could even iterate through regions in the case of some failures that appeared to be regional outages, which is somewhat better than a CNAME that you still have to fix when a region goes down.
I configured CDN in front of my website. Everything works well when you access the website via my custom DNS or CDN DNS. My problem is when I want to use an IP instead of DNS.
When I do a nslookup on my CDN DNS name, I get a list of IP's. If I grab one IP address from there and try to access the website, I get a 403 Forbidden request.
Why is CDN only accepting DNS request and not IP's?
What if I have a proxy in front of my CDN and try to access my website by using the proxy IP, how can I access the website using the proxy IP which points to CDN?
It's a wired requirement and time consuming, I've been looking for the correct answer. No one seems to show a solution.
Cheers!
Why is CDN only accepting DNS request and not IP's?
CloudFront is not designed to work this way. It is a massive, globally-distributed system. When you look up the IP addresses of your CloudFront distribution, you are receiving the list of addresses where CloudFront expects to receive traffic:
for your web site, and
for potentially hundreds or thousands of other sites, and
from browsers in the same geographic area as you
You need a way to identify which distribution you expect CloudFront to use when processing your request.
In HTTP mode, this uses the Host: HTTP header, sent by the browser. In HTTPS mode, this uses the TLS SNI value and the Host: header.
If you were using a proxy to access CloudFront, you would need for the proxy to inject a Host header for HTTP and to set the SNI correctly, too, for HTTPS.
In HAProxy, for example, set the host header, overwriting any such header that's already present.
http-request set-header Host dzczcexample.cloudfront.net
Of course, you might use any one of the Alternate Domain Name values configured for your distribution, as well.
For SNI:
backend my-cloudfront-backend
server my-cloudfront dzczcexample.cloudfront.net:443 ssl verify none sni str(dzczcexample.cloudfront.net)
(Source: https://serverfault.com/a/830327/153161)
But this is only a minimum baseline working configuration, because CloudFront has features that this simple setup overlooks.
As noted above, CloudFront is returning a list of IP addresses that should be used to access (1) your site, (2) from where you are, (3) right now. The list of addresses can and will vary. CloudFront appears to be able to dynamically manage and distribute its workload and mitigate DDoS by moving traffic from one set of servers to another, from one edge location to another, etc., by modifying the DNS responses... so your proxy needs to be using the multiple addresses returned, and needs to be refreshing its DNS values so that it always connects to where CloudFront wishes for it to connect, for optimum behavior and performance.
Also, don't overlook the fact that a proxy server will connect to CloudFront via an edge near the proxy, not near the browser, so this is not something you would routinely use in production, though it absolutely does have some valid use cases. (I have used HAProxy on both sides of CloudFront for several years, for certain applications -- some of which have now been obviated by Lambda#Edge, but I digress).
It's a wired [weird?] requirement
Not really. Name-based virtual hosting has been the standard practice for many years. It is -- in my opinion -- almost an accident of history that when you set up a web server, it will commonly respond on the IP address in the Host header, as well. A well-configured web server will not do this -- if you (the web browser) don't know what host you are asking for and are just sending a request to my IP, then I (the web server) should tell you I have no idea what you want from me, because you are more likely than not to be arriving for malicious reasons, or benign but annoying reasons (scanning), or as the result of a misconfiguration. You also don't want search engine spiders finding your content at an IP address. Bad for listings, bad for SEO.
The problem I have with this proxy is I cannot configure the headers. I was thinking to add another proxy where I have control and then modify the headers with the values I want. But this is not really a good solution, is like jumping from one proxy to another.
I think I should rely more on DNS and hostnames, rather than IP's. Which is fine with me, I prefer using proper DNS name.
Thanks for your thorough explanation, you clarify a lot of things.
I am hosting my website using AWS.
The website is on 2 ec2 instances, with a load balancer (ELB) balancing traffic between them.
Currently, I am using my DNS (Route 53) to restrict the access to the website by using Route 53's geolocation routing:
http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy.html#routing-policy-geo
(The geolocation restriction is just to limit the initial release of my website. It is not for security reasons. Meaning the restriction just needs to work for the general public)
This worries me a little because my load balancer is still exposed to access from everywhere. So I am concerned that my load balancer will get indexed by google or something and then people outside of my region will be able to access the site.
Are there any fixes for this? Am I restricting access by location the wrong way? Is there a way perhaps to specify in the ELB's security group that it only receive inbound traffic from my DNS (of course then I would also have to specify that inbound traffic from edge locations be allowed as well for my static content but this is not a problem)?
Note: There is an option when selecting inbound rules for a security group, under "type" to select "DNS(UDP)" or "DNS(TCP)". I tried adding two rules for both DNS types (and IP Address="anywhere") for my ELB but this did not limit access to the ELB to be solely through my DNS.
Thank you.
The simple solution, here, is found in CloudFront. Two solutions, actually:
CloudFront can use its GeoIP database to do the blocking for you...
When a user requests your content, CloudFront typically serves the requested content regardless of where the user is located. If you need to prevent users in specific countries from accessing your content, you can use the CloudFront geo restriction feature[...]
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/georestrictions.html
You can configure CloudFront with which countries are allowed, or which are denied. You can also configure static pages, stored in S3, which are displayed to denied users. (You can also configure static custom error pages for other CloudFront errors that might occur, and store those pages in S3 as well, where CloudFront will fetch them if it ever needs them).
...or...
CloudFront can pass the location information back to your server using the CloudFront-Viewer-Country: header, and your application code, based on the contents accompanying that header, can do the blocking. The incoming request looks something like this (some headers munged or removed for clarity):
GET / HTTP/1.1
Host: example.com
X-Amz-Cf-Id: 3fkkTxKhNxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==
Via: 1.1 cb76b079000000000000000000000000.cloudfront.net (CloudFront)
CloudFront-Viewer-Country: US
CloudFront-Forwarded-Proto: https
Accept-Encoding: gzip
CloudFront caches the responses against the combination of the requested page and the viewer's country, and any other whitelisted headers, so it will correctly cache your denied responses as well as your allowed responses, independently.
Here's more about how you enable the CloudFront-Viewer-Country: header:
If you want CloudFront to cache different versions of your objects based on the country that the request came from, configure CloudFront to forward the CloudFront-Viewer-Country header to your origin. CloudFront automatically converts the IP address that the request came from into a two-letter country code.
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/header-caching.html#header-caching-web-location
Or, of course, you can enable both features, letting CloudFront do the blocking, while still giving your app a heads-up on the country codes for the locations that were allowed through.
But how do you solve the issue with the fact that your load balancer is still open to the world?
CloudFront has recently solved this one, too, with Custom Origin Headers. These are secret custom headers sent to your origin server, by CloudFront, with each request.
You can identify the requests that are forwarded to your custom origin by CloudFront. This is useful if you want to know whether users are bypassing CloudFront[...]
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/forward-custom-headers.html
So, let's say you added a custom header to CloudFront:
X-Yes-This-Request-Is-Legit: TE9MIHdoYXQgd2VyZSB5b3UgZXhwZWN0aW5nIHRvIHNlZT8=
What's all that line noise? Nothing, really, just a made up secret value that only your server and CloudFront know about. Configure your web server so that if this header and value are not present in the incoming request, then access is denied -- this is a request that didn't pass through CloudFront.
Don't use the above secret, of course... make up your own. It's entirely arbitrary.
Caveat applicable to any GeoIP-restricting strategy: it isn't perfect. CloudFront claims 99.8% accuracy.
The most reliable way to implement geographic IP restrictions, is to use a geographic location database or service API, and implement it at the application level.
For example, for a web site in practically any language, it is very simple to add test at the start of each page request, and compare the client IP against the geo ip database or service, and handle the response from there.
At the application level, it is easier to manage the countries you accept/deny, and log those events, as needed, than at the network level.
IP based geo location data is generally reliable, and there are many sources for this data. While you may trust AWS for many things, I do think that there are many reliable 3rd party sources for geo IP dat, that focus on this data.
freegeoip.net provides a public HTTP API to search the geolocation of IP addresses. You're allowed up to 10,000 queries per hour.
ip2location.com LITE is a free IP geolocation database for personal or commercial use.
If your application uses a database, these geo databases are quite easy to import and reference in your app.
I have a post explaining in detail how to whitelist / blacklist locations with Route53: https://www.devpanda.me/2017/10/07/DNS-Blacklist-of-locations-countries-using-AWS-Route53/.
In terms of your ELB being exposed to public that shouldn't be a problem since the Host header on any requests to the ELB over port 80 / 443 won't match your domain name, which means for most web servers a 404 will be returned or similar.
There is a way using AWS WAF
You can select - Resource type to associate with web ACL as ELB.
Select your ELB and create conditions like Geo Match, IP Address etcetera.
You can also update anytime if anything changes in future.
Thanks