I'm running a backend app with several endpoints on Cloud Run(fully-managed). My endpoints are publicly available by its nature so I don't want to authenticate users through my client app hosted on Netlify.
What I do need is to restrict access to my endpoints so that other applications or malicious users can't abuse it. It is not about scaling, I just don't want to exceed the Free Tier limits since it is a demo of an opensource application.
I've already set the concurrency and max instance limits to minimum but this alone is not enough. There is also a product named Google Cloud Armor but it seems an expensive one, not free.
I was expecting to have a simple built-in solution for this but couldn't find it.
What other solutions do I have? How can I block the traffic coming out of my website on Netlify?
You don't have a lot of solution:
You don't want to authenticate your users -> so you need to rely on the technical layers
Netlify is a serverless hosting platform, you don't manage servers/IPs -> So you need to rely on the host name
To filter on the host name, you can use 2 products
External HTTPS only (about $15 per month) with url path matching.
Default URL land on a dummy service
Only request where the host matches your netlify host name are redirected to your backend
Use Cloud Armor on top of External HTTPS load balancer ($15 + Cloud Armor policy x traffic volume). The time, the load balancer redirect the default URL to the correct backend and Cloud Armor check the request origin.
The problem is that this weak solution is easy to overpass. Perform a simple curl with the host as header, and HTTPS Load Balancer and Cloud Armor think that is the correct origin
curl -H 'Host: myNetlifyHost.com' ....
The highest protection is the authentication. Google Cloud itself say: "Don't trust the network".
We are a website builder SaaS platform, where users are building their FREE websites and are connecting their domains.
We would like to set a limit per domain, example 100mb/month, so when that bandwidth limit is hit we can restrict access to that domain with a message like "You have reached your limit".
This limit can be set for example via custom headers in the request for the specific domain or maybe on a Load Balancer's level.
We weren't able to find documentation on that topic, so decided to ask here!
Please help!
Regards,
Gev
It's possible to use Pulse Virtual Traffic Manager for control the network traffic through your domains. It's in Google Marketplace for deploy directly in your infrastructure.
Also, you can think to use the HTTPs load balancing features in combination with Cloud DNS forwarding and Cloud CDN (caching) for optimize the network traffic and redirections to the different domains.
If I'm in the northeast US and trying to talk to a site on AWS, behind CloudFront, I'm generally getting routed to servers in us-east. For a particular hostname (say, myfico.com), is there any way to force e.g. curl to talk with servers in a different region, presumably by routing my traffic to a different CloudFront edge server?
Since amazon publishes the IP ranges of all their edge servers, I suspect it might be possible to do something like overriding some of my DNS records and forcing it to use an IP prefix from a different region. IT might also be possible to do a more general location spoofing by modifying the edns-client-subnet that's passed along in the DNS request, but I think this might not be possible.
AWS regions are not really related to the CloudFront CDN. Generally speaking AWS region is the region where the server is physically located while the CDN offers a termination point to the client so that it can get a reply from the server quickly. You can possibly hit a CDN node in West US but the request may still be routed to us-east if the website is configured that way.
We have two VPCs one in US and the other in Europe. Each VPC has a load balancer in public subnets and we run our web application in private subnets. What we want is, if a user visits our website (makes a request to our host/domain) the request gets redirected to European VPC if the user is from Europe region and to US VPC if the user requests from US.
Route53 can help you with this
Route53 Geo Routing this allows DNS to return different values depending on the place that the DNS request was made from. See the "Geo Routing" section of this blog post for a good summary https://aws.amazon.com/blogs/aws/route-53-domain-reg-geo-route-price-drop/
You should also look at AWS CloudFront CDN. This can move static content to a global service that is based near your customer. So when a customer in for example Australia asks for static files (for example js or css) then these files are actually served from an endpoint in Australia. You don't have to set up any of the end points you just put the files in an S3 bucket
I am hosting my website using AWS.
The website is on 2 ec2 instances, with a load balancer (ELB) balancing traffic between them.
Currently, I am using my DNS (Route 53) to restrict the access to the website by using Route 53's geolocation routing:
http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy.html#routing-policy-geo
(The geolocation restriction is just to limit the initial release of my website. It is not for security reasons. Meaning the restriction just needs to work for the general public)
This worries me a little because my load balancer is still exposed to access from everywhere. So I am concerned that my load balancer will get indexed by google or something and then people outside of my region will be able to access the site.
Are there any fixes for this? Am I restricting access by location the wrong way? Is there a way perhaps to specify in the ELB's security group that it only receive inbound traffic from my DNS (of course then I would also have to specify that inbound traffic from edge locations be allowed as well for my static content but this is not a problem)?
Note: There is an option when selecting inbound rules for a security group, under "type" to select "DNS(UDP)" or "DNS(TCP)". I tried adding two rules for both DNS types (and IP Address="anywhere") for my ELB but this did not limit access to the ELB to be solely through my DNS.
Thank you.
The simple solution, here, is found in CloudFront. Two solutions, actually:
CloudFront can use its GeoIP database to do the blocking for you...
When a user requests your content, CloudFront typically serves the requested content regardless of where the user is located. If you need to prevent users in specific countries from accessing your content, you can use the CloudFront geo restriction feature[...]
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/georestrictions.html
You can configure CloudFront with which countries are allowed, or which are denied. You can also configure static pages, stored in S3, which are displayed to denied users. (You can also configure static custom error pages for other CloudFront errors that might occur, and store those pages in S3 as well, where CloudFront will fetch them if it ever needs them).
...or...
CloudFront can pass the location information back to your server using the CloudFront-Viewer-Country: header, and your application code, based on the contents accompanying that header, can do the blocking. The incoming request looks something like this (some headers munged or removed for clarity):
GET / HTTP/1.1
Host: example.com
X-Amz-Cf-Id: 3fkkTxKhNxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==
Via: 1.1 cb76b079000000000000000000000000.cloudfront.net (CloudFront)
CloudFront-Viewer-Country: US
CloudFront-Forwarded-Proto: https
Accept-Encoding: gzip
CloudFront caches the responses against the combination of the requested page and the viewer's country, and any other whitelisted headers, so it will correctly cache your denied responses as well as your allowed responses, independently.
Here's more about how you enable the CloudFront-Viewer-Country: header:
If you want CloudFront to cache different versions of your objects based on the country that the request came from, configure CloudFront to forward the CloudFront-Viewer-Country header to your origin. CloudFront automatically converts the IP address that the request came from into a two-letter country code.
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/header-caching.html#header-caching-web-location
Or, of course, you can enable both features, letting CloudFront do the blocking, while still giving your app a heads-up on the country codes for the locations that were allowed through.
But how do you solve the issue with the fact that your load balancer is still open to the world?
CloudFront has recently solved this one, too, with Custom Origin Headers. These are secret custom headers sent to your origin server, by CloudFront, with each request.
You can identify the requests that are forwarded to your custom origin by CloudFront. This is useful if you want to know whether users are bypassing CloudFront[...]
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/forward-custom-headers.html
So, let's say you added a custom header to CloudFront:
X-Yes-This-Request-Is-Legit: TE9MIHdoYXQgd2VyZSB5b3UgZXhwZWN0aW5nIHRvIHNlZT8=
What's all that line noise? Nothing, really, just a made up secret value that only your server and CloudFront know about. Configure your web server so that if this header and value are not present in the incoming request, then access is denied -- this is a request that didn't pass through CloudFront.
Don't use the above secret, of course... make up your own. It's entirely arbitrary.
Caveat applicable to any GeoIP-restricting strategy: it isn't perfect. CloudFront claims 99.8% accuracy.
The most reliable way to implement geographic IP restrictions, is to use a geographic location database or service API, and implement it at the application level.
For example, for a web site in practically any language, it is very simple to add test at the start of each page request, and compare the client IP against the geo ip database or service, and handle the response from there.
At the application level, it is easier to manage the countries you accept/deny, and log those events, as needed, than at the network level.
IP based geo location data is generally reliable, and there are many sources for this data. While you may trust AWS for many things, I do think that there are many reliable 3rd party sources for geo IP dat, that focus on this data.
freegeoip.net provides a public HTTP API to search the geolocation of IP addresses. You're allowed up to 10,000 queries per hour.
ip2location.com LITE is a free IP geolocation database for personal or commercial use.
If your application uses a database, these geo databases are quite easy to import and reference in your app.
I have a post explaining in detail how to whitelist / blacklist locations with Route53: https://www.devpanda.me/2017/10/07/DNS-Blacklist-of-locations-countries-using-AWS-Route53/.
In terms of your ELB being exposed to public that shouldn't be a problem since the Host header on any requests to the ELB over port 80 / 443 won't match your domain name, which means for most web servers a 404 will be returned or similar.
There is a way using AWS WAF
You can select - Resource type to associate with web ACL as ELB.
Select your ELB and create conditions like Geo Match, IP Address etcetera.
You can also update anytime if anything changes in future.
Thanks