We have a server configured under a ALB associated with a WAF
As the underlying service receives requests for your web sites, it forwards those requests to AWS WAF for inspection against your rules. Once a request meets a condition defined in your rules, AWS WAF instructs the underlying service to either block or allow the request based on the action you define.
Above is mentioned in AWS FAQ and according to that I have a understanding that requests first comes to ALB and then forward to WAF.
My questions are in my environment I see some bad requests comes and returns 400 on ALB. they are not counted in WAF so is that mean bad requests are process in ALB? Bad requests are not forwarded to WAF? If I want to forward all the ALB request to WAF is it possible?
Related
I've been working on improving performance of my APIs hosted on various AWS services this week, and I've found that a significant portion of the time on some of the non-Lambda calls (namely ECS) is taken by DNS Lookup, TCP Handshake, and SSL Handshake. Below are the response times broken down by Postman.
The endpoint pointing to an ECS service does not cache the handshakes
Headers returned by ECS endpoint
The endpoint pointing to a Lambda does
Headers returned by Lambda endpoint
More details:
Both services are hosted in the same zone, the API Gateway for both is also hosted in the same zone, the only difference is that for the Lambda, the route goes from Route53 -> API Gateway -> Lambda integration, where the ECS goes Route53 -> API Gateway -> Application load balancer (private VPC link, HTTP) -> ECS service, the service being an NGINX reverse proxy listening for HTTP and routing those requests to a Django container hosted in the same service.
Question:
I'm looking to enable or set up caching for the ECS service in order to cut down response times to below or close to 100ms. I haven't found any details on this caching behavior or by extension how to set it up. How can I go about setting it up, and perhaps read more about this behavior? Thanks
Does the Lambda request come second? The cache will reuse the same handshake for each sub/domain within the session, so the first should always have a penalty. Caching the DNS is generally at machine level, but TLS/SSL has to be handshook on every new session.
In both situations, it is API Gateway which terminates your public SSL. API Gateway makes a separate internal request to Lambda or ECS. Depending how you have setup your routing, you could also remove the API Gateway from the ECS route and have the NGINX terminate SSL and serve the certificate and remove one of the (slower) hops?
1.) BotControl
2.) AmazonIpReputationList
3.) AnonymousIp
4.) CommonRuleSet
5.) KnownBadInputs
6.) LinuxRuleSet
7.) SQL injection
these are all rule sets I have set in place but is there a way for me to tests against them to make sure they are working as intended?
basically you would like to test your rules, here is my 2 cents
Counting the web requests that match the rules in a web ACL
Configure all the rules in a web ACL to count web requests
Set the default action for the web ACL to allow requests
Viewing a sample of the web requests that API Gateway CloudFront or an Application Load Balancer has forwarded to AWS WAF Classic
In the AWS WAF Classic console, you can view a sample of the requests
that API
Gateway CloudFront or an Application Load Balancer has forwarded to AWS WAF
Classic for inspection. For each sampled request, you can view detailed data
about the request, such as the originating IP address and the headers included
in the request. You also can view which rule the request matched, and whether
the rule is configured to allow or block requests.
Logging and monitoring web ACL traffic
You can enable logging to get detailed information about traffic that
is analyzed by your web ACL. Logged information includes the time that
AWS WAF received a web request from your AWS resource, detailed
information about the request, and details about the rules that the
request matched. You can send your logs to an Amazon CloudWatch Logs
log group, an Amazon Simple Storage Service (Amazon S3) bucket, or an
Amazon Kinesis Data Firehose.
please refer this doc for more information.
I've been wasting about 12 hours going in circles in what seems like this:
I am trying to just make a simple static landing page in lambda and hook the root of a domain to it.
The landing page works, but api gateway didn't because AWS doesn't seem to set permissions properly by default ("internal server error" with API gateway and lambda on AWS) but now the gateway link works.
So the next steps were the following:
add a custom domain name in the api gateway
add the api mapping in the custom domain name
in route 53, create a wildcard certificate with *.domain.com and domain.com
create an A record that points to the api gateway with domain.com
create a CNAME record that points to the A record
and I get an error 403 with absolutely nothing in the log. I log both 'default' and '$default' stages in the api gateway.
I read https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-403-error-lambda-authorizer/ which is all about looking at what's in the logs...
and I find the doc is both everywhere and nowhere because it's built as chunks of 'do this' and 'do that' without ever painting a whole picture of how each piece is connected to the other, or any graph with the hierarchy of services, etc. Reminds me of code that works only when you follow the example documented and breaks otherwise.
I'm sure I'm doing something wrong, but given the lack of logs and lack of cohesive documentation, I have no idea about the problem.
Not to mention that http doesn't even connect, just https.
Can anyone outline the steps needed to achieve this? essentially: [http|https]://(www).domain.com -> one lambda function
You cannot use API Gateway for an HTTP request; it only supports HTTPS.
From the Amazon API Gateway FAQs (emphasis mine):
Q: Can I create HTTPS endpoints?
Yes, all of the APIs created with Amazon API Gateway expose HTTPS endpoints only. Amazon API Gateway does not support unencrypted (HTTP) endpoints. By default, Amazon API Gateway assigns an internal domain to the API that automatically uses the Amazon API Gateway certificate. When configuring your APIs to run under a custom domain name, you can provide your own certificate for the domain.
You can use CloudFront to automatically redirect HTTP to HTTPS. How do I set up API Gateway with my own CloudFront distribution? provides a pretty simple walkthrough of connecting an API Gateway to CloudFront (you can skip the API Gateway portion and use the one you created). The important thing you'll need to do that is not in that document is to select Redirect HTTP to HTTPS.
If you truly need HTTP traffic you're probably going to need to go with an ALB.
I'm developing little service using lambda functions which returns "Fact of the day" in your CLI using curl.
First, I developed business logic, deployed and created lambda using Serverless.
Second, I bought domain using aws route 53, Provisioned certificate and routed domain using `Custom Domain Name on API gateway.
At the moment if you would visit https://domain.io service works as intented but if you would try call curl domain.io it outputs:
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>CloudFront</center>
</body>
</html>
My goal is to get service running without SSL (or redirect), by calling curl domain.io.
Is it possible to avoid redirection? Or can you create API custom domain name without certificate?
Currently I call curl -F domain.io it will follow redirect, but it's not solution I'm looking for.
Thank you!
Remove the custom domain configuration from API Gateway.
Wait a few minutes for API Gateway to release the custom domain in the AWS Edge Network (there isn't a way to determine when this is complete, but you'll get an error on one of the subsequent steps until it is. 20 minutes should be sufficient).
Create a CloudFront distribution, using the generic ...execute-api...amazonaws.com domain name assigned to your API stage.
For Origin Protocol Policy, select HTTPS Only.
Set the Origin Path to your stage prefix (e.g. /prod or /v1) -- whatever you set up as the stage prefix.
Set the Viewer Protocol Policy to HTTP and HTTPS.
Set the Minimum TTL and Default TTL to 0.
Set the Alternate Domain Name for the distribution to your custom domain.
If you want SSL to optionally work on your custom domain, associate an ACM certificate with the CloudFront distribution.
Change your DNS entry to point to the *.cloudfront.net hostname assigned to your distribution.
Wait for the CloudFront distribution state to change from In Progress to Deployed.
Test.
This seems like a lot of effort to enable HTTP against API Gateway, but it is necessary, because API Gateway was specifically designed not to support HTTP -- it only works with HTTPS, because that's a best-practice for APIs, generally.
Q: Can I create HTTPS endpoints?
Yes, all of the APIs created with Amazon API Gateway expose HTTPS endpoints only. Amazon API Gateway does not support unencrypted (HTTP) endpoints.
https://aws.amazon.com/api-gateway/faqs/
CloudFront is commonly known as a CDN, but it is in fact something of a Swiss Army knife of custom HTTP request manipulation, and this is a case of that.
Once you verify your behavior, you can optionally increase the Default TTL in CloudFront, which will cause it to cache responses for up to that value in seconds, reducing your costs by sending fewer actual requests to API Gateway and replaying cached responses to the callers.
This setup differs from what you have, now, because you are in control of the CloudFront distribution, instead of API Gateway... so you can customize it in ways that API Gateway doesn't allow when it is in control.
I need to use AWS WAF for my web application hosted on AWS to provide additional rule based security to it. I couldnt find any way to directly use WAF with ELB and WAF needs Cloudfront to add WEB ACL to block actions based on rules.
So, I added my Application ELB CNAME to cloudfront, only the domain name, WebACL with an IP block rule and HTTPS protocol was updated with cloudfront. Rest all has been left default. once both WAF and Cloudfront with ELB CNAME was added, i tried to access the CNAME ELB from one of the ip address that is in the block ip rule in WAF. I am still able to access my web application from that IP address. Also, I tried to check cloudwatch metrics for Web ACL created and I see its not even being hit.
First, is there any good way to achieve what I am doing and second, is there a specific way to add ELB CNAME on cloudfront.
Thanks and Regards,
Jay
Service update: The orignal, extended answer below was correct at the time it was written, but is now primarily applicable to Classic ELB, because -- as of 2016-12-07 -- Application Load Balancers (elbv2) can now be directly integrated with Web Application Firewall (Amazon WAF).
Starting [2016-12-07] AWS WAF (Web Application Firewall) is available on the Application Load Balancer (ALB). You can now use AWS WAF directly on Application Load Balancers (both internal and external) in a VPC, to protect your websites and web services. With this launch customers can now use AWS WAF on both Amazon CloudFront and Application Load Balancer.
https://aws.amazon.com/about-aws/whats-new/2016/12/AWS-WAF-now-available-on-Application-Load-Balancer/
It seems like you do need some clarification on how these pieces fit together.
So let's say your actual site that you want to secure is app.example.com.
It sounds as if you have a CNAME elb.example.com pointing to the assigned hostname of the ELB, which is something like example-123456789.us-west-2.elb.amazonaws.com. If you access either of these hostnames, you're connecting directly to the ELB -- regardless of what's configured in CloudFront or WAF. These machines are still accessible over the Internet.
The trick here is to route the traffic to CloudFront, where it can be firewalled by WAF, which means a couple of additional things have to happen: first, this means an additional hostname is needed, so you configure app.example.com in DNS as a CNAME (or Alias, if you're using Route 53) pointing to the dxxxexample.cloudfront.net hostname assigned to your distribution.
You can also access your sitr using the assigned CloudFront hostname, directly, for testing. Accessing this endpoint from the blocked IP address should indeed result in the request being denied, now.
So, the CloudFront endpoint is where you need to send your traffic -- not directly to the ELB.
Doesn't that leave your ELB still exposed?
Yes, it does... so the next step is to plug that hole.
If you're using a custom origin, you can use custom headers to prevent users from bypassing CloudFront and requesting content directly from your origin.
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/forward-custom-headers.html
The idea here is that you will establish a secret value known only to your servers and CloudFront. CloudFront will send this in the headers along with every request, and your servers will require that value to be present or else they will play dumb and throw an error -- such as 503 Service Unavailable or 403 Forbidden or even 404 Not Found.
So, you make up a header name, like X-My-CloudFront-Secret-String and a random string, like o+mJeNieamgKKS0Uu0A1Fqk7sOqa6Mlc3 and configure this as a Custom Origin Header in CloudFront. The values shown here are arbitrary examples -- this can be anything.
Then configure your application web server to deny any request where this header and the matching value are not present -- because this is how you know the request came from your specific CloudFront distribution. Anything else (other than ELB health checks, for which you need to make an exception) is not from your CloudFront distribution, and is therefore unauthorized by definition, so your server needs to deny it with an error, but without explaining too much in the error message.
This header and its expected value remains a secret because it will not be sent back to the browser by CloudFront -- it's only sent in the forward direction, in the requests that CloudFront sends to your ELB.
Note that you should get an SSL cert for your ELB (for the elb.example.com hostname) and configure CloudFront to forward all requests to your ELB using HTTPS. The likelihood of interception of traffic between CloudFront and ELB is low, but this is a protection you should consider implenting.
You can optionally also reduce (but not eliminate) most unauthorized access by blocking all requests that don't arrive from CloudFront by only allowing the CloudFront IP address ranges in the ELB security group -- the CloudFront address ranges are documented (search the JSON for blocks designated as CLOUDFRONT, and allow only these in the ELB security group) but note that if you do this, you still need to set up the custom origin header configuration, discussed above, because if you only block at the IP level, you're still technically allowing anybody's CloudFront distribution to access your ELB. Your CloudFront distribution shares IP addresses in a pool with other CloudFront distribution, so the fact that the request arrives from CloudFront is not a sufficient guarantee that it is from your CloudFront distribution. Note also that you need to sign up for change notifications so that if new address ranges are added to CloudFront, then you'll know to add them to your security group.