We have recently started using AWS ELB. We are struggling to capture the following headers at ELB.
Referer
User-Agent
Any way to get it done?
AFAIK you cannot configure the access logs format for the LB (classic or other wise).
For classic ELB look at the logging documentation to see what fields are logged.
It does log the user agent but the referer is not logged.
Depending on what kind of application architecture you have behind your ELB you can configure that application for access logs and you would have more control over the access logs.
Example: Say you have an Spring boot based EC2 behind your ELB, you can configure the spring boot embedded tomcat to log whatever headers you need in the access logs.
Related
I distributed spring boot to aws ec2 and vue.js to s3 cloudfront.
Request for ec2 in my locality is performed normally performed.
However, requesting ec2 on cloudfront results in 403 access denied on the web without a server response.
It is being requested through vue.js proxy, and all settings such as security policies have been completed.
What is the problem?
There could be multiple issues with it. Follow the steps below to debug it
Ensure that your APIs are working without CloudFront. Try to access the APIs hosted on EC2 instance. (you have to open them up for public before using them from CloudFront so ensure that ports are open, etc.)
Check the cache settings. For APIs path (behaviour) it should be no-cache
Make sure the behaviours are setup correctly in the CloudFormation. Behaviours configuration set the path / routes of incoming requests and map them to origins
Enable logging for CloudFront and analyse that
CloudFront doesn't translate the error messages exactly for security reasons. If an object or path doesn't exists CF will show 403 rather than 404. This is to prevent exploits to identify which HTTP resource exists and which not.
Update your question with the findings which will help others to share the solutions.
1.) BotControl
2.) AmazonIpReputationList
3.) AnonymousIp
4.) CommonRuleSet
5.) KnownBadInputs
6.) LinuxRuleSet
7.) SQL injection
these are all rule sets I have set in place but is there a way for me to tests against them to make sure they are working as intended?
basically you would like to test your rules, here is my 2 cents
Counting the web requests that match the rules in a web ACL
Configure all the rules in a web ACL to count web requests
Set the default action for the web ACL to allow requests
Viewing a sample of the web requests that API Gateway CloudFront or an Application Load Balancer has forwarded to AWS WAF Classic
In the AWS WAF Classic console, you can view a sample of the requests
that API
Gateway CloudFront or an Application Load Balancer has forwarded to AWS WAF
Classic for inspection. For each sampled request, you can view detailed data
about the request, such as the originating IP address and the headers included
in the request. You also can view which rule the request matched, and whether
the rule is configured to allow or block requests.
Logging and monitoring web ACL traffic
You can enable logging to get detailed information about traffic that
is analyzed by your web ACL. Logged information includes the time that
AWS WAF received a web request from your AWS resource, detailed
information about the request, and details about the rules that the
request matched. You can send your logs to an Amazon CloudWatch Logs
log group, an Amazon Simple Storage Service (Amazon S3) bucket, or an
Amazon Kinesis Data Firehose.
please refer this doc for more information.
I am trying to set Listener rules on an ALB. I want to add Google Oauth support to one of my servers.
Here are the Google endpoints I am using
I see google auth page alright, but on the callback url I'm seeing 500 Internal Server Error. I've also set the callback URL. Am at a loss as to what's wrong here. Any help is most appreciated!
After authentication, I'm not redirecting to my application, instead I've set ALP to show a text based simple response.
I struggled with the same problem for hours, and in the end it turned out to be the user info endpoint that was wrong. I was using the same one as you, but it should be https://openidconnect.googleapis.com/v1/userinfo.
I haven’t found any Google documentation saying what the value should be, but found this excellent blog post that contained a working example: https://cloudonaut.io/how-to-secure-your-devops-tools-with-alb-authentication/ (the first example uses Cognito, but the second uses OIDC and Google directly).
From AWS documentation
HTTP 500: Internal Server Error
Possible causes:
You configured an AWS WAF web access control list (web ACL) and there was an error executing the web ACL rules.
You configured a listener rule to authenticate users, but one of the following is true:
The load balancer is unable to communicate with the IdP token endpoint or the IdP user info endpoint. Verify that the security groups for your load balancer and the network ACLs for your VPC allow outbound access to these endpoints. Verify that your VPC has internet access. If you have an internal-facing load balancer, use a NAT gateway to enable internet access.
The size of the claims returned by the IdP exceeded the maximum size supported by the load balancer.
A client submitted an HTTP/1.0 request without a host header, and the load balancer was unable to generate a redirect URL.
A client submitted a request without an HTTP protocol, and the load balancer was unable to generate a redirect URL.
The requested scope doesn't return an ID token.
Amazon introduced support for Request Tracing in 2016 for all of its Application Load Balancers (ALBs).
How are Access Logging, Request Tracing and CloudTrail logging, different from each other?
When to use what?
How is one better than the other and under what circumstances?
CloudTrail logs API calls within your account. Load balancer log files log HTTP client requests to your load balancer (e.g. web traffic). Very different types of information.
Load Balancer access logs record information about who, where and what HTTP requests are being made from clients. This information is useful for traffic analysis.
Request Tracing adds a custom header X-Amzn-Trace-Id into the request to your origin (typically a web server). This is used to trace (track) requests thru your origin servers.
What is my indication that I am using AWS Certificate Manager correctly and that any remaining problems getting my site to load at https are due to a mistake I am making in my Apache configuration?
In AWS Certificate Manager, I see "Success! Your certificate was issued successfully." Does that mean there are no further steps for me to complete in the AWS console, and I need only get my Apache configuration correct to finish?
Currently, when I try to visit a URL at my site with the http protocol, it loads fine, but when I visit at https, the browser tries to load the page but it never loads.
I have followed the instructions for creating an HTTPS listener, but still do not know if I am done with all necessary steps in AWS console. How would I know?
Edit: To clarify, I am using an Elastic Load Balancer (ELB), since the documentation indicated I need to use ELB with AWS Certificate Manager (ACM). However, I do not know how to determine if I have configured everything correctly in AWS console that I need to in order to access the site at HTTPS.
Edit 2: This might come close to answering my question, possibly, but I don't know how to do this: "You can use curl, telnet etc from your local machine to verify 443 port status on ELB" -- #vivekyad4v.
ACM(AWS Certificate Manager) supports the AWS resources like ELB, Cloudfront, API Gateway etc. You can add SSL certificates to these
resources via AWS console.
Currently, it doesn't support EC2. You cannot use ACM with EC2 instances, you will need a Load Balancer in front of it. Once you have a load balancer, SSL termination happens on the load balancer & not on the EC2 instance.
Once it is setup, you can change your apache server config to redirect all HTTP requests to HTTPS.
Add certificate to ELB - "https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-update-ssl-cert.html"
Update apache config - "https://aws.amazon.com/premiumsupport/knowledge-center/redirect-http-https-elb/"
No EC2 support - "https://aws.amazon.com/certificate-manager/faqs/"