We're working with AWS and wish to log all incoming http requests from our API Gateway to the EC2 box in their entirety (i.e. not truncated at 1024 bytes). As we know the logging for API Gateway is not configurable our next thought was to place a lambda between the api gateway and EC2 which would intercept the requests, log them to S3, and then forward the request to EC2.
My question is two fold:
How does one submit an API Gateway 'event' to an EC2 instance manually? I've gone over the boto3 documentation and there doesn't seem to be a good way to do this out of the box.
Is this a logical way to log requests in their entirety, or are we missing a built in connector between lambda -> ec2 or request logging in general?
def lambda_handler(event, context):
#pass event to the ec2
#log event (done)
#return response
You can enable CloudWatch logging for API gateway. By enabling you can find the logs for your requests in the cloud watch.
Create an IAM role that gives permissions for the API gateway to write logs to cloud watch
set the IAM role to your API
set the desired log levels INFO/ERROR
you can also choose to log the full request/responses
To enable access logs, you also need to create a log group and provide the arn.
The detailed steps can be found in the reference below.
Hope this helps.
Reference:
https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-cloudwatch-logs/
Related
I have an docker image with an API that publishes provided request body to a SNS topic. Now I am able to perform this operation locally, but when I deploy this image to ECS through ECR, I get following error form AWS Go SDK(v2) after ~1 min of request initiation.
https response error StatusCode: 0, RequestID: , canceled, context canceled
I have verified following things:
ECS service (as well as EC2 as I am using EC2 launch type) has a role that grants full access to SNS
SNS access policy allows all requests from current AWS account.
I am able to publish to same topic after logging into EC2 instance.
API request has 5 min timeout.
I am using default VPC.
For the sake of experiment, I am not using HTTPS and attached security group allows all traffic on port 80. I tried everything but didn't find what's wrong. I would appreciate your great help, thanks.
I have created an API Gateway in AWS with two resources (endpoints). Let's say /foo and /bar. Each endpoint has a POST method.
I want to monitor how many times each endpoint got invoked - /foo and /bar in our example. And the metrics to show the 2xx, 4xx, and 5xx responses.
I know API Gateway got a generic "API Calls" metric that shows the total invocations of the API. But how do I monitor how many times each endpoint got called?
You can filter API Gateway metrics for the API method with the specified API name, stage, resource, and method.
API Gateway will not send these metrics unless you have explicitly enabled detailed CloudWatch metrics. You can do this in the console by selecting Enable Detailed CloudWatch Metrics under a stage Logs/Tracing tab. Alternatively, you can call the update-stage AWS CLI command to update the metricsEnabled property to true.
Enable Detailed CloudWatch Metrics on AWS Console:
Documentation:
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-metrics-and-dimensions.html#api-gateway-metricdimensions
Currently, my API gateway execution logs are getting logged in cloud-watch in some random path of a particular format. Is there a way for me to customize that log path in SAM Template while deploying the API gateway ??
No, as of April 2021, execution logs are fully managed by AWS API Gateway and the log group name will be in form of API-Gateway-Execution-Logs_{rest-api-id}/{stage_name} and the format is pre-defined.
On a different note: You can define the CloudWatch target ARN and format of access logs though if you have this enabled but this is a different set of logs.
If you are using SAM, you can explicitly define your API using the AWS::Serverless::Api resource type. This type as a property called AccessLogSetting that allows you to provide the ARN of a CloudWatch group to send the the access logs to.
Also check out this guide which doesn't go into SAM automation but explains API Gateway logging configurations and what is possible, a good read.
https://www.alexdebrie.com/posts/api-gateway-access-logs/
We have implemented Dynatrace and we need to add our AWS account to pull CloudWatch logs.
The problem: We have a corporate proxy and firewall which is super locked down and seems to block whichever requests Dynatrace is trying to make to AWS to authenticate with the key and secret.
The infra guys has allowed a bunch of AWS domains per region to read CloudWatch logs, but we still can't authenticate (see image below).
I have set up a Dynatrace security gateway in AWS which our local gateways are able to access. We are successfully getting logs form OneAgents through the SGW. The problem is getting the CloudWatch logs/integration going.
The error in the image below leads me to believe that Dynatrace is not able to communicate to the AWS auth servers at all.
Any advice would be appreciated.
I'm just a dev
We are not allowed to open the amazon domain.
#corporate-devlife
I need a way to log API gateway deployments (date/time, user, swagger diff etc.). Is there an event thats fired that i can attach a lambda to, or alternatively is this information already available on the dashboard somewhere?
As Krishna mentioned, CloudTrail can capture API events (both from the AWS console as well as the AWS APIs) for API Gateway, including the deployment of APIs. Since CloudTrail stores the events in S3, you can take advantage of S3 bucket notifications as a means to trigger your Lambda function.