I configure Custom Access Logging for Amazon API Gateway and I need to specify CloudWatch Group name, but when I put these just name of log group in format like "API-Gateway-Execution-Logs_3j5w5m7kv9/stage-name" I get such error:
Invalid ARN specified in the request. ARNs must start with 'arn:':
API-Gateway-Execution-Logs_3j5w5m7kv9/stage-name
When I open page of this log group in CloudWatch I just see the same name there and don't see ARN value. How can I find it?
Go to Cloudwatch logs, find your log group, open it and you'll see a list of log streams. There is settings icon on top right:
Click it and you'll see an option to show stream arn:
Save the settings and you'll see stream arns. The part before semicolon looks like Log Group arn
The CloudWatch Group ARN format is arn:aws:logs:{region}:{account-id}:log-group:API-Gateway-Execution-Logs_{rest-api-id}/{stage-name}, cf. https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-logging.html
arn:aws:logs:region:account-id:log-group:log_group_name
See this documentation
You can also use AWS CLI
aws logs describe-log-groups | grep <log_group_name> | awk '/arn/'
2022 Update
Select your log group:
Click Log Group Details:
Copy the ARN:
Or you could do a aws logs describe-log-groups | grep <name_of_group>
That works too.
Related
Following this guide and creating a Kinesis Firehose Stream.
I have followed the guide and when I get to creating a subscription filter (step 12), I encounter this error when trying to send to S3:
An error occurred (InvalidParameterException) when calling the PutSubscriptionFilter operation: Could not deliver test message to specified Firehose stream. Check if the given Firehose stream is in ACTIVE state.
I can confirm that the stream is active and I can send test data via the console and it arrives in S3 as expected.
This is the command I am running (changed my account id):
aws logs put-subscription-filter --log-group-name "myLogGroup" --filter-name "Destination" --filter-pattern "{$.userIdentity.type = Root}" --destination-arn "arn:aws:firehose:ap-southeast-1:1234567890:deliverystream/my-delivery-stream" --role-arn "arn:aws:iam::1234567890:role/CWLtoKinesisFirehoseRole"
I have checked the trusted entities and the role has priviliges to logs and firehose. Any ideas?
I also struggle with this for a long time, for me it was those 2 gothca's:
step 4 in the guide:
make sure to change to bucket name to you bucket:
step 8 !!!:
make sure to put your account ID it is not highlighted:
I am sure you already know how to configure logs subscription filter so not adding steps in my answer
Go to firehose and check logs is your firehose has access to execute lambda ,if not please add required role.
Now start dummy data stream using firehose test and see is your data is moving till lambda or S3.
check cloud trail and cloud watch logs and see if found any error .
Open your IAM role and check all required role added to your role , now click trust relationship add- "logs group ","IAM" and component name in my case its "Ec2"
Hope this will helpful to resolve your issue.
https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html#FirehoseExample
I am not able to delete these subscriptions attached to the CloudWatch Logs Groups.
These subscriptions are created by CloudFormation stack via Serverless Framework. However, when I finished testing and deployed to the template, there was a permission error during the cleanup. Hence, these subscriptions became dangled and I am not able to locate it.
Tried with CLI and seems no relevant info regarding that.
$ aws logs describe-log-groups --log-group-name-prefix yyy
{
"logGroups": [
{
"logGroupName": "yyy",
"creationTime": 1555604143719,
"retentionInDays": 1,
"metricFilterCount": 0,
"arn": "arn:aws:logs:us-east-1:xxx:log-group:yyy:*",
"storedBytes": 167385869
}
]
}
Select the Log Group using the radio button on the left of the Log Group name. Then click Actions, Remove Subscription Filter.
Via CLI is listed in AWS document => This link
Via Console UI -> This capture
As you created the subscription with cloudformation stack via serverless, manually removing the subscription filter as jarmod is not a best practice.
What you should do is remove the cloudwatchLog event from the lambda functions and deploy, it should remove the subscriptions.
due to huge costs in our environment, I have a task to create a lambda to tag all log groups like corresponding resources (the source of these log groups). However, I am facing a challenge to identify the resource arn of log groups. There are many logs in our environment like logs for lambda, logs for elastic-beanstalk, logs for ec2. But how can I match the log group with the corresponding resource? I would appreciate any help very much!
I would try using describe-log-groups. To use it you'll need to work your way "backwards" by going from resources to the log-groups, but I can't think of any other way at the moment.
aws logs describe-log-groups --query 'logGroups[*].arn' --log-group-name-prefix '/aws/lambda/[name-of-your-lambda]'
Output:
[
"arn:aws:logs:[region]:[account-id]:log-group:/aws/lambda/[name-of-your-lambda]:*"
]
Hope that helps.
AWS CodeBuild has an option in the UI to upload build output logs to CloudWatch and/or publish the logs to S3 but I do not see an option in Terraform to enable this feature through the "aws_codebuild_project" resource.
Here is a link to the Terraform docs:
https://www.terraform.io/docs/providers/aws/r/codebuild_project.html
Is there another way to possibly enable this feature via Terraform?
I can create a CloudWatch Event to capture CodeBuild events but that doesn't give me the full logs.
So even thought the Terraform documentation does not specify support for enabling CloudWatch logs for CodeBuild jobs, if you create a CodeBuild job with the AWS CodeBuild Project resource it will by default enable that checkbox for you, at least as of the date of this answer.
The Cloudwatch log Log Group will be created with the following format: aws/codebuild/codebuild_project_name
There is an open PR for a Github issue to enable CodeBuild CloudWatch logs, so hopefully we will be able to use the following config soon:
logs_config {
cloudwatch_logs {
status = "ENABLED|DISABLED"
group_name = "..."
stream_name = "..."
}
}
I'm creating a logs aggregator lambda to send Cloudwatch logs to a private log analysis service. Given the number of resources used by my employer, it was decided to create a subscription lambda that handles log group subscription to the aggregator.
The solution works fine, but it requires to manually search a resource's log group via amazon console and then invoke the subscription lambda with it.
My question:
Is there a way to, given a resource arn, find which log group is mapped to it? Since I'm using Cloudformation to create resources it is easy to export a resource's arn.
UPDATE
To present an example:
Let's say I have the following arn:
arn:aws:appsync:<REGION>:<ACCOUNTID>apis/z3pihpr4gfbzhflthkyjjh6yvu
which is an Appsync GraphQL API.
What I want it a method (using te API or some automated solution) to get the Cloudwatch log group of that resource.
You can try the describe-log-groups command. It is available on the cli, must also be there on the API.
To get the names of the log groups you can go with:
aws logs describe-log-groups --query 'logGroups[*].logGroupName' --log-group-name-prefix '/aws/appsync/[name-of-the-resource]'
Output will look like this:
[
"/aws/appsync/[name-of-your-resource]"
]