How to stream tomcat catalinat.out logs to cloud watch? - amazon-web-services

Question

I want to stream tomcat catalinat.out logs to cloud watch:
This is my config I follow:
https://github.com/awsdocs/elastic-beanstalk-samples/blob/master/configuration-files/aws-provided/instance-configuration/logs-streamtocloudwatch-linux.config
But I don't see catalina.out in cloudwatch console :
This is error , I have in awslogs.log
How can I solve it.
2020-05-22 18:15:55,450 - cwlogs.push.batch - WARNING - 3374 - Thread-29 - CreateLogGroup failed with exception An error occurred (AccessDeniedException) when calling the CreateLogGroup operation: User: arn:aws:sts::610232524349:assumed-role/aws-elasticbeanstalk-ec2-role/i-099300c0bfd4b6a28 is not authorized to perform: logs:CreateLogGroup on resource: arn:aws:logs:eu-central-1:610232524349:log-group:/aws/elasticbeanstalk/************/var/log/tomcat8/catalina.out:log-stream:

Answer 1

With the sample provided you are not exporting the catalinat.out you are streaming to cloudwatch the following files:
/var/log/dmesg
/var/log/messages
To stream the catalitat.out you have to add the file to the configuration with the location of the log at the end of content section (Lines 61-71 on the sample provided)
It should be something like this replacing /path/to/catalitat.log with the actual path to the log:
[/path/to/catalitat.log]
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/path/to/catalitat.log"]]}`
log_stream_name = {instance_id}
file = /path/to/catalitat.log

Answer 2

Steps to publish tomcat logs (catalina.out) to the CloudWatch stream
Create a new policy for EC2 to use AWS CloudWatch, providing access to create log groups, log streams and publish logs
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "logs:*",
"Resource": "*"
}
]
}
Attach the policy to newly created or existing IAM role of the EC2 instance
Connect to the instance using SSH, using the PEM/PPK file.
Locate the AWS CloudWatch Logs Agent configuration file
[ec2-user# elasticbeanstalk]$ sudo su
[root# elasticbeanstalk]# find / -name "*awslogs.conf"
/etc/awslogs/awslogs.conf
Edit the configuration file and add the entry for a log stream for tomcat logs. I have used catalina.out
[ec2-user# elasticbeanstalk]$ cat /etc/awslogs/awslogs.conf
[general]
state_file = /var/lib/awslogs/agent-state
[tomcatLogs]
log_group_name = tomcatLogs
log_stream_name = catalinaLogs
time_zone = LOCAL
file = /[your-path-to]/tomcat8/catalina.out
[ec2-user# elasticbeanstalk]$
Restart the service AWS Logs
[ec2-user# elasticbeanstalk]$ sudo service awslogs restart
Revisit the CloudWatch log groups page, where you can see the new group is created with the name “tomcatLogs” and a log stream with the name “catalinaLogs”

Answer 3

I feel your pain! I've detailed in a new Medium blog how this all works and an example .ebextensions file and where to put it.
Below is an excerpt that you might be able to use, though the article explains how to determine the right folder/file(s) to stream.
packages:
yum:
awslogs: []
option_settings:
- namespace: aws:elasticbeanstalk:cloudwatch:logs
option_name: StreamLogs
value: true
- namespace: aws:elasticbeanstalk:cloudwatch:logs
option_name: DeleteOnTerminate
value: false
- namespace: aws:elasticbeanstalk:cloudwatch:logs
option_name: RetentionInDays
value: 90
files:
"/etc/awslogs/awscli.conf" :
mode: "000600"
owner: root
group: root
content: |
[plugins]
cwlogs = cwlogs
[default]
region = `{"Ref":"AWS::Region"}`
"/etc/awslogs/config/logs.conf" :
mode: "000600"
owner: root
group: root
content: |
[/var/log/tomcat/localhost.log]
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/tomcat/localhost.log"]]}`
log_stream_name = {instance_id}
file = /var/log/tomcat/localhost.*
[/var/log/tomcat/catalina.log]
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/tomcat/catalina.log"]]}`
log_stream_name = {instance_id}
file = /var/log/tomcat/catalina.*
[/var/log/tomcat/localhost_access_log.txt]
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/tomcat/access_log"]]}`
log_stream_name = {instance_id}
file = /var/log/tomcat/access_log.*
commands:
"01":
command: systemctl enable awslogsd.service
"02":
command: systemctl restart awslogsd