How to use fluentd log driver on Elastic Beanstalk Multicontainer docker - amazon-web-services

Question

I tried to use fluentd log driver with the following Dockerrun.aws.json,
{
"AWSEBDockerrunVersion": 2,
"containerDefinitions": [
{
"name": "apache",
"image": "php:5.6-apache",
"essential": true,
"memory": 128,
"portMappings": [
{
"hostPort": 80,
"containerPort": 80
}
],
"logConfiguration": {
"logDriver": "fluentd",
"options": {
"fluentd-address": "127.0.0.1:24224"
}
}
}
]
}
but the following error occurred.
ERROR: Encountered error starting new ECS task: {cancel the command.
"failures": [
{
"reason": "ATTRIBUTE",
"arn": "arn:aws:ecs:ap-northeast-1:000000000000:container-instance/00000000-0000-0000-0000-000000000000"
}
],
"tasks": []
}
ERROR: Failed to start ECS task after retrying 2 times.
ERROR: [Instance: i-00000000] Command failed on instance. Return code: 1 Output: beanstalk/hooks/appdeploy/enact/03start-task.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
What sould do I configure?

Answer 1

Seems that you can also accomplish it with .ebextensions/01-fluentd.config file in your application environment directory with the following content:
files:
"/home/ec2-user/setup-available-log-dirvers.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/sh
set -e
if ! grep fluentd /etc/ecs/ecs.config &> /dev/null
then
echo 'ECS_AVAILABLE_LOGGING_DRIVERS=["json-file","syslog","fluentd"]' >> /etc/ecs/ecs.config
fi
container_commands:
01-configure-fluentd:
command: /home/ec2-user/setup-available-log-dirvers.sh
Now you have to deploy a new application version (without fluentd configuration yet), rebuild your environment, add fluentd configuration:
logConfiguration:
logDriver: fluentd
options:
fluentd-address: localhost:24224
fluentd-tag: docker.myapp
and now deploy updated app, everything should work now.

Answer 2

I have resolved the problem myself.
First, I prepare a custom ami having the following user data.
#cloud-config
repo_releasever: 2015.09
repo_upgrade: none
runcmd:
- echo 'ECS_AVAILABLE_LOGGING_DRIVERS=["json-file","syslog","fluentd"]' >> /etc/ecs/ecs.config
Second, I define the ami id which is created custom ami in my environment EC2 settings. Finally, I deploy my application to Elastic Beanstalk. After this, fluentd log driver in my environment works normally.
In order to use fluentd log driver in Elastic Beanstalk Multicontainer Docker, it requires to define ECS_AVAILABLE_LOGGING_DRIVERS variable in /etc/ecs/ecs.config. Elastic Beanstalk Multicontainer Docker is using ECS inside, thus related settings is in the ECS documentation.
Please read logConfiguration section in the following documentation:
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html

Answer 3

I have added a comment already to the accepted answer, just adding the complete ebextension file that I used to make it work for me
files:
"/home/ec2-user/setup-available-log-dirvers.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/sh
set -e
if ! grep fluentd /etc/ecs/ecs.config &> /dev/null
then
echo 'ECS_AVAILABLE_LOGGING_DRIVERS=["json-file","syslog","fluentd"]' >> /etc/ecs/ecs.config
fi
container_commands:
00-configure-fluentd:
command: /home/ec2-user/setup-available-log-dirvers.sh
01-stop-ecs:
command: stop ecs
02-stop-ecs:
command: start ecs
We are just restating ecs after setting logging drivers