AWS ECS Environment variables aren't set - amazon-web-services

Question

For some reason the environment variables, although I've configured them in my ECS task, are not set in the running container. What am I missing? Why are the values empty?
I have the following AWS::ECS::TaskDefinition:
AirflowWebTaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: !Join ['', [!Ref 'AWS::StackName', -dl-airflow-web]]
ContainerDefinitions:
- Name: dl-airflow-web
Cpu: '10'
Essential: 'true'
Image: companyname-docker-snapshot-local.jfrog.io/docker-airflow:1.0
Command: ['webserver']
Memory: '1024'
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref 'AirflowCloudwatchLogsGroup'
awslogs-region: !Ref 'AWS::Region'
awslogs-stream-prefix: dl-airflow-web
PortMappings:
-
ContainerPort: 8080
Environment:
- Name: LOAD_EX
Value: n
- Name: EXECUTOR
Value: Celery
- Name: MYQL_HOST
Value: !Ref 'RDSDNSName'
- Name: MYSQL_PORT
Value: !Ref 'RDSPort'
- Name: MYSQL_DB
Value: !Ref 'AirflowDBName'
- Name: USERNAME
Value: !Ref 'AirflowDBUser'
- Name: PASSWORD
Value: !Ref 'AirflowDBPassword'
And I am using a docker image which is a fork of https://github.com/puckel/docker-airflow. The entrypoint for the image inspects environment variables as follows:
#!/usr/bin/env bash
AIRFLOW_HOME="/usr/local/airflow"
CMD="airflow"
TRY_LOOP="20"
: ${MYSQL_HOST:="default-mysql"}
: ${MYSQL_PORT:="3306"}
Where the $MYSQL_* variables are set to a default if they have not been set in the docker run command.
When I run the container image from docker-compose using the configuration below, it works and the environment variables are all set:
webserver:
image: companyname-docker-snapshot-local.jfrog.io/docker-airflow:1.0
environment:
- LOAD_EX=n
- EXECUTOR=Celery
- MYSQL_HOST=mysql
- MYSQL_PORT=3306
- USERNAME=dev-user
- PASSWORD=dev-secret-pw
- SQS_HOST=sqs
- SQS_PORT=9324
- AWS_DYNAMODB_ENDPOINT=http://dynamodb:8000
ports:
- "8090:8080"
command: webserver
And the following command in my entrypoint.sh:
echo "$(date) - Checking for MYSQL (host: $MYSQL_HOST, port: $MYSQL_PORT) connectivity"
Logs this output:
Fri Jun 2 12:55:26 UTC 2017 - Checking for MYSQL (host: mysql, port: 3306) connectivity
But inspecting my cloudwatch logs shows this output with the default values:
Fri Jun 2 14:15:03 UTC 2017 - Checking for MYSQL (host: default-mysql, port: 3306) connectivity
But I can ssh into the EC2 host, run docker inspect [container_id] and verify that the environment variables are set:
Config": {
"Hostname": "...",
"Domainname": "",
"User": "airflow",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"5555/tcp": {},
"8080/tcp": {},
"8793/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"MYSQL_PORT=3306",
"PASSWORD=rds-secret-pw",
"USERNAME=rds-user",
"EXECUTOR=Celery",
"LOAD_EX=n",
"MYQL_HOST=rds-cluster-name.cluster-id.aws-region.rds.amazonaws.com",
"MYSQL_DB=db-name",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"DEBIAN_FRONTEND=noninteractive",
"TERM=linux",
"LANGUAGE=en_US.UTF-8",
"LANG=en_US.UTF-8",
"LC_ALL=en_US.UTF-8",
"LC_CTYPE=en_US.UTF-8",
"LC_MESSAGES=en_US.UTF-8"
],
"Cmd": [
"webserver"
],
"Image": "companyname-docker-snapshot-local.jfrog.io/docker-airflow:1.0",
"Volumes": null,
"WorkingDir": "/usr/local/airflow",
"Entrypoint": [
"/entrypoint.sh"
],
"OnBuild": null,
"Labels": {
"com.amazonaws.ecs.cluster": "...",
"com.amazonaws.ecs.container-name": "...",
"com.amazonaws.ecs.task-arn": "...",
"com.amazonaws.ecs.task-definition-family": "...",
"com.amazonaws.ecs.task-definition-version": "16"
}
},
And if I run:
$ docker exec [container-id] echo $MYSQL_HOST
The output is empty

Answer 1

your task definition defines env variable MYQL_HOST. You got that right in the docker compose. Just the CF, fix it and it should be fine.