How to specify sensitive environment variables at deploy time with Elastic Beanstalk - amazon-web-services

Question

I am deploying a Python Flask application with Elastic Beanstalk. I have a config file /.ebextensions/01.config where among other things I set some environment variables - some of which should be secret.
The file looks something like this:
packages:
yum:
gcc: []
git: []
postgresql93-devel: []
option_settings:
"aws:elasticbeanstalk:application:environment":
SECRET_KEY: "sensitive"
MAIL_USERNAME: "sensitive"
MAIL_PASSWORD: "sensitive"
SQLALCHEMY_DATABASE_URI: "sensitive"
"aws:elasticbeanstalk:container:python:staticfiles":
"/static/": "app/static/"
What are the best practices for keeping certain values secret? Currently the .ebextensions folder is under source control and I like this because it is shared with everyone, but at the same time I do not want to keep sensitive values under source control.
Is there a way to specify some environment variables through the EB CLI tool when deploying (e.g. eb deploy -config ...)? Or how is this use case covered by the AWS deployment tools?

Answer 1

The AWS documentation recommends storing sensitive information in S3 because environment variables may be exposed in various ways:
Providing connection information to your application with environment
properties is a good way to keep passwords out of your code, but it's
not a perfect solution. Environment properties are discoverable in the
Environment Management Console, and can be viewed by any user that has
permission to describe configuration settings on your environment.
Depending on the platform, environment properties may also appear in
instance logs.
The example below is from the documentation, to which you should refer for full details. In short, you need to:
Upload the file to S3 with minimal permissions, possibly encrypted.
Grant read access to the role of the instance profile for your Elastic Beanstalk autoscaling group. The policy would be like:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "database",
"Action": [
"s3:GetObject"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::my-secret-bucket-123456789012/beanstalk-database.json"
]
}
]
}
Add a file with a name like s3-connection-info-file.config to /.ebextensions in your application bundle root with these contents:
Resources:
AWSEBAutoScalingGroup:
Metadata:
AWS::CloudFormation::Authentication:
S3Auth:
type: "s3"
buckets: ["my-secret-bucket-123456789012"]
roleName: "aws-elasticbeanstalk-ec2-role"
files:
"/tmp/beanstalk-database.json" :
mode: "000644"
owner: root
group: root
authentication: "S3Auth"
source: https://s3-us-west-2.amazonaws.com/my-secret-bucket-123456789012/beanstalk-database.json
Then update your application code to extract the values from the file /tmp/beanstalk-database.json (or wherever you decide to put it in your actual config.)

Answer 2

This question already has an answer, but I want to contribute an alternative solution to this problem. Instead of having to keep secrets in environment variables (which then have to be managed and stored somewhere out of version control, plus you need to remember to set them at deployment), I put all my secrets in an encrypted S3 bucket only accessible from the role the EB is running as. I then fetch the secrets at startup. This has the benefit of completely decoupling deployment from configuration, and you never ever have to fiddle with secrets in the command line again.
If needed (for example if secrets are needed during app setup, such as keys to repositories where code is fetched) you can also use an .ebextensions config file with an S3Auth directive to easily copy the contents of said S3 bucket to your local instance; otherwise just use the AWS SDK to fetch all secrets from the app at startup.
EDIT: As of April 2018 AWS offers a dedicated managed service for secrets management; the AWS Secrets Manager. It offers convenient secure storage of secrets in string or json format, versioning, stages, rotation and more. It also eliminates some of the configuration when it comes to KMS, IAM etc for a quicker setup. I see no real reason using any other AWS service for storing static sensitive data such as private keys, passwords etc.

Answer 3

You should be able to specify sensitive values as environment variables from eb web console: Your EB app -> Your EB environment -> Configuration -> Software Configuration -> Environment Properties
Alternatively, you can make use of this: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb3-setenv.html
EDIT: While this was accepted answer in 2015, this should not be how you handle it. Now you can use AWS Secrets Manager for this purpose.

Answer 4

I have been using another shell script, something ./deploy_production.sh to set environment specific variables. In the shell script, you can use "eb setenv NAME1=VAR1 NAME2=VAR2 ..." to set env var.
And this file doesn't need to go into git repo.

Answer 5

Some of the other answers are mentioning that there might be a better way with Parameter Store / Secrets Manager.
I described how I did this with AWS Systems Manager Parameter Store (which also gives you an interface to Secrets Manager) in this answer: https://stackoverflow.com/a/59910941/159178. Basically, you give your Beanstalk ECS IAM role access to the relevant parameter and then load it from your application code at startup.