AWS CLI create RDS with elasticbeanstalk create-environment - amazon-web-services

Question

How can I create an RDS instance with the create-environment or another subcommand of aws elasticbeanstalk? I've tried several combinations of parameters to no avail. Below is an example.
APP_NAME="randall-railsapp"
aws s3api create-bucket --bucket "$APP_NAME"
APP_VERSION="$(git describe --always)"
APP_FILE="deploy-$APP_NAME-$APP_VERSION.zip"
git archive -o "$APP_FILE" HEAD
aws s3 cp "$APP_FILE" "s3://$APP_NAME/$APP_FILE"
aws --region us-east-1 elasticbeanstalk create-application-version \
--auto-create-application \
--application-name "$APP_NAME" \
--version-label "$APP_VERSION" \
--source-bundle S3Bucket="$APP_NAME",S3Key="$APP_FILE"
aws --region us-east-1 elasticbeanstalk create-environment \
--application-name "$APP_NAME" \
--version-label "$APP_VERSION" \
--environment-name "$APP_NAME-env" \
--description "randall's rails app environment" \
--solution-stack-name "64bit Amazon Linux 2014.03 v1.0.0 running Ruby 2.1 (Puma)" \
--cname-prefix "$APP_NAME-test" \
--option-settings file://test.json
And the contents of test.json:
[
{
"OptionName": "EC2KeyName",
"Namespace": "aws:autoscaling:launchconfiguration",
"Value": "a-key-is-here"
},
{
"OptionName": "EnvironmentType",
"Namespace": "aws:elasticbeanstalk:environment",
"Value": "SingleInstance"
},
{
"OptionName": "SECRET_KEY_BASE",
"Namespace": "aws:elasticbeanstalk:application:environment",
"Value": "HAHAHAHAHAHA"
},
{
"OptionName": "DBPassword",
"Namespace": "aws:rds:dbinstance",
"Value": "hunter2"
},
{
"OptionName": "DBUser",
"Namespace": "aws:rds:dbinstance",
"Value": "random"
},
{
"OptionName": "DBEngineVersion",
"Namespace": "aws:rds:dbinstance",
"Value": "9.3"
},
{
"OptionName": "DBEngine",
"Namespace": "aws:rds:dbinstance",
"Value": "postgres"
}
]
Anyone know why this is failing? Anything I specify with a aws:rds:dbinstance namespace seems to get removed from the configuration.

Answer 1

Just setting the aws:rds:dbinstance options does not create an RDS database.
Currently you can create an RDS instance using one of the following techniques:
Create using AWS Console
Use eb cli
Use Resources section of ebextensions to create an RDS resource
The first two approaches are the most convenient as they do all the heavy lifting for you but for the third one you have to do some extra work. The third approach is what you would want to use if you are not using the console or eb CLI.
You can create an RDS resource for your beanstalk environment using the following ebextension snippet. Create a file called 01-rds.config in the .ebextensions directory of your app source.
Resources:
AWSEBRDSDatabase:
Type: AWS::RDS::DBInstance
Properties:
AllocatedStorage: 5
DBInstanceClass: db.t2.micro
DBName: myawesomeapp
Engine: postgres
EngineVersion: 9.3
MasterUsername: myAwesomeUsername
MasterUserPassword: myCrazyPassword
This file is in YAML format so indentation is important. You could also use JSON if you like.
These are not option settings so you cannot pass it as --option-settings test.json. You just need to bundle this file with your app source.
Read more about what properties you can configure on your RDS database here. On this page you can also find what properties are required and what properties are optional.
Let me know if the above does not work for you or if you have any further questions.

Answer 2

As of December 2017 we use the following ebextensions
$ cat .ebextensions/rds.config
Resources:
AWSEBRDSDBSecurityGroup:
Type: AWS::RDS::DBSecurityGroup
Properties:
EC2VpcId:
Fn::GetOptionSetting:
OptionName: "VpcId"
GroupDescription: RDS DB VPC Security Group
DBSecurityGroupIngress:
- EC2SecurityGroupId:
Ref: AWSEBSecurityGroup
AWSEBRDSDBSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: RDS DB Subnet Group
SubnetIds:
Fn::Split:
- ","
- Fn::GetOptionSetting:
OptionName: DBSubnets
AWSEBRDSDatabase:
Type: AWS::RDS::DBInstance
DeletionPolicy: Delete
Properties:
PubliclyAccessible: true
MultiAZ: false
Engine: mysql
EngineVersion: 5.7
BackupRetentionPeriod: 0
DBName: test
MasterUsername: toor
MasterUserPassword: 123456789
AllocatedStorage: 10
DBInstanceClass: db.t2.micro
DBSecurityGroups:
- Ref: AWSEBRDSDBSecurityGroup
DBSubnetGroupName:
Ref: AWSEBRDSDBSubnetGroup
Outputs:
RDSId:
Description: "RDS instance identifier"
Value:
Ref: "AWSEBRDSDatabase"
RDSEndpointAddress:
Description: "RDS endpoint address"
Value:
Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Address"]
RDSEndpointPort:
Description: "RDS endpoint port"
Value:
Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Port"]
AWSEBRDSDatabaseProperties:
Description: Properties associated with the RDS database instance
Value:
Fn::Join:
- ","
- - Ref: AWSEBRDSDatabase
- Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Address"]
- Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Port"]
With such custom options
$ cat .ebextensions/custom-options.config
option_settings:
"aws:elasticbeanstalk:customoption":
DBSubnets: subnet-1234567,subnet-7654321
VpcId: vpc-1234567
The only things - you have to explicitly pass RDS_* env variables to your application.

Answer 3

The other answers did not work in my environment as of Sept 2015. After much trial and error, the following worked for me:
config template snippet (YAML):
aws:rds:dbinstance:
DBAllocatedStorage: '5'
DBDeletionPolicy: Delete
DBEngine: postgres
DBEngineVersion: 9.3.9
DBInstanceClass: db.t2.micro
DBPassword: PASSWORD_HERE
DBUser: USERNAME_HERE
MultiAZDatabase: false
.ebextensions/rds.config file (JSON):
{
"Parameters": {
"AWSEBDBUser": {
"NoEcho": "true",
"Description": "The name of master user for the client DB Instance.",
"Default": "ebroot",
"Type": "String",
"ConstraintDescription": "must begin with a letter and contain only alphanumeric characters"
},
"AWSEBDBPassword": {
"NoEcho": "true",
"Description": "The master password for the DB instance.",
"Type": "String",
"ConstraintDescription": "must contain only alphanumeric characters"
},
"AWSEBDBName": {
"NoEcho": "true",
"Description": "The DB Name of the RDS instance",
"Default": "ebdb",
"Type": "String",
"ConstraintDescription": "must contain only alphanumeric characters"
}
},
"Resources": {
"AWSEBAutoScalingGroup": {
"Metadata": {
"AWS::ElasticBeanstalk::Ext": {
"_ParameterTriggers": {
"_TriggerConfigDeployment": {
"CmpFn::Insert": {
"values": [
{
"CmpFn::Ref": "Parameter.AWSEBDBUser"
},
{
"CmpFn::Ref": "Parameter.AWSEBDBPassword"
},
{
"CmpFn::Ref": "Parameter.AWSEBDBName"
}
]
}
}
},
"_ContainerConfigFileContent": {
"plugins": {
"rds": {
"Description": "RDS Environment variables",
"env": {
"RDS_USERNAME": {
"Ref": {
"CmpFn::Ref": "Parameter.AWSEBDBUser"
}
},
"RDS_PASSWORD": {
"Ref": {
"CmpFn::Ref": "Parameter.AWSEBDBPassword"
}
},
"RDS_DB_NAME": {
"Ref": {
"CmpFn::Ref": "Parameter.AWSEBDBName"
}
},
"RDS_HOSTNAME": {
"Fn::GetAtt": [
"AWSEBRDSDatabase",
"Endpoint.Address"
]
},
"RDS_PORT": {
"Fn::GetAtt": [
"AWSEBRDSDatabase",
"Endpoint.Port"
]
}
}
}
}
}
}
}
},
"AWSEBRDSDatabase": {
"Type": "AWS::RDS::DBInstance",
"DeletionPolicy": "Delete",
"Properties": {
"DBName": {
"Ref": {
"CmpFn::Ref": "Parameter.AWSEBDBName"
}
},
"AllocatedStorage": "5",
"DBInstanceClass": "db.t2.micro",
"Engine": "postgres",
"DBSecurityGroups": [
{
"Ref": "AWSEBRDSDBSecurityGroup"
}
],
"MasterUsername": {
"Ref": {
"CmpFn::Ref": "Parameter.AWSEBDBUser"
}
},
"MasterUserPassword": {
"Ref": {
"CmpFn::Ref": "Parameter.AWSEBDBPassword"
}
},
"MultiAZ": false
}
},
"AWSEBRDSDBSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"DBSecurityGroupIngress": {
"EC2SecurityGroupName": {
"Ref": "AWSEBSecurityGroup"
}
},
"GroupDescription": "Enable database access to Beanstalk application"
}
}
}
}

Answer 4

I had the same problem, couldn't get it to work via .ebextensions, and I don't like the EB CLI tool.
EB CLI uses an undocumented API feature, and a customized version of the botocore library ('eb_botocore') to make this work. :(
So I went ahead and forked botocore, and merged in the API data file used by eb_botocore: https://github.com/boto/botocore/pull/396
Then I ran 'python setup.py install' on both my modified botocore and aws-cli (both at master), and aws-cli now accepts a --template-specification option on the 'aws elasticbeanstalk create-environment' command. Hooray!
Example usage:
aws elasticbeanstalk create-environment\
...various options...\
--option-settings file://option-settings.json
--template-specification file://rds.us-west-2.json
where rds.us-west-2.json is:
{
"TemplateSnippets": [{
"SnippetName": "RdsExtensionEB",
"Order": 10000,
"SourceUrl":
"https://s3.amazonaws.com/elasticbeanstalk-env-resources-us-west-2/eb_snippets/rds/rds.json"
}]
}
(it appears you must select a snippet specific to your EB region).
and option-settings.json contains RDS-related settings similar to ones listed in the question (DBEngine, DBInstanceClass, DBAllocatedStorage, DBPassword).
It works great. I hope the AWS CLI team allows us to use this feature in the official tool in the future. I'm guessing it's not a trivial change or they would have done it already, but it's a pretty major omission functionality-wise from the Elastic Beanstalk API and AWS CLI tool, so hopefully they take a crack at it.