So i have a java application with the appropriate Procfile/Buildfile.
I have ran eb create in our scratch Elastic Beanstalk environment but i have to follow up with a manual configuration change to make it a single instance type vs a load balanced.
How would i use the eb-cli where upon eb create $ENVIRONMENT_NAME it generates a single instance environment?
There is a .elasticbeanstalk/config.yml
branch-defaults:
development:
environment: development
group_suffix: null
staging:
environment: staging
group_suffix: null
production:
environment: production
group_suffix: null
global:
application_name: feed-engine
branch: null
default_ec2_keyname: null
default_platform: Java 8
default_region: us-east-1
profile: prod
repository: null
sc: git
I figured out. You have to create a file with the extension .config under in the directory .ebextensions of your project.
{
"option_settings" : [
{
"namespace" : "aws:elasticbeanstalk:application",
"option_name" : "Application Healthcheck URL",
"value" : "/"
},
{
"namespace" : "aws:autoscaling:asg",
"option_name" : "MinSize",
"value" : "1"
},
{
"namespace" : "aws:autoscaling:asg",
"option_name" : "MaxSize",
"value" : "1"
},
{
"namespace" : "aws:autoscaling:asg",
"option_name" : "Custom Availability Zones",
"value" : "us-east-1a"
},
{
"namespace" : "aws:elasticbeanstalk:environment",
"option_name" : "EnvironmentType",
"value" : "SingleInstance"
},
]
}
// Create single instance (without load balancers)
eb create -s
Related
I'm building up a custom platform to run our application. We have default VPC deleted, so according to the documentation I have to specify the VPC and subnet id almost everywhere. So the command I run for ebp looks like following:
ebp create -v --vpc.id vpc-xxxxxxx --vpc.subnets subnet-xxxxxx --vpc.publicip{code}
The above spins up the pcakcer environment without any issue however when the packer start to build an instance I'm getting the following error:
2017-12-07 18:07:05 UTC+0100 ERROR [Instance: i-00f376be9fc2fea34] Command failed on instance. Return code: 1 Output: 'packer build' failed, the build log has been saved to '/var/log/packer-builder/XXX1.0.19-builder.log'. Hook /opt/elasticbeanstalk/hooks/packerbuild/build.rb failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
2017-12-07 18:06:55 UTC+0100 ERROR 'packer build' failed, the build log has been saved to '/var/log/packer-builder/XXX:1.0.19-builder.log'
2017-12-07 18:06:55 UTC+0100 ERROR Packer failed with error: '--> HVM AMI builder: VPCIdNotSpecified: No default VPC for this user status code: 400, request id: 28d94e8c-e24d-440f-9c64-88826e042e9d'{code}
Both the template and the platform.yaml specify vpc_id and subnet id, however this is not taken into account by packer.
platform.yaml:
version: "1.0"
provisioner:
type: packer
template: tomcat_platform.json
flavor: ubuntu1604
metadata:
maintainer: <Enter your contact details here>
description: Ubuntu running Tomcat
operating_system_name: Ubuntu Server
operating_system_version: 16.04 LTS
programming_language_name: Java
programming_language_version: 8
framework_name: Tomcat
framework_version: 7
app_server_name: "none"
app_server_version: "none"
option_definitions:
- namespace: "aws:elasticbeanstalk:container:custom:application"
option_name: "TOMCAT_START"
description: "Default application startup command"
default_value: ""
option_settings:
- namespace: "aws:ec2:vpc"
option_name: "VPCId"
value: "vpc-xxxxxxx"
- namespace: "aws:ec2:vpc"
option_name: "Subnets"
value: "subnet-xxxxxxx"
- namespace: "aws:elb:listener:80"
option_name: "InstancePort"
value: "8080"
- namespace: "aws:elasticbeanstalk:application"
option_name: "Application Healthcheck URL"
value: "TCP:8080"
tomcat_platform.json:
{
"variables": {
"platform_name": "{{env `AWS_EB_PLATFORM_NAME`}}",
"platform_version": "{{env `AWS_EB_PLATFORM_VERSION`}}",
"platform_arn": "{{env `AWS_EB_PLATFORM_ARN`}}"
},
"builders": [
{
"type": "amazon-ebs",
"region": "eu-west-1",
"source_ami": "ami-8fd760f6",
"instance_type": "t2.micro",
"ami_virtualization_type": "hvm",
"ssh_username": "admin",
"ami_name": "Tomcat running on Ubuntu Server 16.04 LTS (built on {{isotime \"20060102150405\"}})",
"ami_description": "Tomcat running on Ubuntu Server 16.04 LTS (built on {{isotime \"20060102150405\"}})",
"vpc_id": "vpc-xxxxxx",
"subnet_id": "subnet-xxxxxx",
"associate_public_ip_address": "true",
"tags": {
"eb_platform_name": "{{user `platform_name`}}",
"eb_platform_version": "{{user `platform_version`}}",
"eb_platform_arn": "{{user `platform_arn`}}"
}
}
],
"provisioners": [
{
"type": "file",
"source": "builder",
"destination": "/tmp/"
},
{
"type": "shell",
"execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo {{ .Path }}",
"scripts": [
"builder/builder.sh"
]
}
]
}
Appreciate any idea on how to make this work as expected. I found couple of issues with the Packer, but seems to be resolved on their side so the documentation just says that the template must specify target VPC and Subnet.
The AWS documentation is a little misleading in this instance. You do need a default VPC in order to create a custom platform. From what I've seen, this is because the VPC flags that you are passing in to the ebp create command aren't passed along to the packer process that actually builds the platform.
To get around the error, you can just create a new default VPC that you just use for custom platform creation.
Packer looks for a default VPC (default behavior of Packer) while creating the resources required for building a custom platform which includes launching an EC2 instance, creating a Security Group etc., However, if a default VPC is not present in the region (for example, if it is deleted), Packer Build Task would fail with the following error:
Packer failed with error: '--> HVM AMI builder: VPCIdNotSpecified: No default VPC for this user status code: 400, request id: xyx-yxyx-xyx'
To fix this error, use the following attributes in the "builders" section of the 'template.json' file for packer to use a custom VPC and Subnets while creating the resources :
▸ vpc_id
▸ subnet_id
I have a CloudFormation template that starts an Elastic Beanstalk service. I want to set the health-check URL to /health in the template.
What is the convention for this in JSON?
You need to set the OptionSettings property of the AWS::ElasticBeanstalk::Environment:
For example:
{
"Type" : "AWS::ElasticBeanstalk::Environment",
"Properties" : {
"ApplicationName" : { "Ref" : "sampleApplication" },
"Description" : "AWS Elastic Beanstalk Environment running PHP Sample Application",
"VersionLabel" : "Initial Version",
"OptionSettings" : [
{
"Namespace" : "elasticbeanstalk:application",
"OptionName" : "Application Healthcheck URL",
"Value" : "/health"
}
],
"TemplateName" : "DefaultConfiguration",
}
}
In this example, the option Application Healthcheck URL is set to /health
For more information see:
AWS::ElasticBeanstalk::Environment
Configuration Options
ConfigurationOptionSettings
I added below config under the .ebextension directory
example:
option_settings:
AWSEBV2LoadBalancerTargetGroup.aws:elasticbeanstalk:environment:process:default:
HealthCheckPath: /health
Rodrigo's answer is good but incomplete.
You also need to modify the health_check_type to ELB, otherwise, when your web application process crash, but your EC2 instead remains healthy, the node won't get replaced.
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environmentconfig-autoscaling-healthchecktype.html
this is the format I am using
$ cat .ebextensions/0090_healthcheckurl.config
Resources:
AWSEBAutoScalingGroup:
Type: "AWS::AutoScaling::AutoScalingGroup"
Properties:
HealthCheckType: "ELB"
HealthCheckGracePeriod: "600"
option_settings:
- namespace: aws:elasticbeanstalk:application
option_name: Application Healthcheck URL
value: /_status
I am using Elastic Beanstalk's aws:elasticbeanstalk:application:environment namespace to configure my environment with env vars. How can I set different values for different environments (e.g. development versus production)?
Development:
option_settings:
aws:elasticbeanstalk:application:environment:
REDIS_HOST: localhost
Production:
option_settings:
aws:elasticbeanstalk:application:environment:
REDIS_HOST: prod.redis.server.com
The AWS CLI has a convenient way of doing this for you as the update-environment command allows you to set env vars from a specially formatted json file. Create a separate json file for each environment you will be deploying to.
Example json file named deploy-dev.json:
[
{
"Namespace": "aws:elasticbeanstalk:application:environment",
"OptionName": "NODE_ENV",
"Value": "dev"
},
{
"Namespace": "aws:elasticbeanstalk:application:environment",
"OptionName": "LOG_LEVEL",
"Value": "silly"
}
]
Deploy app and then update env vars:
aws elasticbeanstalk create-application-version --application-name "$EB_APP_NAME" --version-label "$EB_VERSION"
aws elasticbeanstalk update-environment --environment-name "$EB_ENV_NAME" --version-label "$EB_VERSION" --option-settings file://.ebextensions/deploy-dev.json
How it's changed depends on your deployment method.
One option you can use is to set a bogus value in your .config file:
option_settings:
aws:elasticbeanstalk:application:environment:
REDIS_HOST: change me
Then, after deployment, modify the variable either using the AWS Management Console or using the EB CLI:
eb setenv REDIS_HOST=prod.redis.server.com
If you are using CloudFormation to deploy your EB application, you can feed the value as part of the OptionSettings field in your CloudFormation template:
"EBConfigurationTemplate" : {
"Type" : "AWS::ElasticBeanstalk::ConfigurationTemplate",
"Properties" : {
"ApplicationName" : {
"Ref" : "EBApplication"
},
"Description" : "Configuration Template",
"OptionSettings" : [
{
"Namespace" : "aws:elasticbeanstalk:application:environment",
"OptionName" : "REDIS_HOST",
"Value" : {
"Ref" : "RedisHostInputParameter"
}
},
]
There are probably other methods too, but they will depend on the method of deployment.
My goal is to have a Cloud Formation template which not only automatically creates a VPC with a NAT host and bastion host, but which deploys a .NET app pulled from S3 into an Elastic Beanstalk which is load balanced and more importantly only allows access to the app from my office, NOT the whole internet. It seems that even though the app might be in a VPC in a Private subnet and the ELB is in the Public subnet, that the Network ACL on the Public subnet is irrelevant. If I lock down the Public subnet to only my office, connections from outside the office can still come into the ELB and hit the application.
What seems to work is applying a Security group to the ELB, but I do not see any way to force the creation of a specific ELB with a specific SG inside a "AWS::ElasticBeanstalk::Environment" object. The ELB and ELB SG are created automatically by beanstalk and must be manually altered after CF runs. I don't want that. I want a way to create everything in CF in an automated way with no manual steps after the fact. yes, I've tried Cloud Former on a manually created stakc. No, it doesn't give me what I want.
Here's an excerpt from my CF Template:
"MyWebApp": {
"Type": "AWS::ElasticBeanstalk::Application",
"Properties": {
"ApplicationName" : "AlmDemoWebApp",
"Description": "MyWebapp"
}
},
"MyWebAppVersion": {
"Type": "AWS::ElasticBeanstalk::ApplicationVersion",
"Properties": {
"ApplicationName": {"Ref": "MyWebApp"},
"SourceBundle": {
"S3Bucket": "mywebapp",
"S3Key": {"Fn::Join" : ["", ["MyWebApp.", {"Ref":"Version"}, ".zip"]]}
}
}
},
"MyWebAppEnvironment" : {
"DependsOn" : ["MyWebApp", "MyWebAppVersion", "BastionSecurityGroup", "BeanstalkSecurityGroup", "VPC", "EBLoadBalancer", "EBLoadBalancerSecurityGroup", "PrivateSubnet", "PublicSubnet"],
"Type" : "AWS::ElasticBeanstalk::Environment",
"Properties" : {
"ApplicationName" : { "Ref" : "MyWebApp" },
"Description" : "MyWebApp Target Environment",
"SolutionStackName": "64bit Windows Server 2012 R2 running IIS 8.5",
"OptionSettings" : [
{"Namespace" : "aws:autoscaling:launchconfiguration", "OptionName" : "SecurityGroups", "Value" : { "Ref" : "BeanstalkSecurityGroup" }},
{"Namespace" : "aws:autoscaling:launchconfiguration", "OptionName" : "EC2KeyName", "Value" : { "Ref" : "InstanceKeyName" }},
{"Namespace" : "aws:ec2:vpc", "OptionName" : "VPCId", "Value" : { "Ref" : "VPC" }},
{"Namespace" : "aws:ec2:vpc", "OptionName" : "Subnets", "Value" : { "Ref" : "PrivateSubnet" }},
{"Namespace" : "aws:ec2:vpc", "OptionName" : "ELBSubnets", "Value" : { "Ref" : "PublicSubnet" }}],
"VersionLabel": {"Ref": "MyWebAppVersion"}
}
}
Is there some mysterious and poorly documented option that I can put in the AWS::ElasticBeanstalk::Environment -> Properties -> OptionSettings that will force the Elastic Beanstalk to use a specific ELB configured previously in the CF template ("EBLoadBalancer") rather than automatically create one with a random name? Applying ingress rules to the "BeanstalkSecurityGroup" doesn't seem to help. The SG rules have to be on the ELB to actually work, apparently.
Sounds like you need to create an internal ELB for your Elastic Beanstalk stack. For this, create a property as below.
"ELBScheme":{
"Type":"String",
"AllowedValues":[
"internal"
],
"Default":"internal",
"Description":"Internal load balancer in VPC so that your Elastic Beanstalk application cannot be accessed from outside your VPC."
}
And refer it as a namespace in your AWS::ElasticBeanstalk::Environment "MyWebAppEnvironment"
{
"Namespace":"aws:ec2:vpc",
"OptionName":"ELBScheme",
"Value":"internal"
}
How can I create an RDS instance with the create-environment or another subcommand of aws elasticbeanstalk? I've tried several combinations of parameters to no avail. Below is an example.
APP_NAME="randall-railsapp"
aws s3api create-bucket --bucket "$APP_NAME"
APP_VERSION="$(git describe --always)"
APP_FILE="deploy-$APP_NAME-$APP_VERSION.zip"
git archive -o "$APP_FILE" HEAD
aws s3 cp "$APP_FILE" "s3://$APP_NAME/$APP_FILE"
aws --region us-east-1 elasticbeanstalk create-application-version \
--auto-create-application \
--application-name "$APP_NAME" \
--version-label "$APP_VERSION" \
--source-bundle S3Bucket="$APP_NAME",S3Key="$APP_FILE"
aws --region us-east-1 elasticbeanstalk create-environment \
--application-name "$APP_NAME" \
--version-label "$APP_VERSION" \
--environment-name "$APP_NAME-env" \
--description "randall's rails app environment" \
--solution-stack-name "64bit Amazon Linux 2014.03 v1.0.0 running Ruby 2.1 (Puma)" \
--cname-prefix "$APP_NAME-test" \
--option-settings file://test.json
And the contents of test.json:
[
{
"OptionName": "EC2KeyName",
"Namespace": "aws:autoscaling:launchconfiguration",
"Value": "a-key-is-here"
},
{
"OptionName": "EnvironmentType",
"Namespace": "aws:elasticbeanstalk:environment",
"Value": "SingleInstance"
},
{
"OptionName": "SECRET_KEY_BASE",
"Namespace": "aws:elasticbeanstalk:application:environment",
"Value": "HAHAHAHAHAHA"
},
{
"OptionName": "DBPassword",
"Namespace": "aws:rds:dbinstance",
"Value": "hunter2"
},
{
"OptionName": "DBUser",
"Namespace": "aws:rds:dbinstance",
"Value": "random"
},
{
"OptionName": "DBEngineVersion",
"Namespace": "aws:rds:dbinstance",
"Value": "9.3"
},
{
"OptionName": "DBEngine",
"Namespace": "aws:rds:dbinstance",
"Value": "postgres"
}
]
Anyone know why this is failing? Anything I specify with a aws:rds:dbinstance namespace seems to get removed from the configuration.
Just setting the aws:rds:dbinstance options does not create an RDS database.
Currently you can create an RDS instance using one of the following techniques:
Create using AWS Console
Use eb cli
Use Resources section of ebextensions to create an RDS resource
The first two approaches are the most convenient as they do all the heavy lifting for you but for the third one you have to do some extra work. The third approach is what you would want to use if you are not using the console or eb CLI.
You can create an RDS resource for your beanstalk environment using the following ebextension snippet. Create a file called 01-rds.config in the .ebextensions directory of your app source.
Resources:
AWSEBRDSDatabase:
Type: AWS::RDS::DBInstance
Properties:
AllocatedStorage: 5
DBInstanceClass: db.t2.micro
DBName: myawesomeapp
Engine: postgres
EngineVersion: 9.3
MasterUsername: myAwesomeUsername
MasterUserPassword: myCrazyPassword
This file is in YAML format so indentation is important. You could also use JSON if you like.
These are not option settings so you cannot pass it as --option-settings test.json. You just need to bundle this file with your app source.
Read more about what properties you can configure on your RDS database here. On this page you can also find what properties are required and what properties are optional.
Let me know if the above does not work for you or if you have any further questions.
As of December 2017 we use the following ebextensions
$ cat .ebextensions/rds.config
Resources:
AWSEBRDSDBSecurityGroup:
Type: AWS::RDS::DBSecurityGroup
Properties:
EC2VpcId:
Fn::GetOptionSetting:
OptionName: "VpcId"
GroupDescription: RDS DB VPC Security Group
DBSecurityGroupIngress:
- EC2SecurityGroupId:
Ref: AWSEBSecurityGroup
AWSEBRDSDBSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: RDS DB Subnet Group
SubnetIds:
Fn::Split:
- ","
- Fn::GetOptionSetting:
OptionName: DBSubnets
AWSEBRDSDatabase:
Type: AWS::RDS::DBInstance
DeletionPolicy: Delete
Properties:
PubliclyAccessible: true
MultiAZ: false
Engine: mysql
EngineVersion: 5.7
BackupRetentionPeriod: 0
DBName: test
MasterUsername: toor
MasterUserPassword: 123456789
AllocatedStorage: 10
DBInstanceClass: db.t2.micro
DBSecurityGroups:
- Ref: AWSEBRDSDBSecurityGroup
DBSubnetGroupName:
Ref: AWSEBRDSDBSubnetGroup
Outputs:
RDSId:
Description: "RDS instance identifier"
Value:
Ref: "AWSEBRDSDatabase"
RDSEndpointAddress:
Description: "RDS endpoint address"
Value:
Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Address"]
RDSEndpointPort:
Description: "RDS endpoint port"
Value:
Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Port"]
AWSEBRDSDatabaseProperties:
Description: Properties associated with the RDS database instance
Value:
Fn::Join:
- ","
- - Ref: AWSEBRDSDatabase
- Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Address"]
- Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Port"]
With such custom options
$ cat .ebextensions/custom-options.config
option_settings:
"aws:elasticbeanstalk:customoption":
DBSubnets: subnet-1234567,subnet-7654321
VpcId: vpc-1234567
The only things - you have to explicitly pass RDS_* env variables to your application.
The other answers did not work in my environment as of Sept 2015. After much trial and error, the following worked for me:
config template snippet (YAML):
aws:rds:dbinstance:
DBAllocatedStorage: '5'
DBDeletionPolicy: Delete
DBEngine: postgres
DBEngineVersion: 9.3.9
DBInstanceClass: db.t2.micro
DBPassword: PASSWORD_HERE
DBUser: USERNAME_HERE
MultiAZDatabase: false
.ebextensions/rds.config file (JSON):
{
"Parameters": {
"AWSEBDBUser": {
"NoEcho": "true",
"Description": "The name of master user for the client DB Instance.",
"Default": "ebroot",
"Type": "String",
"ConstraintDescription": "must begin with a letter and contain only alphanumeric characters"
},
"AWSEBDBPassword": {
"NoEcho": "true",
"Description": "The master password for the DB instance.",
"Type": "String",
"ConstraintDescription": "must contain only alphanumeric characters"
},
"AWSEBDBName": {
"NoEcho": "true",
"Description": "The DB Name of the RDS instance",
"Default": "ebdb",
"Type": "String",
"ConstraintDescription": "must contain only alphanumeric characters"
}
},
"Resources": {
"AWSEBAutoScalingGroup": {
"Metadata": {
"AWS::ElasticBeanstalk::Ext": {
"_ParameterTriggers": {
"_TriggerConfigDeployment": {
"CmpFn::Insert": {
"values": [
{
"CmpFn::Ref": "Parameter.AWSEBDBUser"
},
{
"CmpFn::Ref": "Parameter.AWSEBDBPassword"
},
{
"CmpFn::Ref": "Parameter.AWSEBDBName"
}
]
}
}
},
"_ContainerConfigFileContent": {
"plugins": {
"rds": {
"Description": "RDS Environment variables",
"env": {
"RDS_USERNAME": {
"Ref": {
"CmpFn::Ref": "Parameter.AWSEBDBUser"
}
},
"RDS_PASSWORD": {
"Ref": {
"CmpFn::Ref": "Parameter.AWSEBDBPassword"
}
},
"RDS_DB_NAME": {
"Ref": {
"CmpFn::Ref": "Parameter.AWSEBDBName"
}
},
"RDS_HOSTNAME": {
"Fn::GetAtt": [
"AWSEBRDSDatabase",
"Endpoint.Address"
]
},
"RDS_PORT": {
"Fn::GetAtt": [
"AWSEBRDSDatabase",
"Endpoint.Port"
]
}
}
}
}
}
}
}
},
"AWSEBRDSDatabase": {
"Type": "AWS::RDS::DBInstance",
"DeletionPolicy": "Delete",
"Properties": {
"DBName": {
"Ref": {
"CmpFn::Ref": "Parameter.AWSEBDBName"
}
},
"AllocatedStorage": "5",
"DBInstanceClass": "db.t2.micro",
"Engine": "postgres",
"DBSecurityGroups": [
{
"Ref": "AWSEBRDSDBSecurityGroup"
}
],
"MasterUsername": {
"Ref": {
"CmpFn::Ref": "Parameter.AWSEBDBUser"
}
},
"MasterUserPassword": {
"Ref": {
"CmpFn::Ref": "Parameter.AWSEBDBPassword"
}
},
"MultiAZ": false
}
},
"AWSEBRDSDBSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"DBSecurityGroupIngress": {
"EC2SecurityGroupName": {
"Ref": "AWSEBSecurityGroup"
}
},
"GroupDescription": "Enable database access to Beanstalk application"
}
}
}
}
I had the same problem, couldn't get it to work via .ebextensions, and I don't like the EB CLI tool.
EB CLI uses an undocumented API feature, and a customized version of the botocore library ('eb_botocore') to make this work. :(
So I went ahead and forked botocore, and merged in the API data file used by eb_botocore: https://github.com/boto/botocore/pull/396
Then I ran 'python setup.py install' on both my modified botocore and aws-cli (both at master), and aws-cli now accepts a --template-specification option on the 'aws elasticbeanstalk create-environment' command. Hooray!
Example usage:
aws elasticbeanstalk create-environment\
...various options...\
--option-settings file://option-settings.json
--template-specification file://rds.us-west-2.json
where rds.us-west-2.json is:
{
"TemplateSnippets": [{
"SnippetName": "RdsExtensionEB",
"Order": 10000,
"SourceUrl":
"https://s3.amazonaws.com/elasticbeanstalk-env-resources-us-west-2/eb_snippets/rds/rds.json"
}]
}
(it appears you must select a snippet specific to your EB region).
and option-settings.json contains RDS-related settings similar to ones listed in the question (DBEngine, DBInstanceClass, DBAllocatedStorage, DBPassword).
It works great. I hope the AWS CLI team allows us to use this feature in the official tool in the future. I'm guessing it's not a trivial change or they would have done it already, but it's a pretty major omission functionality-wise from the Elastic Beanstalk API and AWS CLI tool, so hopefully they take a crack at it.