AWS Elastic Beanstalk - switching environment variables between environments - amazon-web-services

I am using Elastic Beanstalk's aws:elasticbeanstalk:application:environment namespace to configure my environment with env vars. How can I set different values for different environments (e.g. development versus production)?
Development:
option_settings:
aws:elasticbeanstalk:application:environment:
REDIS_HOST: localhost
Production:
option_settings:
aws:elasticbeanstalk:application:environment:
REDIS_HOST: prod.redis.server.com

The AWS CLI has a convenient way of doing this for you as the update-environment command allows you to set env vars from a specially formatted json file. Create a separate json file for each environment you will be deploying to.
Example json file named deploy-dev.json:
[
{
"Namespace": "aws:elasticbeanstalk:application:environment",
"OptionName": "NODE_ENV",
"Value": "dev"
},
{
"Namespace": "aws:elasticbeanstalk:application:environment",
"OptionName": "LOG_LEVEL",
"Value": "silly"
}
]
Deploy app and then update env vars:
aws elasticbeanstalk create-application-version --application-name "$EB_APP_NAME" --version-label "$EB_VERSION"
aws elasticbeanstalk update-environment --environment-name "$EB_ENV_NAME" --version-label "$EB_VERSION" --option-settings file://.ebextensions/deploy-dev.json

How it's changed depends on your deployment method.
One option you can use is to set a bogus value in your .config file:
option_settings:
aws:elasticbeanstalk:application:environment:
REDIS_HOST: change me
Then, after deployment, modify the variable either using the AWS Management Console or using the EB CLI:
eb setenv REDIS_HOST=prod.redis.server.com
If you are using CloudFormation to deploy your EB application, you can feed the value as part of the OptionSettings field in your CloudFormation template:
"EBConfigurationTemplate" : {
"Type" : "AWS::ElasticBeanstalk::ConfigurationTemplate",
"Properties" : {
"ApplicationName" : {
"Ref" : "EBApplication"
},
"Description" : "Configuration Template",
"OptionSettings" : [
{
"Namespace" : "aws:elasticbeanstalk:application:environment",
"OptionName" : "REDIS_HOST",
"Value" : {
"Ref" : "RedisHostInputParameter"
}
},
]
There are probably other methods too, but they will depend on the method of deployment.

Related

Retrieving Id elasticbeanstalk (EBS) environment in the terminal?

How do I get the Id of my elasticbeanstalk (EBS) environment in the terminal?
This command returns an object in the terminal with some properties for the environment:
aws elasticbeanstalk describe-environments --environment-names my-env
Is it possible to get only the EnvironmentId from that object in the terminal?
{
"Environments": [
{
"ApplicationName": "xxxx-xxxx-xxxx-xxxxx",
"EnvironmentName": "my-env",
"VersionLabel": "Initial Version",
"Status": "Ready",
"EnvironmentArn": "arn:aws:elasticbeanstalk:eu-central-1:xxxxxxx:environment/xxxx-xxxxx-xxxx-xxxx/my-env",
"EnvironmentLinks": [],
"PlatformArn": "arn:aws:elasticbeanstalk:eu-central-1::platform/Multi-container Docker running on 64bit Amazon Linux/2.11.0",
"EndpointURL": "awseb-e-2-xxxxx-xxxxxx-xxxxx.eu-central-1.elb.amazonaws.com",
"SolutionStackName": "64bit Amazon Linux 2018.03 v2.11.0 running Multi-container Docker 18.03.1-ce (Generic)",
"EnvironmentId": "e-1234567",
"CNAME": "my-env.elasticbeanstalk.com",
"AbortableOperationInProgress": false,
"Tier": {
"Version": "1.0",
"Type": "Standard",
"Name": "WebServer"
},
"Health": "Green",
"DateUpdated": "2018-07-12T06:10:17.056Z",
"DateCreated": "2018-07-11T20:03:26.970Z"
}
]
}
In this case, the result that I'm expecting in my terminal to appear is e-1234567
If you want to use the AWS CLI for this, you would need to filter the output of aws elasticbeanstalk describe-environments --environment-names my-env using a tool such as grep. One possible (by no means optimal/concise) solution :
aws elasticbeanstalk describe-environments --environment-names my-env | grep EnvironmentId | grep -Eo "e-[A-Za-z0-9_]+"
The better solution is to use an AWS SDK such as boto3 (the Python AWS SDK).
import boto3
elasticbeanstalk = boto3.client(
'elasticbeanstalk',
region_name='us-west-2'
)
response = elasticbeanstalk.describe_environments(
EnvironmentNames=['my-env']
)
if response['Environments']:
print(response['Environments'][0]['EnvironmentId'])
AWS SDKs are available in other popular languages such as Go, Java, Ruby, JavaScript, PHP as well.

EB Custom Platform without default VPC fails

I'm building up a custom platform to run our application. We have default VPC deleted, so according to the documentation I have to specify the VPC and subnet id almost everywhere. So the command I run for ebp looks like following:
ebp create -v --vpc.id vpc-xxxxxxx --vpc.subnets subnet-xxxxxx --vpc.publicip{code}
The above spins up the pcakcer environment without any issue however when the packer start to build an instance I'm getting the following error:
2017-12-07 18:07:05 UTC+0100 ERROR [Instance: i-00f376be9fc2fea34] Command failed on instance. Return code: 1 Output: 'packer build' failed, the build log has been saved to '/var/log/packer-builder/XXX1.0.19-builder.log'. Hook /opt/elasticbeanstalk/hooks/packerbuild/build.rb failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
2017-12-07 18:06:55 UTC+0100 ERROR 'packer build' failed, the build log has been saved to '/var/log/packer-builder/XXX:1.0.19-builder.log'
2017-12-07 18:06:55 UTC+0100 ERROR Packer failed with error: '--> HVM AMI builder: VPCIdNotSpecified: No default VPC for this user status code: 400, request id: 28d94e8c-e24d-440f-9c64-88826e042e9d'{code}
Both the template and the platform.yaml specify vpc_id and subnet id, however this is not taken into account by packer.
platform.yaml:
version: "1.0"
provisioner:
type: packer
template: tomcat_platform.json
flavor: ubuntu1604
metadata:
maintainer: <Enter your contact details here>
description: Ubuntu running Tomcat
operating_system_name: Ubuntu Server
operating_system_version: 16.04 LTS
programming_language_name: Java
programming_language_version: 8
framework_name: Tomcat
framework_version: 7
app_server_name: "none"
app_server_version: "none"
option_definitions:
- namespace: "aws:elasticbeanstalk:container:custom:application"
option_name: "TOMCAT_START"
description: "Default application startup command"
default_value: ""
option_settings:
- namespace: "aws:ec2:vpc"
option_name: "VPCId"
value: "vpc-xxxxxxx"
- namespace: "aws:ec2:vpc"
option_name: "Subnets"
value: "subnet-xxxxxxx"
- namespace: "aws:elb:listener:80"
option_name: "InstancePort"
value: "8080"
- namespace: "aws:elasticbeanstalk:application"
option_name: "Application Healthcheck URL"
value: "TCP:8080"
tomcat_platform.json:
{
"variables": {
"platform_name": "{{env `AWS_EB_PLATFORM_NAME`}}",
"platform_version": "{{env `AWS_EB_PLATFORM_VERSION`}}",
"platform_arn": "{{env `AWS_EB_PLATFORM_ARN`}}"
},
"builders": [
{
"type": "amazon-ebs",
"region": "eu-west-1",
"source_ami": "ami-8fd760f6",
"instance_type": "t2.micro",
"ami_virtualization_type": "hvm",
"ssh_username": "admin",
"ami_name": "Tomcat running on Ubuntu Server 16.04 LTS (built on {{isotime \"20060102150405\"}})",
"ami_description": "Tomcat running on Ubuntu Server 16.04 LTS (built on {{isotime \"20060102150405\"}})",
"vpc_id": "vpc-xxxxxx",
"subnet_id": "subnet-xxxxxx",
"associate_public_ip_address": "true",
"tags": {
"eb_platform_name": "{{user `platform_name`}}",
"eb_platform_version": "{{user `platform_version`}}",
"eb_platform_arn": "{{user `platform_arn`}}"
}
}
],
"provisioners": [
{
"type": "file",
"source": "builder",
"destination": "/tmp/"
},
{
"type": "shell",
"execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo {{ .Path }}",
"scripts": [
"builder/builder.sh"
]
}
]
}
Appreciate any idea on how to make this work as expected. I found couple of issues with the Packer, but seems to be resolved on their side so the documentation just says that the template must specify target VPC and Subnet.
The AWS documentation is a little misleading in this instance. You do need a default VPC in order to create a custom platform. From what I've seen, this is because the VPC flags that you are passing in to the ebp create command aren't passed along to the packer process that actually builds the platform.
To get around the error, you can just create a new default VPC that you just use for custom platform creation.
Packer looks for a default VPC (default behavior of Packer) while creating the resources required for building a custom platform which includes launching an EC2 instance, creating a Security Group etc., However, if a default VPC is not present in the region (for example, if it is deleted), Packer Build Task would fail with the following error:
Packer failed with error: '--> HVM AMI builder: VPCIdNotSpecified: No default VPC for this user status code: 400, request id: xyx-yxyx-xyx'
To fix this error, use the following attributes in the "builders" section of the 'template.json' file for packer to use a custom VPC and Subnets while creating the resources :
▸ vpc_id
▸ subnet_id

How do I set the health-check URL on an Elastic Beanstalk instance in a AWS CloudFormation template?

I have a CloudFormation template that starts an Elastic Beanstalk service. I want to set the health-check URL to /health in the template.
What is the convention for this in JSON?
You need to set the OptionSettings property of the AWS::ElasticBeanstalk::Environment:
For example:
{
"Type" : "AWS::ElasticBeanstalk::Environment",
"Properties" : {
"ApplicationName" : { "Ref" : "sampleApplication" },
"Description" : "AWS Elastic Beanstalk Environment running PHP Sample Application",
"VersionLabel" : "Initial Version",
"OptionSettings" : [
{
"Namespace" : "elasticbeanstalk:application",
"OptionName" : "Application Healthcheck URL",
"Value" : "/health"
}
],
"TemplateName" : "DefaultConfiguration",
}
}
In this example, the option Application Healthcheck URL is set to /health
For more information see:
AWS::ElasticBeanstalk::Environment
Configuration Options
ConfigurationOptionSettings
I added below config under the .ebextension directory
example:
option_settings:
AWSEBV2LoadBalancerTargetGroup.aws:elasticbeanstalk:environment:process:default:
HealthCheckPath: /health
Rodrigo's answer is good but incomplete.
You also need to modify the health_check_type to ELB, otherwise, when your web application process crash, but your EC2 instead remains healthy, the node won't get replaced.
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environmentconfig-autoscaling-healthchecktype.html
this is the format I am using
$ cat .ebextensions/0090_healthcheckurl.config
Resources:
AWSEBAutoScalingGroup:
Type: "AWS::AutoScaling::AutoScalingGroup"
Properties:
HealthCheckType: "ELB"
HealthCheckGracePeriod: "600"
option_settings:
- namespace: aws:elasticbeanstalk:application
option_name: Application Healthcheck URL
value: /_status

Create a single instance Elastic Beanstalk application with eb-cli

So i have a java application with the appropriate Procfile/Buildfile.
I have ran eb create in our scratch Elastic Beanstalk environment but i have to follow up with a manual configuration change to make it a single instance type vs a load balanced.
How would i use the eb-cli where upon eb create $ENVIRONMENT_NAME it generates a single instance environment?
There is a .elasticbeanstalk/config.yml
branch-defaults:
development:
environment: development
group_suffix: null
staging:
environment: staging
group_suffix: null
production:
environment: production
group_suffix: null
global:
application_name: feed-engine
branch: null
default_ec2_keyname: null
default_platform: Java 8
default_region: us-east-1
profile: prod
repository: null
sc: git
I figured out. You have to create a file with the extension .config under in the directory .ebextensions of your project.
{
"option_settings" : [
{
"namespace" : "aws:elasticbeanstalk:application",
"option_name" : "Application Healthcheck URL",
"value" : "/"
},
{
"namespace" : "aws:autoscaling:asg",
"option_name" : "MinSize",
"value" : "1"
},
{
"namespace" : "aws:autoscaling:asg",
"option_name" : "MaxSize",
"value" : "1"
},
{
"namespace" : "aws:autoscaling:asg",
"option_name" : "Custom Availability Zones",
"value" : "us-east-1a"
},
{
"namespace" : "aws:elasticbeanstalk:environment",
"option_name" : "EnvironmentType",
"value" : "SingleInstance"
},
]
}
// Create single instance (without load balancers)
eb create -s

How to publish kubernetes LoadBalancer Ingress URL to aws route53

Today when I launch an app using kubernetes over aws it exposes a publicly visible LoadBalancer Ingress URL, however to link that to my domain to make the app accessible to the public, I need to manually go into the aws route53 console in a browser on every launch. Can I update the aws route53 Resource Type A to match the latest Kubernetes LoadBalancer Ingress URL from the command line ?
Kubernetes over gcloud shares this challenge of having to either predefine a Static IP which is used in launch config or manually do a browser based domain linkage post launch. On aws I was hoping I could use something similar to this from the command line
aws route53domains update-domain-nameservers ???
__ OR __ can I predefine an aws kubernetes LoadBalancer Ingress similar to doing a predefined Static IP when over gcloud ?
to show the deployed app's LoadBalancer Ingress URL issue
kubectl describe svc
... output
Name: aaa-deployment-407
Namespace: ruptureofthemundaneplane
Labels: app=bbb
pod-template-hash=4076262206
Selector: app=bbb,pod-template-hash=4076262206
Type: LoadBalancer
IP: 10.0.51.82
LoadBalancer Ingress: a244bodhisattva79c17cf7-61619.us-east-1.elb.amazonaws.com
Port: port-1 80/TCP
NodePort: port-1 32547/TCP
Endpoints: 10.201.0.3:80
Port: port-2 443/TCP
NodePort: port-2 31248/TCP
Endpoints: 10.201.0.3:443
Session Affinity: None
No events.
UPDATE:
Getting error trying new command line technique (hat tip to #error2007s comment) ... issue this
aws route53 list-hosted-zones
... outputs
{
"HostedZones": [
{
"ResourceRecordSetCount": 6,
"CallerReference": "2D58A764-1FAC-DEB4-8AC7-AD37E74B94E6",
"Config": {
"PrivateZone": false
},
"Id": "/hostedzone/Z3II3949ZDMDXV",
"Name": "chainsawhaircut.com."
}
]
}
Important bit used below : hostedzone Z3II3949ZDMDXV
now I craft following using this Doc (and this Doc as well) as file /change-resource-record-sets.json (NOTE I can successfully change Type A using a similar cli call ... however I need to change Type A with an Alias Target of LoadBalancer Ingress URL)
{
"Comment": "Update record to reflect new IP address of fresh deploy",
"Changes": [{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "chainsawhaircut.com.",
"Type": "A",
"TTL": 60,
"AliasTarget": {
"HostedZoneId": "Z3II3949ZDMDXV",
"DNSName": "a244bodhisattva79c17cf7-61619.us-east-1.elb.amazonaws.com",
"EvaluateTargetHealth": false
}
}
}]
}
on command line I then issue
aws route53 change-resource-record-sets --hosted-zone-id Z3II3949ZDMDXV --change-batch file:///change-resource-record-sets.json
which give this error message
An error occurred (InvalidInput) when calling the ChangeResourceRecordSets operation: Invalid request
Any insights ?
Here is the logic needed to update aws route53 Resource Record Type A with value from freshly minted kubernetes LoadBalancer Ingress URL
step 1 - identify your hostedzone Id by issuing
aws route53 list-hosted-zones
... from output here is clip for my domain
"Id": "/hostedzone/Z3II3949ZDMDXV",
... importantly never populate json with hostedzone Z3II3949ZDMDXV its only used as a cli parm ... there is a second similarly named token HostedZoneId which is entirely different
step 2 - see current value of your route53 domain record ... issue :
aws route53 list-resource-record-sets --hosted-zone-id Z3II3949ZDMDXV --query "ResourceRecordSets[?Name == 'scottstensland.com.']"
... output
[
{
"AliasTarget": {
"HostedZoneId": "Z35SXDOTRQ7X7K",
"EvaluateTargetHealth": false,
"DNSName": "dualstack.asomepriorvalue39e7db-1867261689.us-east-1.elb.amazonaws.com."
},
"Type": "A",
"Name": "scottstensland.com."
},
{
"ResourceRecords": [
{
"Value": "ns-1238.awsdns-26.org."
},
{
"Value": "ns-201.awsdns-25.com."
},
{
"Value": "ns-969.awsdns-57.net."
},
{
"Value": "ns-1823.awsdns-35.co.uk."
}
],
"Type": "NS",
"Name": "scottstensland.com.",
"TTL": 172800
},
{
"ResourceRecords": [
{
"Value": "ns-1238.awsdns-26.org. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400"
}
],
"Type": "SOA",
"Name": "scottstensland.com.",
"TTL": 900
}
]
... in above notice value of
"HostedZoneId": "Z35SXDOTRQ7X7K",
which is the second similarly name token Do NOT use wrong Hosted Zone ID
step 3 - put below into your change file aws_route53_type_A.json (for syntax Doc see link mentioned in comment above)
{
"Comment": "Update record to reflect new DNSName of fresh deploy",
"Changes": [
{
"Action": "UPSERT",
"ResourceRecordSet": {
"AliasTarget": {
"HostedZoneId": "Z35SXDOTRQ7X7K",
"EvaluateTargetHealth": false,
"DNSName": "dualstack.a0b82c81f47d011e6b98a0a28439e7db-1867261689.us-east-1.elb.amazonaws.com."
},
"Type": "A",
"Name": "scottstensland.com."
}
}
]
}
To identify value for above field "DNSName" ... after the kubernetes app deploy on aws it responds with a LoadBalancer Ingress as shown in output of cli command :
kubectl describe svc --namespace=ruptureofthemundaneplane
... as in
LoadBalancer Ingress: a0b82c81f47d011e6b98a0a28439e7db-1867261689.us-east-1.elb.amazonaws.com
... even though my goal is to execute a command line call I can do this manually by getting into the aws console browser ... pull up my domain on route53 ...
... In this browser picklist editable text box (circled in green) I noticed the URL gets magically prepended with : dualstack. Previously I was missing that magic string ... so json key "DNSName" wants this
dualstack.a0b82c81f47d011e6b98a0a28439e7db-1867261689.us-east-1.elb.amazonaws.com.
finally execute the change request
aws route53 change-resource-record-sets --hosted-zone-id Z3II3949ZDMDXV --change-batch file://./aws_route53_type_A.json
... output
{
"ChangeInfo": {
"Status": "PENDING",
"Comment": "Update record to reflect new DNSName of fresh deploy",
"SubmittedAt": "2016-07-13T14:53:02.789Z",
"Id": "/change/CFUX5R9XKGE1C"
}
}
.... now to confirm change is live run this to show record
aws route53 list-resource-record-sets --hosted-zone-id Z3II3949ZDMDXV
You can also use external-dns project.
AWS specific setup can be found here
After installation it can be used with an annotation e.g.: external-dns.alpha.kubernetes.io/hostname: nginx.external-dns-test.my-org.com.
Note the IAM permissions needs to be set properly.