Elastic Beanstalk. (Created with code star). Java. Spring Boot. EC2.
The documentation states that we can change the environment type within the GUI to load balanced from a single instance:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features-managing-env-types.html
However, when trying it out. After the changes have taken hold and my environment was load balanced. Upon further deployments, my changes appear to have been reset and the environment type is now shown to be single-instance.
Upon examination of the service package, there is cloud formation that states the environment type
template.yml:
EBConfigurationTemplate:
Description: The AWS Elastic Beanstalk configuration template to be created for this project, which defines configuration settings used to deploy different versions of an application.
Type: AWS::ElasticBeanstalk::ConfigurationTemplate
Properties:
ApplicationName: !Ref 'EBApplication'
Description: The name of the sample configuration template.
OptionSettings:
- Namespace: aws:elasticbeanstalk:environment
OptionName: EnvironmentType
Value: SingleInstance
Which must have overwritten any of the GUI changes that have been made by hand upon further deployment.
I presume such changes (anything already in cloud formation) must then be done through cloud formation and not be done by hand in the GUI? (What changes can be done by GUI is it changes related to .ebextensions files?)
Bonus: Has someone seen a GitHub project that have used ElasticBeanstalk with EC2 and hence must have done this in their cloud formation?
I presume such changes (anything already in cloud formation) must then be done through cloud formation and not be done by hand in the GUI?
Yes. It is a bad practice to manually modify resources created and managed by CFN, which can lead to issues. From docs:
changes made outside of CloudFormation can complicate stack update or deletion operations.
What happens when you do this, is so called drift. This means that CFN is simply not aware of any outside changes made to its resources. So once you start developing your solutions through CFN, every change to them should also be done through CFN.
Sadly I'm not sure what you are after in the bonus questions, thus can't comment on it.
Related
Context:
I have been successful in setting up an eks cluster using eksctl with the help of AWS's documentation. The subsections that were main useful to me are:
clusters
> creating a cluster
Networking
> pod networking
> Installing AWS LB controller.
workloads
> application load balancing
Now I am trying use the same command that I learnt from those subsection to write a shell script and automate the entire setup process up until the sample app deployment.
I am stuck where I have to find out the VPC id and region code created via the eksctl and fill them into the v2_4_3_full.yaml file. (This file creates all the components necessary to provision an ingress component under the working namespace in Kubernetes).
I feel totally blank when I think of ways to automatically do that instead of manually referring the vpc and region ids.
Below is the part of the YAML file where that has to be done. Not only do the valueS have to be filled in automatically but even the corresponding parameters have to be added in. Those parameters will be the last two from below under args. I haven't a clue on how to achieve that.
spec:
containers:
- args:
- --cluster-name=your-cluster-name
- --ingress-class=alb
- --aws-vpc-id=vpc-xxxxxxxx
- --aws-region=region-code
I am not sure what ways exist if they do. But I imagine it would be some shell command that I don't know or some python package that does that. Nonetheless, any suggestion is greatly appreciated.
Good afternoon, all:
I have a CloudFormation template that creates an EBS Web Environment with an internal NLB. My plan for this is to then create a VPC Link and API Gateway to proxy to the web worker, essentially keeping the VPC private. I can accomplish this through the Console, and the POC for that works great. But what I would really love to do is take the ARN of the NLB created for the EBS Web Environment, and use that as the output for the CloudFormation template, which I can then use as the input for the CF template that will create the VPC Link. The questions I have are, is this possible, and if so, how do I go about getting the ARN for the NLB in this scenario? I can accomplish the feat in a two-step process; passing the ARN manually as a parameter to the second template. But I'd really like to do it programmatically if possible.
Any links, examples or advice that you can provide on this use case would be very much appreciated.
I want to do something similar but the problem is that the first CF stack that contains the EBS resources actually spins off second CF stack that contains the Web application resources. The NLB is in the second stack.
The second stack template is generated by EBS and you don't have control over it, which means you can't define outputs.
You can use the AWS Cli to list the second stack's resources and look for the NLB resource based on resource type, then grab the ARN. The problem with this is that you don't know the name of the second stack, so would again require the two stage deployment that you describe in your question.
Not much of an answer but I'll keep digging.
Update
This is similar How To Extract Load Balancer Name from Elastic Beanstalk Environment in CloudFormation
You could also create a Customization using the .ebextensions mechanism.
Recently I had to attach a WAF to an ElasticBeanstalk Application Load Balancer. If you create a .config file and place it in .ebextensions configure your environment and customize the AWS resources that it contains. If you haven't given a custom name to any of your resources you can reference it using the standard Resource names found here
(https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-format-resources-eb.html)
NLBs, like ALBs use the same default resource name, so AWSEBV2LoadBalancer is what the doctor ordered.
My config looked like
Resources:
PublicWAF:
Type: AWS::WAFRegional::WebACLAssociation
Properties:
ResourceArn: {"Ref" : "AWSEBV2LoadBalancer" }
WebACLId:
Fn::GetOptionSetting:
OptionName: waf_id
Where Ref returns the Amazon Resource Name (ARN) of the load balancer.
I imagine you could place your Cloudformation for the VPC Link in a .config file for your Elastic Beanstalk App. Fair warning I used YAML for my config, but had to use the JSON format on the Reference function to get it work in my Environment.
ResourceArn: {"Ref" : "AWSEBV2LoadBalancer" }
I have an elastic beanstalk running with the default instance profile aws-elasticbeanstalk-ec2-role. The application uses private SSH keys to access external resources during installation, which means I am using an .ebextensions config file to copy stuff from S3.
However, for various reasons I need to change the instance profile, but I have found this surprisingly difficult, probably because my code has dependencies to the instance profile, and it seems like a configuration change through the AWS web console is only successful if the application is successfully deployed when the configuration change launches the new instance (changing instance profile results in a new instance being launched, then the old one is terminated).
I think the key issue is that my .ebextensions file references the instance profile (aws-elasticbeanstalk-ec2-role):
S3Auth:
buckets:
- <my-bucket>
roleName:
? "Fn::GetOptionSetting"
:
DefaultValue: aws-elasticbeanstalk-ec2-role
Namespace: "aws:asg:launchconfiguration"
OptionName: IamInstanceProfile
type: s3
If I change instance profile through the console, a new instance is launched and my application is deployed, which will fail, hence the config change is reverted.
If I first change my .ebextensions to use the new role, deploying that will fail since the referenced instance profile does not match what the EB currently has. And even if deploying first (which will of course fail) and then changing the configuration, it seems like the newly deployed (failed) version will not be used but instead EB will probably fall back to the previous version - so this seems like a catch 22 scenario.
The solutions I have found are
Create a new environment from scratch and specify instance profile from the start, using
$ eb create -ip <instance profile>
This works, but it may be highly undesirable having to terminate existing environments to release CNAMEs etc, to enable re-creating environments from scratch.
Deploy a dummy Hello World application without dependencies to the instance profile, then change instance profile in the web console, then deploy the real application making use of the new instance profile.
Now, my questions.
Since it seems like the problem is that the instance profile change and the application change need to be done atomically, is there a way to change the instance profile as part of an eb deploy command, e.g. by specifying the instance profile through .elasticbeanstalk/config.yml?
Is there a way to create an .ebextensions config file with a S3 auth rule that doesn't directly reference the instance profile to use (which has to match what the EB has anyway!!), but rather "use the current instance profile"? It seems a bit redundant that .ebextensions files must state what instance profile to authenticate with, when the only one that will ever be usable is the one that the EB actually holds?
Any other way around this problem?
I´m using AWS CLI and CloudFormation, and I could not find any reference in the documentation.
Does anybody know if it´s possible to create a CloudFormation template from a current configuration.
Let´s say that I want to get a CloudFormation template from my current security group configuration.
Any idea if it´s possible to export that configuration as a template using CLI?
Based on our experience we found 3 possible ways to translate existing manually deployed (from Web Console UI) AWS infra to Cloudformation (CF).
Using a new CloudFormation native introduced feature (since Nov 2019) that allows you to Import existing resources into a CloudFormation stack
Using aws cli execute $aws service_name_here describe for each element that make up your stack eg for RDS Database Stack:
RDS Instance -> Type: AWS::RDS::DBInstance,
RDS (EC2) SG -> Type: AWS::EC2::SecurityGroup,
RDS Subnet Group -> Type: AWS::RDS::DBSubnetGroup and
RDS DB Param Group -> Type: AWS::RDS::DBParameterGroup
And manually translate to CF based on the outputs obtained from the aws cli for each of the components. This approach usually requires more experience in both AWS and CF but the templates that you are creating can be structured and designed under good practices, fully parameterized (Sub, Ref, Join, Fn::GetAtt:, Fn::ImportValue), modular, applying conditions and in a 1st iteration the result would probably be close to the final state of the templates (interesting reference examples: https://github.com/widdix/aws-cf-templates/).
Extra points! :)
Some other new alternatives to export your current deployed AWS infra to Cloudformation / Terraform code:
https://former2.com
https://modules.tf
https://www.brainboard.co/
Related Article: https://medium.com/#exequiel.barrirero/aws-export-configuration-as-code-cloudformation-terraform-b1bca8949bca
It's not possible using the AWS CLI but you can use the CloudFormer [1] tool to create a CloudFormation template from existing resources. I've had decent success with it. The templates aren't as "pretty" as hand-made templates but they provide a good starting point.
[1] http://aws.amazon.com/developertools/6460180344805680
In addition to CloudFormer, you might want to take a look at Bellerophon: https://github.com/arminhammer/bellerophon.
I had some problems getting the tradidtional tools - mentioned above - working in our environment; we have a complicated API Gateway. Former2 didnt' find it at all (although seemed ideal for other resources)
I found another tool, "Terraformer" which extracts AWS into Terraform, which can then be turned into CloudFormation -or used directly as IaC.
https://github.com/GoogleCloudPlatform/terraformer#installation
Maybe that will work for others if the above tools don't.
I deployed a template with an s3 bucket. I noticed that the 'Logical ID' field is the resource name, but the 'Physical ID' field is in the format of --. Is there a way to override this behavior? I need to be able to specify an EXACT name. Everywhere Ive looked suggests that I make it through the console, but that's useless to me. I need automation.
Can anyone help me?
I have been coming up against the same issue. I know it is possible to set the physical name of a resource when you create it using the Ruby SDK for AWS. We use this ability so we can tear down/create ELBs with known and stable physical names, based on the environment and server role, and the backend nodes can auto-join upon deployment by looking for the ELB corresponding to their own environment and role.
However, when using CloudFormation, certain resources (ELBs, SGs) take the resource name you specify and prefix or suffix it with things like the stackID etc. Which breaks the above system by having ELB names which are always different after each deployment.
The CloudFormation documentation is by no means bad, but it does seem to lack a lot of edge cases. I haven't yet found out if setting the physical name is possible or not.
The AWS SDK for Ruby is very easy to use - I have no prior experience of Ruby - but have found it straightforward to deploy/destroy resources, so you may find this an acceptable solution to your issue if you have not had any CloudFormation joy by now but, like me, require an automated route. In fact you can use the Ruby SDK to trigger the CloudFormation stack for the items whose names you can control, and then create the others fully under the Ruby SDK.
Yes, it is possible to set a custom physical id for a few kinds of ressources. ELB is one of them. ATM (Dec/2014) this ressources support custom pysical ids:
AWS::CloudWatch::Alarm
AWS::DynamoDB::Table
AWS::ElasticBeanstalk::Application
AWS::ElasticBeanstalk::Environment
AWS::ElasticLoadBalancing::LoadBalancer
AWS::ElastiCache::CacheCluster
AWS::RDS::DBInstance
AWS::S3::Bucket
AWS::SNS::Topic
AWS::SQS::Queue
For more Infos see http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-name.html
To set a custom name for a S3-Bucket set the property BucketName (http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket.html#cfn-s3-bucket-name)