The Elastic Beanstalk environment is running on t1.micro & db.t1.micro, where we tried to update the rds to db.m1.small instance. This configuration change has been made on the Elastic Beanstalk and it is on updating state for around 4 hours, we thought AWS stuck somewhere internally. In this meantime, the AWS has not updated the Trace information. In order to retain the environment back (rollback of configuration update operation), we have aborted the environment configuration update. This has ended up on invalid state with the health turned to grey. This because AWS will not allow creating t1 environment nowadays, however, the rollback is trying to bring back the older t1 setup with an error--> RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.t1.micro, Engine=mysql, EngineVersion=5.6.37, LicenseModel=general-public-license. For supported combinations of instance class and database engine version.
Any suggestion to bring back the environment to green would be really helpful.
Yes, with the source of the snapshot we could able to create a new EBS instance. The only problem we faced during the snapshot to clone an instance is the application level configuration file change not done. BTW there is no proper root cause for the AWS DB upgrade failure from AWS.
Related
I need to install a gRPC PHP extension on my elastic beanstalk created EC2 instances. I have auto-scaling enabled, and when a new EC2 instance is kicked in, I lose all my installations.
From the documentation, I found two ways to fix this:
Create an instance and download everything required and take an image of that instance. And add the image id (AMI ID) in the Elastic Beanstalk environment (Under Configuration -> Instances). And every new instance created by auto-scaling will be from the image I provide. This approach never worked for me. Am I missing something here?
Write a config file in the .ebextensions to automatically install all the required extensions whenever a new instance is kicked in. And for this, we need to create a yaml/json file as per the documentation in cloud.google.com/php/grpc.
Can someone guide which approach should be taken? And help me create yaml/json file to automate the process for all the instances in auto scaling?
As per the AWS documentation here, to customize your Elastic Beanstalk environment you should use .ebextensions configuration files.
Creating .ebextensions provides the ability to completely customize the instances and environment that your application is running on/in, and makes upgrades, changes and/or additions to your instances and environment straightforward and efficient.
As a sidenote, ssh’ing to ElasticBeanstalk instances, and making on-instance changes, should be avoided. The autoscaling issue you are facing is one reason, however the other major reason is that making changes on the instance itself will cause the instances state to be out of sync with the EB state is expecting. If the state is out of sync, subsequent deployments will fail because the application version EB is expecting has drifted. Managing your application and environment through code and .ebextensions eliminates this issue.
I know this has been partially answered in a bunch of places, but the answers are so.. all over the map, dated and not well explained. I'm looking the best practice as of February 2016.
The setup:
A PHP-based RESTful application service that lives in an EC2 instance. The EC2 instance uses S3 for uploaded user data (image files), and RDS MySql for its DB (these two points aren't particularly important.)
We develop in PHPStorm, and our source control is GitHub. When we deploy, we just use PHPStorm's built-in SFTP deployment to upload files directly to the EC2 instance (we have one instance for our Staging environment, and another for our Production environment). I deploy to Staging very often. Could be 20 times a day. I just click on a file in PHPStorm and say 'deploy to Staging', which does the SFTP transfer. Or, I might just click on the entire project and click 'deploy to Staging' - certain folders and files are excluded from the upload, which is part of PHPStorm's deployment configuration.
Recently, I put our EC2 instance behind a Load Balancer. I did this so that I can take advantage of Amazon's free SSL offering via the Certificate Manager, which does not support individual EC2 instances.
So, right now, there's a Load Balancer with only a single EC2 instance behind it. I maintain an Elastic IP pointing to the EC2 instance so that I can access it directly (see my current deployment method above).
Question:
I have not yet had the guts to create additional (clone) EC2 instances behind my Load Balancer, because I'm not sure how I should be deploying to them. A few ideas came to mind, but they're all pretty hacky.
Given the scenario above, what is currently the smoothest and best way to A) quickly deploy a codebase to a set of EC2 instances behind a Load Balancer, and B) actually 'clone' my current EC2 instance to create additional instances.
I haven't been able to really paint a clear picture of the above in my head yet, despite the fact that I've gone over a few (highly technical) suggestions.
Thanks!
You need to treat your EC2 instance as 100% dispensable. Meaning, that it can be terminated at any time and you should not care. A replacement EC2 instance would start and take over the work.
There are 3 ways to accomplish this:
Method 1: Each deployment creates a new AMI image.
When you deploy you app, you deploy it to a worker EC2 instance whose sole purpose is for "setup" of your app. Once the new version is deployed, you create a fresh AMI image from the EC2 instance and update your Auto Scaling launch configuration with the new AMI image. The old EC2 instances are terminated and replaced with the new code.
New EC2 instances have the recent code already on them so they're ready to be added to the load balancer.
Method 2: Each deployment is done to off-instance storage (like Amazon S3).
The EC2 instances will download the recent code from Amazon S3 and install it on boot.
So to put the new code in action, you terminate the old instances and new ones are launched to replace them which start using the new code.
This could be done in a rolling-update fashion, or as a blue/green deployment.
Method 3: Similar to method 2, but this time the instances have some smarts and can be signaled to download and install the code.
This way, you don't need to terminate instances: the existing instances are told to update from S3 and they do so on their own.
Some tools that may help include:
Chef
Ansible
CloudFormation
Update:
Methods 2 & 3 both start with a "basic" AMI which is configured to pull the webpage assets from S3. This AMI is not changed from version-to-version of your website.
For example, the AMI can have Apache and PHP already installed and on boot it pulls the .php website assets from S3 and puts them in /var/www/html.
CloudFormation works well for this. In addition, for method 3, you can use cfn-hup to wait for update signals. When signaled, it'll pull updated assets from S3.
Another possibility is using Elastic Beanstalk which could be used to manage all of this for you.
Update:
To have your AMI image pull from Git, try the following:
Setup an EC2 instance with everything installed that you need to have installed for your web app
Install Git and setup a local repo ready to Git pull.
Shutdown and create an AMI of your instance.
When you deploy, you do the following:
Git push to GitHub
Launch a new EC2 instance, based on your AMI image.
As part of the User Data (specified during the EC2 instance launch), specify something like the following:
#!/bin/sh
cd /git/app
git pull
; copy files from repo to web folder
composer install
When done like this, that user data acts as a script which will run on first boot.
We needed to change the DB tier, but when doing it via the EB console it gave an odd error message...
Updating RDS database named: ******** failed Reason: Cannot upgrade mysql from 5.6.22 to 5.5.41
However, I managed successfully to change the DB tier via the RDS console instead.
The problem is, is that the EB console still shows the old DB tier, whilst RDS now shows it on the new one!
It has upgraded, but how do I get the EB console to show the correct information?
Currently you can only specify the DB version on environment launch in elastic beanstalk. You may do this via the console or using the DBEngineVersion option setting. You cannot update it on an existing environment.
Since you changed the value out-of-band on the RDS directly, Elastic Beanstalk still thinks that the value hasn't changed. Hence it will continue to show you the old version. Your environment should continue to function correctly though. Let me know if you are facing any issues.
One caveat with making the change out-of-band is that if you save the environment configuration, clone or rebuild the environment you will get the old database version again on the new environment. So your change is not persisted across these operations.
I am new to AWS EB and I am trying to figure out how to backup and restore an entire EB environment. I created an AMI based on the EC2 instance generated by EB, and took a snapshot of RDS, also created by EB.
The problem I have is, how do I restore it, assuming that this is the correct approach of backup. Also, I am doing it manually, shouldn't there be an automated way of doing this within EB? By the way, when I created the AMI, it destroyed the source and the EB just created a new EC2 instance without all my changes.
How do I save & restore configuration changes to my application that impact both filesystem and database?
Unfortunately, Amazon AWS Elastic Beanstalk (EB) does not support restoring databases that contain live data, if those databases were created with EB. If you reload (AKA AWS "deploy") the EB saved configuration, you get a blank database!
I called them and they told me to create the RDS DB separately and update the application code to connect to the DB once you know it's name. If you restore the RDS DB it will have a new name too! So you have to update your code again to connect to it.
Also, if you code and environment is fine, but you want to restore your database, again it will have a new name and you will need to change your code.
How to change your code easily and automatically deploy it is a whole other question for which I don't have an answer yet.
So basically the RDS DB provisioning within Elastic Beanstalk has very limited uses, maybe coding and debugging and testing, but not live production use. :(
This is as of Jan 2015.
First go into your EB environment and save the current config. You should go to a running EC2 instance created by EB and make an Image. Then use that new AMI ID by going to the EB configuration and setting it. It will rebuild the environment tearing down all running instances and creating new ones.
For your RDS instance you should make a backup and restore with a new instance name as the docs say you will lose it if the environment is destroyed. You should probably just manually set the environment variables like RDS does and setup the proper security groups between RDS and EC2.
One option I think could work is just renaming the RDS instance name as the environment seems to break and then destroy the environment and create a new one with an attached RDS instance and then destroy that one and rename the old one to the new one's name which may work.
As always make proper backups before proceeding with any of these ideas.
I created AWS EB environments with RDS MySQL DB 1st time. But, I realized that I don't need to use MySQL so, I switched to sqlite3 and terminate RDS MySQL DB. After this, I can't update application version by using git aws.push. AWS EB shows the below error message all the time.
Service:AmazonCloudFormation, Message:Stack named 'awseb-e-xxxxxx-stack' aborted operation.
Current state: 'UPDATE_ROLLBACK_IN_PROGRESS' Reason: The following resource(s) failed to update: [AWSEBAutoScalingGroup].
Updating Auto Scaling group failed Reason: Template error: DBInstance xxxxxxx doesn't exist
How can I fix this issue? Thanks.
As some have noted, you actually cannot manually delete your RDS instance from an elasticbeanstalk environment.
Per another answer you can change this via some options using one of the aws apis:AWS Elastic Beanstalk change RDS Endpoint
My solution was to terminate the environment and just start a new one from a saved configuration.
I was unable to terminate an EB environment of mine due to previously having manually deleted the RDS instance associated with the environment.
I was able to solve this issue by creating a new RDS instance with the same identifier as the deleted instance, then terminating the environment like normal.
If you deleted your RDS instance manually, Elastic BeansTalk does not know about that change and must be updated too.
You can go to Elastic Beanstalk console, pick up your Application and your Environment, go to "Configuration" and delete your RDS instance from the Elastic Beanstalk configuration.