AWS Elastic Beanstalk: terminate RDS then fail to update application version - amazon-web-services

I created AWS EB environments with RDS MySQL DB 1st time. But, I realized that I don't need to use MySQL so, I switched to sqlite3 and terminate RDS MySQL DB. After this, I can't update application version by using git aws.push. AWS EB shows the below error message all the time.
Service:AmazonCloudFormation, Message:Stack named 'awseb-e-xxxxxx-stack' aborted operation.
Current state: 'UPDATE_ROLLBACK_IN_PROGRESS' Reason: The following resource(s) failed to update: [AWSEBAutoScalingGroup].
Updating Auto Scaling group failed Reason: Template error: DBInstance xxxxxxx doesn't exist
How can I fix this issue? Thanks.

As some have noted, you actually cannot manually delete your RDS instance from an elasticbeanstalk environment.
Per another answer you can change this via some options using one of the aws apis:AWS Elastic Beanstalk change RDS Endpoint
My solution was to terminate the environment and just start a new one from a saved configuration.

I was unable to terminate an EB environment of mine due to previously having manually deleted the RDS instance associated with the environment.
I was able to solve this issue by creating a new RDS instance with the same identifier as the deleted instance, then terminating the environment like normal.

If you deleted your RDS instance manually, Elastic BeansTalk does not know about that change and must be updated too.
You can go to Elastic Beanstalk console, pick up your Application and your Environment, go to "Configuration" and delete your RDS instance from the Elastic Beanstalk configuration.

Related

Elastic Beanstalk - Delete environment failed

In AWS Elastic Beanstalk, I created an environment and through it a database.
I went to RDS and renamed the instance name and then delete the RDS instance
Now I am trying to delete the environment and the procedure fails:
ERROR Stack deletion failed: The following resource(s) failed to delete: [AWSEBRDSDatabase].
What can I do in order to detach the environment from the RDS?
Any ideas?
Thank you very much
#Andreas What you can do is rename the RDS instance back to the original one and takes a Snapshot of it.
if you don't rename and try to delete the beanstalk what happens is beanstalk will keep on looking for the RDS Instance.
Now you can delete the Beanstalk, after successful deletion of the environment you can again create the RDS Instance via that Snapshot (which you have created previously)
What else you can try is go to the Configuration tab of the beanstalk and change the configuration of Database and update the beanstalk. This will preserve your beanstalk env also and changes the RDS configuration also.
For me helped to going to Cloud Formation and deleting stack there
https://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-deleted-rds-database/

Elastic Beanstalk went to invalid state with t1

The Elastic Beanstalk environment is running on t1.micro & db.t1.micro, where we tried to update the rds to db.m1.small instance. This configuration change has been made on the Elastic Beanstalk and it is on updating state for around 4 hours, we thought AWS stuck somewhere internally. In this meantime, the AWS has not updated the Trace information. In order to retain the environment back (rollback of configuration update operation), we have aborted the environment configuration update. This has ended up on invalid state with the health turned to grey. This because AWS will not allow creating t1 environment nowadays, however, the rollback is trying to bring back the older t1 setup with an error--> RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.t1.micro, Engine=mysql, EngineVersion=5.6.37, LicenseModel=general-public-license. For supported combinations of instance class and database engine version.
Any suggestion to bring back the environment to green would be really helpful.
Yes, with the source of the snapshot we could able to create a new EBS instance. The only problem we faced during the snapshot to clone an instance is the application level configuration file change not done. BTW there is no proper root cause for the AWS DB upgrade failure from AWS.

Change AWS RDS Tier In Elastic Beanstalk

We needed to change the DB tier, but when doing it via the EB console it gave an odd error message...
Updating RDS database named: ******** failed Reason: Cannot upgrade mysql from 5.6.22 to 5.5.41
However, I managed successfully to change the DB tier via the RDS console instead.
The problem is, is that the EB console still shows the old DB tier, whilst RDS now shows it on the new one!
It has upgraded, but how do I get the EB console to show the correct information?
Currently you can only specify the DB version on environment launch in elastic beanstalk. You may do this via the console or using the DBEngineVersion option setting. You cannot update it on an existing environment.
Since you changed the value out-of-band on the RDS directly, Elastic Beanstalk still thinks that the value hasn't changed. Hence it will continue to show you the old version. Your environment should continue to function correctly though. Let me know if you are facing any issues.
One caveat with making the change out-of-band is that if you save the environment configuration, clone or rebuild the environment you will get the old database version again on the new environment. So your change is not persisted across these operations.

Amazon Web Service can't delete an Elastic Beanstalk environment

I have a problem with AWS Elastic Beanstalk. I tried to delete an environment. It started the process, but after a few minutes the environment "health bar" went to grey and gave me the following errors:
"Deleting security group named: XXXXX failed Reason: resource YYYYY has a dependent object"
"Stack deletion failed: The following resource(s) failed to delete: [AWSEBSecurityGroup]."
I tried to delete the security group from the error message, but I got this:
"XXXXXX: resource XXXXX has a dependent object"
After this I wanted to delete the dependence from the EB environment, but because it's Grey, it didn't allow me to do that.
I browsed the internet for hours, found a possible solution, where I need to do something at the EC2's Network Interfaces page, but it doesn't say any specific option or information.
Try this, I was suggested by AWS support and it worked for me.
You need to go to your CloudFormation console and retry deletion of the CloudFormation stack which the Beanstalk environment used.
The deletion may fail, but after retrying it will prompt you if you want to skip the "AWSEBRDSDatabase" resource that failed to delete. You can just confirm that you want to skip deletion (since you have actually already deleted it).
This should remove the CloudFormation stack
Then you can retry deletion of the Beanstalk environment from the Beanstalk console.
Is the security group being referenced by RDS or something in S3? If that is the case, you'll have to delete the dependency in either RDS or S3.
The error message is saying something outside of your environment is still using the security group and it can't be deleted for this reason.
Go to EC2 under AWS console's Compute
Go to Security Groups under NETWORK & SECURITY on the sidebar
Find your misbehaving security group on the page
Check it and choose Delete Security Group from the Actions menu
You will be presented with a link that will lead you to the security group or instances it depends on.
Keep going until you get to the parent Security Group or instances and delete them.
Make sure you don't delete any important Security Groups or Instances!
In case this happens due to a similar error but due to RDS attached to this,
Stack deletion failed: The following resource(s) failed to delete: [AWSEBRDSDatabase].
This happens when you terminate the RDS instance manually from RDS listing console. I resolved this by launching another instance with the same DB instance ideIntifier name.
Once this is created, you can terminate the Elastic Beanstalk environment successfully. This works.
Use https://github.com/mingbowan/sgdeps to find your security group dependencies and then break the dependencies.
Had this happen where I was using the security group created by EB with a non-eb created RDS instance.
I modified the RDS instance to stop using the EB-created security group and was able to do a successful termination of the environment and application. I used the eb cli 3.x and eb terminate --all --force to get a fresh start on the application.
In my case i have white list EBS instance into RDS security group , so deleted from RDS solved problem.

AWS Elastic Beanstalk change RDS Endpoint

How do I change the configured RDS endpoint of an AWS Elastic Beanstalk environment?
E.g. after the RDS database was deleted or should be replaced with a new RDS database.
Update
The topic remains complex and the AWS Elastic Beanstalk (EB) documentation could still do a better job to clarify available options. The question has been about how to change an RDS endpoint, which seems to be read in two different ways:
One could interpret it about how to attach an existing externally managed RDS endpoint to an existing (not new!) EB environment - this is indeed not possible, rather one would need to resort to handling this scenario from within the app itself as e.g. outlined in section Using an Existing Amazon RDS DB Instance with Python within Using Amazon RDS with Python.
Rather, the OP asked about how to do that after the RDS database was deleted or should be replaced with a new RDS database, i.e. the RDS endpoint change is implied in the process of creating a new RDS database for an existing EB environment that already had one - this is indeed possible by means of the DBSnapshotIdentifier Option Value, which denotes The identifier for the DB snapshot to restore from. Once again the EB docs aren't exactly conclusive what this means, however, EB is using AWS CloudFormation under the hood, and the resp. entry for AWS::RDS::DBInstance - DBSnapshotIdentifier provides more details:
By specifying this property, you can create a DB instance from the
specified DB snapshot. If the DBSnapshotIdentifier property is an
empty string or the AWS::RDS::DBInstance declaration has no
DBSnapshotIdentifier property, the database is created as a new
database. If the property contains a value (other than empty string),
AWS CloudFormation creates a database from the specified snapshot. If
a snapshot with the specified name does not exist, the database
creation fails and the stack rolls back.
In other words, the typical result of updating any of the General Option Values from namespace aws:rds:dbinstance for an existing EB environment is the creation of a respectively adjusted RDS instance managed by EB, and thus a new RDS endpoint.
A specific sub scenario is the use of DBSnapshotIdentifier, which yields a new RDS instance managed by EB based on the referenced snapshot and can therefore be used to migrate (rather than attach) an existing externally managed RDS instance, albeit with considerable downtime based on the snapshot size.
Initial Answer
While unfortunately not specifically addressed within Configuring Databases with AWS Elastic Beanstalk, the AWS Elastic Beanstalk settings for an optional Amazon RDS database are handled via Option Values, see namespace aws:rds:dbinstance within General Options.
While the AWS Management Console hides many of those option values behind its UI, you can specify them explicitly when using the API via other means, both when creating an environment as well as when updating one (which is how you would change any settings of an RDS database instance) - see e.g. parameter --option-settings for update-environment from the the AWS Command Line Interface:
If specified, AWS Elastic Beanstalk updates the configuration set associated with the running environment and sets the specified configuration options to the requested value.
I created a config file under .ebextensions folder that had the following content:
option_settings:
- namespace: aws:rds:dbinstance
option_name: DBSnapshotIdentifier
value: <name-of-snapshot>
Upload and deploy and it will create a new RDS db using this snapshot.
Hot-swapping out the data tier within an environment is discouraged because it breaks down the integrity of the environment. What you want to do is clone the environment, with a restored snapshot of the RDS instance. This means you'll have an identical environment with a different url 'host', and if everything went without a hitch, then you can swap environment urls in order to initiate a DNS swap.
After the swap happens and everything is good to go, you can proceed to deflate the old environment
Follow the steps in the resolution to:
Use an Elastic Beanstalk blue (environment A)/green (environment B) deployment to decouple an RDS DB instance from environment A.
Create a new Elastic Beanstalk environment (environment B) with the necessary information to connect to the RDS DB instance.
check out the official answer below for more detailed solution
https://aws.amazon.com/premiumsupport/knowledge-center/decouple-rds-from-beanstalk/?nc1=h_ls