In trying to move a website to operate via Elastic Beanstalk (ELB), I chose the t2 series of EC2 instances, and in doing so, was forced to create a Virtual Private Cloud (VPC). This site connects to a MySQL database via RDS, and I'm not having any luck getting the ELB site to access the database.
I've tried reviewing this:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo.RDS.html?icmpid=docs_elasticbeanstalk_console
The above link starts by saying "works great for development and testing environments, but is not ideal for a production environment", which confuses me as it doesn't say what would be better in its place - I need a database connected to the site!
It has all sorts of information and I tried several of the things it suggested regarding connecting to an existing database (not creating a new one). It mentions on step 6 of the "To modify the ingress rules on your RDS instance's security group" section to access the ingress tab, which doesn't exist for me.
I've tried editing the security group associated with the database via the RDS dashboard under "security groups", but it does not list the security groups that are associated with the VPC or the EC2 instance launched by ELB. I tried pushing the IP addresses, elastic IPs, and still can't get the site to see the database.
I'm at a loss. Can anyone explain how to connect an ELB distributed EC2 instance with an RDS database through the VPC required by t2 instances?
The statement that "This works great for development and testing environments, but is not ideal for a production environment" is just referring to having ElasticBeanstalk create the RDS instance for you. This can be done by configuring the "Database" section when creating a new EB environment.
The downside of letting EB create the RDS instance for you is that your web instance and database instance will be strongly connected, and if you ever terminate your web instance, your database will also be terminated, including all of your snapshots.
However, I think you're taking the "external" part of "external database" too literally. Your RDS instance should definitely be within the same VPC as your web instance. However, you should create it and connect your web instance to it manually. Connecting to the database involves setting five environment variables (listed below) and configuring the security group to allow connections from the web instance to the database.
The environment variables you'll need to set on your web instance are as follows:
RDS_HOSTNAME=instancename.region.rds.amazonaws.com
RDS_DB_NAME=databasename
RDS_PASSWORD=databasepassword
RDS_USERNAME=databaseuser
RDS_PORT=5432
Related
I made and deployed my Django application in AWS Elastic Beanstalk. It has a connection to a Postgres DB in RDS, through the EBS console.
When I click configuration -> network in EBS, I see: "This environment is not part of a VPC."
How can I make it a part of a VPC? Thanks!
NOTE: very new to this ;)
You have to recreate the Elastic Beanstalk environment and pick the VPC during the creation. It is not possible to move an existing environment into a VPC.
But, unless you have access to EC2-Classic, the EC2 servers that were launched are already be in a VPC. They are just in the default VPC. But as far as Elastic Beanstalk is concerned, it seems oblivious to this.
I am not sure if there are any features that are exclusively available to VPC environments. My suggestion is to try to use your current environment, and if you happen to recreate the environment later for some other reason, then you can try picking a VPC and see if it offers anything new.
As already explained by #stefansundin you can't move existing EB into a custom VPC. You have to create new one.
These are general steps to consider:
Create a custom VPC with public and private subnets as described in the docs: VPC with public and private subnets (NAT). NAT is needed for instances and rds in private subnet to communicate with internet, but no inbound internet traffic will be allowed. This ensures that your instances and rds are not accessible from the outside.
Create new RDS, external to EB. This is good practice as otherwise the lifetime of your RDS is coupled with EB environment. Starting point is the following AWS documentation: Launching and connecting to an external Amazon RDS instance in a default VPC
Create new EB environment and ensure to customize its settings to use the VPC. Pass the RDS endpoint to the EB instances using environmental variables. Depending on how you want to handle password to the RDS, there are few options, starting from using environmental variables (low security) through SSM Parameter Store (free) or AWS Secrets Manager (not free).
Setting this all up correctly can be difficult for someone new to AWS. But with patience and practice it can be done. Thus, I would recommend with starting with default VPC, as you have now. Then once you are comfortable with how to work with external RDS, think on creating custom VPC as described.
If one has a publicly accessible rds database on aws, and wants to instead use a bastion ec2 instance to access and perform database functions (anyone on the internet should be able to use the app and perform database functions in accordance with the features provided by the app), how should one go about performing this shift? I have tried searching the internet but often I get loads of information with terminology that isn't entirely easy to digest. Any assistance would be greatly appreciated.
Again, I want the general public to be able to use and access the app's provided db functions, but not have them be able to access the database directly.
A typical 3-tier architecture is:
A Load Balancer across public subnets, which sends traffic to...
Multiple Amazon EC2 instances in private subnets, preferable provisioned through Amazon EC2 Auto Scaling, which can scale based on demand and can also replace failed instances, which are all talking to...
A Database in a private subnet, preferably in Multi-AZ mode, which means that a failure in the database or in an Availability Zone will not lose any data
However, your application may not require this much infrastructure. For low-usage applications, you could just use:
An Amazon EC2 instance as your application server running in a public subnet
An Amazon RDS database in a private subnet, with a security group configured to permit access from the Amazon EC2 instance
Users would connect to your application server. The application server would connect to the database. Users would have no direct access to the database.
However, YOU might require access to the database for administration and testing purposes. Since the database is in a private subnet, it is not reachable from the Internet. To provide you with access, you could launch another Amazon EC2 instance in a public subnet, with a security group configured to permit you to access the instance. This instance "sticks out" on the Internet, and is thus called a Bastion server (named after the part of a castle wall that sticks out to allow archers to fire on invaders climbing the caste wall).
You can use port forwarding to connect to the Bastion server and then through to the database. For example:
ssh -i key.pem ec2-user#BASTION-IP -L 3306:DATABASE-DNS-NAME:3306
This configures the SSH connection to forward localhost:3306 to port 3306 on the named database server. This allows your local machine to talk to the database via the Bastion server.
You will need to create private subnets for this and update DBsubnet groups accordingly with private subnets only. Moreover in DB security group add bastion and app instances security group as source for db port.
Like if you're using mysql engine, allow 3306 for target instances secuirty group id's.
Before moving to Amazon Web Services, I was using Google Cloud Platform to develop my aplication, CloudSQL to be specific, and GCP have something called Cloud SQL Proxy that allows me to connect to my CloudSQL instance using my computer, instead of having to deploy my code to the server and then test it. How can I make the same thing using AWS?
I have a python environment on Elastic Beanstalk, that uses Amazon RDS.
AWS is deny be default so you cannot access an RDS instance outside of the VPC that your application is running in. With that being said... you can connect to the RDS instance via a VPN that can be stood up in EC2 that has rules open to the RDS instance. This would allow you to connect to the VPN on whatever developer machine and then access the RDS instance as if your dev box was in the VPC. This is my preferred method because it is more secure. Only those with access to the VPN have access to the RDS instance. This has worked well for me in a production sense.
The VPN provider that I use is https://aws.amazon.com/marketplace/pp/OpenVPN-Inc-OpenVPN-Access-Server/B00MI40CAE
Alternatively you could open up a hole in your VPC to the RDS instance and make it publicly available. I don't recommend this however because it will leave your RDS instance open to attack as it is publicly exposed.
You can expose your AWS RDS to the internet by proper VPC setting, I did it before.
But it has some risks
So usually you can use those ways to figure it out:
Create a local database server and restore snapshot from your AWS RDS
or use VPN to connect to your private subnet which hold your RDS
A couple people have suggested putting your RDS instance in a public subnet, and allowing access from the internet.
This is generally considered to be a bad idea, and should be the last resort.
So you have a couple of options for getting access to RDS in a private subnet.
The first option is to set up networking between your local network and your AWS VPC. You can do this with Direct Connect, or with a point-point VPN. But based on your question, this isn't something you feel comfortable with.
The second option is to set up a bastion server in the public subnet, and use ssh port forwarding to get local access to the RDS over the SSH tunnel.
You don't say if you on linux or Windows, but this can be accomplished on either OS.
What I did to solve was:
Go to Elastic Beanstalk console
Chose you aplication
Go to Configurations
Click on the endpoint of your database in Databases
Click on the identifier of your DB Instance
In security group rules click in the security groups
Click in the inbound tab
Click edit
Change type to All Traffic and source to Anywhere
Save
This way you can expose the RDS connected to your Elastic Beanstalk aplication to the internet, which is not recommended as people sugested, but it is what I was looking for.
When you set up a new Elastic Beanstalk cluster you can access your EC2 instance by doing this:
eb ssh
However, it's not clear how to access the RDS instance.
How do you access an RDS in an Elastic Beanstalk context in order to perform CRUD operations?
The RDS command-line can be accessed from anywhere, by adjusting the RDS security group.
Check your AWS VPC configuration.
The security-group will need to be
adjusted to allow you to connect from a new source/port.
Find the security Group-id for the RDS.
Find that group in AWS Console > VPC > secuirty groups
Adjust the Inbound and Outbound Rules accordingly.
You need to allow access to/from the IP or security group that needs to connect to the RDS.
FROM: https://stackoverflow.com/a/37200075/1589379
After that, all that remains is configuring whatever local DB tool you would like to use to operate on the database.
EDIT:
Of additional note, if the ElasticBeanstalk Environment is configured to use RDS, the EC2 Instances will have environment variables set with the information needed to connect to the RDS.
This means that you can import those variables into any code that needs access.
Custom environment variables may also be set in Elastic Beanstalk Environment Configuration, and these too may be included this way.
PHP
define('RDS_HOSTNAME', getenv('RDS_HOSTNAME'));
$db = new rds(RDS_HOSTNAME);
Linux CommandLine
mysql --host=$RDS_HOSTNAME --port=$RDS_PORT -u $RDS_USERNAME -p$RDS_PASSWORD
RDS is a managed database service, which means it is that you can only access it through database calls.
If it is a MySQL database you can access through your EC2 instance through mysql like this:
mysql -u user -p password -h rds.instance.endpoint.region.rds.amazonaws.com
or set it up to work with your app with settings needed for that.
Make sure that you set up security groups correctly so that your EC2/other service has access to your RDS instance.
Update:
If you want what you are asking for then you should use an EC2 instance with a mysql server on. It would cost the same (even though a fraction of performance is lost in comparison). An EC2 instance you can turn off when you are not using as well.
When creating an application in Amazon Elastic Beanstalk, you have the option of creating a new Amazon RDS database instance. Is it possible to associate an existing RDS database instance with an Elastic Beanstalk application?
If you want to share an RDS instance between multiple applications, the best thing to do is set that up independent of your beanstalk application.
Set up privs for each application and configure each application to use the RDS instance.
You will probably have to manually configure your application security group to access the database instance.
I was facing same issue and fixed using following steps :
1) Go to EC2 instance and note your security group example "sg-121212121212"
2) Go to RDS Security Group ad=nd inbound traffic
3) Edit rule select all traffic and add your new ebs security group "sg-121212121212"
Hope it will helpful
A single database can only be used with one Elastic Beanstalk environment. Although, it can be moved between environments.
Also important to remember that a clone operation doesn't clone the database.