We're using IronWorkers from http://www.iron.io/ to do some heavy image and PDF processing.
I want to connect an IronWorker instance to a RDS MySQL database on our Amazon AWS account, so that the our code running on the IronWorker can directly make chances in this database.
I'm not too sure how to go about this, as we have a few technical issues to work around.
My understanding is that IronWorkers that get launched won't be in the RDS instance's security group, and would be blocked from accessing our RDS MySQL instance. It won't be possible to create a security group, by creating an CIDR/IP entry either, since we don't know what the IP address for the launched server is going to be.
Another approach would be to somehow get the .pem file on the launched instance, and configure the MySQL connection to use a PEM file through SSH, but I'm not too convinced that it's the most secure way to go about achieving a connection.
Does anyone know of any means a direct connection can be made from an IronWorker instance to a MySQL RDS instance?
There's a tricky way to get around this by finding the internal IP of your RDS instance then using that instead of the dns entry AWS gives you, example:
$ ping myserverabcdefg.us-east-1.rds.amazonaws.com
PING ec2-X-XX-XXX-X.compute-1.amazonaws.com (10.111.222.33) 56(84) bytes of data.
Then add a dns entry using that internal host and and use the new dns entry in your workers.
Let us know how that works.
Related
I lost my PEM and had to create a new EC2. I duplicated it with "make same". I then tried to connect to the RDS database and it gives me cannot connect with user#. now, the IP address of the server has changed, but it is in the same security group, so shouldn't matter.
I changed the user to be from anywhere % and flushed privileges. I even rebooted the DB. I've tried messing with the security stuff, but everything is as it should be, the same as it was before hand when it was working on the previous server.
either something was not "duplicated" when the EC2 was duplicated or there is something somewhere I have to change because the IP of the EC2 changed.
if they are in the same security group, this shouldn't be an issue however.
any help is appreciated. as I can no longer connect to the DB with the app.
the EC2 is linux running a docker container for node.
I can still connect to the RDS via local machine.
So i still don't know what the issue was, but I was able to get around it by creating a new user. I'm thinking that the host wasn't actually changing when I was changing it in RDS - even with a reboot.
If somebody comes across this same issue, try creating a new MySQL user.
I'm a newer AWS user and today I got stuck while working on a sample project. I successfully created a docker container that runs a simple R script that connects to my AWS RDS MySQL Database and creates & writes some basic files to it. I built a public ECR repository, pushed my docker image there, and built a ECS cluster & task choosing Fargate and using the container image from my repository. My task ran and I could see the R code being executed when I went through the logs, but it was never able to connect to the SQL Database and exited afterwards.
I've had to whitelist my own IP address in the security group for the RDS Database so that I can connect to it, so I'm aware I probably have to do that for my ECS task to establish that connection too. But won't that IP address constantly change because I won't have a static IP for the Fargate Server that is executing my task? I'm trying to stay on the free tier so I'm not sure I want to setup an elastic IP address for this server.
These 2 articles seem close if not the same issue I'm having but I can't figure out a solution. I haven't found any other info.
https://aws.amazon.com/premiumsupport/knowledge-center/ecs-fargate-task-database-connection/
https://aws.amazon.com/premiumsupport/knowledge-center/ecs-fargate-static-elastic-ip-address/
The end goal is to get this sample project successfully running on a scheduled fixed interval, and then running actual scripts on there to help automate things and make my life easier, so this sample project is a first step towards that. Any help or info on the questions I'm having would be appreciated !
Yes, your task is ephemeral (whether you launch it manually or as part of an ECS service) and its private/public ip address may change over time if it gets replaced. The way you'd make the connectivity rules to stick is to assign a security group to the task (that may have inbound access on a specific port you need I assume and outbound to everything) and assign another security group to the RDS db that has inbound access on port 3306 for the security group you assigned to the task (this is the trick, the SG will not change and you are telling RDS to allow access to ALL traffic coming from that SG). I see the first article you posted doesn't talk about this part (it should).
I am trying to set up a bastion host in AWS in order to perform administrative options on an RDS instance in a private subnet. I am following the instructions from the official documentation (https://docs.aws.amazon.com/quickstart/latest/linux-bastion/step1.html), but there it is not clear how the bastion will know which RDS instance to connect to. How would I make sure that it can 'talk to' my intended RDS? (as far as I understand, the key pair is just something I can create anytime and enter to connect to the bastion itself, but not the RDS, or am I wrong?)
The documentation you linked uses an AWS CloudFormation stack to deploy the Bastion. I'm not sure exactly what configuration it is using, so my answer will be generic, rather than applying to this specific situation.
The normal configuration is:
A database in a private subnet
A Bastion server (EC2 instance) in a public subnet
A connection is made to the Bastion, which then allows an on-connection to the database
There are a number of ways of connecting to the database through the Bastion. Here's one that I use:
ssh -i key.pem ec2-user#BASTION-IP -L 3306:DATABASE-DNS-NAME:3306
This tells the SSH connection to forward any traffic sent to my local port 3306 (the first number), through the SSH connection, but then send it to DATABASE-DNS-NAME:3306 (the database server). Any response from the database will come back the same way.
Then, when I wish to refer to the database from my computer, I reference:
localhost:3306
It appears that the database is on my own computer, but the traffic is actually sent across SSH to the Bastion, then onto the database.
There are newer and better ways of doing this forwarding that other people might (hopefully) add as a comment or another answer, but this is the way I make my connections through a Bastion.
Fun fact: A Bastion is the bit of a castle wall that sticks out, allowing defenders to shoot arrows at attackers attempting to climb the wall. In a similar way, the Bastion Server sticks out into the Internet, beyond the protected part of the network.
On AWS, I know how to set up a web server with inbound rules allowing HTTP and HTTPS and a database security group that only connect to the web server. The issue is I need to create a front end to manage the databases without using Internet access - this will be internal only and precludes the use of a public IP / public DNS. Does anyone know how I would do this?
To further elaborate, some of our AWS accounts are for internal use only - we can log in to the console, use CygWin to SSH in, see what's there, etc. But these accounts are for development purposes, and in a large enterprise such as this one, these are not allowed an IGW. So - no inbound Internet access is allowed. How do I create an app (e.g., phpMyAdmin type) in which our manager can easily view and edit the data in the database given the restriction that this must be done without inbound Internet access?
Host your database on RDS inside a VPC and create a VPN connection between your client network and your VPC.
host your database on one EC2 and also upload your front end there. your database will be running on locally on EC2 and you can connect front end to database. where database will not have public DNS it will running locally you can access only using SSH and front end script.
you check this official documentation from aws : https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_VPC.WorkingWithRDSInstanceinaVPC.html
for frontend script you can use https://www.adminer.org/ which is one file database management system. one simple file is there using this make connection to locally running database on EC2
I have a load balancer and EC2 instance with AWS. I had problems with e-mail restrictions and was recommended to use an elastic IP. I then read somethere that you can't use elastic IP and a load balancer so I removed the elastic IP. I can no longer access my instance even when I've rebooted it and waiting 2 hours later. I can ping it (after enabling ICMP with network security) but I can't SSH or go to the web server. All the network settings remain, which included allowing TCP ports for HTTP and SSH. Does anyone know what has happened to make port 80 and 23 no longer accessible? This is a real nightmare for me because I did a bit of a marketing campaign, got increased traffic, noticed emails weren't getting sent, then in an attempt to fix that I've screwed the server completely so the website is down at the worst possible time :(
I fixed it all up. This isn't a direct solution to the problem, more like a workaround. I couldn't connect to that server no matter what, so I created a new instance and that worked. It was as if the Linux server itself was corrupt, not the AWS settings. I detached the volume from the old instance and attached it as a secondary volume on the new instance. When I logged into the new instance I was able to mount the secondary volume as a new drive and I just copied the files over that way. I don't have a bloated server so this wasn't really a big deal to pull off. Anyway, if you can't log in to a server anymore, you can always mount it to a new instance and access it via the file-system