I created an RDS instance (Postgres, free tier, in default VPC) and selected public access. Now, I am trying to connect to it in Python like this:
import psycopg2 as ps
# define credentials
credentials = {'POSTGRES_ADDRESS' : '', # change to your endpoint
'POSTGRES_PORT' : '', # change to your port
'POSTGRES_USERNAME' : '', # change to your username
'POSTGRES_PASSWORD' : '', # change to your password
'POSTGRES_DBNAME' : ''} # change to your db name
# create connection and cursor
conn = ps.connect(host=credentials['POSTGRES_ADDRESS'],
database=credentials['POSTGRES_DBNAME'],
user=credentials['POSTGRES_USERNAME'],
password=credentials['POSTGRES_PASSWORD'],
port=credentials['POSTGRES_PORT'])
cur = conn.cursor()
Here is the security group inbound:
However, this times out. What am I doing wrong? I believe I put in all the credentials above, so maybe it is a problem with the AWS side of things? Please let me know if you need any more info.
Thanks!
You will need to allow inbound access on the Security Group:
Port 5432
Source: Your IP address (for good security)
It is best to create a new Security Group with these settings then modify the database to use this new security group. That way, it will not impact other systems, and future changes will not impact this security group.
See also: Change security group on AWS RDS Database Instance
Based on the comments.
The issue turned out to be due to missing default database in the RDS.
Recreating the RDS with the default database solved the problem.
Related
I'm using AWS CDK rds.DatabaseCluster to create a new RDS Postgres SQL cluster. It's working fine.
However, I can't find an existing way to assign an existing security group to this cluster for security. Has anyone done it before?
You can find an existing security group by using the following:
const securityGroup = SecurityGroup.fromSecurityGroupId(this, 'SG', 'sg-12345', {
mutable: false
});
https://docs.aws.amazon.com/cdk/api/latest/docs/#aws-cdk_aws-ec2.SecurityGroup.html
I'm trying to find security group ID in metadata, but there is only a name of the group.
The get the name with curl like this:
curl http://169.254.169.254/latest/meta-data/security-groups/
RESULT
some-secgroup-name
I need the ID like this one sg-1234567911
Any Idea how to I find in metadata (whitout CLI)?
You'll need the MAC address of the instance. To obtain that you can do a curl using the metadata url:
http://169.254.169.254/latest/meta-data/mac
Once you have that, you can get the security groups from the network section:
http://169.254.169.254/latest/meta-data/network/interfaces/macs/<your_mac>/security-group-ids
Ref: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-data-categories
Recently it was create the possibility to access RDS instances with IAM users and roles. I am confused about how to configure a python connection, since I would not use the database authentication data with psycopg2.
Now I am using like this:
conn = psycopg2.connect("dbname='%s' user='%s' host='%s' password='%s'" % (db_name, db_user, db_host, db_pass))
I have not idea how to use IAM credentials to connect my lambda function with IAM auth.
Please help.
First, you need to create an IAM policy and a DB user as described here:
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html
Then you need to create IAM role for your Lambda function and attach the IAM policy created above to it. Your Lambda function will need to be executed with this role to be able to create a temporary DB password for the DB user.
Finally, you can generate a temporary password for your DB user (created above) within your Lambda using a code snippet like this:
from urllib.parse import quote_plus
import boto3
def get_password(rds_hostname, db_user, aws_region=None, url_encoded=True):
if (not aws_region):
aws_region = boto3.session.Session().region_name
if (not aws_region):
raise Exception("Error: no aws_region given and the default region is not set!")
rds_port = 5432
if (":" in rds_hostname):
split_hostname = rds_hostname.split(":")
rds_hostname = split_hostname[0]
rds_port = int(split_hostname[1])
rds_client = boto3.client("rds")
password = rds_client.generate_db_auth_token( Region=aws_region,
DBHostname=rds_hostname,
Port=rds_port,
DBUsername=db_user)
if url_encoded:
return quote_plus( password )
else:
return password
Do not assign the the password to a variable. Get a new password on every run, since the password has limited time validity and your Lambda container might not be recycled before it expires...
Finally, create the DB connection string for whatever python package you use (I would suggest some pure Python implementation, such as pg8000) from your RDS hostname, port, username and the temporary password obtained with the function above (<user>:<password>#<hostname>:<port>/<db_name>).
Connecting to the RDS might be a bit tricky. If you don't know how to set up VPC's properly I would suggest you run your Lambda outside of VPC and connect to the RDS over a public IP.
Additionally, you will probably need to enforce SSL connection and possibly include the RDS CA file in your Lambda deployment package. The exact way how to do this depends on what you use to connect (I could only describe how to do this with pymysql and sqlalchemy).
Each of these steps could be described in a tutorial of it's own, but knowing about them should be enough to get you started.
Good luck!
I currently want to deploy a simple Django app in AWS using Elastic Beanstalk and RDS, following this tutorial: http://www.1strategy.com/blog/2017/05/23/tutorial-django-elastic-beanstalk/. To create the Beanstalk app I use the command eb create --scale 1 -db -db.engine postgres -db.i db.t2.micro.
In the creation process, the tool fails to create the [AWSEBRDSDBSecurityGroup]. Here is the output:
2018-07-28 06:07:51 ERROR Stack named 'awseb-e-ygq5xuvccr-stack' aborted
operation. Current state: 'CREATE_FAILED' Reason: The following resource(s)
failed to create: [AWSEBRDSDBSecurityGroup].
2018-07-28 06:07:51 ERROR Creating RDS database security group named:
awseb-e-ygq5xuvccr-stack-awsebrdsdbsecuritygroup-oj71kkwnaaag failed Reason:
Either the resource does not exist, or you do not have the required permissions.
I am using an access token with full administrator rights.
How can I solve this issue?
Are you sure you want to use a DB Security group and not a VPC Security group? AFAIK, db security groups should no longer be needed in new accounts, you should just be able to attach an existing VPC security group directly to your instance.
If you do need it, then its most likely one of these:
A badly worded error for hitting the limits for max db security groups
You actually don't have the admin permissions as you claimed.
Do try out and let us know what you find.
I have created an instance using the Boto 22 interface, my connection arguments including key_id and access_key, and a new security group authorized with the following.
security_group.authorize_ingress(IpProtocol="tcp",CidrIp="0.0.0.0/0",FromPort=22,ToPort=22)
I create the instance with
instance = ec2.create_instances(ImageId='ami-5b41123e', KeyName='test_pair57', InstanceType="t2.micro", MinCount=1, MaxCount=1)
I set the program to wait in a while loop until it finds the instance state is running. However, I still can't SSH into the public IP address it then prints out. The connection always times out. I have tried specifying the port, but that does not change it.
Do I need to pass my new keypair's fingerprint somewhere or is there something else I'm missing?
My security settings were incorrect. Checking them in the AWS online interface allowed me to verify the settings and showed me the issue.
I needed to add my created security settings to the instance like so:
instance = ec2.create_instances(ImageId=image.id, KeyName='test_pair' + str(rand), InstanceType="t2.micro", MinCount=1, MaxCount=1, SecurityGroupIds=[security_group.id])