I have tried configuring the RDS cluster using cluster.connections.allowDefaultPortFromAnyIpv4();
but still I am not able to connect to my postgres instance, it keeps timing out.
I've been trying to figure this out from 2 days but still no luck, not sure what to do
Here is the full code for CDK config.
import { CdkWorkshopStack } from "../stacks/cdk-workshop-stack";
import * as rds from "#aws-cdk/aws-rds";
import * as ec2 from "#aws-cdk/aws-ec2";
import { ServerlessCluster } from "#aws-cdk/aws-rds";
import { Duration } from "#aws-cdk/core";
export const createDbInstance = (
scope: CdkWorkshopStack
): { cluster: ServerlessCluster; dbName: string } => {
// Create the VPC needed for the Aurora Serverless DB cluster
const vpc = new ec2.Vpc(scope, "AuroraVPC");
const dbName = "yt_backup";
// Create the Serverless Aurora DB cluster; set the engine to Postgres
const cluster = new rds.ServerlessCluster(scope, "yt_backup_cluster", {
engine: rds.DatabaseClusterEngine.AURORA_POSTGRESQL,
parameterGroup: rds.ParameterGroup.fromParameterGroupName(
scope,
"ParameterGroup",
"default.aurora-postgresql10"
),
defaultDatabaseName: dbName,
//#ts-ignore
vpc: vpc,
//#ts-ignore
scaling: { autoPause: Duration.minutes(10) }, // Optional. If not set, then instance will pause after 5 minutes
});
cluster.connections.allowDefaultPortFromAnyIpv4();
return { cluster, dbName };
};
This opens the security group to all connections:
cluster.connections.allowDefaultPortFromAnyIpv4();
This (see the link for exactly where you would specified this) would give the database server a public IP, allowing connections from outside the VPC:
publiclyAccessible: true,
However, you are creating a Serverless cluster, which does not support the publicly accessible feature at this time.
Like Mark B mentions a Serverless Aurora DB is not publicly accessible.
Having a database publicly accessible is a bad idea from a security point of view in my opinion. (and definitely not open to 0.0.0.0/0)
An application inside your VPC should connect to the database and if you need to access the database you can use a BastionHostLinux , ssh tunnel or Direct Connect.
You can switch is an "non serverless" database if you really need to as this is publicly accessible if it's on a public subnet and there is an internet gateway for the VPC.
Related
I'm able to create an EC2 instance and an RDS db via the cdk but can't find out how to connect them outside of the aws console.
When setting up the connection manually I get the screen that describes the changes saying "To set up a connection between the database and the EC2 instance, VPC security group xxx-xxx-x is added to the database, and VPC security group xxx-xxx-x is added to the EC2 instance."
Is there a way to do this via cdk?
Is it possible to do this in the instanceProps of the DatabaseCluster?
My existing code for the RDS db cluster looks something like this:
this.dbCluster = new DatabaseCluster(this, 'MyDbCluster',{
//
instanceProps: { vpc: props.vpc, vpcSubnets: { subGroupName: 'private' } }
})
How would I add the group to my existing code for the EC2 instance - in the vpcSubnets section?
const ec2Instance = new Instance( this, 'ec2instance', {
//
vpcSubnets: {
subnetGroupName: 'private',
},
})
You need to allow EC2 to connect to RDS.
You do this by using the DatabaseCluster.connections prop (docs).
Look at the examples in the Connections class docs.
this.dbCluster.connections.allowFrom(ec2Instance, ec2.Port.tcp(5432));
Adjust the port to match your database flavor port.
I am wondering if it is possible to configure the “public access source allowlist” from CDK. I can see and manage this in the console under the networking tab, but can’t find anything in the CDK docs about setting the allowlist during deploy. I tried creating and assigning a security group (code sample below), but this didn't work. Also the security group was created as an "additional" security group, rather than the "cluster" security group.
declare const vpc: ec2.Vpc;
declare const adminRole: iam.Role;
const securityGroup = new ec2.SecurityGroup(this, 'my-security-group', {
vpc,
allowAllOutbound: true,
description: 'Created in CDK',
securityGroupName: 'cluster-security-group'
});
securityGroup.addIngressRule(
ec2.Peer.ipv4('<vpn CIDR block>'),
ec2.Port.tcp(8888),
'allow frontend access from the VPN'
);
const cluster = new eks.Cluster(this, 'my-cluster', {
vpc,
clusterName: 'cluster-cdk',
version: eks.KubernetesVersion.V1_21,
mastersRole: adminRole,
defaultCapacity: 0,
securityGroup
});
Update: I attempted the following, and it updated the cluster security group, but I'm still able to access the frontend when I'm not on the VPN:
cluster.connections.allowFrom(
ec2.Peer.ipv4('<vpn CIDER block>'),
ec2.Port.tcp(8888)
);
Update 2: I tried this as well, and I can still access my application's frontend even when I'm not on the VPN. However I can now only use kubectl when I'm on the VPN, which is good! It's a step forward that I've at least improved the cluster's security in a useful manner.
const cluster = new eks.Cluster(this, 'my-cluster', {
vpc,
clusterName: 'cluster-cdk',
version: eks.KubernetesVersion.V1_21,
mastersRole: adminRole,
defaultCapacity: 0,
endpointAccess: eks.EndpointAccess.PUBLIC_AND_PRIVATE.onlyFrom('<vpn CIDER block>')
});
In general EKS has two relevant security groups:
The one used by nodes, which AWS calls "cluster security group". It's setup automatically by EKS. You shouldn't need to mess with it unless you want (a) more restrictive rules the defaults (b) open your nodes to maintenance taks (e.g.: ssh access). This is what you are acessing via cluster.connections.
The Ingress Load Balancer security group. This is an Application Load balancer created and managed by EKS. In CDK, it can be created like so:
const cluster = new eks.Cluster(this, 'HelloEKS', {
version: eks.KubernetesVersion.V1_22,
albController: {
version: eks.AlbControllerVersion.V2_4_1,
},
});
This will will serve as a gateway for all internal services that need an Ingress. You can access it via the cluster.albController propriety and add rules to it like a regular Application Load Balancer. I have no idea how EKS deals with task communication when an Ingress ALB is not present.
Relevant docs:
Amazon EKS security group considerations
Alb Controller on CDK docs
The ALB propriety for EKS Cluster objects
I have an application deployed on Cloud Run. It runs behind an HTTPS Load balancer.
I want it to be able to cache some data using memory store service. I basically followed the documentation to use a serverless vpc connector but this exception keeps poping:
Unable to create a new session key. It is likely that the cache is
unavailable.
I am guessing that my cloud run service can't access memorystore.
On Django I have:
CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": f"redis://{CHANNEL_REDIS_HOST}:{CHANNEL_REDIS_PORT}/16",
"OPTIONS": {
"CLIENT_CLASS": "django_redis.client.DefaultClient",
"IGNORE_EXCEPTIONS": True,
},
"KEY_PREFIX": "api"
}
}
where CHANNEL_REDIS_HOST is the IP from my memorystore primary endpoint and CHANNEL_REDIS_PORT is the port.
When I run this command:
gcloud redis instances describe instance_name --region region --format "value(authorizedNetwork)"
it returns projects/my_project/global/networks/default.
Then, on the VPC network, I clicked on 'default' and then on 'ADD SUBNET'. I created my subnet with IP Address range 10.0.0.0/28. Maybe the problem comes from this step as I do not get a lot about this all IP Communication thing..
When I run this command:
gcloud compute networks subnets describe my subnet
purpose is PRIVATE as intended and network is https://www.googleapis.com/compute/v1/projects/my_project/global/networks/default.
So I think that my memorystore instance and my subnet are able to connect.
Then, I created a serverless VPC connector, using the same region, the default network and the subnet I just created.
Finally on my service I set the VPC connector to the one I just created and I redeploy using Only route requests to private IPs through the VPC connector option, if I choose Route all traffic through the VPC connector my deployment fails, I think probably because I am behind a load balancer, anyway I do not want to route all traffic to my connector.
And after doing this, I still receive the error mention at the beginning of the message..
Any ideas ?
Thanks
So I think my issue was using the db 16. As the maximum number of database on memorystore is 16 it must be from 0 to 15. Changing it make it works.
I am connecting .NET core app to AWS ElastiCache Redis.
The AWS Elasticache Redis is running and port 6329 is open on security group. The client code is using StackExchange.Redis:
var redis = ConnectionMultiplexer.Connect("redis-1.xxxx.0001.use1.cache.amazonaws.com:6379");
//var redis = ConnectionMultiplexer.Connect("localhost"); //working
IDatabase db = redis.GetDatabase();
string value = "abcdefg";
db.StringSet("mykey", value);
string value2 = db.StringGet("mykey");
Console.WriteLine(value2); // writes: "abcdefg"
Console.ReadLine();
Exception is:
It was not possible to connect to the redis server(s). UnableToConnect
on redis-1.xxx.0001.use1.cache.amazonaws.com:6379/Interactive,
Initializing/NotStarted, last: NONE, origin: BeginConnectAsync,
outstanding: 0, last-read: 5s ago, last-write: 5s ago, keep-alive:
60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat:
never, global: 0s ago, v: 2.0.601.3402
Added to ,abortConnect=false to the endpoint, I am able to connect.
Now the error is at StringSet
StackExchange.Redis.RedisConnectionException: 'No connection is
available to service this operation: SET mykey; UnableToConnect ...
The .NET code is working fine with my local Redis service so looks like something incorrect with my AWS setup.
Successfully ssh to my EC2-instance and redis-cli to my AWS redis service:
Any idea please?
NOTE: AWS Elasticache Redis is NOT available for public access as default.
https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/accessing-elasticache.html#access-from-outside-aws
Can you connect to Amazon ElastiСache Redis outside of Amazon?
Question: "...port 6329 is open on security group.." Dont you mean 6379?
Even that, i think i cn have an issue on the "default" subnet groups.
Does the VPC_Subnet (in that group) where the AWS ElastiCache Redis is, has a Routing Table with access to the Internet Gateway?
If not, you need to attache the InternetGteway to the Route Table and than use that Route Table on the Subnet that you want.
It should be like the pick:
Internet GW
I have created a neptune instance and per the documentation here...
I create the following yaml...
hosts: [xxx.xxx.us-east-2.neptune.amazonaws.com]
port: 8182
serializer: { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { serializeResultToString: true }}
And when I try to connect everything seems to work I see...
==>All scripts will now be sent to Gremlin Server - [xx.xx.us-east-2.neptune.amazonaws.com/172.xx.x.xxx:8182] - type
':remote console' to return to local mode
What am I missing why is the query failing?
Have you enabled IAM authentication on your instance?
If yes, you will have to perform some additional steps to connect to the DB.
If no, double check the following:
EC2 instance is in the same VPC as the cluster.
Check the inbound settings for security group attached to the cluster and outbound setting for your EC2 instance.
If it still doesn't connect, I would suggest you to contact AWS Support.
Are you trying to connect from your pc or ec2 instance ?
I think that Neptune is not publicly accessibly
create ec2 instance in the same VPC and try to connect from it