When a new security group is added, or the existing one is modified, the affects are not visible. For instance, I have a security group called “mdi-sg-redshift” with two rules:
As you can see, these rules allow inbounds from anyone across the globe. When applied to the cluster, they should allow inbounds at those ports. Does NOT work! I have rebooted the cluster to no affect.
Here is the snapshot of my Redshift Cluster:
Here is the snapshot of the port scanner.
The cluster was rebooted several times to no effect.
Also noted that the cluster belongs to the same region as the VPC and the security group. The cluster belongs to the VPC that has the security group applied.
I have seen similar issues on EC2 side, but reboots usually fixed it. Not this time.
Anyone with insights? Thanks!
This sounds mostly a VPC rules issue.
Things I will check:
Do you get the same issue if you create your cluster outside of VPC?
Check Cluster Subnet group. It says default in your screen shot. Which subnet groups is dded to this default subnet group? Make sure your cluster is running in the subnet which is added to default subnet group.
Check VPC security group policy for the Red-shift cluster
Did this set-up ever worked in the past ? OR is it the 1st time you are working on this cluster? If it worked in the past, then what setting with respect to VPC/cluster subnet group/ VPC security groups has changed?
Where are you accessing Redshift from?
If you ar trying to access Redshift from outside VPC then please check the Route Table for an entry of Internet Gateway (to verify if the Redshift cluster is publicly available over internet)
If you are trying to access Redshift from within VPC then there might be some other issue that might be stopping access
Related
I saw this today and it seemed odd. In one region I have no VPC, but I apparently have a default security group that can't be deleted because "This is a default security group. Default security groups cannot be deleted."
How did this default security group come to exist without a VPC?
At one point there must have been a VPC, which came with this default security group, but I thought if the VPC is deleted the corresponding default security group would be deleted along with it.
All modern AWS accounts are provisioned with a VPC. VPCs can be deleted but the default security group can not.
I believe this dates back to the beginning of the EC2 service (i.e. EC2 Classic). In 2013 and earlier, users were able to create instances on a flat network that was shared with other customers and there was no concept of VPCs. AWS required EC2 instances to be associated with a security group but if none were created/selected by the user, AWS could rely on the default security group being available.
So the reason that we can have a default security group without a VPC is that security groups were actually a concept before VPCs and AWS still keeps this default security group requirement as a side effect of some of those earlier infrastructure requirements.
Ran into this yesterday.
Was using EC2 Classic Resource Finder and came across this resource along with some Security Groups(including the default) and an EIP. Deleted the non-default SG's and released the EIP without issue. Was unable to delete the default SG with the error "The following security groups can't be deleted. They are either default security groups, referenced by other security groups or they are associated with instances or network interfaces".
Ensuring this was not associated with a VPC or any other resources were referencing it, we reached out to support. Support confirmed that this resource will be sunset with EC2 Classic in 2023.
I think that must be it. I remembered deleting a default VPC for security reasons, but the default security group persisted.
Security is one of the key aspects in this term.
It is useful to note that security is always considered a top priority job at AWS. Therefore, Default Security Group cannot be deleted partially because they it is the last line of defense for potentially malicious attack on your Amazon EC2 instances. Amazon relies on Security Groups as virtual firewalls that your instances will be paired-up with exclusively for safety reasons. That is also why default SGs by default block any incoming traffic.
On the hand, protective NACL (Network Access Control Lists) feature is only one aspect of the VPC setup. VPCs are primarily logical networking compartments in your cloud infrastructure and you have absolute freedom to define them in any way you see fit. One use case is that you can have complex routing solutions that define many of the aspects of your cloud infrastructure, and accidentally adding Amazon EC2 instances to a default VPC by your developers can lead to hours of debugging why your resource is not behaving as expected.
In summary, security and the mentioned legacy aspects of the Amazon EC2 Classic instances related to the VPC concept (EC2-Classic and a VPC ) are one of the key reasons why default SGs can not be deleted compared to the default VPCs. Still, if you delete the default VPC, you will not be able to restore it or to provision new Amazon EC2 instances until you create at least one custom VPC in the respective region.
If you delete the default VPC, any running EC2 instances, network interfaces, it will automatically delete the default security group as well.
Can you delete a default VPC for a region?
You can delete a default subnet or default VPC just as you can delete
any other subnet or VPC.
However, if you delete your default subnets or default VPC, you must
explicitly specify a subnet in another VPC in which to launch your
instance, because you can't launch instances into EC2-Classic. If you
do not have another VPC, you must create a nondefault VPC and
nondefault subnet. For more information, see Create a VPC.
Reference: https://docs.aws.amazon.com/vpc/latest/userguide/default-vpc.html
Background: I have a kubernetes cluster set up in one AWS account that needs to access data in an RDS MySQL instance in a different account and I can't seem to get the settings correct to allow traffic to flow.
What I've tried so far:
Setup a peering connection between the two VPCs. They are in the same region, us-east-1.
Created Route table entries in each account to point traffic on the corresponding subnet to the peering connection.
Created a security group in the RDS VPC to allow traffic from the kubernetes subnets to access MySql.
Made sure DNS Resolution is enabled on both VPC's.
Kubernetes VPC details (Requester)
This contains 3 EC2's (looks like each has its own subnet) that house my kubernetes cluster. I used EKS to set this up.
The route table rules I set up have the 3 subnets associated, and point the RDS VPC CIDR block at the peering connection.
RDS VPC details (Accepter)
This VPC contains the mysql RDS instance, as well as some other resources. The RDS instance has quite a few VPC security groups assigned to it for access from our office IP's etc. It has Public Accessibility set to true.
I repeated the route table setup (in reverse) and pointed back to the K8s VPC subnet / peering connection.
Testing
To test the connection, I've tried 2 different ways. The application that needs to access mysql is written in node, so I just wrote a test connector and example query and it times out.
I also tried netcat from a terminal in the pod running in the kubernetes cluster.
nc -v {{myclustername}}.us-east-1.rds.amazonaws.com 3306
Which also times out. It seems to be trying to hit the correct mysql instance IP though so I'm not sure if that means my routing rules are working right from the k8s vpc side.
DNS fwd/rev mismatch: ec2-XXX.compute-1.amazonaws.com != ip-{{IP OF MY MYSQL}}.ec2.internal
I'm not sure what steps to take next. Any direction would be greatly appreciated.
Side Note: I've read thru this Kubernetes container connection to RDS instance in separate VPC
I think I understand what's going on there. My CIDR blocks do not conflict with the default K8s ips (10.0...) so my problem seems to be different.
I know this was asked a long time ago, but I just ran into this problem as well.
It turns out I was editing the wrong AWS routing table! When I ran kops to create my cluster, it created a new VPC with its own routing table but also another routing table! I needed to add the peer connection route to the cluster's routing table instead of the VPC's Main routing table.
I have doubt on how RDS behaves regarding removing subnets from subnet groups.
In my subnet group configuration, I realized I have configured an unwanted subnet. It's unwanted because it doesn't (and shouldn't) have the routing as the rest of the subnets that form the subnet group, for a specific RDS instance.
I want to remove this subnet from the subnet group, and can do so directly by going to RDS > Subnet Groups > Edit
My doubt is on failover behavior if the instance is sitting in this unwanted subnet. What does RDS do in this case? I don't have Multi-AZ enabled for this instance. Want to keep downtime to a minimum, or zero of possible.
Appreciate the help.
You can view the AZ in which the instance is running by looking in the Amazon RDS console. The Security and Network section shows the current Availability Zone.
You can edit the Subnet Group to remove AZs. (It even lets you remove the AZ in which the database is currently running, but that won't affect the database.)
If you are uncomfortable with making changes, you can always spin-up a test environment with the same settings and play around with the changes.
Bottom line: You can view which AZ is being used. Feel free to remove the AZ from the Subnet Group.
For anyone else reading this, I'm trying something similar with an aurora PSQL cluster and getting
Some of the subnets to be deleted are currently in use: subnet-xxxx (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 1111111-1111-1111-1111-11111; Proxy: null)
AWS documentation[1] hints this is not possible, because the subnet/az has already been used to store data.
[1] https://aws.amazon.com/premiumsupport/knowledge-center/change-vpc-rds-db-instance/
Update: aws support confirmed this and it's also mentioned in [2]. This applies even if there's currently no instance in the subnet I'm trying to remove.
[2] https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Overview.StorageReliability.html#Aurora.Overview.Storage
I am faced with a chicken and egg problem. I currently have a server in EC2 classic, as well as an RDS instance -- in EC2 classic as well. The EC2 instances also interact with Cassandra cluster, which also resides in EC2 classic.
However, I need to move RDS into the VPC. Now, in an ideal world, I'd have all of my stuff in VPC at this point. However, that presents a major migration challenge and I'd like to minimize impact on users and keep steps to minimum -- this is mainly because of the Cassandra cluster.
It turns out that I cannot create security group rules between VPC and Non-VPC security groups.
So, how can I have RDS in VPC that my EC2 instances can access w/o having to open up my RDS to the entire world ?
Any help is greatly appreciated.
UPDATE: So, one idea I had is to assign elastic IPs to my EC2 instances and add IPs explicitly to the security group for RDS within VPC. Would that work ? (trying it now using https://github.com/skymill/aws-ec2-assign-elastic-ip)
Yes, unfortunately that's the only way to do it. You cannot use DNS in security groups, so you're stuck with IP address.
So, I ended up solving it exactly like I described -- assign elastic IPs to my EC2 instances and add IPs explicitly to the security group for RDS within VPC. It ended up working great.
We have couple of RDS that are not added under VPC, so we need to bring them under VPC. Please let me know the steps and downtime expected. Also let me know if there need to be any changes in the webservers, so that everything works fine after RDS is under VPC.
You must have a VPC created before hand that have subnet in atleast two different regions.
After this go and create a "subnet group" for RDS and add two existing subnet in that.
Next take a snapshot of your RDS instance and start a new RDS instance from snapshot in VPC.
That should be it.
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_VPC.html#USER_VPC.Non-VPC2VPC - official documentation from Amazon.
Depending on your configuration and if the RDS needs to be accessible from the Internet you will have to check the option "Publicly Accessible" in the creation wizard and (in addition to the subnets mentioned in other answers) ensure that the Security has port for the DB properly enabled (and maybe from 0.0.0.0/0).