I am unable to connect from a Fargate container to an RDS instance when its ingress is limited through security groups. I can connect with lambdas though.
The container has no issue hitting SQS, or the internet. Only has issues hitting the RDS endpoint.
Here is an excerpt from the template, where the database ingress is open. Fargate can connect without issue.
Service:
Type: AWS::ECS::Service
Properties:
ServiceName: !Ref ServiceName
Cluster: !Ref Cluster
TaskDefinition: !Ref TaskDefinition
PlatformVersion: 1.3.0
DeploymentConfiguration:
MinimumHealthyPercent: 100
MaximumPercent: 200
DesiredCount: 0
LaunchType: FARGATE
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: DISABLED
Subnets:
- !Ref PrivateSubnet1
SecurityGroups:
- !Ref DatabaseAccessSecurityGroup
DatabaseInstance:
Type: AWS::RDS::DBInstance
Properties:
Engine: mysql
EngineVersion: 8.0.16
AvailabilityZone: !GetAtt PrivateSubnet1.AvailabilityZone
PubliclyAccessible: false
...
VPCSecurityGroups:
- !Ref DatabaseSecurityGroup
DatabaseSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: Cloudformation managed Db subnet group
SubnetIds:
- !Ref PrivateSubnet1
- !Ref PrivateSubnet2
DatabaseSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: !Ref VPC
GroupName: database-sg
GroupDescription: Database security group
SecurityGroupIngress:
- Description: Access to RDS
# allowing all works with Fargate
CidrIp: 0.0.0.0/0
FromPort: 3306
ToPort: 3306
IpProtocol: tcp
DatabaseAccessSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: database-access-sg
GroupDescription: Security group for accessing db
VpcId: !Ref VPC
But if I change the DatabaseSecurityGroup Group ingress to only allow ingress through DatabaseAccessSecurityGroup I get errors when trying to connect through Fargate. Lambdas using the same security group have no issue.
SecurityGroupIngress:
- Description: Access to RDS
CidrIp: 0.0.0.0/0
SourceSecurityGroupId: !GetAtt DatabaseAccessSecurityGroup.GroupId
FromPort: 3306
ToPort: 3306
IpProtocol: tcp
Is there any way to get the Fargate Service to respect security group rules?
This issue was occurring because I was using a Service in the Cloudformation template, but spinning up the tasks via ecs.runTask, which overrode the security groups in the Service.
Related
i am pretty new to CloudFormation templates. I have already created a VPC with 2 public and 4 private subnets. Now, i want to create an EC2 instance in 2 of the private subnets, which is then load balanced using ELB created on a public subnet. Below is the CFT template for the same.
Parameters:
SecurityGroupDescription:
Description: Security Group Description
Type: String
KeyName:
Description: Key Pair for EC2
Type: 'AWS::EC2::KeyPair::KeyName'
VPC:
Description: Select VPC.
Type: AWS::EC2::VPC::Id
Subnet1:
Description: Private Subnet to Deploy Docker MFA.
Type: AWS::EC2::Subnet::Id
Subnet2:
Description: Private Subnet to Deploy Docker MFA.
Type: AWS::EC2::Subnet::Id
Mappings:
RegionMap:
us-west-2:
AMI: ami-0c54e4ec017b92f04
Resources:
EC2InstanceMule1:
Type: AWS::EC2::Instance
Properties:
InstanceType: t2.micro
ImageId:
Fn::FindInMap:
- RegionMap
- Ref: AWS::Region
- AMI
SubnetId:
Ref: Subnet1
SecurityGroups:
- !GetAtt EC2SecurityGroup.GroupId
KeyName: !Ref KeyName
EC2InstanceMule2:
Type: AWS::EC2::Instance
Properties:
InstanceType: t2.micro
ImageId:
Fn::FindInMap:
- RegionMap
- Ref: AWS::Region
- AMI
SubnetId:
Ref: Subnet2
SecurityGroups:
- !GetAtt EC2SecurityGroup.GroupId
KeyName: !Ref KeyName
# security group
ELBSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: ELB Security Group
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0
EC2SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: !Ref SecurityGroupDescription
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
SourceSecurityGroupId:
Fn::GetAtt:
- ELBSecurityGroup
- GroupId
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
# Load Balancer for EC2
LoadBalancerforEC2:
Type: AWS::ElasticLoadBalancing::LoadBalancer
Properties:
Instances:
- !Ref EC2InstanceMule1
- !Ref EC2InstanceMule2
Listeners:
- LoadBalancerPort: '80'
InstancePort: '80'
Protocol: HTTP
HealthCheck:
Target: HTTP:80/
HealthyThreshold: '3'
UnhealthyThreshold: '5'
Interval: '30'
Timeout: '5'
SecurityGroups:
- !GetAtt ELBSecurityGroup.GroupId
I am getting the following error :
The parameter groupName cannot be used with the parameter subnet (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterCombination
I have gone through the previous question of the same error and used the security group ID that is being created. Still the error persists. Also, any other modifications required would be appreciated.
You should be using SecurityGroupIds, rather then SecurityGroups.
As part of the CI/CD Jenkins pipeline, I am deploying a springboot application to AWS EC2/Fargate using cloudformation from an image available at dockerhub. I have my access key, the secret, region, and subnet defined as secrets that are passed at runtime. The cloudformation deployment is failing with status CREATE_FAILED and the following error:
Invalid request provided: CreateService error: Security group
sg-0da667222da8a6eb2 does not appear to belong to the same VPC as the
input subnets. (Service: Ecs, Status Code: 400, Request ID:
503ce486-c3db-4d35-bb92-5f4770662c05, Extended Request ID: null)
Here is my cloudformation yaml file content:
AWSTemplateFormatVersion: "2010-09-09"
Parameters:
SubnetID:
Type: String
ServiceName:
Type: String
ServiceVersion:
Type: String
DockerHubUsername:
Type: String
Resources:
Cluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: deployment-example-cluster
ServiceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: ServiceSecurityGroup
GroupDescription: Security group for service
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 8080
ToPort: 8080
CidrIp: 0.0.0.0/0
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: !Sub ${ServiceName}-task
Cpu: 256
Memory: 512
NetworkMode: awsvpc
ContainerDefinitions:
- Name: !Sub ${ServiceName}-container
Image: !Sub ${DockerHubUsername}/${ServiceName}:${ServiceVersion}
PortMappings:
- ContainerPort: 8080
RequiresCompatibilities:
- EC2
- FARGATE
Service:
Type: AWS::ECS::Service
Properties:
ServiceName: !Sub ${ServiceName}-service
Cluster: !Ref Cluster
TaskDefinition: !Ref TaskDefinition
DesiredCount: 1
LaunchType: FARGATE
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: ENABLED
Subnets:
- !Ref SubnetID
SecurityGroups:
- !GetAtt ServiceSecurityGroup.GroupId
Here is screenshot of cloudformation stack building process:
Surprisingly, sg-0da667222da8a6eb2 is not one of my security groups. Any help will be appreciated.
Your ServiceSecurityGroup, as it is defined, it is created in a default VPC. However, your SubnetID probably belongs to a custom VPC. Therefore, you have to provide VpcId for your ServiceSecurityGroup:
Parameters:
VpcId:
Type: AWS::EC2::VPC::Id
# others not shown
Resources:
# only relevant part shown
ServiceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: ServiceSecurityGroup
GroupDescription: Security group for service
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 8080
ToPort: 8080
CidrIp: 0.0.0.0/0
VpcId: !Ref VpcId
Service:
Type: AWS::ECS::Service
Properties:
ServiceName: !Sub ${ServiceName}-service
Cluster: !Ref Cluster
TaskDefinition: !Ref TaskDefinition
DesiredCount: 1
LaunchType: FARGATE
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: ENABLED
Subnets:
- !Ref SubnetID
SecurityGroups:
- !GetAtt ServiceSecurityGroup.GroupId
I am using a custom VPC and not the default VPC in my setup. When cloudformation kicks in, it looks to build the resources in the default VPC in the absence of the information provided. In my case when I pass the custom VPC from my Jenkins as an environment variable, the stack comes up as expected.
I am deploying an EC2 instance using CloudFormation. Then I installed apache and uploaded the files to EC2 instance after deployment. When the instance is deployed I cannot access it using public DNS from browser.
This is my EC2 instance resource and its security group.
WebServerInstance:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref InstanceType
KeyName: !Ref KeyName
SubnetId: !Ref PublicSubnet1
ImageId:
Fn::FindInMap:
- AWSRegionArch2AMI
- Ref: AWS::Region
- Fn::FindInMap:
- AWSInstanceType2Arch
- Ref: InstanceType
- Arch
AvailabilityZone: !Select
- 0
- Fn::GetAZs: !Ref AWS::Region
WebServerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Enable HTTP access via port 80
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '80'
ToPort: '80'
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp:
Ref: SSHLocation
VpcId: !Ref Vpc
When I access it from the browser, it just keeps loading loading and loading. I set the inbound rules on the security group too. What is wrong with it and how can I fix it?
This is my public DNS,
http://ec2-3-{xxx-xxx-xx}.eu-west-1.compute.amazonaws.com/
This is the Public subnet resource.
PublicSubnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref Vpc
CidrBlock: !Select [ 0, !Cidr [ !Ref VpcCidr, 12, 8 ] ]
MapPublicIpOnLaunch: True
AvailabilityZone: !Select
- 0
- Fn::GetAZs: !Ref AWS::Region
There is a route table for public subnet.
In the internet gateway console, there is only one gateway and which is not attached to the VPC in the template. Can this be the issue?
Edit
I got this error
There are several reasons outside the security group allowing access. The following should be checked:
Check your instances subnet has a route within its route table for 0.0.0.0/0 which has a destination of a internet gateway.
Each subnet will have an available route table (this will be the default route table if you did not specify one).
This can be completed by using the CloudFormation below
InternetGateway:
Type: AWS::EC2::InternetGateway
AttachGateway:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId:
Ref: VPC
InternetGatewayId:
Ref: InternetGateway
RouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId:
Ref: myVPC
Route:
Type: AWS::EC2::Route
DependsOn: InternetGateway
Properties:
RouteTableId:
Ref: RouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId:
Ref: InternetGateway
SubnetRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId:
Ref: Subnet
RouteTableId:
Ref: RouteTable
If you updated the default NACL make sure you added both port 80 and ephemeral ports to the rules.
Make sure apache is running on the host (not just installed). This can be done by running systemctl start apache on debian based OS or systemctl start httpd on a RHEL based.
I am trying to autoScale this ec2 instance please guide me how to do it. Any template that might be helpful so that I can get started with the autoscaling. I am attaching only ec2 instance template which I want to autoScale.
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
SourceStackName:
Description: "Source stack name"
Type: String
AllowedPattern: "^[a-zA-Z][-a-zA-Z0-9]*$"
Default: "shifa-vpc"
Resources:
webserver:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: webserver-sg
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
Description: For traffic from Internet
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0
Description: For traffic from Internet
GroupDescription: Security Group for demo server
VpcId:
Fn::ImportValue:
Fn::Sub: "${SourceStackName}-VpcID"
EC2Instance:
Type: AWS::EC2::Instance
Properties:
AvailabilityZone: us-east-1a
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
DeleteOnTermination: "true"
VolumeSize: "8"
VolumeType: gp2
ImageId: ami-09d95fab7fff3776c # ami-0bdcc6c05dec346bf
InstanceType: t2.micro
#IamInstanceProfile: !Ref ListS3BucketsInstanceProfile
#KeyName: ky-webserver
NetworkInterfaces:
- Description: Primary network interface
DeviceIndex: 0
SubnetId:
Fn::ImportValue:
Fn::Sub: "${SourceStackName}-PublicSubnet"
GroupSet:
- !Ref webserver
Outputs:
ec2:
Description: ec2
Value: !Ref EC2Instance
Export:
Name:
Fn::Sub: "${AWS::StackName}-server"
sgGroupId:
Description: ec2
Value: !GetAtt webserver.GroupId
Export:
Name:
Fn::Sub: "${AWS::StackName}-sgid"
I am new to cloudformation and I am in training.
Amazon have some examples for AutoScaling instances.
Importantly EC2 instance resources are not part of the autoscaling configuration within CloudFormation.
Instead you would use either a Launch Template or a Launch Configuration resource. The Launch Template is newer so preferably you should use this. These will define the instance configuration such as volumes, instance type etc.
The other component is the Autoscaling Group this will reference either one of previous components and define how the instance should scale.
If you're trying to scale an existing instance you will need to make an AMI from it first and the reference it.
AWS has an example template with autoscaling groups here.
I want to automate the process of creating RDS. I would like to create RDS Aurora.
When deploying the application, stack cloudFormation is validated and I have an error:
An error occurred: DatabaseCluster - The DB instance and EC2 security group are in different VPCs.
Can you say what's wrong?
I followed this post Issue with creating a Postgres RDS in Cloudformation Template but this doesn't work.
Here is part of my serverless.yml file
resources:
Resources:
DatabaseCluster:
Type: AWS::RDS::DBCluster
Properties:
DatabaseName: name${opt:stage, self:provider.stage}
Engine: aurora
MasterUsername: ${ssm:MasterUsername-${opt:stage, self:provider.stage}}
MasterUserPassword: ${ssm:MasterUserPassword-${opt:stage, self:provider.stage}}
Port: "3306"
VpcSecurityGroupIds:
- !Ref VpcSecurityGroup
ServerlessRDS:
Type: AWS::RDS::DBInstance
Properties:
Engine: aurora
DBClusterIdentifier: !Ref "DatabaseCluster"
DBInstanceIdentifier: db-name-${opt:stage, self:provider.stage}
DBInstanceClass: db.t2.medium
VPCSecurityGroups:
- !Ref VpcSecurityGroup
DBSubnetGroupName: !Ref myDBSubnetGroup
VpcSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId:
Ref: ServerlessVPC
GroupDescription: "Allow all traffic"
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
SecurityGroupIngress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
ServerlessVPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: "10.0.0.0/16"
myDBSubnetGroup:
Type: "AWS::RDS::DBSubnetGroup"
Properties:
DBSubnetGroupDescription: "description"
SubnetIds:
- !Ref ServerlessSubnetA
- !Ref ServerlessSubnetB
ServerlessSubnetA:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: ServerlessVPC
AvailabilityZone: "eu-west-1b"
CidrBlock: "10.0.0.0/24"
ServerlessSubnetB:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: ServerlessVPC
AvailabilityZone: "eu-west-1a"
CidrBlock: "10.0.1.0/24"
You need to add DBSubnetGroupName parameter to AWS::RDS::DBCluster Resource.
DatabaseCluster:
Type: AWS::RDS::DBCluster
Properties:
DatabaseName: name${opt:stage, self:provider.stage}
Engine: aurora
MasterUsername: ${ssm:MasterUsername-${opt:stage, self:provider.stage}}
MasterUserPassword: ${ssm:MasterUserPassword-${opt:stage, self:provider.stage}}
Port: "3306"
VpcSecurityGroupIds:
- !Ref VpcSecurityGroup
DBSubnetGroupName:
Ref: myDBSubnetGroup
Also you may want to add explicit dependencies on ServerlessSubnetA and ServerlessSubnetB in VpcSecurityGroup Resource to kind of group resource creation by the service and to avoid any race conditions.
VpcSecurityGroup:
Type: AWS::EC2::SecurityGroup
DependsOn:
- ServerlessSubnetA
- ServerlessSubnetB
Properties:
VpcId:
Ref: ServerlessVPC
GroupDescription: "Allow all traffic"
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
SecurityGroupIngress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
I encountered this situation and found someone had changed the dBs VPC.
Someone had taken a snapshot and restored it to a different VPC and that obviously didn't have the Security Group needed.
This probably happened because the website talking to the database was running on a different VPC.
To resolve it, you need to make a tough decision, because at this point you may end up having to blow the whole thing away and if its part of a group of CFN templates then that could spell even more problems with domino effects.
It's probably worth you try restore the dB back to the original VPC (or change Subnet Group/VPC) where the CFN template thinks it lives and where the security group was created by the CFN template and then re-run the CFN template.
If this fails, for example the database version has recently been updated you might get stuck with:
Cannot upgrade postgres from 9.6.22 to 9.6.11
At this point, the only recourse I know of is to delete the CFN stack and re-run it.