Related
I have created two EC2 instances using CloudFormation - one for the Apache web-server, another one for the PostgreSQL DB. For some reason the web-server cannot telnet into the DB instance on port 5432 even though the DB instance can telnet into the web-server instance on port 80. When I check the DB instance from localhost, it is working fine and telnetting to the localhost 5432 successfully. There are two security groups for each instance:
"TheWebServerSecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Security Group for The web-server instance",
"SecurityGroupIngress" : [
{ "IpProtocol" : "tcp", "FromPort" : "80", "ToPort" : "80", "CidrIp" : "0.0.0.0/0" },
{ "IpProtocol" : "tcp", "FromPort" : "22", "ToPort" : "22", "CidrIp" : "0.0.0.0/0" }
],
"VpcId" : { "Ref": "TheVPC" },
"Tags" : [ { "Key": "Name", "Value": "TheWebServerSecurityGroup" } ]
}
},
"TheDBSecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Security Group for The DB instance",
"SecurityGroupIngress" : [
{ "IpProtocol" : "tcp", "FromPort" : "22", "ToPort" : "22", "CidrIp" : "0.0.0.0/0" },
{ "IpProtocol" : "tcp", "FromPort" : "5432", "ToPort" : "5432", "CidrIp" : "0.0.0.0/0" }
],
"VpcId" : { "Ref": "TheVPC" },
"Tags" : [ { "Key": "Name", "Value": "TheDBSecurityGroup" } ]
}
},
What might be wrong with the configuration? Any help is appreciated.
UPD: I tried adding the following inbound/outbound rules, but with that it becomes impossible to install packages via yum:
"TheOutboundRule": {
"Type": "AWS::EC2::SecurityGroupEgress",
"Properties":{
"IpProtocol": "tcp",
"FromPort": 0,
"ToPort": 5432,
"DestinationSecurityGroupId": {
"Fn::GetAtt": [
"TheDBSecurityGroup",
"GroupId"
]
},
"GroupId": {
"Fn::GetAtt": [
"TheWebServerSecurityGroup",
"GroupId"
]
}
}
},
"TheInboundRule": {
"Type": "AWS::EC2::SecurityGroupIngress",
"Properties":{
"IpProtocol": "tcp",
"FromPort": 0,
"ToPort": 5432,
"SourceSecurityGroupId": {
"Fn::GetAtt": [
"TheWebServerSecurityGroup",
"GroupId"
]
},
"GroupId": {
"Fn::GetAtt": [
"TheDBSecurityGroup",
"GroupId"
]
}
}
},
I also tried adding just the inbound rule (without an outbound one), but it does not work either.
i tried to create an EC2 instance with the template below,
{
"Description" : "Create an EC2 instance running the Amazon Linux 64 bit AMI.",
"Parameters" : {
"KeyPair" : {
"Description" : "The EC2 Key Pair to allow SSH access to the instance",
"Type" : "String",
"Default" : "formationKey"
}
},
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "formationKeyPair" },
"ImageId" : "ami-0eb14fe5735c13eb5",
"SecurityGroups" : [ { "Ref" : "FormationSecurityGroup" } ],
"InstanceType" : "t2.micro",
"UserData": {
"Fn::Base64": {
"Fn::Join": [ "",
[ "#!/bin/bash\n",
"/opt/aws/bin/cfn-init --region ", { "Ref": "AWS::Region" },
" -s ", { "Ref": "AWS::StackName" },
" -r Ec2Instance\n" ]
]
}
}
},
"Metadata" : {
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"httpd" : []
}
},
"services" : {
"sysvinit" : {
"httpd" : {
"enabled" : "true",
"ensureRunning" : "true"
}
}
}
}
}
}
},
"InstanceSecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Allow HTTP and SSH access",
"SecurityGroupIngress" : [ {
"IpProtocol" : "tcp",
"FromPort" : "22",
"ToPort" : "22",
"CidrIp" : "0.0.0.0/0"
}, {
"IpProtocol" : "tcp",
"FromPort" : "80",
"ToPort" : "80",
"CidrIp" : "0.0.0.0/0"
} ]
}
}
},
"Outputs" : {
"InstanceId" : {
"Description" : "The InstanceId of the newly created EC2 instance",
"Value" : {
"Ref" : "FormationEC2"
}
}
},
"AWSTemplateFormatVersion" : "2010-09-09"
}
but i get this error
" Template format error: Unresolved resource dependencies
[formationKeyPair, FormationSecurityGroup] in the Resources block of
the template "
it seems to be a problem caused by not defining a parameter.
but, isn't it that i defined the parameter in the first place??
i saw a question article similar to my problem, but it was written in YAML
how can i troubleshoot in JSON?
There are some errors with your json. The reference of the security group and SSH is wrong. I've edited it and it looks like it is working.
{
"Description": "Create an EC2 instance running the Amazon Linux 64 bit AMI.",
"Parameters": {
"KeyPair": {
"Description": "The EC2 Key Pair to allow SSH access to the instance",
"Type": "String",
"Default": "formationKey"
}
},
"Resources": {
"Ec2Instance": {
"Type": "AWS::EC2::Instance",
"Properties": {
"KeyName": {
"Ref": "KeyPair"
},
"ImageId": "ami-0eb14fe5735c13eb5",
"SecurityGroups": [
{
"Ref": "InstanceSecurityGroup"
}
],
"InstanceType": "t2.micro",
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash\n",
"/opt/aws/bin/cfn-init --region ",
{
"Ref": "AWS::Region"
},
" -s ",
{
"Ref": "AWS::StackName"
},
" -r Ec2Instance\n"
]
]
}
}
},
"Metadata": {
"AWS::CloudFormation::Init": {
"config": {
"packages": {
"yum": {
"httpd": []
}
},
"services": {
"sysvinit": {
"httpd": {
"enabled": "true",
"ensureRunning": "true"
}
}
}
}
}
}
},
"InstanceSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Allow HTTP and SSH access",
"SecurityGroupIngress": [
{
"IpProtocol": "tcp",
"FromPort": "22",
"ToPort": "22",
"CidrIp": "0.0.0.0/0"
},
{
"IpProtocol": "tcp",
"FromPort": "80",
"ToPort": "80",
"CidrIp": "0.0.0.0/0"
}
]
}
}
},
"Outputs": {
"InstanceId": {
"Description": "The InstanceId of the newly created EC2 instance",
"Value": {
"Ref": "Ec2Instance"
}
}
}
}
When running WSO2 Micro-integrator inside a docker container as a task in AWS, I get an error in the logs about the hostname? In the Dockerfile I don't specifically set the hostname of the container in any way. I created the task using the Cloudformation tool and do not get this error when running the container locally. I tried running on a different VPC as well, without any result. The error remains.
FYI: It is supposed to accept traffic on port 8290 and allow it to send outbound to any IP in the world. Currently I have both inbound and outbound rules set to allow on 0.0.0.0/0 with all protocols.
The full error is as follows:
at
org.eclipse.osgi.internal.framework.BundleContextImpl.registerService(BundleContextImpl.java:544)
at
org.wso2.micro.integrator.ntask.core.internal.TasksDSComponent.activate(TasksDSComponent.java:88)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at
org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at
org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
at
org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at
org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
In case anyone is wondering:
I have setup a new VPC, complete with internet gateway, routes and route tables. The instance will run but I am unable to connect to it in any way.
The following script is run to get the task and make it available:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"myVPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.0.1.0/16",
"Tags": [
{"Key":"Name", "Value":"myVPC"
}
]
},
},
"myInternetGateway" : {
"Type" : "AWS::EC2::InternetGateway",
"Properties" : {
}
},
"myRouteTable": {
"Type" : "AWS::EC2::RouteTable",
"Properties" : {
"VpcId" : {
"Ref": "myVPC"
}
}
},
"mySubPublic": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"AvailabilityZone": "eu-central-1a",
"CidrBlock": "10.0.1.0/28",
"MapPublicIpOnLaunch": true,
"VpcId": {
"Ref": "myVPC"
}
},
"DependsOn": "myInternetGateway"
},
"mySubnetRoutetable": {
"Type" : "AWS::EC2::SubnetRouteTableAssociation",
"Properties" : {
"RouteTableId" : {
"Ref": "myRouteTable"
},
"SubnetId" : {
"Ref": "mySubPublic"
}
}
},
"myVPCGatewayAttachment": {
"Type" : "AWS::EC2::VPCGatewayAttachment",
"Properties" : {
"InternetGatewayId" : {
"Ref": "myInternetGateway"
},
"VpcId" : {
"Ref": "myVPC"
}
}
},
"myRoute": {
"Type" : "AWS::EC2::Route",
"Properties" : {
"GatewayId" : {
"Ref": "myInternetGateway"
},
"DestinationCidrBlock": "0.0.0.0/0",
"RouteTableId" : {
"Ref": "myRouteTable"
}
}
},
"mySecGroup": {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "my security group for all incoming and outgoing.",
"GroupName" : "mySecGroup",
"SecurityGroupEgress" : [ {
"CidrIp" : "0.0.0.0/0",
"Description" : "Allow machine to reach internet.",
"FromPort" : -1,
"IpProtocol" : -1,
"ToPort" : -1
} ],
"SecurityGroupIngress" : [ {
"CidrIp" : "0.0.0.0/0",
"Description" : "Allow machine to be reached from the entire internet.",
"FromPort" : -1,
"IpProtocol" : -1,
"ToPort" : -1
} ],
"VpcId" : {"Ref": "myVPC"}
},
"DependsOn": "myVPC"
},
"myCluster": {
"Type": "AWS::ECS::Cluster",
"Properties": {
"ClusterName": "myCluster"
},
"DependsOn": [
"myVPC"
]
},
"myLogs": {
"Type" : "AWS::Logs::LogGroup",
"Properties" : {
"LogGroupName" : "myLogGroup",
"RetentionInDays" : 7
}
},
"myDockerTask": {
"Type": "AWS::ECS::TaskDefinition",
"Properties": {
"ContainerDefinitions": [
{
"Cpu": 1024,
"Image": "<NRHERE>.dkr.ecr.eu-central-1.amazonaws.com/my",
"Memory": 2048,
"MemoryReservation": 2048,
"Name": "myESBContainer",
"LogConfiguration": {
"LogDriver": "awslogs",
"Options": {
"awslogs-group": {"Ref": "myLogs"},
"awslogs-region": "eu-central-1",
"awslogs-stream-prefix": "my"
}
}
}
],
"Cpu": "1024",
"ExecutionRoleArn": "arn:aws:iam::<NRHERE>:role/ecsTaskExecutionRole",
"Family": "myESB",
"Memory": "2048",
"NetworkMode": "awsvpc",
"RequiresCompatibilities": [
"FARGATE",
"EC2"
],
"TaskRoleArn": "arn:aws:iam::<NRHERE>:role/ecsTaskExecutionRole"
},
},
"myService": {
"Type" : "AWS::ECS::Service",
"Properties" : {
"Cluster" : {"Fn::GetAtt": ["myCluster", "Arn"]},
"DesiredCount" : 1,
"DeploymentController": {"Type": "ECS"},
"LaunchType" : "FARGATE",
"NetworkConfiguration" : {
"AwsvpcConfiguration" : {
"AssignPublicIp" : "ENABLED",
"SecurityGroups" : [ {"Fn::GetAtt": ["mySecGroup", "GroupId"]} ],
"Subnets" : [ {"Ref": "mySubPublic"}]
}
},
"SchedulingStrategy" : "REPLICA",
"ServiceName" : "myService",
"TaskDefinition": {"Ref": "myDockerTask"}
},
"DependsOn": "mySubPublic"
},
"myDeadLetterQueue": {
"Type" : "AWS::SQS::Queue",
"Properties" : {
"QueueName" : "myDeadLetterQueue"
}
},
"myQueue": {
"Type" : "AWS::SQS::Queue",
"Properties" : {
"QueueName" : "myQueue",
"RedrivePolicy": {
"deadLetterTargetArn" : {"Fn::GetAtt": ["myDeadLetterQueue", "Arn"]},
"maxReceiveCount" : 2
}
},
"DependsOn": "myDeadLetterQueue"
}
}
}
Ultimately found the problem. The software could not identify itself because it used localhost instead of 127.0.0.1 for local loopback.
Since I am not in control of the software I tried adding the following to the VPC:
"EnableDnsSupport": true,
"EnableDnsHostnames": true,
This worked and the task is now able to resolve it's own hostname, no longer crashing.
I'm trying to automate the launch of my instances with a Cloudformation template.
When the stack is being created I get the following error :
Security group sg-xxxxx and subnet subnet-xxxxx belong to different networks.
This is my current template :
{
"Description": "AWS Cloudformation template to launch a Docker Swarm cluster of two nodes.",
"Resources" : {
"TwitterappVPC": {
"Type": "AWS::EC2::VPC",
"Properties" : {
"CidrBlock" : "10.0.0.0/16",
"EnableDnsSupport" : "true",
"EnableDnsHostnames" : "true",
"InstanceTenancy" : "dedicated"
}
},
"PublicSubnet" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"VpcId" : { "Ref" : "TwitterappVPC" },
"CidrBlock" : "10.0.0.0/16",
"AvailabilityZone": {
"Fn::Select" : [ "0", { "Fn::GetAZs" : "" } ]
}
}
},
"InternetGateway" : {
"Type" : "AWS::EC2::InternetGateway"
},
"AttachGateway" : {
"Type" : "AWS::EC2::VPCGatewayAttachment",
"Properties" : {
"VpcId" : { "Ref" : "TwitterappVPC" },
"InternetGatewayId" : { "Ref" : "InternetGateway" }
}
},
"TwitterappSecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Enable all Swarm, Microservices and SSH traffic ports",
"SecurityGroupIngress" : [
{"IpProtocol" : "tcp", "FromPort" : "22", "ToPort" : "22", "CidrIp" : "0.0.0.0/0"},
{"IpProtocol" : "tcp", "FromPort" : "2377", "ToPort" : "2377", "CidrIp" : "0.0.0.0/0"},
{"IpProtocol" : "tcp", "FromPort" : "4789", "ToPort" : "4789", "CidrIp" : "0.0.0.0/0"},
{"IpProtocol" : "udp", "FromPort" : "4789", "ToPort" : "4789", "CidrIp" : "0.0.0.0/0"},
{"IpProtocol" : "tcp", "FromPort" : "7946", "ToPort" : "7946", "CidrIp" : "0.0.0.0/0"},
{"IpProtocol" : "udp", "FromPort" : "7946", "ToPort" : "7946", "CidrIp" : "0.0.0.0/0"},
{"IpProtocol" : "tcp", "FromPort" : "3306", "ToPort" : "3306", "CidrIp" : "0.0.0.0/0"},
{"IpProtocol" : "tcp", "FromPort" : "8080", "ToPort" : "8095", "CidrIp" : "0.0.0.0/0"}
],
"VpcId" : {"Ref" : "TwitterappVPC"}
}
},
"PublicRouteTable" : {
"Type" : "AWS::EC2::RouteTable",
"Properties" : {
"VpcId" : {"Ref" : "TwitterappVPC"}
}
},
"PublicRoute" : {
"Type" : "AWS::EC2::Route",
"DependsOn" : "AttachGateway",
"Properties" : {
"RouteTableId" : { "Ref" : "PublicRouteTable" },
"DestinationCidrBlock" : "0.0.0.0/0",
"GatewayId" : { "Ref" : "InternetGateway" }
}
},
"PublicSubnetRouteTableAssociation" : {
"Type" : "AWS::EC2::SubnetRouteTableAssociation",
"Properties" : {
"SubnetId" : { "Ref" : "PublicSubnet" },
"RouteTableId" : { "Ref" : "PublicRouteTable" }
}
},
"TwitterappMasterNode": {
"Type": "AWS::EC2::Instance",
"Properties": {
"AvailabilityZone": {
"Fn::Select" : [ "0", { "Fn::GetAZs" : "" } ]
},
"InstanceType": "t2.medium",
"KeyName": "keypair-xxxx",
"ImageId": "ami-ac442ac3",
"SecurityGroupIds": [{"Ref" : "TwitterappSecurityGroup"}]
}
}
}
}
Which led me to the following stackoverflow question
The suggested solution was to add some Network interface properties to the EC2 instance properties:
"NetworkInterfaces": [
{
"SubnetId": {"Ref": "PublicSubnet"},
"AssociatePublicIpAddress": "true",
"DeviceIndex": "0",
"GroupSet": [{ "Ref" : "TwitterappSecurityGroup" }]
}
]
This gave me the following error:
Network interfaces and an instance-level security groups may not be specified on the same request
What am I doing wrong?
Rather stupid of me...
The issue was fixed after removing
"SecurityGroupIds": [{"Ref" : "TwitterappSecurityGroup"}]
From the EC2-instance properties.
And adding the the networkinterfaces properties
"NetworkInterfaces": [
{
"SubnetId": {"Ref": "PublicSubnet"},
"AssociatePublicIpAddress": "true",
"DeviceIndex": "0",
"GroupSet": [{ "Ref" : "TwitterappSecurityGroup" }]
}
]
I am uploading following template to create an EC2 instance in CloudFormation. And when I "Validate Template" from console getting following error- Template contains errors.: Invalid template parameter property 'Properties'
Template Code:
Template is attached. Open template with notepad or notepad++
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "This is an AWS Cloud Formation template to create an EC2 instance in a Custom VPC.",
"Parameters" : {
"KeyName" : {
"Type" : "String",
"Default" : "ec2-us-east",
"Description" : "SSH Key to access the EC2 instance"
},
"MyVpc" : {
"Type" : "AWS::EC2::VPC",
"Properties" : {
"CidrBlock" : "10.0.0.0/16",
"EnableDnsHostnames" : "true"
}
},
"PublicSubnet" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"VpcId" : {"Ref" : "MyVpc"},
"CidrBlock" : "10.0.0.0/24",
"AvailabilityZone" : "us-east-1a"
}
},
"InstanceType" : {
"Type" : "String",
"Default" : "t2.micro",
"Description" : "Select EC2 instance type"
}
},
"Resources" : {
"SecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupName" : "My Security Group",
"GroupDescription" : "My CFSecurity Group",
"VpcId" : {"Ref" : "MyVpc"},
"SecurityGroupIngress" : [{
"CidrIp" : "0.0.0.0/0",
"FromPort" : "22",
"IpProtocol" : "tcp",
"ToPort" : "22"
}]
}
},
"Server" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"ImageId" : "ami-1853ac65",
"InstanceType" : {"Ref" : "InstanceType"},
"KeyName" : {"Ref" : "KeyName"},
"SecurityGroupIds" : {"Ref" : "SecurityGroup"},
"SubnetId" : {"Ref" : "PublicSubnet"}
}
}
},
"Outputs" : {
"PublicName" : {
"Value" : {"Fn::GetAtt" : ["Server", "PublicDnsName"]},
"Description" : "Public Name (connect via ssh)"
}
}
}
Can you please help me to find out What I am doing wrong?
You are creating VPC and public subnet under key Parameters. You need to define vpc and subnet under key resources. This should work:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "This is an AWS Cloud Formation template to create an EC2 instance in a Custom VPC.",
"Parameters": {
"KeyName": {
"Type": "String",
"Default": "ec2-us-east",
"Description": "SSH Key to access the EC2 instance"
},
"InstanceType": {
"Type": "String",
"Default": "t2.micro",
"Description": "Select EC2 instance type"
}
},
"Resources": {
"SecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupName": "My Security Group",
"GroupDescription": "My CFSecurity Group",
"VpcId": {
"Ref": "MyVpc"
},
"SecurityGroupIngress": [{
"CidrIp": "0.0.0.0/0",
"FromPort": "22",
"IpProtocol": "tcp",
"ToPort": "22"
}]
}
},
"Server": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": "ami-1853ac65",
"InstanceType": {
"Ref": "InstanceType"
},
"KeyName": {
"Ref": "KeyName"
},
"SecurityGroupIds": {
"Ref": "SecurityGroup"
},
"SubnetId": {
"Ref": "PublicSubnet"
}
}
},
"MyVpc": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.0.0.0/16",
"EnableDnsHostnames": "true"
}
},
"PublicSubnet": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Ref": "MyVpc"
},
"CidrBlock": "10.0.0.0/24",
"AvailabilityZone": "us-east-1a"
}
}
},
"Outputs": {
"PublicName": {
"Value": {
"Fn::GetAtt": ["Server",
"PublicDnsName"]
},
"Description": "Public Name (connect via ssh)"
}
}
}