Micro integrator hostname problem in cloud - amazon-web-services

When running WSO2 Micro-integrator inside a docker container as a task in AWS, I get an error in the logs about the hostname? In the Dockerfile I don't specifically set the hostname of the container in any way. I created the task using the Cloudformation tool and do not get this error when running the container locally. I tried running on a different VPC as well, without any result. The error remains.
FYI: It is supposed to accept traffic on port 8290 and allow it to send outbound to any IP in the world. Currently I have both inbound and outbound rules set to allow on 0.0.0.0/0 with all protocols.
The full error is as follows:
at
org.eclipse.osgi.internal.framework.BundleContextImpl.registerService(BundleContextImpl.java:544)
at
org.wso2.micro.integrator.ntask.core.internal.TasksDSComponent.activate(TasksDSComponent.java:88)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at
org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at
org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
at
org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at
org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
In case anyone is wondering:
I have setup a new VPC, complete with internet gateway, routes and route tables. The instance will run but I am unable to connect to it in any way.
The following script is run to get the task and make it available:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"myVPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.0.1.0/16",
"Tags": [
{"Key":"Name", "Value":"myVPC"
}
]
},
},
"myInternetGateway" : {
"Type" : "AWS::EC2::InternetGateway",
"Properties" : {
}
},
"myRouteTable": {
"Type" : "AWS::EC2::RouteTable",
"Properties" : {
"VpcId" : {
"Ref": "myVPC"
}
}
},
"mySubPublic": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"AvailabilityZone": "eu-central-1a",
"CidrBlock": "10.0.1.0/28",
"MapPublicIpOnLaunch": true,
"VpcId": {
"Ref": "myVPC"
}
},
"DependsOn": "myInternetGateway"
},
"mySubnetRoutetable": {
"Type" : "AWS::EC2::SubnetRouteTableAssociation",
"Properties" : {
"RouteTableId" : {
"Ref": "myRouteTable"
},
"SubnetId" : {
"Ref": "mySubPublic"
}
}
},
"myVPCGatewayAttachment": {
"Type" : "AWS::EC2::VPCGatewayAttachment",
"Properties" : {
"InternetGatewayId" : {
"Ref": "myInternetGateway"
},
"VpcId" : {
"Ref": "myVPC"
}
}
},
"myRoute": {
"Type" : "AWS::EC2::Route",
"Properties" : {
"GatewayId" : {
"Ref": "myInternetGateway"
},
"DestinationCidrBlock": "0.0.0.0/0",
"RouteTableId" : {
"Ref": "myRouteTable"
}
}
},
"mySecGroup": {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "my security group for all incoming and outgoing.",
"GroupName" : "mySecGroup",
"SecurityGroupEgress" : [ {
"CidrIp" : "0.0.0.0/0",
"Description" : "Allow machine to reach internet.",
"FromPort" : -1,
"IpProtocol" : -1,
"ToPort" : -1
} ],
"SecurityGroupIngress" : [ {
"CidrIp" : "0.0.0.0/0",
"Description" : "Allow machine to be reached from the entire internet.",
"FromPort" : -1,
"IpProtocol" : -1,
"ToPort" : -1
} ],
"VpcId" : {"Ref": "myVPC"}
},
"DependsOn": "myVPC"
},
"myCluster": {
"Type": "AWS::ECS::Cluster",
"Properties": {
"ClusterName": "myCluster"
},
"DependsOn": [
"myVPC"
]
},
"myLogs": {
"Type" : "AWS::Logs::LogGroup",
"Properties" : {
"LogGroupName" : "myLogGroup",
"RetentionInDays" : 7
}
},
"myDockerTask": {
"Type": "AWS::ECS::TaskDefinition",
"Properties": {
"ContainerDefinitions": [
{
"Cpu": 1024,
"Image": "<NRHERE>.dkr.ecr.eu-central-1.amazonaws.com/my",
"Memory": 2048,
"MemoryReservation": 2048,
"Name": "myESBContainer",
"LogConfiguration": {
"LogDriver": "awslogs",
"Options": {
"awslogs-group": {"Ref": "myLogs"},
"awslogs-region": "eu-central-1",
"awslogs-stream-prefix": "my"
}
}
}
],
"Cpu": "1024",
"ExecutionRoleArn": "arn:aws:iam::<NRHERE>:role/ecsTaskExecutionRole",
"Family": "myESB",
"Memory": "2048",
"NetworkMode": "awsvpc",
"RequiresCompatibilities": [
"FARGATE",
"EC2"
],
"TaskRoleArn": "arn:aws:iam::<NRHERE>:role/ecsTaskExecutionRole"
},
},
"myService": {
"Type" : "AWS::ECS::Service",
"Properties" : {
"Cluster" : {"Fn::GetAtt": ["myCluster", "Arn"]},
"DesiredCount" : 1,
"DeploymentController": {"Type": "ECS"},
"LaunchType" : "FARGATE",
"NetworkConfiguration" : {
"AwsvpcConfiguration" : {
"AssignPublicIp" : "ENABLED",
"SecurityGroups" : [ {"Fn::GetAtt": ["mySecGroup", "GroupId"]} ],
"Subnets" : [ {"Ref": "mySubPublic"}]
}
},
"SchedulingStrategy" : "REPLICA",
"ServiceName" : "myService",
"TaskDefinition": {"Ref": "myDockerTask"}
},
"DependsOn": "mySubPublic"
},
"myDeadLetterQueue": {
"Type" : "AWS::SQS::Queue",
"Properties" : {
"QueueName" : "myDeadLetterQueue"
}
},
"myQueue": {
"Type" : "AWS::SQS::Queue",
"Properties" : {
"QueueName" : "myQueue",
"RedrivePolicy": {
"deadLetterTargetArn" : {"Fn::GetAtt": ["myDeadLetterQueue", "Arn"]},
"maxReceiveCount" : 2
}
},
"DependsOn": "myDeadLetterQueue"
}
}
}

Ultimately found the problem. The software could not identify itself because it used localhost instead of 127.0.0.1 for local loopback.
Since I am not in control of the software I tried adding the following to the VPC:
"EnableDnsSupport": true,
"EnableDnsHostnames": true,
This worked and the task is now able to resolve it's own hostname, no longer crashing.

Related

in JSON, Template format error: Unresolved resource dependencies ~~ in the Resources block of the template

i tried to create an EC2 instance with the template below,
{
"Description" : "Create an EC2 instance running the Amazon Linux 64 bit AMI.",
"Parameters" : {
"KeyPair" : {
"Description" : "The EC2 Key Pair to allow SSH access to the instance",
"Type" : "String",
"Default" : "formationKey"
}
},
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "formationKeyPair" },
"ImageId" : "ami-0eb14fe5735c13eb5",
"SecurityGroups" : [ { "Ref" : "FormationSecurityGroup" } ],
"InstanceType" : "t2.micro",
"UserData": {
"Fn::Base64": {
"Fn::Join": [ "",
[ "#!/bin/bash\n",
"/opt/aws/bin/cfn-init --region ", { "Ref": "AWS::Region" },
" -s ", { "Ref": "AWS::StackName" },
" -r Ec2Instance\n" ]
]
}
}
},
"Metadata" : {
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"httpd" : []
}
},
"services" : {
"sysvinit" : {
"httpd" : {
"enabled" : "true",
"ensureRunning" : "true"
}
}
}
}
}
}
},
"InstanceSecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Allow HTTP and SSH access",
"SecurityGroupIngress" : [ {
"IpProtocol" : "tcp",
"FromPort" : "22",
"ToPort" : "22",
"CidrIp" : "0.0.0.0/0"
}, {
"IpProtocol" : "tcp",
"FromPort" : "80",
"ToPort" : "80",
"CidrIp" : "0.0.0.0/0"
} ]
}
}
},
"Outputs" : {
"InstanceId" : {
"Description" : "The InstanceId of the newly created EC2 instance",
"Value" : {
"Ref" : "FormationEC2"
}
}
},
"AWSTemplateFormatVersion" : "2010-09-09"
}
but i get this error
" Template format error: Unresolved resource dependencies
[formationKeyPair, FormationSecurityGroup] in the Resources block of
the template "
it seems to be a problem caused by not defining a parameter.
but, isn't it that i defined the parameter in the first place??
i saw a question article similar to my problem, but it was written in YAML
how can i troubleshoot in JSON?
There are some errors with your json. The reference of the security group and SSH is wrong. I've edited it and it looks like it is working.
{
"Description": "Create an EC2 instance running the Amazon Linux 64 bit AMI.",
"Parameters": {
"KeyPair": {
"Description": "The EC2 Key Pair to allow SSH access to the instance",
"Type": "String",
"Default": "formationKey"
}
},
"Resources": {
"Ec2Instance": {
"Type": "AWS::EC2::Instance",
"Properties": {
"KeyName": {
"Ref": "KeyPair"
},
"ImageId": "ami-0eb14fe5735c13eb5",
"SecurityGroups": [
{
"Ref": "InstanceSecurityGroup"
}
],
"InstanceType": "t2.micro",
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash\n",
"/opt/aws/bin/cfn-init --region ",
{
"Ref": "AWS::Region"
},
" -s ",
{
"Ref": "AWS::StackName"
},
" -r Ec2Instance\n"
]
]
}
}
},
"Metadata": {
"AWS::CloudFormation::Init": {
"config": {
"packages": {
"yum": {
"httpd": []
}
},
"services": {
"sysvinit": {
"httpd": {
"enabled": "true",
"ensureRunning": "true"
}
}
}
}
}
}
},
"InstanceSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Allow HTTP and SSH access",
"SecurityGroupIngress": [
{
"IpProtocol": "tcp",
"FromPort": "22",
"ToPort": "22",
"CidrIp": "0.0.0.0/0"
},
{
"IpProtocol": "tcp",
"FromPort": "80",
"ToPort": "80",
"CidrIp": "0.0.0.0/0"
}
]
}
}
},
"Outputs": {
"InstanceId": {
"Description": "The InstanceId of the newly created EC2 instance",
"Value": {
"Ref": "Ec2Instance"
}
}
}
}

Target group health check for ECS service failing upon creation with cloudformation

I've been hosting an api as a service on ecs with an alb. Whenver the service is running it fails the health check and continually kills and restarts the task. The task logs always show it repsonding to the health checks with response 200 so it should be passing. I've checked that my security groups and vpc's are configured correctly. I will post my cloud formation template below in hopes someone can find my mistake. I've also tried increasing the health check graces period greatly and it hasn't helped the issue.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Metadata": {
"AWS::CloudFormation::Designer": {
"6514f997-7e0a-48c6-9e5f-95465c35ff00": {
"size": {
"width": 60,
"height": 60
},
"position": {
"x": 280,
"y": 140
},
"z": 0,
"embeds": []
}
}
},
"Parameters": {
"ClusterName": {
"Type" : "String",
"Description" : "Name of the ECS Cluster",
"Default" : "Rest-Api-Explorer"
},
"Subnet": {
"Description": "User Specified Subnet",
"Type": "AWS::EC2::Subnet::Id",
"ConstraintDescription": "Must be a valid Subnet of specified VPC."
}
},
"Mappings" : {
"SubnetToVPCMap" : {
"subnet-58863c01" : {"VPC" : "vpc-6581f100"},
"subnet-ae349ec6" : {"VPC" : "vpc-259a344d"},
"subnet-b9349ed1" : {"VPC" : "vpc-259a344d"},
"subnet-d8349eb0" : {"VPC" : "vpc-259a344d"},
"subnet-e9b7d69e" : {"VPC" : "vpc-6581f100"},
"subnet-ee97d48b" : {"VPC" : "vpc-6581f100"}
},
"VPCToSecurityGroupMap" : {
"vpc-6581f100" : {"SecurityGroup" : "sg-a5c672c1"},
"vpc-259a344d" : {"SecurityGroup" : "sg-1d658072"}
},
"VPCToTag" : {
"vpc-6581f100" : {"Tag" : [{ "Key": "Name", "Value": "Rest API Explorer"}, {"Key": "Type", "Value": "Development"}]},
"vpc-259a344d" : {"Tag" : [{ "Key": "Name", "Value": "Rest API Explorer"}, {"Key": "Type", "Value": "Production"}]}
}
},
"Resources": {
"ECSCluster":{
"Type" : "AWS::ECS::Cluster",
"Properties" : {
"ClusterName" : {"Ref" : "ClusterName"},
"Tags": {
"Fn::FindInMap" : [ "VPCToTag", {"Fn::FindInMap" : [ "SubnetToVPCMap", {"Ref" : "Subnet"}, "VPC"]} , "Tag"]
}
}
},
"EC2Instance": {
"Type": "AWS::EC2::Instance",
"Properties": {
"IamInstanceProfile": "ecsInstanceRole",
"ImageId": "ami-00e0090ac21971297",
"InstanceType": "t2.medium",
"KeyName": "community-admin",
"SecurityGroupIds": [
{"Fn::FindInMap" : [ "VPCToSecurityGroupMap", {"Fn::FindInMap" : [ "SubnetToVPCMap", {"Ref" : "Subnet"}, "VPC"]} , "SecurityGroup"]}
],
"SubnetId": {
"Ref": "Subnet"
},
"Tags": {
"Fn::FindInMap" : [ "VPCToTag", {"Fn::FindInMap" : [ "SubnetToVPCMap", {"Ref" : "Subnet"}, "VPC"]} , "Tag"]
},
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash -xe\n",
"echo ECS_CLUSTER=", { "Ref" : "ClusterName" },
" >> /etc/ecs/ecs.config\n"
]
]
}
}
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "6514f997-7e0a-48c6-9e5f-95465c35ff00"
}
}
},
"ECSALB":{
"Type":"AWS::ElasticLoadBalancingV2::LoadBalancer",
"Properties":{
"Name":"rest-api-explorer-2",
"Scheme":"internal",
"LoadBalancerAttributes": [
{
"Key":"idle_timeout.timeout_seconds",
"Value":"60"
}
],
"SecurityGroups":[
{
"Fn::FindInMap" : [ "VPCToSecurityGroupMap", {"Fn::FindInMap" : [ "SubnetToVPCMap", {"Ref" : "Subnet"}, "VPC"]} , "SecurityGroup"]
}
],
"Subnets" : [
"subnet-58863c01",
"subnet-e9b7d69e",
"subnet-ee97d48b"
]
}
},
"ALBListener":{
"Type":"AWS::ElasticLoadBalancingV2::Listener",
"DependsOn": "ECSTG",
"Properties":{
"DefaultActions":[
{
"Type":"forward",
"TargetGroupArn":{
"Ref":"ECSTG"
}
}
],
"LoadBalancerArn":{
"Ref":"ECSALB"
},
"Port":"80",
"Protocol":"HTTP"
}
},
"ECSTG":{
"Type":"AWS::ElasticLoadBalancingV2::TargetGroup",
"DependsOn":"ECSALB",
"Properties":{
"HealthCheckIntervalSeconds":15,
"HealthCheckPath":"/api/rest/console",
"HealthCheckPort":"traffic-port",
"HealthCheckProtocol":"HTTP",
"HealthCheckTimeoutSeconds":10,
"HealthyThresholdCount":3,
"Name":"rest-api-target-group-2",
"Port":3000,
"Protocol":"HTTP",
"TargetType":"ip",
"UnhealthyThresholdCount":5,
"VpcId":{
"Fn::FindInMap" : [ "SubnetToVPCMap", {"Ref" : "Subnet"}, "VPC"]
}
}
},
"service":{
"Type":"AWS::ECS::Service",
"DependsOn": "ALBListener",
"Properties":{
"Cluster":{
"Ref":"ECSCluster"
},
"DeploymentConfiguration": {
"MaximumPercent": 100,
"MinimumHealthyPercent": 0
},
"DesiredCount":"1",
"HealthCheckGracePeriodSeconds" : 90,
"LaunchType": "FARGATE",
"LoadBalancers":[
{
"ContainerName":"api-rest-explorer",
"ContainerPort":"3000",
"TargetGroupArn":{
"Ref":"ECSTG"
}
}
],
"NetworkConfiguration": {
"AwsvpcConfiguration" : {
"AssignPublicIp" : "ENABLED",
"SecurityGroups" : [
{
"Fn::FindInMap" : [ "VPCToSecurityGroupMap", {"Fn::FindInMap" : [ "SubnetToVPCMap", {"Ref" : "Subnet"}, "VPC"]} , "SecurityGroup"]
}
],
"Subnets" : [
{"Ref": "Subnet"}
]
}
},
"SchedulingStrategy": "REPLICA",
"ServiceName" : "rest-api-explorer-fargate",
"TaskDefinition": "rest-api-explorer-fargate:4"
}
}
}
}

CloudFormation - Template contains errors.: Invalid template parameter property 'Properties'

I am uploading following template to create an EC2 instance in CloudFormation. And when I "Validate Template" from console getting following error- Template contains errors.: Invalid template parameter property 'Properties'
Template Code:
Template is attached. Open template with notepad or notepad++
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "This is an AWS Cloud Formation template to create an EC2 instance in a Custom VPC.",
"Parameters" : {
"KeyName" : {
"Type" : "String",
"Default" : "ec2-us-east",
"Description" : "SSH Key to access the EC2 instance"
},
"MyVpc" : {
"Type" : "AWS::EC2::VPC",
"Properties" : {
"CidrBlock" : "10.0.0.0/16",
"EnableDnsHostnames" : "true"
}
},
"PublicSubnet" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"VpcId" : {"Ref" : "MyVpc"},
"CidrBlock" : "10.0.0.0/24",
"AvailabilityZone" : "us-east-1a"
}
},
"InstanceType" : {
"Type" : "String",
"Default" : "t2.micro",
"Description" : "Select EC2 instance type"
}
},
"Resources" : {
"SecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupName" : "My Security Group",
"GroupDescription" : "My CFSecurity Group",
"VpcId" : {"Ref" : "MyVpc"},
"SecurityGroupIngress" : [{
"CidrIp" : "0.0.0.0/0",
"FromPort" : "22",
"IpProtocol" : "tcp",
"ToPort" : "22"
}]
}
},
"Server" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"ImageId" : "ami-1853ac65",
"InstanceType" : {"Ref" : "InstanceType"},
"KeyName" : {"Ref" : "KeyName"},
"SecurityGroupIds" : {"Ref" : "SecurityGroup"},
"SubnetId" : {"Ref" : "PublicSubnet"}
}
}
},
"Outputs" : {
"PublicName" : {
"Value" : {"Fn::GetAtt" : ["Server", "PublicDnsName"]},
"Description" : "Public Name (connect via ssh)"
}
}
}
Can you please help me to find out What I am doing wrong?
You are creating VPC and public subnet under key Parameters. You need to define vpc and subnet under key resources. This should work:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "This is an AWS Cloud Formation template to create an EC2 instance in a Custom VPC.",
"Parameters": {
"KeyName": {
"Type": "String",
"Default": "ec2-us-east",
"Description": "SSH Key to access the EC2 instance"
},
"InstanceType": {
"Type": "String",
"Default": "t2.micro",
"Description": "Select EC2 instance type"
}
},
"Resources": {
"SecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupName": "My Security Group",
"GroupDescription": "My CFSecurity Group",
"VpcId": {
"Ref": "MyVpc"
},
"SecurityGroupIngress": [{
"CidrIp": "0.0.0.0/0",
"FromPort": "22",
"IpProtocol": "tcp",
"ToPort": "22"
}]
}
},
"Server": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": "ami-1853ac65",
"InstanceType": {
"Ref": "InstanceType"
},
"KeyName": {
"Ref": "KeyName"
},
"SecurityGroupIds": {
"Ref": "SecurityGroup"
},
"SubnetId": {
"Ref": "PublicSubnet"
}
}
},
"MyVpc": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.0.0.0/16",
"EnableDnsHostnames": "true"
}
},
"PublicSubnet": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Ref": "MyVpc"
},
"CidrBlock": "10.0.0.0/24",
"AvailabilityZone": "us-east-1a"
}
}
},
"Outputs": {
"PublicName": {
"Value": {
"Fn::GetAtt": ["Server",
"PublicDnsName"]
},
"Description": "Public Name (connect via ssh)"
}
}
}

How can I let my VPC have access to the internet via cloudformation?

I have a VPC setup to that my lambda function can talk to my RDS server. This is working. I also need my lambda functions to have access to the internet. To this end I'm trying to setup an internet gateway and the routes to allow it. I'm failing.
The VPC routes and gateway are created as the following
"VPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.0.0.0/16",
"EnableDnsSupport": "true",
"EnableDnsHostnames": "true",
"InstanceTenancy": "default",
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
"VPCRouteTable" : {
"Type" : "AWS::EC2::RouteTable",
"Properties" : {
"VpcId" : { "Ref" : "VPC" },
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
"InternetGateway" : {
"Type" : "AWS::EC2::InternetGateway",
"Properties" : {
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
"AttachGateway": {
"Type" : "AWS::EC2::VPCGatewayAttachment",
"Properties" : {
"VpcId" : { "Ref" : "VPC" },
"InternetGatewayId" : { "Ref" : "InternetGateway" }
}
},
"InternetRoute" : {
"Type" : "AWS::EC2::Route",
"DependsOn" : "InternetGateway",
"Properties" : {
"RouteTableId" : { "Ref" : "VPCRouteTable" },
"DestinationCidrBlock" : "0.0.0.0/0",
"GatewayId" : { "Ref" : "InternetGateway" }
}
},
I create the subnets and associate them with the route table
"SubnetA": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": { "Ref": "VPC" },
"CidrBlock": "10.0.0.0/24",
"AvailabilityZone": { "Fn::Select": [ "0", { "Fn::GetAZs": { "Ref": "AWS::Region" } }]},
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
"SubnetB": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": { "Ref": "VPC" },
"CidrBlock": "10.0.1.0/24",
"AvailabilityZone": { "Fn::Select": [ "1", { "Fn::GetAZs": { "Ref": "AWS::Region" } }]},
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
"SubnetARouteTableAssociation" : {
"Type" : "AWS::EC2::SubnetRouteTableAssociation",
"Properties" : {
"SubnetId" : { "Ref" : "SubnetA" },
"RouteTableId" : { "Ref" : "VPCRouteTable" }
}
},
"SubnetBRouteTableAssociation" : {
"Type" : "AWS::EC2::SubnetRouteTableAssociation",
"Properties" : {
"SubnetId" : { "Ref" : "SubnetB" },
"RouteTableId" : { "Ref" : "VPCRouteTable" }
}
},
I have the database security groups
"DBSubnetGroup": {
"Type": "AWS::RDS::DBSubnetGroup",
"Properties": {
"DBSubnetGroupDescription": "Database Access",
"SubnetIds" : [{ "Ref": "SubnetA" }, { "Ref": "SubnetB" }],
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
"DBEC2SecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Security group for RDS DB Instance",
"VpcId": {"Ref": "VPC"},
"SecurityGroupIngress" : [{
"IpProtocol": "tcp",
"FromPort": "3306",
"ToPort": "3306",
"CidrIp": "10.0.0.0/16"
}],
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
and the lambda security group
"LambdaSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Security group for Lambda",
"VpcId": {"Ref": "VPC"},
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
So As it stands now, my lambda's can talk to the database just fine. but they can't reach the internet. What am I missing?
If your lambda function needs to have access to both your VPC resources and Internet, then create 2 subnets: public and private. Put your lambda in private subnet and configure NAT in public subnet.
From http://docs.aws.amazon.com/lambda/latest/dg/vpc.html
Therefore, if your Lambda function requires Internet access (for
example, to access AWS services that don't have VPC endpoints, such as
Amazon Kinesis), you can configure a NAT instance inside your VPC or
you can use the Amazon VPC NAT gateway. For more information, see NAT
Gateways in the Amazon VPC User Guide.

How to add a RDS instance to a VPC using aws cloudformation

When I launch a RDS instance manually I'm able to assign what VPC I want it to be part of. I'm trying to create a stack using AWS cloudformation, however I do not see an API to be able to do that. I can create my VPC in the stack and then reference it for security groups both EC2 and DB security groups and they both end up been part of the VPC however the RDS instance itself does not. Is there a way to assign the VPC to the RDS instance?
Below is my template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Metadata": {
"AWS::CloudFormation::Designer": {
"30e03bfc-b61a-4d6c-89db-1b62b258a305": {
"size": {
"width": 80,
"height": 80
},
"position": {
"x": 700,
"y": 170
},
"z": 0,
"embeds": []
}
}
},
"Parameters": {
"DBPreferredBkupWindow": {
"Description" : "The daily time range (in UTC) during which automated backups are created, ideally off peak-hours.",
"Type" : "String",
"MinLength" : "1",
"MaxLength" : "11",
"AllowedPattern" : "\\d[0-23]:\\d[0-59]-\\d[0-23]:\\d[0-59]",
"Default" : "01:00-02:00"
}
},
"Resources": {
"VPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock" : "172.16.0.0/16",
"EnableDnsSupport" : true
}
},
"DB": {
"Type": "AWS::RDS::DBInstance",
"Properties": {
"DBName" : "ems",
"Engine" : "postgres",
"EngineVersion" : "9.4.7",
"DBInstanceClass" : "db.t1.micro",
"DBInstanceIdentifier" : "rltdb",
"MasterUsername" : "pgadmin",
"MasterUserPassword" : "pgadmin1",
"AllocatedStorage" : "100",
"Iops" : "1000",
"BackupRetentionPeriod" : "7",
"PreferredBackupWindow" : { "Ref" : "DBPreferredBkupWindow" },
"MultiAZ" : true,
"PubliclyAccessible" : false,
"AutoMinorVersionUpgrade" : false,
"VPCSecurityGroups" : [{ "Ref" : "SecurityGroup" } ]
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "30e03bfc-b61a-4d6c-89db-1b62b258a305"
}
}
},
"DBSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"EC2VpcId" : { "Ref" : "VPC" },
"DBSecurityGroupIngress" : { "EC2SecurityGroupName": { "Ref": "SecurityGroup"} },
"GroupDescription" : "Database Access"
}
},
"SecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"VpcId" : { "Ref" : "VPC" },
"GroupDescription" : "Enable database access for application",
"SecurityGroupIngress" : [
{"IpProtocol" : "tcp", "FromPort" : "5432", "ToPort" : "5432", "CidrIp" : "0.0.0.0/0"}
]
}
}
}
}
You have to create a DBSubnetGroup and at least two subnets in your CloudFormation template.
"subnet-1" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"CidrBlock" : "172.16.1.0/24",
"VpcId" : { "Ref" : "VPC" }
}
},
"subnet-2" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"CidrBlock" : "172.16.2.0/24",
"VpcId" : { "Ref" : "VPC" }
}
},
"DBSubnetGroup" : {
"Type" : "AWS::RDS::DBSubnetGroup",
"Properties" : {
"SubnetIds" : [
{ "Ref" : "subnet-1" },
{ "Ref" : "subnet-2" }
],
}
},
and in last you have to include DBSubnetGroup in your "DB" Object.
"DBSubnetGroupName": { "Ref": "DBSubnetGroup" }
You need to include the DBSubnetGroupName:
A DB subnet group to associate with the DB instance.
If there is no DB subnet group, then it is a non-VPC DB instance.
Create a DBSubnetGroup resource using subnets in your VPC, then tie that to your DBInstance:
"DBSubnetGroupName": { "Ref": "MySubnetGroup" }