I have the following code to create a ECS ApplicationLoadBalancedEc2Service, however it is stuck in creation for 2 hours and i don't see any errors in events.
Below is my code:
this.cluster = new Cluster(this, 'Cluster', {
vpc: props.vpc
});
this.cluster.addCapacity('DefaultAutoScalingGroupCapacity', {
instanceType: InstanceType.of(InstanceClass.R5D, InstanceSize.XLARGE24),
minCapacity: 2,
maxCapacity: 50,
});
this.service = new ApplicationLoadBalancedEc2Service(this, 'Service', {
cluster: props.ecsCluster,
memoryLimitMiB: 768000,
taskImageOptions: {
containerPort: 8080,
image: new ContainerImage({
package: Package.fromString('ECSMatching'),
transformPackage: Package.fromString('ECSMatchingImage'),
componentName: 'service',
}),
taskRole: getDefaultEcsTaskInstanceRole(this),
environment: {'STAGE': props.stage}
},
});
this.service.service.connections.allowFrom(
Peer.ipv4(props.ecsCluster.vpc.vpcCidrBlock),
Port.allTraffic(),
'Local VPC Access'
);
this.service.targetGroup.setAttribute('deregistration_delay.timeout_seconds', '6000');
Related
I've got the following Fargate service created by an ecs pattern. The CloudMap I create here only points to the underlying task which is a private IP and runs on port 8080 (Tomcat). The ALB forwards properly from 80->8080. How can I get the DNS to properly route to the task? Can I get the DNS service to route directly to the ALB?
const service = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'MyAppWebstartFargateService', {
serviceName: "myapp-service",
cluster: cluster,
cpu: 512,
memoryLimitMiB: 2048,
cloudMapOptions: {
name: "myapp",
containerPort: 8080,
cloudMapNamespace: namespace,
dnsRecordType: svc_dsc.DnsRecordType.A,
dnsTtl: Duration.seconds(300),
},
desiredCount: 1,
publicLoadBalancer: false,
securityGroups: [sg],
listenerPort: 80,
openListener: true,
healthCheckGracePeriod: Duration.seconds(300),
targetProtocol: elbv2.ApplicationProtocol.HTTP,
protocol: elbv2.ApplicationProtocol.HTTP,
enableExecuteCommand: true,
taskImageOptions: {
containerName: "myapp-container",
containerPort: 8080,
enableLogging: true,
image: ecs.ContainerImage.fromEcrRepository(repository, "latest"),
},
});
I figured it out! I needed to call registerLoadBalancer on the Cloud Map service and give it the resulting LB from the Fargate pattern. Hope this helps someone down the road b/c I could not find any solution to this exact use case.
const namespace = svc_dsc.PrivateDnsNamespace.fromPrivateDnsNamespaceAttributes(this, "MyAppCloudMapNamespace", {
namespaceArn: "*****************",
namespaceId: "999999999999999",
namespaceName: "mydomain.com"
});
const mapService = new svc_dsc.Service(this, 'MyAppCloudMapService', {
namespace: namespace,
dnsRecordType: svc_dsc.DnsRecordType.A,
dnsTtl: Duration.seconds(300),
name: "myapp",
routingPolicy: svc_dsc.RoutingPolicy.WEIGHTED,
loadBalancer: true // Important! If you choose WEIGHTED but don't set this, the routing policy will default to MULTIVALUE instead
});
const service = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'MyAppWebstartFargateService', {
serviceName: "myapp-service",
cluster: cluster,
cpu: 512,
memoryLimitMiB: 2048,
desiredCount: 1,
publicLoadBalancer: false,
securityGroups: [sg],
listenerPort: 80,
openListener: true,
healthCheckGracePeriod: Duration.seconds(300),
targetProtocol: elbv2.ApplicationProtocol.HTTP,
protocol: elbv2.ApplicationProtocol.HTTP,
enableExecuteCommand: true,
taskImageOptions: {
containerName: "myapp-container",
containerPort: 8080,
enableLogging: true,
image: ecs.ContainerImage.fromEcrRepository(repository, "latest"),
},
});
mapService.registerLoadBalancer("MyAppLoadBalancer", service.loadBalancer);
I'm working through setting up a new infrastructure with the AWS CDK and I'm trying to get a TypeScript app running in Fargate to be able to read/write from/to a DynamoDB table, but am hitting IAM issues.
I have both my Fargate service and my DynamoDB Table defined, and both are running as they should be in AWS, but whenever I attempt to write to the table from my app, I am getting an access denied error.
I've tried the solutions defined in this post, as well as the ones it links to, but nothing seems to be allowing my container to write to the table. I've tried everything from setting table.grantReadWriteData(fargateService.taskDefinition.taskRole) to the more complex solutions described in the linked articles of defining my own IAM policies and setting the effects and actions, but I always just get the same access denied error when attempting to do a putItem:
AccessDeniedException: User: {fargate-service-arn} is not authorized to perform: dynamodb:PutItem on resource: {dynamodb-table} because no identity-based policy allows the dynamodb:PutItem action
Am I missing something, or a crucial step to make this possible?
Any help is greatly appreciated.
Thanks!
Edit (2022-09-19):
Here is the boiled down code for how I'm defining my Vpc, Cluster, Container Image, FargateService, and Table.
export class FooCdkStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const vpc = new Vpc(this, 'FooVpc', {
maxAzs: 2,
natGateways: 1
});
const cluster = new Cluster(this, 'FooCluster', { vpc });
const containerImage = ContainerImage.fromAsset(
path.join(__dirname, '/../app'),
{
platform: Platform.LINUX_AMD64 // I'm on an M1 Mac and images weren't working appropriately without this
}
);
const fargateService = new ApplicationLoadBalancedFargateService(
this,
'FooFargateService',
{
assignPublicIp: true,
cluster,
memoryLimitMiB: 1024,
cpu: 512,
desiredCount: 1,
taskImageOptions: {
containerPort: PORT,
image: containerImage
}
}
);
fargateService.targetGroup.configureHealthCheck({ path: '/health' });
const serverTable = new Table(this, 'FooTable', {
billingMode: BillingMode.PAY_PER_REQUEST,
removalPolicy: cdk.RemovalPolicy.DESTROY,
partitionKey: { name: 'id', type: AttributeType.STRING },
pointInTimeRecovery: true
});
serverTable.grantReadWriteData(fargateService.taskDefinition.taskRole);
}
}
Apparently either order in which the resources are defined matters, or the inclusion of a property from the table in the Fargate service is what did the trick. I moved the table definition up above the Fargate service and included an environment variable holding the table name and it's working as intended now.
export class FooCdkStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const vpc = new Vpc(this, 'FooVpc', {
maxAzs: 2,
natGateways: 1
});
const cluster = new Cluster(this, 'FooCluster', { vpc });
const containerImage = ContainerImage.fromAsset(
path.join(__dirname, '/../app'),
{
platform: Platform.LINUX_AMD64 // I'm on an M1 Mac and images weren't working appropriately without this
}
);
const serverTable = new Table(this, 'FooTable', {
billingMode: BillingMode.PAY_PER_REQUEST,
removalPolicy: cdk.RemovalPolicy.DESTROY,
partitionKey: { name: 'id', type: AttributeType.STRING },
pointInTimeRecovery: true
});
const fargateService = new ApplicationLoadBalancedFargateService(
this,
'FooFargateService',
{
assignPublicIp: true,
cluster,
memoryLimitMiB: 1024,
cpu: 512,
desiredCount: 1,
taskImageOptions: {
containerPort: PORT,
image: containerImage,
environment: {
SERVER_TABLE_NAME: serverTable.tableName
}
}
}
);
fargateService.targetGroup.configureHealthCheck({ path: '/health' });
serverTable.grantReadWriteData(fargateService.taskDefinition.taskRole);
}
}
Hopefully this helps someone in the future who may come across the same issue.
I want to make RDS and proxy with credential
However, I bumped into this error.
14:32:32 | CREATE_FAILED | AWS::RDS::DBCluster | DatabaseB269D8BB
Could not find a value associated with JSONKey in SecretString
My script is like this below.
const rdsCredentials: rds.Credentials = rds.Credentials.fromGeneratedSecret(dbInfos['user'],{secretName:`cdk-st-${targetEnv}-db-secret`});
const dbCluster = new rds.DatabaseCluster(this, 'Database', {
parameterGroup,
engine: rds.DatabaseClusterEngine.auroraMysql({ version: rds.AuroraMysqlEngineVersion.VER_2_08_1 }),
credentials: rdsCredentials,
cloudwatchLogsExports:['slowquery','general','error','audit'],
backup: backupProps,
instances:2,
removalPolicy: cdk.RemovalPolicy.DESTROY,
clusterIdentifier: dbInfos['cluster'], //clusterIdentifier,
defaultDatabaseName :dbInfos['database'], //defaultDatabaseName,
instanceProps: {
// optional , defaults to t3.medium
instanceType: ec2.InstanceType.of(ec2.InstanceClass.BURSTABLE2, ec2.InstanceSize.SMALL),
vpcSubnets: {
subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
},
vpc,
securityGroups:[dbSecurityGroup],
},
});
const proxy = new rds.DatabaseProxy(this, 'Proxy', {
proxyTarget: rds.ProxyTarget.fromCluster(dbCluster),
secrets: [dbCluster.secret!],
vpc,
});
I guess this error is related to secrets: [dbCluster.secret!] maybe.
I googled around and found this error happens when secrets are deleted.
However I want to use credential which is just generated for RDS
Is it impossible?
how can I fix this?
More Test
I tried another way but this comes the error below
/node_modules/aws-cdk-lib/aws-rds/lib/proxy.ts:239
secret.grantRead(role);
my code is here
dbCluster.addProxy('testProxy',{
secrets: [rdsCredentials.secret!],
vpc
});
I have a CDK project that creates a CodePipeline which deploys an application on ECS. I had it all previously working, but the VPC was using a NAT gateway, which ended up being too expensive. So now I am trying to recreate the project without requiring a NAT gateway. I am almost there, but I have now run into issues when the ECS service is trying to start tasks. All tasks fail to start with the following error:
ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve secret from asm: service call has been retried 5 time(s): failed to fetch secret
At this point I've kind of lost track of the different things I have tried, but I will post the relevant bits here as well as some of my attempts.
const repository = ECR.Repository.fromRepositoryAttributes(
this,
"ecr-repository",
{
repositoryArn: props.repository.arn,
repositoryName: props.repository.name,
}
);
// vpc
const vpc = new EC2.Vpc(this, this.resourceName(props, "vpc"), {
maxAzs: 2,
natGateways: 0,
enableDnsSupport: true,
});
const vpcSecurityGroup = new SecurityGroup(this, "vpc-security-group", {
vpc: vpc,
allowAllOutbound: true,
});
// tried this to allow the task to access secrets manager
const vpcEndpoint = new EC2.InterfaceVpcEndpoint(this, "secrets-manager-task-vpc-endpoint", {
vpc: vpc,
service: EC2.InterfaceVpcEndpointAwsService.SSM,
});
const secrets = SecretsManager.Secret.fromSecretCompleteArn(
this,
"secrets",
props.secrets.arn
);
const cluster = new ECS.Cluster(this, this.resourceName(props, "cluster"), {
vpc: vpc,
clusterName: `api-cluster`,
});
const ecsService = new EcsPatterns.ApplicationLoadBalancedFargateService(
this,
"ecs-service",
{
taskSubnets: {
subnetType: SubnetType.PUBLIC,
},
securityGroups: [vpcSecurityGroup],
serviceName: "api-service",
cluster: cluster,
cpu: 256,
desiredCount: props.scaling.desiredCount,
taskImageOptions: {
image: ECS.ContainerImage.fromEcrRepository(
repository,
this.ecrTagNameParameter.stringValue
),
secrets: getApplicationSecrets(secrets), // returns
logDriver: LogDriver.awsLogs({
streamPrefix: "api",
logGroup: new LogGroup(this, "ecs-task-log-group", {
logGroupName: `${props.environment}-api`,
}),
logRetention: RetentionDays.TWO_MONTHS,
}),
},
memoryLimitMiB: 512,
publicLoadBalancer: true,
domainZone: this.hostedZone,
certificate: this.certificate,
redirectHTTP: true,
}
);
const scalableTarget = ecsService.service.autoScaleTaskCount({
minCapacity: props.scaling.desiredCount,
maxCapacity: props.scaling.maxCount,
});
scalableTarget.scaleOnCpuUtilization("cpu-scaling", {
targetUtilizationPercent: props.scaling.cpuPercentage,
});
scalableTarget.scaleOnMemoryUtilization("memory-scaling", {
targetUtilizationPercent: props.scaling.memoryPercentage,
});
secrets.grantRead(ecsService.taskDefinition.taskRole);
repository.grantPull(ecsService.taskDefinition.taskRole);
I read somewhere that it probably has something to do with Fargate version 1.4.0 vs 1.3.0, but I'm not sure what I need to change to allow the tasks to access what they need to run.
You need to create an interface endpoints for Secrets Manager, ECR (two types of endpoints), CloudWatch, as well as a gateway endpoint for S3.
Refer to the documentation on the topic.
Here's an example in Python, it'd work the same in TS:
vpc.add_interface_endpoint(
"secretsmanager_endpoint",
service=ec2.InterfaceVpcEndpointAwsService.SECRETS_MANAGER,
)
vpc.add_interface_endpoint(
"ecr_docker_endpoint",
service=ec2.InterfaceVpcEndpointAwsService.ECR_DOCKER,
)
vpc.add_interface_endpoint(
"ecr_endpoint",
service=ec2.InterfaceVpcEndpointAwsService.ECR,
)
vpc.add_interface_endpoint(
"cloudwatch_logs_endpoint",
service=ec2.InterfaceVpcEndpointAwsService.CLOUDWATCH_LOGS,
)
vpc.add_gateway_endpoint(
"s3_endpoint",
service=ec2.GatewayVpcEndpointAwsService.S3
)
Keep in mind that interface endpoints cost money as well, and may not be cheaper than a NAT.
I've been trying de deploy a RestApiGateway with 20 lambda functions since a while now, and I keep getting the error message Subnet is out of IP addresses every time I try to deploy. Any idea why this may be happening?
Here's how my Vpc stack looks like:
this.vpc = new ec2.Vpc(this, 'vpc', {
cidr: '10.0.0.0/21',
maxAzs: this.azs,
natGateways: this.azs,
subnetConfiguration: [
{
subnetType: ec2.SubnetType.PUBLIC,
name: `${APP_PREFIX}-public-subnet`,
cidrMask: 24,
},
{
subnetType: ec2.SubnetType.PRIVATE,
name: `${APP_PREFIX}-private-subnet`,
cidrMask: 28,
},
],
});
...and here's the gateway:
const interfaceVpcEndpoint = vpc.addInterfaceEndpoint(`${APP_PREFIX}-endpoint`, {
service: ec2.InterfaceVpcEndpointAwsService.APIGATEWAY,
subnets: { subnetType: ec2.SubnetType.PRIVATE },
});
const restApiLogGroup = new logs.LogGroup(stack, `${APP_PREFIX}-log-group`, {
logGroupName: `${APP_PREFIX}-log-group`,
});
return new gateway.RestApi(stack, `${APP_PREFIX}-rest-api`, {
restApiName: `${APP_PREFIX}-rest-api`,
endpointConfiguration: {
types: [gateway.EndpointType.PRIVATE],
vpcEndpoints: [interfaceVpcEndpoint],
},
retainDeployments: true,
deployOptions: {
stageName: 'v2',
loggingLevel: gateway.MethodLoggingLevel.INFO,
dataTraceEnabled: true,
accessLogDestination: new gateway.LogGroupLogDestination(restApiLogGroup),
accessLogFormat: gateway.AccessLogFormat.jsonWithStandardFields(),
},
policy: new iam.PolicyDocument({
statements: [
new iam.PolicyStatement({
effect: iam.Effect.DENY,
principals: [new iam.AnyPrincipal()],
actions: ['execute-api:Invoke'],
resources: ['execute-api:/*/POST/*'],
conditions: {
StringNotEquals: {
'aws:sourceVpc': vpc.vpcId,
},
},
}),
new iam.PolicyStatement({
effect: iam.Effect.ALLOW,
principals: [new iam.AnyPrincipal()],
actions: ['execute-api:Invoke'],
resources: ['execute-api:/*/POST/*'],
}),
],
}),
});
The idea is that the lambda functions should not be accessible from the internet, only from a Fargate service that I will deploy via another stack.
I think this is the problem:
cidrMask: 28
A /28 network allows 16 IP addresses, and AWS takes 3 of those addresses for itself.
I'm not familiar with the CDK classes to create a subnet, so can't say whether the overall approach that you're using is correct. I'm assuming that it's smart enough to break the VPC CIDR into subnet blocks based on what you provide for a mask.
I recommend giving at least a /24 allocation to each subnet; I prefer a /20. And a /16 to the VPC. Network configuration is free when you start, extremely expensive to correct later.
I ended up explicitly adding the default security group created by the VPC Construct to each Lambda and that worked.