I want to make RDS and proxy with credential
However, I bumped into this error.
14:32:32 | CREATE_FAILED | AWS::RDS::DBCluster | DatabaseB269D8BB
Could not find a value associated with JSONKey in SecretString
My script is like this below.
const rdsCredentials: rds.Credentials = rds.Credentials.fromGeneratedSecret(dbInfos['user'],{secretName:`cdk-st-${targetEnv}-db-secret`});
const dbCluster = new rds.DatabaseCluster(this, 'Database', {
parameterGroup,
engine: rds.DatabaseClusterEngine.auroraMysql({ version: rds.AuroraMysqlEngineVersion.VER_2_08_1 }),
credentials: rdsCredentials,
cloudwatchLogsExports:['slowquery','general','error','audit'],
backup: backupProps,
instances:2,
removalPolicy: cdk.RemovalPolicy.DESTROY,
clusterIdentifier: dbInfos['cluster'], //clusterIdentifier,
defaultDatabaseName :dbInfos['database'], //defaultDatabaseName,
instanceProps: {
// optional , defaults to t3.medium
instanceType: ec2.InstanceType.of(ec2.InstanceClass.BURSTABLE2, ec2.InstanceSize.SMALL),
vpcSubnets: {
subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
},
vpc,
securityGroups:[dbSecurityGroup],
},
});
const proxy = new rds.DatabaseProxy(this, 'Proxy', {
proxyTarget: rds.ProxyTarget.fromCluster(dbCluster),
secrets: [dbCluster.secret!],
vpc,
});
I guess this error is related to secrets: [dbCluster.secret!] maybe.
I googled around and found this error happens when secrets are deleted.
However I want to use credential which is just generated for RDS
Is it impossible?
how can I fix this?
More Test
I tried another way but this comes the error below
/node_modules/aws-cdk-lib/aws-rds/lib/proxy.ts:239
secret.grantRead(role);
my code is here
dbCluster.addProxy('testProxy',{
secrets: [rdsCredentials.secret!],
vpc
});
Related
So I have a basic VPC setup on CDK, after running cdk synth I see that an AWS::EC2::InternetGateway construct has been generated in my template under cdk.out. The problem is that I need to export this resource to another stack, but I can't reference this.vpc.internetGatewayId in my main .ts cdk file to export.
I know that this resource has been created, so I won't use CfnInternetGateway to create another, but at the same time this.vpc.internetGatewayId seems to have returned undefined even though I have a public subnet created and the resulting resource is in the template file.
Below is my VPC construct:
this.vpc = new ec2.Vpc(this, 'VPC', {
cidr: CONFIG.REGIONAL_CONFIG[region].VPC_CIDR,
maxAzs: CONFIG.REGIONAL_CONFIG[region].AVAILABILITY_ZONES,
subnetConfiguration: [
{
name: 'public',
subnetType: ec2.SubnetType.PUBLIC,
},
{
name: 'private',
subnetType: ec2.SubnetType.PRIVATE_WITH_NAT,
},
{
name: 'isolated',
subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
},
],
});
Create a CloudFormation output on the exporting stack with with a CfnOutput:
// MyExportingStack.ts
// Cause the synth to fail if the ID is undefined. Alternatively, throw an error.
if (!vpc.internetGatewayId)
cdk.Annotations.of(this).addError('An Internet Gateway ID is required!');
// Create the CloudFormation Output
new cdk.CfnOutput(this, 'IgwId', { value: vpc.internetGatewayId ?? 'MISSING', exportName: 'my-igw-id',});
I want to create a DB Read Replica using aws CDK. I am creating a RDS DatabaseInstance using method DatabaseInstanceFromSnapshot in aws CDK. I want to use this RDS instance to create a Read Replica using DatabaseInstanceReadReplica method. DatabaseInstanceReadReplica takes a sourceDatabaseInstance as a parameter. For sourceDatabaseInstance I am passing the DatabaseInstance returned from DatabaseInstanceFromSnapshot method. But I get the following error:
Type 'DatabaseInstanceReadReplica' is missing the following properties from type 'DatabaseInstance': sourceCfnProps, singleUserRotationApplication, multiUserRotationApplication, addRotationSingleUser, addRotationMultiUsers(2739)
How do I fix this issue? Any help is appreciated. Below is the code.
mySqlRdsInstance: DatabaseInstance
mySqlRdsReplicaInstance: DatabaseInstance
this.mySqlRdsInstance = new DatabaseInstanceFromSnapshot(this, props.rdsParameters.instanceName, {
instanceIdentifier: props.rdsParameters.instanceIdentifier,
snapshotIdentifier: props.rdsParameters.snapshotIdentifier || '',
engine: DatabaseInstanceEngine.MYSQL,
vpc: props.vpc,
vpcSubnets: {
subnetType: ec2.SubnetType.PUBLIC,
},
})
this.mySqlRdsReplicaInstance = new DatabaseInstanceReadReplica(this, "", {
sourceDatabaseInstance: this.mySqlRdsInstance,
instanceType: ec2.InstanceType.of(ec2.InstanceClass.BURSTABLE2, ec2.InstanceSize.LARGE),
vpc: props.vpc,
vpcSubnets: {
subnetType: ec2.SubnetType.PUBLIC,
},
})
I'm working through setting up a new infrastructure with the AWS CDK and I'm trying to get a TypeScript app running in Fargate to be able to read/write from/to a DynamoDB table, but am hitting IAM issues.
I have both my Fargate service and my DynamoDB Table defined, and both are running as they should be in AWS, but whenever I attempt to write to the table from my app, I am getting an access denied error.
I've tried the solutions defined in this post, as well as the ones it links to, but nothing seems to be allowing my container to write to the table. I've tried everything from setting table.grantReadWriteData(fargateService.taskDefinition.taskRole) to the more complex solutions described in the linked articles of defining my own IAM policies and setting the effects and actions, but I always just get the same access denied error when attempting to do a putItem:
AccessDeniedException: User: {fargate-service-arn} is not authorized to perform: dynamodb:PutItem on resource: {dynamodb-table} because no identity-based policy allows the dynamodb:PutItem action
Am I missing something, or a crucial step to make this possible?
Any help is greatly appreciated.
Thanks!
Edit (2022-09-19):
Here is the boiled down code for how I'm defining my Vpc, Cluster, Container Image, FargateService, and Table.
export class FooCdkStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const vpc = new Vpc(this, 'FooVpc', {
maxAzs: 2,
natGateways: 1
});
const cluster = new Cluster(this, 'FooCluster', { vpc });
const containerImage = ContainerImage.fromAsset(
path.join(__dirname, '/../app'),
{
platform: Platform.LINUX_AMD64 // I'm on an M1 Mac and images weren't working appropriately without this
}
);
const fargateService = new ApplicationLoadBalancedFargateService(
this,
'FooFargateService',
{
assignPublicIp: true,
cluster,
memoryLimitMiB: 1024,
cpu: 512,
desiredCount: 1,
taskImageOptions: {
containerPort: PORT,
image: containerImage
}
}
);
fargateService.targetGroup.configureHealthCheck({ path: '/health' });
const serverTable = new Table(this, 'FooTable', {
billingMode: BillingMode.PAY_PER_REQUEST,
removalPolicy: cdk.RemovalPolicy.DESTROY,
partitionKey: { name: 'id', type: AttributeType.STRING },
pointInTimeRecovery: true
});
serverTable.grantReadWriteData(fargateService.taskDefinition.taskRole);
}
}
Apparently either order in which the resources are defined matters, or the inclusion of a property from the table in the Fargate service is what did the trick. I moved the table definition up above the Fargate service and included an environment variable holding the table name and it's working as intended now.
export class FooCdkStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const vpc = new Vpc(this, 'FooVpc', {
maxAzs: 2,
natGateways: 1
});
const cluster = new Cluster(this, 'FooCluster', { vpc });
const containerImage = ContainerImage.fromAsset(
path.join(__dirname, '/../app'),
{
platform: Platform.LINUX_AMD64 // I'm on an M1 Mac and images weren't working appropriately without this
}
);
const serverTable = new Table(this, 'FooTable', {
billingMode: BillingMode.PAY_PER_REQUEST,
removalPolicy: cdk.RemovalPolicy.DESTROY,
partitionKey: { name: 'id', type: AttributeType.STRING },
pointInTimeRecovery: true
});
const fargateService = new ApplicationLoadBalancedFargateService(
this,
'FooFargateService',
{
assignPublicIp: true,
cluster,
memoryLimitMiB: 1024,
cpu: 512,
desiredCount: 1,
taskImageOptions: {
containerPort: PORT,
image: containerImage,
environment: {
SERVER_TABLE_NAME: serverTable.tableName
}
}
}
);
fargateService.targetGroup.configureHealthCheck({ path: '/health' });
serverTable.grantReadWriteData(fargateService.taskDefinition.taskRole);
}
}
Hopefully this helps someone in the future who may come across the same issue.
I have a CDK project that creates a CodePipeline which deploys an application on ECS. I had it all previously working, but the VPC was using a NAT gateway, which ended up being too expensive. So now I am trying to recreate the project without requiring a NAT gateway. I am almost there, but I have now run into issues when the ECS service is trying to start tasks. All tasks fail to start with the following error:
ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve secret from asm: service call has been retried 5 time(s): failed to fetch secret
At this point I've kind of lost track of the different things I have tried, but I will post the relevant bits here as well as some of my attempts.
const repository = ECR.Repository.fromRepositoryAttributes(
this,
"ecr-repository",
{
repositoryArn: props.repository.arn,
repositoryName: props.repository.name,
}
);
// vpc
const vpc = new EC2.Vpc(this, this.resourceName(props, "vpc"), {
maxAzs: 2,
natGateways: 0,
enableDnsSupport: true,
});
const vpcSecurityGroup = new SecurityGroup(this, "vpc-security-group", {
vpc: vpc,
allowAllOutbound: true,
});
// tried this to allow the task to access secrets manager
const vpcEndpoint = new EC2.InterfaceVpcEndpoint(this, "secrets-manager-task-vpc-endpoint", {
vpc: vpc,
service: EC2.InterfaceVpcEndpointAwsService.SSM,
});
const secrets = SecretsManager.Secret.fromSecretCompleteArn(
this,
"secrets",
props.secrets.arn
);
const cluster = new ECS.Cluster(this, this.resourceName(props, "cluster"), {
vpc: vpc,
clusterName: `api-cluster`,
});
const ecsService = new EcsPatterns.ApplicationLoadBalancedFargateService(
this,
"ecs-service",
{
taskSubnets: {
subnetType: SubnetType.PUBLIC,
},
securityGroups: [vpcSecurityGroup],
serviceName: "api-service",
cluster: cluster,
cpu: 256,
desiredCount: props.scaling.desiredCount,
taskImageOptions: {
image: ECS.ContainerImage.fromEcrRepository(
repository,
this.ecrTagNameParameter.stringValue
),
secrets: getApplicationSecrets(secrets), // returns
logDriver: LogDriver.awsLogs({
streamPrefix: "api",
logGroup: new LogGroup(this, "ecs-task-log-group", {
logGroupName: `${props.environment}-api`,
}),
logRetention: RetentionDays.TWO_MONTHS,
}),
},
memoryLimitMiB: 512,
publicLoadBalancer: true,
domainZone: this.hostedZone,
certificate: this.certificate,
redirectHTTP: true,
}
);
const scalableTarget = ecsService.service.autoScaleTaskCount({
minCapacity: props.scaling.desiredCount,
maxCapacity: props.scaling.maxCount,
});
scalableTarget.scaleOnCpuUtilization("cpu-scaling", {
targetUtilizationPercent: props.scaling.cpuPercentage,
});
scalableTarget.scaleOnMemoryUtilization("memory-scaling", {
targetUtilizationPercent: props.scaling.memoryPercentage,
});
secrets.grantRead(ecsService.taskDefinition.taskRole);
repository.grantPull(ecsService.taskDefinition.taskRole);
I read somewhere that it probably has something to do with Fargate version 1.4.0 vs 1.3.0, but I'm not sure what I need to change to allow the tasks to access what they need to run.
You need to create an interface endpoints for Secrets Manager, ECR (two types of endpoints), CloudWatch, as well as a gateway endpoint for S3.
Refer to the documentation on the topic.
Here's an example in Python, it'd work the same in TS:
vpc.add_interface_endpoint(
"secretsmanager_endpoint",
service=ec2.InterfaceVpcEndpointAwsService.SECRETS_MANAGER,
)
vpc.add_interface_endpoint(
"ecr_docker_endpoint",
service=ec2.InterfaceVpcEndpointAwsService.ECR_DOCKER,
)
vpc.add_interface_endpoint(
"ecr_endpoint",
service=ec2.InterfaceVpcEndpointAwsService.ECR,
)
vpc.add_interface_endpoint(
"cloudwatch_logs_endpoint",
service=ec2.InterfaceVpcEndpointAwsService.CLOUDWATCH_LOGS,
)
vpc.add_gateway_endpoint(
"s3_endpoint",
service=ec2.GatewayVpcEndpointAwsService.S3
)
Keep in mind that interface endpoints cost money as well, and may not be cheaper than a NAT.
I'm trying to use Pulumi within a somewhat restricted AWS environment.
This sandbox requires that I use a specific VPC, and there is no default VPC.
I have tried the examples showing how to reference an existing VPC, but they all fail with some variant of "invoking aws:ec2/getVpc:getVpc: no matching VPC found"
#pulumi/awsx, using code referenced from: https://github.com/pulumi/pulumi-awsx/issues/522:
const vpc = awsx.ec2.Vpc.fromExistingIds('name', {
vpcId: 'id',
publicSubnetIds: ['a', 'b'],
privateSubnetIds: ['a', 'b']
})
#pulumi/aws, using code referenced from https://www.pulumi.com/docs/reference/pkg/aws/ec2/getvpc/:
const vpc = aws.ec2.Vpc.get('vpc-1', 'vpc-1')
Question: what is the correct and complete syntax for referencing an existing VPC within a Pulumi stack?
Note that I would rather not "adopt" this resource as it is shared and the user running the pulumi up command does not have permission to delete VPC resources.
There is a subtle difference between getVpc() that you linked to and Vpc.get() that you tried using. You should use the former:
const vpc = aws.ec2.getVpc({ id: yourVpcId });
This is what worked in the end:
const vpc = aws.ec2.Vpc.get('vpc-123', 'vpc-123')
I don't think I had saved my file correctly before pulumi up after making the above change.
Note that I also had to add subnets manually to my ALB to get this working, as below:
const vpc = aws.ec2.Vpc.get('vpc-123', 'vpc-123')
const clusterName = nameResource('graphQlServiceCluster')
const ecsCluster = new awsx.ecs.Cluster(clusterName, {
name: clusterName,
vpc
})
const PublicSubnet1a = 'subnet-123'
const PublicSubnet1b = 'subnet-123'
const alb = new awsx.lb.ApplicationLoadBalancer(nameResource('graphQlServiceElb'), {
name: nameResource('graphQlServiceElb'),
external: true,
vpc,
subnets: [
PublicSubnet1a,
PublicSubnet1b
]
})
const listener = alb.createListener(nameResource('graphqlServiceListener'), {
name: nameResource('graphqlServiceListener'),
port: 80,
external: true,
vpc
})
Pulumi has multiple Vpc types. You probably want to use the awsx VPC as it's higher level (and required to use other awsx infrastructure).
There's two ways to do this:
Creating a new VPC
const vpc = new awsx.ec2.Vpc(config.vpcName, {
cidrBlock: "10.0.0.0/16",
subnets: [
{
name: "public",
type: "public",
location: {
cidrBlock: "10.0.0.0/24",
availabilityZone: "us-east-2a",
},
},
{
name: "private-a",
type: "private",
location: {
cidrBlock: "10.0.1.0/24",
availabilityZone: "us-east-2a",
},
},
{
name: "private-b",
type: "private",
location: {
cidrBlock: "10.0.2.0/24",
availabilityZone: "us-east-2b",
},
},
],
});
Using an existing VPC
Borrowing from this GitHub thread with the Pulumi CTO produced a correct result:
const vpc = awsx.ec2.Vpc.fromExistingIds("mycompany", {
vpcId: "vpc-myvpcid",
});
// Create an ECS Fargate cluster.
const ecsCluster = new awsx.ecs.Cluster("mycompany-pulumi-cluster", {
vpc,
});