How to import existing VPC in aws cdk? - amazon-web-services

Hi I am working on aws cdk. I am trying to get existing non-default vpc. I tried below options.
vpc = ec2.Vpc.from_lookup(self, id = "VPC", vpc_id='vpcid', vpc_name='vpc-dev')
This results in below error
[Error at /LocationCdkStack-cdkstack] Request has expired.
[Warning at /LocationCdkStack-cdkstack/TaskDef/mw-service] Proper policies need to be attached before pulling from ECR repository, or use 'fromEcrRepository'.
Found errors
Other method I tried is
vpc = ec2.Vpc.from_vpc_attributes(self, 'VPC', vpc_id='vpc-839227e7', availability_zones=['ap-southeast-2a','ap-southeast-2b','ap-southeast-2c'])
This results in
[Error at /LocationCdkStack-cdkstack] Request has expired.
[Warning at /LocationCdkStack-cdkstack/TaskDef/mw-service] Proper policies need to be attached before pulling from ECR repository, or use 'fromEcrRepository'.
Found errors
Other method I tried is
vpc = ec2.Vpc.from_lookup(self, id = "VPC", is_default=True) // This will get default vpc and this will work
Can someone help me to get non-default vpc in aws cdk? Any help would be appreciated. Thanks

Take a look at aws_cdk.aws_ec2 documentation and at CDK Runtime Context.
If your VPC is created outside your CDK app, you can use
Vpc.fromLookup(). The CDK CLI will search for the specified VPC in the
the stack’s region and account, and import the subnet configuration.
Looking up can be done by VPC ID, but more flexibly by searching for a
specific tag on the VPC.
Usage:
# Example automatically generated. See https://github.com/aws/jsii/issues/826
from aws_cdk.core import App, Stack, Environment
from aws_cdk import aws_ec2 as ec2
# Information from environment is used to get context information
# so it has to be defined for the stack
stack = MyStack(
app, "MyStack", env=Environment(account="account_id", region="region")
)
# Retrieve VPC information
vpc = ec2.Vpc.from_lookup(stack, "VPC",
# This imports the default VPC but you can also
# specify a 'vpcName' or 'tags'.
is_default=True
)
Update with a relevant example:
vpc = ec2.Vpc.from_lookup(stack, "VPC",
vpc_id = VPC_ID
)
Update with typescript example:
import ec2 = require('#aws-cdk/aws-ec2');
const getExistingVpc = ec2.Vpc.fromLookup(this, 'ImportVPC',{isDefault: true});
More info here.

For AWS CDK v2 or v1(latest), You can use:
// You can either use vpcId OR vpcName and fetch the desired vpc
const getExistingVpc = ec2.Vpc.fromLookup(this, 'ImportVPC',{
vpcId: "VPC_ID",
vpcName: "VPC_NAME"
});

here is simple example
//get VPC Info form AWS account, FYI we are not rebuilding we are referencing
const DefaultVpc = Vpc.fromVpcAttributes(this, 'vpcdev', {
vpcId:'vpc-d0e0000b0',
availabilityZones: core.Fn.getAzs(),
privateSubnetIds: 'subnet-00a0de00',
publicSubnetIds: 'subnet-00a0de00'
});
const yourService = new lambda.Function(this, 'SomeName', {
code: lambda.Code.fromAsset("lambda"),
handler: 'handlers.your_handler',
role: lambdaExecutionRole,
securityGroup: lambdaSecurityGroup,
vpc: DefaultVpc,
runtime: lambda.Runtime.PYTHON_3_7,
timeout: Duration.minutes(2),
});

We can do it easily using ec2.vpc.fromLookup.
https://kuchbhilearning.blogspot.com/2022/10/httpskuchbhilearning.blogspot.comimport-existing-vpc-in-aws-cdk.html
The following dictates how to use the method.

Related

How to add AWS Lambda to VPC using javascript

I need to add my lambda function to VPC but using javascript code - no from the aws console.
I'm creating my lambda in AWS CDK stack like that:
const myLambda = new lambda.Function(this, lambda-id, {
code: code,
handler: handler,
runtime: runtime,
...
**vpc**:
})
I think I need to pass VPC as an argument to this function. Is there any way to fetch this VPC by its id and then pass it as argument to this function?
You can import an existing VPC by ID and provide it as an attribute on your lambda like:
const vpc = ec2.Vpc.fromLookup(this, 'VPC', {
vpcId: 'your vpc id'
})
const myLambda = new lambda.Function(this, 'your-lambda', {
code,
handler,
runtime,
...,
vpc
})

RDS issues with custom VPC Subnets

I am trying to use CDK to define a Serverless Postgres Aurora cluster but keep running into issues with regards to the VPC subnets either being "invalid" or "not existing", depending on which db cluster construct I attempt to use. In my setup, I have 2 Stacks: 1 for the VPC, and 1 for the RDS.
This is the contents of my Vpc Stack:
const vpc = new Vpc(this, 'Vpc');
const privateSubnetIds = vpc.selectSubnets({
subnetType: SubnetType.PRIVATE
}).subnetIds;
const rdsSecurityGroup = new SecurityGroup(this, 'RdsSecurityGroup', {
securityGroupName: 'rds-security-group',
allowAllOutbound: true,
description: `RDS cluster security group`,
vpc: vpc
});
...
// The rest of the file defines exports.
Case 1:
Initially, I tried using the CfnDBCluster as the DatabaseCluster does not allow you to directly define engineMode: 'serverless' and enableHttpEndpoint: true. Below is the contents of the RDS Stack using the CfnDBCluster construct:
// The beginning of the file imports all the VPC exports from the VPC Stack:
// subnetIds (for the private subnet), securityGroupId
...
const databaseSecret = new DatabaseSecret(this, 'secret', {
username: 'admin'
});
const secretArn = databaseSecret.secretArn;
const dbSubnetGroup = new CfnDBSubnetGroup(this, "DbSubnetGroup", {
dbSubnetGroupDescription: `Database cluster subnet group`,
subnetIds: subnetIds
});
const dbCluster = new CfnDBCluster(this, 'DbCluster', {
dbClusterIdentifier: 'aurora-cluster',
engine: 'aurora-postgresql',
engineMode: 'serverless',
databaseName: DB_NAME,
masterUsername: databaseSecret.secretValueFromJson('username').toString(),
masterUserPassword: databaseSecret.secretValueFromJson('password').toString(),
enableHttpEndpoint: true,
scalingConfiguration: {
autoPause: true,
minCapacity: 1,
maxCapacity: 16,
secondsUntilAutoPause: 300
},
vpcSecurityGroupIds: [securityGroupId],
dbSubnetGroupName: dbSubnetGroup.dbSubnetGroupName
});
Using the CfnDBCluster construct, I get the following error:
Some input subnets in :[subnet-044631b3e615d752c,subnet-05c2881d9b13195ef,subnet-03c63ec89ae49a748] are invalid. (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 5c4e6237-6527-46a6-9ed4-1bc46c38dce0)
I am able to verify that those Subnets do exist before the RDS Stack is run.
Case 2:
After failing to get the CfnDBCluster example above working, I tried using the DatabaseCluster construct with raw overrides. Below is the contents of the RDS Stack using the DatabaseCluster construct:
// The beginning of the file imports all the VPC exports from the VPC Stack:
// subnetIds (for the private subnet), securityGroupId, vpcId, AZs, vpc (using Vpc.fromAttributes)
...
const dbCluster = new DatabaseCluster(this, 'DbCluster', {
engine: DatabaseClusterEngine.auroraPostgres({
version: AuroraPostgresEngineVersion.VER_10_7
}),
masterUser: {
username: databaseSecret.secretValueFromJson('username').toString(),
password: databaseSecret.secretValueFromJson('password')
},
instanceProps: {
vpc: vpc,
vpcSubnets: {
subnetType: SubnetType.PRIVATE
}
},
});
const cfnDbCluster = dbCluster.node.defaultChild as CfnDBCluster;
cfnDbCluster.addPropertyOverride('DbClusterIdentifier', 'rds-cluster');
cfnDbCluster.addPropertyOverride('EngineMode', 'serverless');
cfnDbCluster.addPropertyOverride('DatabaseName', DB_NAME);
cfnDbCluster.addPropertyOverride('EnableHttpEndpoint', true);
cfnDbCluster.addPropertyOverride('ScalingConfiguration.AutoPause', true);
cfnDbCluster.addPropertyOverride('ScalingConfiguration.MinCapacity', 1);
cfnDbCluster.addPropertyOverride('ScalingConfiguration.MaxCapacity', 16);
cfnDbCluster.addPropertyOverride('ScalingConfiguration.SecondsUntilAutoPause', 300);
cfnDbCluster.addPropertyOverride('VpcSecurityGroupIds', subnetIds);
Using the DatabaseCluster construct, I get the following error:
There are no 'Private' subnet groups in this VPC. Available types:
I am able to verify that the VPC does have a Private subnet, I also verified that it was properly imported and that the Subnets all have the expected tags i.e. key: 'aws-cdk:subnet-type' value: 'Private'
This issue has me blocked and confused, I cannot figure out why either of these issues are manifesting and would appreciate any guidance offered on helping resolve this issue.
References:
DatabaseCluster Construct
CfnDBCluster Construct
Database Cluster CloudFormation Properties
Escape Hatches
Notes:
I am using CDK version 1.56.0 with Typescript
In case you visiting this page after getting-
Some input subnets in :[subnet-XXXX,subnet-YYYY,subnet-ZZZZ] are invalid.
You probably checked and confirmed that these subnets do not exist and knock your head struggling to find where the hell these subnets are coming from.
The reason CDK still point to these subnets is since cdk.context.json is still contains values from last deployments.
From the docs-
Context values are key-value pairs that can be associated with a stack
or construct. The AWS CDK uses context to cache information from your
AWS account, such as the Availability Zones in your account or the
Amazon Machine Image (AMI) IDs used to start your instances.
Replace all the JSON content to a valid one ( {} ) and re-deploy the stack.

How to get AWS account id as custom variable in serverless framework?

In serverless framework, I want to set the deployment bucket as
<project_name>-<stage>-<account_id>
I can get the stage using a custom variable, like:
custom:
stage: ${opt:stage, self:provider.stage}
but how can I get the aws account id? I already tried to used serverless-pseudo-parameters, like this below, without success.
custom:
account_id: #{AWS::AccountId}
plugins:
- serverless-pseudo-parameters
Someone could help me to set the account id as a custom variable?
According to the documentation, to get the Account Id, you can use external js files:
// myCustomFile.js
module.exports.getAccountId = async (context) => {
return context.providers.aws.getAccountId();
};
.
# serverless.yml
service: new-service
provider: aws
custom:
accountId: ${file(../myCustomFile.js):getAccountId}
For anyone using Serverless with an "assumed role" where your IAM users are defined in a master AWS account and you're trying to deploy in a child account using a role from that child account: the documented solution - the one in the accepted answer above - does not work.
This setup in described in detail here: https://theithollow.com/2018/04/30/manage-multiple-aws-accounts-with-role-switching/. When using serverless with an --aws-profile that's configured to assume a role defined in another account, sts.getCallerIdentity() returns the account info of your master account from the default profile, and not the account of the assumed role.
To get the account ID of the assumed role (which is where we're deploying to), I did the following:
const { STS } = require('aws-sdk');
module.exports.getAccountId = async (context) => {
// This loads the AWS credentials Serverless is currently using
// They contain the role ARN of the assumed role
const credentials = context.providers.aws.getCredentials();
// init STS using the same credentials
const sts = new STS(credentials);
const identity = await sts.getCallerIdentity().promise();
return identity.Account;
};
Edit:
Found an even better way, that is simpler than the one presented in Serverless docs and also works fine with assumed roles:
module.exports.getAccountId = async (context) => {
return context.providers.aws.getAccountId();
};
You should be able to access them below as per below example https://serverless.com/framework/docs/providers/aws/guide/variables/
Resources:
- 'Fn::Join':
- ':'
- - 'arn:aws:logs'
- Ref: 'AWS::Region'
- Ref: 'AWS::AccountId'
- 'log-group:/aws/lambda/*:*:*'
It seems like your syntax is wrong. Try
custom:
account_id: ${AWS::AccountId}
Because at least in the example that you provided you are using #{AWS::AccountId}
Notice the hashtag in your one?

Using AWS CDK and RDS (Aurora), where can I change the Certificate authority?

I am setting up a database cluster (Aurora MySQL 5.7) using the DatabaseCluster Construct from #aws-cdk/aws-rds.
My question, where in the setup can I change the Certificate authority? I want to programmatically setup the database to use rds-ca-2019 instead of rds-ca-2015. Note, I want to change this using CDK, not by "clicking in the AWS GUI".
The image below shows which setting I am referring to.
I have been browsing the docs for RDS CDK, and tried to Google this without success.
This guide describes the manual steps on how to do this.
AWS CDK RDS module
DatabaseCluster Construct
Low-level Cluster (CfnCluster)
BTW, my current current config looks a bit like this:
const cluster = new rds.DatabaseCluster(this, 'aurora-cluster', {
clusterIdentifier: 'aurora-cluster',
engine: rds.DatabaseClusterEngine.AURORA_MYSQL,
masterUser: {
username: 'someuser',
password: 'somepassword'
},
defaultDatabaseName: 'db',
instances: 2,
instanceIdentifierBase: 'aurora-',
instanceProps: {
instanceType: ...,
vpcSubnets: {
subnetType: ec2.SubnetType.PUBLIC,
},
vpc: myVpc
},
removalPolicy: cdk.RemovalPolicy.DESTROY,
parameterGroup: {
parameterGroupName: 'default.aurora-mysql5.7'
},
port: 3306,
storageEncrypted: true
});
Apparently Cloudformation doesn't support the certificate authority field, and therefore CDK can't either.
https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/211
I upvoted the issue; feel free to join me!

Error while creating interface type vpc endpoint in terraform (aws provider)

I am trying to create an interface type kms endpoint in terraform on aws. While doing so, I get the following error -
Error creating VPC Endpoint: InvalidParameter: Subnet: subnet- does not have corresponding zone in the service com.amazonaws.ap-southeast-1.kms
My endpoint resource looks like --
resource "aws_vpc_endpoint" "kms" {
vpc_id = "${aws_vpc.main.id}"
service_name = "com.amazonaws.${var.aws_region}.kms"
vpc_endpoint_type = "Interface"
subnet_ids = [
<my subnet ids>
]
security_group_ids = [ <my sg ids> ]
private_dns_enabled = true
}
Anyone any clue on what I might be missing. Just FYI -- I haven't added any route53 record for kms. Do i need to?
Looking forward to any replies.
Thanks
Avi
If you have not created the private DNS in Route53, set private_dns_enabled=false . Otherwise create the private zone first.
I solved this issue by creating multiple endpoint resources for different subnets.