I'm using AWS CDK to deploy a Aurora RDS Cluster.
const cluster = new rds.DatabaseCluster(this, 'id', {
...
parameterGroup: new rds.ClusterParameterGroup(this,'id', {
family: 'aurora-postgresql11'
}})
Throws Property Parameters cannot be empty from the ClusterParameterGroup Construct. Any idea why?
When I try to use ClusterParameterGroup.fromParameterGroupName(this, "id", [a group that I manually created in the account] it works.
The reason I don't want to use the default ParameterGroup that RDS creates is because those are not modifiable. In case I need to modify them, I would need to create my own ParameterGroup resource.
I did the following:
const rdsClusterPrameterGroup = ClusterParameterGroup.fromParameterGroupName(
this,
'rdsClusterPrameterGroup',
'default.aurora-postgresql11',
)
Then pass that to DatabaseCluster as parameterGroup value.
It looks like ParameterGroup.fromParameterGroupName does not create separate resource at all (just synthesize and look at the output), just name/reference to the default group.
Eventually seems like below code (python) does the thing. I needed to put one parameter to avoid Property Parameters cannot be empty error, thus the application_name which seems harmless. This is empty in defaults and stays empty in new group (among other unchanged defaults):
rds_db_param_group = rds.ParameterGroup(
scope=self,
id='CustomDbParams',
engine=rds.DatabaseClusterEngine.aurora_postgres(
version=rds.AuroraPostgresEngineVersion.VER_11_9
),
description=f'custom-params-aurora-postgresql',
parameters={
'application_name' : '',
}
)
And then pass rds_db_param_group to the DatabaseCluster and InstanceProps (as parameter_group):
rds.DatabaseCluster(self,
instance_props=rds.InstanceProps(
parameter_group=rds_db_param_group,
[...]
),
parameter_group=rds_db_param_group,
[...]
)
Related
We've set up our TF GKE code so that the user can specify either the region or zone for the cluster.
However, we need to then check this variable and remove the zone suffix (if it exists) for the deployment of static IP addresses.
We have the following variable:
variable "k8s_cluster_location" {
type = string
default = "europe-west2"
validation {
condition = contains(["europe-west2", "europe-west2-a", "europe-west2-b", "europe-west2-c", "us-east4", "us-east4-a", "us-east4-b", "us-east4-c", "europe-west1", "europe-west1-a", "europe-west1-b", "europe-west1-c" ], var.k8s_cluster_location)
error_message = "Given GCP location not (yet) supported. Contact X if you think it should..."
}
description = "Location of the Kubernetes cluster."
}
If, for example, the variable is "europe-west2-a", we need to remove "-a" to acquire the parent region.
Would we need to incorporate a Regex check? Or could we use something like StartsWith()/EndsWith()?
I would definitely recommend the regular expression solution here as you suggest:
variable "k8s_cluster_location" {
type = string
default = "europe-west2"
validation {
condition = can(regex("(?:europe-west[12])|(?:us-east4)", var.k8s_cluster_location))
error_message = "Given GCP location not (yet) supported. Contact X if you think it should..."
}
description = "Location of the Kubernetes cluster."
}
Note that if you are using Terraform 1.3.x, then you can also use the var.k8s_cluster_location value in the error_message instead of "Given GCP location".
For your other suggestion of startswith(), you would need to do something like anytrue(startswith(var.k8s_cluster_location, "europe-west1"), startswith(var.k8s_cluster_location, "europe-west2"), startswith(var.k8s_cluster_location, "us-east4")), but that feels slightly messier to me.
I'm trying to perform some testing on infrastructure that's created using Terraform. This specific test is testing an auto scaling group. I don't know the full name of the resource as it's appended with a dynamic token, but I know the start of the resource name which is set in the asg_name variable. I've got the following test:
asg_name = input('asg_name')
control 'aws_auto_scaling_groups' do
title 'Auto Scale Group'
desc 'Ensures Autoscale Group exists with correct configuration'
describe aws_auto_scaling_group ( name: /^#{asg_name}*/ ) do
it { should exist }
its('min_size') { should be 1}
its('max_size') { should be 1}
end
end
This is failing with the following:
/opt/inspec/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-5.17.4/lib/inspec/profile_context.rb:171:in `instance_eval': aws/controls/loadbalancer.rb:6: syntax error, unexpected tLABEL, expecting ')' (SyntaxError)
... aws_auto_scaling_group ( name: /^#{asg_name}*/ ) do
... ^~~~~
aws/controls/loadbalancer.rb:12: syntax error, unexpected `end', expecting end-of-input
I've tried a number of different options, including using aws_auto_scaling_groups.where which didn't work as expected as it returned an array, but I still haven't been able to get it working. Please can anyone tell me how I do a match against a name for a single resource like this using InSpec.
Thank you in advance!
The way to delete a feature group using the SageMaker Python SDK is as follows:
my_feature_group.delete()
But this only deletes the feature group you are currently working on. How can one delete feature groups from prior sessions? I tried deleting them out of the S3 bucket directly, but they still appear in the Feature Store UI.
It would be great if feature groups could be deleted through the UI. But if not, is there a way to delete a feature group using it's full name; the one that was created using:
my-feature-group-" + strftime("%d-%H-%M-%S", gmtime())
You can create a FeatureGroup object and call delete or via cli or SageMakerFeatureStoreRuntime client
source: aws
You can loop over list_feature_groups as follows:
def extract_feature_groups(feature_groups):
list_feature_groups = []
list_feature_groups.extend([x['FeatureGroupName'] for x in feature_groups['FeatureGroupSummaries']])
next_token = '' if not ('NextToken' in feature_groups.keys()) else feature_groups['NextToken']
while not (next_token==''):
page_feature_groups = boto_client.list_feature_groups(NextToken=next_token)
list_feature_groups.extend([x['FeatureGroupName'] for x in page_feature_groups['FeatureGroupSummaries']])
next_token = '' if not ('NextToken' in page_feature_groups.keys()) else page_feature_groups['NextToken']
return list_feature_groups
region_name = <your_region_name>
boto_client = boto3.client('sagemaker', region_name=region_name)
boto_session = boto3.session.Session(region_name=region_name)
fs_sagemaker_session = sagemaker.Session(boto_session=boto_session)
feature_groups = boto_client.list_feature_groups()
list_features_groups = extract_feature_groups(feature_groups)
for fg in list_features_groups:
<make sure to include appropriate name filter and/or confirmation requests>
feature_group = FeatureGroup(name = feature, sagemaker_session = fs_sagemaker_session)
feature_group.delete()
Feature groups take time to complete deletion; you might want to add a function for checking deletion has concluded successfully.
I defined a variable map my_role in terraform and set its value in abc.tfvar file as follows. if I assign account id as actual value, it works, if I set account id as a variable, it does not work. Does it mean tfvar file only allow actual value, not variable? By the way, I use terraform workspace. Therefore my_role is different based on workspace I select.
The following works:
my_role = {
dev = "arn:aws:iam::123456789012:role/myRole"
test = ...
prod = ...
}
The following does not work:
my_role = {
dev = "arn:aws:iam::${lookup(var.aws_account_id, terraform.workspace)}:role/myRole"
test = ...
prod = ...
}
The following does not work either:
lambdarole = {
dev = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:role/myRole"
test = ...
prod = ...
}
does
Have you tried following the example on Input Variables?
You can define your abc.tfvars file with:
variable "my_role" {
type = "map"
default = {
"dev" = "arn:aws:iam::123456789012:role/myRole"
"test" = "..."
"prod" = "..."
}
}
And access it with "${lookup(var.my_role, terraform.workspace)}".
Also, according to the from a file, the variables defined in .tfvars files are automatically loaded if you name the file terraform.tfvars, if not, you have to pass as an argument with -var-file=...
Cannot test it right now, but probably is something in this way.
I am replying when terraform 0.12 version is latest one. Solution is simple, you can create one file say vars.tf and declare variables in it.
Example - variable "xyz" {}
Now create terraform.tfvars and initialize it.
Example - xyz="abcd"
No need to pass any runtime args, it will be picked directly.
Terraform has aws_caller_identity data source. You do not need to mention or hand code account id anywhere. It can be fetched using this source.
In any of your .tf file, just include this source and then you can fetch relevant argument value.
This is how you can do it. Define this in any *.tf file
data "aws_caller_identity" "current" {}
Now where ever you want the value of arn or account id, it can be fetched using :
For account id(For terraform0.12):
data.aws_caller_identity.current.account_id
In your case, it would be like this :
dev = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:role/myRole"
But in order to this work, you need to define data source like shown above in any *.tf file.
For more help, refer following:
URL : https://www.terraform.io/docs/providers/aws/d/caller_identity.html
In terraform, is there a way to conditionally create an RDS instance from the most recent snapshot of a given database or to create an empty database depending on the value of a parameter?
I tried something like that:
variable "db_snapshot_source" {
default = ""
}
data "aws_db_snapshot" "last_snap" {
count = "${var.db_snapshot_source == "" ? 0 : 1}"
most_recent = true
db_instance_identifier = "${var.db_snapshot_source}"
}
resource "aws_db_instance" "db" {
[...]
snapshot_identifier = "${var.db_snapshot_source == "" ? "" : data.aws_db_snapshot.last_snap.db_snapshot_identifier}"
}
Unfortunately, it does not work because TF seems to dereference data.aws_db_snapshot.last_snap even if the ternary is false. I get the following error message: * aws_db_instance.db: Resource 'data.aws_db_snapshot.last_snap' not found for variable 'data.aws_db_snapshot.last_snap.db_snapshot_identifier'.
How can I achieve a such behaviour? The only option I see is to declare two aws_db_instance resources each with opposed count which is horrifying.
By defining a count you are saying the result of the data resource will be a list even if it is a zero value.
resource "aws_db_instance" "db" {
[...]
snapshot_identifier = "${
var.db_snapshot_source == "" ? "" :
element(
concat(data.aws_db_snapshot.last_snap.*.db_snapshot_identifier, list("")), 0)
}"
}
The concat is required if you expect the list to be empty. Otherwise you get an error
element: element() may not be used with an empty list...
Github issue describing the concat behaviour
The documentation reads as though specifying snapshot_identifier is what triggers using a snapshot or not, so passing in an empty string is not enough to avoid starting from a snapshot. In that case, you would need two aws_rds_instance resources, and then have ternary expressions for count on each resource to decide which one to create. As you mentioned, this is horrifying, but it might work ok.
Another way to think about it is if you had a blank snapshot in your inventory to start from. Then it's just a ternary operator away from deciding to use the custom snapshot or this blank snapshot. I don't know that you can create a blank snapshot in Terraform though, it's creation might be out of band.