InSpec Testing on AWS Auto Scaling Groups - amazon-web-services

I'm trying to perform some testing on infrastructure that's created using Terraform. This specific test is testing an auto scaling group. I don't know the full name of the resource as it's appended with a dynamic token, but I know the start of the resource name which is set in the asg_name variable. I've got the following test:
asg_name = input('asg_name')
control 'aws_auto_scaling_groups' do
title 'Auto Scale Group'
desc 'Ensures Autoscale Group exists with correct configuration'
describe aws_auto_scaling_group ( name: /^#{asg_name}*/ ) do
it { should exist }
its('min_size') { should be 1}
its('max_size') { should be 1}
end
end
This is failing with the following:
/opt/inspec/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-5.17.4/lib/inspec/profile_context.rb:171:in `instance_eval': aws/controls/loadbalancer.rb:6: syntax error, unexpected tLABEL, expecting ')' (SyntaxError)
... aws_auto_scaling_group ( name: /^#{asg_name}*/ ) do
... ^~~~~
aws/controls/loadbalancer.rb:12: syntax error, unexpected `end', expecting end-of-input
I've tried a number of different options, including using aws_auto_scaling_groups.where which didn't work as expected as it returned an array, but I still haven't been able to get it working. Please can anyone tell me how I do a match against a name for a single resource like this using InSpec.
Thank you in advance!

Related

Terraform - Check If Variable Ends With String & Remove

We've set up our TF GKE code so that the user can specify either the region or zone for the cluster.
However, we need to then check this variable and remove the zone suffix (if it exists) for the deployment of static IP addresses.
We have the following variable:
variable "k8s_cluster_location" {
type = string
default = "europe-west2"
validation {
condition = contains(["europe-west2", "europe-west2-a", "europe-west2-b", "europe-west2-c", "us-east4", "us-east4-a", "us-east4-b", "us-east4-c", "europe-west1", "europe-west1-a", "europe-west1-b", "europe-west1-c" ], var.k8s_cluster_location)
error_message = "Given GCP location not (yet) supported. Contact X if you think it should..."
}
description = "Location of the Kubernetes cluster."
}
If, for example, the variable is "europe-west2-a", we need to remove "-a" to acquire the parent region.
Would we need to incorporate a Regex check? Or could we use something like StartsWith()/EndsWith()?
I would definitely recommend the regular expression solution here as you suggest:
variable "k8s_cluster_location" {
type = string
default = "europe-west2"
validation {
condition = can(regex("(?:europe-west[12])|(?:us-east4)", var.k8s_cluster_location))
error_message = "Given GCP location not (yet) supported. Contact X if you think it should..."
}
description = "Location of the Kubernetes cluster."
}
Note that if you are using Terraform 1.3.x, then you can also use the var.k8s_cluster_location value in the error_message instead of "Given GCP location".
For your other suggestion of startswith(), you would need to do something like anytrue(startswith(var.k8s_cluster_location, "europe-west1"), startswith(var.k8s_cluster_location, "europe-west2"), startswith(var.k8s_cluster_location, "us-east4")), but that feels slightly messier to me.

"Property Parameters cannot be empty" when creating a ClusterParameterGroup in Aurora

I'm using AWS CDK to deploy a Aurora RDS Cluster.
const cluster = new rds.DatabaseCluster(this, 'id', {
...
parameterGroup: new rds.ClusterParameterGroup(this,'id', {
family: 'aurora-postgresql11'
}})
Throws Property Parameters cannot be empty from the ClusterParameterGroup Construct. Any idea why?
When I try to use ClusterParameterGroup.fromParameterGroupName(this, "id", [a group that I manually created in the account] it works.
The reason I don't want to use the default ParameterGroup that RDS creates is because those are not modifiable. In case I need to modify them, I would need to create my own ParameterGroup resource.
I did the following:
const rdsClusterPrameterGroup = ClusterParameterGroup.fromParameterGroupName(
this,
'rdsClusterPrameterGroup',
'default.aurora-postgresql11',
)
Then pass that to DatabaseCluster as parameterGroup value.
It looks like ParameterGroup.fromParameterGroupName does not create separate resource at all (just synthesize and look at the output), just name/reference to the default group.
Eventually seems like below code (python) does the thing. I needed to put one parameter to avoid Property Parameters cannot be empty error, thus the application_name which seems harmless. This is empty in defaults and stays empty in new group (among other unchanged defaults):
rds_db_param_group = rds.ParameterGroup(
scope=self,
id='CustomDbParams',
engine=rds.DatabaseClusterEngine.aurora_postgres(
version=rds.AuroraPostgresEngineVersion.VER_11_9
),
description=f'custom-params-aurora-postgresql',
parameters={
'application_name' : '',
}
)
And then pass rds_db_param_group to the DatabaseCluster and InstanceProps (as parameter_group):
rds.DatabaseCluster(self,
instance_props=rds.InstanceProps(
parameter_group=rds_db_param_group,
[...]
),
parameter_group=rds_db_param_group,
[...]
)

SpotFleetRequest - Tag specification resource type must have a value

For the past few weeks I was able to create SpotFleetRequests just fine (via Java). However since yesterday I am getting the following error:
com.amazonaws.services.ec2.model.AmazonEC2Exception: Tag specification resource type must have a value (Service: AmazonEC2; Status Code: 400; Error Code: InvalidSpotFleetRequestConfig; Request ID: ef69f477-e8f3-4d86-aa91-1646c4067d68)
I didn't really change anything and what's even weirder: I've already added a SpotFleetTagSpecification inside the SpotFleetLaunchSpecification of my SpotFleetRequestConfigData.
Here is my code:
List<Tag> tags = new ArrayList<>();
tags.add(new Tag("TEAM", "CROCODILE"));
SpotFleetTagSpecification tagSpec = new SpotFleetTagSpecification().withTags(tags);
SpotFleetLaunchSpecification launchSpec = new SpotFleetLaunchSpecification()
.withSecurityGroups(new GroupIdentifier().withGroupId(securityGroupId))
.withIamInstanceProfile(new IamInstanceProfileSpecification().withArn(instanceProfileArn))
.withImageId(imageId)
.withInstanceType(InstanceType.M3Xlarge)
.withSubnetId(subnetIds)
.withUserData(getUserDataToConfigureECSCluster(ecsClusterName))
.withTagSpecifications(tagSpec);
// Configure the actual request
SpotFleetRequestConfigData config = new SpotFleetRequestConfigData()
.withIamFleetRole(fleetRoleArn)
.withLaunchSpecifications(launchSpec)
.withAllocationStrategy(AllocationStrategy.LowestPrice)
.withTargetCapacity(targetCapacity)
.withType(FleetType.Maintain)
.withClientToken(spotFleetToken);
RequestSpotFleetRequest request = new RequestSpotFleetRequest()
.withSpotFleetRequestConfig(config);
RequestSpotFleetResult result = ec2.requestSpotFleet(request);
LOG.info("Created spot fleet request with ID {}", result.getSpotFleetRequestId());
Changing or removing the tags doesn't work either, the error persists no matter what. Does anyone have a clue what I am doing wrong?
It seems like the documentation was a bit confusing. It mentions that ResourceType is not required, but I had to set it explicitly to instance to successfully create my SpotFleetRequest.

How to create an RDS instance from the most recent snapshot or from scratch

In terraform, is there a way to conditionally create an RDS instance from the most recent snapshot of a given database or to create an empty database depending on the value of a parameter?
I tried something like that:
variable "db_snapshot_source" {
default = ""
}
data "aws_db_snapshot" "last_snap" {
count = "${var.db_snapshot_source == "" ? 0 : 1}"
most_recent = true
db_instance_identifier = "${var.db_snapshot_source}"
}
resource "aws_db_instance" "db" {
[...]
snapshot_identifier = "${var.db_snapshot_source == "" ? "" : data.aws_db_snapshot.last_snap.db_snapshot_identifier}"
}
Unfortunately, it does not work because TF seems to dereference data.aws_db_snapshot.last_snap even if the ternary is false. I get the following error message: * aws_db_instance.db: Resource 'data.aws_db_snapshot.last_snap' not found for variable 'data.aws_db_snapshot.last_snap.db_snapshot_identifier'.
How can I achieve a such behaviour? The only option I see is to declare two aws_db_instance resources each with opposed count which is horrifying.
By defining a count you are saying the result of the data resource will be a list even if it is a zero value.
resource "aws_db_instance" "db" {
[...]
snapshot_identifier = "${
var.db_snapshot_source == "" ? "" :
element(
concat(data.aws_db_snapshot.last_snap.*.db_snapshot_identifier, list("")), 0)
}"
}
The concat is required if you expect the list to be empty. Otherwise you get an error
element: element() may not be used with an empty list...
Github issue describing the concat behaviour
The documentation reads as though specifying snapshot_identifier is what triggers using a snapshot or not, so passing in an empty string is not enough to avoid starting from a snapshot. In that case, you would need two aws_rds_instance resources, and then have ternary expressions for count on each resource to decide which one to create. As you mentioned, this is horrifying, but it might work ok.
Another way to think about it is if you had a blank snapshot in your inventory to start from. Then it's just a ternary operator away from deciding to use the custom snapshot or this blank snapshot. I don't know that you can create a blank snapshot in Terraform though, it's creation might be out of band.

Amazon Lex Error: An error occurred (BadRequestException) when calling the PutIntent operation: RelativeId does not match Lex ARN format

I'm trying to build a chatbot using Amazon's boto3 library. Right now, I am trying to create an intent using the put_intent function. My code is as follows:
intent = lexClient.put_intent(name = 'test',
sampleUtterances = ["Who is messi?"]
)
When I try running this, I get the following exception:
botocore.errorfactory.BadRequestException: An error occurred
(BadRequestException) when calling the PutIntent operation: RelativeId
does not match Lex ARN format: intent:test2:$LATEST
Can anyone tell me what I'm doing wrong?
I got the same error when trying to have a digit in intent name field. Realized that was not allowed when trying to do the same from AWS console.
Error handling could really be more specific.
Try taking the question mark out of the utterance, that has caused me issues in the past!
You need to run GetSlotType. That will return current checksum for that slot. Put that checksum in your PutSlotType checksum. Big bang boom.
https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/LexModelBuildingService.html#getSlotType-property
var params = {
name: "AppointmentTypeValue",
checksum:'54c6ab5f-fe30-483a-a364-b76e32f6f05d',
description: "Type of dentist appointment to schedule'",
enumerationValues: [
{
value: "cleaning"
},
{
value: "whitening"
},
{
value: "root canal"
},
{
value:"punch my face"
}
]
};
For the put_intent function I faced similar issues.
At least the following 3 are worth mentioning.
Sample Utterances
There are requirements for the sample utterances:
An utterance can consist only of Unicode
characters, spaces, and valid punctuation marks. Valid punctuation
marks are: periods for abbreviations, underscores, apostrophes, and
hyphens. If there is a slot placeholder in your utterance, ensure that
it's in the {slotName} format and has spaces at both ends.
It seems like there is no error raised when calling the put_intent function with the following code.
intent = lexClient.put_intent(name = 'test',
sampleUtterances = ["Who is messi"]
)
However, if you try to add it to your bot and start building the bot it will fail.
To fix it remove the question mark at the end of you sampleUtterance.
intent = lexClient.put_intent(name = 'test',
sampleUtterances = ["Who is messi?"]
)
Prior intent version
If your intent already exists you need to add the checksum to your function call. To get the checksum of your intent you can use the get_intent function.
For example docs:
response = client.get_intent(
name='test',
version='$LATEST'
)
found_checksum = response.get('checksum')
After that you can put a new version of the intent:
intent = lexClient.put_intent(name = 'test',
sampleUtterances = ["Who is messi"],
checksum = found_checksum
)
Intent Name (correct in your case, just adding this for reference)
It seems like the name can only contain letters, underscores, and should be <=100 in length. Haven't found anything in the docs. This is just trial and error.
Calling put_intent with the following:
intent = lexClient.put_intent(name = 'test_1',
sampleUtterances = ["Who is messi"]
)
Results in the following error:
BadRequestException: An error occurred (BadRequestException) when calling the PutIntent operation: RelativeId does not match Lex ARN format: intent:test_1:$LATEST
To fix the name you can replace it to:
intent = lexClient.put_intent(name = 'test',
sampleUtterances = ["Who is messi"]
)