Fn::Sub expression does not resolve to a string - amazon-web-services

I have created a parameter:
Parameters:
..
list:
Description: "Provide a list .."
Type: CommaDelimitedList
Default: "test1, test2"
Now I want to reference this list (which will resolve in "test1", "test2", ..) from a file in my cloudformation which looks like this:
configure_xx:
files:
/etc/file.conf:
content: !Sub |
input {
logs {
log_group => [ "${list}" ]
access_key_id => "${logstashUserKey}"
secret_access_key => "${logstashUserKey.SecretAccessKey}"
region => "eu-west-1"
}
}
How can I make this work for the parameter list? (the keys work).
error: Fn::Sub expression does not resolve to a string

Just switch the parameter type for a "String"
Parameters:
..
list:
Description: "Provide a list .."
Type: String
Default: "test1, test2"
If, for some reason, you have no control over this parameter type, you could use Fn::Join to transform the list to a string. For exemple:
configure_xx:
files:
/etc/file.conf:
content:
Fn::Sub:
- |-
input {
logs {
log_group => [ "${joinedlist}" ]
access_key_id => "${logstashUserKey}"
secret_access_key => "${logstashUserKey.SecretAccessKey}"
region => "eu-west-1"
}
}
- joinedlist:
Fn::Join:
- ', '
- !Ref list

Related

Terraform variable inteporlation and evaluation

I'm working with modules in Terraform using Yaml approach to manage variables. I have a very simple module that should create parameter in AWS Parameter Store based on my RDS and IAM User modules output.So, I wrote this module:
resource "aws_ssm_parameter" "ssm_parameter" {
name = var.parameter_name
type = var.parameter_type
value = var.parameter_value
overwrite = var.overwrite
tags = var.tags
}
The variables I'm using are stored into a Yaml file like this:
ssms:
/arquitetura/catalogo/gitlab/token:
type: SecureString
value: ManualInclude
/arquitetura/catalogo/s3/access/key:
type: String
value: module.iam_user.access_key
/arquitetura/catalogo/s3/secret/access/key:
type: SecureString
value: module.iam_user.secret_access_key
/arquitetura/catalogo/rds/user:
type: String
value: module.rds_instance.database_username
/arquitetura/catalogo/rds/password:
type: SecureString
value: module.rds_instance.database_password
As we can see, I have in "value" the module output I would like to send to Parameter Store. I'm loading this variable file using file and yamldecode functions:
ssmfile = "./env/${terraform.workspace}/ssm.yaml"
ssmfilecontent = fileexists(local.ssmfile) ? file(local.ssmfile) : "ssmFileNotFound: true"
ssmsettings = yamldecode(local.ssmfilecontent)
So, I have a local.ssmsettings and I can write a module call like this:
module "ssm_parameter" {
source = "../aws-ssm-parameter-tf"
for_each = local.ssmsettings.ssms
parameter_name = each.key
parameter_type = each.value.type
parameter_value = each.value.value
tags = local.tags
}
Doing this, my parameter is stored as:
{
"Parameter": {
"Name": "/arquitetura/catalogo/rds/user",
"Type": "String",
"Value": "module.rds_instance.database_username",
"Version": 1,
"LastModifiedDate": "2022-12-15T19:02:01.825000-03:00",
"ARN": "arn:aws:ssm:sa-east-1:111111111111:parameter/arquitetura/catalogo/rds/user",
"DataType": "text"
}
}
Value is receiving the string module.rds_instance.database_username instead of the module output.
I know that file function doesn't interpolate variables and I know Terraform doesn't have an eval function.
Does anybody had the same situation that can tell me how you solved the problem or have any clue that I can follow?
I already tried to work with Terraform templates, without success.
Thanks in advance.
Terraform has no way to understand that the value strings in your YAML files are intended to be references to values elsewhere in your module, and even if it did it wouldn't be possible to resolve them from there because this YAML file is not actually a part of the Terraform module, and is instead just a data file that Terraform has loaded into memory.
However, you can get a similar effect by placing all of the values your YAML file might refer to into a map of strings inside your module:
locals {
ssm_indirect_values = tomap({
manual_include = "ManualInclude"
aws_access_key_id = module.iam_user.access_key
aws_secret_access_key = module.iam_user.secret_access_key
database_username = module.rds_instance.database_username
database_password = module.rds_instance.database_password
})
}
Then change your YAML data so that the value strings match with the keys in this map:
ssms:
/arquitetura/catalogo/gitlab/token:
type: SecureString
value: manual_include
/arquitetura/catalogo/s3/access/key:
type: String
value: aws_access_key_id
/arquitetura/catalogo/s3/secret/access/key:
type: SecureString
value: aws_secret_access_key
/arquitetura/catalogo/rds/user:
type: String
value: database_username
/arquitetura/catalogo/rds/password:
type: SecureString
value: database_password
You can then substitute the real values instead of the placeholders before you use the data structure in for_each:
locals {
ssm_file = "${path.module}/env/${terraform.workspace}/ssm.yaml"
ssm_file_content = file(local.ssm_file)
ssm_settings = yamldecode(local.ssm_file_content)
ssms = tomap({
for k, obj in local.ssm_settings.ssms :
k => {
type = obj.type
value = local.ssm_indirect_values[obj.value]
}
})
}
module "ssm_parameter" {
source = "../aws-ssm-parameter-tf"
for_each = local.ssms
parameter_name = each.key
parameter_type = each.value.type
parameter_value = each.value.value
tags = local.tags
}
The for expression in the definition of local.ssms uses the source value string as a lookup key into local.ssm_indirect_values, thereby inserting the real value.
The module "ssm_parameter" block now refers to the derived local.ssms instead of the original local.ssm_settings.ssms, so each.value.value will be the final resolved value rather than the lookup key, and so your parameter should be stored as you intended.

How to test a user has a managed policy using CDK assertions?

I've defined a user and a managed policy in CDK v2, similar to:
const policy = new iam.ManagedPolicy(this, `s3access`, {
statements: [
new iam.PolicyStatement({
effect: iam.Effect.ALLOW,
actions: ['s3:PutObject', 's3:GetObject'],
resources: ['*']
})
]
})
const someUser = new iam.User(this, 'some-user', { managedPolicies: [policy] });
I want to test that the user has the managed policy applied to it using CDK test assertions, however I'm struggling to figure out how using the existing test constructs:
template.hasResourceProperties('AWS::IAM::ManagedPolicy', {
PolicyDocument: Match.objectLike({
Statement: [
{
Action: ['s3:PutObject', 's3:GetObject'],
Effect: 'Allow',
Resource: [
'*'
]
},
]
})
})
...matches the managed policy, but doesn't test that the user has the managed policy applied.
What is the pattern / best practice for doing this?
You need to match the User's Managed Policy Arn as it appears in the template:
"Type": "AWS::IAM::User",
"Properties": {
"ManagedPolicyArns": [
{
"Ref": "s3access10922181"
}
]
The trick is to get the {"Ref": "s3access10922181"} reference to the policy. Here are two equivalent approaches:
Approach 1: stack.node.tryFindChild
const managedPolicyChild = stack.node.tryFindChild('s3access') as iam.ManagedPolicy | undefined;
if (!managedPolicyChild) throw new Error('Expected a defined ManagedPolicy');
const policyArnRef = stack.resolve(managedPolicyChild.managedPolicyArn);
template.hasResourceProperties('AWS::IAM::User', {
ManagedPolicyArns: Match.arrayWith([policyArnRef]),
});
Approach 2: template.findResources
const managedPolicyResources = template.findResources('AWS::IAM::ManagedPolicy');
const managedPolicyLogicalId = Object.keys(managedPolicyResources).find((k) => k.startsWith('s3access'));
if (!managedPolicyLogicalId) throw new Error('Expected to find a ManagedPolicy Id');
template.hasResourceProperties('AWS::IAM::User', {
ManagedPolicyArns: Match.arrayWith([{ Ref: managedPolicyLogicalId }]),
});

How to form key using query arguments in Dynamodb resolver request mapping?

Schema snippet:
type Query {
getPCSData(country: $String, year: $String, payCycle: $String): PCSDataOutput
}
Dynamodb Resolver:
PCSDataResolver:
Type: AWS::AppSync::Resolver
DependsOn: PCSGraphQLSchema
Properties:
ApiId:
Fn::GetAtt: [PCSGraphQLApi, ApiId]
TypeName: Query
FieldName: getPCSData
DataSourceName:
Fn::GetAtt: [PCSGraphQLDDBDataSource, Name]
RequestMappingTemplate: |
{
"version": "2017-02-28",
"operation": "GetItem",
"key": {
"Country_Year_PayCycle": ?????????
}
}
ResponseMappingTemplate: "$util.toJson($ctx.result)"
Here, I am looking to form key Country_Year_PayCycle using all three arguments passed from the query something like this Country_Year_PayCycle = country+ year+payCycle
Is it possible to concat the query arguments and form key ?
This is how I resolve
RequestMappingTemplate: |
## Set up some space to keep track of things we're updating **
#set($concat ="_")
#set($country = $ctx.args.country )
#set($year = $ctx.args.year )
#set($payCycle = $ctx.args.payCycle )
#set($pk = "$country$concat$year$concat$payCycle")
{
"version": "2017-02-28",
"operation": "GetItem",
"key": {
"Country_Year_PayCycle": $util.dynamodb.toDynamoDBJson($pk)
}
}

How to input the dictionary structure in 'values' argument using the aws_ssm_parameter in terraform?

I have one parameter in AWS System Manager, type of the value is string, but value has a dictionary structure:
Value:
{"key1": "value1","key2": "value2","key3": "value3"}
Now i'm trying to create this parameter using the Terraform. But i received an error, when trying to write this in aws_ssm_parameter resource:
resource "aws_ssm_parameter" "Paramet" {
name = "/dev/parameter/new"
description = "Sample config values"
type = "String"
value = "{key1" : "value1", "key2" : "value2", "key3" : "value3}"
}
Error:
$ terraform plan
Error: Missing newline after argument
on main.tf line 90, in resource "aws_ssm_parameter" "Paramet": 90:
value = "{key1" : "value1", "key2" : "value2", "key3" : "value3}"
An argument definition must end with a newline.
This error is connected with syntax, but i can't understand, how to fix this correctly.
Please advice, how to input correctly this value in aws_ssm_parameter resource?
You can use heredoc syntax to achieve this. The below should work for you
resource "aws_ssm_parameter" "Parameter" {
name = "/dev/parameter/new"
description = "Sample config values"
type = "String"
value = <<EOF
{
"key1" : "value1",
"key2" : "value2",
"key3" : "value3"
}
EOF
}

GKE Creation from Cloud Deployment Manager

Waiting for create [operation-1544424409972-57ca55456bd22-84bb0f13-64975fdc]...failed.
ERROR: (gcloud.deployment-manager.deployments.create) Error in Operation [operation-1544424409972-57ca55456bd22-84bb0f13-64975fdc]: errors:
- code: CONDITION_NOT_MET
location: /deployments/infrastructure/resources/practice-gke-clusters->$.properties->$.cluster.name
message: |-
InputMapping for field [cluster.name] for method [create] could not be set from input, mapping was: [$.ifNull($.resource.properties.cluster.name, $.resource.name)
], and evaluation context was:
{
"deployment" : {
"id" : 4291795636642362677,
"name" : "infrastructure"
},
"intent" : "CREATE",
"matches" : [ ],
"project" : "resources-practice",
"requestId" : "",
"resource" : {
"name" : "practice-gke-clusters",
"properties" : {
"initialNodeCount" : 1,
"location" : "asia-east2-a",
"loggingService" : "logging.googleapis.com",
"monitoringService" : "monitoring.googleapis.com",
"network" : "$(ref.practice-gke-network.selfLink)",
"subnetwork" : "$(ref.practice-gke-network-subnet-1.selfLink)"
},
"self" : { }
}
}
I always experience this when I try to create a GKE out of deployment manager w/ the jinja template below
resources:
- name: practice-gke-clusters
type: container.v1.cluster
properties:
network: $(ref.practice-gke-network.selfLink)
subnetwork: $(ref.practice-gke-network-subnet-1.selfLink)
initialNodeCount: 1
loggingService: logging.googleapis.com
monitoringService: monitoring.googleapis.com
location: asia-east2-a
You are missing:
properties:
cluster:
name: practice-gke-clusters
initialNodeCount: 3
nodeConfig:
oauthScopes:
- https://www.googleapis.com/auth/compute
- https://www.googleapis.com/auth/devstorage.read_only
- https://www.googleapis.com/auth/logging.write
- https://www.googleapis.com/auth/monitoring
Modify the initialNodeCount and oauthScopes as required.