How dynamically change S3 path inside DefinitionBody - amazon-web-services

My code currently is working like this
DefinitionBody:
'Fn::Transform':
Name: AWS::Include
Parameters:
Location:
Fn::Sub: s3://${Bucket}/${AWS::StackName}/${File}
# Note: Bucket and File comes from Parameters in this case
Now I try to get {$File} dynamically using FindInMap
'Fn::Transform':
Name: AWS::Include
Parameters:
Location: !Sub
- s3://${value-from-parameters-section}/${AWS::StackName}/${spec}
- { spec: !FindInMap [ "Config", "type", "file"]}
But is throwing me this error:
Failed to digest functions within transform parameters, intrinsic functions in transform block must only contain parameter values or stack metadata
So, How I can achieve to dynamically change the File name inside the DefinitonBody?

You can't do this, it is not supported. AWS docs explicitly states that no intrinsic functions are supported:
Supported functions None. CloudFormation passes any intrinsic function calls included in Fn::Transform to the specified macro as literal strings.
You can re-design your function to use custom resources instead of macros. This way you can do whatever you wish.

In this case, I think the message is self-explanatory
Failed to digest functions within transform parameters, intrinsic functions in transform block must only contain parameter values or stack metadata
Only metadata or parameters are allowed, that is the reason this works
DefinitionBody:
'Fn::Transform':
Name: AWS::Include
Parameters:
Location:
Fn::Sub: s3://${Bucket}/${AWS::StackName}/${StackType}
In case somebody needs a workaround I just remove the mapping section and rename the name of the files to match the type I want, something like this
Parameters:
StackType:
Type: String
AllowedValues:
- typeA
- typeB
Bucket:
Type: String
Then I just use this sub to get the DefinitionBody like this:
DefinitionBody:
'Fn::Transform':
Name: AWS::Include
Parameters:
Location:
Fn::Sub: s3://${Bucket}/${AWS::StackName}/${StackType}-API.yaml
This will resolve like this:
s3://example-bucket/name-of-stack/typeA-API.yaml
or
s3://example-bucket/name-of-stack/typeB-API.yaml
So now I just need to have these files named in that way on the S3, this is similar to what mapping should do, but for now, is not supported, this will achieve what I want, is not perfect but did the trick, still remain the confusion because I think the documentation is inaccurate

Related

reuse a value in a sam template

I'm probably missing something very obvious, but I can't find a way of just setting a value that I want to reuse. For instance - I have a sam template that creates a bunch of database tables - I want them to all have the same settings - and I want those settings to depend on whether it's production or not.
so at the moment I do
Resources:
firstTable:
Type: AWS::DynamoDb::Table
...
DeletionPolicy: !If[ isProduction, Retain, Delete ]
secondTable:
Type: AWS::DynamoDb::Table
DeletionPolicy: !If[ isProduction, Retain, Delete ]
in the perfect world, I'd want to say something like "every dynamodb table defined in this template should have this list of settings:" - but I suspect that's not possible, but what I think IS possible - I want to somehow be able to say something like:
somewhere:
deletion_policy_value: !If[ isProduction, Retain, Delete ]
...
firstTable:
Type: AWS::DynamoDb::Table
...
DeletionPolicy: deletion_policy_value
but none of parameters, conditionals globals or environment variables seem to fit - ie I want to define a custom variable that exists only for the life of the template - environment variables seem to exist in the actual cloudformation script - which is NOT what I want (I think)
Sadly its not possible. You would have to develop your own macro to create such substitutions.

!Join in Serverless Framework is broken

RetrieveAllSubscribersLambdaPermissionApiGateway:
Type: 'AWS::Lambda::Permission'
Properties:
FunctionName: { "Fn::GetAtt": [ RetrieveAllSubscribersLambdaFunction, Arn ] }
Action: lambda:InvokeFunction
Principal: apigateway.amazonaws.com
SourceArn:
!Join
- "/"
- - 'arn:aws:execute-api:us-west-2:12345'
- - 'other stuff'
I have this block of code where I try and get a sourceArn for this block. I realize the sourceArn is invalid in this example, but so is the viewed output from Serverless.
An error occurred: RetrieveAllSubscribersLambdaPermissionApiGateway - 1 validation error detected: Value 'arn:aws:execute-api:us-west-2:12345/aws/:execute-api:/us-west-2/:/12345/:/a5dghhjk9//*/*' at 'sourceArn' failed to satisfy constraint: Member must satisfy regular expression pattern: arn:(aws[a-zA-Z0-9-]*):([a-zA-Z0-9\-])+:([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\d{1})?:(\d{12})?:(.*) (Service: AWSLambda; Status Code: 400; Error Code: ValidationException; Request ID: a8db5326-dcdd-4abc-b4ae-e4fa5c03c6bd; Proxy: null).
-- Update 2022/01/16 12:49 AM
The problem was indeed with the tags. Everything can be seen on the issue. TLDR; use the full function name (e.g. 'Fn::Join').
However, I'm almost certain that the shorthand can work, there are just some conditions that need to be met first. I will not investigate, since a working solution for the original problem has been achieved and anything else would be outside the scope.
-- Original
After some back and forth between Ryan and myself. Ryan used an alternative solution involving !Sub and the serverless-cloudformation-sub-variables plugin which can be viewed here.
In short, the problem likely lied in dependencies involving tags not correctly being resolved (most likely due to misconfigurations. Still investigating and will update if the original solution can be resolved with additional plugins).
The following plugin abstracts the use of !Sub which in turn resolved the issue,
serverless-cloudformation-sub-variables

AWS CDK generated resource identifiers are horrible and not readable. Any way to fix this?

Anyone, that has used AWS CDK suffers from horrible resource identifiers.
Examples of Stacks/Nested Stacks names:
Or examples of resource names:
These identifiers are horrible to read. Is there any work-around to override these identifiers?
I have tried to set ids / names / identifiers / alies of the resources. However it seems that cdk or cloudformation itself is generating these strings.
Thank you for suggestions!
All of resources(or at least for most that I know) could be named manually.
For AWS::EC2::SecurityGroup that would be Properties -> GroupName
AWS::CloudWatch::Alarm - Properties -> AlarmName
AWS::Lambda::Function - Properties -> FunctionName
etc.
But for some of them that would lead to consequences - you won't be able to update some of them, because they might need recreation (and the name is already occupied). So in general it's not a good practice.
And obviously you won't be able to create a full env duplicate not changing some parameter for the generated name like this:
FunctionName: !Sub '${InstanceName}-your-resourse-constant-name-${Environment}'
If you don't specify the naming it would create a name like this:
${stackName}-${resourceNameInCF}-${someHashCode}, but in your case it seems you have nested stacks and it becomes pretty unreadable, especially with long names because of the names chaining.
Yeah, this is a good question. There's two types of IDs here:
Logical ID
Physical ID
Typically the Physical ID can be specified as a property on the resources. For instance, if you are using the CDK, you can set the functionName property when creating your Lambda (as below).
The Logical ID is also added when creating the resource and as you mentioned, however, the Logical ID is derived from a combination of what you specify and where it is within your stack. So, for example, if you have a stack that uses constructs, then this ID will be prefixed with the construct's Logical ID as well... and it's definitely not very readable.
I'd be very careful changing these IDs, especially if you have already deployed the stack, but if you really want to override them then you could do something like this in the CDK (TypeScript):
import {
CfnResource,
} from "#aws-cdk/core";
import {
Function,
Runtime,
Code,
} from "#aws-cdk/aws-lambda";
const consumerLambda = new Function(this, 'LogicalIdOnResource', {
runtime: Runtime.NODEJS_12_X,
handler: 'index.handler',
code: Code.fromAsset(path.join(__dirname, 'lambda-handler')),
functionName: 'ds-di-kafka-consumer-lambda' // PhysicalIdOnResource
});
// Override Logical ID
(consumerLambda.node.defaultChild as CfnResource).overrideLogicalId(
'Consumer'
);
Which looks like this on CloudFormation:

Is there a global properties for cloudformation Resources section?

I have a cloudformation template for my lambda:
Resources:
Resource1:
Type: AWS::Res
Properties:
StreamArn:
"Fn::Sub": "${var1}-${var2}"
Resource2:
Type: AWS::Res
Properties:
StreamArn:
"Fn::Sub": "${var1}-${var2}"
Is it possible to move these properties somewhere to Properties field of Resources section or any other place to avoid duplication?
Resources:
Properties:
StreamArn:
"Fn::Sub": "${var1}-${var2}"
I've tried to do it, but it doesn't work.
You can use a Parameters entry with a default value to create the equivalent to a Constant Variable, but it can't accept any values from the Resources section (since they haven't been created at that point).
Otherwise, no -- you'll need to duplicate the values. (As at the time of writing this answer.)
If you're using AWS::Serverless::Function, you can use Globals section to have common properties in a stack in one place. So, you can put the resources you mentioned in a stack for them and define a Globals section that has StreamArn
See docs
If you are using SAM and the right resources then you can use the globals sections for this:
AWS::Serverless::Function
AWS::Serverless::Api, and
AWS::Serverless::SimpleTable
The Globals section is unique to AWS SAM. It defines properties that
are common to all your serverless functions and APIs. All the
AWS::Serverless::Function
AWS::Serverless::Api, and
AWS::Serverless::SimpleTable
resources inherit the properties that are
defined in the Globals section. For more information about the Globals
section, see Globals Section of the Template in the AWS Serverless
Application Model Developer Guide.
Documentation
The most convenient way that I found so far is to use mapping like:
Mappings:
ParametersMap:
Var1:
Value: "A"
Var2:
Value: "B"
and then put line !FindInMap: [ "ParametersMap", "Var1", "Value" ] in all the places were you need Var1 param

CloudFormation: conditional parameters

Building a CloudFormation stack template, I have a setup constellation where upon instantiation I want to reference either the name of another CloudFormation stack or a non-CloudFormation-managed database as a parameter.
Is there a way to represent this constellation in my template? I.e. "Parameter DatabaseHost is mandatory if Parameter DatabaseStack is blank"?
Maybe it wasn't possible at the time of the question, but now, you can include conditions on a CloudFormation template. See docs.
In this example, I use one value or another depending on the environment:
InfrastructurePipelineStack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Sub "https://<masked>.yml"
Parameters:
ProjectName: !Ref ProjectName
...
LambdaNotifications: !If [isDev, !GetAtt NotificationsStack.Outputs.LambdaNotifications, !Ref LambdaNotifications]
If the environment is Development ("isDev" condition), I use the output of other CloudFormation Stack as the value. If not, I use a provided fixed value (non-CloudFormation value).
In this case "isDev" is acting as "parameter DatabaseStack is blank" in the OP question.
I'm not aware of a native option in CloudFormation to make one template parameter conditional on a second template parameter.
Possible workarounds might be:
make both optional, and tell user to supply one of them
use two templates, one for each of the two use cases
programmatically generate your template after asking the user for parameters