Is there a way to store variables in Cloudformation?
I've created a resource with a name which is a stage specific name in the following form:
DeliveryStreamName: {'Fn::Sub': ['firehose-events-${Stage}', 'Stage': {'Ref' : 'Stage' }]}
Now if I've to create a cloudwatch alarm on that resource I'm again following the same pattern:
Dimensions:
- Value: {'Fn::Sub': ['firehose-events-${Stage}', 'Stage': {'Ref' : 'Stage' }]}
Instead if I could store the whole value in one variable, it would be much easier for me to refer it.
I thought initially storing it in parameters, like this:
Parameters:
FirehoseEvent: {Type:String, Default: 'firehose-events-${Stage}'}
But the stage value doesn't seem to get passed in here. And there is no non default value either for this resource name.
The other option I considered was using mapping, but that defeats the purpose of using ${Stage}.
Is there some other way which I've missed?
Sadly you haven't missed anything. Parameters can't reference other parameters in their definition.
The only way I can think of doing what you which would be through a custom macro. In its simplest form the macro would just perform traditional find-and-replace type of template processing.
However, the time required to develop such macro could be not worth its benefits, at least in this simple example you've provided in the question.
Related
I'd like to make CloudFormation look in Parameter Store for a particular parameter - and if not found, look for a different one.
The actual use case is that stacks are deployed for different branches and we'd like to have top-level parameters for all branches that can optionally be overridden by creating a branch-specific parameter. Something like this:
dev-param = 120 <-- top-level, applies if branch-specific parameter doesn't exist
dev-param.mybranch = 60 <-- branch-specific parameter
Have tried a couple of ways but got an error when deploying the stack for both - see below.
When attempting to use dynamic references:
Parameters: [ssm:dev-param.mybranch] cannot be found.
When attempting to use CloudFormation SSM Parameter Types.
Template format error: Every Default member must be a string.
For the latter, the Default: field specifies the Parameter Store key name. This needs to be generated dynamically from other CloudFormation parameters, e.g. there is a parameter for the environment type so development key names begin with dev- and production keys begin with prod-.
Is there another way to achieve this?
You can't do that without a custom resource or a macro.
I'm trying in vain to do this. Here is the scenario:
We are creating a CloudFormation Stack that will generate a CodePipeline, that will pull another stack definition from git and deploy it, using the CloudFormationCreateUpdateStackAction
The repo and branch etc. are provided as CfnParamaters and the subsequent stack name we would like to base off a concatenation of the repo name and branch name.
However in some cases the repo might be named with an underscore or other special character that is invalid for a stackName.
I've attempted to manipulate the parameter strings using Macro's but I didnt get anywhere near something useful and running a .replace() function on the repoStr.valueAsString property modifies the CDK's "Token" pointer, not the resulting paramter which is declared at runtime.
Any ideas?
There are two options I can see:
First is to read the actual parameter value at synthesis time, then you can use the .replace function on parameter value. https://docs.aws.amazon.com/cdk/api/latest/docs/#aws-cdk_aws-ssm.StringParameter.html#static-value-wbr-from-wbr-lookupscope-parametername
Second is to use CloudFormation intristic function to split the parameter and join the fragments back in allowed manner. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-split.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-join.html
Anyone, that has used AWS CDK suffers from horrible resource identifiers.
Examples of Stacks/Nested Stacks names:
Or examples of resource names:
These identifiers are horrible to read. Is there any work-around to override these identifiers?
I have tried to set ids / names / identifiers / alies of the resources. However it seems that cdk or cloudformation itself is generating these strings.
Thank you for suggestions!
All of resources(or at least for most that I know) could be named manually.
For AWS::EC2::SecurityGroup that would be Properties -> GroupName
AWS::CloudWatch::Alarm - Properties -> AlarmName
AWS::Lambda::Function - Properties -> FunctionName
etc.
But for some of them that would lead to consequences - you won't be able to update some of them, because they might need recreation (and the name is already occupied). So in general it's not a good practice.
And obviously you won't be able to create a full env duplicate not changing some parameter for the generated name like this:
FunctionName: !Sub '${InstanceName}-your-resourse-constant-name-${Environment}'
If you don't specify the naming it would create a name like this:
${stackName}-${resourceNameInCF}-${someHashCode}, but in your case it seems you have nested stacks and it becomes pretty unreadable, especially with long names because of the names chaining.
Yeah, this is a good question. There's two types of IDs here:
Logical ID
Physical ID
Typically the Physical ID can be specified as a property on the resources. For instance, if you are using the CDK, you can set the functionName property when creating your Lambda (as below).
The Logical ID is also added when creating the resource and as you mentioned, however, the Logical ID is derived from a combination of what you specify and where it is within your stack. So, for example, if you have a stack that uses constructs, then this ID will be prefixed with the construct's Logical ID as well... and it's definitely not very readable.
I'd be very careful changing these IDs, especially if you have already deployed the stack, but if you really want to override them then you could do something like this in the CDK (TypeScript):
import {
CfnResource,
} from "#aws-cdk/core";
import {
Function,
Runtime,
Code,
} from "#aws-cdk/aws-lambda";
const consumerLambda = new Function(this, 'LogicalIdOnResource', {
runtime: Runtime.NODEJS_12_X,
handler: 'index.handler',
code: Code.fromAsset(path.join(__dirname, 'lambda-handler')),
functionName: 'ds-di-kafka-consumer-lambda' // PhysicalIdOnResource
});
// Override Logical ID
(consumerLambda.node.defaultChild as CfnResource).overrideLogicalId(
'Consumer'
);
Which looks like this on CloudFormation:
I have been trying this example provided in the Google's Deployment Manager GitHub project.
It works, yet I am not sure what is the purpose of creating three instances named instance_create, instance_update and instance_delete.
For example, taken from the link:
instance_create = {
'name':
'instance_create',
'action':
'gcp-types/bigtableadmin-v2:bigtableadmin.projects.instances.create',
'properties': {
'parent': project_path,
'instanceId': instance_name,
'clusters': copy.deepcopy(initial_cluster),
'instance': context.properties['instance']
},
'metadata': {
'runtimePolicy': ['CREATE']
}
}
What is the purpose of `action` and `metadata`.`runtimePolicy`? I have tried to find it in the documentation but failed miserably.
Why there are three `BigTable` instances there?
You are right, the documentation is missing the information, which would answer your questions regarding these parameters.
However, it helps knowing what's going on in the Depoyment Manager example you linked.
First of all, the following line in the config.yaml is where the things get tricky:
resources:
- name: my-bigtable
type: bigtable.py
This line will do a call to the bigtable.py python file, which sets the resource type of the deployment to that which are in it, under the GenerateConfig function. See how this is done here.
The resources are returned as {'resources': resources} at the end of it, being the resources variable a list of templates created there.
These templates have different name identifiers, which are set by the "name" tag.
So you are not creating three different instances with the name of instance_create, instance_update and instance_delete in this file, but you are creating three templates with those names, that will later be appended to the resources list, and later returned to the config.yaml resources.type tag.
These templates then will be sequentially build and executed by the deployment manager, once the create command is used. Note that they might appear out of order, this is due not using a schema.
It's easier to see this structure in a .yaml file format, for example, built with jinja, the template you posted would be:
resources:
- action: gcp-types/bigtableadmin-v2:bigtableadmin.projects.instances.create
name: instance_create
metadata:
runtimePolicy:
- CREATE
properties:
clusters:
initial:
defaultStorageType: HDD
location: projects/<PROJECT_ID>/locations/<PROJECT_LOCATION>
serveNodes: 4
instance:
displayName: My BigTable Instance.
type: PRODUCTION
instanceId: my-instance
parent: projects/<PROJECT_ID>
Notice that the parameters under properties are the fields in the request body to bigtableadmin.projects.instances.create (which is nesting a clusters object parameters and a instance object parameters). Note that the InstanceId under properties is always the same, hence the BigTable instance, on which the templates do the calls, is always the same one.
The thing is that, not only the example you linked creates various templates to be run in the same script, but that the resource type for each template is a call to the BigTable API.
Normally the template resources are specified with the type tag, but since you are calling a resource that is directly running an API call (i.e. instead of just specifying gcp-types/bigtableadmin-v2, you are specifying bigtableadmin-v2:bigtableadmin.projects.instances.create), the action tag is used. I haven't found this difference on usage documented anywhere, but it needs to be specified like that.
You will know if you are calling an API 'endpoint' directly if the resource ends with either create/update/delete.
Finally, the I have investigated in my side, and the metadata.runtimePolicy is tied to the fact that the resource type is an API call (like in the previous point). Once again, I haven't found this documented anywhere.
However, since this is a requirement, you will always have to set the correct value in this field. It basically boils down to have metadata.runtimePolicy set to this values, depending on which type of API call you do:
create -> ['CREATE']
update -> ['UPDATE_ON_CHANGE']
delete -> ['DELETE']
Summarizing:
You are not creating three different instances, but three different templates, which do the work on the same BigTable instance.
You need to change the resource type flag to action if you are calling an API endpoint (create/update/delete), instead of just naming the base API.
The metadata.runtimePolicy value is a requirement when doing a call to one of the aforenamed endpoints.
Would like know if CFT API's or logs from Cloudtrail can provide any intermediate CFT before or while creating the resources. When I mean intermediate CFT(s) , We know the CFT can be kind of dynamic in the form of parameters /condition/mappings/functions those are to be evaluated at run time. I would like to know if this CFT can generate the processed CFT (with all the processed rules/parameters from input/functions...) as though it looks like static for the resource creation process. This approach really helps us in validating the real CFT that is going to be executed with all the values replaced. I'm just looking for the another CFT API. something like ,
String staticCFT = cftClient.getActualCFT("cft_location\cft.json","parameters"...);
If this feature is available , it really saves time and don't have to wait until all the resources are created with wrong values because of wrong logic in CFT.
What you can actually create is what I call a "dummy template". I use it at work as a stand-in to an actual template with real resources which would take time to execute. The dummy template has only one resource which does not actually do anything. I use a CustomResource to invoke a "HelloWorld" Lambda function. This is to get around the restriction that a CFT must have at least one resource. The template also has a bunch of parameters and all those parameters are simply supplied directly to the Outputs section. The execution of this template will hardly take a few seconds and based on the parameters and outputs, you can figure out whether your top-level template is passing in the expected values of the parameters. You can invoke the dummy template from within the top-level template.