aws_ssm_document unable to validate YAML automation file from S3 - amazon-web-services

Getting a returned error when trying to create a AWS Systems Manager automation document via aws_ssm_document Terraform resource.
Error: creating SSM document: InvalidDocumentContent: YAML not well-formed. at Line: 1, Column: 1
Test for sanity to create the YAML automation document manually using the same document and also to import it inline (which is less than ideal due to the size)
Sample below of the Terraform resource and the YAML document.
resource "aws_ssm_document" "rhel_updates" {
name = "TEST-DW"
document_format = "YAML"
content = "YAML"
document_type = "Automation"
attachments_source {
key = "SourceUrl"
values = ["s3://rhel/templates/101/runbooks/test.yaml"]
name = "test.yaml"
}
}
schemaVersion: '0.3'
description: |-
cloud.support#test.co.uk
parameters:
S3ArtifactStore:
type: String
default: rhel01
description: S3 Artifact Store.
ApiInfrastructureStackName:
type: String
description: API InfrastructureStackName.
default: rhel-api
mainSteps:
- name: getApiInfrastructureStackOutputs
action: 'aws:executeAwsApi'
outputs:
- Selector: '$.Stacks[0].Outputs'
Name: Outputs
Type: MapList
inputs:
Service: cloudformation
Api: DescribeStacks
StackName: '{{ApiInfrastructureStackName}}'

Related

AWS SSM document, syntax wrong, cant create document

So i'm trying to create a SSM document with the instance ID below, so i can then link it to event bridge to trigger a powershell script based on a cloud watch alarm. I have selected Target type: /AWS::EC2::instance and then the YAML below. But it isn't letting me create the document?
---
schemaVersion: "2.2"
description: "Command Document Example JSON Template"
InstanceId: "i-0bbec63d8fee3d6e3"
description: "Instance ID"
mainSteps:
- action: "aws:runPowerShellScript"
name: "RunCommands"
inputs:
runCommand:
- "Restart-Service -Name ColdFusion 2018 Application Server"
Trying to create a SSM document, which will target this specific instance, based on this instance ID
It is not a valid SSM document file.
I believe that bellow example should help.
---
schemaVersion: "2.2"
description: "Example document"
parameters:
Message:
type: "String"
description: "Example parameter"
default: "Hello World"
mainSteps:
- action: "aws:runPowerShellScript"
name: "example"
inputs:
timeoutSeconds: '60'
runCommand:
- "Write-Output {{Message}}"
Source: https://docs.aws.amazon.com/systems-manager/latest/userguide/document-schemas-features.html#documents-schema-twox

Create JSON value in SSM Parameter Store in Cloudformation

I need to create SSM parameter store in Cloudformation to store JSON
Here is my Template
Resources:
WebServersSSM:
Type: AWS::SSM::Parameter
Properties:
AllowedPattern: String
DataType: text
Description: WebServers CloudWatch Agent Configuration
Name: WebServersSSM
Type: String
Tier: Standard
Value: |
{
... My JSON File
}
I am facing error
Parameter value, cannot be validated against allowedPattern: String (Service: AmazonSSM; Status Code: 400; Error Code: ParameterPatternMismatchException; Request ID: a7c2f063-9e63-4b4c-981b-c9ad05e56166; Proxy: null)
The AllowedPattern is a regular expression to validate the pattern and not the expected type. Remove it and it should work.

How to get the ARN of an SSM Document in CloudFormation?

I have a CloudFormation template that creates an AWS::Events::Rule and an AWS::SSM::Document. I need to provide a list of Targets for the SSM::Rule, but each target expects an ARN:
mySSMDocument:
Type: AWS::SSM::Document
Properties:
DocumentType: 'Command'
Content:
schemaVersion: '2.2'
description: "Code that will be run on EC2"
mainSteps:
- action: "aws:runShellScript"
name: runShellScript
inputs:
runCommand:
- 'Some command to execute'
myEventRule:
Type: AWS::Events::Rule
Properties:
Description: "A description for the Rule."
EventPattern:
source:
- "aws.autoscaling"
detail-type:
- "EC2 Instance-terminate Lifecycle Action"
detail:
AutoScalingGroupName:
- !Ref 'someAutoScalingGroupInThisTemplate'
RoleArn: 'some role ARN'
State: "ENABLED"
Targets:
- Id: "some-unique-id"
Arn: <-- This is the value that I need to fill in.
RunCommandParameters:
RunCommandTargets:
- Key: "tag: Name"
Values:
- 'The name of the EC2 machine'
I think that I need to replace the <-- This is the value that I need to fill in. with the ARN of mySSMDocument, but I don't see any way to retrieve this value from within the template itself. The documentation does not specify any GetAtt functionality on SSM::Document that allows to get the ARN. Anyone know how to solve this issue?
This is ARN pattern of Document
arn:${Partition}:ssm:${Region}:${Account}:document/${DocumentName}
example:
arn:aws:ssm:us-east-2:12345678912:document/demoooo
You can use Ref function to get name of document, then Sub to create final ARN
refer: https://docs.aws.amazon.com/IAM/latest/UserGuide/list_awssystemsmanager.html#awssystemsmanager-resources-for-iam-policies
!Sub arn:${AWS::Partition}:ssm:${AWS::Region}:${AWS::AccountId}:document/${mySSMDocument}
You can produce the ARN format for AWS::SSM::Document using the return Value for AWS::SSM::Document, the Pseudo Parameters for Partition, Region, and AccountId, and the Sub intrinsic function

Use a Stackdriver resource group's ID in a GCP Deployment Manager configuration

I'm trying to create a Stackdriver alert policy with a Deployment Manager configuration. The same configuration first creates a resource group and a notification channel and then a policy based on those:
resources:
- name: test-group
type: gcp-types/monitoring-v3:projects.groups
properties:
displayName: A test group
filter: >-
resource.metadata.cloud_account="aproject-id" AND
resource.type="gce_instance" AND
resource.metadata.tag."managed"="yes"
- name: test-email-notification
type: gcp-types/monitoring-v3:projects.notificationChannels
properties:
displayName: A test email channel
type: email
labels:
email_address: incidents#example.com
- name: test-alert-policy
type: gcp-types/monitoring-v3:projects.alertPolicies
properties:
enabled: true
displayName: A test alert policy
documentation:
mimeType: text/markdown
content: "Test incident"
notificationChannels:
- $(ref.test-email-notification.name)
combiner: OR
conditions:
- conditionAbsent:
aggregations:
- alignmentPeriod: 60s
perSeriesAligner: ALIGN_RATE
duration: 300s
filter: metric.type="compute.googleapis.com/instance/uptime" group.id="$(ref.test-group.id)"
trigger:
count: 1
displayName: The instance is down
The policy's only condition has a filter based on the resource group, i.e. only the members of the group could trigger this alert.
I'm trying to use a reference to the group's ID, but it doesn't work - "The reference 'id' is invalid, reason: The field 'id' does not exists on the reference schema.
Also when I try to use $(ref.test-group.selfLink) I get The reference 'selfLink' is invalid, reason: The field 'selfLink' does not exists on the reference schema.
I could get the group's name (e.g. "projects/aproject-id/groups/3691870619975147604") but the filters only accept group IDs (e.g. only the "3691870619975147604" part):
'{"ResourceType":"gcp-types/monitoring-v3:projects.alertPolicies","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"Field
alert_policy.conditions[0].condition_absent.filter had an invalid value of \"metric.type=\"compute.googleapis.com/instance/uptime\"
group.id=\"projects/aproject-id/groups/3691870619975147604\"\":
must specify a restriction on \"resource.type\" in the filter; see \"https://cloud.google.com/monitoring/api/resources\"
for a list of available resource types.","status":"INVALID_ARGUMENT","statusMessage":"Bad
Request","requestPath":"https://monitoring.googleapis.com/v3/projects/aproject-id/alertPolicies","httpMethod":"POST"}}'
Try replacing your alert policy with the following:
- name: test-alert-policy
type: gcp-types/monitoring-v3:projects.alertPolicies
properties:
enabled: true
displayName: A test alert policy
documentation:
mimeType: text/markdown
content: "Test incident"
notificationChannels:
- $(ref.test-email-notification.name)
combiner: OR
conditions:
- conditionAbsent:
aggregations:
- alignmentPeriod: 60s
perSeriesAligner: ALIGN_RATE
duration: 300s
filter: metric.type="compute.googleapis.com/instance/uptime" $(ref.test-group.filter)
trigger:
count: 1
displayName: The instance is down
metadata:
dependsOn:
- test-group
This adds 1) an explicit dependency to test-group using a dependsOn clause and 2) $(ref.test-group.filter) to the metric filter so that it, while not strictly linked to test-group, ends up containing all the same resources as test-group.
As Deployment Manager resources are ran in parallel its necessary to use dependsOn to ensure test-group is instantiated before attempting to create test-alert-policy; apparently Deployment Manager isn't quite smart enough to reason this just by the references.

AWS + Serverless - how to get at the secret key generated by cognito user pool

I've been following the serverless tutorial at https://serverless-stack.com/chapters/configure-cognito-user-pool-in-serverless.html
I've got the following serverless yaml snippit
Resources:
CognitoUserPool:
Type: AWS::Cognito::UserPool
Properties:
# Generate a name based on the stage
UserPoolName: ${self:custom.stage}-moochless-user-pool
# Set email as an alias
UsernameAttributes:
- email
AutoVerifiedAttributes:
- email
CognitoUserPoolClient:
Type: AWS::Cognito::UserPoolClient
Properties:
# Generate an app client name based on the stage
ClientName: ${self:custom.stage}-user-pool-client
UserPoolId:
Ref: CognitoUserPool
ExplicitAuthFlows:
- ADMIN_NO_SRP_AUTH
# >>>>> HOW DO I GET THIS VALUE IN OUTPUT <<<<<
GenerateSecret: true
# Print out the Id of the User Pool that is created
Outputs:
UserPoolId:
Value:
Ref: CognitoUserPool
UserPoolClientId:
Value:
Ref: CognitoUserPoolClient
#UserPoolSecret:
# WHAT GOES HERE?
I'm exporting all my other config variables to a json file (to be consumed by a mobile app, so I need the secret key).
How do I get the secret key generated to appear in my output list?
The ideal way to retrieve the secret key is to use "CognitoUserPoolClient.ClientSecret" in your cloudformation template.
UserPoolClientIdSecret:
Value:
!GetAtt CognitoUserPoolClient.ClientSecret
But it is not supported as explained here and gives message as shown in the image:
You can run below CLI command to retrieve the secret key as a work around:
aws cognito-idp describe-user-pool-client --user-pool-id "us-west-XXXXXX" --region us-west-2 --client-id "XXXXXXXXXXXXX" --query 'UserPoolClient.ClientSecret' --output text
As Prabhakar Reddy points out, currently you can't get the Cognito client secret using !GetAtt in your CloudFormation template. However, there is a way to avoid the manual step of using the AWS command line to get the secret. The AWS Command Runner utility for CloudFormation allows you to run AWS CLI commands from your CloudFormation templates, so you can run the CLI command to get the secret in the CloudFormation template and then use the output of the command elsewhere in your template using !GetAtt. Basically CommandRunner spins up an EC2 instance and runs the command you specify and saves the output of the command to a file on the instance while the CloudFormation template is running so that it can be retrieved later using !GetAtt. Note that CommandRunner is a special custom CloudFormation type that needs to be installed for the AWS account as a separate step. Below is an example CloudFormation template that will get a Cognito client secret and save it to AWS Secrets manager.
Resources:
CommandRunnerRole:
Type: AWS::IAM::Role
Properties:
# the AssumeRolePolicyDocument specifies which services can assume this role, for CommandRunner this needs to be ec2
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action: 'sts:AssumeRole'
Path: /
Policies:
- PolicyName: CommandRunnerPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 'logs:CancelUploadArchive'
- 'logs:GetBranch'
- 'logs:GetCommit'
- 'cognito-idp:*'
Resource: '*'
CommandRunnerInstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Roles:
- !Ref CommandRunnerRole
GetCognitoClientSecretCommand:
Type: AWSUtility::CloudFormation::CommandRunner
Properties:
Command: aws cognito-idp describe-user-pool-client --user-pool-id <user_pool_id> --region us-east-2 --client-id <client_id> --query UserPoolClient.ClientSecret --output text > /command-output.txt
Role: !Ref CommandRunnerInstanceProfile
InstanceType: "t2.nano"
LogGroup: command-runner-logs
CognitoClientSecret:
Type: AWS::SecretsManager::Secret
DependsOn: GetCognitoClientSecretCommand
Properties:
Name: "command-runner-secret"
SecretString: !GetAtt GetCognitoClientSecretCommand.Output
Note that you will need to replace the <user_pool_id> and <client_id> with your user pool and client pool id. A complete CloudFormation template would likely create the Cognito User Pool and User Pool Client and the user pool & client id values could be retrieved from those resources using !Ref as part of a !Join statement that creates the entire command, e.g.
Command: !Join [' ', ['aws cognito-idp describe-user-pool-client --user-pool-id', !Ref CognitoUserPool, '--region', !Ref AWS::Region, '--client-id', !Ref CognitoUserPoolClient, '--query UserPoolClient.ClientSecret --output text > /command-output.txt']]
One final note, depending on your operating system, the installation/registration of CommandRunner may fail trying to create the S3 bucket it needs. This is because it tries to generate a bucket name using uuidgen and will fail if uuidgen isn't installed. I have opened an issue on the CommandRunner GitHub repo for this. Until the issue is resolved, you can get around this by modifying the /scripts/register.sh script to use a static bucket name.
As it is still not possible to get the secret of a Cognito User Pool Client using !GetAtt in a CloudFormation Template I was looking for an alternative solution without manual steps so the infrastructure can get deployed automatically.
I like clav's solution but it requires the Command Runner to be installed first.
So, what I did in the end was using a Lambda-backed custom resource. I wrote it in JavaScript but you can also write it in Python.
Here is an overview of the 3 steps you need to follow:
Create IAM Policy and add it to the Lambda function execution role.
Add creation of In-Line Lambda function to CloudFormation Template.
Add creation of Lambda-backed custom resource to CloudFormation Template.
Get the output from the custom Ressource via !GetAtt
And here are the details:
Create IAM Policy and add it to the Lambda function execution role.
# IAM: Policy to describe user pool clients of Cognito user pools
CognitoDescribeUserPoolClientsPolicy:
Type: AWS::IAM::ManagedPolicy
Properties:
Description: 'Allows describing Cognito user pool clients.'
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- 'cognito-idp:DescribeUserPoolClient'
Resource:
- !Sub 'arn:aws:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/*'
If necessary only allow it for certain resources.
Add creation of In-Line Lambda function to CloudFormation Template.
# Lambda: Function to get the secret of a Cognito User Pool Client
LambdaFunctionGetCognitoUserPoolClientSecret:
Type: AWS::Lambda::Function
Properties:
FunctionName: 'GetCognitoUserPoolClientSecret'
Description: 'Lambda function to get the secret of a Cognito User Pool Client.'
Handler: index.lambda_handler
Role: !Ref LambdaFunctionExecutionRoleArn
Runtime: nodejs14.x
Timeout: '30'
Code:
ZipFile: |
// Import required modules
const response = require('cfn-response');
const { CognitoIdentityServiceProvider } = require('aws-sdk');
// FUNCTION: Lambda Handler
exports.lambda_handler = function(event, context) {
console.log("Request received:\n" + JSON.stringify(event));
// Read data from input parameters
let userPoolId = event.ResourceProperties.UserPoolId;
let userPoolClientId = event.ResourceProperties.UserPoolClientId;
// Set physical ID
let physicalId = `${userPoolId}-${userPoolClientId}-secret`;
let errorMessage = `Error at getting secret from cognito user pool client:`;
try {
let requestType = event.RequestType;
if(requestType === 'Create') {
console.log(`Request is of type '${requestType}'. Get secret from cognito user pool client.`);
// Get secret from cognito user pool client
let cognitoIdp = new CognitoIdentityServiceProvider();
cognitoIdp.describeUserPoolClient({
UserPoolId: userPoolId,
ClientId: userPoolClientId
}).promise()
.then(result => {
let secret = result.UserPoolClient.ClientSecret;
response.send(event, context, response.SUCCESS, {Status: response.SUCCESS, Error: 'No Error', Secret: secret}, physicalId);
}).catch(error => {
// Error
console.log(`${errorMessage}:${error}`);
response.send(event, context, response.FAILED, {Status: response.FAILED, Error: error}, physicalId);
});
} else {
console.log(`Request is of type '${requestType}'. Not doing anything.`);
response.send(event, context, response.SUCCESS, {Status: response.SUCCESS, Error: 'No Error'}, physicalId);
}
} catch (error){
// Error
console.log(`${errorMessage}:${error}`);
response.send(event, context, response.FAILED, {Status: response.FAILED, Error: error}, physicalId);
}
};
Make sure you pass the right Lambda Execution Role to the parameter Role. It should contain the policy created in step 1.
Add creation of Lambda-backed custom resource to CloudFormation Template.
# Custom: Cognito user pool client secret
UserPoolClientSecret:
Type: Custom::UserPoolClientSecret
Properties:
ServiceToken: !Ref LambdaFunctionGetCognitoUserPoolClientSecret
UserPoolId: !Ref UserPool
UserPoolClientId: !Ref UserPoolClient
Make sure you pass the Lambda function created in step 2 as ServiceToken. Also make sure you pass in the right values for the parameters UserPoolId and UserPoolClientId. They should be taken from the Cognito User Pool and the Cognito User Pool Client.
Get the output from the custom Ressource via !GetAtt
!GetAtt UserPoolClientSecret.Secret
You can do this anywhere you want.