I want to trigger a lambda whenever a new EC2 instance is registred in SSM's Fleet Manager (meaning the instance can be connected to using SSM), however I can't find what pattern to use in EventBridge.
Within EventBridge, I tried using the following pattern I found in the docs (so far its looks like the closest thing to my goal):
{
"source": ["aws.ssm"],
"detail-type": ["Inventory Resource State Change"]
}
However when I create a new EC2 and wait for its SSM agent to become active, it still doesn't trigger the above pattern.
Any idea how to catch this kind of event?
I think you have to go through CloudTrail API call.
Please find below a CloudFormation template I used in the past that was working. Please note that it just provides the SSM resources. You need to add your own SQS queue as well (see SQS.ARN) and I've used the association with the tag registration set to enabled. So that if you have a lambda function connected, you can set it to false so if the instance connect again, it won't go to the same process again.
AWSTemplateFormatVersion: "2010-09-09"
Description: >
SSM Registration event
# Description of the resources to be created.
Resources:
RegistrationDocument:
Type: AWS::SSM::Document
Properties:
DocumentType: Command
Content:
schemaVersion: "2.2"
description: >
An Automation Document ran by registered instances that gathers their software inventory
and automatically updates their AWS SSM Agent to the latest version.
mainSteps:
- name: GatherSoftware
action: aws:softwareInventory
- name: Sleep
action: aws:runShellScript
inputs:
runCommand:
- sleep 20 || true
- name: UpdateAgent
action: aws:updateSsmAgent
inputs:
agentName: amazon-ssm-agent
source: https://s3.{Region}.amazonaws.com/amazon-ssm-{Region}/ssm-agent-manifest.json
allowDowngrade: "false"
RegistrationDocumentAssociation:
Type: AWS::SSM::Association
Properties:
AssociationName: !Sub registration-association-${AWS::StackName}
Name: !Ref RegistrationDocument
Targets:
- Key: tag:registration
Values:
- enabled
RegistrationEventRule:
Type: AWS::Events::Rule
Properties:
Description: >
Events Rule that monitors registration of AWS SSM instances
and logs them to an SQS queue.
EventPattern:
source:
- aws.ssm
detail-type:
- AWS API Call via CloudTrail
detail:
eventName:
- UpdateInstanceAssociationStatus
requestParameters:
associationId:
- !Ref RegistrationDocumentAssociation
executionResult:
status:
- Success
State: ENABLED
Targets:
- Arn: SQS.ARN
Id: SqsRegistrationSubscription
SqsParameters:
MessageGroupId: registration.events
Related
I'm trying to create a cloudFormation stack with MSK Configuration and associating MSK Configuration with MSK Cluster. Creation of AWS::MSK::Configuration returns only ARN while I need ARN and Revision number to associate MSK Configuration to MSK Cluster. Is there any way to achieve this? Currently I'm hard-coding it to 1 which means it will work only for creating stack.
...
MSKConfiguration:
Type: AWS::MSK::Configuration
Properties:
Name: aws-msk-configuration
ServerProperties: |
auto.create.topics.enable = true
zookeeper.connection.timeout.ms = 1000
log.roll.ms = 604800000
MSKCluster:
Type: AWS::MSK::Cluster
Properties:
ClusterName: !Ref ClusterName
ClientAuthentication: !If
- UsingIamAuthentication
- Sasl:
Iam:
Enabled: true
- Sasl:
Scram:
Enabled: true
ConfigurationInfo:
Arn: !GetAtt MSKConfiguration.Arn
Revision: 1
...
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-msk-configuration.html
You can only get the latest version if you define a custom resource. Since you program the full logic of the resource, you can do what you want, including automatically setting up latest version for MKS.
The task is simple: whenever an EC2 instance is launched with tag key:value I want it to install a specific software. Whenever an EC2 instance is launched with a different tag key:value I want it to install a different software.
I understand that I can create 2 different associations in State Manager that uses runCommand RuneRemoteScript to install software based on the tags, but the goal is to have 1 composite document that can do this.
Any help / guidance would be appreciated!
You can achieve that using SSM Automation documents - https://docs.aws.amazon.com/systems-manager/latest/userguide/automation-branchdocs.html
However, probably you will need to do something like this:
In the State Manager use AWS-RunDocument,
This document should execute SSM Automation document (your Composite document)
Your Composite document should look like this:
I didn't validate this template, and I assume It shouldn't work without a few days of debugging
schemaVersion: '0.3'
parameters:
InstanceId:
type: String
mainSteps:
- name: DescribeEc2
action: 'aws:executeScript'
inputs:
Runtime: python3.7
Handler: script_handler
Script: |
import json
import boto3
def script_handler(events):
ec2_instance = boto3.client('ec2').describe_instances(
InstanceIds=events["instance_id"],
)["Reservations"][0]["Instances"][0]
# thread it like an example,
# Here you should parse your tags and decide what software you
# want to install on the provided instance
return json.dumps(
{
"to_be_installed": "result"
},
sort_keys=True,
default=str
)
InputPayload:
instance_id: '{{ InstanceId }}'
Outputs:
- Name: result
Selector: "$.to_be_installed"
- name: WhatToInstall
action: aws:branch
inputs:
Choices:
- NextStep: InstallSoft1
Variable: "{{DescribeEc2.result}}"
StringEquals: soft_1
- NextStep: InstallSoft1
Variable: "{{DescribeEc2.result}}"
StringEquals: soft_2
- name: InstallSoft1
action: aws:runCommand
inputs:
DocumentName: AWS-RunShellScript
InstanceIds:
- '{{ InstanceId }}'
Parameters:
commands:
...
- name: InstallSoft2
action: aws:runCommand
inputs:
DocumentName: AWS-RunShellScript
InstanceIds:
- '{{ InstanceId }}'
Parameters:
commands:
...
Tbh, you will find a lot of troubles with such solution (IAM and SSM specific issues), so I will recommend using Event Bridge -> Lambda Function(that decides which Document/Automation should be run) -> SSM-RunDocument (executed directly in the Lambda Function).
I have the following policy:
policies:
- name: stop-after-hours
resource: ec2
filters:
- tag:Schedule: "OfficeHours"
actions:
- stop
mode:
type: periodic
schedule: "rate(10 minutes)"
role: arn:aws:iam::XXXXXX:role/LambdaRoleCloudCustodian
Which correctly identified my EC2 tagged with "Schedule: OfficeHours":
$> custodian run --dry-run -s out shutdown-out-of-office.yml
custodian.policy:INFO policy:stop-after-hours-cologne resource:ec2 region:eu-central-1 count:1 time:0.00
However, when I want to set the offhour:
policies:
- name: stop-after-hours
resource: ec2
filters:
- tag:Schedule: "OfficeHours"
- type: offhour
offhour: 11
actions:
- stop
mode:
type: periodic
schedule: "rate(10 minutes)"
role: arn:aws:iam::XXXXXX:role/LambdaRoleCloudCustodian
The instance is not identified anymore.
2022-07-05 12:01:04,541: custodian.policy:INFO policy:stop-after-hours-cologne resource:ec2 region:eu-central-1 count:0 time:0.78
I also tried
- type: value
key: tag:Schedule
value: OfficeHours
which doesn't work.
Any idea on how I can filter on tag name AND value here?
So, after fiddling around quite some time, I finally found the solution.
Here's the complete policy
# Stop instances tagged with "Schedule: OfficeHour" at offhour
- name: stop-after-hours
resource: ec2
filters:
- tag:Schedule: OfficeHours
- State.Name: running
- type: offhour
tag: Schedule
weekends: true
default_tz: cet
offhour: 10
actions:
- stop
mode:
type: periodic
schedule: "rate(10 minutes)"
role: arn:aws:iam::XXXXXXXXX:role/LambdaRoleCloudCustodian
Some things to keep in mind
Here, under filters/type, I have a tag attribute for which the value is Schedule. This will tell Cloudcustodian to look for any instance which has the tag Schedule, whatever its value. If you do not specify this, you need to tag your instance with the default offhour tag which is maid_offhours
I also have tag:Schedule: OfficeHours which will filter out instances based on the tag Schedule's value.
If you want to test your policy with a dry-run, you must test in the current hour. So, if my offhour is set to 10, then the dry-run will only be able to fetch the resource if it is run between 10:00am and 10:59am.
I hope it helps some people, I find the Cloudcustodian documentation quite difficult to understand.
I'm trying to deploy Step Function, but I see no ways to define activity in serverless config.
AWS docs https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-stepfunctions-activity.html saying that activity should be defined that way, but every time I do sls deploy I can't see new activity in step function console. Is it possible at all to create activity via serverless or I have to run script/create it manually?
Resources:
MyActivity:
Type: "AWS::StepFunctions::Activity"
Properties:
Name: myActivity
stepFunctions:
stateMachines:
stepfunctest:
events:
- http:
path: step
method: get
definition:
Comment: "A sample application"
StartAt: extract
States:
extract:
Type: Task
Resource: "arn:aws:state:#{AWS::Region}:#{AWS::AccountId}:activity:MyActivity"
End: true
assuming you're uing the serverless plugin https://github.com/serverless-operations/serverless-step-functions. You can create the activity by adding the activity into the stepFunction
stepFuntions:
activities:
- myActivity
stateMachines:
stepfunctest:
events:
- http:
path: step
method: get
definition:
Comment: "A sample application"
StartAt: extract
States:
extract:
Type: Task
Resource: "arn:aws:state:#{AWS::Region}:#{AWS::AccountId}:activity:MyActivity"
End: true
As described in the documentation, when creating a RDS Event subscription you can select any number Event categories that will produce specific Event messages.
Then you can choose to send a notification to an E-mail, SMS or, which is my case, to a SNS topic that triggers a Lambda execution.
How to access the RDS Event ID - i.e. RDS-EVENT-0006 - from the Lambda event parameter?
Add a trigger event like this cloudformation example :
DbRestoredEventRule:
Type: AWS::Events::Rule
Properties:
Name: "xyz-db-restored"
Description: "xyz restored"
EventPattern:
source:
- "aws.rds"
detail-type:
- "RDS DB Instance Event"
detail:
EventCategories:
- "availability"
Message:
- 'DB instance restarted'
Targets:
- Arn:
Fn::GetAtt:
- "MigrationDataFunction"
- "Arn"
Id: "TargetFunctionV1"