There's an AWS CloudFormation stack which defines an SNS topic and an SNS subscription. In another Ansible task, I want to update another subscription, but this isn't possible as SNS subscriptions can't be updated.
Using Ansible to perform CloudFormation. What are the alternatives?
Club AWS CLI with Ansible and then execute plays which contain AWS CLI content?
Create a custom module in Ansible using boto? But this would be difficult as I should store SNS ARN's and give those to the custom module.
Ansible v2.0 added support of an sns_topic module. You can provide a name or ARN of an existing SNS topic to converge. You also probably want to set purge_subscriptions to False so any existing subscriptions are not removed.
- name: Update SNS topic subscriptions
sns_topic:
name: "my-topic"
purge_subscriptions: False
subscriptions:
- endpoint: "some-email#example.com"
protocol: "email"
http://docs.ansible.com/ansible/sns_topic_module.html
Related
I have an SQS Queue that is subscribed to an SNS Topic and everythning is working as expected. Since I am using Cloudformation for infrastructure provisioning, a few years ago, AWS didn't support RawMessageDelivery via cloudformation, therefore, I had to use a custom resource (lambda) for subscribing our SQS queue to SNS Topic with RawMessageDelivery activated.
Now, cloudformation AWS::SQS::Subscription supports RawMessageDelivery and we can remove the custom resources (lambdas) being used for SNS subscription and use the script below in the template.
MyQueueSubscription:
Type: 'AWS::SNS::Subscription'
Properties:
TopicArn: !Ref TopicArn
Endpoint: !GetAtt
- AbcdEventQueue
- Arn
Protocol: sqs
RawMessageDelivery: 'true'
Now the problem is that custom resource (lambda) that is doing the subscription currently is a separate AWS Stack and if I remove this stack, it will also delete the SQS subscription. Although, I can quickly sync the Queue stack with the code above, however, there's a possibility of losing some messages in this case.
What is the best approach to updating the subscription/stack without losing any messages from the SNS topic to my queue?
Generally what means Property "Not currently supported by AWS CloudFormation" for a CDK implementation, specifically:
In the CloudFormation Properties for the Cognito Userpool Lambda Config it says:
CustomEmailSender - Not currently supported by AWS CloudFormation.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-cognito-userpool-lambdaconfig.html
In the CDK for Cognito.CfnUssrPool this property is described:
https://docs.aws.amazon.com/cdk/api/latest/docs/#aws-cdk_aws-cognito.CfnUserPool.LambdaConfigProperty.html#customemailsender
My question now is whether this can be implemented with CDK at all? Currently, our Cognito is provided completely via CDK and I would like to keep it that way.
Edit:
I found a link (Using CustomEmailSender with CFN) where it says that contrary to the documentation it does seem to work and only the documentation has not been updated, I will test this and give feedback.
After testing CustomEmailSender CDK implementation, I have to say that the AWS CloudFormation Documentation hasn't a current state. So it is possible to use this function by CFN and CDK. AWS Doc CFN Cognito CustomEmailSender
ToDos in CDK:
Configure Cognito: instead of using the property emailConfiguration you have to use lambdaConfig:
lambdaConfig: {
customEmailSender: { lambdaArn: customSenderEmailLambdaArn, lambdaVersion: 'V1_0' },
// the version is an ENUM so be careful when you set it
kmsKeyId: kmsKeyArn,
},
As you see here you have also to set up a KMS Key (you can adjust an existing one or create a new KMS key). The most important thing here is that you allow the action kms:CreateGrant to Cognito AND the Lambda function!
Another important ToDo is to add your Lambda the permission for Cognito, action cognito-idp:InvokeFunction.
The big advantage of using CustomEmailSender instead of the common Lambda Triggers that you don't have to set all the triggers in you CDK code or in the Cognito Console (all email events will be sent automatically to your lambda).
I have created SNS using cloud formation like this -
MYSNS:
Type: AWS::SNS::Topic
Properties:
DisplayName: "MYSNS"
TopicName: "MYSNS"
Now, I want to publish message to this topic using Java but
snsClient.publish(snsARN, snsEvent);
sns client requires ARN to publish the event. How I can get SNS Topic ARN?
Edit 1: Post deployment we can get the ARN, but don't want this way.
Ex - We can get SQS url by it's Name using SQS Client like this -
sqsClient.getQueueUrl(dlqName).getQueueUrl(); I am looking for similar way in case of SNS.
You can construct the ARN yourself:
arn:aws:sns:<region>:<account>:MYSNS
You can try this in cloudformation template to get the created SNS's ARN
!Ref MYSNS
I'm creating a logs aggregator lambda to send Cloudwatch logs to a private log analysis service. Given the number of resources used by my employer, it was decided to create a subscription lambda that handles log group subscription to the aggregator.
The solution works fine, but it requires to manually search a resource's log group via amazon console and then invoke the subscription lambda with it.
My question:
Is there a way to, given a resource arn, find which log group is mapped to it? Since I'm using Cloudformation to create resources it is easy to export a resource's arn.
UPDATE
To present an example:
Let's say I have the following arn:
arn:aws:appsync:<REGION>:<ACCOUNTID>apis/z3pihpr4gfbzhflthkyjjh6yvu
which is an Appsync GraphQL API.
What I want it a method (using te API or some automated solution) to get the Cloudwatch log group of that resource.
You can try the describe-log-groups command. It is available on the cli, must also be there on the API.
To get the names of the log groups you can go with:
aws logs describe-log-groups --query 'logGroups[*].logGroupName' --log-group-name-prefix '/aws/appsync/[name-of-the-resource]'
Output will look like this:
[
"/aws/appsync/[name-of-your-resource]"
]
When I am subscribing a lambda function to an SNS topic through either the aws console or aws cli there is a column shown in the subscription information called "subscriber". It does contain my account id.
I don't seem to be able to control this field through the cli or console however I can through ansible:
---
- name: set up topic and subscribe lambda
sns_topic:
name: "topic_name"
state: present
display_name: "Display Name"
subscriptions:
- endpoint: "arn:aws:lambda:ap-southeast-2:123456789:function:functionName"
protocol: "lambda"
subscriber: 123456789
The subscriber field is optional, however SNS messages do not seem to reach lambda without it.
What is this field for?
Can I subscribe Lambdas in other accounts to my SNS topic?
How can I effect change in this field with AWS CLI?
Why is it optional in Ansible 2.4
Couldn't find reference to ghost field i.e. subscriber.
Maybe it was supported in older sdk/api and now has been dropped.
Reseach:
I don't see subscriber field in the console when subscribing lambda. Image below
Even through CLI, there is no such field
Link for SNS Subscribe command through CLI documentation.
[Extra Research]: Even the java sdk asks for only three parameters in order to subscribe
subscribe(String topicArn, String protocol, String endpoint)
Java doc link