I need help in how to set up AWS cross account messaging using EventBridge? Like a step by step guide.
I looked online and here is the article I looked here at: Sending and Receiving Events Between AWS Accounts - Amazon CloudWatch Events but I am not sure if this article is the right article for the step by step.
Also, is sending and receiving events the same thing as sending and receiving messages in EventBridge? I'd really appreciate your help.
Related
I have a Streamsets pipeline that sends events to AWS SNS currently (Using HTTP client). I now have a requirement to send these events to AWS MSK instead of AWS SNS.
I am not finding a relevant documentation to start with this. Don't know where and how to start.
Can someone help in pointing to a relevant documentation which will be much helpful in starting this change?
Any help will be much appreciated. Thanks much in advance.
MSK is just a managed service around Apache Kafka. In StreamSets there is a Kafka connector.
MSK can use certificates (mTLS) for authentication, and this is the method that might work for StreamSets.
We have an Elastic Beanstalk deployment and wanted to receive events into our Slack channel. We setup EventBridge and Chatbot to deliver notifications. Everything works, except the notifications are useless, because Chatbot does not map the message detail only the detail type, which does not tell much. See the attached image of a notification (cropped to cut out the region/account info), the message is not too useful without telling us what was the actual change, which is in the event received by EventBridge and sent to SNS, but not mapped by Chatbot when sending to Slack.
Even their example test notification demonstrates the same problem:
I checked the Chatbot documentation where they claim support for multiple services via EventBridge. I checked EventBridge documentation where they list Elastic Beanstalk among supported services.
I also subscribed and tested Health service notifications, which work just fine. See the example test notification below.
I could not find a way to configure Chatbot to do the right thing. I could write my own message transformer for certain events and call a Slack webhook instead of using Chatbot, but I wanted to avoid writing custom code here for now.
Does anyone know if it is possible to fix Beanstalk notifications that are sent from EventBridge to Chatbot then to Slack? Or is AWS working on supporting more service notifications and in the future this might start working as it should?
Thank you in advance.
I have a lambda function that is invoking around 60million times from one sns topic, in the sns topic metrics i could see it is publish around 600k msgs but I wanted to know how we can check which iam user has made the publish message request in that topic and how can we get the IP from which message are published?
As for as I know there is no way you can monitor what you are asking for. What you can do is to search ClouTrail logs and figure out yourself which user made how many requests to sns. There is also a way to put the logs(CloudTrail) in CloudWatch but it's not cheap process. If you are interested, you check this.
Hi I am developing an application in Java that parses documents using AWS Textract. For multipage parse I use the startDocumentAnalysis method available on the textract client. This client then responds with a jobId and the jobId is placed on the sqs queue along with it's completion status. I had a personal account previously and managed to get all of it working so a process reads messages from the queue and then get the result of the parse using :
GetDocumentAnalysisRequest documentAnalysisRequest = GetDocumentAnalysisRequest.builder().jobId(jobId)
.maxResults(maxResults).nextToken(paginationToken).build();
Now i have to do that on my company's AWS account and i have retraced my steps and everything works except now when i go to pick up the messages from the queue, all the messages come back as [] i.e. empty array.
I use this code:
messages = sqsClient.receiveMessage(receiveMessageRequest).messages();
So i know the error can't be in the java code because previously it worked.
I also know that i can upload to the new bucket and also do single page parses, so i know my credentials are correct.
I have created a topic on sns and registered my sqs to it but somewhere here there must be a permission or other configuration error. Maybe SNS and SQS are not talking to each other?
Wonder if anyone has any insights that could help me.
Thank you
I found the answer. It's all to do with the naming convention of the SNS topic. Explained in the documentation:
https://docs.aws.amazon.com/textract/latest/dg/api-async-roles.html
Step3 - Create an Amazon SNS topic. Prepend the topic name with AmazonTextract. Note the topic Amazon Resource Name (ARN). Ensure that the topic is in the same Region as the AWS endpoint that you're using.
I want to build a pub/sub messaging system into my services that are hosted on Amazon Web Services, and creating SQS queues that subscribe to SNS topics seems like the obvious direction to take, but I can't get it working at all.
So far my code looks for the topics and the queues at startup and creates anything that's missing. This seems to work, I can see the SNS topic and the SQS queues in the AWS management console, and I can see that the queue is subscribed to the topic, but when I publish messages to the topic nothing ends up in the queue.
Even if I manually publish a message using the 'Publish' button in the management console the queue is still empty.
I changed the permissions on both the topic and the queue to 'everyone can do everything' just to eliminate this possibility. My code receives a message ID in response to the publish and there are no errors, every API call returns a 200 (OK) status.
Where can I go from here to figure out why it's not working?
The SNS --> SQS link has a few gotchas:
"Amazon SNS isn't currently compatible with FIFO queues." per the note on their Subscribing an Amazon SQS Queue to an Amazon SNS Topic Tutorial
You have to fiddle with the IAM permissions (see the page on Sending Amazon SNS Messages to Amazon SQS Queues)
You can't send messages to encrypted queues (see their Server-Side Encryption page)
It would definitely have been easier to figure this out if all this info were consolidated into a single page. The killer for me was #3 - perhaps one of these will be the solution to your issue.
A couple of options -
Enable CloudTrail and monitor the logs
View the CloudWatch logs to identify any permissions issues
Open a ticket with AWS support.
Ideally, you wouldn't be creating the resources in your application but instead decouple those into CloudFormation or at a minimum CLI scripts. If you require the ability to dynamically create these resources, using the AWS IoT Message Broker may be a better option since it supports ephemeral messaging resources - http://docs.aws.amazon.com/iot/latest/developerguide/iot-message-broker.html
I had a similar issue with SQS subscriptions. It turned out that if I create the subscription from the SQS editor it works, but if I create it from the SNS creation screen it accepts the message but never forwards it to the queue.
To get more detailed information about specific failures you can turn on
SNS "Delivery status logging".