I have a Streamsets pipeline that sends events to AWS SNS currently (Using HTTP client). I now have a requirement to send these events to AWS MSK instead of AWS SNS.
I am not finding a relevant documentation to start with this. Don't know where and how to start.
Can someone help in pointing to a relevant documentation which will be much helpful in starting this change?
Any help will be much appreciated. Thanks much in advance.
MSK is just a managed service around Apache Kafka. In StreamSets there is a Kafka connector.
MSK can use certificates (mTLS) for authentication, and this is the method that might work for StreamSets.
Related
I need help in how to set up AWS cross account messaging using EventBridge? Like a step by step guide.
I looked online and here is the article I looked here at: Sending and Receiving Events Between AWS Accounts - Amazon CloudWatch Events but I am not sure if this article is the right article for the step by step.
Also, is sending and receiving events the same thing as sending and receiving messages in EventBridge? I'd really appreciate your help.
We have an Elastic Beanstalk deployment and wanted to receive events into our Slack channel. We setup EventBridge and Chatbot to deliver notifications. Everything works, except the notifications are useless, because Chatbot does not map the message detail only the detail type, which does not tell much. See the attached image of a notification (cropped to cut out the region/account info), the message is not too useful without telling us what was the actual change, which is in the event received by EventBridge and sent to SNS, but not mapped by Chatbot when sending to Slack.
Even their example test notification demonstrates the same problem:
I checked the Chatbot documentation where they claim support for multiple services via EventBridge. I checked EventBridge documentation where they list Elastic Beanstalk among supported services.
I also subscribed and tested Health service notifications, which work just fine. See the example test notification below.
I could not find a way to configure Chatbot to do the right thing. I could write my own message transformer for certain events and call a Slack webhook instead of using Chatbot, but I wanted to avoid writing custom code here for now.
Does anyone know if it is possible to fix Beanstalk notifications that are sent from EventBridge to Chatbot then to Slack? Or is AWS working on supporting more service notifications and in the future this might start working as it should?
Thank you in advance.
We have a provider that wants to relay information information through RabbitMQ.
Our system is completely on AWS and we runs on managed services. Example API Gateway, Lambda and DynamoDB, we would like to have a listener (consumer) of messages on the broker and we write into our DynamoDB.
Is this possible to do with some managed AWS Service? I have googled a lot on this theme but only find things on installing a message broker within AWS.
It would be great is someone could point me to the correct direction.
I am having trouble in defining a webhook on AWS CloudWatch(I am new to this technology). I checked out a lot of forums but I could not find out the right answer. Basically, what I need is a webhook or a similar way for sending an alert from CloudWatch to a web server via http post, every time the alert is generated.
Any help would be greatly appreciated!
Configure CloudWatch to send the alert to an SNS topic. Then have SNS send the message to an HTTP endpoint.
I need to load a SQS queue on the Amazon Web Services to load test an application which consumes the messages from the SQS queue.
Is it possible to load the SQS queue using JMeter?
Are there any better options available to load the SQS queue than using JMeter?
I haven't tried it, but you should be able to send the messages through HTTP requests.
I created JMeter plugin that use AWS SDK to publish events in SNS Standard or FIFO Topic:
https://github.com/JoseLuisSR/awsmeter
You just need AWS IAM user with programmatic access, download JMeter and install awsmeter plugin.
If you have questions or comments let me know.
Thanks.