I am a newbie to AWS. As I understand, both AWS MQ and AWS SQS are Message Queue tools. The only noted difference is that SQS is fully managed.
When should we use SQS or MQ?
SQS is a simple queueing service. It doesn't support many higher level abstractions like message routing, fanouts, distribution lists etc. It is a queue - a message is produced, and a message is delivered. It is useful when you need a Queue with limited backing logic.
AWS MQ is a managed Apache ActiveMQ(or RabbitMQ) broker service.
This provides you a fully managed Apache ActiveMQ system in the cloud, with support for a variety of industry-standard queue and broadcast protocols like AMQP, JMS etc. It is useful when you have complicated delivery rules - or when you're migrating an existing system from outside AWS into AWS, and your systems happen to talk to one another with a standard queueing protocol.
From AWS Documentation, you can search "Q: When should I use Amazon MQ vs. Amazon SQS and SNS?":
Amazon MQ, Amazon SQS, and Amazon SNS are messaging services that are
suitable for anyone from startups to enterprises. If you're using
messaging with existing applications, and want to move your messaging
to the cloud quickly and easily, we recommend you consider Amazon MQ.
It supports industry-standard APIs and protocols so you can switch
from any standards-based message broker to Amazon MQ without rewriting
the messaging code in your applications. If you are building brand new
applications in the cloud, we recommend you consider Amazon SQS and
Amazon SNS. Amazon SQS and SNS are lightweight, fully managed message
queue and topic services that scale almost infinitely and provide
simple, easy-to-use APIs. You can use Amazon SQS and SNS to decouple
and scale microservices, distributed systems, and serverless
applications, and improve reliability.
Also, in this doc, you can check comparisons with other services.
Related
Amazon EventBridge enables developers to connect 3rd party event-driven applications with Amazon services. Amazon AppFlow offers event-driven integration with 3rd party apps as well.
What is the difference between the two services for the event-driven scenarios and when to use one over the other?
The AppFlow FAQ contains this info:
Amazon EventBridge enables developers to build event driven
applications that interact with SaaS applications and AWS services.
SaaS applications that have integrated with EventBridge emit events to
the customer’s event bus, which can then be routed to targets such as
Amazon EC2 instances or Lambda functions for processing. AppFlow
supports bi-directional transfer of data between SaaS applications and
AWS services that may be initiated by humans using a UI, a schedule,
or events - all with a point and click interface.
AppFlow is great for anyone who wants to connect together their applications without writing any code. EventBridge works with Step Functions and Lambda (and various other AWS services), and so is well suited for developers who need the extra flexibility that those services provide. Right now EventBridge is only one way, whereas AppFlow allows you to send data back to SaaS apps. The list of supported partners for each service is also different.
For some of AppFlow's partners, data is received via API polling (from the FAQ: "AppFlow is a fully managed API integration service that replaces custom connectors"), which is slightly different from EventBridge which receives data in an event-driven manner, where an event is sent via an HTTP call from the partner as soon as a change occurs. The pricing of the two services also differs: EventBridge charges $1/million events, and AppFlow charges $0.001 per flow (with an additional charge per GB of data processed).
For the purpose of sending push notifications from the backend, if we need a Push Notification Platform, could you please suggest which of these is intended for that purpose – Amazon SNS or Pinpoint?
Both Amazon SNS and Amazon Pinpoint, supports sending push notification to various devices (e.g Android, iOS etc)
The major difference between Amazon SNS & Amazon Pinpoint is that :
with Amazon SNS you have to set up your application to manage each message's audience, content, and delivery schedule. On the other hand, with Amazon Pinpoint you do not have to code these features, most of them are already built in. With Amazon Pinpoint, you can collect data about your app usage, create highly-targeted segments and send full campaigns(either immediate or scheduled) plus many more features.
I have developed a service that receives messages from AWS/Kinesis and process them. I'd like to have a watchdog service that will catch processing failures. Before the service starts message processing it sends a message to watchdog and after it finishes processing the message it notifies Watchdog that processing completed successfully. If there is no 'success' message after some timeout period Watchdog should fire some event.
The question is: which AWS service, if any, can do this job for me?
You can use SNS, making the service publish a notification to the subscriber (watchdog) for each event that you want to be notified about.
Amazon SNS is a fully managed pub/sub messaging service that makes it easy to decouple and scale microservices, distributed systems, and serverless applications. With SNS, you can use topics to decouple message publishers from subscribers, fan-out messages to multiple recipients at once, and eliminate polling in your applications. SNS supports a variety of subscription types, allowing you to push messages directly to Amazon Simple Queue Service (SQS) queues, AWS Lambda functions, and HTTP endpoints. AWS services, such as Amazon EC2, Amazon S3 and Amazon CloudWatch, can publish messages to your SNS topics to trigger event-driven computing and workflows. SNS works with SQS to provide a powerful messaging solution for building cloud applications that are fault tolerant and easy to scale.
https://aws.amazon.com/sns/
I am looking for a PubSub or Message Queue cloud service with the following requirements:
Pay per message
The ability to subscribe on-demand with a client sdk
Support for topics
Amazon MQ does not satisfy these requirements because it is not pay per message.
Amazon SNS does not satisfy these requirements because it doesn't allow on-demand subscription using a client sdk.
Amazon SQS does not satisfy these requirements because it doesn't support topics.
AWS IoT Core does satisfy these requirements.
However, I'm curious if there are other options - possibly even outside the AWS ecosystem. I'm hesitant to use AWS IoT since I wouldn't be using it for an IoT use case.
Should I move forward with AWS IoT Core or are there other options I should be considering?
Do AWS SNS and SQS support STOMP protocol?
I have tried googling it. I went through several forums but I am not sure yet.
No. Amazon SNS and Amazon SQS have their own API.
STOMP would need to support those services (rather than vice versa), but it appears that STOMP does not support SNS nor SQS.
Actually, AWS have another message queue service for Apache ActiveMQ that does support JMS, NMS, AMQP, STOMP, MQTT, WebSocket and called Amazon MQ