I am having a first look at Amazon Kinesis to compare it to Apache Kafka.
Can Kinesis Producers/Consumers access the Kinesis broker from outside AWS? Or have the (Java) clients to be deployed on a EC2 instance?
Yes, you can create and interact with your streams using the API and the AWS SDKs.
Related
A user has created photo editing software and hosted it on EC2. The software accepts requests from the user about the photo format and resolution and sends a message to S3 to enhance the picture accordingly. Which of the below mentioned AWS services will help make a scalable software with the AWS infrastructure in this scenario?
A. AWS Simple Queue Service
B. AWS Simple Notification Service
C. AWS Glacier
D. AWS Elastic Transcoder
I think it should be D AWS Elastic Transcoder. What do u think?
Amazon Simple Queue Service (SQS. is a fast, reliable, scalable, and fully managed message queuing service. SQS provides a simple and cost-effective way to decouple the components of an application. The user can configure SQS, which will decouple the call between the EC2 application and S3. Thus, the application does not keep waiting for S3 to provide the data.
Glacier is for cold storage, Transcoder is sorely for video, SNS is
for notification, hence only SQS could help in processing fragments by
fragments information for the application. So the answer is A (AWS SQS)
You should use the AWS Elastic Transcoder with AWS Lambda + SQS. So whenever the server has any request from a user it will transfer the payload to SQS and SQS will trigger the Lambda. Now Lambda will do all business logic with AWS Elastic Transcoder. In this way, it can handle lots of traffic using SQL and Lambda. You can manage the Lambda thread and SQS queue pool.
I have an external service that sends me data via MQTT. I need to receive this data on AWS then process it and then write it to an RDS instance. What services are recommended for this purpose? An Amazon MQ and a Lambda? A SQS and a Lambda? Any other?
Thanks for any help.
AWS IoT 'things' have a Message Broker that supports MQTT.
Traditionally, IoT and MQTT involves huge numbers of messages, which are normally stored in DynamoDB. However, you could write an AWS Lambda function to store the received messages in an Amazon RDS database.
What could be the backend storage for Amazon CloudWatch Log Groups.
Is it an Amazon S3 backend?
Or is it a physical managed service that AWS manages?
AWS has not exposed what backend the CloudWatch uses.
But if you are interested in doing it yourself.
You can create a StatsD backend for your applications.
https://github.com/camitz/aws-cloudwatch-statsd-backend
I have use case like once data received in AWS IoT Rules Engine from things, I want to call API instead of dynamoDb, kinesis and lambda. I read FAQ https://aws.amazon.com/iot/faqs/ and doc but now found anything related to routing data to API on EC2 instance. This developer guide http://docs.aws.amazon.com/iot/latest/developerguide/iot-create-rule.html shows how to route data to dynamoDb, S2, Kinesis, SNS and machine learning. Is it possible to route data to EC2 instance from IoT Rules Engine?
You can create a iot rule to connect to lambda and in lambda function you can call your EC2 API.
Is there an AMAZON-SNS mock?
That is, some non-hosted version that I can use for testing or for offline cases? Preferably with the same API.
Like elasticMQ is for SQS?
Thank you
LocalStack - A fully functional local AWS cloud stack
LocalStack provides an easy-to-use test/mocking framework for
developing Cloud applications.
Currently, the focus is primarily on supporting the AWS cloud stack.
LocalStack spins up the following core Cloud APIs on your local
machine:
API Gateway at http://localhost:4567
Kinesis at http://localhost:4568
DynamoDB at http://localhost:4569
DynamoDB Streams at http://localhost:4570
Elasticsearch at http://localhost:4571
S3 at http://localhost:4572
Firehose at http://localhost:4573
Lambda at http://localhost:4574
SNS at http://localhost:4575
SQS at http://localhost:4576
Redshift at http://localhost:4577
If you really need a service you can try https://github.com/s12v/sns
Another option would be mocking. SNS is a HTTP service, so you can mock responses in your application.