We are using the java v2 aws iot core sdk, and when trying to test publish and send 500 messages tp one client at once, only ~320 messages arrive to the client.
is this a throttling issue ? if so how can we check if published messages got throttled ?
Related
I have developed a device that is able to connect to Google IoT Core through the MQTT protocol, I can publish MQTT messages on telemetry topics (/devices/DEVICE_ID/events).
But I can't figure out how to subscribe my device to a generic topic and send messages from cloud to device via this topic.
Through the console I created the topic MyTestTopic and a subscription MyTestTopic-sub, I guess that somehow I have to subscribe my device to the subscription MyTestTopic-sub, but I don't know to which topic I have to subscribe my device.
I tried to subscribe my device to the following topics:
projects/PROJECT_ID/topics/MyTestTopic
projects/PROJECT_ID/topics/MyTestTopic-sub
/devices/DEVICE_ID/events/MyTestTopic
/devices/DEVICE_ID/events/MyTestTopic-sub
the subscription to these topics seems to be successful, but the device does not receive the messages I send from the console.
Any suggestions please?
I found this thread (Google Cloud IoT - Invalid MQTT publish topic) from 4 years ago, from which I understand that it is possible to subscribe only to some predefined topics, also from the various documents I have read I understand that Google IoT Core allows devices to subscribe to only two topics: /devices/{device-id}/config and /devices/{device-id}/commands/#
Has anything changed since then?
So to be clear, there are two different "topics" in play here. There's the MQTT topics that are described in the docs you found, config and commands. Those are subscribed to on the device using an MQTT client (e.g. Paho MQTT), and sent down to the device from somewhere else via the IoT Core Admin APIs. Then there's Pub/Sub topics. Out of the box, a device using IoT Core does not directly interact with Pub/Sub at all. They publish/receive via the MQTT topics only. So if you register a device with IoT Core, all messages come and go via the MQTT bridge (or HTTPS).
What Gambit support linked to in that github repo is a direct subscription to a Pub/Sub topic (not MQTT). If you look at the Python code, it's establishing proper Google Cloud Credentials via a json token (service account bearer token) that needs to be present on the device or a proxy device. Depending on the device you're using, that may not be possible of course (implementing GCP APIs on a microcontroller is difficult at best).
The easiest way I've seen this done is to implement a Cloud Function that subscribes to a Pub/Sub topic you want to use, and it implements the proper IoT Core Admin calls to push the Pub/Sub messages down to the device as needed. But of course, it all depends on what you're trying to do.
I am using AWS IoT in my cloud application.
I have created IoT rules and lambda functions to process the MQTT publish messages.
When the MQTT Client publishes the message to a topic, a rule is triggered and a lambda function is executed.
While carrying out the load testing, I found that, after some time, MQTT broker doesn’t process the messages.
On the AWS IoT ( monitor ) console, I can see the number of successful connections reaching to 400.
My MQTT clients are opening around 50 connections, publishing the message and waiting for async response on another topic for 60 seconds.
After receiving the response or 60 seconds timeout, these connections are closed.
Then next set of 50 connection requests are made to AWS MQTT broker.
The client is developed using AWSIoTPythonSDK.
I don’t see any errors in the IOT console as well as in the Lambda monitoring console.
Also I don’t see any message or log, which says the request are throttled.
I have raised a request to increase the limit for number Connect Requests Per Second to 1000.
Is there any way to find out, if the MQTT broker is throttling the request or if there are any other errors?
Following url is about aws iot limits.
https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html#limits_iot
Did you turn on aws iot log(debug level or info level)? And Can you see connection logs in cloudWatch?
I'm using AWS IoT to get device logs from multiple moving devices. Currently, all devices can publish their status to the AWS IoT Core using MQTT. Now I'd like to create an application service from outside AWS (servers are in my company) that will subscribe and consume device status from AWS IoT. I'm facing the issue like that, the app backend can get data from AWS IoT. However, it cannot be distributed or load balanced. Just the last connected consumer can get the data. How can I distribute/scale the reads from AWS IoT or any alternative/suggested solutions?
Thanks in advance!
I have developed a service that receives messages from AWS/Kinesis and process them. I'd like to have a watchdog service that will catch processing failures. Before the service starts message processing it sends a message to watchdog and after it finishes processing the message it notifies Watchdog that processing completed successfully. If there is no 'success' message after some timeout period Watchdog should fire some event.
The question is: which AWS service, if any, can do this job for me?
You can use SNS, making the service publish a notification to the subscriber (watchdog) for each event that you want to be notified about.
Amazon SNS is a fully managed pub/sub messaging service that makes it easy to decouple and scale microservices, distributed systems, and serverless applications. With SNS, you can use topics to decouple message publishers from subscribers, fan-out messages to multiple recipients at once, and eliminate polling in your applications. SNS supports a variety of subscription types, allowing you to push messages directly to Amazon Simple Queue Service (SQS) queues, AWS Lambda functions, and HTTP endpoints. AWS services, such as Amazon EC2, Amazon S3 and Amazon CloudWatch, can publish messages to your SNS topics to trigger event-driven computing and workflows. SNS works with SQS to provide a powerful messaging solution for building cloud applications that are fault tolerant and easy to scale.
https://aws.amazon.com/sns/
I have my own MQTT client that connect to Google IoT Core. When I send connect message I received connack, and successfully send pingreq and received pingresp, but when I send publish message my connection is broken and I have error:
mqtt: User not authorized to perform this action.
My publish topic is /devices/{device-id}/events
I'd recommend looking at "I'm not receiving telemetry data on Cloud Pub/Sub" in https://cloud.google.com/iot/docs/troubleshooting. I suspect it may be related to the "Cloud IoT Core Service Agent" section.