I am using AWS IoT in my cloud application.
I have created IoT rules and lambda functions to process the MQTT publish messages.
When the MQTT Client publishes the message to a topic, a rule is triggered and a lambda function is executed.
While carrying out the load testing, I found that, after some time, MQTT broker doesn’t process the messages.
On the AWS IoT ( monitor ) console, I can see the number of successful connections reaching to 400.
My MQTT clients are opening around 50 connections, publishing the message and waiting for async response on another topic for 60 seconds.
After receiving the response or 60 seconds timeout, these connections are closed.
Then next set of 50 connection requests are made to AWS MQTT broker.
The client is developed using AWSIoTPythonSDK.
I don’t see any errors in the IOT console as well as in the Lambda monitoring console.
Also I don’t see any message or log, which says the request are throttled.
I have raised a request to increase the limit for number Connect Requests Per Second to 1000.
Is there any way to find out, if the MQTT broker is throttling the request or if there are any other errors?
Following url is about aws iot limits.
https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html#limits_iot
Did you turn on aws iot log(debug level or info level)? And Can you see connection logs in cloudWatch?
Related
We are using the java v2 aws iot core sdk, and when trying to test publish and send 500 messages tp one client at once, only ~320 messages arrive to the client.
is this a throttling issue ? if so how can we check if published messages got throttled ?
Device can send message to the AWS IoT rules engine either via MQTT, HTTP or basic ingest. Rules engine uses SQL query to listen on topic and selects the data when topic matching the FROM clause arrives.
Code to send MQTT message:
device.publish('topic/subtopic, payload)
Code to send message via basic ingest is:
device.publish('aws/rules/topic/subtopic, payload)
URL to send message via HTTP POST is:
https://IoT_data_endpoint/topics/url_encoded_topic_name?qos=1
MQTT involves broker (pub/sub). So message sent to topic via MQTT will reach all subscribers. Similarly message sent via HTTP will also reach to all subscribers. Where as basic ingest bypasses the broker, thus saving cost and also since broker is not involved the message will not reach any subscribers. Have I understood this correctly?
Where as basic ingest bypasses the broker, thus saving cost and also since broker is not involved the message will not reach any subscribers. Have I understood this correctly?
Correct - MQTT subscribers will not receive messages that are sent using basic ingest as the broker is bypassed.
From the documentation at https://docs.aws.amazon.com/iot/latest/developerguide/iot-basic-ingest.html
Basic Ingest optimizes data flow by removing the publish/subscribe message broker from the ingestion path
And the "Designing MQTT Topics for AWS IoT" whitepaper at https://d1.awsstatic.com/whitepapers/Designing_MQTT_Topics_for_AWS_IoT_Core.pdf
Basic Ingest enables you to send data to only cloud services through the Rules Engine.
MQTT clients cannot subscribe to the Basic Ingest reserved topics:
Your devices and rules cannot subscribe to Basic Ingest reserved topics
See https://docs.aws.amazon.com/iot/latest/developerguide/iot-basic-ingest.html
I have requested a dedicated long code. But Whenever I send SMS to this Dedicated long code with Keywords, nothing is happening. I am not receiving any automated response messages (Response messages setup in the AWS Pinpoint Console). The dedicated number is for INDIA and region is AWS MUMBAI REGION.
I have also created SNS which is not at all getting triggered for incoming SMS for this dedicated number. I will provide additional details, if wanted. Please guide me to debug this issue.
I have my own MQTT client that connect to Google IoT Core. When I send connect message I received connack, and successfully send pingreq and received pingresp, but when I send publish message my connection is broken and I have error:
mqtt: User not authorized to perform this action.
My publish topic is /devices/{device-id}/events
I'd recommend looking at "I'm not receiving telemetry data on Cloud Pub/Sub" in https://cloud.google.com/iot/docs/troubleshooting. I suspect it may be related to the "Cloud IoT Core Service Agent" section.
What's the easiest way to save/log every message published on a AWS SNS topic? I thought there might be a magic setting to automatically push them to S3 or a database, or maybe a database service supporting the HTTP destination automatically, but doesn't seem to be the case. Maybe it needs to be done via a Lambda function?
The purpose is just for basic diagnostics and debugging while setting up some SNS publishing. I don't really care about high scale or fast querying, just want to log and perform basic queries on all the activity for a few minutes at a time.
You can setup a trigger to push your SNS messages to SQS queue. Push is automatic and does not require any code.
According to the docs, SNS can publish to:
http – delivery of JSON-encoded message via HTTP POST
https – delivery of JSON-encoded message via HTTPS POST
email – delivery of message via SMTP
email-json – delivery of JSON-encoded message via SMTP
sms – delivery of message via SMS
sqs – delivery of JSON-encoded message to an Amazon SQS queue
application – delivery of JSON-encoded message to an EndpointArn for a mobile app and device.
lambda – delivery of JSON-encoded message to an AWS Lambda function.
So yes, a simple approach would be to trigger a lambda function to write to S3 or CloudWatch.