I have an MQTT publisher and subscriber written in Node JS.
I was wondering if it is possible to have it in the form of an API, that we can connect to and publish messages to using PostMan.
Here is my code for the publisher:
Publisher.js:
const mqtt = require('mqtt');
let client = mqtt.connect('mqtt://broker.hivemq.com');
client.on('connect', () =>{
console.log(`MQTT Client Connected Successfully!`);
client.publish('connected', `${true}`);
});
Here is my code for the Subscriber:
Subscriber.js:
const mqtt = require('mqtt');
let client = mqtt.connect('mqtt://broker.hivemq.com');
let connected = false;
client.on('connect', () =>{
console.log(`MQTT Client Connected Successfully!`);
client.subscribe('connected');
});
client.on('message', (topic, message) =>{
if(topic === "connected"){
return handleGarageConnected(message);
}
console.log("No Handler for Topic %s", topic);
});
I want to be able to communicate with the Publisher / Subscriber over the internet using an API that I create.
Thank you.
MQTT is a asynchronous protocol, HTTP is synchronous.
Combining them is possible but it requires a good understanding of both.
E.g. Messages for a subscription can arrive at any time, so suppose you map that to a GET HTTP request, what should it return?
The last message
Block until the next message (with suitable timeout)
All the previous messages
You can make a function which publishes on the desired topic in publisher.js and export it. This way you can use in any other module.
Publisher.js
const mqtt = require('mqtt');
let client = mqtt.connect('mqtt://broker.hivemq.com');
client.on('connect', () =>{
console.log(`MQTT Client Connected Successfully!`);
client.publish('connected', `${true}`);
});
module.exports = {
publishTopic : function(payload){
client.publish('randomTopic');
}
}
You can make another file for your APIS, where you can import the exported function like so,
const mqttWrapper = require('./Publisher.js');
module.exports = {
randomApi : function (req, res) {
mqttWrapper.publishTopic("message");
}
}
Related
I'm using GCP PubSub (push subscription to be precisely), and Cloud Run to execute subscribed messages.
Recently I've noticed that Cloud Run executes a same message for five times.
I see there is a retry policy for a subscription which its message has been unhandled (or mishandled), but Cloud Run clearly is giving 200 OK response.
So it seems that sending 200 OK response is not enough, but I cannot find a way to send ack sign properly.
Here's the code to publish message.
function publish(message: string) {
const pubsub = new PubSub({ projectId: 'my-project' });
const dataBuffer = Buffer.from(message);
const topicName = 'my-topic';
pubsub
.topic(topicName)
.publish(dataBuffer)
.then((messageId) => {
console.log(`Message ${messageId} published`);
})
.catch((err) => {
console.log(err);
});
}
publish(JSON.stringify({ foo : 'bar' }));
And here's the code of Cloud Run.
// express app
app.post('/run', async (req, res) => {
try {
const body = req.body.message ? Buffer.from(req.body.message.data, 'base64').toString() : req.body;
// req.body came out to be "{"foo":"bar"}"
const deliveredMessage = JSON.parse(body);
// do something
// Do I have to do something like message.ack() here?
return res.status(200).end();
} catch (e) {
return res.status(502).end();
}
});
These are example responses of Cloud Run retrying.
Here's a graph of unacked messages at that time.
Is your cloud run app running when you did the post?
Because there is a timeout for the messages waiting in the queue, maybe pub/sub is resending the messages before you start to consuming it.
Another thing you can try is putting the response at the first line, like:
app.post('/run', async (req, res) => {
try {
res.status(200)
...
i would like to post new messages to a chat channel directly from my back-end. I can see how to do it on feeds, but can't find anything similar for chat.
What's the most straight forward way to do it?
You can send messages from your backend in Stream Chat using something such as this
const searchAndSendMessage = async (channelID, userID, message) => {
const filterID = { id: channelID };
const channel = await client.queryChannels(filterID, {});
channel[0].sendMessage({user: { id: userID }, text: message})
return channel[0];
};
Please note that when sending a message from the backend using server-side auth, then you'll need to include either message.user or message.user_id else you will throw an error.
I am using AWS Websocket api-gateway. I am sending message to connected client by aws:execute-api:region:account-id:api-id/stage-name/POST/#connections.However, the message is getting delayed somehow and the UI won't be able to receive the chat message immediately. I don't why any suggestion where getting something wrong????
One thing very exceptional, when I am sending message, last message getting time.
Example : If I sent one message that take much time. If I send 1st message, as I sent second message, First message will received to client immediately but now 2nd second message take much time. SO basically when I am sending nth message, then every nth message will deliver to client when (n+1)th message sent and (n+1)th messages takes much time to deliver.
Your help much appreciated!!!
This is only Code Snippet which I have
import * as AWS from 'aws-sdk';
export class AWSWebsocketGateway {
websocketInstance;
constructor() {
AWS.config.update({
accessKeyId: <accessKeyId>,
secretAccessKey: <secretAccessKey>,
region: <region>
});
this.websocketInstance = new AWS.ApiGatewayManagementApi({
endpoint: <webSocketDomainName> + <webSocketStage>
});
}
sendMessage(connection, messageObject, callback: Function) {
messageObject = JSON.stringify(messageObject);
this.websocketInstance.postToConnection({ ConnectionId: connection, Data: JSON.stringify(messageObject) }, function (error, data) {
console.log('Error', error, 'Send Message...', data);
return callback(error, data);
});
}
}
I retrieving connection Id from DB in one query and calling this function in loop to send message.
For anyone else who get's bitten by this one ->
You need to wait for the API call to return it's result before finishing the lambda's execution.
E.g. this won't work
module.exports.ping = async (event) => {
... //setting up api gateway, etc
apig.postToConnection({ ConnectionId: connectionId, Data: JSON.stringify(message) }, (err, res) => {})
return {
statusCode: 200
}
}
But this will:
module.exports.ping = async (event) => {
... //setting up api gateway, etc
await apig.postToConnection({ ConnectionId: connectionId, Data: JSON.stringify(message) }).promise();
return {
statusCode: 200
}
}
Any async code that is executing after the return of the lambda appears to be paused til that handler is run again, which is why you see the delayed n+ issue you outlined above.
I am using the AWS SDK SQS (with Nodejs) behind a router, to get through the router I need to include a custom header in the REQUEST.
I've seen this documentation (https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Request.html#build-event) that talks about the .on ('build') event in REQUEST:
var req = s3.putObject(params);
req.on('build', function() {
req.httpRequest.headers['Custom-Header'] = 'value';
});
req.send(function(err, data) { ... });
However, using the SQS service is not working, no error has been thrown and the custom header is not included in the REQUEST.
Is it possible include a custom header using AWS SDK with SQS service ?
How make to make this work?
After help, I was able to send messages to the queue with a custom header, below the sample code:
var sqs = new aws.SQS({http_wire_trace: true});
var params = {
MessageBody: 'Hello world!',
QueueUrl: queueUrl,
DelaySeconds: 0
};
var req = sqs.sendMessage(params);
req.on('build', () => {
req.httpRequest.headers['Custom-Header'] = 'bar';
});
req.on('success', (resp) => {
console.log(resp.request.httpRequest.headers)
});
req.send();
I have been searching all over the web and nothing gives a clear answer to confirm the subscription request from amazon SNS. I already send the subscription from the amazon console to my website, but what's next? I am using amazon EC2 as my server with PHP.
Before you even configure the HTTP/HTTPS endpoint subscription through AWS management console, you need to make sure that the HTTP or HTTPS endpoint of your PHP web site has the capability to handle the HTTP POST requests that Amazon SNS generates. There are several types of SNS messages: SubscriptionConfirmation, Notification and UnsubscribeConfirmation. Your PHP code needs to get the header x-amz-sns-message-type from request and process it based on the message type. For SubscriptionConfirmation message, your PHP application needs to process the POST message body, which is a JSON document. In order to subscribe the topic, your PHP code needs to visit the "SubscriberURL" specified in the JSON body. Optionally, you should verify the signature to make sure the authenticity of message before subscribing the topic.
You can find more details on AWS documentation: http://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp.html
Here is an express application (Node.js) which confirms the SNS subscription:
const express = require('express')
const request = require('request')
// parse urlencoded request bodies into req.body
const bodyParser = require('body-parser')
const app = express()
const port = 8080
app.use(bodyParser.urlencoded({ extended: false }))
app.use(bodyParser.json())
app.post('/', (req, res) => {
let body = ''
req.on('data', (chunk) => {
body += chunk.toString()
})
req.on('end', () => {
let payload = JSON.parse(body)
if (payload.Type === 'SubscriptionConfirmation') {
const promise = new Promise((resolve, reject) => {
const url = payload.SubscribeURL
request(url, (error, response) => {
if (!error && response.statusCode == 200) {
console.log('Yess! We have accepted the confirmation from AWS')
return resolve()
} else {
return reject()
}
})
})
promise.then(() => {
res.end("ok")
})
}
})
})
app.listen(port, () => console.log('Example app listening on port ' + port + '!'))
To use it one needs to install required packages:
yarn add express request body-parser
Once you confirm the subscription AWS will send a POST request to the server with the following content:
{
"Type": "SubscriptionConfirmation",
"MessageId": "XXXXXXXX-1ee3-4de3-9c69-XXXXXXXXXXXX",
"Token": "SECRET_TOKEN",
"TopicArn": "arn:aws:sns:us-west-2:XXXXXXXXXXXX:ses-test",
"Message": "You have chosen to subscribe to the topic arn:aws:sns:us-west-2:XXXXXXXXXXXX:ses-test. To confirm the subscription, visit the SubscribeURL included in this message.",
"SubscribeURL": "https://sns.us-west-2.amazonaws.com/?Action=ConfirmSubscription&TopicArn=arn:aws:sns:us-west-2:XXXXXXXXXXXX:ses-test&Token=SECRET_TOKEN",
"Timestamp": "2018-11-21T19:48:08.170Z",
"SignatureVersion": "1",
"Signature": "SECRET",
"SigningCertURL": "https://sns.us-west-2.amazonaws.com/SimpleNotificationService-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem"
}
The payload contains SubscribeURL which is requested by the server.
The end point you have specified will get data from AWS SNS endpoint verification service, The same end point will be used to verify the end point and to get notifications from aws,
Simply dump the input sent by AWS SNS into one text file like,
$json_write_to_text = json_decode(file_get_contents("php://input"));
You will find all data sent by AWS SNS, but just find SubscriptionUrl (which will be specific for endpoint having valid token), Open this in browser you will have SubscriptionConfirmation status. That's it
Enjoy.
Spring cloud SNS subscription with annotation
spring cloud AWS has support for auto confirmation of subscriber, you just need to put this annotation "#NotificationSubscriptionMapping"
#Controller
#RequestMapping("/topicName")
public class NotificationTestController {
#NotificationSubscriptionMapping
public void handleSubscriptionMessage(NotificationStatus status) throws IOException {
//We subscribe to start receive the message
status.confirmSubscription();
}
#NotificationMessageMapping
public void handleNotificationMessage(#NotificationSubject String subject, #NotificationMessage String message) {
// ...
}
#NotificationUnsubscribeConfirmationMapping
public void handleUnsubscribeMessage(NotificationStatus status) {
//e.g. the client has been unsubscribed and we want to "re-subscribe"
status.confirmSubscription();
}
}
http://cloud.spring.io/spring-cloud-aws/spring-cloud-aws.html#_sns_support
I solved this using NodeJS backend. Lets say you have an API like this in HapiJS (Well it doesnt matter you can have another tech)
{
method: 'POST',
path: '/hello',
handler: ( request, reply ) => {
reply( Hello.print(request.payload) );
},
config: {
tags: ['api']
}
}
Just pass the payload you receive, on to your business logic.
In the business logic process it like this
'use strict';
const request = require('request');
exports.print = (payload) => {
payload = JSON.parse(payload);
if(payload.Type === 'SubscriptionConfirmation'){
return new Promise((resolve, reject) => {
const url = payload.SubscribeURL;
request(url, (error, response) => {
if (!error && response.statusCode == 200) {
console.log('Yess! We have accepted the confirmation from AWS');
return resolve();
}
else
return reject();
});
});
}
I am using request module from NPM to automatically accept such requests.
Another way would be to print the contents of payload and then click on the URL given in payload.SubscribeURL.
Once AWS accepts it you check the confirmation on the Subscriptions page where Subscription ARN would be changed from Pending Confirmation to a complex name-cum-SHA having your Topic name.