When I create a new queue and subscribe it to a topic in Java, no message comes. The same via the AWS web console works fine.
I guess I have to confirm the subscription somehow, but the sns.confirmSubscription method needs a token - where shall I get it?
This is my Java code:
String queueURL = sqs.createQueue("my-queue").getQueueUrl();
sns.subscribe(myTopicARN, "sqs", queueURL);
sns.publish(myTopicARN, "{\"payload\":\"test\"}");
sqs.receiveMessage(queueURL).getMessages()
.forEach(System.out::println); // nothing
What am I doing wrong?
Check this out: https://aws.amazon.com/blogs/developer/subscribing-queues-to-topics/
You should subscribe like this:
Topics.subscribeQueue(sns, sqs, myTopicARN, queueURL);
This convinient method creates a policy for the subscription to allow the topic to send messages to the queue.
subscribing the queue to sns does not automatically create a policy to allow sns to send messages to the queue (based on my experience with sns/sqs) so you need to create the policy yourself and give permission to sns to send messages to your queue this is an example on how to do it using queue url , queue arn and topic arn
import static com.amazonaws.auth.policy.Principal.All;
import static com.amazonaws.auth.policy.Statement.Effect.Allow;
import static com.amazonaws.auth.policy.actions.SQSActions.SendMessage;
import static com.amazonaws.auth.policy.conditions.ArnCondition.ArnComparisonType.ArnEquals;
final Statement mainQueueStatements = new Statement(Allow) //imported above
.withActions(SendMessage) //imported above
.withPrincipals(All) //imported above
.withResources(new Resource(queueArn)) // your queue arn
.withConditions(
new Condition()
.withType(ArnEquals.name()) //imported above
.withConditionKey(SOURCE_ARN_CONDITION_KEY) //imported above
.withValues(topicArn) // your topic arn
);
final Policy mainQueuePolicy = ()
.withId("MainQueuePolicy")
.withStatements(mainQueueStatements);
final HashMap<QueueAttributeName, String> attributes = new HashMap<>();
attributes.put(QueueAttributeName.Policy.toString(), mainQueuePolicy.toJson());
amazonSQS.setQueueAttributes(new SetQueueAttributesRequest().withAttributes(attributes).withQueueUrl(queueUrl)); // your queue url
Related
In AWS IOT, I'm able to publish an MQTT message from a device to the topic "topic/messages" :
{
"id":"messageID",
"value":"messageValue"
}
And I want to "augment" it in the server by adding a timestamp on it and let the message to continue on the SAME TOPIC.
So I'd like that subscribers on "topic/messages" receive this:
{
"id":"messageID",
"value":"messageValue",
"serverTimestamp":"1637867431920" <--- Here
}
However, I don't find the way how to process this message and let it flow in the same topic:
I can add a rule
SELECT * , timestamp() as serverTimestamp FROM 'topic/#'
But the rule does not augment the original message but creates an augmented copy and redirects it to some other service (Lambda, DynamoDB, Republish, etc..)
Those services work with the copy of the given value, but not with the original message, so the subscribers still receives the original sent message.
{
"id":"messageID",
"value":"messageValue"
}
I can republish the message using the same topic BUT as the topic has an attached rule, after the republishing, the rule's action is triggered again and again in a recursive loop)...
All the AWS examples I've read are meant to take the message, transform it , and do some other different thing with it (save in DynamoDB, save in a Bucket, end to Salesforce....) but none of them modify the message to be sent.
So what I'm looking for is a way to receive the message, add a field (or more) to it and let it flow in the same topic .
What is the simplest way to do this?
I have this simple python function where i am just taking the input from pubsub topic and then print it.
import base64,json
def hello_pubsub(event, context):
"""Triggered from a message on a Cloud Pub/Sub topic.
Args:
event (dict): Event payload.
context (google.cloud.functions.Context): Metadata for the event.
"""
pubsub_message = base64.b64decode(event['data'])
data = json.loads(pubsub_message)
for i in data:
for k,v in i.items():
print(k,v)
If i had used the pubsub_v1 library, there i could do following.
subscriber = pubsub_v1.SubscriberClient()
def callback(message):
message.ack()
subscriber.subscribe(subscription_path, callback=callback)
How do i ack the message in pubsub triggered function?
Following your latest message, I understood the (common) mistake. With Pubsub, you have
a topic, and the publishers can publish messages in it
(push or pull) subscriptions. All the messages published in the topic are duplicated in each subscription. The message queue belong to each subscription.
Now, if you look closely to your subscriptions on your topic, you will have at least 2.
The pull subscription that you have created.
A push subscription created automatically when you deployed your Cloud Function on the topic.
The messages of the push subscription is correctly processed and acknowledge. However those of pull subscription aren't, because the Cloud Function don't consume and acknowledge them; the subscription are independent.
So, your Cloud Function code is correct!
The message should be ack'd automatically if the function terminates normally without an error.
I want to be able to send a text message to some number and then (upon receiving the text) basically just send a post request off to a different service after receiving the text. Does anyone know of a service I could use to set this up? Would like for it to be as fast as possible
Here's a summary of the steps to setup a sample app:
Navigate to Amazon SNS Service → Topics
Enter Name and create a new Topic
For the newly created topic, create a subscription where the Protocol is AWS Lambda (see image1 below)
Navigate to Amazon Pinpoint Service, create new Pinpoint application
Enable SMS & voice feature for this Pinpoint application
Get a new Long Code (long code price is $1/month)
For the long code, Enable two-way SMS, select the Choose an existing SNS topic option and select the SNS topic created in Step 2 above (see image2 below)
Finally, now you can send a message to that phone number from your phone and it will trigger your lambda function. In your lambda function, you can send a POST request to a different service or do whatever else. You can also respond back to the user's message - see example below.
Here's an example of how to send a message using Amazon Pinpoint in Java:
public void sendSMS(String pinpointPhoneNumber, String userPhoneNumber, String messageContent) {
// define who the message is going to and via what platform
Map<String, AddressConfiguration> addressMap = new HashMap<>();
addressMap.put(userPhoneNumber, new AddressConfiguration().withChannelType(ChannelType.SMS));
SMSMessage smsMessage = new SMSMessage();
smsMessage.setOriginationNumber(pinpointPhoneNumber);
smsMessage.setMessageType(MessageType.TRANSACTIONAL);
smsMessage.setBody(messageContent);
// add sms message to the direct message config
// this can have many other types of messages
DirectMessageConfiguration directMessageConfiguration = new DirectMessageConfiguration()
.withSMSMessage(smsMessage);
// put the phone numbers and all messages in here
MessageRequest messageRequest = new MessageRequest()
.withAddresses(addressMap)
.withMessageConfiguration(directMessageConfiguration);
// create send request
SendMessagesRequest sendMessagesRequest = new SendMessagesRequest()
.withApplicationId("put-pinpoint-app-id-here")
.withMessageRequest(messageRequest);
// send the message
AmazonPinpoint pinpointClient = AmazonPinpointClientBuilder.standard().build();
SendMessagesResult sendMessagesResult = pinpointClient.sendMessages(sendMessagesRequest);
MessageResponse messageResponse = sendMessagesResult.getMessageResponse();
}
I know how to do this in java, but I just can't figure it out in Go at all.
All I want to do, is have a way to detect that an item got created in an S3 bucket, then have that trigger an SNS topic, which then notifies me of the file location in S3.
Has anybody got a working example of how I can do the go side of this for subscribing to the SNS topic or the SNS queue if I need one? Because all I seem to be able to find is Java and Node. I can find publish examples for go, but they are of little use to my use case.
To use SNS you will need a simple HTTP/HTTPS endpoint to receive SNS notifications. Which is divided into two parts (Confirm the subscription and Processing messages from HTTP/HTTPS endpoint)
1. Confirm the subscription
Do something as simple as this:
func confirmSubscription(subcribeURL string) {
response, err := http.Get(subcribeURL)
if err != nil {
fmt.Printf("Unbale to confirm subscriptions")
} else {
fmt.Printf("Subscription Confirmed sucessfully. %d", response.StatusCode)
}
}
2. Processing messages from HTTP/HTTPS endpoint
Parse the request's body, the documentations mentions how the body should be structured.
Sources:
https://docs.aws.amazon.com/sns/latest/dg/sns-http-https-endpoint-as-subscriber.html
https://github.com/viveksyngh/aws-sns-subscriber/blob/master/subscriber/subscriber.go
I am publishing a String message as message payload using SNS notification from Raspberry Pi using Python program and I want to pass that message payload to a Lambda function.
I have configured the requirement in the SNS console on AWS i.e., I have created a topic and added the lambda function to its subscribers.
Now, I want to get that message payload in the lambda function. But I can't find any method that can help me do that. For example, something like getMessage or something similar to that.
So my questions are: Since I have configured the publishing and subscription on AWS, can I assume that the clients are connected and if I publish a message I should be getting that at the subscriber's end which is my lambda function here?
Also, what's the technique in which I can get the message payload in my lambda function?
I am adding the below as per cjwfuller's suggestion.
Below I have written down the method for publishing in Python
client_boto = boto3.client('sns', aws_access_key_id='###',
aws_secret_access_key='###', region_name='us-west-2')
REGION = 'us-west-2'
TOPIC = 'arn:aws:sns:us-west-2:***:topic_name'
MSG = ntpath.basename(f_string)
SUBJECT_boto = 'File Name'
pub =client_boto.publish(TopicArn = TOPIC, Message = MSG,
Subject=SUBJECT_boto)
I am writing the subscribing code in Java.
Since my lambda func is already subscribed to it on AWS console,
should my Java program include the subscription again or is there a
way to get the msg payload directly.
Which language are you writing the function in? JavaScript as an example:
exports.myHandler = function(event, context, callback) {
console.log("value1 = " + event.key1);
console.log("value2 = " + event.key2);
// ...
}
Source: http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html
It's useful testing the sort of stuff in the AWS Lambda console before writing all the code.
Since I have configured the publishing and subscription on AWS, can I assume that the clients are connected
Clients aren't really "connected", they're simply subscribed to a topic.
publish a message I should be getting that at the subscriber's end which is my lambda function here?
Sounds like you're doing the right sort of thing - posting example code will help us come up with more precise answers.
On searching, I have found the class for SNSEvent which is, https://github.com/aws/aws-lambda-java-libs/blob/master/aws-lambda-java-events/src/main/java/com/amazonaws/services/lambda/runtime/events/SNSEvent.java
This Class contains all the methods related to and needed to get the message payload.
The Lambda function handler in Java goes something like this,
example;
public void handleRequest(SNSEvent input, Context context){
String this_takes_message=input.getRecords().get(0).getSNS().getMessage();
}