How do we pump out messages from service bus topic's subscription.
I have been able to receive from service bus queue successfully.
public class Functions
{
public static void ProcessQueueMessage([ServiceBusTrigger("mseoikeyword")] BrokeredMessage message, TextWriter log)
{
string strMsg;
strMsg = message.GetBody<string>();
log.WriteLine(strMsg);
}
}
Using the same trigger with topic path and subscription name as per documentation.
Ensure topic and subscription exist as they will not be created by the trigger.
Related
Lets say for some reason I want to pause consuming reading messages from my SQS ... like a service on the client side will be down for maintanence. Can I pause from my SQS Listener?
#SqsListener(value = "${aws.sqs.listener.myqueue}", deletionPolicy = SqsMessageDeletionPolicy.ON_SUCCESS)
public void processMessage(MyObj myObj) {
// do something with myObj
// if db is down then throw error and pause reading from SQS
}
I have run access some post that suggest to have a killSwitch but this is not ideal for production as it keeps consuming the messages and reposting to the queue.
I have a springboot micrservices application that needs to subscribe to topic and all copies of the container should get the message. I am used to activemq where you can connect directly to the topic and all containers get the message.
I set my appliation.yaml like:
servicebus:
connection-string: ${AZURE_SERVICE_BUS_URL}
idle-timeout: 1800000
pricing-tier: standard
topic-client-id: ${AZURE_SERVICE_BUS_TOPIC_CLIENT_ID}
listener:
subscription-durable: false
I set the listener as:
#JmsListener(destination = "edge-bindermgmt-request-state", containerFactory = "topicJmsListenerContainerFactory")
public void receiveMessage(RequestProcessingStateChange requestProcessingStateChange){
log.info("Received: {}", requestProcessingStateChange);
}
With the following I am only getting the message in one copy of the service:
#JmsListener(destination = "edge-bindermgmt-request-state", containerFactory = "topicJmsListenerContainerFactory",
subscription = "edge-bindermgmt-request-state-subscription")
public void receiveMessage(RequestProcessingStateChange requestProcessingStateChange){
log.info("Received: {}", requestProcessingStateChange);
}
So what I want is to setup the subscription on the fly, rather than having to set it up in:
screenshot of subscriptions page
If anyone else is wondering, you need the premium subscription for Service Bus for that. I ended up going with ActiveMQ instead.
I am using AWS Lambda function to consume the messages received on AWS SQS. The lambda function will further call my service but for now, I was trying to get the messages sent on SQS queue. I have made a standard queue in SQS and a Lambda function in Java. I made a new lambda project in my Eclipse IDE and this is my function:
package com.amazonaws.lambda.demo;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
public class LambdaFunctionHandler implements RequestHandler<String, String> {
#Override
public String handleRequest(String input, Context context) {
context.getLogger().log("Input: " + input);
return (String) input;
}
}
The file does not show any error. I exported the project as .jar file and uploaded the jar file to my lambda function. I have also specified the handler in my lambda function (packageName.className). If I test the lambda function it executed fine but when I send a message to SQS, it increments the no of messages in message flight which I have read are the messages not consumed by the consumer which is lambda in my case.
Please let me know if anything else is needed.
Amazon does not deserialize Strings and so when I used Map as an input, it worked fine.
public String handleRequest(Map<String,Object> input, Context context) {
context.getLogger().log("Your Input is: " +input);
System.out.println(input);
return "Executed Successfully";
}
I read this from Lionel Port's solution. Thanks.
take SQSEvent event as a parameter with will connect your aws SQS with Java. Than you can get the messages using SQSMessage class and see your messages. see the below codes:
public String handleRequest(SQSEvent event, Context context){
for(SQSMessage msg : event.getRecords()){
System.out.println(msg.getBody());
System.out.println("MSG" + msg.getBody());
}
}
We are currently consuming a single SQS queue to process messages.
However as extension of functionality, we need to support multiple regions with same queue name.
Current implementation of jmsListener is tied to a given SQS queue in a given region as below:
SQSListener.java
#Component
public class SQSListener {
#Override
#JmsListener(destination = "${QueueName}", concurrency = "${JmsThreadCount}")
public void onMessage(Message message) throws JMSException {
}
SQSConfiguration.java
#Component
#EnableJms
public class SQSConfig {
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory = null;
try {
factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(getSqsConnectionFactory()); //passes credentials in local method call
factory.setDestinationResolver(new DynamicDestinationResolver());
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
} catch (Exception e) {
logger.error(e.getLocalizedMessage());
}
return factory;
}
}
Application.properties
QueueName=xyz
JmsThreadCount=1-5
Regions=us-east-1, us-west-2 #*(newly added region)*
How can I make it generic to create multiple listener class implementations by regions specified in the configurations ?
Though there is no direct way using spring framework's annotations to create listeners to listen to multiple queue's in multiple AWS regions (same queue name in multiple AWS regions), I was able to solve above problem by programmatic configuration of JMS Container Factory beans for each region, JMS Templates for each region, and JMS Listeners for each region.
I will post sample code in next few days, that may be useful to the community.
You can listen to SQS from different region . If you provide not the queue name but full https url of the SQS . I have applied this in my application and it worked
eg SQS http URL - https://sqs.us-west-2.amazonaws.com/YOURACCOUNTNUMBER/YOURQUEUENAME
#SqsListener(value = "https://sqs.us-west-2.amazonaws.com/123456789/MYQUEUENAME
", deletionPolicy = SqsMessageDeletionPolicy.ON_SUCCESS)
protected void queueListener(String dataFomSQS, #Header("Acknowledgment") Acknowledgment acknowledgment) {}
When I create a new queue and subscribe it to a topic in Java, no message comes. The same via the AWS web console works fine.
I guess I have to confirm the subscription somehow, but the sns.confirmSubscription method needs a token - where shall I get it?
This is my Java code:
String queueURL = sqs.createQueue("my-queue").getQueueUrl();
sns.subscribe(myTopicARN, "sqs", queueURL);
sns.publish(myTopicARN, "{\"payload\":\"test\"}");
sqs.receiveMessage(queueURL).getMessages()
.forEach(System.out::println); // nothing
What am I doing wrong?
Check this out: https://aws.amazon.com/blogs/developer/subscribing-queues-to-topics/
You should subscribe like this:
Topics.subscribeQueue(sns, sqs, myTopicARN, queueURL);
This convinient method creates a policy for the subscription to allow the topic to send messages to the queue.
subscribing the queue to sns does not automatically create a policy to allow sns to send messages to the queue (based on my experience with sns/sqs) so you need to create the policy yourself and give permission to sns to send messages to your queue this is an example on how to do it using queue url , queue arn and topic arn
import static com.amazonaws.auth.policy.Principal.All;
import static com.amazonaws.auth.policy.Statement.Effect.Allow;
import static com.amazonaws.auth.policy.actions.SQSActions.SendMessage;
import static com.amazonaws.auth.policy.conditions.ArnCondition.ArnComparisonType.ArnEquals;
final Statement mainQueueStatements = new Statement(Allow) //imported above
.withActions(SendMessage) //imported above
.withPrincipals(All) //imported above
.withResources(new Resource(queueArn)) // your queue arn
.withConditions(
new Condition()
.withType(ArnEquals.name()) //imported above
.withConditionKey(SOURCE_ARN_CONDITION_KEY) //imported above
.withValues(topicArn) // your topic arn
);
final Policy mainQueuePolicy = ()
.withId("MainQueuePolicy")
.withStatements(mainQueueStatements);
final HashMap<QueueAttributeName, String> attributes = new HashMap<>();
attributes.put(QueueAttributeName.Policy.toString(), mainQueuePolicy.toJson());
amazonSQS.setQueueAttributes(new SetQueueAttributesRequest().withAttributes(attributes).withQueueUrl(queueUrl)); // your queue url