Catching Listener Exceptions in long running Cloud PubSub Subscriber service - google-cloud-platform

Question

I am trying to write a long running Subscriber service in Java. I have set up the Listeners to listen to any failures inside the Subscriber service. I am trying to make this fault tolerant and I do not quite understand few things, Below are my doubts/questions.
I have followed the basic setup shown here https://github.com/googleapis/google-cloud-java/blob/master/google-cloud-examples/src/main/java/com/google/cloud/examples/pubsub/snippets/SubscriberSnippets.java. Specifically, I have setup addListener as shown below.
As shown in the following code, initializeSubscriber acts a state variable which will determine if the Subscriber service should restart. Inside the while loop, this variable is continuously monitored to determine if the restart is required.
My question here is,
1. How do I raise an exception inside Subscriber.Listener's failed method and capture it in the main while loop. I tried throwing a new Exception() in failed method and catching it in catch block inside while, However, I am unable to compile the code as it is a checked exception.
2. As shown here, I use Java Executor thread to run the Listener. How do I handle the Listener failures ? Will I able to catch Listener failures under general Exception catch block as shown here ?
try {
boolean initializeSubscriber = true;
while (true) {
try {
if (initializeSubscriber) {
createSingleThreadedSubscriber();
addErrorListenerToSubscriber();
subscriber.startAsync().awaitRunning();
initializeSubscriber = false;
}
// Checks the status of subscriber service every minute
Thread.sleep(60000);
} catch (Exception ex) {
LOGGER.error("Could not start the Subscriber service", ex);
cleanupSubscriber();
initializeSubscriber = true;
}
}
} catch (RuntimeException e) {
} finally {
shutdown();
}
private void addErrorListenerToSubscriber() {
subscriber.addListener(
new Subscriber.Listener() {
#Override
public void failed(Subscriber.State from, Throwable failure) throws RuntimeException {
LOGGER.info("Subscriber reached a failed state due to " + failure.getMessage()
+ ",Restarting Subscriber service");
initializeSubscriber = true;
}
},
Executors.newSingleThreadExecutor());
}
private void cleanupSubscriber() {
try {
if (subscriber != null) {
subscriber.stopAsync().awaitTerminated();
}
if (!subscriptionListener.isShutdown()) {
subscriptionListener.shutdown();
}
} catch (Exception ex) {
LOGGER.error("Error in cleaning up Subscriber thread " + ex);
}
}

Answer 1

It should not be necessary to add a listener to the subscriber if you just want to recreate the subscriber on a failure. You could instead catch the exception on awaitTerminated:
try {
boolean initializeSubscriber = true;
while (initializeSubscriber) {
try {
createSingleThreadedSubscriber();
subscriber.startAsync().awaitRunning();
initializeSubscriber = false;
subscriber.awaitTerminated();
} catch (Exception ex) {
LOGGER.error("Error in the Subscriber service", ex);
cleanupSubscriber();
initializeSubscriber = true;
}
}
} catch (RuntimeException e) {
} finally {
shutdown();
}
If the subscriber shutdown successfully because of a call to stopAsync, then awaitTerminated will not throw an exception. If there was some kind of exception, then awaitTerminated will throw an IllegalStateException because the state will be FAILED instead of TERMINATED.
Note that transient errors are handled by the library itself. For example, if the server become briefly unavailable due to a network hiccup, the library will seamlessly reconnect and continue to deliver messages. Failures that result in a change in state for the subscriber are likely permanent failures such as permission issues (where the account running the subscriber does not have permission to subscribe to the subscription) or resource issues (such as the subscription having been deleted). In these permanent failure cases, recreating the subscriber will likely just result in the same error unless one takes manual steps to intervene and fix the problem.