Graceful termination - akka

I am trying to implement the following use case as part of my akka learning
I would like to calculate the total streets in all cities of all states. I have a database that contain the details needed. Here is what i have so far
Configuration
akka.actor.deployment {
/CityActor{
router = random-pool
nr-of-instances = 10
}
/StateActor {
router = random-pool
nr-of-instances = 1
}}
Main
public static void main(String[] args) {
try {
Config conf = ConfigFactory
.parseReader(
new FileReader(ClassLoader.getSystemResource("config/forum.conf").getFile()))
.withFallback(ConfigFactory.load());
System.out.println(conf);
final ActorSystem system = ActorSystem.create("AkkaApp", conf);
final ActorRef masterActor = system.actorOf(Props.create(MasterActor.class), "Migrate");
masterActor.tell("", ActorRef.noSender());
} catch (Exception e) {
e.printStackTrace();
}
}
MasterActor
public class MasterActor extends UntypedActor {
private final ActorRef randomRouter = getContext().system()
.actorOf(Props.create(StateActor.class).withRouter(new akka.routing.FromConfig()), "StateActor");
#Override
public void onReceive(Object message) throws Exception {
if (message instanceof String) {
getContext().watch(randomRouter);
for (String aState : getStates()) {
randomRouter.tell(aState, getSelf());
}
randomRouter.tell(new Broadcast(PoisonPill.getInstance()), getSelf());
} else if (message instanceof Terminated) {
Terminated ater = (Terminated) message;
if (ater.getActor().equals(randomRouter)) {
getContext().system().terminate();
}
}
}
public List<String> getStates() {
return new ArrayList<String>(Arrays.asList("CA", "MA", "TA", "NJ", "NY"));
};}
StateActor
public class StateActor extends UntypedActor {
private final ActorRef randomRouter = getContext().system()
.actorOf(Props.create(CityActor.class).withRouter(new akka.routing.FromConfig()), "CityActor");
#Override
public void onReceive(Object message) throws Exception {
if (message instanceof String) {
System.out.println("Processing state " + message);
for (String aCity : getCitiesForState((String) message)) {
randomRouter.tell(aCity, getSelf());
}
Thread.sleep(1000);
}
}
public List<String> getCitiesForState(String stateName) {
return new ArrayList<String>(Arrays.asList("Springfield-" + stateName, "Salem-" + stateName,
"Franklin-" + stateName, "Clinton-" + stateName, "Georgetown-" + stateName));
};}
CityActor
public class CityActor extends UntypedActor {
#Override
public void onReceive(Object message) throws Exception {
if (message instanceof String) {
System.out.println("Processing city " + message);
Thread.sleep(1000);
}
}}
Did i implement this use case properly?
I cannot get the code to terminate properly, i get dead letters messages. I know why i am getting them, but not sure how to properly implement it.
Any help is greatly appreciated.
Thanks

I tested and ran your use case with Akka 2.4.17. It works and terminate properly, without any dead letters logged.
Here are some remarks/suggestions to improve your understanding of the Akka toolkit:
Do not use Thread.sleep() inside an actor. Basically, it is never a good practice since a same thread may do many tasks for many actors (this is the default behavior with a shared thread pool). Instead, you can use an Akka scheduler or assign a single thread to a specific Actor (see this post for more details). See also the Akka documentation about that topic.
Having some dead letters is not always an issue. It generally arises when the system stops an Actor that had some messages within its mailbox. In this case, the remaining unprocessed messages are sent to deadLetters of the ActorSystem. I recommend you to check the configuration you provided for the logging of dead letters. If the file forum.conf you provided is your complete configuration file for Akka, you may want to customize some additional settings. See the page Logging of Dead Letters and Stopping actors on Akka's website. For instance, you could have a section like this:
akka {
# instead of System.out.println(conf);
log-config-on-start = on
# Max number of dead letters to log
log-dead-letters = 10
log-dead-letters-during-shutdown = on
}
Instead of using System.out.println() to log/debug, it is more convenient to set up a dedicated logger for each Actor that provides you additional information such as dispatchers, Actor name, etc. If your are interested, have a look to the Logging page.
Use some custom immutable message objects instead of systematic Strings. At first, it may seem painful to have to declare new additional classes but in the end it helps to better design complex behaviors and it's more readable. For instance, an actor A can answer to a RequestMsg coming from an actor B with an AnswerMsg or a custom ErrorMsg. Then, for your actor B, you will end up with the following onReceive() method:
#Override
public void onReceive(Object message) {
if (message instanceof AnswerMsg) {
// OK
AnswerMsg answerMsg = (AnswerMsg) message;
// ...
}
if (message instanceof ErrorMsg) {
// Not OK
ErrorMsg errorMsg = (ErrorMsg) message;
// ...
}
else {
// Unexpected behaviour, log it
log.error("Error, received " + message.toString() + " object.")
}
}
I hope that these resources will be useful for you.
Have a happy Akka programming! ;)

Related

BulkProcessor .add() not finishing when number of bulks > concurrentRequests

Here is a sample of the code flow:
Trigger the process with an API specifying bulkSize and totalRecords.
Use those parameters to acquire data from DB
Create a processor with the bulkSize.
Send both the data and processor into a method which:
-iterates over the resultset, assembles a JSON for each result, calls a method if the final JSON is not empty and adds that final JSON to the process using processor.add() method.
This is where the outcome of the code is split
After this, if the concurrentRequest parameter is 0 or 1 or any value < (totalRecords/bulkSize), the processor.add() line is where the code stalls and never continues to the next debug line.
However, when we increase the concurrentRequest parameter to a value > (totalRecords/bulkSize), the code is able to finish the .add() function and move onto the next line.
My reasoning leads me to believe we might be having issues with our BulkProcessListener which is making the .add() no close or finish like it is supposed to. I would really appreciate some more insight about this topic!
Here is the Listener we are using:
private class BulkProcessorListener implements Listener {
#Override
public void beforeBulk(long executionId, BulkRequest request) {
// Some log statements
}
#Override
public void afterBulk(long executionId, BulkRequest request, BulkResponse response) {
// More log statements
}
#Override
public void afterBulk(long executionId, BulkRequest request, Throwable failure) {
// Log statements
}
}
Here is the createProcessor():
public synchronized BulkProcessor createProcessor(int bulkActions) {
Builder builder = BulkProcessor.builder((request, bulkListener) -> {
long timeoutMin = 60L;
try {
request.timeout(TimeValue.timeValueMinutes(timeoutMin));
// Log statements
client.bulkAsync(request, RequestOptions.DEFAULT,new ResponseActionListener<BulkResponse>());
}catch(Exception ex) {
ex.printStackTrace();
}finally {
}
}, new BulkProcessorListener());
builder.setBulkActions(bulkActions);
builder.setBulkSize(new ByteSizeValue(buldSize, ByteSizeUnit.MB));
builder.setFlushInterval(TimeValue.timeValueSeconds(5));
builder.setConcurrentRequests(0);
builder.setBackoffPolicy(BackoffPolicy.noBackoff());
return builder.build();
}
Here is the method where we call processor.add():
#SuppressWarnings("deprecation")
private void addData(BulkProcessor processor, String indexName, JSONObject finalDataJSON, Map<String, String> previousUniqueObject) {
// Debug logs
processor.add(new IndexRequest(indexName, INDEX_TYPE,
previousUniqueObject.get(COMBINED_ID)).source(finalDataJSON.toString(), XContentType.JSON));
// Debug logs
}

Logging handler usage

my question is similar to Why does java.util.logging.Logger print to stderr? post.
I want to use only one handler in my logging, where as it should print INFO statements on to out stream and WARNING & SEVERE onto error stream.
Is this possible ?
If I take two handlers for example,
one for out stream - and level as INFO
another for error stream - and level as WARNING/SEVERE in this case, application is showing messages twice
one with out stream and another with error stream.
So any solution ?
This is possible with one or two handlers. The missing part is that you need to create a filter for the OUT handler that limits the highest level. Also you need to make sure there are no other ConsoleHandlers attached to the root logger which can pollute your test. You can print the logger tree to see what handlers are attached.
Here is a proof of concept:
import java.io.PrintStream;
import java.util.logging.ConsoleHandler;
import java.util.logging.Filter;
import java.util.logging.Handler;
import java.util.logging.Level;
import java.util.logging.LogManager;
import java.util.logging.LogRecord;
import java.util.logging.Logger;
public class OutErrorTest {
private static final Logger log = Logger.getLogger("");
public static void main(String[] args) {
LogManager.getLogManager().reset(); //A quick way to remove all handlers.
Handler out = newSystemOut();
Handler err = newSystemErr();
final Level fence = Level.WARNING;
out.setLevel(Level.ALL);
out.setFilter(new LessThanLevelFilter(fence));
err.setLevel(fence);
log.addHandler(out);
log.addHandler(err);
log.setLevel(Level.ALL);
log.finest("Finest Log");
log.finer("Finer Log");
log.fine("Fine Log");
log.config("Config Log");
log.info("Info Log");
log.warning("Warning Log");
log.severe("Severe Log");
}
private static Handler newSystemErr() {
return new ConsoleHandler();
}
private static Handler newSystemOut() {
Handler h = null;
final PrintStream err = System.err;
System.setErr(System.out);
try {
h = new ConsoleHandler(); // Snapshot of System.err
} finally {
System.setErr(err);
}
return h;
}
public static class LessThanLevelFilter implements Filter {
private final int lvl;
public LessThanLevelFilter(final Level max) {
this(max.intValue());
}
public LessThanLevelFilter(final int max) {
this.lvl = max;
}
#Override
public boolean isLoggable(LogRecord r) {
return r.getLevel().intValue() < lvl;
}
}
}

Connect AWS SQS to Apache-Flink

Why is AWS SQS not a default connector for Apache Flink? Is there some technical limitation to doing this? Or was it just something that didn't get done? I want to implement this, any pointers would be appreciated
Probably too late for an answer to the original question... I wrote a SQS consumer as a SourceFunction, using the Java Messaging Service library for SQS:
SQSConsumer extends RichParallelSourceFunction<String> {
private volatile boolean isRunning;
private transient AmazonSQS sqs;
private transient SQSConnectionFactory connectionFactory;
private transient ExecutorService consumerExecutor;
#Override
public void open(Configuration parameters) throws Exception {
String region = ...
AWSCredentialsProvider credsProvider = ...
// may be use a blocking array backed thread pool to handle surges?
consumerExecutor = Executors.newCachedThreadPool();
ClientConfiguration clientConfig = PredefinedClientConfigurations.defaultConfig();
this.sqs = AmazonSQSAsyncClientBuilder.standard().withRegion(region).withCredentials(credsProvider)
.withClientConfiguration(clientConfig)
.withExecutorFactory(()->consumerExecutor).build();
this.connectionFactory = new SQSConnectionFactory(new ProviderConfiguration(), sqs);
this.isRunning = true;
}
#Override
public void run(SourceContext<String> ctx) throws Exception {
SQSConnection connection = connectionFactory.createConnection();
// ack each msg explicitly
Session session = connection.createSession(false, SQSSession.UNORDERED_ACKNOWLEDGE);
Queue queue = session.createQueue(<queueName>);
MessageConsumer msgConsumer = session.createConsumer(queue);
msgConsumer.setMessageListener(msg -> {
try {
String msgId = msg.getJMSMessageID();
String evt = ((TextMessage) msg).getText();
ctx.collect(evt);
msg.acknowledge();
} catch (JSMException e) {
// log and move on the next msg or bail with an exception
// have a dead letter queue is configured so this message is not lost
// msg is not acknowledged so it may be picked up again by another consumer instance
}
};
// check if we were canceled
if (!isRunning) {
return;
}
connection.start();
while (!consumerExecutor.awaitTermination(1, TimeUnit.MINUTES)) {
// keep waiting
}
}
#Override
public void cancel() {
isRunning = false;
// this method might be called before the task actually starts running
if (sqs != null) {
sqs.shutdown();
}
if(consumerExecutor != null) {
consumerExecutor.shutdown();
try {
consumerExecutor.awaitTermination(1, TimeUnit.MINUTES);
} catch (Exception e) {
//log e
}
}
}
#Override
public void close() throws Exception {
cancel();
super.close();
}
}
Note if you are using a standard SQS queue you may have to de-dup the messages depending on whether exactly-once guarantees are required.
Reference:
Working with JMS and Amazon SQS
At the moment, there is no connector for AWS SQS in Apache Flink. Have a look at the already existing connectors. I assume you already know about this, and would like to give some pointers. I was also looking for an SQS connector recently and found this mail thread.
Apache Kinesis Connector is somewhat similar to what you can implement on this. See whether you can get a start on this using this connector.

Akka.net Testkit does not mark test case failed despite ActorInitializationException exception

Following is the actor, I've defined (trying to get my head around persistent actor!!)
public class Country : ReceivePersistentActor
{
public override string PersistenceId => GetType().Name + state.Id;
private CountryState state;
public Country()
{
Command<CreateCountry>(CreateCountry);
}
private bool CreateCountry(CreateCountry cmd)
{
Persist(new CountryCeated
{
Id = cmd.Id,
Code = cmd.Code,
Description = cmd.Description,
Active = cmd.Active
}, evt =>
{
state = new CountryState
{
Id = evt.Id,
Code = evt.Code,
Description = evt.Description,
Active = evt.Active
};
});
return true;
}
}
Following is unit test case that I've defined:
[TestClass]
public class CountrySpec : TestKit
{
[TestMethod]
public void CountryActor_Should_Create_A_Country()
{
var country = Sys.ActorOf(Props.Create(() => new Country()), "Country");
country.Tell(new CreateCountry(Guid.NewGuid(), "UK", "United Kingdom", true));
ExpectNoMsg();
}
}
When I run the test case, there is an exception that I can see in the output window of the test case
[ERROR][25/08/2016 08:25:07][Thread 0007][akka://test/user/Country] Object reference not set to an instance of an object.
Cause: [akka://test/user/Country#552449332]: Akka.Actor.ActorInitializationException: Exception during creation ---> System.NullReferenceException: Object reference not set to an instance of an object.
at Domain.Country.get_PersistenceId() in Y:\Spikes\StatefulActors\Domain\Country.cs:line 9
at Akka.Persistence.Eventsourced.StartRecovery(Recovery recovery)
at Akka.Persistence.Eventsourced.AroundPreStart()
at Akka.Actor.ActorCell.<>c__DisplayClass154_0.<Create>b__0()
at Akka.Actor.ActorCell.UseThreadContext(Action action)
at Akka.Actor.ActorCell.Create(Exception failure)
--- End of inner exception stack trace ---
at Akka.Actor.ActorCell.Create(Exception failure)
at Akka.Actor.ActorCell.SysMsgInvokeAll(EarliestFirstSystemMessageList messages, Int32 currentState)
but the test case is marked as success
Is there any way/settings in the TestKit, where it can be set such that for any exception, mark the test case failed?
By default, any exceptions inside actors are encapsulated - that means they don't bubble up, blowing the rest of the system.
Actors come in systems, and can be tested by observing the way they communicate with each other. Usually it comes up to providing inputs and asserting outputs from actor system - in your case test has passed, since you haven't validated any output. From the perspective of your test, this actor could be dead and it wouldn't make a difference.
Validating an outputs (either by assertion inside actor itself or i.e. using a custom test journal) is the best way to work with tests.
If for some reason you still have to catch exceptions inside actors, you could create supervision strategy bound to i.e. TestActor, where all exceptions could be forwarded:
public class TestingStrategy : OneForOneStrategy
{
protected TestingStrategy(IActorRef probe) : base(exception =>
{
probe.Tell(exception);
return DefaultDecider.Decide(exception);
}) { }
}

Is it possible / good idea to use Akka for multithreading in a Glassfish EAR?

Context:
This is a client server app. At the moment the EJB looks like:
public class ServerSideJob {
#WebMethod(operationName = "launchJob")
public String launchJob(#WebParam(name = "idUser") String idUser, #WebParam(name = "name") String name, #WebParam(name = "param") Object param) {
Runnable controller = new JobController(screenName, fof, mm, job);
new Thread(controller).start();
return "job launched";
}
}
The job is launching several other threads.
At this point, I'd like to add the possibility for the client to interrupt the job. Interrupting a thread from "the outside" is quite a dirty affair (I'd have to add many more calls per op to a db just for that), and this prompts me to switch to Akka for the multithreading.
Problem: I am not sure how / if I can merge the Akka logic with the code of the EJB above. How do I call the top actor of my hierarchy from my launchJob function? (ok, surely a noob question...)
public class ServerSideJob {
#WebMethod(operationName = "launchJob")
public String launchJob(#WebParam(name = "idUser") String idUser, #WebParam(name = "name") String name, #WebParam(name = "param") Object param) {
//how do I call my Akka actor here?
return "job launched";
}
}
That was indeed a noob question. I did not know how to do that because, following the "hello world" example on akka.io, the app was launched without an ActorSystem so I ignored it. An ActorSystem is simply what you need:
One class for the ActorSystem, to be launched at the start up at the app:
#Singleton
#Startup
// initialize at deployment time instead of first invocation
public class SharedActorSystem {
ActorSystem system;
#PostConstruct
void loadConfiguration() {
system = ActorSystem.create("systemJobs");
}
public ActorSystem getSystem() {
return system;
}
}
And the class with the server side job:
public class ServerSideJob {
#EJB
SharedActorSystem sharedActorSystem;
#WebMethod(operationName = "launchJob")
public String launchJob(#WebParam(name = "idUser") String idUser) {
//getting the ActorSystem
ActorSystem system = sharedActorSystem.getSystem();
final ActorRef myActor = system.actorOf(Props.create(MyActor.class));
MsgLaunchActor msg = new MsgLaunchActor(idUser);
myActor.tell(msg, ActorRef.noSender());
return "job launched";
}
}