With ActorPublisher deprecated in favor of GraphStage, it looks as though I have to give up my actor-managed state for GraphStateLogic-managed state. But with the actor managed state I was able to mutate state by sending arbitrary messages to my actor and with GraphStateLogic I don't see how to do that.
So previously if I wanted to create a Source to expose data that is made available via HTTP request/response, then with ActorPublisher demand was communicated to my actor by Request messages to which I could react by kicking off an HTTP request in the background and send responses to my actor so I could send its contents downstream.
It is not obvious how to do this with a GraphStageLogic instance if I cannot send it arbitrary messages. Demand is communicated by OnPull() to which I can react by kicking off an HTTP request in the background. But then when the response comes in, how do I safely mutate the GraphStateLogic's state?
(aside: just in case it matters, I'm using Akka.Net, but I believe this applies to the whole Akka streams model. I assume the solution in Akka is also the solution in Akka.Net. I also assume that ActorPublisher will also be deprecated in Akka.Net eventually even though it is not at the moment.)
I believe that the question is referring to "asynchronous side-channels" and is discussed here:
http://doc.akka.io/docs/akka/2.5.3/scala/stream/stream-customize.html#using-asynchronous-side-channels.
Using asynchronous side-channels
In order to receive asynchronous events that are not arriving as stream elements (for example a completion of a future or a callback from a 3rd party API) one must acquire a AsyncCallback by calling getAsyncCallback() from the stage logic. The method getAsyncCallback takes as a parameter a callback that will be called once the asynchronous event fires.
Related
Is it possible to ensure that a message was successfully delivered to an Event Hub when sending it with the log-to-eventhub policy in API Management?
Edit: In our solution we cannot allow any request to proceed if a message was not delivered to the Event Hub. As far as I can tell the log-to-eventhub policy doesn't check for this.
Welcome to Stackoveflow!
Note: Once the data has been passed to an Event Hub, it is persisted and will wait for Event Hub consumers to process it. The Event Hub does not care how it is processed; it just cares about making sure the message will be successfully delivered.
For more details, refer “Why send to an Azure Event Hub?”.
Hope this helps.
Event Hubs is built on top of Service Bus. According to the Service Bus documentation,
Using any of the supported Service Bus API clients, send operations into Service Bus are always explicitly settled, meaning that the API operation waits for an acceptance result from Service Bus to arrive, and then completes the send operation.
If the message is rejected by Service Bus, the rejection contains an error indicator and text with a "tracking-id" inside of it. The rejection also includes information about whether the operation can be retried with any expectation of success. In the client, this information is turned into an exception and raised to the caller of the send operation. If the message has been accepted, the operation silently completes.
When using the AMQP protocol, which is the exclusive protocol for the .NET Standard client and the Java client and which is an option for the .NET Framework client, message transfers and settlements are pipelined and completely asynchronous, and it is recommended that you use the asynchronous programming model API variants.
A sender can put several messages on the wire in rapid succession without having to wait for each message to be acknowledged, as would otherwise be the case with the SBMP protocol or with HTTP 1.1. Those asynchronous send operations complete as the respective messages are accepted and stored, on partitioned entities or when send operation to different entities overlap. The completions might also occur out of the original send order.
I think this means the SDK is getting a receipt for each message.
This theory is further aided by the RetryPolicy Class used in the ClientEntity.RetryPolicy Property of the EventHubSender Class.
In the API Management section on logging-to-eventhub, there is also a section on retry intervals. Below that are sections on modifying the return response or taking action on certain status codes.
Once the status codes of a failed logging attempt are known, you can modify the policies to take action on failed logging attempts.
Imagine I have a JobsEndpoint class, which contains a JobSupervisor class which then has two child actors, RepositoryActor and StreamsSupervisorActor. The behavior for different requests to this top level JobSupervisor will need to be performed in the appropriate child actor. For example, a request to store a job will be handled exclusively in the RepositoryActor, etc...
My question, then, is if it is an anti-pattern to pass the request context through these actors via the messages, and then complete the request as soon as it makes sense?
So instead of doing this:
Request -> Endpoint ~ask~> JobSupervisor ~ask~> RepositoryActor
Response <- Endpoint <- JobSupervisor <-|return result
I could pass the RequestContext in my messages, such as StoreJob(..., ctx: RequestContext), and then complete it in the RepositoryActor.
I admittedly haven't been using Akka long but I see a few opportunities for improvement.
First, you are chaining "ask" calls which block threads. In some cases it's unavoidable but I think in your case, it is avoidable. When you block threads, you're potentially hurting your throughput.
I would have the Endpoint send a message with it's ActorRef as a "reply to" field. That way, you don't have to block the Endpoint and JobSupervisor actors. Whenever Repository actor completes the operation, it can send the reply directly to Endpoint without having to traverse middlemen.
Depending on your messaging guarantee needs, the Endpoint could implement retrying and de-duplicating if necessary.
Ideally each actor will have everything it needs to process a message in the message. I'm not sure what your RequestContext includes but I would consider:
1) Is it hard to create one? This impacts testability. If the RequestContext is difficult to create, I would opt for pulling out just the needed members so that I could write unit tests.
2) Can it be serialized? If you deploy your actor system in a cluster environment, then you'll need to serialize the messages. Messages that are simple data holders work best.
I have an application which uses spray-servlet to bootstrap my custom Spray routing Actor via spray.servlet.Initializer. The requests are then handed off to my Actor via spray.servlet.Servlet30ConnectorServlet.
From what I can gather, the Servlet30ConnectorServlet simply retrieves my Actor out of the ServletContext that the Initializer had set when the application started, and hands the HttpServletRequest to my Actor's receive method. This leads me to believe that only one instance of my Actor will have to handle all requests. If my Actor blocks in its receive method, then subsequent requests will queue waiting for it to complete.
Now I realize that I can code my routing Actor to use detach() or a complete that returns a Future, however most of the documentation never alludes to having to do this.
If my above assumption is true (single Actor instance handling all requests), is there a way to configure the Servlet30ConnectorServlet to perhaps load balance the incoming requests amongst multiple instances of my routing Actor instead of just the one? Or is this something I'll have to roll myself by subclassing Servlet30ConnectorServlet?
I did some research and now I understand better how spray-servlet is working. It's not spray-servlet that dictates the strategy for how many Request Handler Actors are created but rather the plumbing code provided with the example I based my application on.
My assumption all along was that spray-servlet would essentially work like a traditional Java EE application dispatcher in a handler-per-request type of fashion (or some reasonable variant of that notion). That is not the case because it is routing the request to an Actor with a mailbox, not some singleton HttpServlet.
I am now delegating the requests to a pool of actors in order to reduce our potential for bottleneck when our system is under load.
val serviceActor = system.actorOf(RoundRobinPool(config.SomeReasonableSize).props(Props(Props[BootServiceActor])), "my-route-actors")
I am still a bit baffled by the fact that the examples and documentation assumes everyone would be writing non-blocking Request Handler Actors under spray. All of their documentation essentially demonstrates non-Future rendering complete, yet there is no mention in their literature that maybe, just maybe, you might want to create a reasonable sized pool of Request Handler Actors to prevent a slew of requests from bottle necking the poor single overworked Actor. Or it's possible I've overlooked it.
I have a situation where I need my API to have a call for triggering a service-side event, no information (besides authentication) is needed from the client, and nothing needs to be returned by the server. Since this doesn't fit well into the standard CRUD/Resource interaction, should I take this as an indicator that I'm doing something wrong, or is there a RESTful design pattern to deal with these conditions?
Your client can just:
POST /trigger
To which the server would respond with a 202 Accepted.
That way your request can still contain the appropriate authentication headers, and the API can be extended in the future if you need the client to supply an entity, or need to return a response with information about how to query the event status.
There's nothing "non-RESTful" about what you're trying to do here; REST principles don't have to correlate to CRUD operations on resources.
The spec for 202 says:
The entity returned with this response SHOULD include an indication of
the request's current status and either a pointer to a status monitor
or some estimate of when the user can expect the request to be
fulfilled.
You aren't obliged to send anything in the response, given the "SHOULD" in the definition.
REST defines the nature of the communication between the client and server. In this case, I think the issues is there is no information to transfer.
Is there any reason the client needs to initiate this at all? I'd say your server-side event should be entirely self-contained within the server. Perhaps kick it off periodically with a cron call?
Should WS-notification (WS Notification) be used to just notify or should the data also be transmitted with the payload to save an extra call(back).
Use Case:
A customer's record has changed. Need to notify other systems. Sends a notification.
Scenario 1.
Send the notification with customer record changes. Could be bad since each listening system might do a different action or may or may not need the customer record.
Scenario 2.
Just send the notification. Means that each listening system will have to "react" in some way. Responsibility is on the listening system.
Two methods.
Pub/Sub Push and Pub/Sub Pull.
Pub/Sub Push is to push out the full data.
Pub/Sub Pull is to send enough data for the target app to call back and request the full data. This allows better control of information passed than the pub/sub push method.
Pub/Sub Push method is the easiest to implement.
Pub-sub kind of implies that the notification consumers are already interested in the topic in question by virtue of the fact that they have subscribed. However, as you say, they may not need to respond. So if you consider the notification to be a true event then the notifying system is saying, "here is a notification that my state has changed". If the notification consumer is interested it can use request-response to get that new state. This would be more flexible and lightweight.
Notifications are inherently more event-oriented and therefore using them to push state should be considered carefully. Particularly as with pub-sub you seldom have an idea as to how many subscribers you have at run time - then capacity planning can be difficult and peak load spikes are not uncommon.
So keep the notifications lightweight. Let the consumers decide if they're to act on the event. You're on your way to a true EDA!