I have a situation where I have a component with a list of users. I need to send an email to each user, and then update the user's record to indicate the time when the email was sent.
This seems like a good first use-case for Akka. I was thinking the component that gets the list of users would be an Actor, and a separate Actor would be created for each outbound email-sending Actor, which could be multithreaded. The email-sending Actor would be responsible for updating the date in the user record.
If the email-sending Actor failed, the user's record wouldn't be done, and the supervising Actor would be able to reschedule it.
Is that the right model? Or should the supervisor be responsible for updating the user record?
Or is this a bad use-case for Akka completely?
BTW, I could send mail via SMTP or a web service. I don't think it matters much for the discussion, but I thought I'd mention it.
I don't think you should get hung up on transactionality when there's IO involved. You have no guarantees that the emails will be read, received, received by the right person, received within a certain period of time, be understood etc.
Just send the email and then update the database.
Happy hAkking!
Related
I have a question regarding the effect of eventually consistent (EC) microservice systems.
Imagine we have a booking system - a user-service A and booking-service B. Each service has its own database. Imagine the system does a concurrent booking of the same resource for distinct users at the same time. Lets assume we have a Runtime Verification System checking the concurrent booking.
Would it be possible that the monitor does not realize the concurrent booking at B, because the update in the database is done delayed because of the EC mechanism?
In your example, the Booking Service is the source of truth (presumably) for whether or not the resource is available to book. So, that service should be pretty clear on allowing the first booking request to happen and rejecting the second.
In a case like this, where "first come first served" is the requirement, you'd want an intermediate state that would wait for a response from the Booking Service and update the User Service only when a response has been received.
If your architecture is set up right, User Service shouldn't be calling Booking Service directly anyway - it should be communicating through a messaging plane. As such, when the User clicks "Book Now," you could generate a resourceBookingRequested message and submit it to the queue. You'd acknowledge this request has been queued to the user and update their UI to "Awaiting Booking Confirmation..." or something similar.
Once the booking is accepted, or rejected, the User Service subscribes to the resulting message and updates the UI (and/or takes other actions like sending an email) to let the user know their request succeeded or didn't.
I am looking into using an actor model framework (akka.net with akka.net persistence, but I am looking for a general case answer) to build an 'widget order processing workflow'.
Pretty standard:
Customer orders widget
Payment is processed
Email confirmation sent to customer
Send picklist message to warehouse
Warehouse sends a 'widget has been shipped' message back
Send a 'your item has shipped' email to customer
Now let's say between 4 and 5 a server deployment/restart happens. This would cause a actor(s) rehydration (let's assume there is no snapshot yet). That means we would process the payment again, and resend the order placed email. However it turns out our customers don't like this 'feature'!
How to I prevent non-idempotent actions from re-occurring when using an actor model framework?
I have thought about having a separate store of 'payment processed for order db table'; but this feels like I am fighting the framework/paradigm and I wonder if there is a 'proper' way of doing this kind of thing!
Ok so it turns out it is pretty simple.
With akka.net persistence, after a system restore, messages are replayed. The correct state can be recreated by (re) processing these messages.
There is however a IsRestoring property, which can be checked to see if this is the first or a subsequent processing. Presumably other actor model framework have something similar.
So you do something like:
private void ProcessPayment (Order message)
{
if(!this.IsRestoring){
//Perform non-idempotent payment process
}
}
To make a robust workflow processor, you have to store ALL data of a workflow process in a permanent storage.
You can employ a database, a messaging system like Kafka, or use ready-made workflow management software.
Since you already use Akka, Akka Persistence also can be an option.
UPDATE
Building a system which continue to work correctly in presence of system failures and restarts is a considerable task, far more complex than developing an actor framework. That is, you cannot just take any actor framework and add fault tolerance to it.
Please help selecting a MQ app/system/approach for the following use-case:
Check for incoming messages for a specific user -> read the message if available -> delete from the queue, ideally, staying within AWS.
Context:
Social networking app, users receiving messages, i.e.
I need to identify incoming messages by recipient ID.
The app is doing long-polls for new messages every 30 seconds.
Message size is <1Kb.
As per current estimates, I'll need 100M+ message checks per months in total (however, much less messages, these are just checks).
While users acknowledge messages choosing OK or Ignore, however not sure if ACK support is required from MQ system for that.
I'm in AWS. Initially thought of SQS, but the more I read the less it looks like a good match - cannot set message recipient ID in a way to filter by recipient, etc, however maybe I'm wrong.
One of the options I also thought about is to just use DynamoDB's "messages" table, partition key being userId and sort key being a messageId, thus I'll be able to easily query by a user, however concerned with costs.
If possible, I would much more prefer to stay within AWS or at least use SAAS like SQS, as being a 1-person startup I really want to avoid headaches supporting self-hosted system.
Thank you!
D
You are right on both these counts:
SQS won't work, because of the limitation you pointed.
DynamoDB would work, but cost a lot.
I can suggest the following:
Create a Redis cluster, possibly on Amazon ElastiCache.
In it, make one List per user.
Whenever a new message comes, append it to concerned User's list.
To deliver the message, just read from the User's list. Also, flush the queue if needed.
What I am suggesting is very similar to how Twitter manages each User's news-feed and home-feed.
It should also be cheap.
I want incorporate a timed based reminder email of the events for the day in django. Basically I have a model which has all the events (including the date of the event). I want to send emails to concerned people at around 8.00 AM in the morning of each day about the events for the day. How do I incorporate this is django?
Thanks
I reckon a custom management command to send the alerts, commanded by django-chronograph should do the trick
I wrote a database-backed email queue, to send out emails from a single django install and not have to worry about SMTP throttling and whatnot. It's dead simple -- one model class for the email with a sendit() method, and a command-line script to flush the queue, which I run with cron.
http://gist.github.com/629663
Company A has async pooling based webservice for notifications. Company B checks for notifications. Every time when it reads new notifications A deletes them from the system. Thus subsequent read requests return only new notifications. There is also requirement for the client B to interrupt the connection if there is no response within 30 sec.
This causes one potential problem: Due to unexpected slowness it is possible for A get the request deleted a notification and send the response back while B is already interrupted the connection. Under this scenario notification gets lost. Now one can argue that the core problem lies within operation realm (the HTTP response must be delivered withing 20 sec ) still on practice it is not always feasible.
How to design B (the client) to avoid this problem?
One way I can see is to do not delete the notifications by A and make B be aware of its state, so that it knows starting from what ID it needs to process notifications, but that presumes that ID will be sequential. Which is controlled by A. Even if B defines its own sequence A still has to be altered to return it back.
Are there any other approaches?
Thanks!
Web services in general are unreliable enough that it's rarely a good idea to make a "read" request serve double-duty as a "delete" request, especially without the client's knowledge. There is just too much risk of a connection dropping or timing out. There is no way to get around this only by modifying the client, because it's the server that is at fault here - the way it's designed is fundamentally unsuited for a web service.
I think you're on the right track with the incrementing IDs idea. The client knows (or can be modified to know) which notifications it's received, so if it can supply the ID of the last message it's received when it polls for notifications, the server should be able to respond based on that ID.
It really seems like Company A's webservice should be synchronous instead of asynchronous. If that is not possible, it may be a good idea to send a "ACK"-like response to a new Company A webservice that indicates a specific notification was received (by Company B) and can be deleted.