I have a balance stored on a server in a database. I would like to have two api's
addToBalance(amountToAdd)
subtractFromBalance(amountToSubtract)
My problem is as follows If a user is on a mobile and calls one of these then loses connection we have no way of telling if the balance was updated. So we do not know whether or not to retry.
A solution I came up with was to give the current balance as the client knows it to verify whether there has already been an update, the update should fail if the balance provided does not match the actual current balance. The problem with this solution is, if the balance is updated from elsewhere then the call will fail. We will not know whether to differentiate between the two failures so still do not know whether or not to retry.
Does anyone have a possible solution to ensure that the same call does not get made multiple times to avoid multiple additions and subtractions?
Thanks,
Andy.
That's a common problem, and HTTP itself has the solution for you, which is to make the request conditional.
When the user retrieves that resource with a GET, you should return the Last-Modified header with a timestamp of the last time the resource was changed. Then you require the user to send the If-Unmodified-Since header with that value when changing it.
If someone else changed it in between, the resource timestamp will be greater than the value sent in the If-Unmodified-Since, and you should fail the request with 412 Precondition Failed. If the user failed to include the If-Unmodified-Since header, you should fail with 428 Precondition Required
Related
I'm making a website right now and need to use django-tracking2 for analytics. Everything works but I would like to allow users to opt out and I haven't seen any options for that. I was thinking modifying the middleware portion may work but honestly, I don't know how to go about that yet since I haven't written middleware before.
I tried writing a script to check a cookie called no_track and if it wasn't set, I would set it to false for default tracking and if they reject, it sets no_track to True but I had no idea where to implement it (other than the middle ware, when I tried that the server told me to contact the administrator). I was thinking maybe I could use signals to prevent the user being tracked but then that would slow down the webpage since it would have to deal with preventing a new Visitor instance on each page (because it would likely keep making new instances since it would seem like a new user). Could I subclass the Visitor class and modify __init__ to do a check for the cookie and either let it save or don't.
Thanks for any answers, if I find a solution I'll edit the post or post and accept the answer just in case someone else needs this.
I made a function in my tools file (holds all functions used throughout the project to make my life easier) to get and set a session key. Inside the VisitorTrackingMiddleware I used the function _should_track() and placed a check that looks for the session key (after _should_track() checks that sessions is installed and before all other checks), with the check_session() function in my tools file, if it doesn't exist, the function creates it with the default of True (Track the user until they accept or reject) and returns an HttpResponse (left over from trying the cookie method).
When I used the cookie method, the firefox console said the cookie will expire so I just switched to sessions another reason is that django-tracking2 runs on it.
It seems to work very well and it didn't have a very large impact on load times, every time a request is made, that function runs and my debug tells me if it's tracking me or not and all the buttons work through AJAX. I want to run some tests to see if this does indeed work and if so, maybe I'll submit a pull request to django-tracking2 just in case someone else wants to use it.
A Big advantage to this is that you can allow users to change their minds if they want or you can reprompt at user sign up depending on if they accepted or not. with the way check_session() is set up, I can use it in template tags and class methods as well.
I have an application developed using SmartGWT,Jaxrs,ejb &jpa.
I have one scenario where user wants to extract the data(called Search Screen) by entering either firstname,lastname or middlebane,ssn,email,etc
Database contains the huge number of records in millions, which takes lot of time to respond back.
for example, user search with firstname which takes lot of time to respond, in that case user wants to cancel/terminate/abort the request.
Is it possible either in smartgwt or jaxrs(web api) to terminate the request.
So that user can terminate the request and move further
PS:: i tried lot of option,but i didn't get the proper solution.
One solution is to put the business logic in stateful bean and put the bean in the http session ... now you have access to the currently used persistence context and the open transaction so you can call Session.cancelQuery() .... but this method has some limitation .. It works only if Result set is not yet returned , if this limitation harms you, check this answer please
There are other workarounds to synchronize the web client with the business method but this is the one I like most
One more thing you need to consider as this is your use case is to introduce a new lexical search engine like solr or elasticsearch which can be updated frequently with data from the database ... It fits perfectly in lexical search, gives the ability to stand typo mistakes and returns result very quickly
I am trying to implement an Event Sourcing system using Kafka and have run into the following issue. During a new user sign-up I want to check if the username the user provided is already taken. However, consider the case where 2 users are trying to sign-up at the same time providing the same username.
In my understanding of how ES works the controller that processes the sign-up request will check if the request is valid, it will then send a new event (e.g. NewUser) to Kafka, and finally that event will be picked up by another controller which will persist it in a materialized view (e.g. Postgres DB). The problem is that the validation of the request is done against the materialized view but the actual persistence to it happens later. So because the 2 requests are being processed in parallel (by different service instances) they might both pass the validation, resulting in 2 NewUser messages. However, when the second controller tries to persist those 2 NewUser messages in the database saving the second event will fail because of the violation of the uniqueness constraint for the username.
Any ideas on how to address this?
Thanks.
UPDATE:
In particular, I would like to verify whether the following are accepted approaches to the problem:
use the username as the userId (restrictive)
send an event to a topic partitioned by username and when validation
is done send an event to another topic
Initial validation against the materialized view won't be enough in most scenarios where you have constraints. There can always be some relevant events haven't been materialized yet. There are two main concurrency control approaches to ensure that correct results are generated:
1. Pessimistic approach:
If you want to validate constraints before you publish an event, you need to lock relevant resources (entity, aggregate or data set). The locking means your services must not be able to publish events on these resources. After this point, to get the current state of your data:
You can wait until all events published before locking are materialized.
You can read current state from the database and apply events on it in a separate process.
2. Optimistic approach:
In this approach, you perform your validations after publishing events. To achieve this, you need to implement a feedback mechanism. The process which consumes events and performs validations should be able to publish validation results. You can perform the validations in-memory when possible. Otherwise, you can rely on your materialized data store.
Martin Kleppman talks about a two-step solution for exactly the same problem here and in his book. In this solution, there are two topics: "claims" and "registrations". First, you publish a claim to take the username, then try to write it to the database, and finally publish the result to the registrations topic. At conceptual level, it follows the same steps in the second approach you have mentioned. In validation step, it avoids implementing validation logic and keeping secondary indexes in memory by relying on the database.
During a new user sign-up I want to check if the username the user provided is already taken.
You may want to review Greg Young's essay on Set Validation.
In my understanding of how ES works the controller that processes the sign-up request will check if the request is valid, it will then send a new event (e.g. NewUser) to Kafka, and finally that event will be picked up by another controller which will persist it in a materialized view (e.g. Postgres DB).
That's a little bit different from the usual arrangement. (You may also want to review Greg's talk on polyglot data.)
Suppose we begin with two writers; that's fine, but if there is going to be a single point of truth, then you are going to need synchronization somewhere.
The usual arrangement is to use a form of optimistic concurrency; when processing a request, you reserve a copy of your original state, then you do your calculation, and finally you send the book of record a `replace(originalState,newState)'.
So at this point, we have two writes racing toward the book of record
replace(red,green)
replace(red,blue)
At the book of record, the writes are processed in series.
[...,replace(red,blue)...,replace(red,green)]
So when the book of record processes replace(red,blue), it performs a check that yes, the state is currently red, and swaps in blue. Later, when the book of record tries to process replace(red,green), the book of record performs the check, which fails because the state is no longer red.
So one of the writes has succeeded, and the other fails; the latter can propagate the failure outwards, or retry, or..., precisely what depends on the specific mechanics in question. A retry should mean, of course, reload the "original state", at which point the model would discover that some previous edit already claimed the username.
Any ideas on how to address this?
Single writer per stream makes the rest of the problem pretty simple, by eliminating the ambiguity introduced by having multiple in memory copies of the model.
Multiple writers using a synchronous write to the durable store is probably the most common design. It requires an event store that understands the idea of writing to a specific location in a stream -- aka "expected version".
You can perform an asynchronous write, and then start doing other work until you get an acknowledgement that the write succeeded (or not, or until you time out, or)....
There's no magic -- if you want uniqueness (or any other sort of invariant enforcement, for that matter), then everybody needs to agree on a single authority, and anybody else who wants to propose a change won't know if it has been accepted without getting word back from the authority, and needs to be prepared for a rejected proposal.
(Note: this shouldn't be a surprise -- if you were using a traditional design with current state stored in a RDBMS, then your authority would be a user table in the database, with a uniqueness constraint on the username column, and the race would be between the two insert statements trying to finish their transaction first....)
First, I understand GET requests should be safe and idempotent. However, my current situation is a little bit different from all the examples I have seen, so I'm not sure what to do.
The web app is some kind of metadata database for all online videos (by "all" I actually mean "all YouTube, Vimeo, XXX, ...", i.e., a known range of mainstream online video websites). Users can POST to http://www.example.com/api/video/:id to add metadata to a certain video, and GET from http://www.example.com/api/video/:id to get back all the current metadata for the given video.
The problem is how to get the video ID for a URL (say https://youtu.be/foobarqwe12). I think the users can query the server somehow, perhaps with a GET at http://www.example.com/api/find_video?url=xxx. The idea is that as long as the URL is valid, the query should always return the information of the video (including its ID); this seems to require that the server creates the record for a video if it doesn't exist yet.
My opinion is that although this seems to violate the safety and idempotence requirements for GET requests, it can also be seen as implementation detail (ideally there is a record for every video for every URL at the beginning of time, and lazily creating records on GETs is just a kind of optimization).
Nonsense, it doesn't violate anything.
If "every valid resource name" has a "valid representation", how that representation is manifested is an internal detail that's outside scope.
Your GET is idempotent. Just because you create a new row in a DB on first access doesn't make it not so.
When you GET /missingurl, you get a representation -- not a 404, but a 200 and some kind of result. This representation could also just be a templated boilerplate that all entities get (only with the URL linked filled in).
Whether you simply print some templated boilerplate, or create a row in the DB, the representation to the client is the same. They make the request, they get the representation -- all the time, all the same. That's idempotent. The fact "something happens" on the backend in an implementation detail hidden from the client.
I'm looking for a way how to deal with a following problem:
Imagine you modify a resource and that subsequently causes update of other resources.
E.g. you issue a PUT to, say /api/orders/1234, which by definition changes state of all other Orders of given user. There may be UI clients that display the table of Orders and they should know that not only single item in the table was updated, but eventually other as well.
Now, is there any standard way how inform a clients about such a situation?
So far I can only think of sending back the 205 Reset Content HTTP status code to inform the client that he should refresh the state, as not just a single thing was changed.
There are multiple solutions.
You can define specific resources as non-cacheable, so the client does not cache them at all. (no-store)
You can try giving a max-age of 0, so the client will have to re-validate those resources always. In this case you might have to implement ETags and conditional GETs, but it will be easier on the server than option 1.
Some push method like WebSockets.
If you really want to "notify" potentially multiple clients of a change, then it sounds like you would need option 3.
However, correctly configured caching is normally good enough. For example you could mark not-yet-executed orders as not cached (max-age=0), but as soon as it is executed, you might mark it to be cached indefinitely, since it can not change anymore.