I am building a recommendation service which recommend the items based on use case. For this client needs to call our API
Functionality of API:
Clients call with the list of items required and the use case.
Based on that we will return the exact items.
Stack:
AWS Lambda
Amazon DynamoDB
Problem: How do we handle concurrent fetch requests for the same use case.
Solutions:
Flow using pessimistic locking:
Acquire dbLock on the list of available items for use case.
Remove the items from the original db
Release the lock
This will increase the latency of the API
Flow using Optimistic locking:
Fetch the available items.
Remove from the list and return.
If any other thread tries to delete the items from available list that was already deleted then send an exception to client to call the API again.
Is there any other more efficient way of handling the concurrent requests?
Related
The title is pretty much the question. Is there some way to limit the number of concurrent workflows running at any given time?
Some background:
I'm using eventarc to dispatch a workflow once a message has been sent to a pubsub topic. The workflow will be used to start some long-running operation (LRO) but for reasons I won't go into, I don't want more than 3 instances of this workflow running at a given time.
Is there some way to do this? - primarily from some type of configuration rather than using another compute resource.
There is no configuration to limit running processes that specifically targets sessions that are executed by a Workflow enabled for concurrent execution.
The existing process limit applies to all sessions without differentiating between those from non-concurrent or concurrent enabled Workflows.
Synchronization enables users to limit the parallel execution of certain workflows or templates within a workflow without having to restrict others.
Users can create multiple synchronization configurations in the ConfigMap that can be referred to from a workflow or template within a workflow. Alternatively, users can configure a mutex to prevent concurrent execution of templates or workflows using the same mutex.
Refer to this link for more information.
Summarizing your requirements:
Trigger workflow executions with Pub/Sub messages
Execute at most 3 workflow executions concurrently
Queue up waiting Pub/Sub messages
(Unspecified) Do you need messages processed in the order delivered?
There is no out-of-the box capability to achieve this. For fun, below is a solution that doesn't need secondary compute (and therefore is still fully managed).
The key to making this work is likely starting new executions for every message, but waiting in that execution if needed. Workflows does not provide a global concurrency construct, so you'll need to use some external storage, such as Firestore. An algorithm like this could work:
Create a callback
Push the callback into a FIFO queue
Atomically increment a counter (which returns the new value)
If the returned value is <= 3, pop the last callback and call it
Wait on the callback
-- MAIN WORKFLOW HERE --
Atomically decrement the counter
If the returned value is < 3, pop the last callback and call it
To keep things cleaner, you could put the above steps in a the triggered workflow and the main logic in a separate workflow that is called as needed.
We have a synchronous REST endpoint that does other processing apart from saving item to DynamoDB database which will be used for later purpose.
The requirement is to not error out if the database save fails due to any type of exception.
How do we handle the case where dynamo db is down in the entire region(rare but possible).Is it the right pattern to publish to SQS and have a seperate process consume and save to DynamoDB by pinging it(ListTables or ping).
Should we fallback to another region or publish to SQS? Is it worth using resilience4j circuit breaker pattern?
It is a common pattern to have the API simple enqueue a request to SQS. This has many benefits such as allowing higher throughput, decoupling the producer and consumer and better fault tolerance.
This would be a fine design but your REST API will no longer be synchronous and the caller won't quite know whether the operation was successfully processed so you may need to add another endpoint to get the status of the request.
I am not super familiar with resilence4j circuit breakers but this may not be necessary as the Amazon SDKs already have built in retries if that is the main benefit you are seeking.
I'm using dfuse streaming API to built a EOS based application.
I want to keep an in-sync view of all the data flowing through my smart contract.
I want to ensure that I will always be in sync so I never miss a single block. Using the dfuse streaming API, how can I achieve that?
Using the with_progress feature of the Websocket API, you will receive one progress message after each block was processed.
For a given stream (corresponding to a request and an associated req_id), when you received a progress for a block, you are guaranteed to have seen all the contents it contained in your query (the actions for get_action_traces, or the rows for get_table_rows).
If you keep track of that block_num and/or block_id, upon disconnection/reconnections, you can provide it back in your request, and be guaranteed never to miss a beat, even if that means you're reprocessing 1M blocks.
As of November 22nd 2018, the get_table_rows request will stream table_delta messages that include an "undo"/"redo" step. This allows your app to navigate forks and ensure absolute sync of your application with the longest chain's state.
Check the docs at https://docs.dfuse.io/ and search for with_progress for more details.
We've created a Google Cloud Function that is essentially an internal API. Is there any way that other internal Google Cloud Functions can talk to the API function without exposing a HTTP endpoint for that function?
We've looked at PubSub but as far as we can see, you can send a request (per say!) but you can't receive a response.
Ideally, we don't want to expose a HTTP endpoint due to the extra security ramifications and we are trying to follow a microservice approach so every function is its own entity.
I sympathize with your microservices approach and trying to keep your services independent. You can accomplish this without opening all your functions to HTTP. Chris Richardson describes a similar case on his excellent website microservices.io:
You have applied the Database per Service pattern. Each service has
its own database. Some business transactions, however, span multiple
services so you need a mechanism to ensure data consistency across
services. For example, lets imagine that you are building an e-commerce store
where customers have a credit limit. The application must ensure that
a new order will not exceed the customer’s credit limit. Since Orders
and Customers are in different databases the application cannot simply
use a local ACID transaction.
He then goes on:
An e-commerce application that uses this approach would create an
order using a choreography-based saga that consists of the following
steps:
The Order Service creates an Order in a pending state and publishes an OrderCreated event.
The Customer Service receives the event attempts to reserve credit for that Order. It publishes either a Credit Reserved event or a
CreditLimitExceeded event.
The Order Service receives the event and changes the state of the order to either approved or cancelled.
Basically, instead of a direct function call that returns a value synchronously, the first microservice sends an asynchronous "request event" to the second microservice which issues a "response event" that the first service picks up. You would use Cloud PubSub to send and receive the messages.
You can read more about this under the Saga pattern on his website.
The most straightforward thing to do is wrap your API up into a regular function or object, and deploy that extra code along with each function that needs to use it. You may even wish to fully modularize the code, as you would expect from an npm module.
I am working on a mobile app that will broadcast a push message to hundreds of thousands of devices at a time. When each user opens their app from the push message, the app will hit our API for data. The API resource will be identical for each user of this push.
Now let's assume that all 500,000 users open their app at the same time. API Gateway will get 500,000 identical calls.
Because all 500,000 nearly concurrent requests are asking for the same data, I want to cache it. But keep in mind that it takes about 2 seconds to compute the requested value.
What I want to happen
I want API Gateway to see that the data is not in the cache, let the first call through to my backend service while the other requests are held in queue, populate the cache from the first call, and then respond to the other 499,999 requests using the cached data.
What is (seems to be) happening
API Gateway, seeing that there is no cached value, is sending every one of the 500,000 requests to the backend service! So I will be recomputing the value with some complex db query way more times than resources will allow. This happens because the last call comes into API Gateway before the first call has populated the cache.
Is there any way I can get this behavior?
I know that based on my example that perhaps I could prime the cache by invoking the API call myself just before broadcasting the bulk push job, but the actual use-case is slightly more complicated than my simplified example. But rest assured, solving this simplified use-case will solve what I am trying to do.
If you anticipate that kind of burst concurrency, priming the cache yourself is certainly the best option. Have you also considered adding throttling to the stage/method to protect your backend from a large surge in traffic? Clients could be instructed to retry on throttles and they would eventually get a response.
I'll bring your feedback and proposed solution to the team and put it on our backlog.