I'm creating an agent with more than 2,000 intents. Basically a FAQ bot that can answer thousands of questions.
In Dialogflow ES, there was the concept of MegaAgents and SubAgent. The maximum number of intent for a SubAgent was 2,000. Using a MegaAgent, I could put together up to 20 Sub Agents (total 20,000 intents). When the user queries the bot, each intent is weighted differently.
Now in Dialogflow CX, although the entire agent has a maximum 10,000 intents, each flow has a 2,000 limit. I can't figure out a way to design the bot to have multiple "parallel" flows where there is equal weighting given to each flow.
The only way is string together multiple flows in sequence, and use a Fallback to transition from one flow to another. However, this doesn't put equal query weighting on my many thousand intents.
Any suggestions?
Related
I am doing the processDocument process using the expense parser as in the example here. Since the billing costs too much, instead of sending the documents one by one, i combine 10 documents into one pdf and use processDocument again. However, DocumentAI sees 10 separate receipts that we have combined as a single receipt, and instead of returning 10 different total_amount entities for each receipt, 1 total_amount returns.I want to combine 10 documents into one pdf and send it for less billing cost. In addition, i am looking for a way to think of each document independently from each other and extract its entities separately. Will batch processing work for me? What can I do for it? Can you help me please?
Unfortunately there is no way to make the billing cheaper because the pricing of Document AI is calculated on a per page/document basis. See Document AI pricing.
With regards to your question:
I am looking for a way to think of each document independently from
each other and extract its entities separately. Will batch processing
work for me?
Yes batch processing will work for you, but pricing is just the same with processDocument. See the pricing info I have attached above.
The only difference between batch processing and processDocument is that instead of sending a single request for a single document, batch processing will send all your documents in a single request. The response will then be stored in a GCS bucket that you have defined on the batch process options. See batch process sample code.
Another thing to add is batch processing process the documents asynchronously. This means that when the request is sent, the processing is done on the backend and you can poll the status of your request to see if it is still processing or it is done.
I have an API that returns most_similar_approx from a magnitude model . The model is built from native Word2Vec format with 50 dimensions and 50 trees. The magnitude model is close to 350MB, with approximately 350000 tokens.
Load testing this API I observed that the performance deteriorates as I increase the topn value for most_similar_approx, I need a high number of similar tokens for downstream activities,
with topn=150 I get a throughput of 500 transactions per second on the API,
while gradually reducing it I get 800 transactions with topn=50 and and ~1300 with topn=10.
The server instance is not under any memory/cpu load, am using a c5.xlarge AWS EC2 instance.
Is there anyway I can tune the model to improve the performance for a high topn value?
My aim is to obtain most_similar tokens from word embeddings, and pymagnitude was the most recommended option I found, are there any similar high performing alternatives.
I use Lambda to read from a JSON Api and write in DynamoDB via http request. The JSON Api is very big (has 200k objects) and my function is extremely slow with writing to DynamoDB. I used the regular write function and after 10 min I could only populate 5k rows in my DynamoDB table. I was thinking about using BatchWriteItem but since it can only do 25 puts in one batch, it would still take too much time to write all 200k rows. Is there any better solution?
This will be because you're being throttled.
For Lambda
There are a maximum number of concurrent invocations of Lambdas that can be running at a time, the default limit is 1000 concurrent requests.
If you have more than 1000 concurrent requests at the same time you will need to reach out to AWS Support to increase this, you will also need to provide a business use case for why it needs to support this.
For DynamoDB
Whether you use batch or single PutItem your DynamoDB table is configured with a number of WCU (Write Credit Units) and RCU (Read Credit Units).
A single write credit unit covers 1 write of an item 1Kb or less (every extra kb is another unit). If you exceed this you will start to be throttled for write requests, if you're using the SDK it may use exponential backoff as well to keep attempting to write.
As a solution for this you should do one of the following:
If this is a one time process you can adjust the WCU as a fixed number, then wait 5 minutes for it to increase and then scale down.
If this is a natural flow on your app then enable DynamoDB autoscaling to naturally increase and decrease throughout the day
In addition look at your data modelling as this can lead to throttling too.
In extreme cases, throttling can occur if a single partition receives more than 3,000 RCUs or 1,000 WCUs
Basically, I have a web service that receives a small json payload (an event) a few times per minute, say 60. This event must be sent to an SQS queue only after 1 year has elapsed (it's ok to have it happen a few hours sooner or later, but the day of month should be exactly the same).
This means I'll have to store more than 31 million events somewhere before the first one should be sent to the SQS queue.
I thought about using SQS message timers, but they have a limit of only 15 minutes, and as pointed out by #Charlie Fish, it's weird to have an element lurking around on a queue for such a long time.
A better possibility could be to schedule a lambda function using a Cron expression for each event (I could end up with millions or billions of scheduled lambda functions in a year, if I don't hit an AWS limit well before that).
Or I could store these events on DynamoDB or RDS.
What would be the recommended / most cost-effective way to handle this using AWS services? Scheduled lambda functions? DynamoDB? PostgreSQL on RDS? Or something entirely different?
And what if I have 31 billion events per year instead of 31 million?
I cannot afford to loose ANY of those events.
DynamoDB is a reasonable option, as is RDS - SQS for long term storage is not a good choice. However - if you want to keep your costs down, I may suggest another: accumulate the events for a single 24 hour period (or a smaller interval if that is desirable), and write that set of data out as an S3 object instead of keeping it in DynamoDB. You could employ dynamodb or rds (or just about anything else) as a place to accumulate events for the day (or hour) before it then writes out that data to S3 as a single set of data for the interval.
Each S3 object could be named appropriately, either indicating the date/time it was created, or the data/time it needs to be used, i.e. 20190317-1400 to indicate that on March 17th, 2019 at 2PM this file needs to be used.
I would imagine a lambda function, called by a cloudwatch event that is triggered every 60 minutes, scans your s3 bucket looking for files that are due to be used, and it then reads in the json data and puts them into an SQS queue for further processing and moves the processed s3 object to another 'already processed' bucket
Your storage costs would be minimal (especially if you batch them up by day or hour), S3 has 11 9's of durability, and you can archive older events off to Glacier if you want to keep them around even after the are processed.
DynamoDB is a great product, it provides redundant storage, and super high performance - but I see nothing in your requirements to that would warrant incurring that cost or requiring the performance of DynamoDB; and why keep millions of records of data in a 'always on' database when you know in advance that you don't need to use or see the records until a year from now.
I mean you could store some form of data in DynamoDB, and run some daily Lambda task to query for all the items that are greater than a year old, remove those from DynamoDB and import it into SQS.
As you mentioned SQS doesn't have this functionality built in. So you need to store the data using some other technology. DynamoDB seems like a responsible choice based on what you have mentioned above.
Of course you also have to think about if doing a cron task once per day is sufficient for your task. Do you need it to be exactly after 1 year? Is it acceptable to have it be one year and a few days? Or one year and a few weeks? What is the window that is acceptable for importing into SQS?
Finally, the other question you have to think about is if SQS is even reasonable for your application. Having a queue that has a 1 year delay seems kinda strange. I could be wrong, but you might want to consider something besides SQS because SQS is meant for much more instantaneous tasks. See the examples on this page (Decouple live user requests from intensive background work: let users upload media while resizing or encoding it, Allocate tasks to multiple worker nodes: process a high number of credit card validation requests, etc.). None of those examples are really meant for a year of wait time before executing. At the end of the day it depends on your use case, but off the top of my head I can't think of a situation that makes sense for delaying entry into an SQS queue for a year. There seem to be much better ways to handle this, but again I don't know your specific use case.
EDIT another question is if your data is consistent? Is the amount of data you need to store consistent? How about the format? What about the number of events per second? You mention that you don’t want to lose any data. For sure build in error handling and backup systems. But for DynamoDB it doesn’t scale the best if one moment you store 5 items then the next moment you want to store 5 million items. If you set your capacity to account for 5 million then it is fine. But the question is will the amount of data and frequency be consistent or not?
I know that DynamoDB is bound to a writes and read per second limit, which I set. This means that when I delete items they are bound to the same limits. I want to be able to delete many records at some point in time, without that having a negative effect on the other operations that my app is doing.
So for example, if I run a script to delete 10,000 items and it takes 1 minutes, I don't want my database to stop serving other users that are using my app. Is there a way to separate the two, one for background processes (admin) and give it its own limits and one for the main process (the app)?
Note: The item deletion will be by date ranges, and I have no way in knowing how much items are there ahead of time.
App in ASP.NET C#
Thanks
The limits are set on the DynamoDB tables themselves, not on the client requests, so the answer is no.
One workaround is to write a script that:
increases the write ops limit
runs the delete queries in a throttled manner so that it uses only the offset between the old limit and the newly set one
decreases the limit back, after the operations are completed.
You could then optimise the amount by which you scale up the writes/second to balance the time it takes for the script to complete.