In the Authorize.net API, when getSettledBatchList returns a settlementState of settlementError, is that the final state for the batch? What should I expect to happen to the batched transactions?
Is the same batch processed again the following day, using the same batch id, possibly resulting in a settlementState of settledSuccessfully? Or are the affected transactions automatically included in a new batch with a new batch id?
If the transactions are included in a new batch, would they then be included in multiple batches? If transactions are included in multiple batches, would getTransactionList for each of these batches return the exact same transactionStatus for transactions that were included in multiple batches, regardless of which batch id was used to make the getTransactionList request?
Question was originally asked at https://community.developer.authorize.net/t5/Integration-and-Testing/What-happens-to-a-batch-having-a-settlementState-of/td-p/58993. If the question is answered there, I'll also add the answer here.
Here's the answer posted in the Authorize.Net community for those who did not follow the link in the question:
Batch status of "settlement error" means that the batch failed. There are different reasons a batch could fail depending on the processor the merchant is using and different causes of failure. A failed batch needs to be reset and this means that the merchant will need to contact Authorize.Net to request for a batch reset. It is important to note that batches over 30 days old cannot be reset. When resetting a batch, merchant needs to confirm first with their MSP (Merchant Service Provider) that the batch was not funded, and the error that failed the batch has been fixed, before submitting a ticket for the batch to be reset.
Resetting a batch doesn't really modify the batch, what it does, is it takes the transactions from the batch and puts them back into unsetttled so they settle with the next batch. Those transactions that were in the failed batch will still have the original submit date.
Authorize.net just sends the batch to your msp, you'll have to contact your msp to have them three way call authorize.net to sort it out.
Related
I'm trying to establish email alerts at a project level to send email alerts for when a certain number of query/job concurrency is reached e.g. 5 concurrent queries. We have a flat-rate pricing model.
I want to do a similar email notification when total slot Usage exceeds a certain threshold as well e.g. slot usage reaching 1000 slots
As a next step I would like to throttle new incoming queries based on the above mentioned thresholds. Meaning if there are already for example 5 queries actively running the 6th one will be put on hold until one of the 5 running earlier have completed.
You may create an Alert Policy in which you can set your desired metric type (eg. slots) and then configure your desired threshold similar to below.
In creating an Alert Policy you may also configure the notification channel to email notification which is also included on the same documentation.
For the available metric types for SLOTS in BigQuery, you may refer to this Google Cloud Metrics for BigQuery documentation.
For your next step, you may code (python, node.js, etc) using BigQuery API to count queries actively running (through JOB ID) and when the count hits 5, you may print "query queue is full" and then wait for the total JOBS to hit below 5 before running the next query. You may refer to this BigQuery Managing Jobs API Documentation.
I am doing the processDocument process using the expense parser as in the example here. Since the billing costs too much, instead of sending the documents one by one, i combine 10 documents into one pdf and use processDocument again. However, DocumentAI sees 10 separate receipts that we have combined as a single receipt, and instead of returning 10 different total_amount entities for each receipt, 1 total_amount returns.I want to combine 10 documents into one pdf and send it for less billing cost. In addition, i am looking for a way to think of each document independently from each other and extract its entities separately. Will batch processing work for me? What can I do for it? Can you help me please?
Unfortunately there is no way to make the billing cheaper because the pricing of Document AI is calculated on a per page/document basis. See Document AI pricing.
With regards to your question:
I am looking for a way to think of each document independently from
each other and extract its entities separately. Will batch processing
work for me?
Yes batch processing will work for you, but pricing is just the same with processDocument. See the pricing info I have attached above.
The only difference between batch processing and processDocument is that instead of sending a single request for a single document, batch processing will send all your documents in a single request. The response will then be stored in a GCS bucket that you have defined on the batch process options. See batch process sample code.
Another thing to add is batch processing process the documents asynchronously. This means that when the request is sent, the processing is done on the backend and you can poll the status of your request to see if it is still processing or it is done.
Problem:
I know that Google Pub/Sub subscription cannot be patched to update the filter. I am just figuring out other ways to handle updates to filter in production.
Approach I could come up with:
Modify the push config to pull in existing subscription (old filter) so that it accumulates messages
Create a new subscription with latest filter
Transfer the messages from old subscription to a topic using dataflow
Detach the old subscription from the topic
Problems I see with the approach:
As both subscriptions exist at a point of time, I could end up processing duplicate messages
Any suggestions on the best way to handle this?
The timing is important to minimize the duplicates or the message loss.
Firstly, I will deploy a service (Cloud Run for exemple) that save the PubSub message as is somewhere (in a Cloud Storage file, in bigQuery, in Firestore,...)
Then, and in the same time, I will change the push of the old subscription to push to my Cloud Run service; and create the new push subscription with the new filter
Finally detach the subscription.
If you have the capacity, in your REAL app to detect the message already processed, you can remove them from your save location (it's easier with BigQuery for example) and then reprocess only the missing messages. With dataflow, or manually
However, it's recommended to have idempotent processing of your message. Keep in mind that PubSub is at least 1 delivery message and even with the same subscription you could have duplicates.
As you note, a filter expression cannot be changed once a subscription has been created. To effectively change a filter, please do the following steps:
Create a snapshot on the existing subscription (subscription A).
Create a new subscription (subscription B) with the new filter expression.
Seek subscription B to the snapshot created in step 1.
Change your subscribers to use subscription B instead of A.
Please note that during the time subscribers are switching from subscription A to B, there will be a high rate of duplicates delivered as messages are acked independently on the two subscriptions.
If you need to minimize the duplicates processed and can afford downtime, you can stop the all subscribing jobs/servers before step 1 and restart them configured to pull from the new subscription, B, after step 4. All steps must be completed well within the configured message_retention_duration to prevent message loss.
I am working on an contact sync solution to be able to keep the contacts in our app in sync with the google contacts of the user.
Our code uses the php library for the google people api latest Version (v1).
Everything is working fine for one week with each user but after that week we get:
400 - Error "Sync token is expired.
Clear local cache and retry call without the sync token".
My question now:
Is this intended behaviour that you have to clear all your cache after one week with no changes or am I doing something wrong?
Is there any possibility to renew a syncToken if there were no changes?
I already checked the whole code to be sure that the new received nextSyncToken is saved at our side and used for the next incremental sync request. It seems that the new sync Token is always the same as the one sent in the request. Thus it is just clear that we get that errors if a sync token expires after exactly one week.
I also tried to set the option requestSyncToken to true for every list request, even if also a syncToken is set. No success. Sync token stays the same after each request with no changes.
Just in case someone is also facing this problem (syncToken expiration after one week without changes in the persons/contacts list):
Our solution was:
Save the creation date and time of a new syncToken each time you
get one together with the syncToken.
When you receive a syncToken in an incremental sync process compare that token to the stored one. If the syncToken is a new
one, overwrite the old one and its creation date/time.
Use a continuous process that checks each syncToken. If one is about one week old (for security reasons we used 6 days) create a new
syncToken (process see below). As the people API does not offer
things like the watch-channels of the calendar API you would anyway
need some continuous processes that do list-calls in fixed time
intervals for a complete real-time synchronization - so maybe you
could combine these tasks depending on your solution for this
problem?!
Process for creation of a new SyncToken:
Do a new list request without providing a syncToken.
For additional security do some checkups like compare the total persons received with the total persons expected by the old/current
data. And do this renew process at a time of the day when almost no
one does changes generally, for example like 2am.
Overwrite the old syncToken and date/time with the new one and the current date/time.
That's it.
But attention! You can still miss some changes that were made if your syncToken renew process is running exactly at the time a change is made!
Create a dummy contact before the sync to get a new syncToken. After the sync delete the dummy contact from both Google Contacts and your cache.
I am working on Amazon Matillion for Redshift and we have multiple jobs running daily triggered by SQS messages. Now I am checking the possibility of creating a UI dashboard for stakeholders which will monitor live progress of jobs and will show report of previous jobs, like Job name, tables impacted, job status/reason for failure etc. Does Matillion maintain this kind of information implicitly? Or I will have to maintain this information for each job.
Matillion has an API which you can use to obtain details of all task history. Information on the tasks API is here:
https://redshiftsupport.matillion.com/customer/en/portal/articles/2720083-loading-task-information?b_id=8915
You can use this to pull data on either currently running jobs or completed jobs down to component level including name of job, name of component, how long it took to run, whether it ran successfully or not and any applicable error message.
This information can be pulled into a Redshift table using the Matillion API profile which comes built into the product and the API Query component. You could then build your dashboard on top of this table. For further information I suggest you reach out to Matillion via their Support Center.
The API is helpful, but you can only pass a date as a parameter (this is for Matillion for Snowflake, assume it's the same for Redshift). I've requested the ability to pass a datetime so we can run the jobs throughout the day and not pull back the same set of records every time our API call runs.