I'm working on an Ad-tech system which serves millions of users.
Basically users (non anonymous users) can see different Ads that are being created by the marketing team.
Our marketing team want to be able to set some Frequency caps on those Ads (among other targeting rules they already have)
For example:
"We should not show this ad for a user if he already seen/click this ad more than X times in the last Y days"
Also ads can be grouped to campaigns, so rules like that are also possibile:
"We should not show this for a user if he viewed more than X times ads in this campaign in the last Y days".
Also our marketing might wanna know how many people viewed/click a specific add in the last Y days.
We have roughly 200K RPM and our responses should be very fast.
The smallest unit of time for our queries is one day and it will not change.
Few questions and thoughts:
Is DynamoDB a good fit?
I thought about creating a table for each event type (Click/View/Close..)
What is the best way to configure the primary key?
I thought about settings the primary key as the user id and the sort key as a combination of the ad id and the current day {dd/mm/yyyy}
I thought about use "ADD" operation to increase the counter when a user click/view/.. an Ad in a specific date. are they expensive operations? do I have an alternative?
What is the best way I can use to also be able to query per ad and campaigns as well (for example: "all users views for all ads in campaign" or "get all ad views in the last 40 days) ) ?
What other considerations should I take in mind?
Thanks a lot
We have been trying to integrate authorize.net payment gateway in one of our clients project based on Asp.net web API. We have few queries that we came across while implementing Recurring Planning scenarios.
Query 1
We checked the API’s for Creating Subscription, Getting Subscription, Updating Subscription. However once we have created subscription, is there any way we can update the amount in the subscription.
Let’s say for example.
We have a created a subscription for our user for 50$ amount on 01st Jan 2021 with 30 days interval.
And on 15th Jan 2021, our user wishes to purchase 1 more license which will cost him 10$ more.
Hence can we increase his billing cycle of subscription by updating the subscription?
We checked in Update Subscription API, & it is only allowing to update credit card info hence is there any way to update amount.
Query 2
Is there any way to implement Autorenewal, hence when a user wishes he/she can set auto renewal on/off for recurring billing.
Query 3
If there is any way to switch off auto renewal of recurring billing, then is there any link that we can generate & send them through which they can pay there next due.
Query 1: You cannot update a subscription amount. If the amount needs to change you either need to cancel the current subscription and create a new one for the new amount (being sure to prorate credit from the previous subscription payment) or use CIM to manage your subscription service which allows you to charge against their card at your discretion but requires you to also manage the subscription yourself.
Query 2: Not through Authorize.Net. If you want a subscription to start or end you need to explicitly do so through their API.
Query 3: Not through Authorize.Net. That application logic and, once again, you would be responsible for managing.
I'm assuming you are using or are aware of the API provided for Authorize.net here: https://github.com/AuthorizeNet/sdk-dotnet/tree/master/Authorize.NET/Api/Controllers
Query 1: As of now, there is a way to update the amount for a given subscription. You can use ARBSubscriptionType class. There is an amount property there you can set. Then you can create the request ARBUpdateSubscriptionRequest, passing in the ARBSubscriptionType class and the subscription Id.
Note: You might have to handle pro-rating.
Query 2: There isn't a built in renewal feature in Authorize.Net as far as I know. It seems like you could potentially update the totalOccurrences by some amount to act as a "renewal", when technically its an extension of the subscription. The method in which you check when to update, either a Modulo operation or a date check is up to you. You can use paymentScheduleType class to update totalOccurrences, passing it along to a ARBUpdateSubscriptionRequest.
Query 3: Authorize.Net does not have any in house link generation.
Let's say I'm creating a PWA (Progressive Web App) where products can be added by users.
Prices of these products are variable from 0,01 EUR to 1,00 EUR.
I'm using Stripe for payments.
The Stripe Order object do not support dynamic price, passed on the fly, without any reference (kind of foreign key).
To accept the Order, Stripe needs a reference to a SKU.
This SKU will be, in my case, a variation of the price, on the product.
It means that, to cover all variations, I need 100 SKUs, from 1 (0.01 EUR) to 100 (1,00 EUR).
So, for each product created in Stripe, I need to create 100 SKUs in Stripe.
I tried to insert a test dataset of 200 products, which means (200 products + (200 x 100 SKUs)) = 20200 requests.
I got a surprising "Request rate limit exceeded" error from Stripe.
Less than half of records where created... :(
That "Request rate limit exceeded" is the core of the problem.
Right now, the insertion process is the following (x 200):
Create product in Firestore.
Firebase cloud function listener :
OMG new product inserted in Firestore. Ok let's :
Import official nodejs Stripe & Algolia libraries
Create product in Stripe to make it billable
Create the 100 SKU related to the product in Stripe, with Promise.all (This is where, at some point, I end up with a rate limit error, because my concurrent cloud functions instances are using the same Stripe key, which means the same Stripe account)
Create product in Algolia to make it searchable
I need solutions to counter this Stripe API rate limit error.
I have several solutions in mind :
Solution 1 :
Be able to increase Stripe rate API limit for a given amount of time.
Not sure this is possible.
Solution 2 :
Be able to use differents Stripe keys, then rotate over them, to perform admin stuff, such inserting multiple products/SKUs in Stripe.
Ultimately on production, be able to create programmatically 1 Stripe key per user, so each user would have its own limit.
Not sure this is possible.
Solution 3 :
Slow down insertion process in javascript.
Don't know how to perform that.
Besides, Cloud functions have a budget/limit of 60 seconds for javascript execution. So I can't delay too much.
Solution 4 :
Delay work using Pub/Sub (?), or Firestore Triggers
For example, having an integer in Firestore, that each function call increments, and same function listen the write to re-increment he number, etc, etc, etc, until the number equals 100 for the 100th SKU. That solution would sequentialize the 100 SKUs writes in Stripe.
Not sure this will really slow down enough the work to be under the API rate limit. In addition, such a solution would cost lots of money : 100+ Firestore writes, and 100+ functions calls to perform these writes, for only one product, which means 20000+/20000+ for the 200 products. That would be expensive.
Solution 5 :
Perform Just-In-Time insertions, when user pays.
The server side algorithm, after a Payment Request API call, might look like this :
Create order in Stripe
If error 'No such sku...' catched {
For each SKU { // Ideally filter here SKUs to create (only those in error)
If price not between 1 and 100 {
continue // Bad price, not legit
}
Create SKU in Stripe
If error 'Already exists' {
continue // no creation needed for that SKU
}
If error 'No such product...' catched {
If productId does not exists in Firestore {
continue // Bad productId, not legit
}
Create product in Stripe
}
Create SKU in Stripe
}
}
Create order in Stripe
This last solution could do the job.
But it might comes with some delay for the user when it executes payment, which could increase stress. Plus it might increase Stripe calls during the business hours. Many purchases in same time could lead to a Stripe API rate limit error, especially with well furnished carts (let's say an average of 30 products in the cart, so in worst case 30+ HTTPS calls during payment, times 1000 users = 30000 calls => Stripe error). That problem might decrease over time for a given product, because once a SKU is created it is created definitively. Still, as there would be new products, so products with zero SKU at creation, every day, the problem remains.
What do you think ?
Do you have any other ideas ?
Solution 3 and Solution 5 with some tweaks will work best.
Solution 3: You can limit number of concurrent requests to Stripe using async module's forEachLimit or queue.
Solution 5 : Just in time insertions is also a good option as it won't put much load on Stripe server at same time. Regarding your concern of getting the same error during business hour, it will a very rare case as Stripe APIs are built to perform very well. But if you still have doubt regarding this what you can do is to have a Background process for adding SKUs during non-business hours, which will keep on creating SKUs for you without encountering Stripe API rate limit error.
Solution 6 (Modified Solution 5): Have just in time insertions but also create an extra API request to your server whenever a product is entered in the cart which will then check if the SKU exist in Stripe and if not then create it in the background before cart payment happens.
Solution 6 :
Same idea (JIT), but moving SKU creation from payment time to product selection time. Each time a product is selected, try to create the product and its current SKU (price variation) in Stripe. This way, Stripe calls should be more distributed in the time. Or maybe it will ends with more API calls, as we select products more often than we pay, because users can select & unselect products, so they might end with more products selected during their journey than the sum of products finally being paid in the cart ?
Solution 7 :
Same idea (JIT), but with SKU cached in Algolia or Firebase, so I can perform "does this SKU exist ?" calls without querying Stripe, which should reduces Stripe calls if the existence test is performed before the create call (we do not call Stripe.skus.create() blindly). The drawback is, that Firebase and Algolia are exposed in Front so the SKUs and prices will be too, and this is a potential source of threat, so another index, dedicated and only known by the server, has to be used.
im working on an integration between our company systems and NetSuite using PHP. Before the newest release of the NetSuite platform we were able to partially apply an existing credit memo to an specific pending invoice, but since their last release a couple of weeks ago every time we try to apply the credit to an specific invoice through our middle PHP system, it also applies all the other pending invoices in the account.
Let me specify a little more:
We have customer accounts, every account can have any number of open invoices. Lets say we have 3 invoices in the account 1. The first is for $20, the second for $30 and the last for $25. Then we create a credit memo associated to the account 1 for $75.
Then a week later we want to kill the first invoice of the account 1, the one with the $20 value. Then we command the credit memo to kill that invoice through our system. That would mean the first invoice is cancelled, we used $20 from the $75 we had in the credit and we still have $55 without use. Right?
Well, now for some reason the credit kills all the 3 invoices the account 1 had pending, even though we specify that only the first one should be applied. That consumes all the $75 of the credit memo, which of course messes up our accounting and prevents us from using it later in other invoices as we need it.
Just a few more points:
We are uploading our credit memos with property "autoapply" set on false.
When we want to apply an invoice we set its "apply" property as true and the ones from the other invoices as false, then we make an update request for the credit memo with that info. Thats how we always did it but now for some reason it doesnt works.
Sorry if its a little long, but i wanted to be really specific. If you have had a similar problem or have any insight on the matter i would greatly appreciate any help you can give. Thanks.
One suggestion you can try as I have experienced this as well. I had an issue where the user whose log in I was using for the integration entered a new credit memo, checked the box to auto apply and from then on every CM I imported through web services auto applied and did not respect the Auto Apply setting I submitted.
Log in as the user that you are authenticating with for the integration. Open a new credit memo and see if the Auto Apply check box is checked by default. This is a "sticky" setting in NetSuite. Enter a new CM As the user with the box unchecked and then delete it if you like. When I did this it stopped checking the box automatically and my CM import worked as expected again.
i am about to start work on a group-buying site. it works like this: a phone costs 100USD, but if 50 people buy it together , u get a price of 70USD.
so the website launches this offer, waits for 50 people to deposit money . once the threshold is reached, everyone is sent a coupon and that person can go and get his phone.
the problem.. there will be few offers which will be time-limited. for example, total of 100 mobile phones available only. the first 100 users can only buy it. the offer starts on 5th janissary at 11am.
now, how we can detect any suspicious attempts. like a person writing a script and buying all the 100 units.
i dont think IP based limits is a good enough criteria as REAL BUYERS may access the website using the same IP.
environment is LAMP.
why should you care , the more you sell the more you make money :) if the transaction is made you can enforce that user can't buy with same payment method and credentials twice. And as you said user must deposit money before he can make the purchase. You can ask user credit card data before the sale is opened, that is rather good limit as scammer must use 50 different credit cards