In developing a new web service I haven't been able to find very much information on how companies bill for their web services.
Do you bill by request or only certain requests ie) GET or POST?
-would these be tracked at the application or server level?
Do you bill by bandwidth?
-again how would this be tracked on a per user basis
Do you charge a subscription to simply have access?
-this is assuming that they are only granted an api key after payment has been made.
A combination of the above or other options?
Thanks for your help.
As all things in a market economy, the price, but also the inconvenience (or convenience) and risk associated with the actual payment (irrespective of the amount) is a function of how unique and cool and valued your service or product is.
It is therefore impossible to answer the question but in very generic terms, i.e. in the form of suggestions. You actual invoicing model may base on one or several of the following
bill for a one-time setup fee
bill on a subscription basis (i.e. for a defined period, with explicitly defined maximum amounts of usage)
bill for maintenance
bill by the act, i.e. a certain amount (possibly on a decreasing unit price schedule). Such acts should be counted at the server level, (The client-side may include some audit/monitoring/log of sorts, but the server-side should be the authoritative source of info)
bill by volume (for example number of MBytes transfered etc.), this is applicable to services where there is a big variation in the volume of info produced for each "act".
In general, the price and the modality of accounting should seem fair, to both parties, particularly to the buyer, and typically, the simpler the better. The price should not necessarily be low, provided you can make the case that the service provided is effectively valuable, and that you either invested and took risk to introduce the service, or the on-going expenses associated with running the service are evident.
I guess It Depends™ on what the service does. Broadly, I'd say you should bill when you provide some intrinsic value; how you determine what that billing criteria is is quite domain-specific. There may be some property of the service provided which allows you to determine how much to bill.
For example, suppose you've a web service that performs a calculation. You might decide that for every successful computation you do, you're going to charge a fixed fee, say $0.01, but let users off if there's a validation problem, such as an invalid request. Alternatively, if those computations are vaguely long-running, you might have a charging model that's based on some sort of CPU-time metric.
Your point about subscriptions is a good one, and this is an area where you might potentially benefit from allowing a couple of commercial models; one to cater for the users who might perform a lot of requests per month, in which case a fixed subscription might make sense, and one to cater for users who make a few ad-hoc requests. In the latter case, of course, if you only attract those customers, then you're not going to make a good return on investment. Some kind of middle ground, whereby you have a small subscription, but then allow customers to buy a "block" or "bundle" of requests on top without incurring additional processing costs, might work.
Most webservices I know of charge for two things:
Volume of "usage". Generally giving low volumes "free" access (i.e., less than X hits/hour from a given IP address account combination). This is similar to say, twitter which gives you 150 hits/hour to its service from either your username, or unique IP or combination of the two (so you dont abuse it by changing IPs frequently). If you want a higher volume you pay for that access and its usually assigned by account (in twitters case you can get a dev account [for free] which gives you 20K or more hits an hour)
Depth of Details, Access to features. Again free accounts get a minimum amount of access, but dont get access to more data or to more advanced features (filtering, etc). Lots of google services work like this, were base access is given to everyone but if you want more refined abilities (greater search, more data, faster results) you have to buy an account code with the corresponding functionality.
I havent really seen or participated in any projects with pay-for-performance, or pay-per-hit/access models as they get very difficult to reliably bill for and very hard to account for to customers, even if you use tiered or banded ranges. How do you tell your customers how many hits they have used, especially in a distributed system, with redundant fail-over, etc. If I had to pay $0.01 cents per access I would want to know exactly how its measured, and what the company had in place to control access, and how accurate their monitoring was, etc.
Its not impossible, and definitely can be done, and may work well in large bulk scenarios.
Many of the ones I have seen bill by time, such as on a monthly or yearly basis. Some allow you to pay by the month, some require some (or all) of the fee up front. Access might be restricted by issuing a security certificate for the web service that expires when the customer's account expires, or possibly by having them send a client ID and letting the server check if that client ID is allowed to have an answer (but that's open to people stealing someone else's client ID ;) ).
I suppose if you have a service that sends and receives very large amounts of data, it might make sense to bill per service request, but the billing for that could get trickier. Are clients likely to make dozens of requests per day, or just a few? How much to bill per transaction? $100? $0.01? That all would depend on the nature of the service. If you want to go that route, you would probably need to be able to ensure that clients only get billed for requests that are successfully answered (I'd hate to get billed even though my client app failed to receive the entire web service message from your server).
Per request or as a subscription, and yes, bandwidth can be a variable that is used to set the fee. Depends of the value of binding the customer close or having a myriad of loosely coupled customers using it. There is no correct answer to the question that fits all or even most cases.
If I look at the services I have made in the past, the subscription model would be the best model to use. Sometime a tick of $ per request seems like the best approach but I have never had a service configured that way yet.
I agree with what has been said by Rob and Des. One thing to remember is that a subscription is a really simple concept that everyone is used to and comfortable with (if you price it right). If you want to cover a wide audience look at how the payment providers do - they have slightly different methods of payment depending on how many transactions you do per year. There'll be a fixed subscription plus a per-transaction charge and they both vary with the number of transactions. This is the most flexible, but it depends if it makes sense for your business.
Related
I work in a company where in order to send staff to the field, employees have to comply with a certification that expires each 4 years, which probably employees did in their previous job, but the previous job just provided a letter that could be falsified, also, others companies don't spend time and resources to share their databases advocating security for the sensitiveness of data (names, nationality, id, company (including governments), date completion), centralization is a risk they don't want to take. As this is a training that is the same, and conducted everywhere that produces a contract between the company and the person, that expires in 4 years, but the person would like to be able to certify itself with the other companies, I was wondering if there is such implementation in the blockchain world, where writers, public ones, with no interest in 3rd parties, but in consensus due to the training nature, are willing to write/read the transaction on a secure manner providing a "self-service" among producers and consumers.
Assuming the other companies want to participate in the scheme, it is perfectly plausible to use some blockchain solution. You might want to use Hyperledger Fabric to create a privately permissioned blockchain network between the relevant stakeholders.
However don't get sucked into the blockchain hype, evaluate objectively whether the rest of the business processes also fit in the paradigm. Can certifications be revoked? Are there regulations on what data you can store on the employees and rights they have to get their data erased?
There might be many other simpler solutions that achieve the same thing. If the company or training provider just has a private key that signs certificates (PDF?) it might already be enough to solve your letter falsification issue.
Maybe each company could host its list of certified trained people on its own repository which could be queried, such that the list of certified people is maintained up to date (and any revocations can also be handled online), would probably achieve exactly the same result. Just like most other blockchain use-cases, there is a simpler solution that achieves the same thing, and is probably more flexible to fit the rest of the business processes.
Finally, before engaging in too much effort, make sure the other third parties are on-board. Do they really want to share the list of people they trained with competitors? If there is a likelihood they will not play along the project will probably be DOA. Case in point what happened here:
https://www.coindesk.com/ibm-blockchain-maersk-shipping-struggling/
In the "Coffee with Blockchain" app produced by IBM it performs price matching between Growers and Buyers. I'm wondering how that matching would be implemented (either in the example app or in an actual implementation).
Example of the app can be seen here: https://www.youtube.com/watch?v=suE5KHkESF4
How does "Coffee with Blockchain" match growers to buyers?
Some questions to help guide the answer:
Does that matching operate entirely as chaincode? If so, how would that be implemented?
Eg, would the Grower submit a transaction that they have a new batch of beans ready for shipment and that triggers the match-making part in the chaincode to find/choose a Buyer?
I have my doubts on making that process deterministic, otherwise a grower's new batch announcement fails and they'd have to retry. Perhaps there's a different way of doing so in chaincode?
Or does the matching operate outside of the chaincode, listening for updates in the ledger, and instead it calls the API to invoke a chaincode method to transfer the beans from Grower-A to Buyer-B?
My doubt on that though is the matching is then centralized to some trusted provider, rather than validated/endorsed chaincode.
Thanks
I can only hypothesise based on my understanding of Blockchain. IBM's algorithms are not public, so I do not have first hand knowledge.
But from what I know - Yes, all your understanding seem accurate.
Unlike crytocurrency, in physical commodity, the process of 'mining' is subject to availability of the underlying asset. Farmers are like miners and they will have to announce the production of the commodity. This 'announcement' would trigger the match-making process where registered 'buyers' will be matched.
However, just like you I have my doubts on it being fully "chain code" based. Reason for that being the underlying commodity - coffee - is also simultaneously being traded on commodity exchanges and if the "chain code" prices drifts too far from the exchange prices then the marketplace ecosystem would revert back to traditional channels outside of blockchain.
So in a way the commodity exchange is still the "trusted provider" of the pricing match (albeit indirectly).
Like you, I am of the opinion that this implementation creates more questions than solutions.
This video tries to explain some details -> Journey of Coffee on Blockchain
In my opinion, it might works in other way round:
When GROWER-A submits a transaction that they have a new batch of beans ready for shipment, then it is added to a list and it will be triggered when a BUYER-B wants to buy some quantity of bean.
Suppose a BUYER-B wants to buy a new batch of beans(some quantity), then it will trigger the API to invoke a chaincode method to transfer the beans from GROWER-A to BUYER-B.
So from this it can be concluded that, there might be a chance of two chain, chain A for GROWER which will be add on block when GROWER-A wants to submit a bean and remove one block when BUYER-B wants to buy it.
And chain B will be for buyer which will work alternatively.
This video explains the example in detail. Hope it answers some of your questions.
Per this explainer video -
1) Buyer brews a cup of coffee, and hence its stock of coffee beans goes down
2) Buyer buys beans to restock
3) Grower ships beans to the buyer
4) Once the Buyer confirms the receipt of the shipment, grower gets paid
The video also explains:
Coffee beans prices may differ based upon the quality and demand. Quality may be determined by popularity or by a certifying body
All the transactions are governed by Smart Contracts. Each participant in this, say Coffee Business, network - the buyer, the grower, the payment company, the bank, the certifying body, the shipping company, even coffee making machines (read, IoT) - agrees to abide by relevant contract(s)
As far as I know, IBM's Blockchain solution is for Enterprises, and not general public, like Bitcoin Exchange. So, all the enterprises become part of that same network.
Consider the following micro services for an online store project:
Users Service keeps account data about the store's users (including first name, last name, email address, etc')
Purchase Service keeps track of details about user's purchases.
Each service provides a UI for viewing and managing it's relevant entities.
The Purchase Service index page lists purchases. Each purchase item should have the following fields:
id, full name of purchasing user, purchased item title and price.
Furthermore, as part of the index page, I'd like to have a search box to let the store manager search purchases by purchasing user name.
It is not clear to me how to get back data which the Purchase Service does not hold - for example: a user's full name.
The problem gets worse when trying to do more complicated things like search purchases by purchasing user name.
I figured that I can obviously solve this by syncing users between the two services by broadcasting some sort of event on user creation (and saving only the relevant user properties on the Purchase Service end). That's far from ideal in my perspective. How do you deal with this when you have millions of users? would you create millions of records in each service which consumes users data?
Another obvious option is exposing an API at the Users Service end which brings back user details based on given ids. That means that every page load in the Purchase Service, I'll have to make a call to the Users Service in order to get the right user names. Not ideal, but I can live with it.
What about implementing a purchase search based on user name? Well I can always expose another API endpoint at the Users Service end which receives the query term, perform a text search over user names in the Users Service, and then return all user details which match the criteria. At the Purchase Service, map the relevant ids back to the right names and show them in the page. This approach is not ideal either.
Am I missing something? Is there another approach for implementing the above? Maybe the fact that I'm facing this issue is sort of a code smell? would love to hear other solutions.
This seems to be a very common and central question when moving into microservices. I wish there was a good answer for that :-)
About the suggested pattern already mentioned here, I would use the term Data Denormalization rather than Polyglot Persistence, as it doesn't necessarily needs to be in different persistence technologies. The point is that each service handles its own data. And yes, you have data duplication and you usually need some kind of event bus to share data across services.
There's another option, which is a sort of a take on the first - making the search itself as a separate service.
So in your example, you have the User service for managing users. The Purchases services manages purchases. Each handles its own data and only the data it needs (so, for instance, the Purchases service doesn't really need the user name, only the ID). And you have a third service - the Search Service - that consumes data produced by other services, and creates a search "view" from the combined data.
It's totally fine to keep appropriate data in different databases, it's called Polyglot Persistence. Yes, you would like to keep user data and data about purchases separately and use message queue for sync. Millions of users seems fine to me, it's scalability, not design issue ;-)
In case of search - you probably want to search more than just username, right? So, if you use message queue to update data between services you can also easily route this data to ElasticSearch, for example. And from ElasticSearch perspective it doesn't really matter what field to index - username or product title.
I usually use both approaches. Sometimes i have another service which is sitting on top on x other services and combines the data. I don't really like this approach because it is causing dependencies and coupling between services. So in general, within my last projects we tried to stick to polyglot persistence.
Also think about, if you need to have x sub http requests for combining data in some kind of middleware service, it will lead you to higher latency. We always try to cut down the amount of requests for one task and handle everything what is possible through asynchronous queues. ( especially data sync )
If you conceptualize modules as the owners and controllers of the data they work on, then your model must also communicate that data out of that module to others. In contrast, the modules in a manufacturing process have the access to change data without possessing and controlling it.
Microservices is an architecture for distributed processing, like most code, where modules pass the data around to work on it. From classic articles by Harvard Business Review and McKinsey on the subject of owning members of a supply chain, I identified complexities arising from this model and wrote an article teaching programmers what you need to know: http://www.powersemantics.com/p.html
Manufacturing is an architecture for integrated processing, where modules work on the data without passing it around from point to point. This can be accomplished by having modules configured to access the same memory, files or database tables. My architecture shows how to accomplish this on memory via reference properties.
When you consider "exposing an API at the Users Service end which brings back user details based on given ids", you need to be aware that creates what HBR calls "irreversible" complexity, which I've dubbed centralization complexity. Don't build A->B (distributed) systems, because you can't decentralize them later after failing to separate requirements. Requirements in production processes represent user instructions, and centralized modules only enable you to change the wrong users' processes. In other words, centralized modules don't document user groups or distinguish them from derived-product-users.
I am working on a civics related project and I need to be able to display all the properties in the City of Philadelphia on a map, so I'll need to get the latitude & longitude for all 580,000 properties. (Only once)
Most APIs like Google/Yahoo have limits of 5,000 per day, and even BatchGeo has a similar limit.
Is there a way I can do a one-time geocoding of all these addresses?
You can find a list of free and paid geocoding services at USC site.
Also check Microsoft's Geocode Dataflow API, it allows up to 200,000 entries / 300 Mb and takes up to 14 days.
Another possibility to combine several services at once: use 4 services that allow 5,000 entries a day and you'll finish your task in a month.
You can use Map Quest of Cloud Made.
I have created a small utility to help compare these API's.
The utility is hosted at below url:
http://ankit-zalani.appspot.com/GeoCode/index.jsp
Tobias, I work for an address verification (and recently, geocoding) company called SmartyStreets.
Many services have usage restrictions based on volume and license agreements which prevent users from storing the results of geocoding queries. There are some vendors, however, which don't have limits or restrictions like that.
I would recommend something like LiveAddress which will not only geocode the addresses but also perform CASS-Certified verification to make sure your addresses are correct before giving you potentially faulty coordinates. You can run 580,000 or even millions at a time in a few minutes, and we allow you to store your results.
Hope this helps. If you have any more questions about addresses, I'll personally assist.
This thread is pretty old by now, but there have been some developments in recent years making bulk geocoding very cheap. My favorite option is to just obtain a geocoding server on AWS ( google: geocoding on aws), many options there, some free some with low hourly rates (total cost depends on the server you choose, of course.)
I've been tasked with setting up a society's website. I'm a full time Django (at al) web developer so I was happy to take on the task.
Going through the specs, they want to control memberships so that all applications need a "second" (read: sponsor, referee, etc) and then they need to pay a subscription fee to be part of the club.
This club has a number of events with variable ticket prices for lunches and talks to name two. Only members are allowed to see the price per ticket and therefore only members are allowed to buy the tickets.
I had originally planned on farming the event management off to EventBrite and pulling the upcoming events back to the website through EB's API but this members-only constraint looks like something EventBrite can't do.
Then there's processing members subscriptions. I had hoped to allow anybody to register a django.contrib.auth account but leave subscription payment offline but the client would be happier if they could mark accounts as "members", store the subscription data in the database and let the members pay online.
Like with EventBrite, I was hoping I could store rough membership data (whether or not they're allowed to subscribe, a unique token for the user on the API service, their level of membership and their membership's expiry) and there'd be something I could post users off to to process their subscription payment.
I basically don't want to touch any payment systems. Even something as simple as Paypal+IPN is something I'd rather not do (I can and have in the past on other projects) but it's the layer of management that I'd have to build around it (messaging members, creating recurring events, etc) that I'd like to farm out to a third party... Even if they do want an additional percent of the payments processed.
Do any of you know any suitable APIs that cover membership or events or both?
Or is this so complex that I should give up hoping for external help and just knuckle down and do it myself?
I think the google search you are looking for is online membership management. I don't know if any of them play particularly nicely with Django/python, but some of them do include APIs. Almost all of these are companies that charge, either for the system, or on a per-user basis.
If you don't mind installing something yourself, CiviCRM is a free, open source solution that I found with a bit of googling. It's integrates with either Joomla or Drupal (so probably PHP-based). You'd have to put the payment processing in yourself, but it does support payments using PayPal which would take handling payments mostly out of the equation. If you can, choose PayPal Express rather than PayPal Website Payments Pro since you may need to be PCI-DSS compliant to use the latter.