I wanted a free web service to get cities list and found geobytes. Its good. I wanted to know What is the meaning of 50000 request? On every key pressed it makes a HTTP request.So do they count this way?
but if you expect to be performing more than 50,000 requests per day (your average unique visitors X 5), then please tell us
Anyone who has used this please help.
I would imagine what it means is that going over 50,000 requests can be penalized in someway. A key press is not a request - but entering a city and fetching that cities' details would constitute 1 of the 50,000 requests.
Hope this helps.
I am the author and administrator of Goebytes'AutoCompleteCity API and there is now no practical limit to genuine use, and the reference to 50,000 lookup per day has been removed from the web site. I say practical, because it does have DOS attack prevention measures, but as the API is intended to be called from the browser (as opposed to a server - for that you would use the GetCityDetails API) its DOS protection measure of "1024 look ups per IP Address, per hour", should never cut in under any circumstances that I can imagine.
Related
Say I want to develop a web application that will have registered users and will be registered as a twitter app (allowing users to give it permissions to view their timeline and post on their behalf). The sole function of the application will be to re-tweet tweets from users' timeline according to users' settings and desires.
I understand that the website for this app will use the common technologies like HTML, CSS and JS on the client-side. The server side (where the user defines what kind of tweets the application should retweet) will have to be coded in PHP/Python/Perl/... based on a DB MySQL/Postgre/...
What I don't understand, and would really appreciate your help with, is where the real "business logic" will be coded? For example, what technology should I use to code the function that will sit on my server: contacting Twitter server every 5 minutes, reading the timeline of every user I have, checking whether there are tweets worth retweeting (according to what the user has defined), and sending Tweeter the necessary commands to retweet the chosen tweets on behalf of my users.
All that will happen off-line for the user, and will be an on-going and cyclic process - but what technology should I use to code it?
Thanks!
I have heard about this API for PHP. It is actually the only one that I have heard of for PHP, though. I know that there are some good Python libraries out there, but I don't know about Perl.
I am actually working on a new API for C# (won't be a good fit for you, as you're clearly not using Windows Servers), and started building it while working on an enterprise web application that prompted several questions similar to your own.
Here is what you are going to have to do:
Before you start, you are going to have to get in touch with one of Twitter's data partners (I believe that you may contact Twitter for the reference)
The reason for this is that you are going to be requiring many more requests than you think
Twitter's time interval used for Twitter's recorded rate cap is 900 seconds (5 minutes)
With the general rate limit if you are querying a user's timeline only once every rate limit, you are limiting your number of visitors on your site to 300 at a time
Here's where it gets tricky - if every user makes one Tweet (meaning you send the Tweet - not rate limited - and then refresh the timeline - rate limited - so that they can see the updated tweet) you have now dropped your maximum number of active users at any given time to 150
Factor in the company's own timeline (-1 visitor), plus the number of visitors who leave their browsers open (now you need more logic, and you have to either kick them off or simply keep track of whose timeline you won't be refreshing), the number of users who make more than one tweet (-1 visitor for each Tweet), etc.
Moral of the story: contact one of their data partners and get yourself either unlimited requests, or at least a significant enough amount to accommodate your number of visitors/users (plus a bit of padding)
If you adhere to this advise, skip steps 2 and 3, otherwise, skip step 4
(Note: Steps 2 and 3 are only for rate-capped implementations) Using your desired language, make a service that runs on the server and makes the queries to Twitter
Based on the information that you gave, I suggest that you use Python to make this service
The service will run at all times and be on it's own clock to base the 5-minute intervals between the requests on
You will have to use a caching or a database system for storing the data
(Note: Steps 2 and 3 are only for rate-capped implementations) Add the necessary code to make a request to the service that you created for the data and perform these requests every 5 minutes
I suggest that the clock used for making these requests to the service be running a little bit behind the clock used for the service to account for instances of slow data transfer, etc.
You will also have to have to call some methods on the service for adding/removing users from the queue
(Note: Step 4 is only for unlimited request implementations) Forget about the service and simply include the request code directly in the page that the user is on.
The user's timeline will be updated based on when they visited the site or when their timeline was last refreshed (if a Tweet was made)
The only caveat to this implementation is that you will have to pay for the unlimited/larger data rate limit
I am planning to get an app developed but the developer has told me that there is a limit of 600 calls per 600 seconds per IP. The app has plenty of scenarios in which this would not suffice. Is there a way to increase the limit somehow? Or does Facebook offer any premium account or something probably with a yearly fee that does not have such a limit?
Thanks.
If you exceed, or plan to exceed, any of the following thresholds please contact us as you may be subject to additional terms: (>5M MAU) or (>100M API calls per day) or (>50M impressions per day).
Pulled from : https://developers.facebook.com/policy/
100M API Queries per day should be for a single app. So that should restrict you, but I dont think that matters.
Another thing, what you mentioned in your question, and I have read that elsewhere as well.
I've found 600 calls per 600 seconds, per token & per IP to be about where they stop you.
Pulled from : http://www.quora.com/Whats-the-Facebook-Open-Graph-API-rate-limit
Note, it is per token. Every other user has a different access token and IP as well. If it happens to be a cron running from the server, still I dont think they would catch you for the IP as long as you keep changing the tokens.
Another thing to implement is the Real time updates API, which will ping you when something changes so you dont have to run a 24*7 monitoring script.
P.S : Real Time Updates is Buggy! Have experienced it myself.
I am working on a civics related project and I need to be able to display all the properties in the City of Philadelphia on a map, so I'll need to get the latitude & longitude for all 580,000 properties. (Only once)
Most APIs like Google/Yahoo have limits of 5,000 per day, and even BatchGeo has a similar limit.
Is there a way I can do a one-time geocoding of all these addresses?
You can find a list of free and paid geocoding services at USC site.
Also check Microsoft's Geocode Dataflow API, it allows up to 200,000 entries / 300 Mb and takes up to 14 days.
Another possibility to combine several services at once: use 4 services that allow 5,000 entries a day and you'll finish your task in a month.
You can use Map Quest of Cloud Made.
I have created a small utility to help compare these API's.
The utility is hosted at below url:
http://ankit-zalani.appspot.com/GeoCode/index.jsp
Tobias, I work for an address verification (and recently, geocoding) company called SmartyStreets.
Many services have usage restrictions based on volume and license agreements which prevent users from storing the results of geocoding queries. There are some vendors, however, which don't have limits or restrictions like that.
I would recommend something like LiveAddress which will not only geocode the addresses but also perform CASS-Certified verification to make sure your addresses are correct before giving you potentially faulty coordinates. You can run 580,000 or even millions at a time in a few minutes, and we allow you to store your results.
Hope this helps. If you have any more questions about addresses, I'll personally assist.
This thread is pretty old by now, but there have been some developments in recent years making bulk geocoding very cheap. My favorite option is to just obtain a geocoding server on AWS ( google: geocoding on aws), many options there, some free some with low hourly rates (total cost depends on the server you choose, of course.)
Hi actually this is a simple question but just came up out of the curiosity...
I have seen a web evaluation online tool recently called teqpad.com.I have lots of queries on it
How do they do it?? eg:page views daily visitors etc. without mapping real website??...
Website worth...is this getting any near to any site??
I don't know how do they got daily revenue??
I like traffic by country..it has seen same like in Google analytic s..how they got that info??
another one is ISP info and Google map location of server..
is there any one here done similar scripts?? if so what is your opinion??
They may be tracking user browser stats like Alexa does. (More info on Wikipedia.) A group of users installs a plug-in that reports which sites each user visits, like TV ratings work in most (all?) countries. This method is obviously not very reliable, and often nowhere near the actual numbers of visitors.
This is usually based on bullshit pseudo-scientific calculations and never a viable basis for evaluating the "value" of a web site, even though it may be possible to guesstimate the approximate ad revenues a site yields (see 3) But that is only one revenue stream - it says nothing about how expensive the site's daily maintenance is - servers, staff, content creation....
It should be possible to very roughly estimate daily revenue by taking the guesses on daily visitors/page views, count the frequency with which ads are shown, and look at what those ads usually yield per page view. It is probably pretty easy to get some rough numbers on what an ad view is worth on a big site if you're in the market.
and 5. It is possible to track down most IP addresses down to the visitor's country and sometimes even city. See the Geo targeting article on Wikipedia
In developing a new web service I haven't been able to find very much information on how companies bill for their web services.
Do you bill by request or only certain requests ie) GET or POST?
-would these be tracked at the application or server level?
Do you bill by bandwidth?
-again how would this be tracked on a per user basis
Do you charge a subscription to simply have access?
-this is assuming that they are only granted an api key after payment has been made.
A combination of the above or other options?
Thanks for your help.
As all things in a market economy, the price, but also the inconvenience (or convenience) and risk associated with the actual payment (irrespective of the amount) is a function of how unique and cool and valued your service or product is.
It is therefore impossible to answer the question but in very generic terms, i.e. in the form of suggestions. You actual invoicing model may base on one or several of the following
bill for a one-time setup fee
bill on a subscription basis (i.e. for a defined period, with explicitly defined maximum amounts of usage)
bill for maintenance
bill by the act, i.e. a certain amount (possibly on a decreasing unit price schedule). Such acts should be counted at the server level, (The client-side may include some audit/monitoring/log of sorts, but the server-side should be the authoritative source of info)
bill by volume (for example number of MBytes transfered etc.), this is applicable to services where there is a big variation in the volume of info produced for each "act".
In general, the price and the modality of accounting should seem fair, to both parties, particularly to the buyer, and typically, the simpler the better. The price should not necessarily be low, provided you can make the case that the service provided is effectively valuable, and that you either invested and took risk to introduce the service, or the on-going expenses associated with running the service are evident.
I guess It Depends™ on what the service does. Broadly, I'd say you should bill when you provide some intrinsic value; how you determine what that billing criteria is is quite domain-specific. There may be some property of the service provided which allows you to determine how much to bill.
For example, suppose you've a web service that performs a calculation. You might decide that for every successful computation you do, you're going to charge a fixed fee, say $0.01, but let users off if there's a validation problem, such as an invalid request. Alternatively, if those computations are vaguely long-running, you might have a charging model that's based on some sort of CPU-time metric.
Your point about subscriptions is a good one, and this is an area where you might potentially benefit from allowing a couple of commercial models; one to cater for the users who might perform a lot of requests per month, in which case a fixed subscription might make sense, and one to cater for users who make a few ad-hoc requests. In the latter case, of course, if you only attract those customers, then you're not going to make a good return on investment. Some kind of middle ground, whereby you have a small subscription, but then allow customers to buy a "block" or "bundle" of requests on top without incurring additional processing costs, might work.
Most webservices I know of charge for two things:
Volume of "usage". Generally giving low volumes "free" access (i.e., less than X hits/hour from a given IP address account combination). This is similar to say, twitter which gives you 150 hits/hour to its service from either your username, or unique IP or combination of the two (so you dont abuse it by changing IPs frequently). If you want a higher volume you pay for that access and its usually assigned by account (in twitters case you can get a dev account [for free] which gives you 20K or more hits an hour)
Depth of Details, Access to features. Again free accounts get a minimum amount of access, but dont get access to more data or to more advanced features (filtering, etc). Lots of google services work like this, were base access is given to everyone but if you want more refined abilities (greater search, more data, faster results) you have to buy an account code with the corresponding functionality.
I havent really seen or participated in any projects with pay-for-performance, or pay-per-hit/access models as they get very difficult to reliably bill for and very hard to account for to customers, even if you use tiered or banded ranges. How do you tell your customers how many hits they have used, especially in a distributed system, with redundant fail-over, etc. If I had to pay $0.01 cents per access I would want to know exactly how its measured, and what the company had in place to control access, and how accurate their monitoring was, etc.
Its not impossible, and definitely can be done, and may work well in large bulk scenarios.
Many of the ones I have seen bill by time, such as on a monthly or yearly basis. Some allow you to pay by the month, some require some (or all) of the fee up front. Access might be restricted by issuing a security certificate for the web service that expires when the customer's account expires, or possibly by having them send a client ID and letting the server check if that client ID is allowed to have an answer (but that's open to people stealing someone else's client ID ;) ).
I suppose if you have a service that sends and receives very large amounts of data, it might make sense to bill per service request, but the billing for that could get trickier. Are clients likely to make dozens of requests per day, or just a few? How much to bill per transaction? $100? $0.01? That all would depend on the nature of the service. If you want to go that route, you would probably need to be able to ensure that clients only get billed for requests that are successfully answered (I'd hate to get billed even though my client app failed to receive the entire web service message from your server).
Per request or as a subscription, and yes, bandwidth can be a variable that is used to set the fee. Depends of the value of binding the customer close or having a myriad of loosely coupled customers using it. There is no correct answer to the question that fits all or even most cases.
If I look at the services I have made in the past, the subscription model would be the best model to use. Sometime a tick of $ per request seems like the best approach but I have never had a service configured that way yet.
I agree with what has been said by Rob and Des. One thing to remember is that a subscription is a really simple concept that everyone is used to and comfortable with (if you price it right). If you want to cover a wide audience look at how the payment providers do - they have slightly different methods of payment depending on how many transactions you do per year. There'll be a fixed subscription plus a per-transaction charge and they both vary with the number of transactions. This is the most flexible, but it depends if it makes sense for your business.