I have a long list of towns and cities, and I'd like to add latitude and longitude information to each of them.
Does anyone know the easiest way to generate this information once?
See also Geocode multiple addresses
The first part of the third video shows how to get latitude and Longitude using Google Refine and geocoding. No need to write a new script. Ideal for doing this kind of change once.
http://code.google.com/p/google-refine/
Or use www.geonames.org - there's language APIs for that. Or Open Street Map's Nominatim: http://wiki.openstreetmap.org/wiki/Nominatim - google have slightly more restrictive terms of service.
You can use the Google Geocoding API. Check the API at this URL: http://code.google.com/apis/maps/documentation/geocoding/
What follows next is writing some code. I am doing something similar in C# and it is quite easy here.
Most geocoding services can handle queries with only administrative names which is what you're after, e.g., municipality and region. So I'd choose one you like that also handles batch or bulk requests, e.g., the Bing Spatial Data API (here's an article on batch geocoding with it.)
An alternative approach that might be useful if you're on a budget and have a lot of these to do would be to download the Geonames database and write a bit of code to import it into your database or index it; then query it however and how often you like, e.g., if you put your places in another table you could SELECT [...] FROM my_places LEFT JOIN geonames [...]. I used to import Geonames DB into a vanilla PostgreSQL nightly and probably still have the code in a git repo somewhere if that's a route you want to try (comment and I'll find it and attach.)
For a service that uses google, which I find most accurate.
Look at http://www.torchproducts.com/tools/geocode
Related
I have a very large list of lat/lon coordinate pairs (>50 million). I want to attach address information to each one. Most geo/revgeo services have strict call limits. Assuming computing power isn't the issue, how can I accomplish this? Also note that time/speed are not the primary concern.
One place to start might be the
You can get one of the dedicated AWS geocoders for unlimited volumme processing: https://aws.amazon.com/marketplace/search/results?x=0&y=0&searchTerms=geocoder
Intro
I have experience working with SmartyStreets's batch processing tool. They don't have call limits (paid version). But, they also don't have a Reverse Geocode API (yet!). Their batch processing is strictly for flexibility and ease-of-use in addition to normal calls. But, I am aware of a couple services that do Reverse Geocoding, and they mention batch processing on their website.
How they work
Batch processing services generally allow you to upload your data, even arbitrarily large files. You probably want to put your data in a CSV file (type of spreadsheet) as latitude and longitude pairs. Then, their servers will process the data and alert you when you can download. It's common practice to charge money for this download, but maybe TAMU's is free?
Suggestions on who to use
Texas A&M Geoservices
MapLarge
Both of these services have demos and developer portals to guide you along if there is something you want to research before using them.
(Full disclosure: I have worked for SmartyStreets.)
I need to write an API which would provide access to data being served as HTML documents from a web server. I need for my users to be able to perform queries over the data.
Say on a web site there is a page which lists items and their owners. Then there is additional set of profile pages for owners which for each owner provide information about their reputation. An example query I may need to answer is "Give me ID's and owners of all items submitted in 2013 whose owners have reputation of at least 10".
Given a query to answer, I need to be able to screen scrape only the parts of the web site I need for answering the query at hand. And ideally cache the obtained information for future use with new queries.
I have no problem writing the screen scraping part, but I am struggling with designing the storage/query/cache part. Is there something about Clojure/Datomic that makes it an especially suitable technology choice for this kind of processing of data? I have been pointed in this direction before.
It seems a nice challenge but not sure about a few things: a) would you like to expose to your users a Datalog query box and so make them learn datalog-like syntax? b) what exact kind of results do you wish to cache, raw DB responses, html fomatted text, json ?
Anyway I suggest you to install and play a little bit with the Datomic console to get a grasp if you didn't before as it seems to me the more close idea to what you want to achieve atm https://www.youtube.com/watch?v=jyuBnl0XQ6s http://blog.datomic.com/2013/10/datomic-console.html
For the API I suggest you to use http://clojure-liberator.github.io/liberator/ as it provides sane defaults to implement REST services and let you focus on your app behaviour
Is it possible to use the services of Google-Map to get all the cities in a given country?
No, all the Google services are limited. Try looking at geonames, they have services that may do what you need or possibly you can download the data you need from there and serve it locally.
Disclaimer, it is not possible now. Often if I make an absolute statement like this, the functionality becomes available later. You can try adding an enhancement request to the issues list.
Take a look at the Google Places API
I need to be able to search for a product based on barcode, keyword or ASIN and show the results from amazon.co.uk, amazon.com, amazon.de and amazon.fr (UK, USA, Germany and France).
Is that possible? If so, how do I do it?
At the bottom of this page, you will find links to the various Locales where this Product Advertising API works.
However, you need to make a separate subscription to each one of these different APIs. In other words, you need a different AssociateTag, AccessKeyId and a secret key in order to access those different APIs.
You can implement a search engine that uses all of those APIs in sequence. Start by searching for the product in the US API, then UK, then DE, ... You can encapsulate this complexity in a facade, so that your client simply makes a single call to search. You might also encounter some performance issues with this idea, so the use of cache (and common sense) is advisable.
In dopplr [http://www.dopplr.com] there is an option to fill the city of travel and the site will automatically find the city around the world. Is there any web service or database for such a city lookup?
http://www.geonames.org/ is useful - HTML query, download and web service.
Also http://developer.yahoo.com/geo/ - you'll need a Yahoo developer ID.
Google maps API includes a geocoding service and there are plenty of other options
You could download and process this list on wikipedia. If you trust it.
The best free database of world cities I have found is from MaxMind. See http://www.maxmind.com/app/geolitecity It also includes geocoding information and population. Uses ISO standard for country and regions.
There's the OpenStreetMap Name Finder.
I don't know if it's necessarily the most complete resource, but AFAIK OSM is free for any use, which may/may not be the case with other options.