We started using the GeoLite2 Country DB and saw some very strange results. Looking closely it appeared that the DB itself had incorrect data. Looking more closely I saw that for the same IP the GeoLite2 Country and GeoLite2 City gave different results.
(I may also be reading this wrong so any advice on that most welcome!)
The IP in question is 46.251.120.133
Maxmind – country
- Doesn't have 46.251.120.0
- It does have 46.251.0.0 which is found as location 719819 (Hungary) which is incorrect.
Maxmind - city
- Has 46.251.120.0 which is found as location 146268 (Nicosia, Cyprus) which is, correct.
To be specific, we're using the csv files found here:
http://dev.maxmind.com/geoip/geoip2/geolite2/
Really hoping I'm reading something wrong in the db...
Thanks!
::46.251.120.0 is part of the ::ffff:46.251.96.0/115 network, which is mapped to 146669. 146669 is the blocks record for Cyrus.
Related
I've downloaded the latest updates of the Maxmind's GeoLite2 City database (both in MaxMind DB binary and CSV formats). When I tried to look up "88.184.98.0" here's what I got:
{"city":{"geoname_id":2982652,"names":{"de":"Rouen","en":"Rouen","es":"Ruan","fr":"Rouen","ja":"ルーアン","pt-BR":"Ruão","ru":"Руан","zh-CN":"鲁昂"}},"continent":{"code":"EU","geoname_id":6255148,"names":{"de":"Europa","en":"Europe","es":"Europa","fr":"Europe","ja":"ヨーロッパ","pt-BR":"Europa","ru":"Европа","zh-CN":"欧洲"}},"country":{"geoname_id":3017382,"is_in_european_union":true,"iso_code":"FR","names":{"de":"Frankreich","en":"France","es":"Francia","fr":"France","ja":"フランス共和国","pt-BR":"França","ru":"Франция","zh-CN":"法国"}},"location":{"accuracy_radius":5,"latitude":49.4431,"longitude":1.0993,"time_zone":"Europe/Paris"},"postal":{"code":"76100"},"registered_country":{"geoname_id":3017382,"is_in_european_union":true,"iso_code":"FR","names":{"de":"Frankreich","en":"France","es":"Francia","fr":"France","ja":"フランス共和国","pt-BR":"França","ru":"Франция","zh-CN":"法国"}},"subdivisions":[{"geoname_id":11071621,"iso_code":"NOR","names":{"de":"Normandie","en":"Normandy","es":"Normandía","fr":"Normandie"}},{"geoname_id":2975248,"iso_code":"76","names":{"de":"Seine-Maritime","en":"Seine-Maritime","es":"Sena Marítimo","fr":"Seine-Maritime","pt-BR":"Sena Marítimo"}}]}
However, there's no corresponding geoname_id for returned subdivisions in CSV files (e.g. cat GeoLite2-City-Locations-en.csv | grep 11071621 returns nothing).
Is it a bug or expected behavior? I couldn't find anything in the documentation.
I've reached out to support and the answer is that this is by design. The CSV files do not include locations if there's no IP directly associated with them.
Read more here: https://github.com/maxmind/MaxMind-DB/issues/61
Recently I have stumbled upon a problem - MaxMind's GeoIPCity file is way too big for our needs and contains A LOT of data we don't need and won't need.
The question is: is there a way to limit the City database to a single country? let's say, Canadian cities only?
You cannot just conveniently download the database for Canadian cities only, but you can certainly prune the database once you have downloaded and loaded it. This is true whether you use the MaxMind DB or download the CSV format, just trim out the lines that do not represent Canada's country code or geoname_id (depending on v1 or v2 of the dataset).
If you identify your specific coding environment and language, I'm certain someone can help you write a few lines of code that chops out all the fat.
i want to list all available industries ( like: http://biz.yahoo.com/p/ ) and show all corresponding stocks.
Until now I'm using YAHOO.Finance.SymbolSuggest.ssCallback for the symbol suggestion and http://finance.yahoo.com/d/quotes.csv?s=... for getting the stock's data.
Does anyone have any idea how to get all industries and corresponding stocks?
Is there another hidden Yahoo API?
Lists of all available industries are called GICS Sectors for Standard and Poor's (S&P500 will use that) and ICB for Dow Jones and FTSE. Hence it used by Nasdaq, Nyse and others markets.
It seems like Yahoo uses a third industry classification by Morning Star, but since I'm not quite sure I will give both ways of retrieving data.
Morning Star
I don't know if Yahoo really sticks to this classification, but some names were really close so let's see it:
You need to go to their Index Data and in each sector, click on it and then at the bottom View complete index holdings.
It's not as precise as in Yahoo industry list, but it's all you can do with Morning Star. Not very convincing, I know...
GICS Sectors
GICS Sectors are now a trademark of Standard and Poor's and then data have to be sought for in S&P's website.
Short answer: take a look at this page, you will need to be registered (it's free and easy) and you can download spreadsheets (xls) with stocks and corresponding sectors. Nevertheless, things aren't always easy, and you will have to do a bit of a search to retrieve all stocks with their corresponding industries. For example, the file INDICATED_RATE_CHANGE.xls will give you some companies and their sectors in each month of 2012. Using that and SP500_DividendAristocrats_2012.xls you should be able to retrieve at least a large part of S&P 500 companies.
ICB
ICB is used by NYSE, NASDAQ etc... Then it's a lot simpler than S&P and MorningStar. Here is your answer. BOOM! Direct link!
Link is dead :(
Finally
I strongly advise you to use the simpler and most-used industry classification index: the ICB. It will always be available and publicly displayed since millions of investors relay everyday on it, without having to use S&P financial services or MorningStar brokerage services...
EDIT
You can look at nasdaq.com to retrieve all companies and their corresponding sector: here for Nasdaq and here for Nyse
Get all industry-IDs from here:
http://biz.yahoo.com/ic/ind_index.html
(look at the links)
Then use YQL ( https://developer.yahoo.com/yql/console/ )
with a query like this:
select * from yahoo.finance.industry where id=912
I'm brand new to geocoding and have a relatively large dataset of addresses 100,000+. When I geocode them using MapMarker Professional I get about 10% that I'm not able to geocode with a high level of precision (I get mostly S2 precision values back which mean that it was able to match to a Primary Postal Code centroid, centerpoint of the Primary Postal Code boundary). Each of the addresses has already been standardized so they should be valid (I have taken a random sample and run them through the USPS Zip Code Lookup process to verify this). My question is, should I be able to geocode addresses with a higher degree of accuracy than what I'm seeing or am I expecting too much of the products currently on the market? I've tried geocoding using google and yahoo's services without any better luck. All of the services appear to be able to give me the postal code centroid, but none of them appear to have enough information to be able to give me distinct coordinates for houses in at least 98% of the addresses I send to it.
Thanks for any guidance you can provide,
Jeremy
Geocoding is an imprecise process. The addresses you are geocoding that don't have good precision are likely in rural areas, where it is not uncommon for addresses to be off by up to a mile.
They only know where addresses are by taking the number at the start and end of a street segment, and dividing from there.
How Do I work around a problem with the yahoo map geocode result set? The result set being returned is wrong. The city field contains the city, region and postal code. As seen below.
Is there a way to work around this issue without breaking scalability.
-33.924320
151.187057
203 Coward St
MASCOT NSW 2020
Australia
AU
The Yahoo geoencoding returns usually an XML or a PHP serialized. By querying the encoding service I suppose you already have the address and you want to get the coordinates for your geoPoint. It is possible that you are feeding the maps engine with a wrong request.
If you think you found a bug you can send them an email, but I suggest you to check with other locations or to publish first here your code in order to spot the eventual errors.