I was wondering if there is any way to find historical geolocation IP information? Everything I've been able to find discusses current data, but I've been unable to find any way to query a service or DB to, for example, look for information on an IP from a few years ago.
I found this article titled 'How to perform historical IP geolocation lookups' and it does mention maxmind as a potential tool for availability of historical versions but it doesn't mention how to access this data.
Looking through the maxmind api docs I can't seem to find any way to query by date either.
How could one go about finding the location of an IP at a given time in the past? (Extreme accuracy not necessary, an example would be Find the country of this IP address in 2012)
Perhaps there is an easier solution, but it seems that you can download quite a few (not all) old versions of the MaxMind database via the Wayback Machine. For example, this shows the snapshots they have of the binary database. For the CSV files, you'd need to look up the snapshots for specific filenames, and manually change the filenames to the desired 1st-of-month date. For example, here's one.
That said, these posts seem to suggest that IP ranges aren't reassigned to different geographies very often--might not be necessary to worry about getting old versions of the database:
https://serverfault.com/questions/286025/historical-ip-geo-location
https://serverfault.com/questions/59167/how-often-do-ip-blocks-get-reassigned-to-different-regions
Related
(I'm using MaxMind's databases)
I'm using SourceMod which is a game server management addon for Source Engine, which provides plugin support.
Two months ago, I have added a plugin that does the following:
Check if the connecting user is from Israel. (Where I host the server, you're required to speak Hebrew in order to play)
If the user is not from Israel, check if his IP address is whitelisted. If it isn't, disallow the connection.
Now recently, one of the Israeli ISPs got new IP addresses (all starting with 77.138.*.*) which GeoLite shows as if they're French. So far I've got to whitelist 12 IP addresses because of this.
SourceMod has a third-party API that lets me use GeoLite2 City, GeoLite City (free ones) and GeoIPCity/GeoIPISP if I own them. For now I've tried using both of the free databases, sadly, no success.
I know MaxMind will update GeoLite again in 6 days. What I'm afraid of, is that I'm not going to achieve anything because MaxMind doesn't offer support for GeoLite, so I might be stuck with outdated databases that provide incorrect information.
What is a recently updated alternative that uses the same format as GeoLite (either legacy or new), or, is it 100% confirmed that those incorrect locations for IPs will be updated soon for GeoLite? I really don't feel like spending cash monthly just for GeoIP.
I have a list of IP addresses (IPv4 in the form 122.XXX.XXX.XXX) which I want to be assigned to a pair of latitude and longitude coordinates; I would be happy also with a approximate location. I downloaded the free GeoLite2 database (here) but I don't understand how should I proceed, since the IP information in the database is stored as IPv6 (on which I have no experience).
Is there any way to proceed? Should I use drop everything after ::ffff: and then match the rest of the string with my IPv4? Or should I also drop part of my IPv4, which I want to geolocate?
The GeoLite2 database is in hybrid notation to be forwards compatible. While dropping everything after ::ffff: would work, I would recommend the opposite, convert the IPs you get into hybrid notation and then match. (Just add ::ffff: to the front). This should help if you ever have to start supporting IPv6.
What is the best, most efficient way to test city detection? I have IP-based location detection implemented via www.maxmind.com, but now I'd like to test it's accuracy.
I know there are various proxy services out there such as https://www.geoedge.com/ and various similar websites, but most of these services have a very limited number of proxy servers. It would be great to have an automated solution which could iterate through hundreds if not thousands of proxy servers, hit a test page, and tabulate the results. I'm sure there are others who have had to deal with the same challenge. What is the defacto way to test this? For example, is cURL'ing to spoof IP addresses a possibility?
Note: many people have suggested that you can never achieve perfect accuracy when it comes to city detection due to the lack of reliability of IP addresses, and I am aware of this (http://www.maxmind.com/en/city_accuracy). I'd still like a way of testing for sanity / maintenance purposes. Thanks!
Related: How do sites like Groupon segment geolocation based on the cities they have deals in?
I split this answer into two sections for the sake of clarity.
IP Geolocation
You may want to stick with MaxMind unless you have a very good reason to question the MaxMind data. I built a very similar service to the one you are describing a few years ago and, like you, wanted a way to verify MaxMind's accuracy. I evaluated 10+ IP geolocation solutions running the entire gamut; free JSON APIs to enterprise-centric, database subscriptions. It became apparent rather quickly that most of the platforms were either using MaxMind directly or combining MaxMind data with metadata from other sources. The spelling, capitalization, and common abbreviations of ISP metadata
This paper, despite being a few years old, is also quite telling. The authors ascertain the accuracy of a handful of IP geolocation tools (including MaxMind) by comparing their results to a dataset they refer to as "ISP Groundtruth", a mashup of EU ISP router data and the actual GPS coordinates of the routers. The paper puts forth a technical explanation of inaccurate geolocation data at the city level.
Proxy Scanning
With respect to automated proxy scanning, I highly recommend checking out nmap and its Lua-based scripting engine (NSE). Here are a few scripts and libraries you may find useful:
open proxy detection
HTTP proxies
SOCKS proxies
proxy testing
ip geolocation
MaxMind
IPInfoDB
geoPlugin
database support
PostgreSQL
MySQL
I know the question borders between something that can be answered or discussed, but with just examples should be enough.
I'm working on BI and part of the data contained is the ISP from visitors. I'm trying to extract as much information from the data in the dataware, so I just wanted to know how I can help transform this specific piece of data into something valuable to the client.
Any samples of the value of knowing the ISP? The site I'm working for is massive (can't say which one), so that should block responses as "it would matter if you were <giant-company>". I need exactly THAT info.
One of the ways I have used that information in the past is to determine what peering agreements are appropriate. That is, if you determine that 40% of users are coming from ISP X, then getting a peering arrangement setup with ISP X will improve user experience due to decreased latency. I've also used this information for selecting a CDN (pick the CDN that provides the best footprint for the ISP's your users are on). It could also be useful for determining where to put edge caches (for instance, varnish instances). There are a whole host of uses for the data but at the top of my mind the class of uses are limited to performance and availability.
Sorry for so noobish question but how can I do it with C++? I need to get country names and their 2-letter or 3-letter ISO codes (or whatever it's called) using a given IP address. Is there any way to do this easily?
The only two ways I found are to use GeoIP C library or to download a free database but is there something more dynamic? I mean, is there a free web-service with up-to-date db I can utilize for my purpose using curl or something?
Thanks in advance.
MaxMind.com exposes a set of web services for geolocation. They're not free, but cheap per lookup.
One of my clients uses their city-level lookup service, with each result cached for a reasonable period (from memory, 2 or 3 weeks), since IP address blocks don't change location that often.
The application also counts the number of actual lookups to send an admin notification when the number remaining in the subscription gets low, so it can be topped up manually.
I also use the downloadable database in several applications, with a scheduled update process to keep it up to date.
http://www.hostip.info provides an API.