Geolocate multiple IP addresses(获取大量IP地址的地理位置)–[1]

王 茂南 2017年10月4日23:29:06
评论
3788字阅读12分37秒

 

问题

来源:https://mathematica.stackexchange.com/questions/155076/geolocate-multiple-ip-addresses

原文

I would like to geolocate a few hundred IP addresses (as part of analysing server logs). What is the best way to do this using Mathematica?

Does Mathematica have any builtin functionality for this?

Is the best approach to try to query an online service instead? Is there one which is easy to use from Mathematica and would be able to handle up to 1000 addresses?

Should I try to find a database and use it offline from Mathematica?

I am not necessarily looking for working code. A general high-level guide on the best approach will be sufficient.

Update: For this application, performance is important. Here's a dataset to use as benchmark:

 

websites =
WolframAlpha[
"top websites", {{"OrdinalRankings", 1}, "ComputableData"},
PodStates -> {"OrdinalRankings  More", "OrdinalRankings  More",
"OrdinalRankings  More", "OrdinalRankings  More",
"OrdinalRankings  More"}];

Length[websites]
(* 160 *)

iplist = First /@
Flatten@DeleteMissing[HostLookup /@ websites[[All, 2]]];

Length[iplist]
(* 355 *)

翻译

我想获取成百个IP地址的地理位置(作为分析服务器日志的一部分)。使用mathematica来做这件事最好的方法是什么? mathematica是否有内置的函数来完成这个功能? 尝试查询在线服务是最好的方法吗?是否有这样一个便于被mathematica使用并且能处理1000IP地址的网站? 我是否应该寻找一个可以离线使用的数据库并使用mathematica来调用他。 我不一定要找能运行的代码。关于最佳方法的高质量的指导就可以了。

更新:对于这个应用,性能是十分重要的。下面是用来做基准的数据集:

websites =
WolframAlpha[
"top websites", {{"OrdinalRankings", 1}, "ComputableData"},
PodStates -> {"OrdinalRankings  More", "OrdinalRankings  More",
"OrdinalRankings  More", "OrdinalRankings  More",
"OrdinalRankings  More"}];

Length[websites]
(* 160 *)

iplist = First /@
Flatten@DeleteMissing[HostLookup /@ websites[[All, 2]]];

Length[iplist]
(* 355 *)

 

高票回答

 

原文

Batch mode API call, for best performance

 

From the documentation at http://ip-api.com/docs/api:batch "Batch JSON", a batch processing with the ability to query multiple IP addresses in one HTTP request, significantly faster than submitting individual queries. A batch request requires a POST request to http://ip-api.com/batch with a Body string in JSON array format, containing up to 100 objects. Therefore, here the ipLongListarguments is Partition into as many ipShortList as necessary.

Free for non-commercial use only!

 

BatchIP2Location[ipLongList : List[_String ..]] :=
AssociationThread[ipLongList,
Flatten@Table[
Query[Values, GeoPosition][
ImportString[
URLRead[
HTTPRequest[
"http://ip-api.com/batch",
<|
Method -> "POST",
"Query" -> {"fields" -> "lat,lon"},
"Body" ->
ExportString[
Map[{"query" -> #} &, ipShortList],
"JSON"]
|>]
, "Body"], {"RawJSON"}]
],
{ipShortList, Partition[ipLongList, UpTo[99]]}
]]

 

翻译

查看一下网址 http://ip-api.com/docs/api:batch关于批量处理JSON的文档,可以使用批处理在一条HTTP请求查询多个IP地址,很明显的比单独提交查询快。一个批量的请求需要提交一个POST请求给http://ip-api.com/batch,其中包含至多100个对象。因此,ipLongList被分割成ipShortList是必须的。

仅供非商业用途使用!

 

BatchIP2Location[ipLongList : List[_String ..]] :=
AssociationThread[ipLongList,
Flatten@Table[Query[Values, GeoPosition][(*将经纬度使用GeoPosition进行包装*)
ImportString[(*把字符串以JSON的格式输入,返回提交ip地址的经纬度*)
URLRead[
HTTPRequest[
"http://ip-api.com/batch",(*生成要提交的内容*)
<|
Method -> "POST",
"Query" -> {"fields" -> "lat,lon"},
"Body" ->
ExportString[
Map[{"query" -> #} &, ipShortList],
"JSON"]
|>],
"Body"], {"RawJSON"}
]],
{ipShortList,
Partition[ipLongList,
UpTo[99]](*将列表划分,每个里面最多包含99个元素(因为每次提交数据只能提交100个),ipShortList就是将ipLongList重新划分后的数据,分次提交*)}
]]

 

运行结果

BatchIP2Location[iplist][[;; 5]]
(*IP地址->经纬度*)
(*
<|"216.58.200.238" -> GeoPosition[{37.4192, -122.057}],
"8.7.198.45" -> GeoPosition[{39.2139, -104.846}],
"93.46.8.89" -> GeoPosition[{37.6167, 15.1667}],
"111.13.101.208" -> GeoPosition[{39.9289, 116.388}],
"220.181.57.217" -> GeoPosition[{39.9289, 116.388}]|>
*)

 

生词&&一些好的翻译

  • query:询问
  • dataset:数据集
  • benchmark:基准
  • coordinates:座标
  • batch:批量 batch precessing:批处理,批量处理
  • Is the best approach to try to query an online service instead?:尝试查询在线服务是最好的方法吗?
  • I am not necessarily looking for working code. A general high-level guide on the best approach will be sufficient.:我不一定要找能运行的代码。关于最佳方法的高质量的指导就可以了。
  • Free for non-commercial use only!:仅供非商业用途使用

 

 

 

  • 微信公众号
  • 关注微信公众号
  • weinxin
  • QQ群
  • 我们的QQ群号
  • weinxin
王 茂南
  • 本文由 发表于 2017年10月4日23:29:06
  • 转载请务必保留本文链接:https://mathpretty.com/8219.html
匿名

发表评论

匿名网友 填写信息

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: