• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

rsain/GitHub-Crawler: A Python script to collect data from GitHub using its API.

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称:

rsain/GitHub-Crawler

开源软件地址:

https://github.com/rsain/GitHub-Crawler

开源编程语言:

Python 100.0%

开源软件介绍:

GitHub-Crawler

A Python script to collect information from GitHub using its API.

This script allows to crawl information and repositories from GitHub using the GitHub REST API (https://developer.github.com/v3/search/).

Given a query, the script downloads for each repository returned by the query its ZIP file. In addition, it also generates a CSV file containing the list of repositories queried. For each query, GitHub returns a json file which is processed by this script to get information about repositories.

The GitHub API limits the queries to get 100 elements per page and up to 1,000 elements in total. To get more than 1,000 elements, the main query should be splitted in multiple subqueries using different time windows through the constant SUBQUERIES (it is a list of subqueries).

NOTE: please, take a look at the GitHub website to be sure that you do not violate any GitHub rule.




鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap