Scraping GitHub Trending Repositories
github-trend is a library for scraping GitHub trending repositories.
Only scraping
const Trending = ;const scraper = ; // Empty string means 'all languages'scraper; // For other languagesscraper;scraper;scraper;
Scraper
only scrapes GitHub trending page. This returns an array of repository information.
This method is relatively faster because of sending request only once per language.
Scraping and getting full repository information
const Trending = ;const client = ; 'lient.fetchTrending('').then(repos => { for (const repo of repos) { // Result of https://api.github.com/repos/:user/:name console.log(repo); }}).catch(err => { console.error(err.message);}); // Fetch all API call asynchronouslyclient.fetchTrendings(['', 'vim', 'go']).then(repos => { for (const lang in repos) { for (const repo of repos[lang]) { // Result of https://api.github.com/repos/:user/:name console.log(repo); } }}).catch(err => { console.error(err.message);});
Client
contains scraper and scrapes GitHub trending page, then gets all repositories' full information using GitHub /repos/:user/:name
API.
This takes more time than only scraping, but all requests are sent asynchronously and in parallel.
Scraping language information
const Trending = ;const scraper = ; scraper;
This returns all languages information detected in GitHub by scraping here. The result is cached and reused.
Scraping language colors
const Trending = ;const scraper = ; scraper; // If you want only language names:scraper;
Collect trending repositories by scraping and GitHub API
By scraping GitHub Trending Repositories page, the information is restricted to the information rendered in the page. This library also supports to getting information of trending repositories using GitHub Repositories API.
Although an API token (at the first parameter of new Client
) is not mandatory, it is recommended
for avoiding API rate limit.
const Client = ;const client = token: 'API access token here'; client;
License
Distributed under the MIT license.