lycos

1.0.1 • Public • Published

🐾 lycos.js

All the goodies you'll ever need to scrape the web

Documentation

In-browser Playground

You can try the library on codesandbox, it uses a cors proxy fetcher to let you grab contents from any website inside your browser.

Installation

yarn add lycos
# or 
npm i lycos

Features

  • ⚡️️ All in one package to fetch and scrape data from the web
  • ⭐ Node & Browser Support
  • 💡 Powerful declarative API
  • 🚀 Blazingly fast (supports concurrency)
  • 🔧 Extensible

Quick Example

const lycos = require('lycos');
 
// Get page from the url and paginate through all
// the next pages to extract quotes
const quotes = await lycos
  .get('http://quotes.toscrape.com/')
  .paginate('.next > a')
  .asyncFlatMap(pageScraper =>
    pageScraper.scrapeAll('.quote', {
      author: '.author@text',
      text: '.text@text'
    })
  );
<!-- quotes -->
[
  { 
  "author": "Albert Einstein", 
  "text": "“The world as we have created it is a process of our thinking.“"
  },
  ...
]

Credits

• FB55: his work reprensents the core of this library.

• Matt Mueller and cheerio contributors : A good portion of the code and concepts are copied/derived from the cheerio and x-ray libraries.

License

MIT © 2019 Jimmy Laurent

Package Sidebar

Install

npm i lycos

Weekly Downloads

0

Version

1.0.1

License

MIT

Unpacked Size

1.12 MB

Total Files

51

Last publish

Collaborators

  • jimmylaurent