urlint-core

Get all the links behind an url and classify them based on HTTP status code.

Install

$ npm install @urlint/core --save

Usage

;(async () => {
  const urlint = require('@urlint/core')
  const url = 'https://kikobeats.com'
  const emitter = await urlint(url)

  emitter.on('*', function (event, data) {
    console.log(event, data)
  })

  emitter.on('end', function (data) {
    console.log('finished!')
  })
})()

API

urlint(urls, [options])

urls

Required
Type: string|string[]

The target URL(s) for detecting content.

options

concurrence

Type: number
Default: 8

The number of URLs that can be resolved in parallel.

quiet

Type: boolean
Default: false

When is true, it avoid print progress on terminal. This is oriented for CI environment and the value will be automatically inferred.

verbose

Type: boolean
Default: false

When is true, it will be print more detailed information per every link as output.

prerender

Type: string|boolean
Default: auto
Values: auto|true|false

Get HTML markup from the target URL using prerendering.

retries

Type: number
Default: 2

Maximum quantity of retries to do per every link after consider the URL is not reachable.

timeout

Type: timeout
Default: 30000

Maximum quanitty of time in milliseconds to wait until consider the URL is not reachable.

followRedirect

Type: boolean
Default: true

Defines if redirect responses should be followed automatically.

whitelist

Type: string[]

A set of URLs that can be ignored.

selector

Type: string

Specify a CSS Classname selector for getting the URLs from HTML markup.Specify a CSS Classname selector for getting the URLs from HTML markup.

License

urlint © Kiko Beats, released under the MIT License.
Authored and maintained by Kiko Beats with help from contributors.

kikobeats.com · GitHub @Kiko Beats · Twitter @Kikobeats