@tryghost/mg-assetscraper

Migrate Assetscraper

Install

npm install @tryghost/mg-assetscraper --save

or

yarn add @tryghost/mg-assetscraper

Usage

// Default
const assetScraper = new AssetScraper(fileCache, {}, {
    logger: myLogger
});

// Later on, create the Listr tasks
const tasks = ctx.imageScraper.fetch(ctx);

// Create the Listr task runner, ensuring its not concurrent:
// Each task *must* be run sequentially
const taskRunner = makeTaskRunner(tasks, {concurrent: false});

// And later still, run those tasks
await taskRunner.run();

// `ctx` is now updated

// Set some options
// 2 MB maximum, and no media files
const assetScraper = new AssetScraper(fileCache, {
    sizeLimit: 2, // 2 MB
    allowMedia: false,
    baseDomain: 'https://example.com' // Set this domain to be added to relative asset links
}, {
    logger: myLogger
});

// Don't scrape assets from this URL
assetScraper.addBlockedDomain('https://my-custom-cdn.example.com');

// Now create & run tasks

Develop

This is a mono repository, managed with lerna.

Follow the instructions for the top-level repo.

1. git clone this repo & cd into it as usual
2. Run yarn to install top-level dependencies.

Run

• yarn dev

Test

• yarn lint run just eslint
• yarn test run lint and tests
• yarn benchmark run benchmarks

Copyright & License

Copyright (c) 2013-2023 Ghost Foundation - Released under the MIT license.