@devbookhq/docsets-pipeline-manager

Docsets Manager

CLI for adding new documentation to Devbook. The CLI command name is docsets. You can start documentation scraping and indexing based on configs in the devbook-docsets repository with the create sub-command. Then you can release documentation that finished scraping and indexing with the release sub-command. If you want to update the documentation with new data, you run the create sub-command again and then update the documentation with the update-release sub-command.

Installation

To install the CLI run the following command:

$ npm i @devbookhq/docsets-pipeline-manager@latest -g --unsafe-perm=true --allow-root

You need to set one environment variable before you use the CLI:

1. GOOGLE_APPLICATION_CREDENTIALS - location of a JSON containing the GCP service account with permission to access PubSub and Secret Manager.

CLI Usage

How to scrape and index a documentation

The following command starts scraping and indexing documentation defined by configs <configsList...> to the default environment development:

$ docsets create <configsList...>

If you want to use a different environment then use the -e (--env) flag followed by the <env> which is either development or production.

$ docsets create <configsList...> -e <env>

If you want to also deploy VMs for scraping then use the -w (--workers) flag followed by the <workerCount>.

$ docsets create <configsList...> -w <workerCount>

WARNING: VMs created by specifying the CLI flag -w (--workers) are not yet automatically deleted. If you want the VMs deleted, you must do it manually by going to GCP console and deleting all instances named docsets-pipeline-worker-<environment>-<cuid> or by using the gcloud CLI.

If you want to scrape and index a documentation locally you need to have the docsets pipeline running locally and then use the -l (--local) flag.

$ docsets create <configsList...> -l

NOTE: The environment of the local docsets pipeline and the command environment modyfied by the -e (--env) <env> flag must match.

How to release a newly scraped and indexed documentation

The following command starts an interactive prompt for releasing a scraped and indexed documentation to the default environment development.

$ docsets release

If you want to use a different environment then use the -e (--env) flag followed by the <env> which is either development or production.

$ docsets release -e <env>

How to update an existing documentation

The following command starts an interactive prompt for updating a released documentation in the default environment development.

$ docsets update-release

If you want to use a different environment then use the -e (--env) flag followed by the <env> which is either development or production.

$ docsets update-release -e <env>

How to list all scraping and indexing works

The following command starts an interactive prompt for listing all finished, failed, and running works in the default environment development.

$ docsets list

If you want to use a a different environment then use the -e (--env) flag followed by the <env> which is either development or production.

$ docsets list -e <env>

Release

To update the CLI bump the version in package.json and run the following command:

$ npm publish