@mapbox/watchbot-progress

    1.1.6 • Public • Published

    Build Status

    watchbot-progress

    watchbot-progress a series of tracking tools available on reduce-enabled Watchbot stacks to track the progress of distributed map-reduce operations. See the included examples.md for some basic worker recipes.

    Installation

    npm install watchbot-progress
    

    Using the CLI command

    A CLI command is available to report progress to Watchbot, and is accessible by default on reduce-enabled Watchbot stacks.

    $ watchbot-progress --help
    
      Tracks the progress of a distributed map-reduce operation
    
      USAGE: watchbot-progress <command> <job-id> [options]
    
      Environment variable $ProgressTable must be set as the ARN for the DynamoDB
      table that is used to track progress.
    
      COMMANDS:
        - status: check the status of a job
        - set-total: set the total number of parts in a job
        - set-metadata: set arbitrary metadata on the job record
        - complete-part: complete a single part
        - fail-job: mark a job as a failure
    
      OPTIONS:
        -t, --total     (for set-total) the total number of parts in a job
        -p, --part      (for complete-part) the part number to mark as complete
        -m, --metadata  (for set-metadata) the JSON metadata object to store
        -r, --reason    (for fail-job) a description of why the job failed
    

    Note that by default, workers in reduce-enabled Watchbot stacks will have the $ProgressTable environment variable set automatically.

    Creating a progress table

    A JavaScript module is available to create a DynamoDB table. You will need to provide any CloudFormation permissions.

    var table = require('watchbot-progress').table;

    This function requires 2 parameters:

    • name: DynamoDB table name
    • throughout: Object that contains user specification for progress table read and write capacity units:
      • throughput.readCapacityUnits: Approximate number of reads per second from progress table
      • throughput.writeCapacityUnits: Approximate number of writes per second to progress table

    Reporting progress in JavaScript

    A JavaScript module is also available as a mechanism for progress reporting.

    var progress = require('watchbot-progress').progress;
    • progress.setTotal(jobId, total, [callback]): Set the total number of parts for a particular map-reduce operation.
    • progress.completePart(jobId, part, [callback]): Mark a single part as complete. The response will be a boolean value indicating whether or not the operation is completed.
    • progress.setMetadata(jobId, metadata, [callback]): Associate arbitrary metadata with a particular map-reduce operation.
    • progress.status(jobId, [callback]): Read the status of a particular map-reduce operation.
    • progress.failJob(jobId, reason, [callback]): Mark an operation as a failure, providing a description of what went wrong.

    Keywords

    none

    Install

    npm i @mapbox/watchbot-progress

    DownloadsWeekly Downloads

    5,733

    Version

    1.1.6

    License

    ISC

    Unpacked Size

    46.4 kB

    Total Files

    14

    Last publish

    Collaborators

    • mapbox-admin
    • mapbox-machine-user
    • mapbox-npm
    • mapbox-npm-01
    • mapbox-npm-02
    • mapbox-npm-03
    • mapbox-npm-04
    • mapbox-npm-05
    • mapbox-npm-06
    • mapbox-npm-07
    • mapbox-npm-08
    • mapbox-npm-09
    • mapbox-npm-advanced-actions
    • mapbox-npm-ci