Have ideas to improve npm?Join in the discussion! »

    nuxt-generate-cluster

    2.7.0 • Public • Published

    Multi-threaded generate command for Nuxt.js

    Build Status Windows Build Status Coverage Status npm npm (scoped with tag)

    Use multiple workers to generate the static files for your Nuxt.js project

    Setup

    Install the package

    yarn add nuxt-generate-cluster
    

    Add a generate script to your package.json

      "scripts": {
        "generate": "nuxt-generate -w 4"
      }

    Nuxt config options

    Configure the generate options in nuxt.config.js

      generate: {
        workers: 4,
        workerConcurrency: 500,
        concurrency: 500,
        routes (callback, params) {
          return axios.get('https://api.example.com/routes?since=' + params.lastFinished)
          .then((res) => {
            return res.data
          })
        },
        done ({ duration, errors, workerInfo }) {
          if (errors.length) {
            axios.post('https://api.example.com/routes', { generate_errors: errors })
          }
        }
      }

    workers

    • Default: number of processors

    The number of workers that should be started. It probably has no use to start more workers then number of processors in your system.

    workerConcurrency

    • Default: 500

    To even the load between workers they are sent batches of routes to generate, otherwise a worker with 'easy' routes might finish long before others. Workers will also still use the concurrency option from Nuxt.

    routes

    The default Nuxt.js routes method has been extended so you can pass additional parameters to it, see params parameter in example config under Setup. By default it will list 3 timestamps:

    • lastStarted The unix timestamp when the nuxt-generate command was last executed, should be just now

    • lastBuilt The unix timestamp when the nuxt project was last built by nuxt-generate

    • lastFinished The unix timestamp when nuxt-generate last finished succesfully (eg not interrupted by ctrl+c)

    Timestamps are locally stored in ~/.data-store/nuxt-generate-cluster.json, see data-store

    beforeWorkers

    This method is called on the master just before the workers are started/forked. It receives the Nuxt options as first argument.

    Use this method if you experience serialization issues or when your Nuxt config is too big. The Nuxt options are stringified and then passed as environment variable to the workers, on Windows there seems to be a maximum size of 32KB for env variables.

    Properties which should be safe to remove are (not a complete list):

    • buildModules (and their options)
    • serverMiddleware

    done

    This method will be called when all workers are finished, it receives two arguments:

    • The first argument is an object with statistics:

      • duration The total time in seconds that the command ran

      • errors An array of all the errors that were encountered. Errors can have type handled or unhandled, for the latter the error message will contain the stacktrace

      [ { type: 'handled',
          route: '/non-existing-link',
          error:
           { statusCode: 404,
             message: 'The message from your 404 page' } }
      ]
      • workerInfo An object with detailed information about each worker. Data passed is from the watchdog object that we use internally to monitor the worker status.
      {{ '6707':                            // the process id of the worker
         { start: [ 1929158, 859524606 ],   // process.hrtime the worker was started
           duration: 109567,                // how long the worker was active
           signal: 0,                       // the signal by which the worker was killed (if any)
           code: 0,                         // the exit status of the worker
           routes: 73,                      // the number of routes generated by this worker
           errors: [] },                    // the errors this worker encountered, errors of all workers
                                            // combined is the error argument above
      }
    • The second argument is the Nuxt instance

    Command-line options

    Please note that you need to explicitly indicate with -b that you want to (re-)build your project

    $ ./node_modules/.bin/nuxt-generate -h
    
      Multi-threaded generate for nuxt using cluster
    
      Description
        Generate a static web application (server-rendered)
      Usage
        $ nuxt-generate <dir>
      Options
        -b, --build           Whether to (re-)build the nuxt project
        -c, --config-file     Path to Nuxt.js config file (default: nuxt.config.js)
        -h, --help            Displays this message
        -p, --params          Extra parameters which should be passed to routes method
                                (should be a JSON string or queryString)
        -q, --quiet           Decrease verbosity (repeat to decrease more)
        -v, --verbose         Increase verbosity (repeat to increase more)
        --fail-on-page-error  Immediately exit when a page throws an unhandled error
        -w, --workers [NUM]   How many workers should be started
                                (default: # cpus)
        -wc [NUM],            How many routes should be sent to
        --worker-concurrency [NUM]    a worker per iteration
    
    

    If you need to have more control which routes should be generated, use the -p option to pass additional parameters to your routes method.

    # nuxt.config.js
    generate: {
      routes (callback, params) {
        console.log(params)
      }
    }
    
    $ nuxt-generate -w 2 -p id=1&id=2
    // will print =>
    { id: [ '1', '2' ],
      lastStarted: 1508609323,
      lastBuilt: 0,
      lastFinished: 0 }
    

    If you are using a npm script under bash use -- to pass the parameters to nuxt-generate instead of npm:

    $ npm run generate -- -p '{ "id": [1,2,3] }'
    // will print =>
    { id: [ 1, 2, 3 ],
      lastStarted: 1508786574,
      lastBuilt: 0,
      lastFinished: 0 }
    

    Logging

    You can use multiple -v or -q on the command-line to increase or decrease verbosity. We use consola for logging and set a default log level of 3 unless DEBUG is set, then its 5. If you want to log debug messages without setting DEBUG you can use -vv on the command-line

    The difference between log levels 2, 3 and 4 are:

    • Level 2 (-q) Only print master messages like worker started / exited. No worker messages.

    • Level 3 Also print which routes the workers generated

    • Level 4 (-v) Also print how much time the route generation took and messages between master and workers

    Install

    npm i nuxt-generate-cluster

    DownloadsWeekly Downloads

    2,625

    Version

    2.7.0

    License

    MIT

    Unpacked Size

    202 kB

    Total Files

    32

    Last publish

    Collaborators

    • avatar
    • avatar
    • avatar