dask-labextension
    TypeScript icon, indicating that this package has built-in type declarations

    5.1.0 • Public • Published

    Dask JupyterLab Extension

    Build Status Version Downloads Dependencies

    This package provides a JupyterLab extension to manage Dask clusters, as well as embed Dask's dashboard plots directly into JupyterLab panes.

    Dask Extension

    Explanatory Video (5 minutes)

    Dask + JupyterLab Screencast

    Requirements

    JupyterLab >= 1.0 distributed >= 1.24.1

    Installation

    To install the Dask JupyterLab extension you will need to have JupyterLab installed. For JupyterLab < 3.0, you will also need Node.js version >= 12. These are available through a variety of sources. One source common to Python users is the conda package manager.

    conda install jupyterlab
    conda install -c conda-forge nodejs

    JupyterLab 3.0 or greater

    You should be able to install this extension with pip or conda, and start using it immediately, e.g.

    pip install dask-labextension

    JupyterLab 2.x

    This extension includes both client-side and server-side components. Prior to JupyterLab 3.0 these needed to be installed separately, with node available on the machine.

    The server-side component can be installed via pip or conda-forge:

    pip install dask_labextension
    conda install -c conda-forge dask-labextension

    You then build the client-side extension into JupyterLab with:

    jupyter labextension install dask-labextension

    If you are running Notebook 5.2 or earlier, enable the server extension by running

    jupyter serverextension enable --py --sys-prefix dask_labextension

    Configuration of Dask cluster management

    This extension has the ability to launch and manage several kinds of Dask clusters, including local clusters and kubernetes clusters. Options for how to launch these clusters are set via the dask configuration system, typically a .yml file on disk.

    By default the extension launches a LocalCluster, for which the configuration is:

    labextension:
      factory:
        module: 'dask.distributed'
        class: 'LocalCluster'
        args: []
        kwargs: {}
      default:
        workers: null
        adapt:
          null
          # minimum: 0
          # maximum: 10
      initial:
        []
        # - name: "My Big Cluster"
        #   workers: 100
        # - name: "Adaptive Cluster"
        #   adapt:
        #     minimum: 0
        #     maximum: 50

    In this configuration, factory gives the module, class name, and arguments needed to create the cluster. The default key describes the initial number of workers for the cluster, as well as whether it is adaptive. The initial key gives a list of initial clusters to start upon launch of the notebook server.

    In addition to LocalCluster, this extension has been used to launch several other Dask cluster objects, a few examples of which are:

    • A SLURM cluster, using
    labextension:
        factory:
          module: 'dask_jobqueue'
           class: 'SLURMCluster'
           args: []
           kwargs: {}
    • A PBS cluster, using
    labextension:
      factory:
        module: 'dask_jobqueue'
        class: 'PBSCluster'
        args: []
        kwargs: {}
    labextension:
      factory:
        module: dask_kubernetes
        class: KubeCluster
        args: []
        kwargs: {}

    Development install

    As described in the JupyterLab documentation for a development install of the labextension you can run the following in this directory:

    jlpm  # Install npm package dependencies
    jlpm build  # Compile the TypeScript sources to Javascript
    jupyter labextension develop . --overwrite  # Install the current directory as an extension

    To rebuild the extension:

    jlpm build

    You should then be able to refresh the JupyterLab page and it will pick up the changes to the extension.

    To run an editable install of the server extension, run

    pip install -e .
    jupyter serverextension enable --sys-prefix dask_labextension

    Publishing

    This application is distributed as two subpackages.

    The JupyterLab frontend part is published to npm, and the server-side part to PyPI.

    Releases for both packages are done with the jlpm tool, git and Travis CI.

    Note: Package versions are not prefixed with the letter v. You will need to disable this.

    $ jlpm config set version-tag-prefix ""

    Making a release

    $ jlpm version [--major|--minor|--patch]  # updates package.json and creates git commit and tag
    $ git push upstream main && git push upstream main --tags  # pushes tags to GitHub which triggers Travis CI to build and deploy

    Install

    npm i dask-labextension

    DownloadsWeekly Downloads

    1,535

    Version

    5.1.0

    License

    BSD-3-Clause

    Unpacked Size

    102 kB

    Total Files

    18

    Last publish

    Collaborators

    • jacobtomlinson
    • ian-r-rose