Natively Pronounced Mandarin

    laoban
    TypeScript icon, indicating that this package has built-in type declarations

    0.2.13 • Public • Published

    Laoban

    Laoban or 老板 is chinese for 'boss'. It is a tool for controlling multiple projects. While it is language agnostic it probably offers the most value to javascript/typescript projects

    NPM usage

    NPM does not handle multiple projects well. Each project is a separate project.json that is managed separately. There is no ability to 'inherit' or share configuration, so in a project with many moving parts each of which is implemented with a small bit of javascript/typescript, it can be difficult to keep all the dependancies in line.

    Laoban makes the following easy:

    • Managing config files
      • There are a number of template files (might be just one)
        • These holds files that are copied to the project whenever 'laoban update' is called
        • The package.json in it is 'modified' during the copying based on a file called 'project.details.json' in the project
        • In my projects these files are things like:
          • jest.config.json
          • babel.config.json
          • tsconfig.json
          • the jest adapter for the version of jest
    • Executing things in parallel across all projects
      • tsc: to compile all the typescript
      • npm test: to run all the tests
      • npm install: to make sure everything is loaded
      • npm
      • Any command at all...
    • It keeps track of the status of important things: such as last test execution, last compile, last install

    Other package managers

    Laoban is not opinionated. Replaceing npm with yarn in the config will let you use all the features with yarn. If you want to use it with maven or sbt or... it works fine (although those tools already have much of the capabliities that laoban brings to the javascript world)

    What are the 'golden moments'

    • Running all the tests in parallel across multiple projects
      • Without this I have to either use a pipeline after a commit, or make a script to call them one at a time
    • Seeing the status of the important commands
      • When working with ten or more projects I found it very hard to get a simple of view of how well the code was behaving in each project
    • Updating all the 'react' project settings in one go
      • You can update the template settings, call laoban update followed by laoban install and laoban status
      • Now you know how all the projects have responded to the upgrade: they are all using it, and they have been compiled and tested
    • Updating a global version number
      • If the projects are tightly coupled, I like them to share a version number.
    • When the commands take a long time you can see the tail of the logs of the commands easily
      • Press ? while the commands are running for a menu

    Typical usage

    When loading a project with many subprojects from git

    • git clone the project
    • laoban install will setup and test the subprojects in parallel

    When publishing

    • Change the version in the template directory
    • laoban update will update all the projects
    • laoban publish will publish all the projects

    Important ideas

    laoban.json

    This is a file that configures laoban. The variable values can be seen by laoban config

    The existance of the file marks that this is the root of a 'big project' which is composed of one or more sub projects

    It holds:

    • "templateDir": the directory that the templates can be found in
    • "log": the name of the log file found in the project directories holding the log of executing the commands
    • "status": the name of the file (in each project directory) that holds the status of 'important commands' executed
    • "scriptDir": A place for bash scripts that can be accessed by the laoban commands. You can put your own scripts here
    • "packageManager": defaults to npm

    Templates

    • under templateDir. Each template is a directory holding files that are used by the update command

    variables

    The syntax ${variableName} allows access to variables. These can be used

    • In other variables
    • In commands

    Legal variables are the variables in laoban.json and the variables in the project.details.json. Examples are

    • laoban run 'echo ${projectDetail.template}' -a. Note the use of '' which is used to prevent bash from attemption to dereference the variables
    • laoban run 'echo ${scriptDir}'

    When debugging executing laoban <scriptname> -adsv can provide a lot of help. This will show the value of the command being executed in each direcoty, and if the variable is not 'correct' you can often see what are legal values, and why it's not working

    project.details.json

    {
      "template"      : "noreact",
      "name"          : "@phil-rice/lens",
      "description"   : "A simple implementation of lens using type script",
      "projectDetails": {
        "generation"  : 0,
        "publish"     : true,
        "links"       : [],
        "extraDeps"   : {},
        "extraDevDeps": {}
      }
    }
    

    If this is present in a directory it tells laoban that the directory is a project directory.

    • template is the name of the subdirectory that holds the configuration files that laoban will place in the project
    • name is the name of the project. This is injected into package.json by update
    • description is the name of the project. This is injected into package.json by update
    • extraDeps are the names of dependancies that this project needs and are to be added to the template
    • extraDevDeps are the names of developer dependancies that this project needs and are to be added to the template
    • links are used within the 'master project' that laoban is looking after. * It allows laoban to set up symbolic links so that changes in one project are immediately reflected * These are added as dependencies to the project, with the 'current version number'
    • publish should this project be affected by commands with the guard condition ${projectDetails.details.publish} * Typically these are projects to be published to npmjs * typicall commands are laoban pack, laoban publish, laoban ls-publish
    • generation the projects are sorted in generation order so that all generation 0 projects are processed before generation 1 * See the 'TODO' section at the end: generations are only respected in display order at the moment
    • throttle sets the maximum number of parallel activites that will be executed. The default is 0 which doesn't limit things

    Scripts

    Scripts are lists of commands (sometimes just one) that are executed when you type laoban scriptName.

    • Scripts can be run in more than one directory at once. Most commands run in 'all projects' unless the current directory is a project directory, in which case they run in the current * -a means 'run in all project directories' * -1 means 'just run in this directory'
    • Scripts have access to variables

    OsGuard

    Scripts can be marked so that they only run in a particularly OS. Examples can be seen in the laoban.json. If called from the wrong os then an error is given. guardReason can be set to give an error message (and document why)

        "pack"       : {
           ...
          "osGuard":  "Linux",
          "guardReason": "uses the linux 'cp' command",
          "commands"   : [
              ....
          ]
        },
    

    pmGuard

    If a script requires a particular package manager (example npm test and yarn test are both OK but yarn install is not allowed), then a pmGuard can be set. If the command is executed then it will give an error message. As with osGuard, guardReason can be set to document why

    guard

    A command can be set to only execute if a guard is defined. Unlike the osGuard and pmGuard this does not cause an error message: only scripts that match are executed. The example of ls-ports here:

        "ls-ports"  : {
          "description": "lists the projects that have a port defined in project.details.json",
          "guard"      : "${projectDetails.details.port}",
          "commands"   : ["js:process.cwd()"]
        },
    

    Another good example is

        "start"     : {
          "description": "${packageManager} start for all projects that have a port defined in project.details.json",
          "guard"      : "${projectDetails.details.port}",
          "commands"   : ["${packageManager} start"],
          "env"        : {"PORT": "${projectDetails.details.port}"}
        },
    

    So by setting 'ports' to a numeric value in the project.details.json we have 'marked' the directory in such a way that executing laoban start will start up the project. This lets us spin up multiple react projects at once. It's a good idea if all the projects have different ports...

    environment variables

    cwd is added as an environment variable to represent the current directory other environment variables can be added to scripts such as

        "ls-pact":        {
          "osGuard":     "Windows_NT",
          "description": "lists the projects with pact files in them",
          "guard":       "${projectDetails.details.packport}",
          "commands":    ["echo %PORT%  %cwd%"],
          "env":         {"PORT": "${projectDetails.details.packport}"}
        },

    inLinksOrder

    Some scripts don't parallelise that well. For example if we are compiling projects, and some projects depend on other projects then we want to compile them in 'the right order'.

    Setting 'inLinksOrder' means that the links in the projectDetails are used to determine the order in which things are executed

    This can be seen using '-g | --generationPlan' as an option. This behavior can also be forced on any command by selecting -l, --links

    env

    If a command needs access to environment variables (for example a port) these can be added. It is not uncommon to have a guard condition on the command.

    Commands

    These are added to laoban by means of the laoban.json file. An inspection of it should give you a good idea of how to add your own command. Each command can have multiple sections or steps. Each step is executed in it's own shell so for example changing directory or setting environment variables in one step will not impact others

    Simple commands

        "log"       : {"description": "displays the log file", "commands": ["cat ${log}"]},
    

    After adding this command laoban --help will now display the command and the description. The command simply executes cat ${log}.

    Commands with step names and (optional) status

    "test"      : {
    "description": "runs ${packageManager} test",
    "commands"   : [{"name": "test", "command": "${packageManager} test", "status": true}]
    },
    

    Here we can see that the command has one step with name test. Because status is true the step results will be visible in the status

    Javascript

    If the text of a command starts with js: then the command will be executed in javascript.

    Examples

    • js:process.cwd()
    • js:"Hello World"

    This is primarily for js:process.cwd() so that we can run scripts on both windows and linux that want to show the current directory

    directory

    If a command needs to run in a different directory (typically a sub directory) the directory can be set

       "install"    : {
          "commands"   : [
            {"name": "link", "command": "${packageManager} link", "status": true, "directory": "dist"},
    

    This link command is now executed in the dist sub directory

    eachLink

    Some commands need to be accessed once for each link defined in the projectDetails file.

        "remoteLink"   : {
    ...
          "commands"       : [
            {"name": "remoteLink", "command": "${packageManager} link ${link}", "eachLink": true, "status": true},
    

    Here we can see that the command will be executed once for each link. The variable ${link} holds the value of the link

    osGuard and pmGuard

    Scripts can also be set with these. If set at the script it is used to say 'this script will throw an error if you try and use it'. When set at the command level the command is ignored if the guard is not valid. This can be used to give multiple implementations for different operating systems / package managers

        "install": {
          "description"    : "(not working yet ) doing the initial updateConfigFilesFromTemplates/install/link/tsc/test... etc in each project",
          "commands"       : [
    ...
            {"name": "copyPackageJsonLinux", "command": "cp package.json dist", "osGuard": "Linux"},
            {"name": "copyPackageJsonWindows", "command": "copy package.json dist", "osGuard": "Windows"},
          ], "inLinksOrder": true
    

    Monitoring

    While laoban is running you can press ? to get an interactive menu.

    • You can press (capital) S to see the status of all your scripts as they are executed in different directories
      • The status includes 'which commands have finished', 'how long they ran for'
    • You can press (capital) L for the names of the log files if you want to do something like tail -f
    • You can press a number (or letter if there are more than 10 directory) to see the tail of the log for that log

    options

    option -a

    The -a means 'in all projects'. Without this laoban looks at the current directory

    • If it contains a project.details.json, the command is executed in this directory only
    • If it doesn't contain a project.details.json the command is executed as though -a had been specified

    options -p <project>

    You can give a regex for the project name and the command will be executed in those projects

    Example

    • You are in a project X that depends on another project Y
    • you type laoban tsc -p X
      • Now the tsc is executed in project X

    option -s

    This allows a bit of debugging of scripts. If you are having problems adding -s gives a little more information about what is happening

    This is a great option when you want 'information from many places'. Like

    • laoban run 'ls *.config' -as

    option -d

    This is a dryrun. Instead of executing the command, it is just printed. This includes dereferencing the variables. A combination of -ds gives quite nice information about what is executing

    option -v

    Only used when debugging to help work out what are legal variables

    option -g

    Rather like '-a' in that it does not display any commands. Instead it outputs the 'generation plan': the directories that will be processed in parallel.

    option -t xxx

    Sets the maximum number of items executed at once, so that the computer doesn't get over loaded. This overrides the setting in laoban.json

    status

    The idea of status is to give a way to visualise what is happening across all the sub projects

    • Some commands are marked as 'status: true'
    • When these are executed they are recorded in the status file for that project
    • laoban status will display the latest status
    • laoban compactStatus will crunch the status down

    TODO

    Make easily available on npmjs with name laoban

    • Have reserved name
    • Need to make it so that we can just mention it and it gets installed.

    Monitor improvements

    • Had a situation where everything was finished but one thing that was hanging
    • It would be nice to be able to 'kill' the one thing that is hanging

    Move to the ts project so we can easily dogfood

    • Consider if this is a good idea

    Make the laoban.json composible

    • Because it can get very big...
    • and we want to be able to be able to reuse bits across projects

    Need a plugin story so that we don't have to shell out for common things

    • For example updating package json... only important for npm/yarn. Currently done by a script
    • We want to be able to use other people's scripts easily

    Bugs?

    check laoban pack and loaban pack -a. Not sure why isn't executing

    Install

    npm i laoban

    DownloadsWeekly Downloads

    51

    Version

    0.2.13

    License

    MIT

    Unpacked Size

    135 kB

    Total Files

    45

    Last publish

    Collaborators

    • phil-rice