Natural Push Machine

    bgtm

    2.1.2 • Public • Published

    Blanksy's Gulp Task Manager (BGTM)

    Breaking Changes

    As of version 2, there are a number of breaking changes:

    Gulp 4 Support

    Support for Gulp 4.x has broken support for verion 3.x, you must ensure you are using the latest gulp-cli as well.

    Engine Changes may require updates

    Engines need to be updated to include a 3rd parameter. This third parameter is used if you are using the streams for your engine to allow for correctly returning a complete state to gulp. See the 'Engine' section for further information.

    Introduction

    This is a wrapper for gulp that allows for a consistent template-based approach to project setup for gulp absed on your development teams standards for build processes.

    Why?

    This task manager is a solution to a problem that has been faced countless times in development when dealing with large amounts of projects and a constantly changing team of developers and it usually consists of 3 main challenges that are faced, these are:

    Consistency & Team Best Practices

    Challenge

    You have a team, they have outlined a build process including tasks that they consider best practice, however these tasks are in their own files, still in gulpfile.js and they clone this from project to project.

    What generally happens with this process is that you copy it from project to project, and you tweak it to optimise some steps by changing a build step here, a build step there and then all of a sudden you have all your projects all similar but all slightly different in subtle ways, they have been updated for the latest best practices but the changes were not ported back to the other projects.

    This inconsistency can be an absolute pain for maintaining a certain level of quality across all your projects.

    Solution

    The concept that this framework goes with is to create engines to solve this problem.

    What are engines? You can think of them as an NPM Package containing your best practices for a specific build pipeline.

    Example: You currently have a build pipeline that does the following:

    • Gets the source file(s)
    • Concatenate the source files into a single file
    • Minify the file to reduce the filesize.
    • Generate the sourcemaps for that file.
    • Ensure that the file always has an extension of .min.js
    • Always have the sourcemap have the same slug as the main file
    • Output the new minified file.

    This pipeline can be considered to be a set of best practices you follow and are the rules that make up an engine that should apply to all your projects.

    Maintainability of the Build Process Best Practices

    So we have talked about the Consistency and Best Practices which brings us on to the Maintainability of the build process which constantly shifts as new concepts come out and new idea's are formed.

    Challenge

    Say that maintain a large number of projects or have projects being outsourced and you want to keep a consistent standard for your builds.

    What can happen is a developer modifies the build steps in one project.

    This then brings that project out of sync with your established best practices and will have the potential to cause chaos a little down the line when you go to integrate it with CI, pick the project up for maintenance expecting it to work one way but works another entirely different way.

    But now you want to port those changes back, to do that you now have to go through each project manually and that could lead to issues in one project but not the other.

    Solution

    Again with the concept of engines on your side and the power of npm you have the ability to update the underlying best practices you have specified and bring those projects up to date.

    For this you will need to install the following:

    npm install -g npm-check-updates
    

    And then run the following commands

    ncu -u
    npm install
    

    It does take a little change of process of course as you need to think if what you are adding is going to be a part of your best practices, you need to then checkout or even create the engine code and then publish this new engine into NPM.

    This may seem tiresome and a little long winded however you are actually setting a precedent by saying the rules that make up my best practices are important and should be considered before updating them.

    CI (Continuous Integration) Friendly

    Challenge

    Generally with CI implementation you want to keep it repeatable, as repeatable makes it very easy to setup new projects as the steps for building a project are based on a template that does a specific set of steps and relies on things being done in a similar way.

    Because the setup process is templated and repeatable you want to be able to ensure that the build files all have the same base level tasks as well to ensure that when you want to build a production ready version you can do so, when you want to trigger a build you can.

    Solution

    The following tasks are ALWAYS available:

    • build
    • compile (alias of build)

    There is a --production and --environment argument that can be passed in for the purposes of only doing specific build steps during the build as part of the engine makeup.

    So as a CI implementor you would implement the frontend build by running:

    gulp build
    

    or

    gulp build --production
    

    or

    gulp build --environment <ENVIRONMENT_NAME>
    

    Getting Started

    To get started create your project using

    cd /to/your/project/directory/
    npm init
    npm i --save-dev gulp bgtm
    

    How to structure your gulpfile.js

    var gulp = require('gulp');
    var tm = require('bgtm')(gulp);
    
    ... Any task require statements ...
    
    ... Any tasks to be added to the manager ...
    
    // ALWAYS run this method last.
    tm.run();
    

    Tasks

    The task manager contains tasks, just like gulp. Except that the manager has rules surrounding how the tasks can be run.

    The following tasks are ALWAYS defined automatically without you having to add them:

    • compile
    • build
    • watch
    • default

    compile is an alias of build.

    build is the default task that runs all the tasks needed to build a project, this is specified as part of the task addition.

    watch sets the file watchers up to run a specific task.

    default ALWAYS runs the build task then it runs the watcher to watch for file changes.

    Adding a task

    The below is an example only

    First install the engine npm package:

    npm i --save-dev bgtm-engine-scss
    

    Then load the dependency:

    var scss = require('bgtm-engine-scss');
    

    Then add the engine to the task manager:

    tm.add('taskName', {
        runOnBuild: false,
        runOnBuildEnvironment: 'all',
        watch: false,
        watchSource: [],
        liveReload: false,
        engine: scss,
        engineOptions: {
            src: ['static/css/src/main.scss'],
            dest: 'static/css/
        }
    })
    

    Option Outline

    Option Type Value
    runOnBuild boolean Adds this task to the list of tasks that should be run when gulp build is run.
    runOnBuildEnvironment string or array The ability to only run this task when a specific environment is specified. You add the environment or environment names you want to be able to run this task for.
    watch boolean Adds this task to the list of tasks that have files being watched and should be actioned when you update a certain set of files.
    watchSource array An array of patterns for the files/file types to be matched when watching for file changes.
    liveReload boolean Set this to true if you are using livereload to automatically update your browser when the task has finished updating the files.
    engine function The callback (npm package or custom function) that can be used to run the gulp pipeline.
    engineOptions object An object containing all options for the enging to function, such as source and destination folders etc.

    Group tasks into a Sequence Group

    You can also group tasks so that they can be run in the order that they appear by using the sequence task name.

    There are some limitations:

    • sequence groups cannot live reload and this is turned off for the groups
    • the tasks in the group cannot be run on build, watched or live reload.

    To use this functionality you can wrap them in the following construct:

    task.addSequence(
        'static-files',
        {
            runOnBuild: true,
            watch: true,
            liveReload: true
        },
        function (tm) {
    
            // Normal tasks get added here.
    
        }
    );
    

    addSequence Options

    Code

    task.addSequence(taskName, sequenceOptions, callback);
    

    Options

    Option Type Value
    taskName string The name to give the sequence so that it can be run independently.
    sequenceOptions object The options to pass to the sequence on when to run.
    callback function The callback used to wrap the tasks, it contains one parameter which is the instance of BGTM.

    sequenceOptions

    Option Type Value
    runOnBuild boolean Adds this task to the list of tasks that should be run when gulp build is run.
    runOnBuildEnvironment string or array The ability to only run this task when a specific environment is specified. You add the environment or environment names you want to be able to run this task for.
    watch boolean Adds this task to the list of tasks that have files being watched and should be actioned when you update a certain set of files.
    watchSource array An array of patterns for the files/file types to be matched when watching for file changes.

    Running the tasks

    You would run the tasks exactly the same way you would run a normal gulp process.

    gulp taskName
    

    Environment Mode

    The task manager has an environment system that allows for customisation of actions and what can run on build based on the environment you specify during build.

    There are 3 default environment reserved words.

    ENVIRONMENT JS ACCESS
    all tm.environment.ALL
    development tm.environment.DEVELOPMENT
    production tm.environment.PRODUCTION

    You can also specify your own by simply implementing this in the:

    • task option runOnBuildEnvironment value.
    • hard-wiring it as part of your own custom engine.

    How to change environment mode

    If the environment argument is not specified then the default environment is development

    You can change the environment mode that the gulp task runner is running under by using either:

    --environment <ENVIRONMENT>
    

    or

    --production
    

    The functionality of --environment production and --production are exactly the same and are interchangeable, so use whichever you prefer.

    The reason for the difference is that the production flag has a special function in that it also sets the process environment for node to production mode as well.

    How runOnBuildEnvironment words

    This is an option that is available for every task that only adds the task to the global build task when that specific environment is run.

    By default this is set to all which means that it runs no matter what environment is specified using the --environment <ENVIRONMENT> argument.

    If you wish to only run a task in development mode then you can specify this value as being development.

    This is very useful if you want a copy task to be only run during development but not during the CI processes for production / release etc.

    e.g. The following task will only run when the development and release environments are specified during the gulp runner.

    tm.add('taskName', {
        runOnBuild: true,
        runOnBuildEnvironment: [
            'development',
            'release'
        ],
        ...
    })
    

    To run this task you will need to use either:

    gulp
    gulp --environment development
    gulp --environment release
    gulp build
    gulp build --environment development
    gulp build --environment release
    

    The following will not run this task

    gulp build --production
    gulp build --environment production
    gulp build --environment anyotherenvironment
    

    How to use the environments in an engine.

    You can read more on building engines below, however the code effectively equates to the following:

    if(tm.isEnvironment(tm.environment.PRODUCTION)) {
        // Perform the action you want for production environment.
    }
    if(tm.isEnvironment(tm.environment.DEVELOPMENT)) {
        // Perform the action you want for development environment.
    }
    if(tm.isEnvironment('release')) {
        // Perform the action you want for custom release environment.
    }
    

    Engines

    Pre-Existing Engines

    There are a number of engines already built that can be used if you find that they match your own best practices, however these may not suit your purposes.

    Creating an Engine

    Engine Types

    Stream Based

    Stream based engines are just standard tasks created in the same way as normal with gulp, except in a package which can be reused without changing the rules behind it for consistency in the way it works.

    Standalone

    A standalone engine is an engine that does not need to use the stream but can be used independently without the need of the gulp stream.

    Using your new engine

    Create the index.js and package.json files for your project like a normal NPM package.

    Ensure that any dependencies are added to your package.json file.

    Example Engine Code

    package.json
    {
      "name": "bgtm-engine-example",
      "version": "0.0.1",
      "description": "",
      "author": "Blanksy",
      "license": "MIT",
      "dependencies": {},
      "main": "index.js",
      "devDependencies": {
        "gulp-plumber": "^1.1.0",
        "gulp-rename": "^1.2.2"
      }
    }
    
    index.js
    Stream Based
    var plumber = require('gulp-plumber');
    var rename = require('gulp-rename');
    
    module.exports = function(taskManager, engineOptions, done) {
        var task = this.src(engineOptions.src);
        task.pipe(plumber());
        if(tm.isEnvironment(tm.environment.PRODUCTION)) {
            // When in production mode rename the files.
            task.pipe(rename({extname: '.min.css'}));
        }
        task.pipe(this.dest(engineOptions.dest));
        task.pipe(done());
        return task;
    };
    
    Standalone
    module.exports = function(taskManager, engineOptions, done) {
        // Sample Async Call
        setTimeout(function() {
            done();
        }, 10000);
    };
    

    If you need to access gulp this can be achieved by using this as part of the engine code.

    There are also 3 parameters to be aware of that make up the engine itself, these are:

    Parameter Type Value
    taskManager TaskManager The instance curently running for BGTM itself.
    engineOptions object The object containing all the options passed into the engine.
    done function This is the callback that is called when your engine has finished its processing. If you are using a stream based engine use .pipe(done()) otherwise you can use done() when your engine has finished processing.

    When you are happy with the engine, commit the code to git and then do an npm publish

    Inline Callback (Not Recommended)

    Whilst I do not recommend this approach as it goes against the spirit of this manager, it is important to outline that this can be done and how to do it.

    In the engine option for the task addition you can (should not) specify a callback function as follows:

    tm.add( 'taskName', {
        runOnBuild: true,
        watch: true,
        watchSource: [
            'static/fonts/**/*'
        ],
        liveReload: false,
        engine: function(taskManager, engineOptions, done) {
            var task = this.src(engineOptions.src)
            task.pipe(plumber());
            if(tm.isEnvironment(tm.environment.PRODUCTION)) {
                // When in production mode rename the files.
                task.pipe(rename({extname: '.min.css'}));
            }
            task.pipe(this.dest(engineOptions.dest));
            task.pipe(done());
            return task;
        },
        engineOptions: {
            src: 'static/fonts/**/*',
            dest: 'built/static/fonts/'
        }
    });
    

    As you can see however this is exactly the same format as you building your engine into its own package and it would be more ideal for you to build your own package as mentioned above and do an npm publish on it so that you can get all the benefits of being able to update the package across many many projects easily.

    License

    Copyright 2017-2019 Ben Blanks

    Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

    The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

    Keywords

    none

    Install

    npm i bgtm

    DownloadsWeekly Downloads

    21

    Version

    2.1.2

    License

    MIT

    Unpacked Size

    33 kB

    Total Files

    5

    Last publish

    Collaborators

    • blanksy
    • oldben