Noisy Pneumatic Mnemonic

    webjobs

    1.1.0 • Public • Published

    WebJobs

    Links

    Badges

    Circle CI Build

    CircleCI

    npm Version

    npm version

    npm big badge

    NPM

    Abstract

    A web worker that runs small scripts with a certain interface and wraps the work in a promise.

    The idea would be to figure out this mechanism and how to run tests in it as well then publish it.

    So for some reason they named threads web workers in the browser so I have named this package webjobs.

    So the idea is a library where you learn to kick small jobs to threading and then they notify you when it is done.

    How it works.

    What WebJobs(tm) does is load a thread and then pass it the url for the require library. It loads require and then the client sends it a job require path.

    A job is a simple object that has a dispatch method. with 3 parameters.

    1. parameters passed.
    2. workerId (unique id generated by library)
    3. callback to reply with result.
    define('jobs/TestJob', [], function() {
     
      /*
       * Just create a regular vanilla javascript class in requirejs format.
       *
       * A constructor
       * A prototype decalration with the method dispatch.
       * return the constructor at the end.
       *
       * Within dispatch you can call other styatic or instamce functions
       * but you have 2 main exits..
       *
       * callback({
       *   isError: true,
       *   payload: e
       * })
       *
       * OR
       *
       * callback({
       *   payload: result
       * })
       *
       */
     
     
      function TestJob() {}
     
      TestJob.prototype = {
        dispatch: function(workerId, params, callback) {
          callback({
            payload: 42
          });
        }
      }
     
      return TestJob;
    });

    Things Learned

    It is very import to set rules about the messages passed back and forth. I am sure the future holds some shared memory and locks to make things easier but for now it is entirely message based.

    Where the thread loads is where importScripts() starts relative paths. Some people thought it was where you spawned the Worker() constructor but I have confirmed that it is where the actual thread script resides.

    So if you load the thread from apps/src/webjobs/MyThread.js

    relative paths for importScripts() within that thread will be apps/src/webjobs/

    This is okay if you are doing development with a test website like mine but more complex for a production website. I believe I will have to make it configurable.

    Samples

    define('SomeProcessingModule',
      ['src/TroubleMaker'],
      function(TroubleMaker) {
        var jobPromise = TroubleMaker.start({
          jobPath: 'src/Addition',
          jobparams: {
            param1: 10,
            param2: 20
          }    
        });
     
        jobPromise.then(function(results) {
          console.log(results);
        }).catch(function(err) {
          console.error(err);
        });
      });

    Usage

    In the beginning I was going to write automated tests that excercised this library. But I found the tests would always timeout in playback and pass in debugging so I moved to a demo website to debug the kinks and be able to add features.

    REM this runs an express server that serves a basic test website that you can run several tests.
    node scripts/server.js

    For you to use it in your project there is a script for modifying the define() signature of the files.

    node scripts/modufy.js YourSiteRoot/thirdparty/jobs YourSiteRoot/thirdparty/src

    I am assuming everyone has their own toolchain for minimizing and I leave that up to you. You do however want to minimize the BaseThread.js separate from the other source when deploying to other projects.

    TroubleMaker

    At some point some night... I decided JobStarter and JobManager were too lame. and I said who is going to start and thought TroubleMaker and the name stuck in my head.

    the coolest thing is git made it easy to do this.

    Tasks

    • Get basic tests running with client code running webworkers.

    Dispatching

    The main key to managing a thread was devising a protocol to initialize, start, and complete work.

    Each message is an int because I prefer int compares to string compares for efficiency.

    Sequence

    Sequence

    Some enums that matter

    Enums

    And a state machine describing how a thread is managed.

    State

    Status

    6/09/2018

    Got tests running...got requirejs laoding in test not in regular browser... upon configure of requirejs...we cannot request the job maybe we need base/

    Late ... the requirejs is configured and we supposdely succeed in loading Job script but it is NOT being called. we cannot instantiate...

    6/10/2018

    SO if we give the full path to the job script the importScripts() call works and we get called back but the object is null because the define does not match the url...

    Very difficult problem to solve...

    10/3/2018

    Got testsite doing its first job. Got different operations working... Cleaned up some vulnerabilities. Putting Down.

    10/4/2018

    had the idea that the test site should have 4 or more sections with testers in them. A little tired to try coding right now.

    State machine should be less states.

    10/7/2018

    Started looking at layout to tester. This is what we get a new index.html with... https://codepen.io/mozilladevelopers/pen/JrReJE

    Maybe we should finally load a style sheet.

    Lets get the longing recusivejob working and then a xhr job...

    We got layout of 4 grid...now we need to figure out how to keep divs in parent div.

    10/11/2018

    Got layout for tester webapp working. 3 separate tests working.

    Next major hurdle...thread pooling and reinitializing...

    10/13/2018

    ThreadPooling first started working...

    10/19/2018

    Removed resolver dependency. Also installed rollup() which will be our packager. Will work on rollup later.

    10/20/2018

    Get rollup to build in a dist directory then build website that runs off of dist.

    10/21/2018

    Rollup is discarded we will just compress with uglify.

    10/31/2018

    new idea... instead of compressing...figure out how to map to different apps using different requirejs.config() and paths

    requirejs.config({
      'webjobs': 'src'
      webjobs_jobs: 'jobs'
    });

    11/24/2018

    The modify script is working. Now we should cleanout unused code and retest...

    12/18/2018

    The next demo we should build is a random generation of a bitmap. it should be a fractal algorithm or something... we should pass it back and display it on a canvas. using Uint8Array.

    3/14/2019

    Integrated with codebase at work and found I needed extra configuration fields.

    TroubleMaker.setup({
      fullPathToRequire: amdLoaderNode.src,
      baseUrl: WebappsUtils.getWebappsBaseUrl(),
      appPath: WebappsUtils.getWebappsBaseUrl() + 'WebJobs/'
    });
     
    // fullPathToRequire = the full path to requirejs or amdloader.
    // baseUrl: baseUrl to initialize require with in the thread.
    // appPath to where the BaseThread exists.
     
    // Sometimes baseUrl and appPath cand be 2 differnet things...sometiems the same..

    6/8/2019

    Updated README for more specific instructions for usage. Preparing for publish.

    6/10/2019

    Attached keywords to package.json at a colleagues suggestion.

    9/24/2019

    Figuring out coverage. Reraching some of the internals. MesageIDs is not loading....why NOT.

    Install

    npm i webjobs

    DownloadsWeekly Downloads

    5

    Version

    1.1.0

    License

    MIT

    Unpacked Size

    84.1 kB

    Total Files

    41

    Last publish

    Collaborators

    • cbuteau