Full-lifecycle starter project for SPAs
hi_score is a full-lifecycle starter project for web application development. It embodies good practice from over 20 years of experience for every lifecycle stage.
hi_score embraces the feature-module (or "web component") design pattern, so it should work well with React or Vue.js. Please do swap assets and libraries as required - that's the point.
js/xhilibs and documentation
bin/xhi help build
superpackto be more reliable
Download the latest latest virual appliance to try
hi_score with the minimum of time and hassle. Pick the latest
ova2 image for virutal box, and the latest
vmx.zip image for VMware or Parallels. If you need more help with installing a VM or wish to consider other options, please consult the Development platform section.
Open a terminal and enter the following lines. Wait for each to complete before proceeding to the next. The build process prompts the user to review TODO notes. Just press
return to accept them.
git clone email@example.com:mmikowski/hi_score.gitcd hi_scorebin/xhi build,dev_startgoogle-chrome http://localhost:8080/build/latest/dist
bin/xhi tool guides developers through lifecycle stages and employs sophisticated dependency checking to help avoid mistakes. The
bin/xhi build,dev_start command installs vendor assets; copies, configures, and patches vendor files for development; configures and starts an HTTP server; lints code with ESLint and other checks, lists TODO items for developer review; runs all regression test suites in
test.d; calculates and reports test coverage; minimizes, obsfucates, and creates a unique distribution directory containing multiple applications. The latest build can always be found in
hi_score has three example applications. Two (
app-ex02.html) are quite simple. The third, Typebomb2 (
The best way to use
hi_score is as a
git upstream source. One may then create new application using the
hi_score infrastructure without losing upstream improvements or bug fixes.
First create a new empty repository on Github and copy the
ssh repository URL, which should look similar to
firstname.lastname@example.org:<user>/<repo_name> and then proceed as below:
mkdir -p ~/Githubcd ~/Github# Clone the empty repositorygit clone email@example.com:<user>/<repo_name>cd <repo_name># Create master branchtouch README.<app_name>.md # Add app specific docs heregit add .git commit -m 'First commit for <repo_name>'git push# Verify origingit remote -v# origin firstname.lastname@example.org:<user>/<repo_name>.git (fetch)# origin email@example.com:<user>/<repo_name>.git (push)# Add upstream repositorygit remote add upstream firstname.lastname@example.org:mmikowski/hi_score.git# Verify upstreamgit remote -v# origin email@example.com:<user>/<repo_name>.git (fetch)# origin firstname.lastname@example.org:<user>/<repo_name>.git (push)# upstream email@example.com:mmikowski/hi_score.git (fetch)# upstream firstname.lastname@example.org:mmikowski/hi_score.git (push)# Merge changes from upstream and push to origingit fetch upstreamgit merge --allow-unrelated-histories upstream/mastergit push
We suggest you structure your apps as illustrated by the Typebomb2 app. This follows the feature module pattern which has been embraced by the recent libraries such as React and Vue.js (we've been advocating it since 2011, go figure). You may view this guide any time using
bin/xhi design. The core concept is to create feature modules that contain their own isolated data and models when appropriate. This is pragmatic and recognizes the fractal nature of MVC. A slicker image is shown below.
We have provided the
js/xhi libraries to either provide capabilities directly or as an illustration. For example, in Typebomb2 you will notice the following files for each layer shown in the diagram:
Module layers ======================================= tb02.00_root.js | ^ tb02.01_util.js load | tb02.02_data.js order | tb02.03_model.js | call, tb02.04_utilb.js | init tb02.06_css.js events order tb02.06_<feature>.js | | tb02.07_shell.js | | tb02.08_app.js v |
All these modules claim a slice of the application namespace (
tb02) and use
js/xhi libraries in one of three ways:
More specific notes about Typebomb2 app are provide in
README.app-tb02.md. One can omit unused layers for a given app. However, for illustrative purposes, we have included all layers for Typebomb2. One can copy these to a new namespace to create a new app and then edit from there.
cd hi_scorecp app-tb02.html app-<ns>.htmlcd jscp tb02.00_root.js <ns>.00_root.jscp tb02.01_util.js <ns>.01_util.jscp tb02.02_data.js <ns>.02_data.jscp tb02.03_model.js <ns>.03_model.jscp tb02.04_utilb.js <ns>.04_utilb.jscp tb02.05.css_<feature>.js <ns>.05.css_<feature>.jscp tb02.06_css.js <ns>.06_css.jscp tb02.06_<feature>.js <ns>.06_<feature>.jscp tb02.07_shell.js <ns>.07_shell.jscp tb02.08_app.js <ns>.08_app.jscp tb02.08_app-build.js <ns>.08_app-build.jsgit add .cd ../templatecp app-tb02.html app-<ns>.html
We need to change all references from
tb02 in these new files to our new namespace.
<ns>. We will also need to add a new build manifest in package.json. See the
xhi_11_BuildMatrix configuration for tb02 as your guide. The result is an app architecture that is designed to work well for every phase of the SPA lifecycle.
One can delete all the example apps (
ex02) from the project if they get in our way. However we recommend retaining at least
tb02 for reference because it will continue to be refined along with the
hi_score project. One can refresh upstream at any time as shown below.
git fetch upstreamgit merge upstream/master# use --allow-unrelated-histories if needed
We recommend you run
bin/xhi install,setup after any such merge.
bin/xhi tool automates best practice for almost every conceivable stage of the SPA development life cycle. Configuration of all stages is found in the NPM
package.json file. It resolves goal and environment prerequisites to ensure necessary stages are completed to meet a desired goal. For example, if we run
bin/xhi build right after cloning the Github repository, it will run all the stages needed to ensure a quality build including installation of the npm libraries. If we run it again, many stages will be omitted because they are known as complete. If we run
dev_upgrade all the
npm packages be updated to the latest revision. On any subsequent run,
bin/xhi will do the right thing and re-install our
npm packages, reset the environment, and re-test the codebase. See the Run Lifecycle Stages section below for more detail.
The full-lifecycle stages supported by
bin/xhi are shown below. Those marked
placeholder are placeholder which we plan to address in future releases. Use the command
bin/xhi help all to see this list.
$ bin/xhi help all xhi> START Stage 00 help xhi> 00 help : Help on 'xhi' tool, use -v for verbose xhi> 01 install : Download and install npm modules xhi> 02 setup : Patch and distribute vendor npm assets xhi> 03 design : Show architecture docs xhi> 04 dev_pull : Download and merge SCMS assets (git pull) xhi> 05 dev_upgrade : Upgrade packages to latest xhi> 06 dev_start : Start local HTTP server xhi> 07 dev_test : Run regression tests xhi> 08 dev_lint : Lint changed code xhi> 09 dev_cover : Create coverage report xhi> 10 dev_commit : Commit changes with git xhi> 11 build : Build a distribution xhi> 12 publish : Upload to publishers xhi> 13 dev_restart : Cycle local HTTP server xhi> 14 dev_stop : Stop local HTTP server xhi> 15 deploy : Upload distribution # placeholder xhi> 16 prod_start : Start production server(s) # placeholder xhi> 17 prod_restart: Cycle production server(s) # placeholder xhi> 18 prod_stop : Stop production server(s) # placeholder xhi> 19 fetch_info : Fetch feedback # placeholder xhi> 20 uninstall : Remove xhi # placeholder xhi> END Stage 00 help
This tool is used for all NPM lifecycle scripts (such as
Many sections of this document have been removed because the information is now directly available from
bin/xhi help. One can see detailed help on a stage or range of stages by including a
-v flag as shown below.
$ bin/xhi help dev_lint -v xhi> START Stage 00 help xhi> 08 dev_lint: xhi> Check lint quality of changed JS code. xhi> Files in 'vendor|node_modules' directories are ignored. xhi> Four tests are performed on each file: xhi> 1. Check for tab characters or trailing space xhi> 2. Ensure 'use strict'; is employed xhi> 3. Run 'eslint' on each file (config in package.json) xhi> 4. List TODO items for developer to review and approve xhi> Any failed step causes this stage to report failure. xhi> xhi> This stage does not "short-circuit" so any and all issues are xhi> shown for each run. xhi> xhi> NPM SCRIPTS : none. xhi> SUCCESS CRITERIA : All tests complete without error xhi> END Stage 00 help
A typical workflow is shown below.
# Get list of stages $ xhi help all # Run desired stage-range $ xhi dev_cover,build
bin/xhi tool takes a
<stage-range> argument. Stages that are provided out-of-order are sorted before running. Example use is shown below.
# Run a single stage $ xhi install # Run all stages between 'install' and 'dev_commit' inclusive $ xhi install-dev_commit # Run individual stages $ xhi update,dev_cover # Run a range using stage numbers $ xhi 0-5 # Get help on ranges $ xhi help install -v $ xhi help install-dev_commit $ xhi update,dev_cover $ xhi help 0-5
Even if we specify only one stage the
bin/xhi tool will often run more. That is because many stages require prequisites as discussed in the following section.
bin/xhi has a sophisticated prerequisite resolver that ensures required stages are run only if required.
Goal prerequisites are stages that are always run before before the target stage. For example, if we run
bin/xhi dev_commit the
dev_test stages will be run first to ensure the code quality is acceptable. If either prerequisite fails,
bin/xhi exits immediately (with an exit code of 1) and the target stage is not attempted. Goal prequesites are configuired in
These are stages that must be successfuly completed in the development environment. For example, if we run
bin/xhi dev_commit but have not run
bin/xhi install, the
install stage will be run before the
dev_commit stage. The success or failure of each stage is saved in the state file (
lib/xhi_state.json) and the next stage is run. If the
install stage succeeds it will not be included in future prerequisite calculations.
Environment prerequisites may be invalidated. For example, if
bin/xhi install or
bin/xhi upgrade fail, the tool will mark the
install stage as failed and this will be attempted again in the next
bin/xhi invocation that require it as a prerequisite.
Explicitly requested stages will run again regardless of their last success statuses. For example,
bin/xhi dev_lint may or may not run the
install stage, but
bin/xhi install,dev_lint will always run the
install stage because it is explicitly listed.
bin/xhi help-dev_lint will also run
install since it is explicitly within the range provided (
help-dev_lint). We can reset the status by removing the
stage_status_map from the
If all the stages of a range are successful an exit status of
0 is provided. If any stage fails processing of the range stops and an exit status of
1 is provided. In Bash, the return status is available in the
$? environment variable. If we apply minor adjustments to disable terminal interaction
bin/xhi should be capable of integration to other tool chains.
Our baseline compatibility is IE9+. Those supporting IE 8 have our sympathy.
The server component of hi_score is designed to run on industry-standard hardware, cloud instances like Amazon EC2, and containers. Our server platform is Ubuntu 16.04 LTS. Later version of Ubuntu and other distributions should work well.
Everything should just work on recent Ubuntu 16.04+ and derivatives like Mint or Kubuntu. The steps to install all required and recommended libraries are shown below.
# install development and useful libs sudo wajig install apt-file build-essential git \ htop kdiff3 libfile-slurp-perl \ liblist-moreutils-perl meld mysql-client mysql-server \ net-tools openssh-server pandoc pandoc-citeproc \ ppa-purge sysstat unzip vim-gtk vim-nox \ vim-syntax-gtk zip # Install nodejs curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash - sudo apt-get install -y nodejs # Install MongoDB 3.x (optional) # See their website for details.
Other Linux distributions should generally work as long as the same libraries can be installed with Ubuntu. It works fine on current versions of CentOS. Development libraries should be installed as shown below.
$ yum install gcc gcc-c++ make openssl-devel
The easiest way path to get familiar with this project on Mac is probably to use a product like Parallels or VMFusion to import the VMX image (unzip this file before use). VirtualBox also runs on Mac but it doesn't integrate as well to the host OS as Parallels, for example.
We should be able to run this natively on the Mac but we haven't tested it. We would need at the very least Bash 4+, GNU Core utilities, NodeJS, Git, PanDoc, Perl File::Slurp, and SSH server.
We recommend using a virtual machine as detailed above.
bin/xhi setup stage patches and deploys vendor assets using the
xhi_02_SetupMatrix configuration found in the
package.json file. This field is correlated with the with the
devDependencies map to ensure assets are properly label, patched, distributed, and ignored by Git.
Assets are copied to their destination directory with their version number appended to their names. The
.gitignore file instructs
git to ignore all these files as their development management is external to our project. Everytime
bin/xhi setup is run the vendor directories are deleted and recreated.
Vendor executables are copied to the
Vendor font files are copied to the
font/vendor directory. These fonts are currently installed:
Vendor images are be copied to the
Client libraries are copied to the
js/vendor directory. This makes them available to the web server. The following libraries are installed:
NodeJS libraries are not copied to a
vendor directory. We may changes this if we decide to create a server distribution. The following libraries are installed:
Developent libraries are used for testing a building code. They are not copied to a
vendor directory and probably never will be as they are for development, not deployment. The following libraries are installed:
Vendor CSS libraries are copied to the
css/vendor directory. We copy the Font Awesome CSS files to this directory:
xhi_02_SetupMatrix.patch_matrix directs patch application.
bin/xhi setup stage applies patches to vendor assets. The configuration for patches are in
package.json in the
xhiPatchMatrix map. The patches are stored in the
The patches mechanism allows us to use great tools tweaked for our needs while maintaining the upstream source. For example, we patch
uglify-js to support object property name compression and shuffling by
bin/xhi build or
bin/xhi make or
build/<build_id>/dist). The result loads faster, runs faster, and typically consumes roughly 1/50th (2%) of the development environment.
$ ## Show disk usage of all development files $ cd hi_score && export PATH=`pwd`/bin:$PATH; $ du -sh . 148M $ ## Get disk usage of all distribution files $ xhi build && cd build/latest && du -sh . 3.6M
bin/xhi build stage uses uses
superpack to analyze symbols (variable names, object properties, and labels) and replaces them with shortened and shuffled keys. The shortest keys are used for the most frequently found symbols.
superpack reports the key-to-symbol mapping and the frequency of use which makes further optimizations by pruning code easier (see
build/<build-number>/stage/<name>-sp.diag for mapping and key use). Projects with many object properities can be compressed an additional 50% using
superpack and hinder reverse-engineering of the compressed code.
The build process enhances security because only a tiny, currated, obsfucated portion of our code is published and sensitive data such as SCMS metadata, documentation, lookup-maps, and development assets are omitted for us to publish elsewhere at our discretion. The distribution also reduces the dozens of HTTP calls to just a few. This can reduce load time significantly as illustrated below.
|Attribute||Original (%)||Minified (%)||Superpack (%)|
|Size||601,027 (100.0%)||215,400 ( 35.8%)||162,494 ( 27.1%)|
|Gzipped||151,716 ( 25.2%)||62,895 ( 10.4%)||57,275 ( 09.5%)|
|Attribute||Original||Minified (%)||Superpack (%)|
|HTTP reqs||27 (100.0%)||4 ( 15.4%)||4 ( 15.4%)|
|Local ms||231 (100.0%)||166 ( 71.2%)||144 ( 62.3%)|
|Deploy Size||121 MB||8 MB ( 6.6%)||8 MB ( 6.5%)|
The load time measurements were made using a local HTTP server which is almost certainly a best-case scenario. We hope to add results for a remote server soon.
Namespaces enable us to provide a suite of web apps that share a great deal of code but have instances and data cleanly isolated. Namespacing across JS and CSS can help one trace behaviors to the controlling code faster and with greater accuracy. We can open them in the Chrome browser (
bin/xhi install && google-chrome ex*.html) and use the developer tools to see this in practice.
When we view Example 1 (
ex01.html) we can open the browser development tools (press
<shift>-<cmd>-i on a Mac), type
<return> to inspect that value. We can see that this single variable that contains our entire application. When we enter
ex02 we see that it is
undefined. When we visit the Example 2 (
ex02.html) instead we can see that
ex02 contains our app code using a similar process.
We also namespace our CSS classes to avoid collisions. When we inspect the HTML of the Example 1 app we can see that nearly all classes start with an
ex01- prefix. When we inspect Example 2 tab we find the prefix is
ex02-_lb_ class was generated for use by the
ex02-_lb_ module. During the build process selectors are shortened along with property values as long as they use the property-name structure. For example,
ex02-_shell_title_underscore_ will compress to something like
Any improvements or suggestions are welcome through the issues tracker. Pull requests are especially appreciated.
2016, 2017 Michael S. Mikowski (mike[dot]mikowski[at]gmail[dotcom])
setDeepMapValwith more powerful and tested
jscoveragewith much more complete and recent
castroutines and detail their use
jsdomto expand testing to modules that use jQuery
npm run buildify
npm run prep-libsto
npm run setup
npm run coverto
npm run coverage
npm run coverato
npm run publish-coverage
npm run buildifyto
npm run make
bin/xhitool development capabilities
bin/xhi setup: Implement env prequisites and
bin/xhi setup: Auto-create
bin/xhi build: Create build directory like
bin/xhi build: Link
dist/latestto latest build
bin/xhi build: Do not auto-increment build until next commit
bin/xhi dev_cover: Move to
"help" : "bin/xhi help"
"make" : "bin/xhi make"
"setup": "bin/xhi setup"
"test" : "bin/xhi test"
js/xhilibs and documentation
bin/xhi help build
superpackto be more reliable
bin/xhi buildconvert: superpack Perl to JS, use
bin/xhi dev_start, prod_startHTTPS : Use LetsEncrypt to use HTTPS by default
bin/xhi dev_start, prod_startHTTP/2: Configure for HTTP/2 if feasible
bin/xhi deployimplement: Add configuration and capability
bin/xhi publish: Push to NPM