Builder Initializer
Initialize projects from builder archetypes.
Installation
Install this package as a global dependency.
$ npm install -g builder-init
Although we generally disfavor global installs, this tool creates new projects from scratch, so you have to start somewhere...
Usage
builder-init
can initialize any package that npm
can
install, including npm, GitHub, file, etc.
It uses the denim template engine with some customizations specifically for
builder
projects.
Invocation:
$ builder-init [flags] <archetype>
Flags:
--help
--version
--prompts
Examples:
$ builder-init builder-react-component$ builder-init builder-react-component@0.2.0$ builder-init FormidableLabs/builder-react-component$ builder-init FormidableLabs/builder-react-component#v0.2.0$ builder-init git+ssh://git@github.com:FormidableLabs/builder-react-component.git$ builder-init git+ssh://git@github.com:FormidableLabs/builder-react-component.git#v0.2.0$ builder-init /FULL/PATH/TO/builder-react-component
Internally, builder-init
utilizes npm pack
to download (but not install) an archetype package from npm, GitHub, file, etc.
There is a slight performance penalty for things like local files which have to
be compressed and then expanded again, but we gain the very nice benefit of
allowing builder-init
to install anything npm
can in exactly the same
manner that npm
does.
Installing from a Relative Path on the Local Filesystem
One exception to the "install like npm
does" rule is installation from the
local filesystem. Internally, builder-init
creates a temporary directory
to expand the download from npm pack
and executes the process in that
directory, meaning that relative paths to a target archetype are now incorrect.
Accordingly, if you want to simulate a relative path install, you can try something like:
# Mac / Linux $ builder-init "${PWD}/../builder-react-component" # Windows $ builder-init "%cd%\..\builder-react-component"
Automating Prompts
To facilitate automation, notably testing an archetype by generating a project
with builder-init
and running the project's tests as part of CI, there is a
special --prompts=JSON_OBJECT
flag that skips the actual input prompts and
injects fields straight from a JSON object.
$ builder-init <archetype> \ --prompts'{"name":"bob","quest":"popcorn","destination":"my-project"}'
Note that all required fields must be provided in the JSON object, no defaults
are used, and the init process will fail if there are any missing fields.
Tip: You will need a destination
value, which is added to all prompts.
A working example is available at:
builder-react-component/.travis.yml
which initializes the archetype's templates for a fresh project with canned
--prompts
values, npm installs dependencies, then runs the same builder
tasks used in the project's CI.
Archetype Templates
Authoring templates for an archetype consists of adding the following to your archetype source:
init.js
: A control file for user prompts and data. See, e.g.,builder-react-component/blob/master/init.js
init/
: A directory of templates to inflate during initialization. See, e.g.,builder-react-component/blob/master/init/
This directory can be configured with user prompts / data by setting the special_templatesDir
variable to something different than"init"
.
For example, in builder-react-component
, we have a control file and templates
as follows:
init.js
init/
.babelrc
.builderrc
.editorconfig
.travis.yml
CONTRIBUTING.md
demo/app.jsx
demo/index.html
LICENSE.txt
package.json
README.md
src/components/{{componentPath}}.jsx
src/index.js
test/client/main.js
test/client/spec/components/{{componentPath}}.spec.jsx
test/client/test.html
{{_gitignore}}
{{_npmignore}}
Archetype Data
Archetypes provide data for template expansion via an init.js
file in the
root of the archetype. The structure of the file is:
moduleexports = destination: // A special prompt for output destination directory. prompts: // Questions and responses for the user derived: // Other fields derived from the data provided by the user;
Note that builder-init
requires destination
output directories to not exist
before writing for safety and initialization sanity.
Imports and Dependencies
The init.js
file is require
-ed from a temporary extracted
directory
containing the full archetype. However, an npm install
is not run in the
archetype directory prior to starting the initialization process. This means
that you can require
in:
- Files contained in the archetype itself.
- Any standard node libraries. (E.g.,
require("path")
,require("fs")
).
Unfortunately, you cannot require third party libraries or things that may
be found in <archetype>/node_modules/
. (E.g., require("lodash")
).
This is a good thing, because the common case is that you will need nearly
none of the dependencies in init.js
prompting that are used in the archetype
itself, so builder-init
remains lightening quick by not needing to do any
npm install
-ing.
There is a future ticket
to consider supporting custom npm
dependencies in the `init.js file.
User Prompts
User prompts and responses are ingested using inquirer. The prompts
field
of the init.js
object can either be an array or object of inquirer
question objects. For example:
moduleexports = // Destination directory to write files to. // // This field is deep merged and added _last_ to the prompts so that archetype // authors can add `default` values or override the default message. You // could further override the `validate` function, but we suggest using the // existing default as it checks the directory does not already exist (which // is enforced later in code). destination: { // Use the early `name` prompt as the default value for our dest directory return dataname; } prompts: name: "name" message: "What is your name?" { // Validate functions return `true` if valid. // If invalid, return `false` or an error message. return !!val || "Must enter a name!"; } name: "quest" message: "What is your quest?" ;
builder-init
provides a short-cut of placing the name
field as the key
value for a prompts
object instead of an array:
moduleexports = prompts: name: message: "What is your name?" { return !!val || "Must enter a name!"; } quest: message: "What is your quest?" ;
Note - Async: Inquirer has some nice features, one of which is enabling
functions like validate
to become async by using this.async()
. For
example:
name: message: "What is your name?" { var done = this; // Let's wait a second. ; }
Derived Data
Archetype authors may not wish to expose all data for user input. Thus,
builder-init
supports a simple bespoke scheme for taking the existing user
data and adding derived fields.
The derived
field of the init.js
object is an object of functions with
the signature:
derived: // - `data` All existing data from user prompts. // - `callback` Callback of form `(error, derivedData)` { // Uppercase the existing `name` data. ; }
Special Data and Scenarios
.npmignore
, .gitignore
, etc.
The Problem
Special files like .npmrc
, .npmignore
, and .gitignore
in an init/
templates directory are critical to the correct publishing / git lifecycle of a
created project. However, publishing init/
to npm as part of publishing the
archetype and even initializing off of a local file path via npm pack
does not
work well with the basic layout of:
init/
.gitignore
.npmignore
.npmrc
The problem is that the .npmignore
affects and filters out files that will
be available for template use in an undesirable fashion. For example, in
builder-react-component
which has an .npmignore
which includes:
demo
test
.editor*
.travis*
natural npm
processes would exclude all of the following template files:
init/.editorconfig
init/.travis.yml
init/test/client/main.js
init/test/client/spec/components/{{componentPath}}.spec.jsx
init/test/client/test.html
init/demo/app.jsx
init/demo/index.html
Adding even more complexity to the situation is the fact that if npm
doesn't
find a .npmignore
on publishing or npm pack
it will rename .gitignore
to
.npmignore
.
The Solution
To address this, we have special derived
values built in by default to
builder-init
. You do not need to add them to your init.js
:
{{_gitignore}}
->.gitignore
{{_npmignore}}
->.npmignore
{{_npmrc}}
->.npmrc
{{_eslintrc}}
->.eslintrc
In your archetype init
directory you should add any / none of these files
with the following names instead of their real ones:
init/
{{_gitignore}}
{{_npmignore}}
{{_npmrc}}
{{_eslintrc}}
As a side note for your git usage, this now means that init/.gitignore
doesn't
control the templates anymore and your archetype's root .gitignore
must
appropriately ignore files in init/
for git commits.
<archetype>/package.json
, <archetype>/dev/package.json
There is often a "chicken vs. egg" situation of an archetype under update vs.
the init/
templates installed from and using the archetype. To help a variety
of situations, we provide a special archetype
data variable with the
following data:
archetype:
package // `<archetype>/package.json` if it exists, else `{}`
devPackage // `<archetype>/dev/package.json` if it exists, else `{}`
This enables you to have "always correct" version values for init/package.json
by doing something like:
"dependencies": "builder": "^2.5.0" "builder-react-component": "<%= archetype.package.version ? '^' + archetype.package.version : '*' %>" "devDependencies": "builder-react-component-dev": "<%= archetype.devPackage.version ? '^' + archetype.devPackage.version : '*' %>"
In your template content.
Templates Directory Ingestion
As a preliminary matter, init/
is the out-of-the box templates directory
default for a special prompts variable _templatesDir
. You can override this in
an init.js
either via prompts
(allowing a user to pick a value) or derived
data. Either of these approaches can choose 1+ different directories to find
templates than the default init/
.
builder-init
mostly just walks the templates directory of an archetype looking
for any files with the following features:
- An empty templates directory is permitted, but a non-existent one will produce an error.
- If an
<_templatesDir>/.gitignore
file is found, the files matched in the templates directory will be filtered to ignore any.gitignore
glob matches. This filtering is done at load time before file name template strings are expanded (in case that matters).
builder-init
tries to intelligently determine if files in the templates
directory are actually text template files with the following heuristic:
- Inspect the magic numbers for known text files and opportunistically the byte range of the file buffer with https://github.com/gjtorikian/isBinaryFile. If binary bytes detected, don't process.
- Inspect the magic numbers for known binary types with https://github.com/sindresorhus/file-type If known binary type detected, don't process.
- Otherwise, try to process as a template.
If this heuristic approach proves too complicated / problematic, we'll consider
a more significant revision of processing with something more heavy-handed like
an opt-in file naming scheme or a blessed "unprocessed" directory
(such as init-raw/
).
Template Parsing
builder-init
uses Lodash templates, with the following customizations:
- ERB-style templates are the only supported format. The new ES-style template strings are disabled because the underlying processed code is likely to include JS code with ES templates.
- HTML escaping by default is disabled so that we can easily process
<
,>
, etc. symbols in JS.
The Lodash templates documentation can be found at: https://github.com/lodash/lodash/blob/master/lodash.js#L12302-L12365
And, here's a quick refresher:
Variables
var compiled = _;console;// => "Hi Bob!"
var compiled = _;console;// => Hi BOB, SALLY!
JavaScript Interpolation
var compiled = _;console;// => Hi BOB, SALLY!
File Name Parsing
In addition file content, builder-init
also interpolates and parses file
names using an alternate template parsing scheme, inspired by Mustache
templates. (The rationale for this is that ERB syntax is not file-system
compliant on all OSes).
So, if we have data: packageName: "whiz-bang-component"
and want to create
a file-system path:
src/components/whiz-bang-component.jsx
The source archetype should contain a full file path like:
init/src/components/{{packageName}}.jsx
builder-init
will validate the expanded file tokens to detect clashes with
other static file names provided by the generator.
Tips, Tricks, & Notes
npmrc File
If you use Private npm, or a non-standard registry, or anything leveraging a
custom npmrc
file, you need to set
a user (~/.npmrc
) or global ($PREFIX/etc/npmrc
) npmrc file.
builder-init
relies on npm pack
under the hood and runs from a temporary
directory completely outside of the current working directory. So, while
npm info <module>
or npm pack <module>
would work just fine with an
.npmrc
file in the current working directory, builder-init
will not.
Archetype Development Guide
There is a "chicken vs. egg" problem when developing changes to both an
archetype and the init/
templates. Here is a workflow that should be
appropriate for most scenarios using builder-react-component
as an example.
First, npm link
your archetype and its -dev
version if applicable.
# Link prod archetype $ cd /PATH/TO/builder-react-component$ npm link # Link dev archetype (if you have one) $ cd dev$ npm link
Next, install off directory in workspace of your choosing:
$ cd /PATH/TO/TEMP_WORKSPACE$ npm install -g builder-init$ builder-init /PATH/TO/builder-react-component# ... answer prompts, etc. [builder-init] New builder-react-component project is ready at: PROJECT_NAME
Then, change to project directory, npm link as appropriate and install.
$ cd PROJECT_NAME$ npm link builder-react-component$ npm link builder-react-component-dev$ npm install
You can check you are using the appropriately symlinked modules on Mac/Linux with:
$ ls -l node_modules | grep ^llrwxr-xr-x 1 USER COMPUTER Users 64 Jan 29 16:20 builder-react-component -> ../../../../.nvm/v4.2.4/lib/node_modules/builder-react-componentlrwxr-xr-x 1 USER COMPUTER Users 68 Jan 29 16:20 builder-react-component-dev -> ../../../../.nvm/v4.2.4/lib/node_modules/builder-react-component-dev
All actions in your generated project will now use your "under development" archetype on your local filesystem.
Side Note - our CI checks for initializing a new project from scratch for
archetypes like builder-react-component
pretty much follows this exact scheme.
See our above section on Automating Prompts for links
and other setup information.