Nearly Perfect Mediocracy

    metalsmith-data-loader

    1.3.0 • Public • Published

    metalsmith-data-loader

    Metalsmith plugin to add extra metadata from external files. Very similar to metalsmith-models with the difference that this allows you to load information from either a folder outside of the source or files within the source tree. Also, this one lets you use JSON or YAML files.

    npm version Build Status Dependencies Dev Dependencies codecov.io

    The "dependencies" badge says there's a potential problem with js-yaml and suggests that packages use .safeLoad(), which is exactly what this plugin does.

    What It Does

    When working with templates, you sometimes want to generate something like a table in Markdown from external data. You can do this with metalsmith-hbt-md but you need to get the extra data in your metadata. That can be accomplished by metalsmith-models as long as you are willing to separate your source data from the template that needs it.

    This plugin differs from that approach, allowing you to have your Markdown files adjacent to the data that is inserted into the table. This plugin look at the data property (configurable) in your file's metadata, then replace that filename with the parsed contents of the file. This works better with an example.

    table.md:

    ---
    title: Just a test file to illustrate why the module is useful.
    data: contacts.yaml
    ---
    
    Here's a great list of contacts:
    
    | First | Last | Email |
    |-------|------|-------|
    {{#data}}| {{first}} | {{last}} | {{email}} |
    {{/data}
    

    contacts.yaml (right next to the markdown):

    -
        first: Tyler
        last: Akins
        email: fidian@rumkin.com
    -
        first: Jane
        last: Doe
        email: j.doe@example.com
    

    After parsing, the metalsmith file object for table.md would look like this:

    ---
    title: Just a test file to illustrate why the module is useful.
    data:
        -
            first: Tyler
            last: Akins
            email: fidian@rumkin.com
        -
            first: Jane
            last: Doe
            email: j.doe@example.com
    ---
    
    Here's a great list of contacts:
    
    | First | Last | Email |
    |-------|------|-------|
    {{#data}}| {{first}} | {{last}} | {{email}} |
    {{/data}
    

    This isn't limited to table generation. You could load metadata specific to a collection of pages. Maybe you have a site where different authors maintain different pages and you could point to a single source for the author's information.

    If you prefer to work with code, there is an example repository set up that illustrates how the plugin functions. Please pay special attention to how the extra data is referenced in the templates.

    Note: You can also use an array of filenames or an object whose values are all filenames. More information in the "Usage" section.

    Installation

    npm can do this for you.

    npm install --save metalsmith-data-loader
    

    Usage

    Include this like you would include any other plugin. First, a CLI example that shows the default options. You don't need to specify any options unless you want to override their values.

    {
        "plugins": {
            "metalsmith-data-loader": {
                "dataProperty": "data",
                "directory": "models/",
                "ignoreReadFailure": false,
                "match": "**/*",
                "matchOptions": {},
                "removeSource": false
            }
        }
    }
    

    And here is the JavaScript example. This includes brief descriptions of each of the options.

    // Load this, just like other plugins.
    var dataLoader = require("metalsmith-data-loader");
    
    // Then in your list of plugins you use it.
    .use(dataLoader())
    
    // Alternately, you can specify options.  The values shown here are
    // the defaults.
    .use(dataLoader({
        // Property name to scan for files to include.
        dataProperty: "data",
    
        // Directory containing models.  This is relative to the working
        // directory.  It does not need to exist unless you start loading
        // files from here.
        directory: "models/",
    
        // Pattern of files to match in case you want to limit processing
        // to specific files.
        match: "**/*",
    
        // Options to ignore read failures, useful if you want to continue
        // processing even if the file is not found. When this is set to false,
        // the build will throw an exception when trying to load a file that
        // does not exist and is typically what people want.
        ignoreReadFailure: false,
    
        // Options for matching files.  See metalsmith-plugin-kit for
        // more information.
        matchOptions: {},
    
        // Flag indicating that the loaded data object should be removed
        // if it is found in the source.  Use this so your built project
        // doesn't include the metadata in the rendered output and another
        // copy that was consumed during the build.
        removeSource: false
    })
    

    This uses metalsmith-plugin-kit to match files. The .matchOptions object can be filled with options control how the matching behaves.

    From here, you now need to specify the files to include. These examples all assume you didn't change dataProperty.

    ---
    title: A single string loads one file
    data: my-model.json
    ---
    

    When loaded this way, the data metadata is replaced with the parsed contents of my-model.json.

    ---
    title: An object holding a map of filenames.
    data:
        users: users.json
        services: services.yaml
    ---
    

    This lets you load multiple files and assign them to properties on the data object. One would access the information by using {{data.users}} or {{data.services}} in Mustache templates.

    ---
    title: An array of files.  The filenames are replaced with their contents.
    data:
        - file1.yaml
        - file2.yaml
    ---
    

    The two files are loaded asynchronously and will replace the file1.yaml and file2.yaml. You would access these in Mustache by using {{data.0}} and {{data.1}} or you can iterate over {{#data}}.

    Resolving Files

    The combination of the directory configuration option and the data metadata property dictate which files are loaded. This table can help illustrate the relationship. In all of the examples, the directory configuration option is set to "models/" and the source file is always "src/path/file.md".

    Metadata Path File to Load Description
    file.yaml src/path/file.yaml Relative to source file.
    ../file.yaml src/file.yaml Relative to the source file.
    /other-path/file.yaml src/other-path/file.yaml Resolved from root of source folder.
    /../file.yaml file.yaml Can load things outside the source folder.
    !file.yaml models/file.yaml Resolved from the directory, not source folder. See the note!
    !../file.yaml file.yaml Can load items from outside the models directory. See the note!

    Note: When the metadata path starts with an exclamation point (!), you must use quotes in your YAML frontmatter. Here's a sample.

    ---
    title: Sample file that loads a file from a models directory
    data: "!file-from-models.json"
    ---
    

    You can play with the example repository to get a better handle on how the plugin works.

    API

    metalsmith-data-loader

    Params

    • destination Object - Metalsmith Data Loader

    Loads files and places their data structures in as file metadata.

    Example

    var dataLoader = require("metalsmith-data-loader");
    
    // Create your Metalsmith instance and add this like other middleware.
    metalsmith.use(dataLoader({
        // configuration goes here
    }));

    module.exports(options) ⇒ function

    Factory to build middleware for Metalsmith.

    Kind: Exported function
    Params

    module.exports~removeSource(sourceFile, modelFile, files, options)

    Removes a file from the list of files to process. This only happens when it is included by another file through this data loader and when the option is enabled.

    Kind: inner method of module.exports
    Params

    module.exports~resolveFile(metalsmith, sourceFile, modelFile, options) ⇒ string

    Resolves a file path and checks to make sure it exists.

    Paths can look like any of these:

    model.json -> Relative to file ./model.json -> Relative to file dir/model.yaml -> Relative to file ../model.json -> Relative to file /model.yaml -> From root of source /dir/model.json -> From root of source !model.yaml -> From models folder (not in source) !dir/model.json -> From models folder (not in source)

    Kind: inner method of module.exports
    Params

    • metalsmith module:metalsmith - Metalsmith instance.
    • sourceFile string - Filename being processed.
    • modelFile string - Reference inside the file's metadata.
    • options options

    module.exports~loadModel(dataFile) ⇒ Promise.<*>

    Loads a file asynchronously and places metadata on the destination object.

    Kind: inner method of module.exports
    Returns: Promise.<*> - The loaded data.
    Params

    • dataFile string - Resolved filename to load.

    module.exports~metalsmithFile : Object

    Metalsmith file object.

    Kind: inner typedef of module.exports
    Properties

    • contents Buffer
    • mode string

    module.exportsmetalsmithFileCollection : Object.<string, module:metalsmith-data-loader--module.exportsmetalsmithFile>

    Metalsmith collection of file objects.

    Kind: inner typedef of module.exports

    module.exports~options : Object

    Options for the middleware factory.

    Kind: inner typedef of module.exports
    See: https://github.com/fidian/metalsmith-plugin-kit
    Properties

    • dataProperty string - Name of property to use for loading models.
    • directory string - Path for storing external modules.
    • match module:metalsmith-plugin-kit~matchList - Files to match. Defaults to all files.
    • matchOptions module:metalsmith-plugin-kit~matchOptions - Options controlling how to match files.
    • removeSource boolean - When truthy, remove the model from the build result.

    Development

    This uses Jasmine, Istanbul and ESLint for tests.

    # Install all of the dependencies
    npm install
    
    # Run the tests
    npm run test
    

    This plugin is licensed under the MIT License with an additional non-advertising clause. See the full license text for information.

    Keywords

    none

    Install

    npm i metalsmith-data-loader

    DownloadsWeekly Downloads

    430

    Version

    1.3.0

    License

    MIT

    Unpacked Size

    57.4 kB

    Total Files

    14

    Last publish

    Collaborators

    • fidian