es-module-shims
    TypeScript icon, indicating that this package has built-in type declarations

    1.4.2 • Public • Published

    ES Module Shims

    Shims modern ES Modules features like import maps on top of the baseline modules support in browsers supported by 95% of users.

    When running in polyfill mode, the 67% of users with import maps entirely bypass the shim code entirely.

    For the remaining 30% of users, the highly performant (see benchmarks) production and CSP-compatible shim kicks in to rewrite module specifiers driven by the Web Assembly ES Module Lexer.

    The following modules features are polyfilled:

    When running in shim mode, module rewriting is applied for all users and custom resolve and fetch hooks can be implemented allowing for custom resolution and streaming in-browser transform workflows.

    Because we are still using the native module loader the edge cases work out comprehensively, including:

    • Live bindings in ES modules
    • Dynamic import expressions (import('src/' + varname'))
    • Circular references, with the execption that live bindings are disabled for the first unexecuted circular parent.

    Usage

    Include ES Module Shims with a async attribute on the script, then include an import map and module scripts normally:

    <script async src="https://ga.jspm.io/npm:es-module-shims@1.4.2/dist/es-module-shims.js"></script>
    
    <!-- https://generator.jspm.io/#U2NhYGBkDM0rySzJSU1hKEpNTC5xMLTQM9Az0C1K1jMAAKFS5w0gAA -->
    <script type="importmap">
    {
      "imports": {
        "react": "https://ga.jspm.io/npm:react@18.0.0-rc.0/index.js"
      },
      "scopes": {
        "https://ga.jspm.io/npm:react@18.0.0-rc.0/": {
          "object-assign": "https://ga.jspm.io/npm:object-assign@4.1.1/index.js"
        }
      }
    }
    </script>
    
    <script type="module">
      import react from 'react';
      console.log(react);
    </script>

    In browsers without import maps support, a console error will be given:

    Uncaught TypeError: Failed to resolve module specifier "app". Relative references must start with either "/", "./", or "../".
      at <anonymous>:1:15
    

    This execution failure is a feature - it avoids the polyfill causing double execution. The first import being a bare specifier in the pattern above is important to ensure this.

    This is because the polyfill cannot disable the native loader - instead it will only execute modules that would otherwise fail resolving or parsing to avoid duplicate fetches or executions that would cause performance and reliability issues.

    Shim Mode

    Shim mode is an alternative to polyfill mode and doesn't rely on native modules erroring - instead it is triggered by the existence of any <script type="importmap-shim"> or <script type="module-shim">, or when explicitly setting the shimMode init option.

    In shim mode, only the above importmap-shim and module-shim tags will be parsed and executed by ES Module Shims.

    Shim mode also provides some additional features that aren't yet natively supported such as supporting multiple import maps, external import maps with a "src" attribute, dynamically injecting import maps, and reading current import map state, which can be useful in certain applications.

    Benchmarks

    ES Module Shims is designed for production performance. A comprehensive benchmark suite tracks multiple loading scenarios for the project.

    Benchmark summary:

    • ES Module Shims Chrome Passthrough (for 70% of users) results in ~5ms extra initialization time over native for ES Module Shims fetching, execution and initialization, and on a slow connection the additional non-blocking bandwidth cost of its 10KB compressed download as expected.
    • ES Module Shims Polyfilling (for the remaining 30% of users) is on average 1.4x - 1.5x slower than native module loading, and up to 1.8x slower on slow networks (most likely due to the browser preloader), both for cached and uncached loads, and this result scales linearly up to 10MB and 20k modules loaded executing on the fastest connection in just over 2 seconds in Firefox.
    • Very large import maps (100s of entries) cost only a few extra milliseconds upfront for the additional loading cost.

    Features

    Browser Support

    Works in all browsers with baseline ES module support.

    Browser Compatibility with ES Module Shims:

    ES Modules Features Chrome (61+) Firefox (60+) Safari (10.1+) Edge (17+)
    Executes Modules in Correct Order ✔️ ✔️ ✔️ ✔️1
    Dynamic Import ✔️ ✔️ ✔️ ✔️
    import.meta.url ✔️ ✔️ ✔️ ✔️
    Module Workers ✔️ ~68+ 2 2 2
    modulepreload ✔️ ✔️ ✔️ ✔️
    Import Maps ✔️ ✔️ ✔️ ✔️
    JSON Modules ✔️ ✔️ ✔️ ✔️
    CSS Modules ✔️3 ✔️3 ✔️3 ✔️3
    import.meta.resolve ✔️ ✔️ ✔️ ✔️
    Top-Level Await ✔️ 89+ ✔️ 89+ 4 4
    • 1: The Edge parallel execution ordering bug is corrected by ES Module Shims with an execution chain inlining approach.
    • 2: Module worker support cannot be implemented without dynamic import support in web workers.
    • 3: CSS module support requires a separate Constructable Stylesheets polyfill.
    • 4: Top-level await support is possible for ES Module Shims to implement, with the feature request tracking in https://github.com/guybedford/es-module-shims/issues/5.

    Browser compatibility without ES module shims:

    ES Modules Features Chrome (61+) Firefox (60+) Safari (10.1+) Edge (17+)
    Executes Modules in Correct Order ✔️ ✔️ ✔️ 1
    Dynamic Import ✔️ 63+ ✔️ 67+ ✔️ 11.1+
    import.meta.url ✔️ ~76+ ✔️ ~67+ ✔️ ~12+ 1
    Module Workers ✔️ ~68+
    modulepreload ✔️ 66+
    Import Maps ✔️ 89+
    JSON Modules ✔️ 91+
    CSS Modules ✔️ 95+
    import.meta.resolve
    Top-Level Await ✔️ 89+ ✔️ 89+
    • 1: Edge executes parallel dependencies in non-deterministic order. (ChakraCore bug).
    • ~: Indicates the exact first version support has not yet been determined (PR's welcome!).
    • 1: On module redirects, Safari returns the request URL in import.meta.url instead of the response URL as per the spec.

    Import Maps

    Stability: WhatWG Standard, Single Browser Implementer

    Import maps allow for importing "bare specifiers" in JavaScript modules, which prior to import maps throw in all browsers with native modules support.

    Using this polyfill we can write:

    <script type="importmap-shim">
    {
      "imports": {
        "test": "/test.js"
      },
      "scopes": {
        "/": {
          "test-dep": "/test-dep.js"
        }
      }
    }
    </script>
    <script type="module-shim">
      import test from "test";
      console.log(test);
    </script>

    All modules are still loaded with the native browser module loader, but with their specifiers rewritten then executed as Blob URLs, so there is a relatively minimal overhead to using a polyfill approach like this.

    Multiple Import Maps

    Multiple import maps are not currently supported in any native implementation, Chromium support is currently being tracked in https://bugs.chromium.org/p/chromium/issues/detail?id=927119.

    In polyfill mode, multiple import maps are therefore not supported.

    In shim mode, support for multiple importmap-shim scripts follows the import map extensions proposal.

    External Import Maps

    External import maps (using a "src" attribute) are not currently supported in any native implementation.

    In polyfill mode, external import maps are therefore not supported.

    In shim mode, external import maps are fully supported.

    Dynamic Import Maps

    Support for dynamically injecting import maps with JavaScript via eg:

    document.body.appendChild(Object.assign(document.createElement('script'), {
      type: 'importmap',
      innerHTML: JSON.stringify({ imports: { x: './y.js' } }),
    }));

    is supported in Chromium, provided it is injected before any module loads and there is no other import map yet loaded (multiple import maps are not supported).

    Both modes in ES Module Shims support dynamic injection using DOM Mutation Observers.

    While in polyfill mode the same restrictions apply that multiple import maps, import maps with a src attribute, and import maps loaded after the first module load are not supported, in shim mode all of these behaviours are fully enabled for "importmap-shim".

    Reading current import map state

    To make it easy to keep track of import map state, es-module-shims provides a importShim.getImportMap utility function, available only in shim mode.

    const importMap = importShim.getImportMap()
    
    // importMap will be an object in the same shape as the json in a importmap script

    Dynamic Import

    Stability: Stable browser standard

    Dynamic import(...) within any modules loaded will be rewritten as importShim(...) automatically providing full support for all es-module-shims features through dynamic import.

    To load code dynamically (say from the browser console), importShim can be called similarly:

    importShim('/path/to/module.js').then(x => console.log(x));

    import.meta.url

    Stability: Stable browser standard

    import.meta.url provides the full URL of the current module within the context of the module execution.

    modulepreload

    Stability: WhatWG Standard, Single Browser Implementer

    Preloading of modules can be achieved by including a <link rel="modulepreload" href="/module.js" /> tag in the HTML or injecting it dynamically.

    This tag also supports the "integrity", "crossorigin" and "referrerpolicy" attributes as supported on module scripts.

    This tag just initiates a fetch request in the browser and thus works equally as a preload polyfill in both shimmed and unshimmed modes, with integrity validation support.

    Unlike the browser specification, the modulepreload polyfill does not request dependency modules by default, in order to avoid unnecessary code analysis in the polyfill scenarios. It is recommended to preload deep imports anyway so that this feature shouldn't be necessary.

    CSP Support

    By default ES Module Shims provides full support for CSP by using the asm.js ES Module Lexer build. This is absolutely identical in performance to the Wasm version in Firefox and Chrome, while in Safari the asm.js version is actually faster than Wasm making this build preferable.

    The CSP nonce to use for module scripts will be picked up from the first script on the page or via the nonce init option.

    A full example of such a CSP workflow is provided below:

    <meta http-equiv="Content-Security-Policy" content="script-src 'self' 'nonce-n0nce'" />
    <script async src="es-module-shims.js"></script>
    <script type="importmap" nonce="n0nce">
    {
      "pkg": "/pkg.js"
    }
    </script>
    <script type="module" nonce="n0nce">
    import pkg from 'pkg';
    </script>

    Wasm Build

    To use the Web Assembly / non-CSP build of ES Module Shims, this is available as a self-contained single file at es-module-shims/wasm or es-module-shims/dist/es-module-shims.wasm.js in the package folder.

    JSON Modules

    Stability: WhatWG Standard, Single Browser Implementer

    In shim mode, JSON modules are always supported. In polyfill mode, JSON modules require the polyfillEnable: ['json-modules'] init option.

    JSON Modules are currently supported in Chrome when using them via an import assertion:

    <script type="module">
    import json from 'https://site.com/data.json' assert { type: 'json' };
    </script>

    In addition JSON modules need to be served with a valid JSON content type.

    Checks for assertion failures are not currently included.

    CSS Modules

    Stability: WhatWG Standard, Single Browser Implementer

    In shim mode, CSS modules are always supported. In polyfill mode, CSS modules require the polyfillEnable: ['css-modules'] init option.

    CSS Modules are currently supported in Chrome when using them via an import assertion:

    <script type="module">
    import sheet from 'https://site.com/sheet.css' assert { type: 'css' };
    </script>

    To support the polyfill or shim of this feature, the Constructable Stylesheets polyfill must be separately included in browsers not supporting Constructable Stylesheets eg via:

    <script async src="https://unpkg.com/construct-style-sheets-polyfill@3.0.0/dist/adoptedStyleSheets.js"></script>

    For more information see the web.dev article.

    In addition CSS modules need to be served with a valid CSS content type.

    Checks for assertion failures are not currently included.

    Resolve

    Stability: No current browser standard

    import.meta.resolve provides a contextual resolver within modules. It is asynchronous, like the Node.js implementation, to support waiting on any in-flight import map loads when import maps are loaded dynamically.

    The second argument to import.meta.resolve permits a custom parent URL scope for the resolution, which defaults to import.meta.url.

    // resolve a relative path to a module
    var resolvedUrl = await import.meta.resolve('./relative.js');
    // resolve a dependency from a module
    var resolvedUrl = await import.meta.resolve('dep');
    // resolve a path
    var resolvedUrlPath = await import.meta.resolve('dep/');
    // resolve with a custom parent scope
    var resolvedUrl = await import.meta.resolve('dep', 'https://site.com/another/scope');

    This implementation is as provided experimentally in Node.js - https://nodejs.org/dist/latest-v14.x/docs/api/esm.html#esm_no_require_resolve.

    Polyfill Mode Details

    In polyfill mode, feature detections are performed for ES modules features. In browsers with full feature support no further processing is done.

    In browsers with variable feature support, sources are analyzed with module specifiers rewritten using the very fast Wasm / asm.js lexer while sharing the source network fetch cache with the native loader.

    Polyfill Features

    The current default native baseline for the ES module shims polyfill mode is browsers supporting import maps.

    If using more modern features like CSS Modules or JSON Modules, these need to be manually enabled via the polyfillEnable init option to raise the native baseline to only browsers supporting these features:

    <script>
    window.esmsInitOptions = { polyfillEnable: ['css-modules', 'json-modules'] }
    </script>

    To verify when the polyfill is actively engaging as opposed to relying on the native loader, a polyfill hook is also provided.

    Polyfill Edge Case: Dynamic Import

    The guarantee of the polyfill is that any module graph that would have failed will be reexecuted through the shim layer. This leaves any edge case where execution succeeds but not as expected. For example when using dynamic imports:

    <script type="module">
      console.log('Executing');
      const dynamic = 'bare-specifier';
      import(dynamic).then(x => {
        console.log('Ok');
      }, err => {
        console.log('Fail');
      });
    </script>

    The native browser loader without import maps support will execute the above module fine, but fail on the lazy dynamic import.

    ES Module Shims will not reexecute the above in browsers without import maps support though because it will see that the execution did complete successfully therefore it will not attempt reexecution and as a result, "Ok" is never logged.

    Other examples include dynamically injecting import maps, or using import maps with a "src" attribute, which aren't supported in native Chrome.

    This is why it is advisable to always ensure modules use syntax that will fail early to avoid non-execution.

    Polyfill Edge Case: Instance Sharing

    When running in polyfill mode, it can be thought of that are effectively two loaders running on the page - the ES Module Shims polyfill loader, and the native loader.

    Note that instances are not shared between these loaders for consistency and performance.

    As a result, if you have two module graphs - one native and one polyfilled, they will not share the same dependency instance, for example:

    <script type="importmap">
    {
      "imports": {
        "dep": "/dep.js"
      }
    }
    </script>
    <script type="module">
    import '/dep.js';
    </script>
    <script type="module">
    import 'dep';
    </script>

    In the above, on browsers without import maps support, the /dep.js instance will be loaded natively by the first module, then the second import will fail.

    ES Module Shims will pick up on the second import and reexecute /dep.js. As a result, /dep.js will be executed twice on the page.

    For this reason it is important to always ensure all modules hit the polyfill path, either by having all graphs use import maps at the top-level, or via importShim directly.

    Skip Polyfill

    Adding the "noshim" attribute to the script tag will also ensure that ES Module Shims skips processing this script entirely:

    <script type="module" noshim>
      // ...
    </script>

    Init Options

    Provide a esmsInitOptions on the global scope before es-module-shims is loaded to configure various aspects of the module loading process:

    <script>
    window.esmsInitOptions = {
      shimMode: true, // default false
      polyfillEnable: ['css-modules', 'json-modules'], // default empty
      nonce: 'n0nce', // default null
      noLoadEventRetriggers: true, // default false
      skip: /^https:\/\/cdn\.com/, // defaults to null
      onerror: (e) => { /*...*/ }, // default noop
      onpolyfill: () => {}, // default logs to the console
      resolve: (id, parentUrl, resolve) => resolve(id, parentUrl), // default is spec resolution
      fetch: (url, options) => fetch(url, options), // default is native
      revokeBlobURLs: true, // default false
      enforceIntegrity: true, // default false
    }
    </script>
    <script async src="es-module-shims.js"></script>

    If only setting JSON-compatible options, the <script type="esms-options"> can be used instead:

    <script type="esms-options">
    {
      "shimMode": true,
      "polyfillEnable": ["css-modules", "json-modules"],
      "nonce": "n0nce",
      "onpolyfill": "polyfill"
    }
    </script>

    This can be convenient when using a CSP policy.

    Function strings correspond to global function names.

    See below for a detailed description of each of these options.

    Shim Mode Option

    Shim Mode can be overridden using the shimMode option:

    <script type="esms-options">
    {
      "shimMode": true
    }
    </script>

    For example, if lazy loading <script type="module-shim"> scripts alongside static native module scripts, shim mode would not be enabled at initialization time.

    DOM load events are fired for all "module-shim" scripts both for success and failure just like for native module scripts.

    Pollyfill Enable Option

    The polyfillEnable option allows enabling polyfill features which are newer and would otherwise result in unnecessary polyfilling in modern browsers that haven't yet updated.

    Currently this option supports just "css-modules" and "json-modules".

    <script type="esms-options">
    {
      "polyfillEnable": ["css-modules", "json-modules"]
    }
    </script>

    Enforce Integrity

    When enabled, enforceIntegrity will ensure that all modules loaded through ES Module Shims must have integrity defined either on a <link rel="modulepreload" integrity="..."> or on a <link rel="modulepreload-shim" integrity="..."> preload tag in shim mode. Modules without integrity will throw at fetch time.

    For example in the following, only the listed app.js and dep.js modules will be able to execute with the provided integrity:

    <script type="esms-options">{ "enforceIntegrity": true }</script>
    <link rel="modulepreload-shim" href="/app.js" integrity="sha384-..." />\
    <link rel="modulepreload-shim" href="/dep.js" integrity="sha384-..." />
    <script type="module-shim">
      import '/app.js';
    </script>

    Strong execution guarantees are only possible in shim mode since in polyfill mode it is not possible to stop the native loader from executing code without an integrity.

    Future versions of this option may provide support for origin-specific allow lists.

    Nonce

    The nonce option allows setting a CSP nonce to be used with all script injections for full CSP compatibility supported by the CSP build of ES Module Shims.

    Alternatively, add a blob: URL policy with the CSP build to get CSP compatibility.

    <script type="esms-options">
    {
      "nonce": "n0nce"
    }
    </script>

    No Load Event Retriggers

    Because of the extra processing done by ES Module Shims it is possible for static module scripts to execute after the DOMContentLoaded or readystatechange events they expect, which can cause missed attachment.

    In order to ensure libraries that rely on these event still behave correctly, ES Module Shims will always double trigger these events that would normally have executed before the document ready state transition to completion, once all the static module scripts in the page have been completely executed through ES module shims.

    In such a case, this double event firing can be disabled with the noLoadEventRetriggers option:

    <script type="esms-options">
    {
      // do not re-trigger DOM events (onreadystatechange, DOMContentLoaded)
      "noLoadEventRetriggers": true
    }
    </script>
    <script async src="es-module-shims.js"></script>

    Skip Processing

    When loading modules that you know will only use baseline modules features, it is possible to set a rule to explicitly opt-out modules from rewriting. This improves performance because those modules then do not need to be processed or transformed at all, so that only local application code is handled and not library code.

    This can be configured by providing a URL regular expression for the skip option:

    <script type="esms-options">
    {
      "skip": "/^https?:\/\/(cdn\.skypack\.dev|jspm\.dev)\//`
    }
    </script>
    <script async src="es-module-shims.js"></script>

    Polyfill hook

    The polyfill hook is called when running in polyfill mode and the polyfill is kicking in instead of passing through to the native loader.

    This can be a useful way to verify that the native passthrough is working correctly in latest browsers for performance, while also allowing eg the ability to analyze or get metrics reports of how many users are getting the polyfill actively applying to their browser application loads.

    <script>
    window.polyfilling = () => console.log('The polyfill is actively applying');
    </script>
    <script type="esms-options">
    {
      "onpolyfill": "polyfilling"
    }
    </script>

    The default hook will log a message to the console with console.info noting that polyfill mode is enabled and that the native error can be ignored.

    In the above, running in latest Chromium browsers, nothing will be logged, while running in an older browser that does not support newer features like import maps the console log will be output.

    Error hook

    You can provide a function to handle errors during the module loading process by providing an onerror option:

    <script>
      window.esmsInitOptions = {
        onerror: error => console.log(error) // defaults to `((e) => { throw e; })`
      }
    </script>
    <script async src="es-module-shims.js"></script>

    Resolve Hook

    The resolve hook is supported for shim mode only and allows full customization of the resolver, while still having access to the original resolve function.

    <script>
      window.esmsInitOptions = {
        shimMode: true,
        resolve: async function (id, parentUrl, defaultResolve) {
          if (id === 'custom' && parentUrl.startsWith('https://custom.com/'))
            return 'https://custom.com/custom.js';
    
          // Default resolve will handle the typical URL and import map resolution
          return defaultResolve(id, parentUrl);
        }
      }
    </script>

    Fetch Hook

    The fetch hook is supported for shim mode only.

    The ES Module Shims fetch hook can be used to implement transform plugins.

    For example:

    <script>
      window.esmsInitOptions = {
        shimMode: true,
        fetch: async function (url, options) {
          const res = await fetch(url, options);
          if (!res.ok)
            return res;
          if (res.url.endsWith('.ts')) {
            const source = await res.body();
            const transformed = tsCompile(source);
            return new Response(new Blob([transformed], { type: 'application/javascript' }));
          }
          return res;
        } // defaults to `((url, options) => fetch(url, options))`
      }
    </script>
    <script async src="es-module-shims.js"></script>

    Because the dependency analysis applies by ES Module Shims takes care of ensuring all dependencies run through the same fetch hook, the above is all that is needed to implement custom plugins.

    Streaming support is also provided, for example here is a hook with streaming support for JSON:

    window.esmsInitOptions = {
      shimMode: true,
      fetch: async function (url, options) {
        const res = await fetch(url, options);
        if (!res.ok || !/^application\/json($|;)/.test(res.headers.get('content-type')))
          return res;
        const reader = res.body.getReader();
        const headers = new Headers(res.headers);
        headers.set('Content-Type', 'application/javascript');
        return new Response(new ReadableStream({
          async start (controller) {
            let done, value;
            controller.enqueue(new TextEncoder.encode('export default '));
            while (({ done, value } = await reader.read()) && !done) {
              controller.enqueue(value);
            }
            controller.close();
          }
        }), { headers });
      }
    }

    Revoke Blob URLs

    When polyfilling the missing features es-module-shims would create in-memory blobs using URL.createObjectURL() for each processed module. In most cases, memory footprint of these blobs is negligible so there is no need to call URL.revokeObjectURL() for them, and we don't do that by default.

    That said, in some scenarios, e.g. when evaluating some continuously changing modules without a page reload, like in a web-based code editor, you might want to reduce the growth of memory usage by revoking those blob URLs after they were already imported.

    You can do that by enabling the revokeBlobURLs init option:

    <script type="esms-options">
    {
      "revokeBlobURLs": true
    }
    </script>
    <script type="module" src="es-module-shims.js"></script>

    NOTE: revoking object URLs is not entirely free, while we are trying to be smart about it and make sure it doesn't cause janks, we recommend enabling this option only if you have done the measurements and identified that you really need it.

    Implementation Details

    Import Rewriting

    • Sources are fetched, import specifiers are rewritten to reference exact URLs, and then executed as BlobURLs through the whole module graph.
    • The lexer handles the full language grammar including nested template strings, comments, regexes and division operator ambiguity based on backtracking.
    • When executing a circular reference A -> B -> A, a shell module technique is used to "shim" the circular reference into an acyclic graph. As a result, live bindings for the circular parent A are not supported, and instead the bindings are captured immediately after the execution of A.

    Inspiration

    Huge thanks to Rich Harris for inspiring this approach with Shimport.

    License

    MIT

    Keywords

    none

    Install

    npm i es-module-shims

    DownloadsWeekly Downloads

    32,119

    Version

    1.4.2

    License

    MIT

    Unpacked Size

    145 kB

    Total Files

    7

    Last publish

    Collaborators

    • guybedford