be-importing

Import Static, Declarative HTML Web Components with Streaming HTML

Sample syntax

<xtal-side-nav be-importing=xtal-side-nav/xtal-side-nav.html>
    <section style='color:white'>
        <div>Menu Item 1</div>
        <div>Menu Item 2</div>
    </section>
</xtal-side-nav>

<script type=importmap>{
    "imports":{
        "xtal-side-nav/": "https://cdn.jsdelivr.net/npm/xtal-side-nav@0.0.110/"
    }
}</script>
<script type=module>
    import 'https://esm.run/be-importing@0.0.50';
</script>

Backdrop

With the advent of declarative shadow DOM, many useful web components that require little to no js could be less taxing on the browser if they were imported as pre-rendered HTML rather than JavaScript.

When the user loads an HTML page in their browser, served by an ancient web server, it streams. This was engineered by Netscape/Apache in a fortnight(?), when even elite users had to suffer with 19,200 bit/s.

Three decades later, two of the browser engines have now enabled this streaming optimization, even for content that has style isolation (Shadow DOM), which is fantastic news!

But what if we need a portion of the page to stream, for example as the content of that part of the page becomes out of date? Or maybe the data for that portion of the page wasn't available at the time the page loaded?

That would have been the natural evolution of things, to support this scenario, once asynchronous http requests could be made within a page session, circa 1999. Browsers would have needed to make certain adjustments to make it so. But asynchronous HTTP landed at about the same time as Road Runner and V8, so the whole streaming concept became passé at that point. Instead, we went all API-happy, with unfortunate future consequences.

One would have thought that with the introduction of smart phones, the browser vendors would have gone back to the drawing board, and rushed to fulfill this poverty-and-global-warming-reducing functionality (streaming partial page reloads), but alas, they had higher priorities, like preventing one website from gleaning whether the user visited another site that also uses JQuery (how embarrassing that would be if it were known!). So browsers made sure to prevent that from happening. Downloading a fresh copy of JQuery every 10 minutes is precisely what mobile phone users on a pay-as-you-go plan have been clamoring for.

Now that every household in Silicon Valley has intravenous 5g connectivity, it is not surprising that implementing streaming for partial page reloads has been a dystopian, Kafkaesque, waiting-for-Godot's-second-coming kind of a rollout.

Still, progress has been made, and today, all the browsers do have good api support for streaming partial page reloads. There are some rough edges, I'm finding, which will hopefully be ironed out soon [TODO, check if issues still persist]. be-written exists to provide declarative support on top of these API's. It provides a kind of inline iframe, but without the baggage of iframes -- the slow performance / being limited to a rectangle, to name the top two issues iframes have. Here's to hoping the browser vendors choose to show some much needed HTML love (like they've been doing for years with JavaScript) and provide first class support for declarative inclusiveness, making be-written a welcomed casualty.

(Sadly, the industry itself hasn't exactly made itself "worthy" of much HTML love from the browser vendors. Most advertisements follow the same bad practices the rest of the development community follows, maybe even worse, delivering their advertisement via JS alone, I'm finding. It seems like a catch-22 situation. What a waste (sigh).)

Anyway, a strong case can be made that to benefit from caching, lazy loading, etc, in some cases it is better to reference the HTML for a web component via client-side fetch.

So yes, this is yet another client-side include implementation, but one specifically for streaming declarative shadow DOM / declarative web components to the browser.

Security

It should be noted that be-written also has rudimentary support for import maps, as well as a custom link preload solution containing an onerror attribute. This in fact forms the cornerstone of the security checks, to prevent an attribute that might be corrupted via an XSS attack to reference any url arbitrarily. Only url's that are resolved by import maps and/or link preload (or any other value of rel, as long as the link tag was able to obtain an onerror attribute) can be imported via be-importing (and CSP can also help here).

Bundling

It often makes sense to want to bundle the HTML-based web component definitions when deploying to production. be-importing's base class be-written has that covered as well.

Functionality

be-importing extends be-written, by simply defaulting some of be-written's options to settings most applicable for (declarative) web components:

{
    between: ["<!--begin-->", "<!--end-->"],
    shadowRootMode: "open",
    once: true
};

The "between" setting allows us to create web components as html files that can also be opened directly in a web browser, making rudimentary demo's possible.

But more importantly, it also solves some difficult to overcome obstacles as far as managing where the light children should go, and the ability to pass properties down to the custom element ahead of the downloading completing. And also allowing the same file to be used as an embedded server-side include in scenarios where the benefits outweigh the costs of that approach.

The "shadowRoot" setting allows us to specify whether to wrap the imported content inside a shadowRoot. This is "off" by default for be-written, but defaults to "open" for be-importing, as that is more typically desired for web components.

The "once" setting allows us have repeated instances of the web component, including the import statement, without having to worry that multiple requests to the same resource will be made:

<xtal-side-nav be-importing=xtal-side-nav/xtal-side-nav.html>
    <section style='color:white'>
        <div>Menu Item 1</div>
        <div>Menu Item 2</div>
    </section>
</xtal-side-nav>
...
<xtal-side-nav be-importing=xtal-side-nav/xtal-side-nav.html>
    <section style='color:white'>
        <div>Menu Item 3</div>
        <div>Menu Item 4</div>
    </section>
</xtal-side-nav>

... only downloads the resource once, and (using loosely coupled functionality not discussed here) only defines one custom element.

Working example:

Using ES modules, import maps exclusively

<xtal-side-nav be-importing=xtal-side-nav/xtal-side-nav.html>
    <section style='color:white'>
        <div>Menu Item 1</div>
        <div>Menu Item 2</div>
    </section>
</xtal-side-nav>
<script type=importmap>{
    "imports":{
        "stream-orator/": "../node_modules/stream-orator/",
        "trans-render/": "../node_modules/trans-render/",
        "xtal-element/": "../node_modules/xtal-element/",
        "be-based/": "../node_modules/be-based/",
        "be-decorated/": "../node_modules/be-decorated/",
        "be-exportable/": "../node_modules/be-exportable/",
        "be-having/": "../node_modules/be-having/",
        "be-hive/": "../node_modules/be-hive/",
        "be-importing/": "../node_modules/be-importing/",
        "be-written/": "../node_modules/be-written/",
        "xtal-side-nav/": "../node_modules/xtal-side-nav/"
    }
}</script>
<script type=module>
    import 'be-importing/be-importing.js';
</script>

Note: We can give any name we want to the custom element, it doesn't have to match the default name specified by the html file!

Using CDN:

<xtal-side-nav be-importing=xtal-side-nav/xtal-side-nav.html>
    <section style='color:white'>
        <div>Menu Item 1</div>
        <div>Menu Item 2</div>
    </section>
</xtal-side-nav>

<script type=importmap>{
    "imports":{
        "xtal-side-nav/": "https://cdn.jsdelivr.net/npm/xtal-side-nav@0.0.110/"
    }
}</script>
<script type=module>
    import 'https://esm.run/be-importing@0.0.50';
</script>

Since be-importing is not a standard, CDN's have no out-of-the-box support for it, of course, thus the developer is burdened with specifying the import map base package location for where the HTML file can be found.

An alternative way of mapping the bare import specifier of the html file to a precise location is using the link preload tag:

<!-- This should probably go in index.html / head tag: -->
<head>
    ...
<link id=xtal-side-nav/xtal-side-nav.html 
    rel=preload as=fetch 
    href=https://cdn.jsdelivr.net/npm/xtal-side-nav@0.0.110/xtal-side-nav.html 
    onblur=console.error(href)>
...
</head>

<body>
    ...

<xtal-side-nav be-importing=xtal-side-nav/xtal-side-nav.html>
    <section style='color:white'>
        <div>Menu Item 1</div>
        <div>Menu Item 2</div>
    </section>
</xtal-side-nav>


<script type=module>
    import 'https://esm.run/be-importing@0.0.47';
</script>


...
</body>

Demo

Viewing Locally

Any web server that can serve static files will do but...

1. Install git.
2. Fork/clone this repo.
3. Install node.
4. Open command window to folder where you cloned this repo.

5. npm install

6. npm run serve

7. Open http://localhost:3030/demo/dev in a modern browser.

Importing in ES Modules:

import 'be-importing/be-importing.js';

Using from CDN:

<script type=module crossorigin=anonymous>
    import 'https://esm.run/be-importing';
</script>