External is a dual purpose library. It ships with a client-side framework renders third party or external pages in the most optimal way as possible. This is done using various of techniques:
- The payload is downloaded using a fully async streaming XHR request. This way we can continuously update and render our placeholder while data flows and therefor reducing the time to render.
- All assets of the page are loaded async, this includes the CSS.
- The received client code is wrapped before execution so client code can re-use our dependencies while keeping a sandboxed approach.
- While the client was specifically written for the BigPipe framework it should work against any back-end as long as it returns the same data structure.
- Templates are rendered using React so it's easy to compose and update.
But we also ship with a server-side framework implementation for BigPipe which makes it possible to serve the client and automatically format all the output in the expected HTML structure.
Table of Contents
The client-side component is composed from various of tiny modules and can be
build using Browserify. It can be build-in to other browserify components by
simply requiring the
external module in your client-code.
The server side part of this framework can be installed through npm:
npm install external
In addition to providing a browserify-able client-side script there is also a
compiled version of this code which lives in the
dist folder called
extern.js. This pre-compiled library exposes it self using the
and therefor does not introduced
require statement as globals. In all the code
examples in documentation we assume that you have an
Extern global. If you use
dist build you can skip the following example:
var Extern = ;
If you want to generate new stand alone bundles of the Extern library you can
dev scripts using the
npm run command. These
commands do assume that you've installed the
devDependencies of this project.
To generate a new production build,
npm run prepublish
As this is a
prepublish script, it means that every release to npm will have
dist/extern.js included. So if browserify isn't your think, you can just
To generate an un-minified build for development purposes you can run:
npm run dev
This will generate a new
Now that you know how to install it and what type of bundles there are you can decide how to serve the library. When this module is used as plugin in BigPipe it will automatically serve the browserify and plugin combined bundle from:
We also mount our
dist folder on the server so the static assets in this
folder can also be served:
Now that you've picked your build, and know how the files are served you can simply put the script tag in your page and your ready to display external pages/apps.
The easiest way to have
Extern load your remote pages is by using the
Extern.listen method in combination with the
rel="extern" attributes on
Extern.listen method will gather all
<a> elements and search for a
that is set to
extern and uses the set
href of the element as URL that needs
to be remotely loaded.
The following options are supported:
timeoutTimeout for dependency loading. If assets take longer we should render and error template instead. The timeout is in milliseconds.
documentReference to the
documentglobal can be useful if assets need to be loaded in iframes instead of the global document.
classNameIf a link has this className we will automatically load it in the placeholder. This className will also automatically be add and removed once the link is clicked. Defaults to
var extern = '' documentbodytimeout: 10000;
extern instance is actually an
EventEmitter3 instance so you
can listen to the various of events that we're emitting:
- error Emitted when something went so horribly wrong that we decided to
show the error template. This event receives the actual
- done The streaming XHR is finished with loading.
- name:render Called when a fragment is about to render in to the
namepart in the event should be name of the fragment you want to listen for.
- name:loaded All the assets are loaded for the given placeholder name.
The client code for each fragments are loaded through an XHR connection. This
way we can safely executed third party code by wrapping the execution in a
try/catch statement. But not only does this allow us to wrap code, it also
allows us to introduce variables in the function. The following variables are
introduced as "globals":
React, This is the
require, Reference to our
requirestatement so you can re-use all the bundled things.
The following properties and methods are exposed on the Extern instance.
Exposed on the constructor
Scan the current document for all
<a rel="extern"> elements and attach click
listeners to it so we can automatically update the supplied placeholder with the
contents of the set URL. This method accepts one argument and that is the
placeholder DOM element where all pages should loaded in
Exposed on the constructor
Merge the object of the second argument in to the first argument. It returns the fully merged first argument.
var x = Extern;
Exposed on the constructor
A reference to the
requests module that we're using for our XHR requests.
var requests = Externrequests
See unshiftio/requests for more information.
This library ships with a custom Fittings framework implementation for
BigPipe which allows us to control how everything is processed inside of
BigPipe. Adding it to your BigPipe instance is just as simple as passing a
framework option while creating a new instance:
'use strict';var BigPipe =Extern = ;var app = BigPipe;
But the framework can also be set after the construction using the
Please do note that the current Fittings implentation is in the BigPipe master branch but will out in the release that follows 0.9
Once the fittings are installed on the application, it will start spitting out responses based on the specified Wire Format below. The processing instructions can be found in the instructions folder in the root of this repository. But before fiddling with these files I would suggest giving the README.md of Fittings a read so you know how the data formatting works.
In order to have the broadest support within this framework we came up with a dedicated wire-format in order to have the server-side and client-side components interact with each other. While this wire-format is mostly catered to the needs of an application that is build using the BigPipe framework it should be relatively easy to produce exactly the same output in different frameworks and programming languages. This wire format is also required in order to make streaming data as simple as possible as we can trigger buffer flushes based on this.
The format that we're using is
JSON. Every time we
\u1337 character on the client-side we assume it's the end of
chunk that requires processing. The JSON payload that is send should contain the
_idA unique id for the payload that is flushed.
nameName of the payload that is flushed. This is used to track potential child->parent references throughout the flushed payload.
detailsAn object that contains:
cssArray with path names for the CSS files that need to be loaded on the page. We will automatically prepend the server address to these assets.
stateAdditional state that will be spread on the component when we render it.
templateAn initial HTML template that should be rendered in the given placeholder.
In order to be able to load CSS assets fully async in every browser we need to
know when the styles are applied. This is done by Extern client by adding a DOM
element to the page that has an
id attribute which contains
_ and the filename of
the asset that is being downloaded (
#_yourfilename). We therefor require
that the CSS file contains CSS selector and sets the
height property to
42px. This allows us to poll the element for height changes to know when the
CSS is fully loaded. So if we have a file called
1aFafa801jz09.css it should
have the following selector in the source:
This project has been released under the MIT license, see [LICENSE].