451-tools

1.0.3 • Public • Published

451 Tools

Censorship resilient and distributed publishing. This repository contains a collection of scripts and tools to help you run your own censorship resilient web services. We mostly use Service Workers to achieve this.

Installation

npm install 451-tools

Table of contents

Getting started

To ensure smooth integration between our modules it is necessary to use the registerServiceWorkerController function. Make sure to include it in the options parameter when calling one of the modules, as illustrated below.

Usage

Inside your service worker:

import { registerBookmarkApi, registerServiceWorkerController } from '451-tools';

const serviceWorkerController = registerServiceWorkerController();
registerBookmarkApi({ serviceWorkerController });

Configuration

Individual modules can be configured by passing an object when calling the module function from your service worker. Next to that you can also configure all modules from one configuration file. You need to host that file on your server and pass the path of the file as a url parameter to your service worker registration path, as illustrated below.

From the client:

if ('serviceWorker' in navigator) {
  window.addEventListener('load', () => {
    const configurationVersion = 1;
    const serviceWorkerUrl = `/service-worker.js?configuration=${encodeURIComponent(`/static/service-worker-configuration.json?v=${configurationVersion}`)}`;
    navigator.serviceWorker
      .register(serviceWorkerUrl)
      .then((registration) => registration.update())
  });
}

Make sure to revision hash or version the configuration file, so that you can update it without having to update your service worker.

The configuration file should look like this, all modules are optional.

{
  "bookmark": {},
  "fallbackPages": {},
  "fallbackAssets": {}
}

For more information about the configuration options, see the documentation of the individual modules.

Timeout

The default timeout for network requests is 3 seconds. To adjust this timeout, pass it as an option while creating the Service Worker Controller:

const serviceWorkerController = registerServiceWorkerController({
  networkTimeoutSeconds: 10,
});

Bookmarking

We provide some API endpoints defined in the service worker to save pages for offline availability.

Usage

Inside your service worker:

import { registerBookmarkApi, registerServiceWorkerController } from '451-tools';

const serviceWorkerController = registerServiceWorkerController();
registerBookmarkApi({ serviceWorkerController });

When your service worker is installed, you can now call the bookmark API, which is available at /451-tools/bookmark/.

API endpoints

GET /451-tools/bookmark/

Get all pages from the bookmark cache.

Response

An array of bookmarked pages.

Property Type Description Example
url string The url of the bookmarked page htts://domain.org/path/to/the/page/
path string The path of the bookmarked page /path/to/the/page/
html string The html of the bookmarked pagw <html><head><title>Page title</title></head><body>Page content</body></html>
metadata object Optional metadata of the bookmarked page.
Passed when a page was bookmarked
{ title: 'Page title' }
Example
fetch('/451-tools/bookmark/', { method: 'GET' })
  .then(response => response.json())
  .then(pages => {
    console.log(pages);
  });

GET /451-tools/bookmark/{path}

Get a single page from the bookmark cache. Where the path is a url encoded path.

Response
200: A single bookmarked page.
Property Type Description Example
url string The url of the bookmarked page htts://domain.org/path/to/the/page
path string The path of the bookmarked page /path/to/the/page
html string The html of the bookmarked page <html><head><title>Page title</title></head><body>Page content</body></html>
metadata object Optional metadata of the bookmarked page.
Passed when a page was bookmarked
{ title: 'Page title' }
404: The page was not found in the bookmark cache.
400: The path is not a valid url encoded path.
Example
fetch('/451-tools/bookmark/%2Fpath%2Fto%2Fthe%2Fpage%2F', { method: 'GET' })
  .then(response => response.json())
  .then(page => {
    console.log(page);
  });

POST /451-tools/bookmark/{path}

Add a page to the bookmark cache. Where the path is a url encoded path.

Request body
Property Type Description Example
metadata object Optional metadata for the bookmarked page.
It can be anything you want as long as it is a valid object.
{ title: 'Page title' }
Example
fetch('/451-tools/bookmark/%2Fpath%2Fto%2Fthe%2Fpage%2F', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({ metadata: { title: 'Page title' } }),
});

DELETE /451-tools/bookmark/{path}

Delete a page from the bookmark cache. Where the path is a url encoded path.

Example
fetch('/451-tools/bookmark/%2Fpath%2Fto%2Fthe%2Fpage%2F', { method: 'DELETE' });

Fallback pages

We provide a fallback strategy for pages to use when the network is not available.

Usage

Inside your service worker:

import { registerFallbackPages, registerServiceWorkerController } from '451-tools';

const serviceWorkerController = registerServiceWorkerController();
registerFallbackPages({ serviceWorkerController });

Calling registerFallbackPages is enough to get you started. It will render a default offline page whenever the network is not available. By default, it looks like this:

Screenshot of the default offline page.

This page is somewhat customizable, it has support for basic theming and the default UI texts can be overwritten. However, in case this is not enough, we also support custom offline pages:

import { registerFallbackPages, registerFallbackHandlersPlugin } from '451-tools';

const serviceWorkerController = registerServiceWorkerController();
registerFallbackPages({
  serviceWorkerController,
  pages: [{
    url: '/my-offline-page/',
    revision: '1',
  }],
});

revision is not required, but it is highly recommended. Changing this value will cause a cache invalidation, which means we will re-cache the latest version.

⚠️ Please note that this module takes over the /451-tools/dashboard/ route regardless of the user's online status. The route is used to make the default offline page available for online users.

Options

Pass an object with the following properties to the registerFallbackPages function:

Option Description Required Type Default
serviceWorkerController See Getting started for more information. true true ServiceworkerController
pages A single or a list of offline fallback pages. true String | Object | Array -
language The language for the default offline page. The value will be used as a lang attribute on the HTML tag. See MDN for possible values. false String en
textDirection The text direction for the default offline page. The value will be used as a dir attribute on the HTML tag. Can be either ltr or rtl. false String ltr
customProperties Will be used to overwrite the default styles of the default offline page. The full list of over writable values can be found below. false Object {}
templateStrings Will be used to overwrite the default UI text strings of the default offline page. The full list of over writable values can be found below. false Object {}
showHiddenTabs Toggle the display of tabs for unused features on the default offline page. false Boolean false

customProperties

A list of all the over writable custom properties and their default values.

Key Description Default
background-color Color used as the page's background. #f9fbfc
border-color Color used for stylistic borders and separators. #e6e8ec
card-background-color Color used as the card's background. #ffffff
card-box-shadow Box shadow used by the cards, it can be removed by passing it a value of none. 0px 8px 24px rgba(149, 157, 165, 0.2)
container-width The main container's max width. 1200px
icon-color Color used for the header's offline icon. #e6e8ec
primary-color Color used for links, highlights, active & focus states, etc. Essentially, whenever we need to pull a user's attention to something. #1993f6
tab-text-color Color used as the tab's primary text color (non-active state). #7a7d95
text-color Color used as the primary text color. #2d2d3b

templateStrings

A list of all the over writable UI text strings and their default values.

Key Description Default
pageTitle Page title Woops, No Internet Connection
pageDescription Page description Please check your internet connection and try again.
homeLinkText The link text for the home link in the header Home
bookmarksTabTitle The title for the bookmarks tab Bookmarks
editorialTabTitle The title for the editorial(content bundles) tab Editorial
aboutTabTitle The title for the about tab About
copyright Copyright Copyright © ${new Date().getFullYear()}. All rights reserved.
aboutInnerHTML Inner HTML used to render the about tab -

Examples

Default offline page with customization

import { registerFallbackPages, registerServiceWorkerController } from '451-tools';

const serviceWorkerController = registerServiceWorkerController();
registerFallbackPages({
  serviceWorkerController,
  customProperties: {
    'primary-color': 'blue',
  },
  templateStrings: {
    pageTitle: 'My Offline Page',
  },
});

Custom offline page

import { registerFallbackPages, registerServiceWorkerController } from '451-tools';

const serviceWorkerController = registerServiceWorkerController();
registerFallbackPages({
  serviceWorkerController,
  pages: [{
    url: '/my-offline-page/',
    revision: '1',
  }],
});

Multiple offline pages

For most cases, a single fallback is enough. However, we support an optional refererRegex property. This can, for example, be used to render a different fallback page for users who came from a search engine:

import { registerFallbackPages, registerServiceWorkerController } from '451-tools';

const serviceWorkerController = registerServiceWorkerController();
registerFallbackPages({
  serviceWorkerController,
  pages: [{
    url: '/my-offline-page/',
    revision: '1',
  },
    {
      url: "/google-search-offline-page/",
      revision: "1",
      refererRegex: "https://(www.)?(google).com/",
    },
  ],
});

Fallback assets

We provide a fallback strategy for assets to use when the network is not available.

Usage

Inside your service worker:

import { registerFallbackAssets, registerServiceWorkerController } from '451-tools';

const serviceWorkerController = registerServiceWorkerController();
registerFallbackAssets({
  serviceWorkerController,
  assets: [
    {
      url: '/main.css',
      revision: '1',
    },
    {
      url: '/main.js',
      revision: '1',
    },
  ],
});

revision is not required, but it is highly recommended. Changing this value will cause a cache invalidation, which means we will re-cache the latest version.

Options

Pass an object with the following properties to the registerFallbackAssets function:

Option Description Required Type
serviceWorkerController See Getting started for more information. true ServiceworkerController
assets A list of offline fallback assets. true String | Object | Array

Content bundles

You can define a list of pages as content bundles to be cached by the service worker. These are pages that you want to be available offline.

Usage

Inside your service worker:

import { registerContentBundles, registerServiceWorkerController } from '451-tools';

const serviceWorkerController = registerServiceWorkerController();
registerContentBundles({
  serviceWorkerController,
  pages: [
    {
      url: '/content-bundle-1/',
      revision: '1',
    },
    {
      url: '/content-bundle-2/',
      revision: '1',
    },
  ],
});

Options

Option Description Required Type
serviceWorkerController See Getting started for more information. true ServiceworkerController
pages A list of pages. true String | Object | Array

Fallback image

We provide a fallback strategy for images, it returns a fallback SVG for uncached images.

Usage

Inside your service worker:

import { registerFallbackImage, registerServiceWorkerController } from '451-tools';

const serviceWorkerController = registerServiceWorkerController();
registerFallbackImage({ serviceWorkerController });

Mirroring

In order to keep a website more available you can configure alternative mirrors where the website is hosted. The service worker will always try to get resources from the primary domain. But once the primary domain fails and mirrors are configured content will be fetched from one of the configured mirrors. We will timeout and cancel a request after 3 seconds.

We mark a mirror and its status, which means that for subsequent requests a mirror that was marked as up will be used for the next 10 minutes. We feel that this leads to a faster user experience because we will not encounter the timeout for every request that is done.

If all mirrors fail the offline fallback pages strategy as defined in fallback pages will be used. If the offline fallbakc page is not configured the request will simply fail.

Usage

Inside your service worker:

import { registerMirroring, registerServiceWorkerController } from '451-tools';

const serviceWorkerController = registerServiceWorkerController();
registerMirroring({ serviceWorkerController });

⚠️ Please note that the order of registering modules is important. Especially with mirroring. Since it will match all document requests and same origin requests it is important to register it last.

Once the service worker is installed, you can call the mirror status API endpoint, which is available at /451-tools/mirror-status/. It will return the status of where the content is coming from. The response looks like this:

{
  "status": "up" // or "unknown", "down", "low" 
}
Status Description
unknown The status of the primary domain is not known (yet).
up Content is coming from the primary domain.
down It is not possible to get up to date resources from either the primary domain or any (all) of the defined mirrors.
low It is not possible to get resources from the primary domain, but it is still possible to get up to date resources from one of the mirrors.

Options

Pass an object with the following properties to the registerMirroring function:

Option Description Required Type
serviceWorkerController See Getting started for more information. true ServiceworkerController
urls A list of alternative urls where the website is available. true String[]

UI components

In addition to our modules, we offer a set of UI components that come in the form of web components. While these components may offer added functionality, their primary function is to provide an alternative approach to implementing our modules without requiring any JavaScript coding.

Bookmark button

The button displays the bookmark status of the related page - whether it is currently bookmarked or not. In addition, it enables users to bookmark or unbookmark the page with just a click. Please note that this button requires the bookmarking module to function properly.

Usage

Inside your HTML:

<bookmark-button
  path="/my-page"
  bookmark-text="Bookmark"
  remove-bookmark-text="Remove Bookmark"
>
</bookmark-button>

Attributes

Pass the following attributes to the bookmark-button component:

Attribute Description Required Default
path The page's path. true -
bookmark-text The label displayed when the page is not bookmarked. false Bookmark
remove-bookmark-text The label displayed when the page is bookmarked. false Remove Bookmark
metadata-title Optional page metadata can be added to provide additional information about the page. This is also used by the default offline page to render the cards. false -
metadata-description Optional page metadata can be added to provide additional information about the page. This is also used by the default offline page to render the cards. false -
metadata-image-src Optional page metadata can be added to provide additional information about the page. This is also used by the default offline page to render the cards. false -
metadata-image-height Optional page metadata can be added to provide additional information about the page. This is also used by the default offline page to render the cards. false -
metadata-image-width Optional page metadata can be added to provide additional information about the page. This is also used by the default offline page to render the cards. false -

Styling

Use the following selector:

bookmark-button::part(button) {
  background-color: rebeccapurple;
  color: white;
}

Traffic light

The component displays the user's network status - it can either be online or offline.

Usage

Inside your HTML:

<traffic-light
  href="/offline/"
  online-text="You are online."
  offline-text="You are offline."
>
</traffic-light>

Attributes

Pass the following attributes to the traffic-light component:

Attribute Description Required Default
online-text The label read out loud by screen readers when the user is online. false Online
offline-text The label read out loud by screen readers when the user is offline. false Offline
href The URL that the hyperlink points to. false -
target Where to display the linked URL. false -
rel The relationship of the linked URL as space-separated link types. false -

Styling

Use the following selectors:

traffic-light::part(button) {
  /* General button selector. */
}

traffic-light::part(online) {
  /* Online button selector. */
}

traffic-light::part(offline) {
  /* Offline button selector. */
}

Package Sidebar

Install

npm i 451-tools

Weekly Downloads

4

Version

1.0.3

License

none

Unpacked Size

191 kB

Total Files

26

Last publish

Collaborators

  • article19