❤
npm
Sign UpSign In

@idio/idio

1.6.2 • Public • Published

@idio/idio

npm version Node.js CI

@idio/idio contains Koa's fork called Goa — web server compiled with Closure Compiler so that its source code is optimised and contains only 1 external dependency (mime-db). Idio adds essential middleware to Goa for session, static files, CORS and compression and includes the router. As the project grows, more middleware will be added and optimised.

This is a production-ready server that puts all components together for the ease of use, while providing great developer experience using JSDoc annotations for auto-completions. Idio is not a framework, but a library that enables idiomatic usage and compilation of the server and middleware.

Developer-Friendly Suggestions For Middleware 

idio~:$ \
yarn add @idio/idio
npm install @idio/idio

Example Apps

There are some example apps that you can look at.

  1. File Upload: a front-end + back-end application for uploading photos. Demo requires GitHub authorisation without any scope permissions to enable session middleware showcase.
  2. Akashic.Page: a service for managing email and web-push subscriptions, with JS widgets and Mongo database connection.

Table Of Contents

API

The package is available by importing its default function and named components:

import idio, { Keygrip, Router } from '@idio/idio'

async idio(
  middlewareConfig=: !MiddlewareConfig,
  config=: !Config,
): !Idio

Start the server. Sets the proxy property to true when the NODE_ENV is equal to production.

  • middlewareConfig !MiddlewareConfig (optional): The middleware configuration for the idio server.
  • config !Config (optional): The server configuration object.

The app can be stopped with an async .destroy method implemented on it that closes all connections.

There are multiple items for middleware configuration:

MiddlewareConfig extends FnMiddlewareConfig: Middleware configuration for the idio server.

Name Type Description
static (!StaticOptions | !Array<!StaticOptions>) Static middleware options.
compress (boolean | !CompressOptions) Compression middleware options.
session !SessionOptions Session middleware options.
cors !CorsOptions CORS middleware options.
form !FormDataOptions Form Data middleware options for receiving file uploads and form submissions.
frontend !FrontEndOptions Front End middleware allows to serve source code from node_modules and transpile JSX.
neoluddite !NeoLudditeOptions Records the usage of middleware to compensate their developers' intellectual work.
csrfCheck !CsrfCheckOptions Enables the check for the presence of session with csrf property, and whether it matches the token from either ctx.request.body or ctx.query.
github (!GitHubOptions | !Array<!GitHubOptions>) Sets up a route for GitHub OAuth authentication. The returned middleware will be installed on the app automatically so it doesn't need to be passed to the router.
jsonErrors (boolean | !JSONErrorsOptions | !Array<!JSONErrorsOptions>) Tries all downstream middleware, and if an error was caught, serves a JSON response with error and stack properties (only if exposeStack is set to true). Client errors with status code 4xx (or that start with !) will have full message, but server errors with status code 5xx will only be served as { error: 'internal server error '} and the app will emit an error via app.emit('error') so that it's logged.
jsonBody (boolean | !JSONBodyOptions) Allows to parse incoming JSON request and store the result in ctx.request.body. Throws 400 when the request cannot be parsed.
logarithm !LogarithmOptions Options to record hits in ElasticSearch.

The types for starting the server include the address, port and router configuration.

Config: Server configuration object.

Name Type Description Default
port number The port on which to start the server. 5000
host string The host on which to listen. 0.0.0.0
router !_goa.RouterConfig The configuration for the router. -

After the app is started, it can be accessed from the return type.

Idio: The return type of the idio.

Name Type Description
url* string The URL on which the server was started, such as http://localhost:5000.
server* Node.JS Docs!http.Server The server instance.
app* !Application The Goa application instance (with additional .destroy method).
middleware* !ConfiguredMiddleware An object with configured middleware functions, which can be installed manually using app.use, or router.use. The context will be a standard Goa context with certain properties set by bundled middleware such as .session.
router* !Router The router instance.

All middleware can be accessed from the middleware property, so that it can be installed on individual basis on specific routes, if it's not used app-wise.

ConfiguredMiddleware extends MiddlewareObject: Idio-specific properties of the middleware object.
Name Type Description
form !_multipart.FormData An instance of the form data class that can be used to create middleware.
session !Middleware The session middleware to be installed on individual routes.
frontend !Middleware The frontend middleware.
csrfCheck !Middleware Configured CSRF check middleware.
jsonErrors (!Middleware | !Array<!Middleware>) Middleware to server errors as JSON.

The example below starts a simple server with session and custom middleware, which is installed (used) automatically because it's defined as a function.

Source Output 
const { url, app,
  middleware: { session, form },
  router,
} = await idio({
  // Developers' payment scheme neoluddite.dev
  neoluddite: {
    env: process.env.NODE_ENV,
    key: '0799b7f0-d2c7-4903-a531-00c8092c2911',
    app: 'idio.example',
  },
  // Idio's bundled middleware.
  session: {
    algorithm: 'sha512',
    keys: ['hello', 'world'],
    prefix: 'example-',
  },
  static: {
    use: true,
    root: 'upload',
  },
  form: {
    config: {
      dest: 'upload',
    },
  },
  // Any middleware function to be use app-wise.
  async middleware(ctx, next) {
    console.log('//', ctx.method, ctx.path)
    await next()
  },
})
app.use(router.routes())
router.get('/', session, (ctx) => {
  ctx.body = 'hello world'
})
router.post('/upload', session, async (ctx, next) => {
  if (!ctx.session.user) {
    ctx.status = 403
    ctx.body = 'you must sign in to upload'
    return
  }
  await next()
}, form.single('/upload'), (ctx) => {
  // db.create({
  //  user: ctx.session.id,
  //  file: ctx.req.file.path,
  // })
  ctx.body = 'Thanks for the upload. Link: ' +
    `${url}/${ctx.file.filename}`
})
 
http://localhost:5000
// GET /
hello world

Middleware

Idio's advantage is that is has the essential middleware, that was compiled together with the server, so that the packages are reused and memory footprint is low.

Static

🗂 Explore Static Middleware Configuration

Used to serve static files, such as stylesheets, images, videos, html and everything else. Will perform mime-type lookup to serve the correct content-type in the returned header.

Static source The Output 
const { url, app } = await idio({
  static: {
    root: 'example', use: true,
  },
// or multiple locations
  static: [{
    root: ['example'], use: true,
  }, {
    root: ['wiki'], use: true,
  }],
}, { port: null })
 
/** http://localhost:57537/app.css */ 

body {
  font-size: larger;
}
Show Response Headers 
Content-Length: 29
Last-Modified: Thu, 18 Jul 2019 14:34:31 GMT
Cache-Control: max-age=0
Content-Type: text/css; charset=utf-8
Date: Thu, 05 Mar 2020 13:30:57 GMT
Connection: close
Content-Length: 114
Last-Modified: Sat, 28 Dec 2019 18:07:31 GMT
Cache-Control: max-age=0
Content-Type: image/svg+xml
Date: Thu, 05 Mar 2020 13:30:59 GMT
Connection: close

Session

👳‍♂️Explore Session Middleware Configuration

Allows to store data in the .session property of the context. The session is serialised and placed in cookies. When the request contains the cookie, the session will be restored and validated (if signed) against the key.

Session Config
const { url, app } = await idio({
  session: { use: true, keys:
    ['hello', 'world'], algorithm: 'sha512' },
  async middleware(ctx, next) {
    if (ctx.session.user)
      ctx.body = 'welcome back '
        + ctx.session.user
    else {
      ctx.session.user = 'u'
        + (Math.random() * 1000).toFixed(1)
      ctx.body = 'hello new user'
    }
    await next()
  },
})
The session data is encrypted with base64 and signed by default, unless the .signed option is set to false. Signing means that the signature will contain the hash which will be validated server-side, to ensure that the session data was not modified by the client. The default algorithm for signing is sha1, but it can be easily changed to a more secure sha512. 
// GET /
"hello new user"
/* set-cookie */
[
  {
    name: 'koa:sess',
    value: 'eyJ1c2VyIjoidTg2LjciLCJfZXhwaXJlIjoxNTgzNTAxNDU5ODQzLCJfbWF4QWdlIjo4NjQwMDAwMH0=',
    path: '/',
    expires: 'Fri, 06 Mar 2020 13:30:59 GMT',
    httponly: true
  },
  {
    name: 'koa:sess.sig',
    value: '5hRueSOyLuhp6nZvOi4TcziXNiADlaIhE6fJHruR-I8cGtEVDYCgNe9t3LS0SyV-SEN1kPa8ZwIz-a91GWPw-A',
    path: '/',
    expires: 'Fri, 06 Mar 2020 13:30:59 GMT',
    httponly: true
  }
]
// GET /
"welcome back u86.7"

CORS

👮‍♀️Explore CORS Middleware Configuration

To enable dynamic communication between clients and the server via JavaScript requests from the browser, the server must respond with Access-Control-Allow-Origin header that sets the appropriate allowed Origin. This middleware is easy to use on production and development environments.

CORS source The Output 
const { NODE_ENV } = process.env

const { url, app } = await idio({
  async example(ctx, next) {
    console.log('//', ctx.method,
      ctx.path, 'from', ctx.get('Origin'))

    ctx.body = 'hello world'
    await next()
  },
  cors: {
    use: true,
    origin: NODE_ENV == 'production' ?
      'http://prod.com' : null,
    allowMethods: ['GET', 'POST'],
  },
})
 
// GET / from https://3rd.party
{
  vary: 'Origin',
  'access-control-allow-origin': 'http://prod.com',
  date: 'Thu, 05 Mar 2020 13:31:01 GMT',
  connection: 'close'
}

// GET / from http://prod.com
{
  vary: 'Origin',
  'access-control-allow-origin': 'http://prod.com',
  date: 'Thu, 05 Mar 2020 13:31:01 GMT',
  connection: 'close'
}

// OPTIONS / from http://prod.com
{
  vary: 'Origin',
  'access-control-allow-origin': 'http://prod.com',
  'access-control-allow-methods': 'GET,POST',
  date: 'Thu, 05 Mar 2020 13:31:01 GMT',
  connection: 'close'
}

Compression

🗜Explore Compression Middleware Configuration

When the body of the response is non-empty, it can be compressed using gzip algorithm. This allows to save data transmitted over the network. The default threshold is 1024 bytes, since below that the benefits of compression are lost as the compressed response might end up being even larger.

Compression source The Output 
const { url, app } = await idio({
  async serve(ctx, next) {
    console.log('//',
      ctx.method, ctx.path)

    ctx.body = packageJson
    await next()
  },
  compress: {
    use: true,
  },
})
 
// GET /
{
  'content-type': 'application/json; charset=utf-8',
  vary: 'Accept-Encoding',
  'content-encoding': 'gzip',
  date: 'Thu, 05 Mar 2020 13:31:01 GMT',
  connection: 'close',
  'transfer-encoding': 'chunked'
}

File Upload

🖼Explore Form Data Middleware Configuration

Browser will submit forms and send files using multipart/form-data type of request. It will put all fields of the form together and stream them to the server, sending pairs of keys/values as well as files when they were attached. The Form Data middleware is the Multer middleware specifically rewritten for Koa that can handle file uploads.

File Upload source The Output 
const { url, app, router, middleware: {
  form,
} } = await idio({
  form: {
    dest: 'example/upload',
  },
})
app.use(router.routes())
router.post('/example',
  form.single('bio'),
  (ctx) => {
    delete ctx.file.stream
    ctx.body = { file: ctx.file,
      body: ctx.request.body }
  }
)
 
{
  file: {
    fieldname: 'bio',
    originalname: 'bio.txt',
    encoding: '7bit',
    mimetype: 'application/octet-stream',
    destination: 'example/upload',
    filename: '106e5',
    path: 'example/upload/106e5',
    size: 29
  },
  body: { hello: 'world' }
}

Front End

🌐Explore Front End Middleware Configuration

Web applications are always full stack and involve both back-end together with front-end. Whereas all previously described middleware was for the server only, the front-end middleware facilitates browser development, as it allows to serve source code from the node_modules directory and transpile JSX. Modern browsers support modules, but JavaScript needs to be patched to rename imports like

// was
import X from 'package-name'
// becomes
import X from '/node_modules/package-name/src/index.mjs'

This is achieved by resolving the module field from package.json of served packages (with fallback to the main field, but in that case require statements will not work).

Configuration JSX Component 
const { url, app } = await idio({
  frontend: {
    use: true,
    directory: 'example/frontend',
  },
})
 
import { render, Component } from 'preact'

class MyComp extends Component {
  render() {
    return (<div className="example">
      Hello World!
    </div>)
  }
}

render(MyComp, document.body)
Using the simple configuration from above, and a JSX file, the browser will receive the following patched source code. The middleware will also look for requests that start with the /node_modules path, and serve them also. The pragma (import { h } from 'preact') is also added automatically, but it can be configured. 
import { h } from '/node_modules/preact/dist/preact.mjs'
import { render, Component } from '/node_modules/preact/dist/preact.mjs'

class MyComp extends Component {
  render() {
    return (h('div',{className:"example"},
      `Hello World!`
    ))
  }
}

render(MyComp, document.body)

The idea here is to provide a basic mechanism to serve front-end JavaScript code, without inventing any module systems, adapting to CommonJS, or transpiling old features. We simply want to execute our modern code and browsers are more than capable to do that, without us having to run complex build systems on the development code. Our simple JSX parser is not rocket science either and works perfectly well without building ASTs (but check for minor limitations in Wiki).

Additional Middleware

There are some small bits of middleware that can be used in server as well, but which are not essential to its functioning. They are listed in 📖 Wiki.

  • csrfCheck: Ensures that the csrf token from session matches one in the request.
  • jsonErrors: Allows to serve errors as JSON, which is useful for APIs.
  • jsonBody: Parses requests with the application/json content type into ctx.request.body.
  • logarithm: Record hits in ElasticSearch.
  • github: Sets up GitHub OAuth routes.

Custom Middleware

When required to add any other middleware in the application not included in the Idio bundle, it can be done in several ways.

  1. Passing the middleware function as part of the MiddlewareConfig. It will be automatically installed to be used by the Application. All middleware will be installed in order it is found in the MiddlewareConfig. 
    import idio from '@idio/idio'

const APIServer = async (port) => {
  const { url } = await idio({
    // 1. Add logging middleware.
    async log(ctx, next) {
      await next()
      console.log(' --> API: %s %s %s', ctx.method, ctx.url, ctx.status)
    },
    // 2. Add always used error middleware.
    async error(ctx, next) {
      try {
        await next()
      } catch (err) {
        ctx.status = 403
        ctx.body = err.message
      }
    },
    // 3. Add validation middleware.
    async validateKey(ctx, next) {
      if (ctx.query.key !== 'app-secret')
        throw new Error('Wrong API key.')
      ctx.body = 'ok'
      await next()
    },
  }, { port })
  return url
}

export default APIServer
    Started API server at: http://localhost:5005
 --> API: GET / 403
 --> API: GET /?key=app-secret 200
  2. Passing a configuration object as part of the MiddlewareConfig that includes the middlewareConstructor property which will receive the reference to the app. Other properties such as conf and use will be used in the same way as when setting up bundled middleware: setting use to true will result in the middleware being used for every request, and the config will be passed to the constructor. 
    import rqt from 'rqt'
import idio from '@idio/idio'
import APIServer from './api-server'

const ProxyServer = async (port) => {
  // 1. Start the API server.
  const API = await APIServer(5001)
  console.log('API server started at %s', API)

  // 2. Start a proxy server to the API.
  const { url } = await idio({
    async log(ctx, next) {
      await next()
      console.log(' --> Proxy: %s %s %s', ctx.method, ctx.url, ctx.status)
    },
    api: {
      use: true,
      async middlewareConstructor(app, config) {
        // e.g., read from a virtual network
        app.context.SECRET = await Promise.resolve('app-secret')

        /** @type {import('@typedefs/goa').Middleware} */
        const fn = async (ctx, next) => {
          const { path } = ctx
          const res = await rqt(`${config.API}${path}?key=${ctx.SECRET}`)
          ctx.body = res
          await next()
        }
        return fn
      },
      config: {
        API,
      },
    },
  }, { port })
  return url
}
    API server started at http://localhost:5001
Proxy started at http://localhost:5002
 --> API: GET /?key=app-secret 200
 --> Proxy: GET / 200

Router Set-up

After the Application and Router instances are obtained after starting the server as the app and router properties of the returned object, the router can be configured to respond to custom paths. This can be done by assigning configured middleware from the map and standalone middleware, and calling the use method on the Application instance.

import { collect } from 'catchment'
import idio from '@idio/idio'

const Server = async () => {
  const {
    url, router, app, middleware: { pre, post, bodyparser },
  } = await idio({
    // 1. Configure middlewares via middlewareConstructor without installing them.
    pre: {
      middlewareConstructor() {
        return async function(ctx, next) {
          console.log('  <-- %s %s',
            ctx.request.method,
            ctx.request.path,
          )
          await next()
        }
      },
    },
    post: {
      middlewareConstructor() {
        return async function(ctx, next) {
          console.log('  --> %s %s %s',
            ctx.request.method,
            ctx.request.path,
            ctx.response.status,
          )
          await next()
        }
      },
    },
    bodyparser: {
      middlewareConstructor() {
        return async (ctx, next) => {
          let body = await collect(ctx.req)
          if (ctx.is('application/json')) {
            body = JSON.parse(body)
          }
          ctx.request.body = body
          await next()
        }
      },
    },
  }, { port: 5003 })

  // 2. Setup router with the bodyparser and path-specific middleware.
  router.post('/example',
    pre,
    bodyparser,
    async (ctx, next) => {
      ctx.body = {
        ok: true,
        request: ctx.request.body,
      }
      await next()
    },
    post,
  )
  app.use(router.routes())
  return url
}
Logging Response 
Page available at: http://localhost:5003
  <-- POST /example
  --> POST /example 200

// server response:
{ ok: true, request: { hello: 'world' } }

Also checkout the Router package that allows to automatically initialise routes from a given directory, and watch for changes in them during development. This means you don't have to refresh the server manually after a change to a route.

const w = await initRoutes(router, 'routes', {
  middleware,
})
if (process.env.NODE_ENV == 'prod') watchRoutes(w)

SSR

Idio supports Server-Side rendering of JSX components (same restrictions apply as for front-end). You can easily mark up your back-end pages using full-scale HTML, or basic placeholders in which you can then render your front-end app.

import idio, { render } from '@idio/idio'

const { url, app, router } = await idio()
router.get('/', (ctx) => {
  ctx.body = render(<html>
    <head>
      <title>Example</title>
    </head>
    <body>
      Hello World!
    </body>
  </html>, {
    addDoctype: true,
    pretty: true,
  })
})
app.use(router.routes())
<!doctype html>
<html>
  <head><title>Example</title></head>
  <body>Hello World!</body>
</html>

NeoLuddite.Dev

This web server integrates with NeoLuddite: the package monetary reward scheme. It's currently in beta, and this section will be relevant when it's open to the public.

Every time you invoke certain functionality in a package somebody has written (e.g., koa-static for static files, koa-session for creation of session), via Idio, your usage will be counted and your balance in Ludds on the neoluddite server will be transferred to the software engineer as a reward for his/her intellectual work. Contact license@neoluddite.dev for any requests.

const { url, app,
  middleware: { session, form },
  router,
} = await idio({
  // Developers' payment scheme neoluddite.dev
  neoluddite: {
    env: process.env.NODE_ENV,
    key: '0799b7f0-d2c7-4903-a541-10d8092c2911',
    app: 'idio.example',
  },
  // ...
}

The usage will be billed for apps running in production mode, therefore the env variable is needed. Setting the app has no effect but allows to break down statistics by web application on the portal. See the license section for more info.

NeoLudditeOptions: Options for the neoluddite.dev client.

Name Type Description Default
key* string The API key received from the app. -
env string The environment (e.g., dev/staging). The production env must be indicated as prod which is billed. -
host string The hostname of the server. https://neoluddite.dev
app string The name of the application. -

WebSocket

We've implemented a library to upgrade requests into WebSocket connections. You can read more at the actual package page. Idio simply exports this method via its API. You need to configure it yourself.

import idio, { websocket } from '@idio/idio'

const { url, app, server } = await idio()
// clients stores current connections against ID
const clients = websocket(server, {
  onConnect(clientId) {
    // the value is a function to send messages
    clients[clientId]('intro-event', 'Hello Client!')
  },
})

Copyright & License

GNU Affero General Public License v3.0

Affero GPL means that you're not allowed to use this web server on the web unless you release the source code for your application. This is a restrictive license which has the purpose of defending Open Source work and its creators.

To be able to use the server, just set up a monthly payment on Open Collective for any amount of your choice.

All original work on middleware and Koa are under MIT license. See Goa Page for the list of packages and modules used in compilation of the Goa server, and the package.json file for dependencies of this project (todo: create wiki page w/ licenses table).

Art Deco © Art Deco™ for Idio 2020 Idio

Readme

Keywords

Package Sidebar

Install

npm i @idio/idio

Repository

github.com/idiocc/idio

Homepage

www.idio.cc/

Weekly Downloads

4

Version

1.6.2

License

AGPL-3.0

Unpacked Size

1.13 MB

Total Files

34

Last publish

Collaborators

  • zvr
Try on RunKit
Report malware