kettle

Declarative IoC-based framework for HTTP and WebSockets servers on node.js based on express and ws

Kettle

Kettle is an integration technology which promotes the expression of servers handling HTTP and WebSockets endpoints. With a few exceptions, Kettle implements no primary functionality of its own, but aggregates the facilities of express and ws, as well as middleware held in the wider pillarjs "Bring your own HTTP Framework Framework" ecosystem. Kettle applications can easily incorporate any express-standard middleware, as well as coexisting with standard express apps targeted at the same node.js http.Server. Since Kettle applications are expressed declaratively, in the JSON format encoding Infusion's component trees, it is possible to adapt existing applications easily, as well as inserting middleware and new handlers anywhere in the pipeline without modifying the original application's code. This makes Kettle suitable for uses where application functionality needs to be deployed flexibly in a variety of different configurations.

In fact, Kettle's dependency on express itself is minimal, since the entirety of the Kettle request handling pipeline is packaged as a single piece of express-compatible middleware – Kettle could be deployed against any other consumer of middleware or even a raw node.js HTTP server.

Contents of this repository

This is packaged as Infusion grades derived from kettle.server, kettle.request and kettle.app. The first two of these exist in variants specialized both for plain HTTP (with the .http suffix) and for WebSockets (with the .ws suffix) – kettle.app does not specialize.

As well as the integration technology implementing Kettle itself, this repository also contains functionality helpful for testing HTTP and WebSockets servers written in arbitrary technologies. This is accessed by running kettle.loadTestingSupport() after having called require("kettle"). Kettle testing support allows HTTP and WebSockets client requests to be packaged as Infusion components, suitable for use with Infusion's IoC Testing Framework. Any user of Kettle's testing support needs to have node-jqunit registered as a member of their own project's devDependencies in their own package.json.

Kettle runs on node.js version 4.x (see package.json for current dependency profile).

The Kettle repository also contains a few implementations of the simple DataSource contract for read/write access to data with a simple semantic (broadly the same as that encoded in CRUD although the current DataSource semantic does not provide explicitly for deletion). See the documentation section on DataSources for details of this contract, the available implementations and how to use them.

This repository contains DataSource implementations suitable for HTTP endpoints (with a particular variety specialised for accessing CouchDB databases with CRUDlike semantics) as well as the filesystem, with an emphasis on JSON payloads.

Getting Started and Community

Firstly, install node and npm by running a standard installer from node.js. Clone this repository and then run npm install.

Issue tracking is at http://issues.fluidproject.org/browse/KETTLE .

Visit #fluid-work on EFNet – community resources are linked at Fluid's IRC Channels.

Contact us on the fluid-work mailing list with any problems or comments.

The primary user of Kettle is the GPII's autopersonalisation infrastructure, held at GPII/universal. Kettle is used to provide a flexible means of deploying the GPII's "Flow Manager" and related components distributed across multiple local and remote installations.

A closely related project to Kettle is gpii-express which is used in other GPII projects such as the terms registry and unified listing. This is similar in architecture to Kettle (wrapping express primitives such as servers and requests into dynamically constructed Infusion components) but slightly different in emphasis –

  • gpii-express allows independently mounted application units with nested routing, in the Express 4.x style – whereas Kettle is currently limited to flat Express 3.x-style routing
  • Kettle incorporates support for WebSockets endpoints, whereas gpii-express does not
  • Kettle incorporates support for DataSources (see DataSources )

The request-handling architecture for gpii-express and Kettle is quite similar and the projects will probably converge over time. gpii-express currently already depends on Kettle to get access to its HTTP testing support.

Documentation

Documentation and sample code for working with Kettle now follow:

The top-level structure of a Kettle applications can be described by a "config" file in JSON format. A Kettle "config" describes the configuration of a number of "Kettle apps" (grade kettle.app) hosted in a number of "Kettle servers" (grade kettle.server). The config JSON file represents an Infusion component tree. If you aren't familiar with the syntax and meaning of component trees, it is a good idea to browse the documentation, tutorials and examples at the Infusion documentation site. Kettle components are currently derived from the base grade fluid.component, so you can ignore for these purposes the parts of the Infusion documentation relating to model and view components.

In this section, we will construct a simple Kettle application within JavaScript code, to produce a self-contained example. You can find and try out this same application represented in two forms in the examples/simpleConfig directory.

 
fluid.defaults("examples.simpleConfig", {
    gradeNames: "fluid.component",
    components: {
        server: {
            type: "kettle.server",
            options: {
                port: 8081,
                components: {
                    app: {
                        type: "kettle.app",
                        options: {
                            requestHandlers: {
                                getHandler: {
                                    "type": "examples.simpleConfig.handler",
                                    "route": "/handlerPath",
                                    "method": "get"
                                }
                            }
                        }
                    }
                }
            }
        }
    }
});
 
fluid.defaults("examples.simpleConfig.handler", {
    gradeNames: "kettle.request.http",
    invokers: {
        handleRequest: "examples.simpleConfig.handleRequest"
    }
});
 
examples.simpleConfig.handleRequest = function (request) {
    request.events.onSuccess.fire({
        message: "GET request received on path /handlerPath"
    });
};
 
// Construct the server using the above config 
examples.simpleConfig();

The JSON "config" form of the application itself is held at examples/simpleConfig/examples.simpleConfig.json, which encodes the same information as in the first fluid.defaults call above. The definitions for request handlers such as examples.simpleConfig.handler and examples.simpleConfig.handleRequest, which in our sample are held in examples/simpleConfig/simpleConfig-config-handler.js, always need to be supplied in standard .js files required by the application – although future versions of Kettle may allow the defaults for the handler grade to be encoded in JSON. Consult the Infusion framework documentation on grades if you are not familiar with this kind of configuration.

You can try out these samples in examples/simpleConfig by, for example, from that directory, typing node simpleConfig-config-driver.js. The last line of the driver files load a common module, simpleConfig-client.js which tests the server by firing an HTTP request to it and logging the payload – this uses one of the HTTP client drivers taken from Kettle's testing definitions. Later on, we will see how to issue formal test fixtures against this application by using the Kettle testing framework.

Kettle applications can be started in a variety of ways, both programmatically and from the command line - as well as easily embedded into other applications, be they Infusion component trees or raw Express apps.

Kettle includes a driver function, kettle.config.loadConfig which will load and run a Kettle application defined as a JSON file in the filesystem. It accepts an options structure which includes the following fields:

Member name Type Description
configName String The name of the config (the bare filename, minus any extension) which is to be loaded
configPath String The directory holding the config. This path may start with a symbolic module reference, e.g. of the form %kettle, to a module which has been registered using Infusion's module API fluid.module.register

kettle.config.loadConfig will return the (Infusion) component instance of the initialised application. You could use this, for example, to terminate the application using its destroy() method.

An alternative to kettle.config.loadConfig is kettle.config.createDefaults. This accepts the same arguments but simply loads the config as a grade rather than instantiating it as well. The return value from kettle.config.loadConfig is the grade name of the application. You can construct this application later by use of Infusion's invokeGlobalFunction API, or else embed it in a wider application as a subcomponent.

Kettle includes a top-level driver file named init.js which will accept values from the command line and the environment variable NODE_ENV in order to determine which application config to start. For example, from Kettle's top-level directory you can run

    node init.js <configPath> [<configName>]

The configPath argument is required - its meaning is as given in the configPath option to kettle.config.loadConfig call described in the previous section.

The configName argument is optional. If this value is not supplied at the command line, it will be read from the environment variable NODE_ENV. The meaning is as given in the configName option to kettle.config.loadConfig described in the previous section.

For example, you can start the sample app from the previous section by running

   node init.js examples/simpleConfig examples.simpleConfig

from the root directory of a Kettle checkout.

In the previous section, we saw a simple example of a JSON-formatted Kettle config, which declaratively encodes a Kettle application structure. In this section we describe the top-level members of this structure, and in the next secion we'll look at the containment structure in terms of grades. In the rest of this document we'll describe the options accepted by the various grades which make up a Kettle application (kettle.server, kettle.app, kettle.middleware and kettle.request). The structure of our minimal application will serve as a general template – the full definition of a Kettle application consists of a config, plus definitions of request handler grades plus implementations of request handler functions.

Top-level members of a Kettle "config" configuration file
Member Type Description
type String (grade name) The type name for this config. This should be a fully-qualified grade name – it is suggested that it agree with the file name of the config file without the .json extension.
options Object (component options) The options for the application structure. These should start with the application's gradeNames which will usually just be fluid.component, and then continue with components designating the next level of containment of the application, the kettle.server level – see the next section on containment structure of a Kettle application for the full structure
mergeConfigs (optional) String/Array of String A filename (or array of these) of other config files which are to be included into this application. These names may begin with a module-relative path such as %kettle or else will be interpreted as paths relative to this config's location in the filesystem. Each filename listed here will be loaded and resolved as a grade and then merged with the structure of this config (via an algorithm similar to jQuery.extend – note that because of a current Infusion framework bug FLUID-5614, all of the semantics of nested options merging will not be respected and the merging will occur in a simple-minded way below top level)
loadConfigs (optional) String/Array of String A filename (or array of these) of other config files which will be loaded before this config is interpreted. These names may begin with a module-relative path such as %kettle or else will be interpreted as paths relative to this config's location in the filesystem. Each filename listed here will be loaded and resolved as a grade. The workflow is similar to that with mergeConfigs, only the grades represented in loadConfigs will not be automatically merged with the current config as parent grades. Instead, the user is free to refer to them as required - for example as the type or gradeNames of a subcomponent
require (optional) String/Array of String A module identifier (or array of these) that will be loaded when this config is loaded. These modules will be loaded as if by the standard node.js API require operating from the config's directory (the module-relative path such as %kettle to indicate a path relative to a module registered with Infusion.

The overall structure of a Kettle application (a "config") shows a 4-level pattern:

  • At top level, the application container – this has the simple grade fluid.component and does not carry any functionality – it is simply used for grouping the definitions at the next level
    • At 2nd level, one or more Kettle servers – these have the grade kettle.server – in the case there is just one server it is conventionally named server
      • At 3rd level, one more Kettle apps – these have the grade kettle.app – this is the level at which independently mountable segments of applications are grouped (an app is a grouping of handlers)
        • At 4th level, one or more Kettle request handlers – these have the grade kettle.request – each of these handles one endpoint (HTTP or WebSockets) routed by URL and request method

This expression is much more verbose in simple cases than the traditional raw use of express apps, but in larger and more complex applications this verbosity is amortised, with the ability to easily customise and reassort groups of handlers and servers from application to application.

Note that the containment relationships between the top 3 levels need not be direct – servers may be nested any number of levels below the config root, and apps may be nested any number of levels below a server. However, request handlers must be defined as direct children of their parent apps, in the options section named requestHandlers.

The kettle.server grade accepts the following options which can usefully be configured by the user. Naturally by virtue of being a fluid.component there are numerous other parts of its lifecycle which can be customised, but these are the options principally supported for user configuration:

Supported configurable options for a kettle.server
Option Type Description
members.expressApp express The express application which this server is to be bound to. If this option is not overriden, the server will automatically construct one using the express() constructor.
members.httpServer http.Server The node.js HTTP server which this server is to be bound to. If this option is not overriden, the server will use the one extracted from the expressApp member
port Number The port number which this server is to listen on. Defaults to 8081.
rootMiddleware middlewareSequence The group of middleware which is to be executed for every request handled by this server
components.middlewareHolder Component A plain component container for middleware that is intended to be resolvable throughout the server's component tree
events.onListen Event An event fired once this server has started listening on its port. Fired with one argument, the server component itself
events.beforeStop Event An event fired just before this server is about to be stopped. This is an opportunity to clean up any resource (e.g. close any open sockets). Fired with one argument, the server component itself
events.onStopped Event An event fired after the server is stopped and the HTTP server is no longer listening. Fired with one argument, the server component itself
events.onContributeMiddleware Event This event is useful for authors trying to integrate with 3rd-party express applications. This is a useful lifecycle point, before Kettle registers its own middleware to the express application, for an external integrator to register their own middleware first, e.g. using app.use. Fired with one argument, the server component itself – typically only that.expressApp will be of interest to the listener
events.onContributeMiddleware Event This event is useful for authors trying to integrate with 3rd-party express applications. This is a useful lifecycle point, before Kettle registers its own rout handlers to the express application, for an external integrator to register their own rout handlers first, e.g. using app.get etc.. Fired with one argument, the server component itself – typically only that.expressApp will be of interest to the listener

A WebSockets-capable server exposes all of the configurable options supported by a kettle.server in addition to the ones in the table below:

Supported configurable options for a kettle.server.ws
Option Type Description
wsServerOptions Object Any options to be forwarded to the constructor of the ws.Server. Note that after construction, this server will be available as the top-level member named wsServer on the overall component.

Request handlers are registered in the requestHandlers section of the options of a kettle.app – see the sample app for positioning of this component in the containment structure. This consists of a free hash of handlerName strings to handlerRecord structures.

{
<handlerName> : <handlerRecord>,
<handlerName> : <handlerRecord>,
...
}

Note that the handlerNames are simply free strings and have no function other than to uniquely name the handler in the context of its app. These strings exist to allow easy alignment when multiple apps are merged together from different sources to produce combined apps.

    <tr>
        <td><code>method</code> (optional)</td>
        <td><code>String</code> value – one of the valid <a href="https://github.com/nodejs/node/blob/master/deps/http_parser/http_parser.h#L88">HTTP methods</a> supported by node.js, expressed in lower case, or else a comma-separated 
        sequence of such values.
        </td>
        <td>The HTTP request type(s) which this handler will match. <code>method</code> is omitted in the 
        case that the request handling grade is not descended from <code>kettle.request.http</code> – the only currently supported requests of that type are WebSockets requests descended from <code>kettle.request.ws</code> 
    </tr>
</tbody>
Members of an handlerRecord entry within the requestHandlers block of a kettle.app component
Member Type Description
type String The name of a request handling grade, which must be descended from kettle.request. If you supply the method field, your grade must be descended from kettle.request.http
route String An express-compatible routing string, expressing the range of HTTP paths to be handled by this handler, together with any named parameters and query parameters that should be captured. The exact syntax for route matching is documented more precisely at pillarjs
prefix (optional) String A routing prefix to be prepended to this handler's route. The prefix plus the route expression must match the incoming request in order for this handler to be activated – but if it is, it will only see the portion of the URL matched by route in the member request.req.url. The entire incoming URL will remain visible in request.req.originalUrl – this is the same behaviour as express.js routing system. It is primarily useful when using static middleware which will compare the req.url value with the filesystem path relative to its mount point.

A handler for a particular request must have a grade registered with Infusion whose name matches the type field in the handlerRecord structure just described. The parent grades of this grade must be consistent with the the request you expect to handle – descended from kettle.request.http in the case of an HTTP request, or kettle.request.ws in the case of a WebSockets request. In addition, the grade must define (at minimum) an invoker named handleRequest. This invoker will be called by Kettle when your route is matched, and be supplied a single argument holding the request object, an object whose grade is your request handler's grade, which the framework has constructed to handle the request.

We duplicate the definitions from the sample application in order to show a minimal request handler grade and request handler function:

fluid.defaults("examples.simpleConfig.handler", {
    gradeNames: "kettle.request.http",
    invokers: {
        handleRequest: "examples.simpleConfig.handleRequest"
    }
});
 
examples.simpleConfig.handleRequest = function (request) {
    request.events.onSuccess.fire({
        message: "GET request received on path /handlerPath"
    });
};

In the next section we will talk more about request (handler) objects, the members you can expect on them, and how to use them.

A request component is constructed by Kettle when it has determined the correct handler record which matches the incoming request. This request component will be usefully populated with material drawn from the request and node.js initial process of handling it. It also contains various elements supplied by Kettle in order to support you in handling the request. You can add any further material that you like to the request object by adding entries to its grade definition, of any of the types supported by Infusion's component configuration options. Here we will document the standard members that are placed there by Kettle for the two standard request types which are supported, kettle.request.http and kettle.request.ws.

Members defined by default at top-level on a (HTTP) request component
Member Type Description
req http.IncomingMessage The request object produced by node.js – this is the value which is commonly referred to as req in the standard express middleware pattern
res (only for kettle.request.http) http.ServerResponse The response object produced by node.js – this is the value which is commonly referred to as res in the standard express middleware pattern
events.onSuccess (only for kettle.request.http) Event A standard Infusion Event which should be fired if the request is to produce a response successfully. The event argument will produce the response body – if it is of type Object, it will be JSON-encoded.
events.onError Event A standard Infusion Event which should be fired if the request is to send an error response. For a request of type kettle.request.http, the argument to the event must be an Object with at least a field message of type String holding the error message to be returned to the client. The argument can also include a member statusCode of type Number holding the HTTP status code to accompany the error – if this is not supplied, it will default to 500.
handlerPromise (only for kettle.request.http) Promise This promise is a proxy for the two events onSuccess and onError, packaged as a Promise. This promise exposes methods resolve and reject which forward their arguments to onSuccess and onError respectively. In addition, the promise exposes a then method which accepts two callbacks which can be used to listen to these event firings respectively. Note that this promise is not compliant with any particular specification for promises, including ES6, A+, etc. – in the language of those specifications, it is simply a thenable which also includes the standard resolution methods resolve and reject. Implementation at FluidPromises.js.
events.onSuccess (only for kettle.request.http) Event A standard Infusion Event which should be fired if the request is to produce a response successfully. The event argument will produce the response body – if it is of type Object, it will be JSON-encoded.

Note that, in return, the Kettle request component will be marked onto the node.js request object so that it can easily be retrieved from standard middleware, etc. – it will be available as req.fluidRequest where req is the request object described in the table above. More details follow on middleware in the section working with middleware.

WebSockets communications in a Kettle application are mediated by the ws WebSockets library – you should get familiar with the documentation for that library if you intend to use this functionality significantly. It is also worth spending some time familiarising yourself with at least some of the ws implementation code since there are several aspects not properly covered by the documentation.

The request component for a WebSockets request, derived from the grade kettle.request.ws includes the members in the above table which are not marked as kettle.request.http only, as well as several more members described in the following table:

Members defined by default at top-level on a WebSockets request component
Member Type Description
ws ws.WebSocket The ws.WebSocket advertised by the ws WebSockets library as allocated to handle one end of an established WebSockets connection. This will be of the variety referred to in the ws docs as "a WebSocket constructed by a Server".
events.onReceiveMessage Event A standard Infusion Event which is fired by the framework when a message is received from the client at the other end of the WebSockets connection. The arguments to the event are (that, payload) where that represents this request component itself, and payload represnts the payload sent by the client. If the receiveMessageJSON option is set to true for this component (the default), the payload will have been decoded as JSON.
events.onSendMessage Event A standard Infusion Event which can be fired by the implementor of the request handler when they want to send a message to the client. This event expects to receive just a single argument, the message payload. If the sendMessageJSON option is set to true for this component (the default), the payload will be encoded by the framework as JSON.
events.onError Event A standard Infusion Event which should be fired if the request is to end an error response. This event has the same name as the one fired by a kettle.request.http but the behaviour and semantic is different. Rather than sending an HTTP error response, the framework instead emits a WebSockets event of type error. Because of this, the statusCode field of the event argument should not be used. However, it is recommended that the event payload still includes a field message of type String holding the error message to be returned to the client, as well as a boolean member isError with the value true.

The most crucial structuring device in the expressjs (or wider pillarjs) community is known as middleware. In its most basic form, a piece of middleware is simply a function with the following signature:

middleware(req, res, next)

The elements req and res have been described in the section on request components. The element next is a callback provided by the framework to be invoked when the middleware has completed its task. This could be seen as a form of continuation passing style with 0 arguments – although only in terms of control flow since in general middleware has its effect as a result of side-effects on the request and response. In express, middleware are typically accumulated in arrays or groups of arrays by directives such as app.use. If a piece of middleware completes without error, it will invoke the next callback with no argument, which will signal that control should pass to the next middleware in the current sequence, or back to the framework if the sequence is at an end. Providing an argument to the callback next is intended to signal an error and the framework will then abort the middleware chain and propagate the argument, conventionally named err, to an error handler. This creates an analogy with executing promise sequences which we will return to when we construct middleware components.

In Kettle, middleware can be scheduled more flexibly than by simply being accumulated in arrays – the priority of a piece of middleware can be freely adjusted by assigning it a Priority as seen in many places in the Infusion framework, and so integrators can easily arrange for middleware to be inserted in arbitrary positions in already existing applications.

Middleware is accumulated at two levels in a Kettle application – firstly, overall middleware is accumulated at the top level of a kettle.server in an option named rootMiddleware. This is analogous to express app-level middleware registered with app.use. Secondly, individual request middleware can be attached to an individual kettle.request in its options at requestMiddleware. This is analogous to express router-level middleware. The structure of these two options areas is the same, which we name middlewareSequence. When the request begins to be handled, the framework will execute the following in sequence:

  • The root middleware attached to the kettle.server
  • The request middleware attached to the resolved kettle.request component
  • The actual request handler designated by the request component's invoker handleRequest

If any of the middleware in this sequence signals an error, the entire sequence will be aborted and an error returned to the client.

A middlewareSequence is a free hash of keys, considered as namespaces for the purpose of resolving Priorities onto records of type middlewareEntry:

{
    <middlewareKey> : <middlewareEntry>,
    <middlewareKey> : <middlewareEntry>,
...
}
Members of an middlewareEntry entry within the middlewareSequence block of a component (rootMiddleware for kettle.server or requestMiddleware for kettle.request)
Member Type Description
middleware String (IoC Reference) An IoC reference to the middleware component which should be inserted into the handler sequence. Often this will be qualified by the context {middlewareHolder} – e.g. {middlewareHolder}.session – to reference the core middleware collection attached to the kettle.server but middleware could be resolved from anywhere visible in the component tree. This should be a reference to a component descended from the grade kettle.middleware
priority (optional) String (Priority) An encoding of a priority relative to some other piece of middleware within the same group – will typically be before:middlewareKey or after:middlewareKey for the middlewareKey of some other entry in the group

A piece of Kettle middleware is derived from grade kettle.middleware. This is a very simple grade which defines a single invoker named handle which accepts one argument, a kettle.request, and returns a promise representing the completion of the middleware. Conveniently a fluid.promise implementation is available in the framework, but you can return any variety of thenable that you please. Here is a skeleton, manually implemented middleware component:

fluid.defaults("examples.customMiddleware", {
    gradeNames: "kettle.middleware",
    invokers: {
        handle: "examples.customMiddleware.handle"
    }
});
 
examples.customMiddleware.handle = function (request) {
    var togo = fluid.promise();
    if (request.req.params.id === 42) {
        togo.resolve();
    } else {
        togo.reject({
            isError: true,
            statusCode: 401,
            message: "Only the id 42 is authorised"
        });
    }
    return togo;
};

The framework makes it very easy to adapt any standard express middleware into a middleware component by means of the adaptor grade kettle.plainMiddlware. This accepts any standard express middleware as the option named middleware and from it fabricates a handle method with the semantic we just saw earlier. Any options that the middleware accepts can be forwarded to it from the component's options. Here is an example from the framework's own json middleware grade:

kettle.npm.bodyParser = require("body-parser");
 
fluid.defaults("kettle.middleware.json", {
    gradeNames: ["kettle.plainMiddleware"],
    middlewareOptions: {}, // see https://github.com/expressjs/body-parser#bodyparserjsonoptions 
    middleware: "@expand:kettle.npm.bodyParser.json({that}.options.middlewareOptions)"
});

Consult the Infusion documentation on the compact format for expanders if you are unfamiliar with this syntax for designating elements in component options which arise from function calls.

Here we describe the built-in middleware supplied with Kettle, which is mostly sourced from standard middleware in the express and pillarjs communities. You can consult the straightforward implementations in KettleMiddleware.js for suggestions for how to implement your own.

Grade nameMiddleware nameDescriptionAccepted optionsStandard IoC Path
kettle.middleware.json expressjs/body-parser Parses JSON request bodies, possibly with compression. middlewareOptions, forwarded to bodyParser.json(options) {middlewareHolder}.json
kettle.middleware.urlencoded expressjs/body-parser Applies URL decoding to a submitted request body middlewareOptions, forwarded to bodyParser.urlencoded(options) {middlewareHolder}.urlencoded
kettle.middleware.cookieParser expressjs/cookie-parser Parses the Cookie header as well as signed cookies via req.secret. secret and middlewareOptions, forwarded to the two arguments of cookieParser(secret, options) none
kettle.middleware.session expessjs/session Stores and retrieves req.session from various backends middlewareOptions, forwarded to session(options) {middlewareHolder}.session when using kettle.server.sessionAware server
kettle.middleware.static expressjs/serve-static Serves static content from the filesystem root and middlewareOptions, forwarded to the two arguments of serveStatic(root, options) none – user must configure on each use
kettle.middleware.CORS Kettle built-in Adds CORS headers to outgoing HTTP request to enable cross-domain access allowMethods {String} (default "GET"),
origin {String} (default *`),
credentials {Boolean} (default true)
- see CORS response headers
{middlewareHolder}.CORS
kettle.middleware.null Kettle built-in No-op middleware, useful for overriding and inactivating undesired middleware none {middlewareHolder}.null

Middleware which it makes sense to share configuration application-wide is stored in a standard holder of grade kettle.standardMiddleware which is descended from the grade kettle.middlewareHolder – the context reference {middlewareHolder} is recommended for referring to this if required – e.g. {middlewareHolder}.session.

Here is an example of mounting a section of a module's filesystem path at a particular URL. In this case, we want to mount the src directory of our Infusion module at the global path /infusion/, a common enough requirement. Note that this is done by registering a handler just as with any other Kettle request handler, even though in this case the useful request handling function is actually achieved by the middleware. The only function of the request handler is to serve the 404 message in case the referenced file is not found in the mounted image – in this case, it can refer to the standard builtin handler named kettle.request.notFoundHandler. Note that the request handler must declare explicitly that it will handle all URLs under its prefix path by declaring a route of /* – this is different to the express model of routing and middleware handling. Kettle will not dispatch a request to a handler unless its route matches all of the incoming URL.

Note that our static middleware can refer symbolically to the path of any module loaded using Infusion's module system fluid.module.register by means of interpolated terms such as %infusion.

Our config:

{
    "type": "examples.static.config",
    "options": {
        "gradeNames": ["fluid.component"],
        "components": {
            "server": {
                "type": "kettle.server",
                "options": {
                    "port": 8081,
                    "components": {
                        "infusionStatic": {
                            "type": "kettle.middleware.static",
                            "options": {
                                "root": "%infusion/src/"
                            }
                        },
                        "app": {
                            "type": "kettle.app",
                            "options": {
                                "requestHandlers": {
                                    "staticHandler": {
                                        "type": "examples.static.handler",
                                        "prefix": "/infusion",
                                        "route": "/*",
                                        "method": "get"
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

Our handler:

fluid.defaults("examples.static.handler", {
    gradeNames: "kettle.request.http",
    requestMiddleware: {
        "static": {
            middleware: "{server}.infusionStatic"
        }
    },
    invokers: {
        handleRequest: {
            funcName: "kettle.request.notFoundHandler"
        }
    }
});
 

A DataSource is an Infusion component which meets a simple contract for read/write access to indexed data. DataSource is a simple semantic, broadly the same as that encoded in CRUD, although the current DataSource semantic does not provide explicitly for deletion.

The concrete DataSources in Kettle provide support for HTTP endpoints (with a particular variety specialised for accessing CouchDB databases with CRUDlike semantics) as well as the filesystem, with an emphasis on JSON payloads.

The DataSource API is drawn from the following two methods – a read-only DataSource will just implement get, and a writeable DataSource will implement both get and set:

/* @param directModel {Object} A JSON structure holding the "coordinates" of the state to be read -  
 * this model is morally equivalent to (the substitutable parts of) a file path or URL
 * @param options {Object} [Optional] A JSON structure holding configuration options good for just 
 * this request. These will be specially interpreted by the particular concrete grade of DataSource 
 * – there are no options valid across all implementations of this grade.
 * @return {Promise} A promise representing successful or unsuccessful resolution of the read state
 */
dataSource.get(directModel, options);
/* @param directModel {Object} As for get
 * @param model {Object} The state to be written to the coordinates
 * @param options {Object} [Optional] A JSON structure holding configuration options good for just 
 * this request. These will be specially interpreted by the 
 * particular concrete grade of DataSource – there are no options valid across all implementations 
 * of this grade. For example, a URL DataSource will accept an option `writeMethod` which will 
 * allow the user to determine which HTTP method (PUT or POST) will be used to implement the write
 * operation.
 * @return {Promise} A promise representing resolution of the written state,
 * which may also optionally resolve to any returned payload from the write process
 */
dataSource.set(directModel, model, options);

In this example we define and instantiate a simple HTTP-backed dataSource accepting one argument to configure a URL segment:

var fluid = require("infusion"),
    kettle = require("../../kettle.js"),
    examples = fluid.registerNamespace("examples");
 
 
fluid.defaults("examples.httpDataSource", {
    gradeNames: "kettle.dataSource.URL",
    url: "http://jsonplaceholder.typicode.com/posts/%postId",
    termMap: {
        postId: "%directPostId"
    }
});
 
var myDataSource = examples.httpDataSource();
var promise = myDataSource.get({directPostId: 42});
 
promise.then(function (response) {
    console.log("Got dataSource response of ", response);
}, function (error) {
    console.error("Got dataSource error response of ", error);
});

You can run this snippet from our code samples by running node simpleDataSource.js from examples/simpleDataSource in our samples area. This contacts the useful JSON placeholder API service at jsonplaceholder.typicode.com to retrieve a small JSON document holding some placeholder text. If you get a 404 or an error, please contact us and we'll update this sample to contact a new service.

An interesting element in this snippet is the termMap configured as options of our dataSource. This sets up an indirection between the directModel supplied as the argument to the dataSource.get call, and the URL issued in the HTTP request. The keys in the termMap are interpolation variables in the URL, which in the URL are prefixed by %. The values in the termMap represent either

  • Plain values to be interpolated as strings directly into the URL, or
  • If the first character of the value in the termMap is %, the remainder of the string represents a path which will be dereferenced from the directModel argument to the current set or get request.

In addition, if the term value has the prefix noencode:, it will be interpolated without any URI encoding.

We document these configuration options in the next section:

Supported configurable options for a kettle.dataSource.URL
Option Path Type Description
writable Boolean (default: false) If this option is set to true, a set method will be fabricated for this dataSource – otherwise, it will implement only a get method.
writeMethod String (default: PUT) The HTTP method to be used when the set method is operated on this writable DataSource (with writable: true). This defaults to PUT but POST is another option. Note that this option can also be supplied within the options argument to the set method itself.
url String A URL template, with interpolable elements expressed by terms beginning with the % character, for the URL which will be operated by the get and set methods of this dataSource.
termMap Object (map of String to String) A map, of which the keys are some of the interpolation terms held in the url string, and the values will be used to perform the interpolation. If a value begins with %, the remainder of the string represents a path into the directModel argument accepted by the get and set methods of the DataSource. By default any such values looked up will be URI Encoded before being interpolated into the URL – unless their value in the termMap is prefixed by the string noencode:.
notFoundIsEmpty Boolean (default: false) If this option is set to true, a fetch of a nonexistent resource (that is, a nonexistent file, or an HTTP resource giving a 404) will result in a resolve with an empty payload rather than a reject response.
components.encoding.type String (grade name) A kettle.dataSource.URL has a subcomponent named encoding which the user can override in order to choose the encoding used to read and write the model object to and from the textual form in persistence. This defaults to kettle.dataSource.encoding.JSON. Other builtin encodings are kettle.dataSource.encoding.formenc operating HTML form encoding and kettle.dataSource.encoding.none which applies no encoding. More details in Using Content Encodings with a DataSource.
setResponseTransforms Array of String (default: ["encoding"]) Contains a list of the namespaces of the transform elements (see section transforming promise chains that are to be applied if there is a response payload from the set method, which is often the case with an HTTP backend. With a JSON encoding these encoding typically happens symmetrically - with a JSON request one will receive a JSON response - however, with other encoding such as form encoding this is often not the case and one might like to defeat the effect of trying to decode the HTTP response as a form. In this case, for example, one can override setResponseTransforms with the empty array [].
charEncoding String (default: utf8 The character encoding of the incoming HTTP stream used to convert its data to characters - this will be sent directly to the setEncoding method of the response stream

In addition, a kettle.dataSource.URL component will accept any options accepted by node's native http.request constructor – supported in addition to the above are protocol, host, port, headers, hostname, family, localAddress, socketPath, auth and agent. All