node package manager

barnacles

barnacles

A real-time location & sensor data aggregator for the IoT

barnacles consume real-time spatio-temporal information about wireless devices and emit notification events based on changes such as appearances, displacements and disappearances. barnacles receive this information stream from barnowl and other barnacles instances, and maintain the current state of all detected devices. barnacles ensure that contextual information propagates efficiently from a local to a global scale in the Internet of Things.

barnacles can notify other barnacles and any third-party service that has a REST API. Currently supported platforms include Google Analytics and several proprietary IoT and analytics services. And it's easy to connect your service too.

In the scheme of Things (pun intended)

The barnowl, barnacles, barterer and chickadee packages all work together as a unit, conveniently bundled as hlc-server. Check out our developer page for more resources on reelyActive software and hardware.

barnacles logo

What's in a name?

As Wikipedia so eloquently states, "To facilitate genetic transfer between isolated individuals, barnacles have extraordinarily long penises." And given the current state of isolation of nodes in today's IoT, this package (pun intended) needs "the largest penis to body size ratio of the animal kingdom".

Also, we hope the name provides occasions to overhear our Québecois colleagues say things like "Tu veux tu configurer ta barnacle!?!"

Installation

npm install barnacles

barnacles are tightly coupled with barnowl, our IoT middleware package. The latter provides a source of data to the former, as the following example will show.

Hello barnacles & barnowl

var barnacles = require('barnacles');
var barnowl = require('barnowl');
 
var notifications = new barnacles();
var middleware = new barnowl();
 
middleware.bind( { protocol: 'test', path: 'default' } ); // See barnowl 
 
notifications.bind({ barnowl: middleware });
 
notifications.on('appearance', function(event) {
  console.log(event.deviceId + ' has appeared on ' + event.receiverId);
});
 
notifications.on('displacement', function(event) {
  console.log(event.deviceId + ' has displaced to ' + event.receiverId);
});
 
notifications.on('disappearance', function(event) {
  console.log(event.deviceId + ' has disappeared from ' + event.receiverId);
});
 
notifications.on('keep-alive', function(event) {
  console.log(event.deviceId + ' remains at ' + event.receiverId);
});

When the above code is run, you should see output to the console similar to the following:

001bc50940100000 has appeared on 001bc50940800000
fee150bada55 has appeared on 001bc50940810000
001bc50940100000 has displaced to 001bc50940800001
fee150bada55 has displaced to 001bc50940810001
...

Events

events animation

The events in the example above have the following structure:

Basic

Contains only the minimum required fields.

{
  "event": "appearance",
  "time": 1420075425678,
  "deviceId": "fee150bada55",
  "receiverId": "001bc50940810000",
  "rssi": 150,
  "tiraid": { /* Included for legacy purposes only */ }
}

Contextual

Same as above, but adds metadata associated with both the device and the receiver. Requires a connection to a chickadee instance (see Where to bind?).

{
  "event": "appearance",
  "time": 1420075425678,
  "deviceId": "fee150bada55",
  "deviceAssociationIds": [],
  "deviceUrl": "https://myjson.info/stories/test",
  "deviceTags": [ 'test' ],
  "receiverId": "001bc50940810000",
  "receiverUrl": "https://sniffypedia.org/Product/reelyActive_RA-R436/",
  "receiverTags": [ 'test' ],
  "receiverDirectory": "test",
  "rssi": 150,
  "tiraid": { /* Included for legacy purposes only */ }
}

RESTful interactions

Include Content-Type: application/json in the header of all interactions in which JSON is sent to barnacles.

GET /statistics

Retrieve the latest real-time statistics. The response will be as follows:

{
  "_meta": {
    "message": "ok",
    "statusCode": 200
  },
  "_links": {
    "self": {
      "href": "http://localhost:3005/statistics"
    }
  },
  "statistics": {
    "devices": 2,
    "tiraids": 2,
    "appearances": 0,
    "displacements": 0,
    "disappearances": 0
  }
}

where devices is the number of devices in the current state and all other values are the average number of events per second in the last statistics period.

POST /events

Create an event. Each event includes a tiraid and an event type, the latter being one of the following:

  • appearance
  • displacement
  • disappearance (ignored)
  • keep-alive

For instance, if the event is an appearance of transmitting device id 2c0ffeeb4bed on receiving device id 001bc50940810000, the JSON would be as follows:

{ 
  "event": "appearance", 
  "tiraid": {
    "identifier": {
      "type": "ADVA-48",
      "value": "2c0ffeeb4bed",
      "advHeader": {
        "type": "SCAN_REQ",
        "length": 12,
        "txAdd": "public",
        "rxAdd": "public"
      },
      "advData": {}
    },
    "timestamp": "2015-01-01T01:23:45.678Z",
    "radioDecodings": [
      {
        "rssi": 169,
        "identifier": {
          "type": "EUI-64",
          "value": "001bc50940810000"
        }
      }
    ]
  }
}

A successful response would be as follows:

{
  "_meta": {
    "message": "ok",
    "statusCode": 200
  },
  "_links": {
    "self": {
      "href": "http://localhost:3005/events"
    }
  }
}

Querying the current state

It is possible to query the current state of barnacles. There are the following four query options:

  • "transmittedBy" returns the transmissions by the devices with the given ids
  • "receivedBy" returns every transmission received by the devices with the given ids
  • "receivedStrongestBy" returns every transmission received strongest by the devices with the given ids
  • "receivedBySame" returns every transmission received by the same devices which decoded the given ids

For example, based on the Hello barnacles & barnowl example above, the following would query the most recent transmission by device 001bc50940100000:

var options = { query: "transmittedBy",
                ids: ["001bc50940100000"] };
notifications.getState(options, function(state) { console.log(state) } );

The results of the above query might resemble the following:

{
  "devices": {
    "001bc50940100000": {
      "identifier": {
        "type": "EUI-64",
        "value": "001bc50940100000",
        "flags": {
          "transmissionCount": 0
        }
      },
      "timestamp": "2014-01-01T12:34:56.789Z",
      "radioDecodings": [
        {
          "rssi": 135,
          "identifier": {
            "type": "EUI-64",
            "value": "001bc50940800000"
          }
        }
      ]
    }
  }
}

It is possible to include an omit option if either the timestamp, radioDecodings and/or identifier of the tiraid are not required. For example to query which device transmissions are received by devices 001bc50940800000 and 001bc50940810000, omitting their timestamp and radioDecodings:

var options = { query: "receivedBy",
                ids: ["001bc50940800000", "001bc50940810000"],
                omit: ["timestamp", "radioDecodings"] };
notifications.getState(options, function(state) { console.log(state) } );

The results of the above query might resemble the following:

{
  "devices": {
    "001bc50940100000": {
      "identifier": {
        "type": "EUI-64",
        "value": "001bc50940100000",
        "flags": {
          "transmissionCount": 0
        }
      }
    },
    "fee150bada55": {
      "identifier": {
        "type": "ADVA-48",
        "value": "fee150bada55",
        "advHeader": {
          "type": "ADV_NONCONNECT_IND",
          "length": 22,
          "txAdd": "random",
          "rxAdd": "public"
        },
        "advData": {
          "flags": [
            "LE Limited Discoverable Mode",
            "BR/EDR Not Supported"
          ],
          "completeLocalName": "reelyActive"
        }
      }
    }
  }
}

Querying real-time statistics

It is possible to query the latest real-time statistics as follows:

notifications.getStatistics();

This query will return the following:

{ devices: 0,
  tiraids: 0,
  appearances: 0,
  displacements: 0,
  disappearances: 0 }

where devices is the number of devices in the current state and all other values are the average number of events per second in the last statistics period.

Connecting with services

It is possible to connect different services such that they receive the notifications via their API. The following services are supported:

Barnacles (via REST)

barnacles can POST notifications to another barnacles instance (or any other server) via REST. This way the remote barnacles instance is aware of the local state. For instance to POST notifications to a barnacles instance hosted at www.remotebarnacles.com:

notifications.addService( { service: "barnaclesrest",
                            hostname: "www.remotebarnacles.com",
                            port: 80,
                            ignoreInfrastructureTx: false,
                            whitelist: [ "001bc50940800000", "001bc50940810000" ] } );

In the case above, only notifications relative to the two whitelisted devices will be POSTed. To POST notifications for all devices, omit the whitelist property. The default path is '/events'.

Barnacles (via MQTT)

barnacles can publish notifications to another barnacles instance (or any other broker) via MQTT. This way the remote barnacles instance is aware of the local state. For instance to publish notifications to a barnacles instance hosted at www.remotebarnacles.com:

notifications.addService( { service: "barnaclesmqtt",
                            hostname: "www.remotebarnacles.com",
                            topic: "events"
                            clientOptions: { keepalive: 3600 },
                            ignoreInfrastructureTx: false,
                            whitelist: [ "001bc50940800000", "001bc50940810000" ] } );

In the case above, only notifications relative to the two whitelisted devices will be published. To publish notifications for all devices, omit the whitelist property. The default topic is 'events'.

This service requires the mqtt package which is not listed as a dependency. To use this service, you must manually install the package:

npm install mqtt

Websockets

barnacles can send notifications via websockets. For instance to set up websockets on a namespace called test:

notifications.addService( { service: "websocket",
                            namespace: "/test",
                            ignoreInfrastructureTx: false,
                            whitelist: [ "001bc50940800000", "001bc50940810000" ] } );

If the barnacles instance were to be running on localhost port 3005, the notification stream could be consumed at localhost:3005/test.

Logfile

barnacles can write events as comma-separated values (CSV) to a local logfile. For instance to write to a file called eventlog:

notifications.addService( { service: "logfile",
                            logfileName: "eventlog",
                            ignoreInfrastructureTx: false,
                            whitelist: [ "001bc50940800000", "001bc50940810000" ] } );

The output file name will be, for example, eventlog-160101012345.csv, where the numeric portion is the date and timestamp when the file is created.

Google Universal Analytics

barnacles can send notifications to Google's Universal Analytics platform such that a wireless device being detected by a sensor is analagous to a user hitting a webpage. In other words, imagine a physical location as a website, and the "invisible buttons" are webpages. A wireless device moving through that space triggering invisible buttons is equivalent to a user browsing a website. And it's all possible in one line of code:

notifications.addService( { service: "google",
                            hostname: "http://hlc-server.url",
                            accountId: "UA-XXXXXXXX-X",
                            ignoreInfrastructureTx: false,
                            whitelist: [ "001bc50940800000", "001bc50940810000" ] } );

The optional hostname can be used to specify the URL of an hlc-server instance. This could be useful if you want to collect both physical and online "hits" of the same resource. The accountId is provided by Google when you set up Google Analytics Reporting. The optional whitelist limits the notifications to those with tiraids containing one or more of the given receiver ids.

The pageview path is recorded as /id/receiverID where the receiverID would for instance be 001bc50940800001. Each wireless device is given a UUID and CID based on its identifier which allows tracking so long as the identifier does not change.

This service requires the universal-analytics package which is not listed as a dependency. To use this service, you must manually install the package:

npm install universal-analytics

Initial State

barnacles can send notifications to the Initial State platform. This allows for infrastructure statistics to be logged and visualised. For instance to stream reelceiver send counts to an Initial State bucket:

notifications.addService( { service: "initialstate",
                            bucketType: "sendCount",
                            bucketKey: "Bucket Key",
                            accessKey: "Your-Access-Key-Here",
                            whitelist: [ "001bc50940800000", "001bc50940810000" ] } );

The data key is always the reelceiverId and the value depends on the bucketType. Currently, the following bucketTypes are supported:

  • uptimeSeconds
  • sendCount
  • crcPass
  • crcFail
  • maxRSSI
  • avgRSSI
  • minRSSI
  • temperatureCelcius
  • radioVoltage

This service requires the initial-state package which is not listed as a dependency. To use this service, you must manually install the package:

npm install initial-state

mnubo

barnacles can send notifications to the mnubo platform. For instance to stream real-time events to mnubo:

notifications.addService( { service: "mnubo",
                            clientId: "Your-ID-Here",
                            clientSecret: "Your-Secret-Here",
                            clientEnv: "sandbox",
                            ignoreInfrastructureTx: false,
                            whitelist: [ "001bc50940800000", "001bc50940810000" ] } );

This service requires the mnubo-sdk package which is not listed as a dependency. To use this service, you must manually install the package:

npm install mnubo-sdk

Connecting your service

Prefer instead to connect your own service to barnacles so that it receives a real-time stream of spatio-temporal events? There's an easy way and there's an even easier way. We'll start with the latter.

Use the Barnacles REST service

The barnacles API is incredibly simple to copy, and we suggest that you set up an endpoint on your service that can ingest events from a POST request. See POST /events for the structure of the data you'll receive. And then set up your barnacles instance to post to that service by configuring the hostname, port and path as explained here, for example:

notifications.addService( { service: "barnaclesrest",
                            hostname: "www.myservice.com",
                            path: "/mypath" } );

Create your own service within barnacles

If the barnacles API is unsuitable for your service, or if your service already has an npm package, it might be preferable to write your own service to add to the barnacles code base. Inspire yourself from the existing services in the lib/services folder, and then get in touch and/or make a pull request on the develop branch.

Where to bind?

barnowl

barnowl provides a real-time stream of events. barnacles can bind to multiple instances of barnowl.

notifications.bind( { barnowl: middleware } );

chickadee

chickadee provides a contextual associations store. When bound, barnacles will append any contextual information from chickadee to the events it propagates. barnacles can bind to a single instance of chickadee only. Compatible with chickadee@0.3.24 and above.

notifications.bind( { chickadee: associations } );

websocket

barnacles can listen for events emitted by a websocket client. For example to bind your barnacles instance to your real-time Pareto data feed, first create and connect the websocket client, then bind it to barnacles.

var io = require('socket.io-client');
var PARETO_TOKEN = 'xxx';  // Listed in your Pareto account 
 
process.env.PARETO_TOKEN = PARETO_TOKEN;
const mysocket = io('https://pareto.reelyactive.com',
                    { query: { token: PARETO_TOKEN } });
 
notifications.bind( { websocket: mysocket } );

The above example requires the socket.io-client package which is not listed as a dependency. To use this service, you must manually install the package:

npm install socket.io-client

Options

The following options are supported when instantiating barnacles (those shown are the defaults):

{
  httpPort: 3005,
  useCors: false,
  delayMilliseconds: 1000,
  minDelayMilliseconds: 100,
  historyMilliseconds: 5000,
  disappearanceMilliseconds: 10000,
  keepAliveMilliseconds: 5000,
  enableMasterSocketForwarding: false,
  acceptStaleEvents: false,
  acceptFutureEvents: true
}

Notes:

  • delayMilliseconds specifies how long to wait for data to arrive from all possible sources before determining if an event occurred - note that this introduces the given amount of latency
  • minDelayMilliseconds specifies the minimum time between successive batches of event calculations - this can be tweaked to reduce CPU load
  • historyMilliseconds specifies how long to consider historic spatio-temporal data before it is flushed from memory - to avoid the possibility of data being ignored, ensure that historyMilliseconds = keepAliveMilliseconds
  • disappearanceMilliseconds specifies how long to wait after the most recent decoding before considering the transmitting device as disappeared and removing the record from memory
  • keepAliveMilliseconds specifies the maximum time between subsequent events for each transmitting device - if no displacement events occur, barnacles will emit a keep-alive notification every given period
  • enableMasterSocketForwarding specifies whether all events are forwarded on the master websocket
  • acceptStaleEvents specifies whether received events with a timestamp further in the past than the disappearanceMilliseconds are discarded (false) or accepted and updated to the current time (true)
  • acceptStaleEvents specifies whether received events with a timestamp in the future are discarded (false) or accepted and updated to the current time (true)

What's next?

This is an active work in progress. Expect regular changes and updates, as well as improved documentation! If you're developing with barnacles check out:

License

MIT License

Copyright (c) 2014-2017 reelyActive

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.