Nuclear Powered Mushroom

hull-connector

1.0.44 • Public • Published

Hull Connector Generator

Think of this connector generator is a kind of create-react-app for Hull connectors. It helps you go from 0 to fully setup environment in about 30 seconds.

It generates a fully working connector setup that you can immediately boot and use to receive Hull data streams and process them. It has sane defaults, with

  • prettier
  • eslint
  • babel
  • webpack
  • flow
  • mocha / sinon / chai / nock (for server testing)
  • jest (for client testing)

Plus a bunch of hull-specific tooling to make it easy to simulate traffic:

At the same time, it doesn't force you into any specific setup. It's just an express app at the core.

If you need to build a Dashboard UI, React hot-reload and Webpack are pre-configured, you just have to start writing your code.

Installing

We use builder-Init to generate the boilerplate app. Installing and generating a connector takes 2 lines:

~/test ❯❯❯ yarn global add builder builder-init
~/test ❯❯❯ builder-init hull-connector;

preview


hull-connector-1.0.44.tgz
[builder-init] Preparing templates for: hull-connector
? Human-readable Name (e.g., 'Intercom') Test
? Package / GitHub project name hull-test
? GitHub organization name hull-ships
? Package description Synchronize Data with Test
? License organization (e.g., you or your company) hull-ships
? Destination directory to write hull-test

[builder-init] Wrote files:
 - hull-test/.babelrc
 - hull-test/.builderrc
 - hull-test/.editorconfig
 - hull-test/.eslintignore
 - hull-test/.flowconfig
 - hull-test/jest.config.json
 - hull-test/manifest.json
 - hull-test/newrelic.js
 - [....]

[builder-init] New hull-connector project is ready at: hull-test

Usage

~/test ❯❯❯ cd hull-test
~/test ❯❯❯ yarn
~/test ❯❯❯ yarn dev   //Start development server
~/test ❯❯❯ yarn build //Build project
~/test ❯❯❯ yarn start //Start production server

When your connector is ready (as soon as you have ran yarn in it's folder), you can start using it as follows:

command description Notes
yarn dev Start in development mode
yarn ngrok Start a ngrok server be sure to have your account setup to choose the subdomain
yarn build Build client and server assets
yarn start Start in production mode
yarn build:client Build Client-files from src to dist Uses Webpack & Babel
yarn build:server Build Server assets from server to lib Uses Babel
yarn clean Remove build files
yarn prettier Prettifies source files
yarn flow Checks Flow annotations
yarn lint Lint and surface errors
yarn test:electrode Starts Hull's Repl, Electrode
yarn test Run Server & Client tests
yarn test:client Jest client tests
yarn test:units Unit Tests
yarn test:units:watch Starts a mocha --watch server so you can quickly work on unit tests
yarn test:specs Runs integration tests Use minihull
yarn test:specs:watch Starts a mocha --watch server so you can quickly work on unit tests

Electrode

Simulating real traffic on connectors.

Electrode is an easy-to-use REPL you can use to send any kind of traffic to connectors, while being in a real hull environment, and see what the connector sends to Hull in response.

  • It takes a real connector ID & Secret to boot
  • It fetches the actual settings for this ID & Secret and uses them on the connector.
  • It accepts User or Account notification payloads,
  • It displays what the connetor sends back to Hull

Getting an example payload is very easy: just install a Processor connector, search for the User you want and copy the payload on the left column.

~/hull-test ❯❯❯ yarn test:electrode --port 8083
# yarn run v1.3.2 
# $ node ./tests/electrode 
? SHIP_ID my-ship-id
? SHIP_SECRET my-ship-secret
? SHIP_ORG my-org.hullapp.io
 
  [...]
  "schedules": [
    {
      "url""/status",
      "type""interval",
      "value""5",
      "next_run_time": 1517693700,
      "last_run_time": null
    }
  ]
  },
  "resources": {}
}
 
╔═════════════════════════════════════════════════════════════╗
║                                                             ║
║                                                             ║
║   Connector Started on port 8083 with instance my-ship-id   ║
║   Organization: my-org.hullapp.io                           ║
║                                                             ║
║                                                             ║
╚═════════════════════════════════════════════════════════════╝
 
{"port":8083,"level":"info","message":"connector.server.listen"}
 
hull ❯ _
hull ❯ .load file.json user # loads `file.json` as `user` 
hull ❯ send(user) # sends user as a User Notification 
 
# What's available out of the box :  
# colorize -> colorize console dumps with console.log(colorize(object)); 
# segments -> segments on currently booted Hull organization 
# minihull -> hull server 
# hull -> hull client 
# connector -> connector object 
# organization -> org name 
# moment -> moment.js 
# lo -> lodash (renamed to avoid conflicts) 
# urijs 
#  
# Of course you can require your own modules: 
hull ❯ const boxen = require('boxen');

Writing tests

mockr

Integration-testing for logs and connector responses

Mockr is a testing addition to make it easy to simulate calls and settings and write assertions on the connector's responses to Hull

Testing environment

Boilerplate comes with mocha/chai/sinon/nock already setup for server tests.

You can then simply require hull-connector-dev/lib/mockr package and start writing assertions.

It sets up some mocks and minihull, which is a stripped down version of hull that's able to send messages to connectors and offer expectations on what the connector should send to the Firehose.

Here's how:
import { expect } from "chai";
import mockr from "hull-connector-dev/lib/mockr";
 
// Your server's entry point, with the same format as the one `hull-connector` bundles.
// Options will be passed to it.
import server from "../../server/server";
 
describe("Test Group", () => {
  // Start the mocks. they will run `beforeEach` and `afterEach` cleanups for you,
  // Start a development server
  const mocks = mockr({
    server
    beforeEach,
    afterEach,
    port: 8000,
    segments: [{ id: "1", name: "A" }], // Segments that should exist on the server
  });
 
  it("should behave properly", done => {
    const myNock = mocks
      .nock("https://api.myremote.test.com")
      .get("/test")
      .query({ foo: "bar" })
      .reply(200, [{ email: "foo@foo.bar", id: "foobar" }]);
 
    // Optional, if you want to stub more things that your connector
    // will access during it's flow.
    mocks.minihull.stubApp("/api/v1/search/user_reports").respond({
      pagination: { total: 0 },
      aggregations: {
        without_email: { doc_count: 0 },
        by_source: { buckets: [] },
      },
    });
 
    // Send a `user:update` call to the connector.
    mocks.minihull.userUpdate(
      {
        // Connector Settings
        connector: {
          id: "123456789012345678901234",
          private_settings: {
            api_key: "123",
            handle_accounts: true,
            prospect_enabled: true,
            prospect_segments: ["1"],
            prospect_filter_titles: ["foo"],
            prospect_limit_count: 2,
          },
        },
        // Message payload
        messages: [
          {
            user: { id: "abc", "traits/clearbit/source": "reveal" },
            account: { id: "ACCOUNTID", domain: "domain.com" },
            segments: [{ id: "1" }],
          },
        ],
      },
      // This is what the Firehose receives.
      ({ batch, logs }) => {
        const [first, second, third, fourth] = batch;
        expect(batch.length).to.equal(4);
        expect(logs[1].message).to.equal("outgoing.user.start");
        myNock.done();
        done();
      }
    );
  });
});

If you use Atom, i strongly recommend you install mocha-test-runner which will let you run test by hitting ctrl-alt-m


Developing the Archetype

yarn gen-dev;
npm link ; cd dev ; npm link

Publishing the archetype

# Bump version in package.json
npm run publish:both

Setup

  • Will monitor, bundle and build all .js files in the src folder for Client-side code
  • Will build the server folder as lib for Server-side code

install

npm i hull-connector

Downloadsweekly downloads

49

version

1.0.44

license

MIT

homepage

hull.io

last publish

collaborators

  • avatar
Report a vulnerability