qtopology
    TypeScript icon, indicating that this package has built-in type declarations

    2.3.2 • Public • Published

    QTopology

    npm

    QTopology is a distributed stream processing layer, written in node.js.

    NPM package: https://www.npmjs.com/package/qtopology

    Documentation https://qminer.github.io/qtopology

    Installation

    npm install qtopology

    Intro

    QTopology is a distributed stream processing layer, written in node.js.

    It uses the following terminology, originating in Storm project:

    • Topology - Organization of nodes into a graph that determines paths where messages must travel.
    • Bolt - Node in topology that receives input data from other nodes and emits new data into the topology.
    • Spout - Node in topology that reads data from external sources and emits the data into the topology.
    • Stream - When data flows through the topology, it is optionaly tagged with stream ID. This can be used for routing.

    When running in distributed mode, qtopology also uses the following:

    • Coordination storage - must be resilient, receives worker registrations and sends them the initialization data. Also sends shutdown signals. Implementation is custom. QTopology provides REST-based service out-of-the-box, but the design is similar for other options like MySQL storage etc.
    • Worker - Runs on single server. Registers with coordination storage, receives initialization data and instantiates local topologies in separate subprocesses.
      • Leader - one of the active workers is announced leader and it performs leadership tasks such as assigning of topologies to workers, detection of dead or inactive workers.

    Quick start

    See documentation

    Keywords

    Install

    npm i qtopology

    DownloadsWeekly Downloads

    6

    Version

    2.3.2

    License

    BSD-2-Clause

    Unpacked Size

    3.68 MB

    Total Files

    253

    Last publish

    Collaborators

    • rupnikj
    • bergloman