Nitrate Processed Mincemeat

    eth-indexer

    0.1.12 • Public • Published

    High Performance Ethereum Indexer

    Syncs events from Ethereum and indexes them for fast retrieval. This is useful and often essential for production Dapps.

    • Support for 6 different persistence stores
    • Includes "single-node friendly" stores such as flat-file JSON
    • Comprehensive error retry logic when communicating with Ethereum node
    • Maximum throughput via JSON RPC using batched parallel fetching

    Todo

    • Keep track of blockNumbers in persistence for fast-resume
    • Benchmark documentation
    • Support going forward syncing for Ethereum
    • NPM package
    • Improved failsafes for data consistency
    • Split up stores into separate dependencies

    Supported Indexing Stores and Benchmarks

    The following indexing implementations have been benchmarked on an Ethereum blockchain section with a relatively large number of events (EtherDelta exchange blocks 4800000 - 5002718) on a single 8-core 2017 Macbook Pro:

    Store Name Store ID Events Per Second Blocks Per Second
    Local Memory memory 4072.7/s 464.5/s
    Redis redis 2049.55/s 235.25/s
    Local Flat File file 1798.75/s 201.75/s
    MongoDB mongodb 795.5/s 94.6/s
    Elasticsearch elasticsearch 452.25/s 57.1/s
    LevelDB level 207.35/s 26.95/s

    Install Dependencies

    yarn install
    

    Testing & Linting

    yarn test
    yarn lint
    

    Integration tests (requires synced ethereum node at localhost:8545):

    yarn test:integration
    

    Example: Sync trading and balance events from decentralized trading contract: EtherDelta

    Contract address: https://etherscan.io/address/0x8d12a197cb00d4747a1fe03395095ce2a5cc6819

    See examples/etherdelta/sync.js.

    Define which events to index on which keys:

    const indexing = {
      events: {
        Withdraw: {
          keys: ['user'],
        },
        Trade: {
          keys: ['tokenGive', 'tokenGet', 'get', 'give'],
        },
      },
    };

    Then create a store:

    const store = new LevelStore(indexing, '/tmp/etherdelta.db');

    Now boot up the indexer:

    const indexer = new Indexer(store, EtherdeltaABI, '0x8d12a197cb00d4747a1fe03395095ce2a5cc6819');
    await indexer.syncAll({
      fromBlock: 4906764,
    });

    Once indexing is complete events can be retrieved as follows:

    const events = await store.get('Withdraw', 'user', '0x13d8d38421eb02973f3f923a71a27917bd483190');

    Directory Structure

    • package.json - Configure dependencies
    • dist/* - Files generated by babel
    • src - All source code
    • src/*/__tests__ - Unit tests
    • integration/__tests__ - Integration tests

    Keywords

    none

    Install

    npm i eth-indexer

    DownloadsWeekly Downloads

    0

    Version

    0.1.12

    License

    MIT

    Unpacked Size

    289 kB

    Total Files

    42

    Last publish

    Collaborators

    • dominiek