cdap-avsc

4.1.13 • Public • Published

Avsc

NOTE:

This is a clone of mtth/avsc repo.

The reason we have a separate repo is because CDAP supports avro-based schemas which has subtle changes (relaxed rules)

  • Map can support complex types
  • Names can have - (hyphens).

So this repo adds those rules to the parser

Pure JavaScript implementation of the Avro specification.

Features

Installation

$ npm install cdap-avsc

avsc is compatible with all versions of node.js since 0.11 and major browsers via browserify (see the full compatibility table here). For convenience, you can also find compiled distributions with the releases (but please host your own copy).

Documentation

Examples

Inside a node.js module, or using browserify:

const avro = require('cdap-avsc');
  • Encode and decode values from a known schema:

    const type = avro.parse({
      name: 'Pet',
      type: 'record',
      fields: [
        {name: 'kind', type: {name: 'Kind', type: 'enum', symbols: ['CAT', 'DOG']}},
        {name: 'name', type: 'string'}
      ]
    });
    const buf = type.toBuffer({kind: 'CAT', name: 'Albert'}); // Encoded buffer.
    const val = type.fromBuffer(buf); // {kind: 'CAT', name: 'Albert'}
  • Infer a value's type and encode similar values:

    const val = {city: 'Cambridge', zipCodes: ['02138', '02139'], visits: 2};
    const type = avro.infer(val);
    // We can now encode the value:
    const buf = type.toBuffer(val);
    // And also any values with a matching structure:
    const bufs = [
      type.toBuffer({city: 'Seattle', zipCodes: ['98101'], visits: 3}),
      type.toBuffer({city: 'NYC', zipCodes: [], visits: 0})
    ];
  • Get a readable stream of decoded values from an Avro container file:

    avro.createFileDecoder('./values.avro')
      .on('metadata', (type) => { /* `type` is the writer's type. */ })
      .on('data', (val) => { /* Do something with the decoded value. */ });
  • Implement a TCP server for an IDL-defined protocol:

    avro.assemble('./Ping.avdl', (err, attrs) => {
      // Generate the protocol and attach a handler for `ping` messages:
      const protocol = avro.parse(attrs)
        .on('ping', (req, ee, cb) => {
          cb(null, 'pong');
        });
      // Respond on any incoming connection:
      require('net').createServer()
        .on('connection', (con) => {
          protocol.createListener(con);
        })
        .listen(8000);
    });

Package Sidebar

Install

npm i cdap-avsc

Weekly Downloads

133

Version

4.1.13

License

MIT

Unpacked Size

398 kB

Total Files

17

Last publish

Collaborators

  • ajainarayanan