node package manager

nxus-data-manager

nxus-data-manager

Import file contents as arrays of objects. Includes parsers and exporters for:

  • CSV / TSV
  • JSON
  • GeoJSON / ArcJSON

Provides convenience integration with nxus-storage to import and save to a storage model.

Installation

npm install nxus-data-manager --save

Options

All responses accept an options argument, in addition to any parser-specific options you can indicate:

  • mapping: object of {field: newField} name mappings
  • identityFields: for importing to models, array of fields to be used for createOrUpdate query
  • truncate: for importing to models, true/false for deleting existing collection data before import. Ignored if identityFields is provided.
  • strict: defaults to true. Only import columns/data that matches the attribute names for the model. Set to false to import everything.

Events

You can modify the records during import with the following specific events:

  • records.type: e.g. dataManager.after('records.csv', (results) => {})
  • record.type: e.g. dataManager.after('record.csv', (one) => {})
  • models.identity: e.g. dataMangaer.after('models.user', (results) => {})
  • model.identity: e.g. dataManager.after('model.user', (user) => {})

record_ events occur after parsing and name mapping model_ events occur after record events and before models are created/updated.

API


parser

Provide a parser for a particular type (file extension)

Parameters

  • type string The type (e.g. 'html') this renderer should handle
  • handler function Function to receive (content, options) and return parsed array of result objects

exporter

Provide an exporter for a particular type (file extension)

Parameters

  • type string The type (e.g. 'html') this exporter creates
  • handler function Function to receive (content, options) and return formatted output content

export

Request formattted output based on type

Parameters

  • type string The type (e.g. 'html') of the output content
  • records [object] The records to export
  • opts object Options for the exporter context

Returns Promise String of formatted output

import

Request parsed results based on type

Parameters

  • type string The type (e.g. 'html') of the content
  • content string The contents to parse
  • opts object Options for the parser context

Returns Promise Array of parsed result objects

importFile

Request parsed results from a file path

Parameters

  • filename string The filename to read and parse
  • opts object Options for the parser context

Returns Promise Array of parsed result objects

importToModel

Import string contents to a model

Parameters

  • model string The identity of the model to populate
  • type string The type (e.g. 'html') of the content
  • content string The contents to parse
  • opts object Options for the parser context

Returns Promise Array of instances

importFileToModel

Import file contents to a model

Parameters

  • model string The identity of the model to populate
  • filename string The filename to read and parse
  • opts object Options for the parser context

Returns Promise Array of instances

fixture

Import a data file as fixture data. Can specify environment option to only load for e.g. test

Parameters

  • modelId string The identity of the model to import
  • path string The path to a file
  • options object Options to pass to data-loader.importFile

API

src/index.js:32-32

Import file contents as arrays of objects

Installation

npm install nxus-data-loader --save

Options

All responses accept an options argument, in addition to any parser-specific options you can indicate:

  • mapping: object of {field: newField} name mappings
  • identityFields: for importing to models, array of fields to be used for createOrUpdate query
  • truncate: for importing to models, true/false for deleting existing collection data before import. Ignored if identityFields is provided.
  • strict: defaults to true. Only import columns/data that matches the attribute names for the model. Set to false to import everything.

Events

You can modify the records during import with the following specific events:

  • records.type: e.g. app.get('data-loader').after('records.csv', (results) => {})
  • record.type: e.g. app.get('data-loader).after('record.csv', (one) => {})
  • models.identity: e.g. app.get('data-loader').after('models.user', (results) => {})
  • model.identity: e.g. app.get('data-loader).after('model.user', (user) => {})

record_ events occur after parsing and name mapping model_ events occur after record events and before models are created/updated.

API


parser

src/index.js:87-91

Provide a parser for a particular type (file extension)

Parameters

  • type string The type (e.g. 'html') this renderer should handle
  • handler function Function to receive (content, options) and return parsed array of result objects

exporter

src/index.js:97-99

Provide an exporter for a particular type (file extension)

Parameters

  • type string The type (e.g. 'html') this exporter creates
  • handler function Function to receive (content, options) and return formatted output content

export

src/index.js:113-117

Request formattted output based on type

Parameters

  • type string The type (e.g. 'html') of the output content
  • records [object] The records to export
  • opts object Options for the exporter context

Returns Promise String of formatted output

import

src/index.js:126-133

Request parsed results based on type

Parameters

  • type string The type (e.g. 'html') of the content
  • content string The contents to parse
  • opts object Options for the parser context

Returns Promise Array of parsed result objects

importFile

src/index.js:141-150

Request parsed results from a file path

Parameters

  • filename string The filename to read and parse
  • opts object Options for the parser context

Returns Promise Array of parsed result objects

importToModel

src/index.js:160-164

Import string contents to a model

Parameters

  • model string The identity of the model to populate
  • type string The type (e.g. 'html') of the content
  • content string The contents to parse
  • opts object Options for the parser context

Returns Promise Array of instances

importFileToModel

src/index.js:173-177

Import file contents to a model

Parameters

  • model string The identity of the model to populate
  • filename string The filename to read and parse
  • opts object Options for the parser context

Returns Promise Array of instances

_storeResultsWithModel

src/index.js:245-250

Simple do-storage function to help mock up storage for tests, other applications.

Parameters

  • model [type] [description]
  • values [type] [description]
  • uniqueCriteria [type] [description]

Returns [type] [description]

export

src/JSONExporter.js:14-21

Stringify an array of results into JSON. Assumes top-level is array, unless opts.key is provided to wrap results in an object.

Parameters

parse

src/JSONParser.js:14-20

Parse JSON into an array of results. Assumes top-level is array, unless opts.key is provided to pick a top-level key from parsed object as results.

Parameters