node package manager
Orgs are free. Discover, share, and reuse code in your team. Create a free org »

algolia-csv

Algolia CSV toolbox

What is this project about?

It's a CLI and library to easily upload CSV files to Algolia in an efficient manner without limitations.

Cool features:

  • uses the header (first line of the csv file) to set the name of the columns
  • automatically detects the types of the fields
  • adds an objectID automatically if not set (useful for public datasets)
  • uses batch best practices to upload to Algolia
  • based on streams

Installation / usage

Requirements

  • Node + npm
  • an algolia account and and API key that can upload (not the search API key)

CLI

npm install -g algolia-csv

You must have a file in which the first row contains the name of all the fields.

algolia-upload $APP_ID $API_KEY $indexName $file|$url [-d $delimiter] [-b $batchSizer] [--clear-index] [--parse-arrays=$column]

Mandatory parameters are the aplication id, a key with write rights, the target index name and the input CSV (locally or accessible with http/https).

Other parameters:

  • -d let you set the delimiter used in your file. This should be set in quotes. Default is ','.
  • -b let you set the batch size. Default is 10000.
  • --clear-index forces the index to be cleared before uploading the new data.
  • --parse-arrays= let you specify if a column value should be splitted with ',' before uploading the data. More than one column can be set.
  • --geo-colums=latCol,longCol let you specify two columns that are to be used for creating the special algolia attribute _geoloc

As a library

npm install --save algolia-csv
var algoliaCsvTools = require('algolia-csv');
 
algoliaCsvTools.upload({
  appId: '',
  apiKey: '',
  indexName: '',
  inputFile: '',
  batchSize: 1000,
  delimiter: ',',
  clearIndex: false,
  parseArrays: ['column'],
  geoColumns: {lat: 'latCol', 'lng': 'longColumn'}
});