Elasticsearch _bulk File
The easiest way to get large amounts of data into an Elasticsearch index is by using the _bulk API.
This script creates _bulk-compatible files from regular JavaScript arrays.
Usage
const batch = require('elastic-batch-file')
const arr = [
{ id: 'hello', name: 'world' },
{ id: 'whats', name: 'up' }
]
batch(arr, 'indexname', 'out')
The file can be sent straight to Elasticsearch:
curl -s -H "Content-Type: application/x-ndjson" -u 'elastic:auth' "https://elastic-db:port/_bulk" --data-binary "@batch_indexname"; echo
Parameters
- arr: Array of Objects. Each entry requires an "id" field
- indexname: the name of the index for the bulk operation
- outfile: which file to write in the current directory. Starts with
batch_
Output
{"index":{"_index":"indexname","_type":"_doc","_id":"hello"}}
{"name":"world"}
{"index":{"_index":"indexname","_type":"_doc","_id":"whats"}}
{"name":"up"}
License
MIT © Patrick Heneise