node-file-dedup

A node module that removes duplicate lines from one or more files.

The output file will be sorted.

Example

const nodeFileDedup = require('node-file-dedup');
 
nodeFileDedup.dedup([
    'input1.txt',
    'input2.txt',
    'input3.txt'
  ], 'output.txt', function() {
    console.log('Files merged and deduplicated lines');
});

API

The API is pretty straight forward.

nodeFileDedup.dedup(files, output, callback)

• files: either one file or an array of files
• output: the output file
• callback: is called when deduplication is finished