A Node Module that accepts a JSON formatted dataset of leads, de-duplicates them, and creates a new data set without duplicates, according to certain prioritization rules.
I decided to use some of the ES6 syntax that I am learning in my own time on this project as well; the upshot is that this module wants Node v4 or better. NPM will give you a friendly reminder if you npm install this with a lower version of Node.
Review the Code
The GitHub repo for this module is here
$ npm install -g dmitry-dedup
$ dmitry-dedup <path to leads JSON file> [-n|--no-log]
The module will create a new JSON file adjacent to the passed leads file, named
deduplicated-<original file name>.
Additionally, the module will create an additional file logging the changes that it made to the original file, again adjacent to the original file location, and named
--no-log command line option is also passed, the module will not create the changeLog file.
Specific Implementation and Prioritization Notes
This module expects the leads data to be in the following format:
Any other fields are optional. However, this module will obliterate the entire older record with a newer record; fields that were present on an old record but not on a new record will NOT be retained.
entryDate is equal to the first instance, priority will be given to a record with a higher index (found later) in the list.