The OpenAddresses importer is used to process data from OpenAddresses for import into the Pelias geocoder.
Node.js 4 or higher is required.
git clonecd openaddressesnpm install
# show full command line optionsnode import.js --help# run an importnode import.js
OpenAddresses records do not contain information about which city, state (or
other region like province), or country that they belong to. Pelias has the
ability to compute these values from Who's on First data.
For more info on how admin lookup works, see the documentation for
pelias/wof-admin-lookup. By default,
adminLookup is enabled. To disable, set
false in Pelias config.
Note: Admin lookup requires loading around 5GB of data into memory.
OpenAddresses data tends to contain lots of duplicate records. In order to avoid inserting this duplicate data into Pelias, the importer can optionally run all records through the openvenues address deduplicator.
Note: The address deduper is a Python/Flask app and defaults to running on port 5000. It must be running in order for deduplication to take place!
The address deduplicator will filter out records that are both nearby, and have
nearly identical values. The deduplicator knows to expand common abbreviations,
123 Main Street and
123 Main St would be matched as duplicates.
Like admin lookup, this process slows down the importer quite a bit, and requires a large amount of disk space (it uses a LevelDB store to track which addresses it has seen), so it defaults to off.
This importer can be configured in pelias-config, in the
hash. A sample configuration file might look like:
"imports":"openaddresses":"deduplicate": false"datapath": "/tmp/oa-data""files": "us/ny/city_of_new_york.csv"
The following properties are recognized:
deduplicate: Boolean flag to enable deduplication (see above).
datapath: The absolute path of the directory containing OpenAddresses files. Must be specified if no directory is given as a command-line argument.
files: An array of the names of the files to import. If specified, only these files will be imported, rather than all
.csvfiles in the given directory. If the array is empty, all files will be imported.