denmark-dawa-replicate
Replicate the DAWA database.
Installation
npm install denmark-dawa-replicate
Example
This module just provided the necessary logic for fetching the data.
The actual insert
, update
and delete
logic is a bit more complicated,
a full postgresql example exists inside the example/
directory.
Documentation
const replicate =
DAWA (Danmarks Adressers Web API) is a service provided by the danish government, which exposes multiply APIs for getting address related information. The service supports replication of its tables, this module helps getting that data and keeping it up to date.
First create a new DAWAReplicate
object.
const replicator = version;
If there is no data in the database, set version
to 0
or just don't
specify it. The replicator
object is a Readable
object stream,
where each item has the following structure.
replicator;
To get all possible table names and associated information, see the denmark-dawa-schema module.
Using a parallel writeable stream
Since there is a lot of data, it is recommended to implement a Writeable
stream
that supports parallel writes. This can be done by implementing the
Writeable._writev
method, that takes a chunk of items.
{ streamWritable; thisdb = ;}util; InsertStreamprototype { thisdbitemoperationitemtable itemdata callback;}; InsertStreamprototype { const self = this; async;};
You then just pipe the replicator
as usual.
replicator;
replicator.update([version], callback)
Updates the current version to version
. If no version
is specified
the module will fetch the latest version for you. After the new version
has been determined, all new data (events) from DAWA
will be piped to
the replicator
object.
If a newer version exists a new-version
event will be emitted with
the new version number. As each table starts being fetched the update-table
will emit, with the corresponding table name.
The callback is called when all data has been fetched. Before this
you are not allowed to execute .update
again.
// Standard synchronization pattern { replicate;};
replicator.getLatestVersion(callback)
In case you want to be more in control, this method just fetches the latest version number, without any side effects.
replicator;
Source
DAWA documents the replication protocol at http://dawa.aws.dk/replikeringdok.