kazana
A modular data warehouse system
Kazana is a data warehouse system framework. It provides high level APIs to extract, transform and index data for analysis.
See also: Glossary
Quick start
Create a folder and create a package.json
mkdir myapp
cd myapp
npm init --yes
Add kazana
as dependency
npm install --save kazana
Add a "start" script to the package.json
Create myapp/index.js
with the following content:
moduleexports = name: 'myapp' version: '1.0.0'
In future, that will be enough, but for now the public
option must be set,
see https://github.com/eHealthAfrica/kazana/issues/195.
So add a public option like this:
moduleexports = name: 'myapp' version: '1.0.0' public: 'public'
And create a public/index.html
file with
this content.
Now start the app
npm start
You need to create a test user at http://localhost:4999/_utils/#/database/_users/new
with this content
Now start the app
npm start
Once running, you can open http://localhost:5000. It will show a simple html app that lets you submit data.
Congratulations, you have built your first Kazana app and can already start accepting data submissions!
What’s next?
- Create a custom html app with user authentication
- Transform raw data reports into indicators
- Indices
- Bootstrap users & data
- build a plugin
Core philosophies
-
Quick deployments
Kazana accepts all kind of data out of the box through its generic REST API and client SDKs.
-
Iterative development
Transform scripts are not required to accept raw data reports, they simply start integrating existing raw data when they are ready.
-
Tiny core, many plugins
Instead of a monolithic architecture, Kazana is focussed on a tiny core with great code quality and stability. Context-specific features can be added with plugins.
Features
- Flexible Raw Data Reports handling Zero-setup, secure raw data warehouse
- Asynchronous Data Transformation
Integrate existing and new raw data - Indices map & reduce integrated data for analysis
- Entities
e.g. involved agencies, locations, projects, etc - Custom routes
e.g. for hook APIs from 3rd party services - Events✸
e.g. send notifications when new data submitted - Auditing✸
Get notified if data is missing - Services
e.g. to constantly pull data from 3rd party services) - Plugins
(reusable) modules with all extension points above - User authentication & permissions✸
Built-in user management and role-based permissions - Client APIs with Offline Support✸
Capture data while offline, sync later
✸ Work in progress
How it works
Kazana is a server built on top of the Hapi framework and uses CouchDB as its data store✸.
A Kazana app is defined by a Kazana Manifest. It’s the app’s main file (index.js
by default) and exports an object with pre-defined properties. A minimal manifest looks like this:
moduleexports = name: 'myapp'
Starting the app will serve a generic app for data submissions at http://localhost:5000. (TODO: https://github.com/eHealthAfrica/kazana/issues/195)
✸ Kazana spawns a PouchDB Server as a CouchDB drop-in replacement by default, unless otherwise configured, so CouchDB is not a hard requirement for development, but recommended for production use.
Local setup and Testing
git clone git@github.com:eHealthAfrica/kazana.git
cd kazana
npm install
npm test
CI test with selenium / chrome
npm run test:ci
CI test with Saucelabs
SAUCE_USERNAME=*** SAUCE_ACCESS_KEY=*** TEST_CLIENT="saucelabs:internet explorer:10:Windows 8" npm run test:ci
Credit
Brought to you by eHealth Africa — good tech for hard places.