Datree
Declare the data structure, transform it when needed, and send requests to update!
Datree is about a concept of full-stack web programming. This is the core implementation of this idea.
Development Status
Although datree core is done, some other important pieces should also be finished to make the whole thing works.
- datree-mongodb The MongoDB binding for datree. It might be complex in design.
- datree-socketio Using socket.io to sync data between server and client. It won't be complex.
- datree-react The React binding for datree. It could be as simple as a mixin.
I am trying hard to finish them. Contributions are welcomed!
Concepts
Declarative
While writing web apps, data connects the database and the interface. In "the REST part" of apps, we are always fetching data from server, listening user modifications, and send them back to the server. These make our code awful.
Actually, it is just the transformation of data. We do not need to care about how to send or receive data. We just need to declare what the data "shape" looks like, and which is the source of the data.
With datree, we need
- declare the "shape" and source of the data
- raise requests if update is required
- declare some filters to filter operations or change the data
In this point of view, we could see the dataflow as below.
Database (or similar data sources) <--> Datree A <--> Datree B <--> ... <--> Interface
Now, the key to write apps is defining data structures and the connections. Datree solves how data flows between databses and datrees. The interface? Fortunately, we have React, VUE, and some others which works in a declarative way.
Connective
In the datree system, we could define many pieces of data. One piece of data is called a "node". The node structure looks like a JavaScript object. Non-leaf nodes are common objects. Leaf nodes are strings, numbers, booleans, etc.
A node could be linked to a source node (of cause, the link state should be declared in advance). It is a special two-way connection. The node's value changes if its source changes. However, if the node want to change its value, it should send a request to its source, waiting the source to "permit" changes.
Shaped
Different from JavaScript objects, the data in datree is shaped. It means
- leaf nodes have a fixed type (string, number, boolean, function, or json)
- non-leaf nodes have static children declared in advance, which are not removable
But there are relaxations for some cases
- json-typed leaf nodes could hold any data that is JSON-compatible
- non-leaf nodes could be "dynamic"
Dynamic nodes could have dynamic children. The dynamic children could be inserted or removed, and have a changeable order for iteration.
The Datree Core
Datree core handles datree declaration and transformation between datrees. Here is a small example.
var Node = Node;var MemorySource = MemorySource; MemorySource;
Guide
Using Sources
To use datree, firstly a source node is required.
The easy way to create sources is using database bindings or the built-in MemorySource
to store something temporarily.
var MemorySource = MemorySource; MemorySource;
The MemorySource.create(obj, cb)
call creates a node that works as a data source.
The node could contain children just like JavaScript objects. Each property is called a field of its parent.
The obj
could be common JavaScript objects with strings, numbers, booleans, objects, and arrays.
Strings, numbers, and booleans are default values for fields.
Types of fields are inferred from default values. Types for fields could not be changed after created.
Once created, cb
is called with the created source node as the first argument.
Transformation
Once you need to transform a source node or watch the changes of some fields, you need to transform the source.
sourceNode;
You could also create a new node instead, using Node.create(def, cb)
.
The only difference is that you should specify the source node in the link statement.
var Node = Node; Node;
Understanding Nodes
Each value of a field is also a node. Every time you create a node, you are actually creating a tree of nodes. So nodes have parent/child relationships, and nodes could be leaf nodes (typed string, number, boolean, json, or function) or non-leaf nodes.
Sometimes you would like to mix source fields and linked fields in a single tree.
It is also possible through Node.create(def, cb)
, but you should handle the requests of source fields yourself.
Node;
There is a special type for leaf nodes - "function".
Function-typed nodes could be called with .exec(cb)
. It works just like calling the declared function.
However, you could not pass any parameters.
Updates
When a value of a node need to be updated, you need to update on the source (you could ONLY do it on the source nodes).
If you define a source manually (without database bindings or MemorySource), you should do it with .update(newValue, cb)
.
The new values are always converted to the type of corresponding source nodes.
sourceNode;// update to the source node itself is finished synchronouslyconsole; // === newValue
If the new value equals to the old one, the update process is not triggered at all.
Requests
You could also raise requests to update the source with .request(newValue, cb)
.
If a request is not send from the source node, it would be piped to the source.
Requested values are not updated automatically. They are handled by source nodes.
node;// you cannot assume that the value is changed or would be changed sometime - it all depends to the source
Dynamic Nodes
Sometimes you would like to dynamically add/remove children for a non-leaf node (i.e. working with arrays). You could do it by declaring the node as a "dynamic" node. Dynamic nodes could have static fields, but it is not recommended.
When you want to update the field list of a dynamic source node, you should use .createField(fieldName, def, cb)
to create a new field for it (if not created before), and then update the list with .updateFields(newFieldList, cb)
.
Node;
Dynamic fields have orders. You could iterate them with .forEach(cb)
or .forIn(cb)
.
sourceNode; sourceNode;
Using Filters
Filters are allowed in request
, update
, and updateFields
data flows.
They are quite useful for filtering data and authentication.
sourceNode;
A filtering function is sync by default.
The return value of sync filters are used as the filtered value.
You could use change the function to async by setting this.async = true
.
Then the filtered value could be provided in the callback.
If the filtered value is undefined, then the value would not be changed.
The data flow bubbles up to ancestors, so that you could declare filters in the parent to catch all dataflow to its children.
this.node
is the current node, and this.path
is an array to describe the path to the target node that raise the target data flow.
You could use this.node.getDescendant(this.path)
to get the target node.
this.interrupted
could be set to true if you want to interrupt the whole data flow and also the filter chain.
Datree obtains special locks to prevent race conditions for a single datree. There could not be two data flows running in one datree at any moment. If one data flow is unfinished (i.e. an async filter did not called the callback), the other data flow on the same datree would wait for it. Sometimes it causes deadlocks. BE AWARE OF THIS.
Busy States
A node is busy if one or more requests on itself and its descendants are waiting callbacks.
Node;
If a node is writable, there is always a non-writable node keeping track of the busy state of it.
You could get the node by node.getBusyNode()
, but you should NEVER update it yourself.
Synchronous Creation
Node creation calls Node.create(defOrShape, cb)
, node.transform(def, cb)
, and node.createField(fieldName, def, cb)
have an synchronously returned value, indicating the half-created node.
A half-created node has the same structure of the created node, but the current value might not have been transferred from its parent.
This happens in following cases.
- The node or its descendants has asynchronous
update
filters, or one of their non-cacheable sources has asynchronousupdate
filters. - The node or its descendants has asynchronous
create
filters.MemorySource
use them on arrays.
API
Install with npm install datree
.
Shape
manages the declaration of datrees.
You could create datrees with the same declaration using Shape
.
Shape = require('datree').Shape
Shape.create(def)
create and return a new shape withdef
.def
is the object to declare datrees. The format of def is listed below.def.source
could be a reference node. It used as the relative node fordef.link
, and does NOT mean this node is linked to the source. It could also be a path to reference node declared in the node's closest ancestor.def.link
should be given only when this node is linked. The value could be the source node or the path to the reference node declared bydef.source
. If a path is given, it internally callsNode.getDescendant(path)
on the reference node to find the source node.def.type
should be the type of this node if it is a leaf node, or undefined if it is a non-leaf node. The allowed types are "string", "number", "boolean", "json", and "function". You could also declare it to JavaScript built-in objects -String
,Number
,Boolean
,JSON
, andFunction
. It could also be ignored ifdef.value
is given - it could be inferred.def.value
defines the default value for initializing this node. If the node is updated with a new value and the new value could not be converted to the declared type (i.e. "str" to number), it would be set to the default value. Default values could be inferred fromdef.type
.def.cache
defines a leaf node or descendants of a non-leaf node would be cacheable or not. If a leaf node is cacheable, its value could be accessed any time. Otherwise its value could only be accessed indef.update
filters.def.writable
if set to false, the node and its descendants would be not writable. A non-writable node silently ignores the requests to it.def.fields
defines the fields of a non-leaf node. The fields could also be defined in thedef
itself if the field name is not collided with preserved definition key words. If no fields are found, all static fields are linked to the reference node (i.e. defined bydef.source
).def.addFields
links all static fields from the reference node, and add some extra fields defined here. Useful for transformation.def.removeFields
could be an array of field names. These arrays would be removed. Useful for transformation.def.dynamic
whether the node is dynamic or not. Default to false.def.update
the update filters. See guides for usages.def.updateFields
the updateFields filters. See guides for usages.def.request
the request filters. See guides for usages.def.create
the create filters. These filters are not in any data flow, have no value to filter, and would not bubble up. It just allows you do something after node creation (i.e. inserting dynamic children).def.destroy
the destroy filters. Similar todef.create
, these filters are not in any data flow, have no value to filter, and would not bubble up.
Node
provides basic support for node manipulation.
Node = require('datree').Node
Node.create(defOrShape, cb)
create a new node and return it in callback.defOrShape
could be a shape or def of a shape.node.destroy()
destroy the node. The node would not receive any requests and updates any more.node.getShape()
get the shape of the node.node.getParent()
get the parent node of the node.node.getFieldName()
get the field name in its parent of the node.node.getCachedValue()
get the cached value if the node is cacheable.node.getStaticChild(fieldName)
get the static child node by thefieldName
of the node.node.getDynamicChild(fieldName)
get the dynamic child node by thefieldName
of the node.node.getChild(fieldName)
get the static child or the dynamic child by thefieldName
of the node.node.getDescendant(path)
get the descendant by thepath
of the node.path
is an array of field names (or a single string of a field name). This method calls.getChild(fieldName)
repeatedly to find the descendant from the node.node.get()
return the cached value for leaf node, or the node itself for non-leaf node. If the node is function-typed, it returns a function that could be directly called (no need to usenode.exec(cb)
).node.get(fieldName)
callnode.get()
on the child node infieldName
of the node (i.e. return cached value for leaf child node, or the child node it self for non-leaf child node). If the field name only contains letters, numbers, and underscores (common varible name compatible),node[fieldName]
is a shortcut for this method.node.getDynamicChildren()
get an array of dynamic children of the node.node.forEach(cb)
iterate dynamic children of the node. See guides for usages.node.forIn(cb)
iterate dynamic children of the node. See guides for usages.node.isBusy([fieldName], cb)
whether the node or the child infieldName
of it is busy or not.node.getBusyNode([fieldName], cb)
get a node indicating the busy state of the node or the child infieldName
of it.node.transform(def, cb)
create a new node withdef.source
default to the node.node.createField(fieldName, def, cb)
create a new dynamic child for the node. See guides for usages.node.updateFields(arr, cb)
update field list for a dynamic node. See guides for usages.node.update([fieldName], newValue, cb)
update a source node or the child infieldName
of it. See guides for usages.node.request([fieldName], requestedValue, cb)
raise a request on a node or the child infieldName
of it. See guides for usages.node.exec([fieldName], cb)
call a function-typed node. A function-typed node is in a special data flow. You could trigger a function-typed node with no arguments using this method.
MemorySource
is a helper for creating common in-memory data sources.
MemorySource = require('datree').MemorySource
MemorySource.create(obj, cb)
create a source node. The obj
contains structure and default values for the node. Arrays in obj
would be translated to dynamic nodes, with pre-defined append and remove features. To append a child to the array, request a new object on append
field (it is a JSON-typed field). To remove a child in the array, request the field name on remove
field.
LICENSE
MIT