A YottaDB and GT.M database driver and language binding for Node.js
Version 0.14.0 - 2019 Jan 17
Copyright and License
Addon Module written and maintained by David Wicksell firstname.lastname@example.org
Copyright © 2012-2019 Fourth Watch Software LC
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License (AGPL) as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.
Full license text: AGPL-3.0
Contact me if you are interested in using Nodem with a different license.
Summary and Info
Nodem is an open source addon module that integrates Node.js with the YottaDB and GT.M implementations of M[UMPS], providing in-process access to their database systems and some language features. Nodem provides access to the basic global database handling operations, as well as providing the ability to invoke M language functions and procedures. It also supports full local symbol table management. Although designed for use with YottaDB and GT.M, Nodem aims to be API-compatible (while in 'strict' mode) with the in-process Node.js interface for Caché.
Nodem uses the YottaDB and GT.M C call-in interface. YottaDB has released a new, faster, low-level database access API, with version r1.20, called the SimpleAPI. Nodem uses YottaDB's SimpleAPI for the data, get, kill, nextNode, order, previous, previousNode, and set APIs, when it is available, and falls back to the call-in interface when it is not. SimpleAPI support for the rest of the API is coming soon.
NOTE: Nodem Developer API and User Guide coming soon. In the meantime, please refer to the Caché Node.js API documentation for further details.
Using Nodem with YottaDB, in the Node.js REPL:
> const ydb = ; // Create YottaDB connection instance - use Gtm() when connecting to GT.Mundefined> ydb; // Open connection to YottaDBok: true pid: 15975> ydbversion;'Node.js Adaptor for YottaDB: Version: 0.13.4 (FWS); YottaDB version: 1.22'> ydb; // write ^test(0,2,0)ok: trueglobal: 'test'subscripts: 0 2 0data: '2 bags of wheat'defined: 1> ydb; // write ^test(0,2,0)'2 bags of wheat'> ydb;undefined> result: ok: trueglobal: 'test'subscripts: 0 2 0data: '2 bags of wheat'defined: 1> ydb;undefined> result: 2 bags of wheat> ydb; // set ^test(0,2,0)="3 bags of wheat"undefined> ydb;ok: trueglobal: 'test'subscripts: 0 2 0data: '3 bags of wheat'defined: 1> ydb;ok: falseerrorCode: 150373498errorMessage:'(SimpleAPI),%YDB-E-NULSUBSC, DB access failed because Null subscripts are not allowed for current region,%YDB-I-GVIS, \t\tGlobal variable: ^test("")'> ydb; // Close connection to YottaDB, releasing resources and restoring terminal settingstrue
In order to use the Nodem addon, you will need to have YottaDB or GT.M installed and configured correctly, including setting up your environment with the required YottaDB/GT.M environment variables. Make sure you have either $ydb_dist (only applicable for YottaDB) or $gtm_dist set to the root of the YottaDB/GT.M instance before you compile Nodem. You will also need to have Node.js installed and working.
ATTENTION: These instructions assume that the nodem repository has been installed in your home directory. The paths will likely be different if you have installed this with npm.
NOTE: If you have installed Nodem using npm, it will attempt to build mumps.node during installation. If there is a file in the nodem/ directory called builderror.log, and if that file contains no build errors for mumps.node, it built without issue. It also attempts to pre-compile the v4wNode.m integration routine, and there might be warnings from that, which won't affect the building of mumps.node itself. If you downloaded Nodem any other way, including cloning it from its github repository, then you'll have to build it from source. Remember to make sure that either $ydb_dist or $gtm_dist is set to the root of the YottaDB/GT.M instance before building Nodem. In order to build it, while in the root of the Nodem repository, run the 'npm install' command, e.g.
$ cd ~/nodem$ npm install
$ node-gyp rebuild 2> builderror.log
In addition you will need to set a few environment variables in order for YottaDB/GT.M to find the call-in table and the M routine that it maps to. The Nodem package supplies a sample environment file, called environ. It has a commented out command to set $LD_LIBRARY_PATH to $gtm_dist, which you will need to uncomment if you need it. It is located in ~/nodem/resources/ and can be sourced into your working environment, either directly, or from your own environment scripts or profile/login script, e.g.
$ cd ~/nodem/resources/$ source environ
$ echo "source ~/nodem/resources/environ" >> ~/.profile
If you don't source the environ file, then you will need to put a copy of v4wNode.m into a directory that is specified in your $gtmroutines routines path, or in the routinesPath property in your call to the open API, so that YottaDB/GT.M can find it. It is located in the ~/nodem/src/ directory. Again, if you don't source the environ file, then you will also need to define the $GTMCI environment variable, or set the callinTable property in your call to the open API, and point it at the file nodem.ci, located in the ~/nodem/resources/ directory, e.g.
$ export GTMCI=~/nodem/resources/nodem.ci$ cp ~/nodem/src/v4wNode.m ~/p/
> const callinTable = '/home/dlw/nodem/resources/nodem.ci';> const routinesPath = 'home/dlw/nodem/src'; // Make sure to include your routine directories> ydb;
You can clone the repository with this command..
$ git clone https://github.com/dlwicksell/nodem.git
You can also install it via npm with this command..
$ npm install nodem
You can update to the latest version with this command..
$ npm update nodem
The open() call works a bit differently than the Caché version; it does not require any arguments, and will connect with the database specified in the environment variable $ydb_gbldir (or $gtmgbldir). If you have more than one database and would like to connect to a different one than what is defined in your environment, you can pass an object, with a property called either globalDirectory or namespace, defined as the path to your global directory file for that database, e.g.
Nodem supports setting up a custom routines path, for resolving calls to other M functions and procedures, via the routinesPath property. Make sure that one of the directories in the routinesPath contains the v4wNode.m routine, located in the Nodem src/ directory, or its compiled object, v4wNode.o, otherwise Nodem will not be fully functional. This could be used to provide a certain level of security, by giving access only to certain routines, within a Nodem process, within an environment that contains routines with unfettered access to the system in its default environment configuration, e.g.
Nodem supports setting the call-in path directly in the open call, via the callinTable property. This can be handy if you are running Nodem in an environment that has other software that uses the YottaDB/GT.M call-in interface, and you don't want to worry about namespace issues. Nor would you need to set the $GTMCI environment variable, in order for NodeM to be fully functional, e.g.
You can configure Nodem to function as a GT.CM client, allowing Nodem to connect with a remote database. In the open() method, you can set an ipAddress, and/or a tcpPort property, and Nodem will set up the environment to connect with a YottaDB/GT.M database on a remote server that already has a GT.CM server running on that address and port. If only ipAddress or tcpPort is defined, the other one will be set with a default value; 127.0.0.1 for ipAddress, or 6879 for tcpPort. Nodem will then set the $GTCM_NODEM environment variable, for that Nodem process only, with the address and port you set in the open() call, e.g.
You will also need to create a global directory file that maps one or more database segments to a data file on the remote server you want to connect with, noting that the prefix to the -file argument in the example below must be NODEM, in order to match the $GTCM_NODEM environment variable name that Nodem sets up for you, e.g.
$ mumps -run GDEGDE> change -segment DEFAULT -file=NODEM:/home/dlw/g/gtcm-server.dat
Then on the server you are connecting to, make sure you have the data file set up at the same path that you set the '-file=' option to in the global directory of your GT.CM client configuration, and have started the GT.CM server on the same ipAddress and tcpPort that you configured in the open() call in Nodem, e.g.
$ $ydb_dist/gtcm_gnp_server -log=gtcm.log -service=127.0.0.1:6879
NOTE: GT.CM only allows remote connections for the database access APIs, not the function or procedure APIs. So while using Nodem in a remote GT.CM configuration, any calls to the function or procedure APIs will result in local calls, not remote calls.
Nodem supports two different character encodings, UTF-8 and M (or binary). It defaults to UTF-8 mode. The character encoding you set in Nodem is decoupled from the underlying character encoding you have set up for the YottaDB/GT.M environment it is running in. So it is possible to work with UTF-8 encoded data in the database, while in Nodem, even if you haven't set up YottaDB/GT.M to work with UTF-8 directly. You can set it to UTF-8 mode directly by passing utf-8 or utf8, case insensitively, to the charset property. If you'd rather work with an older byte-encoding scheme, that represents all characters in a single byte, you can set charset to either m, ascii, or binary, case insensitively. One thing to keep in mind when you do so, is that Node.js internally represents data in UTF-16, but interprets data in UTF-8 in most cases. You can control this through the process stream encoding methods inside of your Node.js code. Call those methods to change the encoding to 'binary' or 'ascii', and it will interpret your data as a byte encoding, using the character glyphs in your current locale, e.g.
> processstdin;> processstdout;> ydb;
There are currently two different data modes that Nodem supports. The mode can be set to either 'canonical' or 'strict'. The default is canonical, and interprets data using the M canonical representation. I.e. Numbers will be represented numerically, etc. Strict mode interprets all data as strings, strictly following the convention set with InterSystems' Node.js driver, e.g.
Nodem also has a debug tracing mode, in case something doesn't seem to be working right. It has four levels of debugging, defaulting to 'off'. The other debug levels are 'low', 'medium', and 'high'. The higher the debug level, the more verbose the debug output will be, e.g.
Nodem handles several common signals by default, closing the database connection and stopping the Node.js process. These signals include SIGINT, SIGTERM, and SIGQUIT. The handling of the SIGQUIT signal will also generate a core dump of the process. All three signal handlers are on by default. However, you can turn the signal handling on or off directly, via passing true or false to a signalHandler object (with properties for each of the signals) for each individual signal, or all of them at once, e.g.
Third, you can also enable it globally, by setting the environment variable NODEM_AUTO_RELINK to 1, or any other non-zero number, e.g.
$ export NODEM_AUTO_RELINK=1$ node function.js
$ NODEM_AUTO_RELINK=1 node function.js
YottaDB/GT.M changes some settings of its controlling terminal device, and Nodem resets them when it closes the database connection. By default, Nodem will restore the terminal device to the state it was in when the open() call was invoked. Normally this is the desired option, however, if you wish to reset the terminal to typically sane settings, the close() call allows this by setting the resetTerminal property to true, or any non-zero number, e.g.
The lock API takes an optional timeout argument. If you do not set a timeout, it will wait to acquire the lock indefinitely. If you wish to come back from the call right away, if the lock is not available, simply pass a timeout argument of 0, e.g.
The kill API takes an optional nodeOnly argument. It can be set to true or false, defaulting to false. If set to true, then it will only remove the node that is passed to it; if set to false, then it will remove the node passed to it, and all of its children, or the full sub-tree, e.g.
Nodem provides a built-in API usage help menu. By calling the help method without an argument, Nodem will display a list of APIs and a short description of what they do. Calling the help method with an argument string of one of those APIs will display more in-depth usage information for that method.
Nodem supports full local symbol table manipulation with the current database APIs. In order to use it, instead of defining a 'global' property in your argument object, you define a 'local' property. For APIs that support passing arguments by-position, you signify that you want them to work on a global, by using a '^' as the first character of the first argument. For the get and set APIs, if the first character of the first argument is a '$', then you are signifying that you are working with a intrinsic special variable (ISV). There is also a localDirectory API, that works the same way as the globalDirectory API, except that it lists the local symbols in the symbol table, rather than the globals in the database. One caveat is that you cannot manipulate any local variable that begins with 'v4w', as Nodem internally uses that namespace to implement the v4wNode.m integration routine. You can also call the kill API with no arguments, and it will clear the local symbol table. This functionality will allow you to call legacy M functions and procedures, without having to write wrappers in M anymore, in most cases. Here is an example of using the local symbol table functionality to call a legacy API directly from Nodem. In this example, the local variable, U, needs to be set before this API is called, as it expects it to be defined already. You can also see how the local symbol table changes, after setting the required local variable, making the call, and then clearing the symbol table, e.g.
> ydb;> ydb;ok: true local: 'U' data: '^'> ydb;'U'> ydb;ok: trueprocedure: 'AGET^ORWORR'arguments: <1 empty item> 9 '2^0' 13 0> ydb;'DILOCKTM' 'DISYS' 'DT' 'DTIME' 'DUZ' 'IO' 'U' 'XPARSYS'> ydb;undefined> ydb;
> ydb;ok: true local: 'U' data: '^'> const arg = type: 'reference' value: 'LIST';undefined> ydb;ok: trueprocedure: 'LISTALL^ORWPT'arguments: type: 'reference' value: 'LIST' 'A' 1> ydb;'LIST' 'U'> ydbdatalocal: 'LIST';ok: true local: 'LIST' defined: 10> ydb;ok: truelocal: 'LIST'subscripts: 1data: '1^ZZ PATIENT,TEST ONE^^^^ZZ PATIENT,TEST ONE'defined: 1> ydb;ok: truelocal: 'LIST'subscripts: 2data: '3^ZZ PATIENT,TEST THREE^^^^ZZ PATIENT,TEST THREE'defined: 1> ydb;ok: truelocal: 'LIST'subscripts: 3data: '2^ZZ PATIENT,TEST TWO^^^^ZZ PATIENT,TEST TWO'defined: 1> ydb;ok: true local: 'LIST' defined: 0
|close||Close the database connection|
|data||Determine whether a global or local node has data and/or children|
|function||Call an extrinsic function|
|get||Retrieve the value of a global, local, or intrinsic special variable node|
|globalDirectory||List the names of the globals in the database|
|help||Display a help menu of method usage|
|increment||Atomically increment the value stored in a global or local node|
|kill||Delete a global or local node, and optionally, all of its children; or delete all local variables|
|localDirectory||List the names of the variables in the local symbol table|
|lock||Lock a global or global node, or local or local node, incrementally|
|merge||Merge a global or local tree/sub-tree, or data node, to a global or local tree/sub-tree, or data node|
|nextNode||Retrieve the next global or local node, regardless of subscript level|
|open||Open the database connection|
|order or next||Retrieve the next global or local node, at the current subscript level|
|previous||Same as order, only in reverse|
|previousNode||Same as nextNode, only in reverse in YottaDB r1.10 or newer, otherwise not yet implemented|
|procedure or routine||Call a procedure/routine/subroutine|
|retrieve||Not yet implemented|
|set||Set a global, local, or intrinsic special variable node, to a new value|
|unlock||Unlock a global or global node, or local or local node, incrementally; or release all locks|
|update||Not yet implemented|
|version or about||Display version information; if database connection open, display its version|
Fourth Watch Software endeavors not to make any breaking changes to APIs, but as Nodem is still in development, its interface may change in future versions.