dynamodb-data-types

A utility for Amazon DynamoDb Data Types for AWS SDK for Node.js.

DynamoDb-Data-Types

A utility for Amazon DynamoDB data types.

This utility is designed to be used along with the Amazon SDK for Node.js. It helps represent AWS DynamoDb data types.

Version 2.0.0 of DynamoDb-Data-Types was recently released (Feb 15 2015) with a rewrite of file AttributeValue.js. Incase you encounter any issues, please report them on github.

Following are some key-value pairs:

var data = { 
  name: 'Java Script',
  age: 18,
  fav: {
    food: ['Rice', 'Noodles'],
    colors: ['Orange', 'Blue']
  },
  engines: [ 'Rhino', 'v8', 'SpiderMonkey', 'Carakan', 'JavaScriptCore' ]
}

In order to put the above data into DynamoDB, the AWS SDK requires it to be represented as:

{
  "name": { "S": "Java Script" },
  "age": { "N": "18" },
  "fav": { 
    "M": {
      "food": { "SS": [ "Rice", "Noodles" ] },
      "colors": { "SS": ["Orange", "Blue" ] }
    }
  },
  "engines": { "SS": ["Rhino", "v8", "SpiderMonkey", "Carakan", "JavaScriptCore" ] }
}

This utility helps to construct such representations required by the AWS SDK for Node.js

To represent the above var data as a DynamoDB AttributeValue do:

var attr = require('dynamodb-data-types').AttributeValue;
attr.wrap(data);
// { 
//   "name": { "S": "Java Script" }, 
//   "age": { "N": "18" }, 
//   "fav": {  
//     "M": { 
//       "food": { "SS": [ "Rice", "Noodles" ] }, 
//       "colors": { "SS": ["Orange", "Blue" ] } 
//     } 
//   }, 
//   "engines": { "SS": ["Rhino", "v8", "SpiderMonkey", "Carakan", "JavaScriptCore" ] } 
// } 
var attr = require('dynamodb-data-types').AttributeValue;
 
var infoAttrValue = attr.wrap({ name: "Foo", age: 50 });
console.log(JSON.stringify(infoAttrValue));
// {"name":{"S":"Foo"},"age":{"N":"50"}} 
 
var experience = {
  count: {"N": "4" },
  languages: { "SS": ["Java Script", "Ruby", "GLSL", "C" ] }
}
console.log(JSON.stringify(attr.unwrap(experience)));
// {"count":4,"languages":["Java Script","Ruby","GLSL","C"]} 

Refer to docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Types.html

DynamoDb-Data-Types supports:

  • AttributeValue
  • AttributeValueUpdate

Refer to docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_AttributeValue.html

DynamoDb-Data-Types supports:

  • B
  • BOOL
  • BS
  • L
  • M
  • N
  • NS
  • NULL
  • S
  • SS

Its trivial to detect N, NS, S, SS, NULL and BOOL. The other types M, L, B, BS are not difficult but need some explaining.

For any a given value val, wrap() detects the AWS Data types as follows:

How wrap() detects them (psuedo-code):

IF val is typeof boolean
    THEN detect as type BOOL
ELSE IF val is null
    THEN detect as type NULL
ELSE IF val is typeof number or if val instanceof Number
    THEN detect as type N
ELSE IF val is typeof string or if val is instanceof String
    THEN detect as type S

How wrap() detects type B (psuedo-code):

IF val is instanceof Buffer
    THEN detect as type B

There maybe other type which should get detected as B. Please let me know if you have suggestions.

How wrap() detects type M (psuedo-code):

IF (val is none of: BOOL, NULL, N, S, B)
    AND (typeof val === 'object')
        THEN detect as type M
ELSE
    wrap() ignores val

When wrap() sees an Array, here's what it does (psuedo-code):

IF val is an Array
    IF (every element in Array is type N)
        THEN detect as type NS
    ELSE IF (every element in Array is type S)
        THEN detect as type SS
    ELSE IF (every element in Array is type B)
        THEN detect as type BS
    ELSE 
        detect as type L

The source is available for download from github.com/kayomarz/dynamodb-data-types.

To install using Node Package Manager (npm):

npm install dynamodb-data-types

DynamoDb-Data-Types version 2.0.0 introduces support for AttributeValue types BOOL, NULL, M, L.

DynamoDb-Data-Types uses M to nest objects. Consider the following data:

var data = {
  polygon: {
    quadrilateral: {
        sides: 4
    }
  }
}

wrap() represents the above data as follows:

{
  "polygon": {
    "M": {
      "quadrilateral": {
        "M": {
          "sides": {
            "N": "4"
          }
        }
      }
    }
  }
}

DynamoDb-Data-Types uses L to represent mixed arrays. Consider the following data:

{
  strs: ['abc', 'def'],
  nums: [123, 456],
  mix: [1, 'abc', true, false, null, [1,2,3]]
}

wrap() represents the above data as follows:

{
  strs: { 
    SS: ["abc","def"] 
  },
  nums: { 
    NS: ["123","456"] 
  },
  mix: {
    "L": [
      { N: "1" },
      { S: "abc" },
      { BOOL: true },
      { BOOL: false },
      { NULL: true },
      { NS: ["1","2","3"] }
    ]
  }
}

Read this only if you need DynamoDb-Data-Types version 1.0.0 or below.

If you are already using version 1.0.0 or 0.2.7 you may continue to do so.

If you are using DynamoDb-Data-Types version 1.0.0 or 0.2.7, wrapping / unwrapping B and BS will not work when used with AWS SDK 1.x.x but should automagically work with AWS SDK 2.x.x. although it has not been tested. This is related to automatic conversion of base64 done by AWS SDK version 2.x. See AWS Upgrading Notes (1.x to 2.0).

AWS API Reference - AttributeValue

AWS API Reference - AttributeValueUpdate

Wrap object properties into DynamoDB's AttributeValue data type.

  • @param {Object} item The object to wrap.
  • @return {Object} A DynamoDb AttributeValue.

Example

var attr = require('dynamodb-data-types').AttributeValue;
attr.wrap({name: "Foo", age: 50});
// {"name":{"S":"Foo"},"age":{"N":"50"}} 

Unwrap DynamoDB AttributeValue to values of the appropriate types.

  • @param {Object} attributeValue The DynamoDb AttributeValue to unwrap.
  • @return {Object} Unwrapped object with properties.

Example

var attr = require('dynamodb-data-types').AttributeValue;
attr.unwrap({"name":{"S":"Foo"},"age":{"N":"50"}});
// {name: "Foo", age: 50} 

Wrap a single value into DynamoDB's AttributeValue.

  • @param {String|Number|Array}
  • @return {Object} DynamoDB AttributeValue.

Example

var attr = require('dynamodb-data-types').AttributeValue;
attr.wrap1(50);    // {"N":"50"} 
attr.wrap1("50");  // {"S":"50"} 

Unwrap a single DynamoDB's AttributeValue to a value of the appropriate javascript type.

@param {Object} attributeValue The DynamoDB AttributeValue. @return {String|Number|Array} The javascript value.

Example

var attr = require('dynamodb-data-types').AttributeValue;
attr.unwrap1({"N":"50"});  // 50 
attr.unwrap1({"S":"50"});  // "50" 
 

Append attributes to be updated with action "ADD". This function can be chained with further calls to add, put or delete.

  • @param {Object} attrs Object with attributes to be updated.
  • @return {Updates} Object with all update attributes in the chain.

Example - put, add, delete.

See note: duplicate attribute names

Append attributes to be updated with action "PUT". This function can be chained with further calls to add, put or delete.

  • @param {Object} attrs Object with attributes to be updated.
  • @return {Updates} Object with all update attributes in the chain.

Example - put, add, delete.

See note: duplicate attribute names

Append attributes to be updated with action "DELETE". This function can be chained with further calls to add, put or delete.

  • @param {Object|String|Array} attrs If this argument is an an Object,the Object's property values must be an array, containing elements to be removed, as required by DynamoDb SDK. If this argument is a String, it should contain comma seperated names of properties to be deleted. If its an Array, each array element should be a property name to be deleted.

  • @return {Updates} Object with all update attributes in the chain.

See note: duplicate attribute names

var attrUpdate = require('dynamodb-data-types').AttributeValueUpdate;
 
var dataUpdate = attrUpdate
    .put({name: "foo"})
    .add({age: 1})
    .delete("height, nickname")
    .add({favColors: ["red"]})
    .delete({favNumbers: [3]});
 
console.log(JSON.stringify(dataUpdate));
// { 
//   "name": { "Action": "PUT", "Value": { "S": "foo" } }, 
//   "age": { "Action": "ADD", "Value": { "N": "1" } }, 
//   "height": { "Action": "DELETE" }, 
//   "nickname": { "Action": "DELETE" }, 
//   "favColors": { "Action": "ADD", "Value": { "SS": ["red" ] } }, 
//   "favNumbers": { "Action": "DELETE", "Value": { "NS": ["3"] } } 
// } 

Each attribute name can appear only once in the AttributeUpdates object of the itemUpdate call. This is a feature of the AWS API. However its easy to overlook this when chaining add, put and delete updates.

For example, following is an attribute colors of type SS (String set)

var item = {
    id: ...,
    colors: ["red", "blue"]
}

Suppose, we want to delete "red" and add "orange".

To add "orange", the AttributeUpdates object is created as: attrUpdate.add({colors: ["orange"]}). Similarly, to delete "red" the AttributeUpdates object is created as attrUpdate.delete({colors: ["red"]})

However, both actions cannot be represented in the same AttributeUpdates object.

// Will not work as expected 
attrUpdate.add({colors: ["orange"]}).delete({colors: ["red"]});

The action to delete "red" overwrites the action to add "orange". This is simply because colors is a property of the AttrubuteUpdates object.

The following code demonstrates the above note:

JSON.stringify(attrUpdate.add({colors: ["orange"]}));
//{"colors":{"Action":"ADD","Value":{"SS":["orange"]}}} 
 
JSON.stringify(attrUpdate.delete({colors: ["red"]}));
//{"colors":{"Action":"DELETE","Value":{"SS":["red"]}}} 
 
// The below does not work as expected 
JSON.stringify(attrUpdate.add({colors: ["orange"]}).delete({colors: ["red"]}));
//{"colors":{"Action":"DELETE","Value":{"SS":["red"]}}} 
 

It is upto the application to ensure that the application follows the SDK requirements. This utility does not perform any checks.

This utility is designed for Node.js. For use in the browser, it will need to be adapted. This is in lieu of the October 2013 Developer Preview - AWS SDK for JavaScript in the Browser

To adapt this utility for the browser, following are few todos. (This list is not exhaustive)

  • Ensure browser compatibility of functions which iterate over objects. Currently lib/util.js has a function to iterate over object properties which is sufficient for use with Node.js.
  • Browser compatible ways to detect array Array.
  • Ways to detect binary data commonly used by browser applications. For instance, currently for Node.js, wrap() detects Buffer as binary type B.

Change log

2015-02-15

  • Fixed README
  • Committed modified package.json (just realised it wasn't committed)

2015-02-15

  • Implemnted M
  • Implemented L
  • Added example to put and get binary data (examples/02-binary-image.js)

2015-02-11

Note: There are no source code changes in version 1.0.0. Functionally, 1.0.0 is identical to 0.2.7.

  • Bumped from version 0.2.7 to version 1.0.0.
  • Update documentation especially with regard to B and BS data types.
  • Added development deps into pacakge.json instead of tests/package.json (It should have been this way to begin with)

Note: Change log dates are yyyy-mm-dd.

License