@necrobits/aggregator

Aggregator.js

An flexible aggregator component for your backend (or even frontend).

An implementation for entity source with cache (CachedEntitySource) is also shipped in the library.

You can freely define how to gather your data by implementing your own lookup function.

Install

Using npm

npm install @necrobits/aggregator

or using yarn

yarn add @necrobits/aggregator

Quick start

Given the following data and getters

const users = [
    { id: 'A', name: 'Andy' },
    { id: 'B', name: 'Hai' }
];
const findUsers = ids => users.filter(user => ids.includes(user.id));

const todos = [
    { id: 'T1', task: 'Study' },
    { id: 'T2', task: 'Code' }
];
const findTodos = ids => todos.filter(todo => ids.includes(todo.id));

// You want aggregation for this data
const data = [
    {
        date: '22-02-2022',
        tasks: [
            {assigneeId: 'A', taskId: 'T1'}
            {assigneeId: 'B', taskId: 'T2'}
        ],
    },
// {...},
]

The code for the aggregation looks like this:

const userSource = new SimpleEntitySource('user', {
  lookupUsing: findUsers,
  entityIdBy: 'id',
});

const todoSource = new SimpleEntitySource('todo', {
  lookupUsing: findTodos,
  entityIdBy: 'id',
});
const aggregator = new Aggregator({
  user: userSource,
  todo: todoSource,
});
// or
const aggregator = new Aggregator();
aggregator.register('user', userSource).register('todo', todoSource);

Use the aggregator

const opts = {
  'tasks.*.assigneeId': {
    source: 'user',
    to: {
      key: 'assignee',
    },
    removeIdKey: true,
  },
  'tasks.*.taskId': {
    source: 'todo',
    removeIdKey: true,
  },
};
const aggregatedData = await aggregator.aggregate(data, opts);

console.log(aggregatedData);

Output:

[{
    date: '22-02-2022',
    tasks: [
        {
            id: 'T1',
            name: 'Study'
            assignee: {
                id: 'A',
                name: 'Andy'
            }
        },
        {
            id: 'T2',
            name: 'Code',
            assignee: {
                id: 'B',
                name: 'Hai'
            }
        }
    ]
},
{...}]

Syntax

The syntax to define a aggregation process is as follows:

{
    "<path to the object's ID>": {
        //<Aggregation Options>
    }
}

While declare a path to the object's ID, sometimes you have to access to an array. You can simply use * to tell the aggregator to process every element in that array (see example above).

However, if the data itself is an array, you don't need to use the asterik * at the beginning. The aggregator can recognize that automatically. Meaning, don't write *.userId if you have an array of multiple objects that contains userId,

[{ userId: '1' }, { userId: '2' }];

you can simply use userId directly.

{
    "userId":{
        // options
    }
}

Aggregation Options

Name	Type	Description	Required	Default
source	string	Name of the entity source to gather the data	Yes
to	-	If specified, the result is put into another key
to.key	string	The name of the new field to inject the data into	If to is specified
to.omitNull	boolean	If the lookuped object is null, remove the key	No	false
removeIdKey	boolean	Remove the id field after injecting the data	No	false
transform	(any) => (any)	A function to transform the data before the injection	No	Identity function (v) => v

Using cache with CachedEntitySource

You can implement an adapter that implements the EntityCache interface to use cache in CachedEntitySource.

Example

This is an example for node-cache. You can also use Typescript if you want to.

export class NodeCacheAdapter<T> implements EntityCache<T> {
  constructor(private nodeCache: NodeCache) {}

  async invalidate(keys: string[]): Promise<void> {
    this.nodeCache.del(keys);
    return;
  }

  async get(key: string): Promise<T> {
    return this.nodeCache.get(key);
  }

  async setBatch(batch: { key: string; value: any }[]): Promise<void> {
    this.nodeCache.mset(
      batch.map((b) => ({
        key: b.key,
        val: b.value,
      }))
    );
  }
}

const cache = new NodeCache();
const userSource = new CachedEntitySource<User>("user",{
    cache: new NodeCacheAdapter<User>(cache);
    lookupUsing: findUsers,
    entityIdBy: "id"
});

CachedEntitySource Options

Name	Type	Description
cache	EntityCache	The cache instance that implements the EntityCache interface
lookupUsing	EntityLookupFunction (string[]) => (T[] | Promise<T[]>)	A function that receives an array of IDs and returns an array of entities (or an Promise)
entityIdBy	string | (T) => string	The name of the ID field in the entity, or a function that receives an entity and returns its ID.

License

MIT