Get unlimited public & private packages + team-based management with npm Teams.Learn more »

elastictractor

1.2.13 • Public • Published

Elastic Tractor Build Status

This is a tool to extract, transform and load data based on Grok Patterns.

Firstly it was designed to run as a AWS Lambda, but with some efforts you could use with any serverless platform.

Currently we are supporting events from:

  • Source: Kinesis, S3, Cloudwatch Logs and ECS Events.
  • Destination: Kinesis, Kinesis Firehose and ElasticSearch.

Current Status

Stable, but under constant development.

Installation

npm install elastictractor

Features

  • Listen events from Kinesis, S3, CloudWatch Logs and ECS Events;
  • Extract information based on Grok Patterns;
  • Apply some pre-defined transformations like, parseInt, parseJson, add and delete fields;
  • Load data into Kinesis, Kinesis Firehose or ElasticSearch.

Introduction

It is a simple example to use with AWS Lambda:

var elasticTractor = require('elastictractor')
var processor = new elasticTractor({
  elkHost: "http://localhost:9200"
});
 
exports.handler = function(event, context, callback) {
  processor.handler(event, context, callback);
};

It is a basic authentication example to use with AWS Lambda:

var elasticTractor = require('elastictractor')
var processor = new elasticTractor({
  elkHost: "http://localhost:9200",
  username: "test",
  password: "123"
});
 
exports.handler = function(event, context, callback) {
  processor.handler(event, context, callback);
};

It is a cloud example to use with AWS Lambda:

var elasticTractor = require('elastictractor')
var processor = new elasticTractor({
  cloudId: "name:SJHSJSHJSHHDIhb23331=",
  username: "test",
  password: "123"
});
 
exports.handler = function(event, context, callback) {
  processor.handler(event, context, callback);
};

Install

npm i elastictractor

DownloadsWeekly Downloads

0

Version

1.2.13

License

Apache-2.0

Unpacked Size

49.5 kB

Total Files

7

Last publish

Collaborators

  • avatar