@beardedtim/wordsmith

Wordsmith

I shot an elephant in my pajamas last night!

API

This module exposes three functions:

makeGrammarizer

• (normalizer, tokenizer, opts) => str|[str] => [objects]
  • Defaults all to semi-sane defaults
  • normailzer must normalize words for tokenizer
  • if returned function is passed an array, it is assumed to be normalized
  • Returns information about the sentences

makeNormalizer

• (opts = {}) => str => str
  • Normalizes values based on opts.rules and opts.conversions
  • opts
    • .rules: array of { regex, replace } objects
    • .coversions: array of { regex, replace } objects

makeTokenizer

• `(opts ={}) => { toSentences: str -> [str], toWords: str -> [str] }
  • Splits words based on opts.sentenceRegex and opts.wordRegex
  • opts
    • .wordRegex: Regex to split at for words
    • .sentenceRegex: Regex to split at for sentences

Usage

const { makeTokenizer, makeNormalizer, makeGrammarizer } = require('@beardedtim/wordsmith');

const normalizerRules = [
  { regex: /some-custom-stuff/gi, replace: 'cuz' },
];

const normalizer = makeNormalizer({ rules: normalizerRules });

const tokenizer = makeTokenizer();

const sentences = tokenizer
  .toSentences(normalizer('I do stuff some-custom-stuff. With my friends!'))
  // [ 'I do stuff cuz.', 'With my friends!']

const grammarize = makeGrammarizer(normalizer, tokenizer);

const types = grammarize(sentences); // [ { type: 'declrative' }, { type: 'exlamatory' }]

RoadMap

• [ ] Parts of Speech
• [ ] Positivity/Overall emotion