@mindlearn/tokenizer

0.0.1 • Public • Published

MindScript Tokenizer

This is a simple mindscript tokenizer to make your chatbot better understand user questions.

How to Use

  1. Install the dependencies:
npm install @mindlearn/tokenizer
  1. Use the tokenizer in your code:
const tokenizer = require("@mindlearn/tokenizer");

const word = "Hello World, here is a basic mindscript tokenizer example.";
console.log(tokenizer(word));

/@mindlearn/tokenizer/

    Package Sidebar

    Install

    npm i @mindlearn/tokenizer

    Weekly Downloads

    1

    Version

    0.0.1

    License

    MIT

    Unpacked Size

    1.95 kB

    Total Files

    5

    Last publish

    Collaborators

    • aethera