Search results

simple-text2word-tokens

simple text to word tokenizer. accepts a custom regex. returns an array of tokens

• tokenize
• text

bboydflo

published 0.0.2 • 5 years agopublished 0.0.2 5 years ago

overview-js-tokenizer

Splits text into tokens in as many languages as possible, using libicu

• token
• word
• tokenize
• tokenizer
• natural
• language
• processing
• text
• icu
• libicu

adamhooper

published 0.1.0 • 9 years agopublished 0.1.0 9 years ago

monaco-editor-node

Wrapper of monaco-editor to use some functionalities in Node.js.

• monaco
• editor
• vscode
• node
• synthax
• code
• ast
• highlight
• text
• tokenize

inokawa

published 0.3.0 • 2 years agopublished 0.3.0 2 years ago

3h-ast

A simple AST generator.

• ast
• tokenize
• token
• 3h

3h

published 0.14.0 • 2 years agopublished 0.14.0 2 years ago

parse-regex

Parse regular expressions into an AST.

• ast
• tokenize
• tokens
• parse
• parse-regex
• parser
• regex
• regular expression

jonschlinkert

published 0.1.0-beta • 6 years agopublished 0.1.0-beta 6 years ago

onaml

Oh No, Another Markup Language. Based on YAML, but stores considerably more object metadata.

• JSON
• YAML
• ONAML
• persistence
• serialization
• stringify
• tokenize

stefanhamminga

published 0.0.4 • 9 years agopublished 0.0.4 9 years ago