Search results
11 packages found
Sort by: Default
- Default
- Most downloaded this week
- Most downloaded this month
- Most dependents
- Recently published
Multilingual tokenizer that automatically tags each token with its type
published version 5.3.0, 3 years ago18 dependents licensed under $MIT
56,845
Port from Apache Lucene
published version 5.3.3, 9 years ago0 dependents licensed under $ISC
121
A text tokenizing library that handles strings, by tokenizing them into arrays, depending on intended format, like sentences, sub sentences, paragraphs, words...
- NLP tokenizer
- Tokenizer
- Sentence tokenizer
- Text tokenizer
- Node.js tokenizer
- Flexible text tokenizer
- CommonJS text tokenizer
- Paragraph tokenizer
- Sub sentence tokenizer
- Stable tokenizer
- Easy tokenizer
- Light text tokenizer
published version 1.2.0, 7 months ago0 dependents licensed under $MIT
75
NFA Tokenizer
published version 0.1.8, 7 years ago1 dependents licensed under $MIT
69
A super simple basic parser / tokenizer for easier processing of various configuration files
published version 1.1.0, 6 months ago0 dependents licensed under $MIT
46
RWKV / gpt-NeoX / Pythia, 0-dep tokenizer library, for nodejs
published version 1.0.5, 2 years ago1 dependents licensed under $MIT
44
Mathematical expression solver / Reverse Polish Notation calculator for NodeJS
published version 1.0.4, 4 years ago0 dependents licensed under $MIT
37
Lexer generator from RegExp spec
published version 0.1.10, 4 years ago0 dependents licensed under $MIT
39
Lexical tokenizer with grammar classes ready to go.
published version 1.0.10, 3 years ago0 dependents licensed under $ISC
32
module for nksoft tokens
published version 1.1.2, 4 years ago0 dependents licensed under $ISC
31
A simple lexer for javascript
published version 1.0.2, 4 years ago1 dependents licensed under $GPL-3.0-or-later
21