Search results
456 packages found
Lightweight trimmed down encoder/decoder/tokenizer/token counter for gpt3 that is compatible with both node and browser environments
- BPE
- encoder
- decoder
- tokenizer
- GPT
- GPT-2
- GPT-3
- GPT-3.5
- GPT-4
- NLP
- Natural Language Processing
- Text Generation
- OpenAI
- Machine Learning
Tokenizes a text using regex rules and returns the tokens with their positions in the text given.
n8n node for working with BPE Tokens with OpenAI's GPT models in mind.
React-based typeahead and typeahead-tokenizer with mouseover
The API for building streaming tokenizers and lexers.
A parser generator with Bison's API
Navigate within a string
Tokenize (Scan) JSDoc buffer
Fast & simple lexical analyzer
tokenize any text stream given some basic regex rules to match tokens
Straightforward HTML parser for Node.js and browser
- html
- parser
- html5
- html5 parser
- htmlparser
- html parser
- html tree-constructor
- html to JSON
- html to AST
- html tokenizer
- tokenize
- tokenizer
- stream parsing
- stream parser
- View more
Bredon plugin to normalize case
Bredon plugin to transform initial values
Bredon plugin to trim hex values
Bredon plugin to normalize floating-point precision
Bredon color to precalculate calc expressions
Bredon plugin to remove unnecessary units from values
A lightly-typescriptified version of jison
A parser generator with Bison's API
Break down text into array of words.