Search results
58 packages found
Skyflow SDK for Node.js
Taiwanese Hokkien Transliterator and Tokeniser
Tokenizing strings of text. Extracting arrays of words and optionally number, emojis, tags, usernames and email addresses from strings. For Node.js and the browser. When you need more than just [a-z] regular expressions.
Textmate token-based language service for Visual Studio Code.
- vscode
- vscode-extension
- textmate
- grammar
- language-features
- language-service
- language
- lsp
- parse
- syntax
- tokenization
- tokenizer
Very fast Porter Stemmer implementation written in Javascript.
- porter
- stemmer
- nlp
- text-analysis
- language-processing
- linguistics
- information-retrieval
- tokenization
- algorithms
- text-search
- text-mining
- word-stemming
- javascript-nlp
- text-preprocessing
- View more
## How To Use
Use LLMs to run map-reduce summarization tasks on large documents until a target token size is met.
- text-distillation
- text-summarization
- text-compression
- token-management
- llm-distillery
- large-language-model
- LLM
- semantic-chunking
- text-processing
- AI-text-reduction
- openai-api
- tokenization
text segmentation into sentences
A powerful and flexible text search library for JavaScript that enables you to build a simple text search engine.
- search
- text-search
- query
- query-parser
- parser
- tokenizer
- interpreter
- AST
- abstract-syntax-tree
- boolean-search
- evaluation
- normalization
- factory
- search-engine
- View more
Basis Theory utility for decrypting Apple Pay Tokens
NLP (natural language processing) for server and the browser in TypeScript. All lightweight and super-fast.
- nlp
- natural language processing
- sentiment analysis
- word vector
- text classification
- tokenization
- naive bayes
- english
- portuguese
A small wrapper module to simplify files and buffers tokenization using langchain
A tokenizer and lemmatizer for canonical terms in text
package to deploy the smart contracts that associate the tokens to the various clients proportional to their contribution
Multi-criteria Cantonese segmentation with dashes, intermediates, pipes, and spaces.
Wix Restaurants credit-cards tokenizer
Bedrock Tokenization Engine
RWA Token SDK for the development of permissioned tokens on SVM blockchains.
Basis Theory utility for decrypting Google Pay Tokens