Search results
433 packages found
Tiny JavaScript tokenizer.
React typeahead with Bootstrap styling
- auto complete
- auto suggest
- auto-complete
- auto-suggest
- autocomplete
- autosuggest
- bootstrap
- bootstrap tokenizer
- bootstrap typeahead
- bootstrap-tokenizer
- bootstrap-typeahead
- react
- react autocomplete
- react autosuggest
- View more
A tool set for CSS: fast detailed parser (CSS → AST), walker (AST traversal), generator (AST → CSS) and lexer (validation and matching) based on specs and browser implementations
small commonmark compliant markdown parser with positional info and concrete tokens
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
- BPE
- encoder
- decoder
- tokenizer
- GPT
- GPT-2
- GPT-3
- GPT-3.5
- GPT-4
- NLP
- Natural Language Processing
- Text Generation
- OpenAI
- Machine Learning
Chevrotain is a high performance fault tolerant javascript parsing DSL for building recursive decent parsers
React typeahead with Bootstrap styling
- auto complete
- auto suggest
- auto-complete
- auto-suggest
- autocomplete
- autosuggest
- bootstrap
- bootstrap tokenizer
- bootstrap typeahead
- bootstrap-tokenizer
- bootstrap-typeahead
- react
- react autocomplete
- react autosuggest
- View more
stream-json is the micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. It can parse JSON files far exceeding available memory streaming individual primitives using a SAX-inspired API. I
JS tokenizer for LLaMA-based LLMs
Isomorphic utilities for GPT-3 tokenization and prompt building.
Simple algorithm to tokenize Chinese texts into words using CC-CEDICT.
General natural language (tokenizing, stemming (English, Russian, Spanish), part-of-speech tagging, sentiment analysis, classification, inflection, phonetics, tfidf, WordNet, jaro-winkler, Levenshtein distance, Dice's Coefficient) facilities for node.
- natural language processing
- artifical intelligence
- statistics
- Porter stemmer
- Lancaster stemmer
- tokenizer
- bigram
- trigram
- quadgram
- ngram
- stemmer
- bayes
- classifier
- phonetic
- View more
Parse PHP code from JS and returns its AST
This project is about to implement a simple javascript parser and visualize the ast
A promise based streaming tokenizer
Dot notation tokenizer
JavaScript implementation of Japanese morphological analyzer
A CLI tool to concatenate all text files in your CWD with headers for GPT prompt engineering.
- text
- concatenate
- CLI
- Current Working Directory
- GPT
- tokens
- tokenizer
- ChatGPT
- prompt engineering
- token-count
- text manipulation
- file concatenation
- command line interface
- GPT-3
- View more