Search results
149 packages found
docast utility to parse docblocks
A CLI tool for converting JSX files into standard JS.
A CLI command for converting JSX files into standard JS.
Uses esprima to extract line and block comments from a string of JavaScript. Also optionally parses code context (the next line of code after a comment).
Middleware for calling blockchain interactions in a web2 environment
safely inject a block of tokens into a shader
Transform stream that tokenizes CSS
Tokenizes a text using regex rules and returns the tokens with their positions in the text given.
NLP Functions for amplifying negations, managing elisions, creating ngrams, stems, phonetic codes to tokens and more.
- Tokenize
- Stem
- NGrams
- Bag of Words
- Phonetize
- Soundex
- Stop Words
- Sentence Breaking
- Regex
- NLP
- Natural Language Processing
Multilingual tokenizer that automatically tags each token with its type
A moo compatible tokenizer/lexer generator, sacrificing some performance for features.
JavaScript library for tokenizing strings with regular expressions
Converts a string into an array of tokens, with useful methods for looking ahead and behind, capturing, matching, et cetera.
Uses snapdragon to tokenize a single JavaScript block comment into an object, with description, tags, and code example sections that can be passed to any other comment parsers for further parsing.
A gemtext (`text/gemini`) parser with support for streaming, ASTs, and CSTs
Uses snapdragon to tokenize a single JavaScript block comment into an object, with description, tags, and code example sections that can be passed to any other comment parsers for further parsing.
Tokenize a graphql schema
Split text into tokens.
Time a JavaScript tokenizer