keywords:tokenizer - npm search

js-tokens

Tiny JavaScript tokenizer.

published version 9.0.1, 4 months ago5820 dependents licensed under $MIT

250,350,357

css-tree

A tool set for CSS: fast detailed parser (CSS → AST), walker (AST traversal), generator (AST → CSS) and lexer (validation and matching) based on specs and browser implementations

lahmatiy

published version 3.1.0, 3 months ago903 dependents licensed under $MIT

152,249,743

ret

Tokenizes a string that represents a regular expression.

fent

published version 0.5.0, 2 years ago828 dependents licensed under $MIT

68,732,771

micromark

small commonmark compliant markdown parser with positional info and concrete tokens

wooorm

published version 4.0.2, 20 days ago124 dependents licensed under $MIT

47,613,568

strtok3

A promise based streaming tokenizer

borewit

published version 10.2.2, 10 days ago50 dependents licensed under $MIT

25,499,492

@csstools/css-tokenizer

Tokenize CSS

romainmenke

published version 3.0.3, 5 months ago45 dependents licensed under $MIT

31,264,188

@tokenizer/token

TypeScript definition for strtok3 token

borewit

published version 0.3.0, 4 years ago10 dependents licensed under $MIT

23,085,662

stream-json is the micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. It can parse JSON files far exceeding available memory streaming individual primitives using a SAX-inspired API. I

elazutkin

published version 1.9.1, 4 months ago323 dependents licensed under $BSD-3-Clause

7,903,989

leac

Lexer / tokenizer

killy.mxi

published version 0.7.0-preview.1, 22 days ago4 dependents licensed under $MIT

9,060,286

chevrotain

Chevrotain is a high performance fault tolerant javascript parsing DSL for building recursive decent parsers

bd82

published version 11.0.3, 2 years ago351 dependents licensed under $Apache-2.0

6,620,101

scss-tokenizer

A tokenzier for Sass' SCSS syntax

xzyfer

published version 0.4.3, 3 years ago41 dependents licensed under $MIT

5,447,971

simple-html-tokenizer

Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.

rwjblue

published version 0.5.11, 4 years ago49 dependents licensed under $MIT

1,747,719

glsl-tokenizer

r/w stream of glsl tokens

dfcreative

published version 2.1.5, 6 years ago77 dependents licensed under $MIT

1,375,251

parenthesis

Parse parentheses from a string

dy

published version 3.1.8, 3 years ago63 dependents licensed under $MIT

1,300,321

glsl-token-inject-block

safely inject a block of tokens into a shader

mattdesl

published version 1.1.0, 9 years ago3 dependents licensed under $MIT

1,171,216

glsl-token-whitespace-trim

Trim the whitespace within an array of GLSL tokens

hughsk

published version 1.0.0, 9 years ago5 dependents licensed under $MIT

1,151,882

natural

General natural language (tokenizing, stemming (English, Russian, Spanish), part-of-speech tagging, sentiment analysis, classification, inflection, phonetics, tfidf, WordNet, jaro-winkler, Levenshtein distance, Dice's Coefficient) facilities for node.