Search results

424 packages found

Tiny JavaScript tokenizer.

published 9.0.0 12 days ago
M
Q
P
M
Q
P

A tool set for CSS: fast detailed parser (CSS → AST), walker (AST traversal), generator (AST → CSS) and lexer (validation and matching) based on specs and browser implementations

published 2.3.1 a year ago
M
Q
P

Chevrotain is a high performance fault tolerant javascript parsing DSL for building recursive decent parsers

published 11.0.3 6 months ago
M
Q
P

stream-json is the micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. It can parse JSON files far exceeding available memory streaming individual primitives using a SAX-inspired API. I

published 1.8.0 9 months ago
M
Q
P

small commonmark compliant markdown parser with positional info and concrete tokens

published 4.0.0 8 months ago
M
Q
P

A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models

published 2.1.2 4 months ago
M
Q
P

Tokenize CSS

published 2.2.3 2 months ago
M
Q
P
M
Q
P

Simple algorithm to tokenize Chinese texts into words using CC-CEDICT.

published 1.0.1 6 months ago
M
Q
P

Tokenizes a string that represents a regular expression.

published 0.5.0 a year ago
M
Q
P

A parser generator with Bison's API

published 0.4.18 6 years ago
M
Q
P

JavaScript implementation of Japanese morphological analyzer

published 0.1.2 6 years ago
M
Q
P

Isomorphic utilities for GPT-3 tokenization and prompt building.

published 1.2.0 a year ago
M
Q
P

String Tokenizer for Node.js using ICU's BreakIterators

published 2.0.0 4 months ago
M
Q
P

Parse PHP code from JS and returns its AST

published 3.1.5 8 months ago
M
Q
P

A promise based streaming tokenizer

published 7.0.0 2 years ago
M
Q
P

This project is about to implement a simple javascript parser and visualize the ast

published 1.0.9 2 years ago
M
Q
P

A lua parser

published 1.0.9 3 months ago
M
Q
P

Dot notation tokenizer

published 1.3.0 2 years ago
M
Q
P