keywords:tokenize - npm search

js-tokens

Tiny JavaScript tokenizer.

lydell

published version 9.0.1, 8 months ago5710 dependents licensed under $MIT

362,720,742

micromark-util-subtokenize

micromark utility to tokenize subtokens

wooorm

published version 2.1.0, 4 months ago20 dependents licensed under $MIT

42,760,753

hast-util-from-html

hast utility to parse from HTML

wooorm

published version 2.0.3, 10 months ago126 dependents licensed under $MIT

6,742,631

esast-util-from-js

estree (and esast) utility to parse from JavaScript

wooorm

published version 2.0.1, 2 years ago7 dependents licensed under $MIT

3,853,373

@stdlib/string-base-format-tokenize

Tokenize a string into an array of string parts and format identifier objects.

stdlib-bot

published version 0.2.2, a year ago2 dependents licensed under $Apache-2.0

1,815,872

html-tokenize

transform stream to tokenize html

substack

published version 2.0.1, 5 years ago22 dependents licensed under $MIT

2,751,829

glsl-token-inject-block

safely inject a block of tokens into a shader

mattdesl

published version 1.1.0, 9 years ago3 dependents licensed under $MIT

1,218,210

@stdlib/string-base-format-interpolate

Generate string from a token array by interpolating values.

stdlib-bot

published version 0.2.2, a year ago2 dependents licensed under $Apache-2.0

1,815,284

css-tokenize

Transform stream that tokenizes CSS

anandthakker

published version 1.0.1, 11 years ago6 dependents licensed under $MIT

1,717,785

json-lexer

lexing json

finnpauls

published version 1.2.0, 3 years ago9 dependents licensed under $ISC

814,491

extract-comments

Uses esprima to extract line and block comments from a string of JavaScript. Also optionally parses code context (the next line of code after a comment).

jonschlinkert

published version 1.1.0, 7 years ago59 dependents licensed under $MIT

376,210

wink-nlp

Developer friendly Natural Language Processing ✨

sanjaya

published version 2.4.0, 8 days ago35 dependents licensed under $MIT

192,039

parse5-with-errors

Fork: HTML parsing/serialization toolset for Node.js. WHATWG HTML Living Standard (aka HTML5)-compliant. Fork from the HTMLParseErrorWG branch.

pmdartus

published version 4.0.4, 5 years ago0 dependents licensed under $MIT

54,566

excel-formula-tokenizer

Tokenize Excel formulas

psalaets

published version 3.0.0, 4 years ago16 dependents licensed under $MIT

45,019

tokenize-comment

Uses snapdragon to tokenize a single JavaScript block comment into an object, with description, tags, and code example sections that can be passed to any other comment parsers for further parsing.