Search results

nai-js-tokenizer

BPE tokenizer used for the NovelAI frontend.

• gpt
• gpt2
• pile
• sentencepiece
• nerdstash
• tokenizer
• novelai

ght901

published 1.1.0 • 2 months agopublished 1.1.0 2 months ago

@lambda_x/sptk

Sentencepiece tokenization for natural language processing, JS version.

• machine_learning
• albert
• nlp
• sentencepiece

lambda_x

published 1.1.3 • 10 months agopublished 1.1.3 10 months ago

@agnai/sentencepiece-js

Sentencepiece tokenization for natural language processing, JS version.

• machine_learning
• albert
• nlp
• sentencepiece

sceuick

published 1.1.1 • a year agopublished 1.1.1 a year ago

sentencepiece-js

Sentencepiece tokenization for natural language processing, JS version.

• machine_learning
• albert
• nlp
• sentencepiece

devilyouwei

published 1.1.0 • 2 years agopublished 1.1.0 2 years ago