Search results

@lenml/tokenizers

a lightweight no-dependency fork of transformers.js (only tokenizers)

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• transformers
• transformers.js
• gpt4
• gpt4o
• gpt3.5
• vicuna
• chatglm
• baichuan

luke_zhang

• 3.4.0 • 4 months ago • 34 dependents • Apache-2.0published version 3.4.0, 4 months ago34 dependents licensed under $Apache-2.0

@lenml/tokenizer-llama3

llama3 tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer

luke_zhang

• 3.4.2 • 4 months ago • 1 dependents • Apache-2.0published version 3.4.2, 4 months ago1 dependents licensed under $Apache-2.0

@lenml/tokenizer-claude

claude tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• claude

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0

@lenml/tokenizer-gemini

gemini tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• gemini

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0

@lenml/tokenizer-gemma3

gemma3 tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• gemma3

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0

@lenml/tokenizer-qwen2_5

qwen2_5 tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• qwen2_5

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0

aicommit2

A Reactive CLI that generates git commit messages with various AI

• cli
• ai
• git
• commit
• git-commit
• command-line
• commandline
• aipick
• aicommit
• aicommits
• aicommit2
• openai
• huggingface
• anthropic
• View more

njir

• 2.2.24 • 5 days ago • 0 dependents • MITpublished version 2.2.24, 5 days ago0 dependents licensed under $MIT

@lenml/tokenizer-qwen3

qwen3 tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• qwen3

luke_zhang

• 3.4.2 • 2 months ago • 0 dependents • Apache-2.0published version 3.4.2, 2 months ago0 dependents licensed under $Apache-2.0

@lenml/tokenizer-deepseek_v3

deepseek_v3 tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• deepseek_v3

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0

@lenml/tokenizer-gpt4o

gpt4o tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• gpt4o

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0

@lenml/tokenizer-llama2

llama2 tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0

@lenml/tokenizer-qwen1_5

qwen1_5 tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• qwen1_5

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0

@lenml/tokenizer-gemma2

gemma2 tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• gemma2

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0

ai-renamer

A Node.js CLI that uses Ollama and LM Studio models (Llava, Gemma, Llama etc.) to intelligently rename files by their contents

• ai
• files
• ollama
• openai
• llama3
• cli-tool
• lm-studio
• automation
• file-renamer
• image-renamer
• video-renamer
• file-management
• machine-learning

ozgrozer

• 1.0.24 • a year ago • 0 dependents • GPL-3.0published version 1.0.24, a year ago0 dependents licensed under $GPL-3.0

@lenml/tokenizer-gemma

gemma tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• gemma

luke_zhang

• 3.4.2 • 4 months ago • 1 dependents • Apache-2.0published version 3.4.2, 4 months ago1 dependents licensed under $Apache-2.0

@lenml/tokenizer-command_r_plus

command_r_plus tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• command_r_plus

luke_zhang

• 3.4.1 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.1, 4 months ago0 dependents licensed under $Apache-2.0

@lenml/tokenizer-text_embedding_ada002

text_embedding_ada002 tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• text_embedding_ada002

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0

@lenml/tokenizer-gpt4

gpt4 tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• gpt4

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0

@lenml/tokenizer-internlm2

internlm2 tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• internlm2

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0

@lenml/tokenizer-gpt35turbo

gpt35turbo tokenizer for NodeJS/Browser

• llama
• llama2
• llama3
• chatgpt
• mistral
• tokenizer
• gpt35turbo

luke_zhang

• 3.4.2 • 4 months ago • 0 dependents • Apache-2.0published version 3.4.2, 4 months ago0 dependents licensed under $Apache-2.0