⏳ js-tiktoken on Cloudflare Pages
Currently js-tiktoken does not work with Cloudflare Pages due to worker size limitations.
This library uses conditional export to exclude js-tiktoken implementations from Cloudflare worker bundles.
[!IMPORTANT] This is not the library that makes
js-tiktokenwork on Cloudflare Workers.
npm i @jill64/cf-tiktoken// This function can be imported by both edge and browser.
// however, on edge it returns invalid output regardless of the input value.
import { encodingForModel } from '@jill64/cf-tiktoken'
export const fn = () => {
const enc = encodingForModel('gpt-4')
// ...
}