最終更新:2026-04-03 (金) 22:17:16 (105d)
tiktoken
Top / tiktoken
is a fast BPE tokeniser for use with OpenAI's models.
https://github.com/openai/tiktoken
おなかすいた族!
is a fast BPE tokeniser for use with OpenAI's models.
https://github.com/openai/tiktoken