最終更新:2024-11-27 (水) 23:48:43 (72d)  

tiktoken
Top / tiktoken

is a fast BPE tokeniser for use with OpenAI's models.

https://github.com/openai/tiktoken




検索：