最終更新:2025-01-24 (金) 16:13:07 (14d)
Transformers/チャットテンプレート
Top / Transformers / チャットテンプレート
文字列→モデル用のコントロールトークンを使ってトークナイズ
入力
chat = [ {"role": "user", "content": "Hello, how are you?"}, {"role": "assistant", "content": "I'm doing great. How can I help you today?"}, {"role": "user", "content": "I'd like to show off how chat templating works!"}, ]
モデルA
- mistralai/Mistral-7B-Instruct-v0.1
"<s>[INST] Hello, how are you? [/INST]I'm doing great. How can I help you today?</s> [INST] I'd like to show off how chat templating works! [/INST]"
モデルB
- HuggingFaceH4/zephyr-7b-beta
<|user|> Hello, how are you?</s> <|assistant|> I'm doing great. How can I help you today?</s> <|user|> I'd like to show off how chat templating works!</s>