electron-rare/prima-cpp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

llama : add llama-3 chat template (#6751)

Browse Source 
* Added llama-3 chat template

* Update llama.cpp

Co-authored-by: Samuel Tallet <[email protected]>

* Update llama.cpp

Co-authored-by: Samuel Tallet <[email protected]>

* Update tests/test-chat-template.cpp

Co-authored-by: Samuel Tallet <[email protected]>

* Added EOS stop sequence according to https://github.com/ggerganov/llama.cpp/pull/6751#issuecomment-2065602862

* Removed adding of BOS token before first message

* Removed bos token from expected output from llama-3

* Update tests/test-chat-template.cpp

Co-authored-by: Rene Leonhardt <[email protected]>

* Update tests/test-chat-template.cpp

Co-authored-by: Rene Leonhardt <[email protected]>

* Added <|end_of_text|> as another stop token

* Reverted last change of adding the end_of_text stop word for llama 3

---------

Co-authored-by: Wouter Tichelaar <[email protected]>
Co-authored-by: Samuel Tallet <[email protected]>
Co-authored-by: Rene Leonhardt <[email protected]>
Co-authored-by: Georgi Gerganov <[email protected]>
This commit is contained in:
Wouter
2024-04-21 16:03:39 +03:00
committed by GitHub
co-authored by Wouter Tichelaar Samuel Tallet Rene Leonhardt Georgi Gerganov
parent c1386c936e
commit 7dbdba5690
2 changed files with 14 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
llama.cpp
+9
View File
@@ -17247,6 +17247,15 @@ static int32_t llama_chat_apply_template_internal(
if (add_ass) {
ss << "<|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>";
}
} else if (tmpl == "llama3" || (tmpl.find("<|start_header_id|>") != std::string::npos && tmpl.find("<|end_header_id|>") != std::string::npos)) {
// Llama 3
for (auto message : chat) {
std::string role(message->role);
ss << "<|start_header_id|>" << role << "<|end_header_id|>\n\n" << trim(message->content) << "<|eot_id|>";
}
if (add_ass) {
ss << "<|start_header_id|>assistant<|end_header_id|>\n\n";
}
} else {
// template not supported
return -1;