feat: add tensor dumping functionality with shape information

- Add --dump-folder CLI argument to enable tensor dumping during network communication
- Implement binary dump format with tensor shape metadata (n_embed, n_tokens)
- Dump both send and receive tensors with unique filenames and counters
- Include proper parameter passing from CLI to llama_send_tensors/llama_recv_tensors functions

The dump format includes: element_type(1B) + n_embed(8B) + n_tokens(8B) + tensor_size(8B) + data
This commit is contained in:
DandinPower
2025-07-28 09:57:07 +00:00
parent eb0cac1da5
commit 9bc57a10f6
5 changed files with 109 additions and 4 deletions
+7
View File
@@ -2059,6 +2059,13 @@ gpt_params_context gpt_params_parser_init(gpt_params & params, llama_example ex,
}
}
));
add_opt(llama_arg(
{"--dump-folder"}, "FOLDER",
"folder to dump network communication tensors (no dumping if unset)",
[](gpt_params & params, const std::string & value) {
params.dump_folder = value;
}
));
add_opt(llama_arg(
{"--positive-file"}, "FNAME",
format("positive prompts file, one prompt per line (default: '%s')", params.cvector_positive_file.c_str()),