feat: add tensor dumping functionality with shape information

- Add --dump-folder CLI argument to enable tensor dumping during network communication
- Implement binary dump format with tensor shape metadata (n_embed, n_tokens)
- Dump both send and receive tensors with unique filenames and counters
- Include proper parameter passing from CLI to llama_send_tensors/llama_recv_tensors functions

The dump format includes: element_type(1B) + n_embed(8B) + n_tokens(8B) + tensor_size(8B) + data
This commit is contained in:
DandinPower
2025-07-28 09:57:07 +00:00
parent eb0cac1da5
commit 9bc57a10f6
5 changed files with 109 additions and 4 deletions
+3
View File
@@ -378,6 +378,9 @@ extern "C" {
// currently works only with CPU execution
ggml_abort_callback abort_callback;
void * abort_callback_data;
// Tensor dumping path - if provided, network communication tensors will be dumped
const char * dump_folder;
};
// model quantization parameters