96191d3b6f
feat: add CLI flag to control communication and computation logging
2025-08-02 12:55:12 +00:00
e00cef66ac
docs: add comprehensive CHANGES.md documenting prima.cpp enhancements
2025-08-02 12:34:56 +00:00
c949e531d5
feat: make llama-perplexity can support with multi node feature
2025-08-02 11:48:26 +00:00
e48e955e81
Merge pull request #4 from DandinPower/feat/comm_tensors_dump
...
Add tensor dumping functionality for network communication analysis
2025-07-28 18:03:52 +08:00
9bc57a10f6
feat: add tensor dumping functionality with shape information
...
- Add --dump-folder CLI argument to enable tensor dumping during network communication
- Implement binary dump format with tensor shape metadata (n_embed, n_tokens)
- Dump both send and receive tensors with unique filenames and counters
- Include proper parameter passing from CLI to llama_send_tensors/llama_recv_tensors functions
The dump format includes: element_type(1B) + n_embed(8B) + n_tokens(8B) + tensor_size(8B) + data
2025-07-28 09:57:07 +00:00
eb0cac1da5
fix: update the sbatch_tokens from %u to %lu to match unsigned long
2025-07-06 02:38:26 +08:00
adad23df61
feat: add more detail batch information
2025-07-05 21:40:58 +08:00
a399e49194
feat: add comm & compute log for further gantt chart analysis
2025-07-05 18:09:11 +08:00
f1f7e37cdd
Merge pull request #3 from DandinPower/feat/enable-rank-specific-gguf-file-loading
...
Feat/enable rank specific gguf file loading
2025-06-23 01:33:06 +08:00
b51eb86676
doc: update documentation for multi splits feature
2025-06-22 22:58:07 +08:00
17edd6c96f
feat: remove the string_split function
2025-06-22 22:16:15 +08:00
6a898ef9d8
docs: clarify split loading usage
2025-06-19 19:55:56 +08:00
1e7ae71ce5
feat: Add configurable network port options
2025-06-10 23:16:57 +08:00
fbf853341b
add endpoint /v1/cancel
2025-06-07 11:34:38 +04:00
c8af1be27e
Merge pull request #24 from Lizonghang/lizh_dev
...
Fix batch decoding and dynamic batching.
2025-06-07 01:02:10 +04:00
22a6ddef13
fix batch decoding and dynamic batching
2025-06-07 00:53:56 +04:00
e56be76bdf
assume only a single seq_id per token is needed
2025-06-07 00:42:44 +04:00
d8aea899d1
fix n_seq_id and seq_id
2025-06-06 23:58:03 +04:00
a1a2238831
add batch_all.n_seq_id and batch_all.seq_id to sync_meta
2025-06-06 23:36:53 +04:00
68ecc8509d
add batch_all.logits to sync_meta
2025-06-06 22:58:48 +04:00
500e066a2f
fix batch decoding and dynamic batching
2025-06-06 16:53:22 +04:00
ef1e10101e
add test for IQ1 and doc for device selection
2025-06-04 15:12:00 +04:00
27756ee182
fix: enable rolling back set assignment when all devices are assigned to M4 but no feasible solutions
2025-06-04 15:11:29 +04:00
6439090920
reformat code
2025-06-03 23:53:24 +04:00
b6fdbd541b
Merge branch 'dev' of github.com:Lizonghang/prima.cpp into dev
2025-06-03 18:20:17 +04:00
9f0ec78a4b
Merge branch 'dev' of github.com:Lizonghang/prima.cpp into dev
2025-06-03 18:18:53 +04:00
a01fafd126
Merge branch 'main' into dev
2025-06-03 17:56:47 +04:00
1b3b6a506f
fix: add warm-up in profiling to prevent init delay
2025-06-03 17:10:09 +04:00
b30f749e5e
fix n_embd cannot be divided by quantized block size
2025-06-03 14:06:31 +04:00
e25c739ecf
Merge pull request #19 from yezhizi/feat/auto-exit
...
feat: ranks with only 1 layer auto-exit and rebuild topology
2025-05-20 02:04:50 +08:00
7b0ededd24
Merge branch 'dev' into feat/auto-exit
2025-05-20 02:04:14 +08:00
421b3deca5
fix llama-cli pos sync
2025-05-19 18:08:27 +04:00
c54a6a0132
fix context shifting
2025-05-19 16:58:35 +04:00
34eaa8224d
fix: handle socket closure and connection in llama_rebuild_topo
...
Signed-off-by: DeEMO <yzzxrx@gmail.com >
2025-05-19 09:22:35 +00:00
8b61cb2fa4
fix: adapt the new topo
...
Signed-off-by: DeEMO <yzzxrx@gmail.com >
2025-05-19 09:22:29 +00:00
df16b1876f
refactor: add zmq helper to generate message
...
Signed-off-by: DeEMO <yzzxrx@gmail.com >
2025-05-19 09:22:24 +00:00
0ad009a2f4
fix: update serialization and deserialization for next_ip in device_info
...
Signed-off-by: DeEMO <yzzxrx@gmail.com >
2025-05-19 09:22:16 +00:00
4b36aef157
fix some bugs
...
Signed-off-by: DeEMO <yzzxrx@gmail.com >
2025-05-19 09:22:08 +00:00
cc46aa9828
update rank and n_world
...
Signed-off-by: DeEMO <yzzxrx@gmail.com >
2025-05-19 09:22:02 +00:00
fdd6694633
add topo rebuild
...
Signed-off-by: DeEMO <yzzxrx@gmail.com >
2025-05-19 09:21:53 +00:00
26bb86c09b
Add tune_layer_allocation
...
Signed-off-by: DeEMO <yzzxrx@gmail.com >
2025-05-19 09:21:22 +00:00
07c4966a80
reduce fio data size to 1gb to speed up profiling
2025-05-14 21:26:01 +04:00
2cc01483fd
support server mode
2025-05-14 18:28:46 +04:00
ebd09fc83c
Merge branch 'dev'
2025-05-14 14:19:53 +04:00
258fb2d06b
add QA: How to manually profile a device
2025-05-14 14:19:20 +04:00
2fbc0c8da3
fix: reset -ngl to 0 when GPU is not used and reformat code
2025-05-14 13:27:20 +04:00
b44187e3af
Merge pull request #9 from yezhizi/rm-profile
...
Remove unnecessary profiling when `-lw` is specified
2025-05-14 10:05:49 +08:00
e2de4511c5
Update README.md
2025-05-11 18:15:39 +08:00
ca95b1efad
Merge pull request #11 from eltociear/patch-1
...
chore: update debug-test.sh
2025-04-22 00:32:19 +08:00
fa1444ab3b
chore: update debug-test.sh
...
compliation -> compilation
2025-04-21 23:40:40 +09:00
DaninPower