 Li, Zonghang
|
b30f749e5e
|
fix n_embd cannot be divided by quantized block size
|
2025-06-03 14:06:31 +04:00 |
|
|
Li, Zonghang
|
e25c739ecf
|
Merge pull request #19 from yezhizi/feat/auto-exit
feat: ranks with only 1 layer auto-exit and rebuild topology
|
2025-05-20 02:04:50 +08:00 |
|
|
Li, Zonghang
|
7b0ededd24
|
Merge branch 'dev' into feat/auto-exit
|
2025-05-20 02:04:14 +08:00 |
|
|
Lizonghang
|
421b3deca5
|
fix llama-cli pos sync
|
2025-05-19 18:08:27 +04:00 |
|
|
Lizonghang
|
c54a6a0132
|
fix context shifting
|
2025-05-19 16:58:35 +04:00 |
|
|
DeEMO
|
34eaa8224d
|
fix: handle socket closure and connection in llama_rebuild_topo
Signed-off-by: DeEMO <yzzxrx@gmail.com>
|
2025-05-19 09:22:35 +00:00 |
|
|
DeEMO
|
8b61cb2fa4
|
fix: adapt the new topo
Signed-off-by: DeEMO <yzzxrx@gmail.com>
|
2025-05-19 09:22:29 +00:00 |
|
|
DeEMO
|
df16b1876f
|
refactor: add zmq helper to generate message
Signed-off-by: DeEMO <yzzxrx@gmail.com>
|
2025-05-19 09:22:24 +00:00 |
|
|
DeEMO
|
0ad009a2f4
|
fix: update serialization and deserialization for next_ip in device_info
Signed-off-by: DeEMO <yzzxrx@gmail.com>
|
2025-05-19 09:22:16 +00:00 |
|
|
DeEMO
|
4b36aef157
|
fix some bugs
Signed-off-by: DeEMO <yzzxrx@gmail.com>
|
2025-05-19 09:22:08 +00:00 |
|
|
DeEMO
|
cc46aa9828
|
update rank and n_world
Signed-off-by: DeEMO <yzzxrx@gmail.com>
|
2025-05-19 09:22:02 +00:00 |
|
|
DeEMO
|
fdd6694633
|
add topo rebuild
Signed-off-by: DeEMO <yzzxrx@gmail.com>
|
2025-05-19 09:21:53 +00:00 |
|
|
DeEMO
|
26bb86c09b
|
Add tune_layer_allocation
Signed-off-by: DeEMO <yzzxrx@gmail.com>
|
2025-05-19 09:21:22 +00:00 |
|
|
Lizonghang
|
07c4966a80
|
reduce fio data size to 1gb to speed up profiling
|
2025-05-14 21:26:01 +04:00 |
|
|
Lizonghang
|
2cc01483fd
|
support server mode
|
2025-05-14 18:28:46 +04:00 |
|
|
Lizonghang
|
ebd09fc83c
|
Merge branch 'dev'
|
2025-05-14 14:19:53 +04:00 |
|
|
Lizonghang
|
258fb2d06b
|
add QA: How to manually profile a device
|
2025-05-14 14:19:20 +04:00 |
|
|
Lizonghang
|
2fbc0c8da3
|
fix: reset -ngl to 0 when GPU is not used and reformat code
|
2025-05-14 13:27:20 +04:00 |
|
|
Li, Zonghang
|
b44187e3af
|
Merge pull request #9 from yezhizi/rm-profile
Remove unnecessary profiling when `-lw` is specified
|
2025-05-14 10:05:49 +08:00 |
|
|
Li, Zonghang
|
e2de4511c5
|
Update README.md
|
2025-05-11 18:15:39 +08:00 |
|
|
Li, Zonghang
|
ca95b1efad
|
Merge pull request #11 from eltociear/patch-1
chore: update debug-test.sh
|
2025-04-22 00:32:19 +08:00 |
|
|
Ikko Eltociear Ashimine
|
fa1444ab3b
|
chore: update debug-test.sh
compliation -> compilation
|
2025-04-21 23:40:40 +09:00 |
|
|
DeEMO
|
cf4fa04980
|
Add an independent profile tool
|
2025-04-18 03:27:20 +00:00 |
|
|
DeEMO
|
168c14f4e8
|
remove unnecessary profile when --lw is specified
|
2025-04-17 13:49:09 +00:00 |
|
|
DeEMO
|
9cb87f7923
|
add fio file to gitignore
|
2025-04-17 13:44:23 +00:00 |
|
|
leeetao
|
fc1e2d3fc6
|
Added support for iq1s and iq1m quantization type
|
2025-04-17 10:27:53 +00:00 |
|
|
leeetao
|
b212d74dc3
|
update Readme.md
|
2025-04-17 09:17:11 +00:00 |
|
|
Zonghang Li
|
f9702ec4c0
|
update README.md
|
2025-04-16 15:55:43 +04:00 |
|
|
Zonghang Li
|
5a5f103833
|
fix q6k and q80
|
2025-04-16 08:55:07 +04:00 |
|
|
Li, Zonghang
|
b59d6d9cdf
|
Update README.md
|
2025-04-15 09:59:08 +08:00 |
|
|
Li, Zonghang
|
6d13836c44
|
Update README.md
|
2025-04-11 01:41:43 +08:00 |
|
|
Li, Zonghang
|
4845abf25e
|
Update README.md
|
2025-04-11 01:20:36 +08:00 |
|
|
Li, Zonghang
|
4ab9087d69
|
Update LICENSE
|
2025-04-09 21:42:55 +08:00 |
|
|
Lizonghang
|
e48b804730
|
update README.md
|
2025-04-09 13:55:30 +04:00 |
|
|
Li, Zonghang
|
55f8dc588f
|
Update README.md
|
2025-04-09 10:56:25 +08:00 |
|
|
Lizonghang
|
e421d788d3
|
update README
|
2025-04-08 23:15:43 +04:00 |
|
|
Lizonghang
|
03ff9a7654
|
update README
|
2025-04-07 23:28:01 +04:00 |
|
|
Li, Zonghang
|
a3a1f4499b
|
Update README.md
|
2025-04-07 22:14:44 +08:00 |
|
|
Li, Zonghang
|
98d73778a6
|
Update README.md
|
2025-04-07 22:13:31 +08:00 |
|
|
Li, Zonghang
|
5984b1b75f
|
Update README.md
|
2025-04-07 22:13:12 +08:00 |
|
|
Li, Zonghang
|
ebd15b4112
|
Update README.md
|
2025-04-07 22:12:11 +08:00 |
|
|
Li, Zonghang
|
35adc76337
|
Update README.md
|
2025-04-07 22:08:14 +08:00 |
|
|
Lizonghang
|
87eb1aa7ec
|
update README
|
2025-04-07 18:06:57 +04:00 |
|
|
Li, Zonghang
|
67b10034a7
|
update video to compare llama.cpp and prima.cpp
|
2025-04-07 18:01:04 +04:00 |
|
|
Lizonghang
|
7631ddcdc7
|
ignore video
|
2025-04-07 17:59:23 +04:00 |
|
|
Lizonghang
|
fffefb9259
|
update README
|
2025-04-07 17:57:57 +04:00 |
|
|
Lizonghang
|
f97a97003b
|
fix type convert
|
2025-04-07 17:57:50 +04:00 |
|
|
Lizonghang
|
3b264352e7
|
update README
|
2025-03-30 23:39:36 +04:00 |
|
|
Lizonghang
|
3a6cb1768f
|
add logo
|
2025-03-30 17:21:42 +04:00 |
|
|
Zonghang Li
|
63b45a4c26
|
add args -k and --force
|
2025-03-11 22:09:39 +04:00 |
|