 Lizonghang
|
3d75b8576e
|
add api llama_model_set_n_gpu_layers
|
2025-01-15 10:48:19 +04:00 |
|
|
Lizonghang
|
1e1ba5bb91
|
add api llama_model_set_n_gpu_layers
|
2025-01-15 10:47:53 +04:00 |
|
|
Lizonghang
|
9279a2e3ff
|
fix error in llama_context_n_gpu_layers
|
2025-01-15 10:08:41 +04:00 |
|
|
Lizonghang
|
5d9aadf3d5
|
use highs to solve the allocation program
|
2025-01-15 10:04:04 +04:00 |
|
|
Lizonghang
|
b577c10d25
|
add noreturn flag for monitor
|
2025-01-07 20:38:11 +04:00 |
|
|
Zonghang Li
|
0b9720c222
|
modify default max nodes back to 8192
|
2025-01-04 19:18:59 +04:00 |
|
|
Zonghang Li
|
e543fcd55f
|
fix buf_compute_meta size
|
2025-01-04 19:16:38 +04:00 |
|
|
Zonghang Li
|
b2f806a572
|
increase default max nodes from 8192 to 16384
|
2025-01-04 11:08:35 +04:00 |
|
|
Zonghang Li
|
b712ad3044
|
fix flops test error
|
2024-12-31 09:37:46 +04:00 |
|
|
Zonghang Li
|
0a5450487c
|
fix serialize error
|
2024-12-31 08:24:31 +04:00 |
|
|
Zonghang Li
|
e7c75d6b4a
|
fix get device os error on linux
|
2024-12-30 21:03:51 +04:00 |
|
|
Lizonghang
|
fa31ca8e35
|
add os detect
|
2024-12-30 09:13:12 +04:00 |
|
|
Lizonghang
|
d9beb030ee
|
add EPS in device_compute_delay
|
2024-12-29 22:31:45 +04:00 |
|
|
Lizonghang
|
fa210d2034
|
remove duplicate calls
|
2024-12-29 22:00:17 +04:00 |
|
|
Lizonghang
|
a7ec685eda
|
add memcpy speed test
|
2024-12-29 16:19:08 +04:00 |
|
|
Lizonghang
|
5b46c4e848
|
correct GB to GiB
|
2024-12-28 10:09:43 +04:00 |
|
|
Lizonghang
|
b2bc836423
|
reformat
|
2024-12-28 08:36:24 +04:00 |
|
|
Lizonghang
|
70811d85b3
|
remove OOM warning
|
2024-12-22 11:01:45 +04:00 |
|
|
Lizonghang
|
b6fd762fa8
|
fix log
|
2024-12-21 19:01:00 +04:00 |
|
|
Lizonghang
|
aa6c352aa2
|
reverse 128MiB free mem in termux
|
2024-12-12 21:27:36 +04:00 |
|
|
Lizonghang
|
1b169f8f37
|
remove anonpages for repeat count
|
2024-12-12 20:28:59 +04:00 |
|
|
Zonghang Li
|
6ea692d974
|
use small tensors for test on cpu
|
2024-12-12 15:52:27 +04:00 |
|
|
Lizonghang
|
b642d70188
|
fix swappable mem in termux
|
2024-12-12 15:15:16 +04:00 |
|
|
Lizonghang
|
fa05a482f8
|
fix swapout and mmapin delay
|
2024-12-12 09:58:10 +04:00 |
|
|
Lizonghang
|
8c491c0fb1
|
consider swappable mem and swap space in termux
|
2024-12-11 21:24:37 +04:00 |
|
|
Lizonghang
|
28ebc6478a
|
use disk.read_rnd.bw in termux
|
2024-12-11 14:56:08 +04:00 |
|
|
Lizonghang
|
dfb37671d3
|
fix memory count and use sequantial disk read speed on linux
|
2024-12-11 09:30:02 +04:00 |
|
|
Lizonghang
|
bef2662d42
|
fix infinite loop in fio test
|
2024-12-10 20:58:17 +04:00 |
|
|
Lizonghang
|
8e9ab45458
|
fix model bytes counter
|
2024-12-10 14:57:48 +04:00 |
|
|
Zonghang Li
|
2d79554694
|
fix device_physical_memory
|
2024-12-09 14:42:49 +04:00 |
|
|
Zonghang Li
|
4104eb5b7a
|
fix std::max val type
|
2024-12-09 13:36:53 +04:00 |
|
|
Zonghang Li
|
a11e0dd0fa
|
fix device_cgroup_physical_memory in docker container
|
2024-12-09 13:34:01 +04:00 |
|
|
Lizonghang
|
d78fa427e7
|
add memory copy speed test
|
2024-12-09 10:07:42 +04:00 |
|
|
Lizonghang
|
1aee5bd6da
|
hide metal log if not debug
|
2024-12-09 09:17:49 +04:00 |
|
|
Lizonghang
|
cead0f2e5a
|
device_disk_access_delay: adapt for linux (on termux)
|
2024-12-08 22:58:08 +04:00 |
|
|
Lizonghang
|
2c2171cebf
|
fix display
|
2024-12-08 22:57:12 +04:00 |
|
|
Lizonghang
|
aad604ec8b
|
add disk access delay while swapping
|
2024-12-08 20:07:30 +04:00 |
|
|
Zonghang Li
|
f5243571d1
|
use sequential vram read test
|
2024-12-08 10:14:52 +04:00 |
|
|
Zonghang Li
|
d2dc8fd3b8
|
set CUDA_READ_BLOCK_SIZE to 512
|
2024-12-08 10:14:20 +04:00 |
|
|
Zonghang Li
|
df813675d0
|
fix flops count and ram/vram speed test
|
2024-12-08 10:14:05 +04:00 |
|
|
Lizonghang
|
26c2ffb5b7
|
set fio read-job startdelay 1.5s
|
2024-12-06 17:03:42 +04:00 |
|
|
Lizonghang
|
a79ab3dde3
|
device_disk_access_delay: fix delay calculation when physical memory is busy (all mapped tensors will be reloaded)
|
2024-12-06 16:41:55 +04:00 |
|
|
Lizonghang
|
cd823546dd
|
llama_profile_device: add arg n_predict
|
2024-12-06 16:37:25 +04:00 |
|
|
Lizonghang
|
a46d56cc60
|
llama_model_n_flops: remove ctxs
|
2024-12-06 11:31:53 +04:00 |
|
|
Lizonghang
|
f1c1d1b929
|
add support for Q5_K and fix byte count for Q6_K
|
2024-12-06 07:59:45 +04:00 |
|
|
Lizonghang
|
e6f4c009ab
|
device_disk_access_delay: add delay for loading one row from lookup table
|
2024-12-05 20:38:28 +04:00 |
|
|
Lizonghang
|
4fd3b6679e
|
fix n_gpu_layers
|
2024-12-05 20:28:19 +04:00 |
|
|
Lizonghang
|
3c8dbd11fc
|
device_compute_delay: compatible with the shared memory in Metal
|
2024-12-05 20:02:13 +04:00 |
|
|
Lizonghang
|
e74967c488
|
throw error when using async upload, will support later
|
2024-12-05 14:18:39 +04:00 |
|
|
Zonghang Li
|
5a99c5894d
|
update test model, enable warm-up and sched
|
2024-12-05 09:52:16 +04:00 |
|