 Lizonghang
|
8e41362af0
|
fix prefetch
|
2025-02-04 17:38:41 +04:00 |
|
|
Lizonghang
|
215151918f
|
fix gpu mem limit
|
2025-01-31 18:52:13 +04:00 |
|
|
Lizonghang
|
17cd8ba618
|
reverse 300MiB for Metal kernel
|
2025-01-31 16:24:44 +04:00 |
|
|
Lizonghang
|
dd632ee6df
|
ignore the first 5 evals due to preheat
|
2025-01-31 08:53:51 +04:00 |
|
|
Lizonghang
|
b680cb74fe
|
set POSIX_MADV_WILLNEED for the next subgraph
|
2025-01-30 13:29:34 +04:00 |
|
|
Lizonghang
|
f9b4c46b74
|
ignore the first eval to make time test more accurate
|
2025-01-30 11:12:26 +04:00 |
|
|
Lizonghang
|
849b47ccd0
|
fix auto schedule logic
|
2025-01-29 13:13:37 +04:00 |
|
|
Lizonghang
|
631daadd92
|
test
|
2025-01-28 16:36:47 +04:00 |
|
|
Lizonghang
|
2934cf3e8e
|
reserve 200 mib for internal gpu usage
|
2025-01-27 22:14:12 +04:00 |
|
|
Lizonghang
|
1ca9a43bd1
|
keep the output layer weights in shared memory by default
|
2025-01-25 23:31:43 +04:00 |
|
|
Lizonghang
|
f3dd5776eb
|
fix kappa and memory bounds, account for look-up table and input/output layer delay
|
2025-01-25 22:31:40 +04:00 |
|
|
Lizonghang
|
1c0087e919
|
rename arg --keep-inp-out-in-metal to --keep-out-in-metal
|
2025-01-23 23:17:06 +04:00 |
|
|
Lizonghang
|
fb05f80f89
|
remove token_embd from metal mem
|
2025-01-23 16:58:25 +04:00 |
|
|
Lizonghang
|
78a544d716
|
add metal mem limit
|
2025-01-23 16:08:52 +04:00 |
|
|
Zonghang Li
|
33429ec4e1
|
add option --keep-inp-out-in-metal
|
2025-01-22 11:25:09 +04:00 |
|
|
Lizonghang
|
facb4ea736
|
add option --keep-inp-out-in-metal and fix bugs in unmap
|
2025-01-22 11:15:19 +04:00 |
|
|
Lizonghang
|
ce2ef9699f
|
fix mapping unmap_fragment error
|
2025-01-21 22:43:51 +04:00 |
|
|
Lizonghang
|
189ed92cba
|
segment mmap range on Metal shared memory to avoid memory waste
|
2025-01-21 21:07:02 +04:00 |
|
|
Lizonghang
|
e7fae2acdb
|
fix cuda mem limitation
|
2025-01-16 09:48:08 +04:00 |
|
|
Zonghang Li
|
46e99218b4
|
add arg --cuda-mem
|
2025-01-16 09:15:34 +04:00 |
|
|
Lizonghang
|
1e1ba5bb91
|
add api llama_model_set_n_gpu_layers
|
2025-01-15 10:47:53 +04:00 |
|
|
Lizonghang
|
9279a2e3ff
|
fix error in llama_context_n_gpu_layers
|
2025-01-15 10:08:41 +04:00 |
|
|
Lizonghang
|
5d9aadf3d5
|
use highs to solve the allocation program
|
2025-01-15 10:04:04 +04:00 |
|
|
Zonghang Li
|
0b9720c222
|
modify default max nodes back to 8192
|
2025-01-04 19:18:59 +04:00 |
|
|
Zonghang Li
|
e543fcd55f
|
fix buf_compute_meta size
|
2025-01-04 19:16:38 +04:00 |
|
|
Zonghang Li
|
b2f806a572
|
increase default max nodes from 8192 to 16384
|
2025-01-04 11:08:35 +04:00 |
|
|
Zonghang Li
|
b712ad3044
|
fix flops test error
|
2024-12-31 09:37:46 +04:00 |
|
|
Lizonghang
|
fa31ca8e35
|
add os detect
|
2024-12-30 09:13:12 +04:00 |
|
|
Lizonghang
|
a7ec685eda
|
add memcpy speed test
|
2024-12-29 16:19:08 +04:00 |
|
|
Lizonghang
|
b642d70188
|
fix swappable mem in termux
|
2024-12-12 15:15:16 +04:00 |
|
|
Lizonghang
|
8e9ab45458
|
fix model bytes counter
|
2024-12-10 14:57:48 +04:00 |
|
|
Lizonghang
|
d78fa427e7
|
add memory copy speed test
|
2024-12-09 10:07:42 +04:00 |
|
|
Lizonghang
|
2c2171cebf
|
fix display
|
2024-12-08 22:57:12 +04:00 |
|
|
Zonghang Li
|
df813675d0
|
fix flops count and ram/vram speed test
|
2024-12-08 10:14:05 +04:00 |
|
|
Lizonghang
|
cd823546dd
|
llama_profile_device: add arg n_predict
|
2024-12-06 16:37:25 +04:00 |
|
|
Lizonghang
|
a46d56cc60
|
llama_model_n_flops: remove ctxs
|
2024-12-06 11:31:53 +04:00 |
|
|
Lizonghang
|
f1c1d1b929
|
add support for Q5_K and fix byte count for Q6_K
|
2024-12-06 07:59:45 +04:00 |
|
|
Lizonghang
|
4fd3b6679e
|
fix n_gpu_layers
|
2024-12-05 20:28:19 +04:00 |
|
|
Lizonghang
|
e74967c488
|
throw error when using async upload, will support later
|
2024-12-05 14:18:39 +04:00 |
|
|
Lizonghang
|
6f54a12c7d
|
add gpu support in llama_model_kvcache_size and llama_model_compute_buf_size
|
2024-11-29 21:06:32 +04:00 |
|
|
Lizonghang
|
1123c00e45
|
recover comments in llama_profile_device
|
2024-11-29 19:07:07 +04:00 |
|
|
Lizonghang
|
68ecabc8c3
|
add cpu_read_ram_bw, metal_read_vram_bw, cuda_read_vram_bw
|
2024-11-29 19:04:53 +04:00 |
|
|
Lizonghang
|
0f73d12247
|
decrease compute buf from available memory
|
2024-11-29 11:15:54 +04:00 |
|
|
Lizonghang
|
45a1e55eec
|
reduce kv cache from available memory
|
2024-11-28 20:21:21 +04:00 |
|
|
Lizonghang
|
9a7bbce7ad
|
fix t_load_us
|
2024-11-28 15:55:21 +04:00 |
|
|
Lizonghang
|
740f7f0b95
|
use multithread disk r/w test
|
2024-11-27 22:14:17 +04:00 |
|
|
Lizonghang
|
f7507ec20b
|
fix disk r/w test, add disk access latency, and correct units (GB, GiB)
|
2024-11-27 21:36:12 +04:00 |
|
|
Zonghang Li
|
f78c437172
|
add device_inp_embd_delay test, device_memory_bw test, device_cuda_memory_bw test,
|
2024-11-26 22:28:02 +04:00 |
|
|
Lizonghang
|
a7a95b53fe
|
add q80xf32 and count_n_params
|
2024-11-24 23:11:12 +04:00 |
|
|
Lizonghang
|
3fe00a16a0
|
count model flops for f32xf32, f16xf32, q4kxf32, q6kxf32
|
2024-11-24 13:13:32 +04:00 |
|