 Zonghang Li
|
bcfdace59b
|
add args -k and --force
|
2025-03-11 20:44:36 +04:00 |
|
|
Lizonghang
|
9cbdf01645
|
fix support for Q5_0
|
2025-02-27 22:25:03 +04:00 |
|
|
Lizonghang
|
c8e615d69c
|
fix n_m bound error
|
2025-02-27 21:59:04 +04:00 |
|
|
Lizonghang
|
550fdcbc4f
|
add support for Q5_0
|
2025-02-27 21:47:14 +04:00 |
|
|
Lizonghang
|
96e68679ce
|
fix upper bound and set calibration in halda
|
2025-02-27 17:00:27 +04:00 |
|
|
Lizonghang
|
41f3708999
|
fix condition for gpu overload
|
2025-02-25 21:31:55 +04:00 |
|
|
leeetao
|
224d14eb4c
|
Merge branch 'tao' into dev
|
2025-02-24 16:48:43 +00:00 |
|
|
leeetao
|
42da179d66
|
Added parameter display for the distilled model of deepseek-qwen
|
2025-02-24 13:24:56 +00:00 |
|
|
Lizonghang
|
e3a0d0007a
|
add gpu check in set calibration
|
2025-02-23 21:56:59 +04:00 |
|
|
Lizonghang
|
07a397360b
|
fix gpu underutilization
|
2025-02-19 16:30:18 +04:00 |
|
|
Lizonghang
|
863393554a
|
add gpu underutilization calibration step
|
2025-02-17 18:54:18 +04:00 |
|
|
Zonghang Li
|
8532d030f3
|
fix bugs in bo
|
2025-02-15 18:10:11 +04:00 |
|
|
Lizonghang
|
e64f237e04
|
fix bugs in available_mem calculation
|
2025-02-15 17:43:03 +04:00 |
|
|
Lizonghang
|
64c4a47980
|
fix bugs and warnings
|
2025-02-15 17:33:03 +04:00 |
|
|
Zonghang Li
|
630556bc16
|
fix default allocation strategy to avoid OOM
|
2025-02-15 17:23:19 +04:00 |
|
|
Lizonghang
|
fdfaaecd5e
|
disable device profiler in standalone mode
|
2025-02-12 17:12:30 +04:00 |
|
|
Lizonghang
|
c84f9d29fe
|
use arg prefetch and remove arg unload
|
2025-02-12 17:04:41 +04:00 |
|
|
Lizonghang
|
64089236eb
|
fix latency estimation in set m1
|
2025-02-03 07:56:02 +04:00 |
|
|
Lizonghang
|
83b3d01844
|
fix delay estimation on macos
|
2025-02-01 10:37:56 +04:00 |
|
|
Lizonghang
|
849b47ccd0
|
fix auto schedule logic
|
2025-01-29 13:13:37 +04:00 |
|
|
Lizonghang
|
e7c6b830e6
|
fix auto schedule logic
|
2025-01-29 11:15:45 +04:00 |
|
|
Lizonghang
|
631daadd92
|
test
|
2025-01-28 16:36:47 +04:00 |
|
|
Zonghang Li
|
36f353e374
|
check env path before calling fio to ensure we can find it
|
2025-01-28 13:06:08 +04:00 |
|
|
Lizonghang
|
2934cf3e8e
|
reserve 200 mib for internal gpu usage
|
2025-01-27 22:14:12 +04:00 |
|
|
Lizonghang
|
1e2b934d69
|
add bounds n[m]<=0 for devices without GPUs
|
2025-01-27 11:13:09 +04:00 |
|
|
Lizonghang
|
1ca9a43bd1
|
keep the output layer weights in shared memory by default
|
2025-01-25 23:31:43 +04:00 |
|
|
Lizonghang
|
f3dd5776eb
|
fix kappa and memory bounds, account for look-up table and input/output layer delay
|
2025-01-25 22:31:40 +04:00 |
|
|
Lizonghang
|
9e4ba4f06a
|
fix w init error
|
2025-01-24 20:26:18 +04:00 |
|
|
Lizonghang
|
1c0087e919
|
rename arg --keep-inp-out-in-metal to --keep-out-in-metal
|
2025-01-23 23:17:06 +04:00 |
|
|
Lizonghang
|
5fcf020cfb
|
use the recommended working set size as metal mem bound, rather than the total physical mem
|
2025-01-23 23:04:40 +04:00 |
|
|
Lizonghang
|
92f00303d5
|
set reserved mem to 100MiB
|
2025-01-23 16:58:11 +04:00 |
|
|
Lizonghang
|
78a544d716
|
add metal mem limit
|
2025-01-23 16:08:52 +04:00 |
|
|
Lizonghang
|
facb4ea736
|
add option --keep-inp-out-in-metal and fix bugs in unmap
|
2025-01-22 11:15:19 +04:00 |
|
|
Lizonghang
|
871a27f66a
|
use user input setup in standalone mode
|
2025-01-21 21:06:09 +04:00 |
|
|
Lizonghang
|
c701340719
|
set minimal disk speed to 500Mbps
|
2025-01-19 10:09:27 +04:00 |
|
|
Lizonghang
|
4f5265a78d
|
fix beta calculation on devices without GPU
|
2025-01-18 21:32:18 +04:00 |
|
|
Lizonghang
|
6761ca5358
|
show device props
|
2025-01-18 17:25:27 +04:00 |
|
|
Lizonghang
|
f9d16fbf71
|
fix manully set layer window and gpu layers
|
2025-01-18 16:44:26 +04:00 |
|
|
Lizonghang
|
70bd9db008
|
fix kappa
|
2025-01-18 11:51:18 +04:00 |
|
|
Lizonghang
|
c634c2fcbe
|
reformat highs log
|
2025-01-16 09:38:12 +04:00 |
|
|
Lizonghang
|
94f01993fa
|
reformat highs log
|
2025-01-16 09:27:53 +04:00 |
|
|
Zonghang Li
|
46e99218b4
|
add arg --cuda-mem
|
2025-01-16 09:15:34 +04:00 |
|
|
Lizonghang
|
18c96e8042
|
add option USE_HIGHS
|
2025-01-15 20:05:49 +04:00 |
|
|
Zonghang Li
|
790f702d0c
|
report k and W
|
2025-01-15 16:55:21 +04:00 |
|
|
Lizonghang
|
1e1ba5bb91
|
add api llama_model_set_n_gpu_layers
|
2025-01-15 10:47:53 +04:00 |
|
|
Lizonghang
|
9279a2e3ff
|
fix error in llama_context_n_gpu_layers
|
2025-01-15 10:08:41 +04:00 |
|
|
Lizonghang
|
5d9aadf3d5
|
use highs to solve the allocation program
|
2025-01-15 10:04:04 +04:00 |
|
|
Zonghang Li
|
df813675d0
|
fix flops count and ram/vram speed test
|
2024-12-08 10:14:05 +04:00 |
|
|
Lizonghang
|
cd823546dd
|
llama_profile_device: add arg n_predict
|
2024-12-06 16:37:25 +04:00 |
|
|
Lizonghang
|
68ecabc8c3
|
add cpu_read_ram_bw, metal_read_vram_bw, cuda_read_vram_bw
|
2024-11-29 19:04:53 +04:00 |
|