 Lizonghang
|
fdfaaecd5e
|
disable device profiler in standalone mode
|
2025-02-12 17:12:30 +04:00 |
|
|
Lizonghang
|
c84f9d29fe
|
use arg prefetch and remove arg unload
|
2025-02-12 17:04:41 +04:00 |
|
|
Lizonghang
|
708b1d8c89
|
disable force fetching
|
2025-02-12 16:55:44 +04:00 |
|
|
Lizonghang
|
ea0e655a8b
|
disable force feteching
|
2025-02-12 16:55:21 +04:00 |
|
|
Lizonghang
|
b163918b46
|
disable prefetch in standalone mode
|
2025-02-12 00:17:33 +04:00 |
|
|
Lizonghang
|
6a50d494d2
|
increase prefetch dense
|
2025-02-11 17:25:06 +04:00 |
|
|
Lizonghang
|
65ad14140a
|
do not check loaded tensors due to increased latency
|
2025-02-11 17:10:11 +04:00 |
|
|
Lizonghang
|
3dd3138207
|
ignore tensors already in page cache when prefetching
|
2025-02-11 17:00:17 +04:00 |
|
|
Lizonghang
|
24974a488c
|
assume 10% of active pages can be compressed on macOS UMA
|
2025-02-11 11:06:33 +04:00 |
|
|
Zonghang Li
|
261c88f058
|
skip tensors on CUDA in manage_graph_tensors
|
2025-02-11 09:49:17 +04:00 |
|
|
Zonghang Li
|
c4c6a642fc
|
manage_graph_tensors: fix segment prefetch
|
2025-02-08 22:44:38 +04:00 |
|
|
Lizonghang
|
d2bc5cd502
|
add pid as suffix to avoid conflicts with other processes
|
2025-02-07 10:29:22 +04:00 |
|
|
Lizonghang
|
8e41362af0
|
fix prefetch
|
2025-02-04 17:38:41 +04:00 |
|
|
Lizonghang
|
ec73e239c9
|
use 80% available mem as a conservative estimate
|
2025-02-03 18:10:05 +04:00 |
|
|
Lizonghang
|
64089236eb
|
fix latency estimation in set m1
|
2025-02-03 07:56:02 +04:00 |
|
|
Lizonghang
|
83b3d01844
|
fix delay estimation on macos
|
2025-02-01 10:37:56 +04:00 |
|
|
Lizonghang
|
215151918f
|
fix gpu mem limit
|
2025-01-31 18:52:13 +04:00 |
|
|
Lizonghang
|
17cd8ba618
|
reverse 300MiB for Metal kernel
|
2025-01-31 16:24:44 +04:00 |
|
|
Lizonghang
|
dd632ee6df
|
ignore the first 5 evals due to preheat
|
2025-01-31 08:53:51 +04:00 |
|
|
Lizonghang
|
fdecd4b54c
|
more active pages can be compressed
|
2025-01-30 23:17:07 +04:00 |
|
|
Lizonghang
|
2bc7a56790
|
fix available mem estimation in termux
|
2025-01-30 21:23:05 +04:00 |
|
|
Lizonghang
|
b680cb74fe
|
set POSIX_MADV_WILLNEED for the next subgraph
|
2025-01-30 13:29:34 +04:00 |
|
|
Lizonghang
|
f9b4c46b74
|
ignore the first eval to make time test more accurate
|
2025-01-30 11:12:26 +04:00 |
|
|
Lizonghang
|
cd758247e6
|
consider active pages compression in macos available memory estimation
|
2025-01-29 20:33:13 +04:00 |
|
|
Lizonghang
|
27c996835d
|
fix undeclared identifier get_page_size
|
2025-01-29 19:59:02 +04:00 |
|
|
Lizonghang
|
4b616baed4
|
fix macos x86_64 available mem estimation
|
2025-01-29 19:57:06 +04:00 |
|
|
Lizonghang
|
849b47ccd0
|
fix auto schedule logic
|
2025-01-29 13:13:37 +04:00 |
|
|
Lizonghang
|
e7c6b830e6
|
fix auto schedule logic
|
2025-01-29 11:15:45 +04:00 |
|
|
Lizonghang
|
631daadd92
|
test
|
2025-01-28 16:36:47 +04:00 |
|
|
Zonghang Li
|
36f353e374
|
check env path before calling fio to ensure we can find it
|
2025-01-28 13:06:08 +04:00 |
|
|
Zonghang Li
|
2d649c808d
|
add /usr/local/bin on darwin
|
2025-01-28 13:05:25 +04:00 |
|
|
Lizonghang
|
2934cf3e8e
|
reserve 200 mib for internal gpu usage
|
2025-01-27 22:14:12 +04:00 |
|
|
Lizonghang
|
1ca9e7974b
|
device_os returns Linux if in Termux
|
2025-01-27 11:14:21 +04:00 |
|
|
Lizonghang
|
1e2b934d69
|
add bounds n[m]<=0 for devices without GPUs
|
2025-01-27 11:13:09 +04:00 |
|
|
Lizonghang
|
ac5d63b09e
|
add explaination for why the output layer weights should be kept in metal shared memory
|
2025-01-25 23:51:16 +04:00 |
|
|
Lizonghang
|
1ca9a43bd1
|
keep the output layer weights in shared memory by default
|
2025-01-25 23:31:43 +04:00 |
|
|
Lizonghang
|
f3dd5776eb
|
fix kappa and memory bounds, account for look-up table and input/output layer delay
|
2025-01-25 22:31:40 +04:00 |
|
|
Lizonghang
|
9e4ba4f06a
|
fix w init error
|
2025-01-24 20:26:18 +04:00 |
|
|
Lizonghang
|
4948b1004c
|
fix device_name() to get device name from host, not from termux
|
2025-01-24 16:56:23 +04:00 |
|
|
Lizonghang
|
1c0087e919
|
rename arg --keep-inp-out-in-metal to --keep-out-in-metal
|
2025-01-23 23:17:06 +04:00 |
|
|
Lizonghang
|
5fcf020cfb
|
use the recommended working set size as metal mem bound, rather than the total physical mem
|
2025-01-23 23:04:40 +04:00 |
|
|
Lizonghang
|
fb05f80f89
|
remove token_embd from metal mem
|
2025-01-23 16:58:25 +04:00 |
|
|
Lizonghang
|
92f00303d5
|
set reserved mem to 100MiB
|
2025-01-23 16:58:11 +04:00 |
|
|
Lizonghang
|
38e4a3eaa0
|
add device_macos_swappable_memory
|
2025-01-23 16:09:04 +04:00 |
|
|
Lizonghang
|
78a544d716
|
add metal mem limit
|
2025-01-23 16:08:52 +04:00 |
|
|
Zonghang Li
|
33429ec4e1
|
add option --keep-inp-out-in-metal
|
2025-01-22 11:25:09 +04:00 |
|
|
Lizonghang
|
facb4ea736
|
add option --keep-inp-out-in-metal and fix bugs in unmap
|
2025-01-22 11:15:19 +04:00 |
|
|
Lizonghang
|
ce2ef9699f
|
fix mapping unmap_fragment error
|
2025-01-21 22:43:51 +04:00 |
|
|
Lizonghang
|
189ed92cba
|
segment mmap range on Metal shared memory to avoid memory waste
|
2025-01-21 21:07:02 +04:00 |
|
|
Lizonghang
|
871a27f66a
|
use user input setup in standalone mode
|
2025-01-21 21:06:09 +04:00 |
|