Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• 27c996835d fix undeclared identifier get_page_size Lizonghang 2025-01-29 19:59:02 +04:00
• 4b616baed4 fix macos x86_64 available mem estimation Lizonghang 2025-01-29 19:57:06 +04:00
• 849b47ccd0 fix auto schedule logic Lizonghang 2025-01-29 13:13:37 +04:00
• e7c6b830e6 fix auto schedule logic Lizonghang 2025-01-29 11:15:45 +04:00
• 631daadd92 test Lizonghang 2025-01-28 16:36:47 +04:00
• 36f353e374 check env path before calling fio to ensure we can find it Zonghang Li 2025-01-28 13:06:08 +04:00
• 2d649c808d add /usr/local/bin on darwin Zonghang Li 2025-01-28 13:05:25 +04:00
• 2934cf3e8e reserve 200 mib for internal gpu usage Lizonghang 2025-01-27 22:14:12 +04:00
• 1ca9e7974b device_os returns Linux if in Termux Lizonghang 2025-01-27 11:14:21 +04:00
• 1e2b934d69 add bounds n[m]<=0 for devices without GPUs Lizonghang 2025-01-27 11:13:09 +04:00
• ac5d63b09e add explaination for why the output layer weights should be kept in metal shared memory Lizonghang 2025-01-25 23:51:16 +04:00
• 1ca9a43bd1 keep the output layer weights in shared memory by default Lizonghang 2025-01-25 23:31:43 +04:00
• f3dd5776eb fix kappa and memory bounds, account for look-up table and input/output layer delay Lizonghang 2025-01-25 22:31:40 +04:00
• 9e4ba4f06a fix w init error Lizonghang 2025-01-24 20:26:18 +04:00
• 4948b1004c fix device_name() to get device name from host, not from termux Lizonghang 2025-01-24 16:56:23 +04:00
• 1c0087e919 rename arg --keep-inp-out-in-metal to --keep-out-in-metal Lizonghang 2025-01-23 23:17:06 +04:00
• 5fcf020cfb use the recommended working set size as metal mem bound, rather than the total physical mem Lizonghang 2025-01-23 23:04:40 +04:00
• fb05f80f89 remove token_embd from metal mem Lizonghang 2025-01-23 16:58:25 +04:00
• 92f00303d5 set reserved mem to 100MiB Lizonghang 2025-01-23 16:58:11 +04:00
• 38e4a3eaa0 add device_macos_swappable_memory Lizonghang 2025-01-23 16:09:04 +04:00
• 78a544d716 add metal mem limit Lizonghang 2025-01-23 16:08:52 +04:00
• 33429ec4e1 add option --keep-inp-out-in-metal Zonghang Li 2025-01-22 11:25:09 +04:00
• facb4ea736 add option --keep-inp-out-in-metal and fix bugs in unmap Lizonghang 2025-01-22 11:15:19 +04:00
• ce2ef9699f fix mapping unmap_fragment error Lizonghang 2025-01-21 22:43:51 +04:00
• 189ed92cba segment mmap range on Metal shared memory to avoid memory waste Lizonghang 2025-01-21 21:07:02 +04:00
• 871a27f66a use user input setup in standalone mode Lizonghang 2025-01-21 21:06:09 +04:00
• 0b748060ad ignore variable unused warning Lizonghang 2025-01-21 21:04:32 +04:00
• c701340719 set minimal disk speed to 500Mbps Lizonghang 2025-01-19 10:09:27 +04:00
• 4f5265a78d fix beta calculation on devices without GPU Lizonghang 2025-01-18 21:32:18 +04:00
• c19891f7db fix device os detect Lizonghang 2025-01-18 19:56:43 +04:00
• 6761ca5358 show device props Lizonghang 2025-01-18 17:25:27 +04:00
• f9d16fbf71 fix manully set layer window and gpu layers Lizonghang 2025-01-18 16:44:26 +04:00
• 70bd9db008 fix kappa Lizonghang 2025-01-18 11:51:18 +04:00
• e7fae2acdb fix cuda mem limitation Lizonghang 2025-01-16 09:48:08 +04:00
• c634c2fcbe reformat highs log Lizonghang 2025-01-16 09:38:12 +04:00
• 94f01993fa reformat highs log Lizonghang 2025-01-16 09:27:53 +04:00
• 46e99218b4 add arg --cuda-mem Zonghang Li 2025-01-16 09:15:34 +04:00
• dab6b2e1c2 fix option USE_HIGHS Lizonghang 2025-01-15 20:50:36 +04:00
• 18c96e8042 add option USE_HIGHS Lizonghang 2025-01-15 20:05:49 +04:00
• 790f702d0c report k and W Zonghang Li 2025-01-15 16:55:21 +04:00
• 9bead4291c fix path not found Zonghang Li 2025-01-15 16:55:07 +04:00
• 106ff79f08 include conda path only on Linux (if installing highs on Linux via conda) Lizonghang 2025-01-15 13:02:52 +04:00
• 73ed4dec68 add conda path to find highs Zonghang Li 2025-01-15 12:16:38 +04:00
• 3d75b8576e add api llama_model_set_n_gpu_layers Lizonghang 2025-01-15 10:48:19 +04:00
• 1e1ba5bb91 add api llama_model_set_n_gpu_layers Lizonghang 2025-01-15 10:47:53 +04:00
• 9279a2e3ff fix error in llama_context_n_gpu_layers Lizonghang 2025-01-15 10:08:41 +04:00
• 5d9aadf3d5 use highs to solve the allocation program Lizonghang 2025-01-15 10:04:04 +04:00
• b577c10d25 add noreturn flag for monitor Lizonghang 2025-01-07 20:38:11 +04:00
• 0b9720c222 modify default max nodes back to 8192 Zonghang Li 2025-01-04 19:18:59 +04:00
• e543fcd55f fix buf_compute_meta size Zonghang Li 2025-01-04 19:16:36 +04:00
• b2f806a572 increase default max nodes from 8192 to 16384 Zonghang Li 2025-01-04 11:08:35 +04:00
• b712ad3044 fix flops test error Zonghang Li 2024-12-31 09:37:46 +04:00
• 0a5450487c fix serialize error Zonghang Li 2024-12-31 08:24:31 +04:00
• e7c75d6b4a fix get device os error on linux Zonghang Li 2024-12-30 21:03:51 +04:00
• fa31ca8e35 add os detect Lizonghang 2024-12-30 09:13:12 +04:00
• d9beb030ee add EPS in device_compute_delay Lizonghang 2024-12-29 22:31:45 +04:00
• fa210d2034 remove duplicate calls Lizonghang 2024-12-29 22:00:17 +04:00
• a7ec685eda add memcpy speed test Lizonghang 2024-12-29 16:19:08 +04:00
• 5b46c4e848 correct GB to GiB Lizonghang 2024-12-28 10:09:43 +04:00
• b2bc836423 reformat Lizonghang 2024-12-28 08:36:24 +04:00
• 70811d85b3 remove OOM warning Lizonghang 2024-12-22 11:01:45 +04:00
• b6fd762fa8 fix log Lizonghang 2024-12-21 19:01:00 +04:00
• aa6c352aa2 reverse 128MiB free mem in termux Lizonghang 2024-12-12 21:27:36 +04:00
• 1b169f8f37 remove anonpages for repeat count Lizonghang 2024-12-12 20:28:59 +04:00
• 6ea692d974 use small tensors for test on cpu Zonghang Li 2024-12-12 15:52:27 +04:00
• b642d70188 fix swappable mem in termux Lizonghang 2024-12-12 15:15:16 +04:00
• fa05a482f8 fix swapout and mmapin delay Lizonghang 2024-12-12 09:58:10 +04:00
• 8c491c0fb1 consider swappable mem and swap space in termux Lizonghang 2024-12-11 21:24:37 +04:00
• 28ebc6478a use disk.read_rnd.bw in termux Lizonghang 2024-12-11 14:56:08 +04:00
• dfb37671d3 fix memory count and use sequantial disk read speed on linux Lizonghang 2024-12-11 09:30:02 +04:00
• bef2662d42 fix infinite loop in fio test Lizonghang 2024-12-10 20:58:17 +04:00
• 8e9ab45458 fix model bytes counter Lizonghang 2024-12-10 14:57:48 +04:00
• 2d79554694 fix device_physical_memory Zonghang Li 2024-12-09 14:42:49 +04:00
• 4104eb5b7a fix std::max val type Zonghang Li 2024-12-09 13:36:53 +04:00
• a11e0dd0fa fix device_cgroup_physical_memory in docker container Zonghang Li 2024-12-09 13:34:01 +04:00
• d78fa427e7 add memory copy speed test Lizonghang 2024-12-09 10:07:42 +04:00
• 1aee5bd6da hide metal log if not debug Lizonghang 2024-12-09 09:17:49 +04:00
• cead0f2e5a device_disk_access_delay: adapt for linux (on termux) Lizonghang 2024-12-08 22:58:08 +04:00
• 2c2171cebf fix display Lizonghang 2024-12-08 22:57:12 +04:00
• aad604ec8b add disk access delay while swapping Lizonghang 2024-12-08 20:07:30 +04:00
• f5243571d1 use sequential vram read test Zonghang Li 2024-12-08 10:14:52 +04:00
• d2dc8fd3b8 set CUDA_READ_BLOCK_SIZE to 512 Zonghang Li 2024-12-08 10:14:20 +04:00
• df813675d0 fix flops count and ram/vram speed test Zonghang Li 2024-12-08 10:14:05 +04:00
• 26c2ffb5b7 set fio read-job startdelay 1.5s Lizonghang 2024-12-06 17:03:42 +04:00
• a79ab3dde3 device_disk_access_delay: fix delay calculation when physical memory is busy (all mapped tensors will be reloaded) Lizonghang 2024-12-06 16:41:55 +04:00
• cd823546dd llama_profile_device: add arg n_predict Lizonghang 2024-12-06 16:37:25 +04:00
• a46d56cc60 llama_model_n_flops: remove ctxs Lizonghang 2024-12-06 11:31:53 +04:00
• f1c1d1b929 add support for Q5_K and fix byte count for Q6_K Lizonghang 2024-12-06 07:59:45 +04:00
• e6f4c009ab device_disk_access_delay: add delay for loading one row from lookup table Lizonghang 2024-12-05 20:38:28 +04:00
• 4fd3b6679e fix n_gpu_layers Lizonghang 2024-12-05 20:28:19 +04:00
• 3c8dbd11fc device_compute_delay: compatible with the shared memory in Metal Lizonghang 2024-12-05 20:02:13 +04:00
• e74967c488 throw error when using async upload, will support later Lizonghang 2024-12-05 14:18:39 +04:00
• 5a99c5894d update test model, enable warm-up and sched Zonghang Li 2024-12-05 09:52:16 +04:00
• b9538f048a add sched in device_flops but disabled by default Lizonghang 2024-12-04 23:54:57 +04:00
• 7521e532c4 device_memory_bw: simulate cache-friendly block access and multi-threading Zonghang Li 2024-12-04 15:36:59 +04:00
• 44b4718c8b recover device_memory_access_delay Zonghang Li 2024-12-04 12:29:24 +04:00
• 89aad7880e fix sysctlbyname on Linux and Win Zonghang Li 2024-12-04 12:24:07 +04:00
• 585864b05a add inactive_memory in Mac UMA Lizonghang 2024-12-04 11:56:09 +04:00
• 99d48157a0 compatible with Mac ARM64 and x86_64 arch, and ignore memory access delay Lizonghang 2024-12-04 11:33:46 +04:00
• 6d7801de87 fix available_memory calculation Zonghang Li 2024-12-04 10:36:43 +04:00