ce45587ea9
correct GB to GiB
2024-11-29 11:20:19 +04:00
0f73d12247
decrease compute buf from available memory
2024-11-29 11:15:54 +04:00
329d084061
add illustration for memory allocation of activations
2024-11-29 10:34:21 +04:00
45a1e55eec
reduce kv cache from available memory
2024-11-28 20:21:21 +04:00
9858d90ce4
get system readahead size automatically
2024-11-28 16:18:41 +04:00
9a7bbce7ad
fix t_load_us
2024-11-28 15:55:21 +04:00
740f7f0b95
use multithread disk r/w test
2024-11-27 22:14:17 +04:00
f7507ec20b
fix disk r/w test, add disk access latency, and correct units (GB, GiB)
2024-11-27 21:36:12 +04:00
9cd22177d0
remove arg test_file
2024-11-27 21:34:45 +04:00
0a91ad3edc
fix cuda compatibility errors
2024-11-26 22:35:58 +04:00
3f008f2ad9
add op GGML_OP_READ
2024-11-26 22:28:34 +04:00
f78c437172
add device_inp_embd_delay test, device_memory_bw test, device_cuda_memory_bw test,
2024-11-26 22:28:02 +04:00
a7a95b53fe
add q80xf32 and count_n_params
2024-11-24 23:11:12 +04:00
3fe00a16a0
count model flops for f32xf32, f16xf32, q4kxf32, q6kxf32
2024-11-24 13:13:32 +04:00
a5ba34169a
add f32, f16, q4k_f32, q6k_f32 flops test and fix duplicate inp_embd in subgraphs
2024-11-23 21:36:34 +04:00
7ee1423006
add model_flops
2024-11-21 20:06:16 +04:00
80f6b72e71
remove device_flops from profiler api
2024-11-21 08:37:57 +04:00
477ecf2084
add llama_model_n_flops
2024-11-20 19:40:27 +04:00
10f6f92c7e
add f32, f16, q8, q4k speed test for cuda
2024-11-10 23:41:13 +04:00
f4260bb346
add device_flops() for cpu, metal, and cuda
2024-11-10 23:11:05 +04:00
5fae6ac36f
add cpu flops test
2024-11-09 20:53:42 +04:00
2bd4d03aa8
add automatic layer window size assignment workflow
2024-11-08 18:21:03 +04:00
53cb3a6069
synchronize device info
2024-11-07 22:02:01 +04:00
ef7fdf70cc
add LLAMA_API llama_profile_device
2024-11-07 09:30:39 +04:00
b922418cca
convert MB to GB
2024-11-06 20:47:17 +04:00
407c71ae52
add cpu and gpu profile
2024-11-06 20:42:28 +04:00
4e1be1065d
add memory speed test
2024-11-06 10:57:30 +04:00
9a03b52785
fix device get name on linux
2024-11-05 22:07:09 +04:00
a7f3d917a1
add device get name
2024-11-05 22:04:14 +04:00
6657885816
fix swap detect on linux
2024-11-05 21:57:09 +04:00
2d447266e9
add swap capacity test
2024-11-05 21:42:45 +04:00
9eed6b14bf
add disk read speed test
2024-11-05 21:12:02 +04:00
9cd66f2145
add profiler
2024-11-05 20:29:09 +04:00
766ec7862b
test
2024-11-05 17:22:24 +04:00
976a4c3534
fix cuda support
2024-11-04 11:00:01 +04:00
9d6a6845ac
fix cuda support
2024-11-03 12:02:30 +04:00
684b2ac05b
fix cuda support
2024-11-03 11:19:50 +04:00
a83f577c63
reset backend_embd and out_embd as input tensors
2024-11-02 20:40:21 +04:00
7aa771e701
set backend_embd and out_embd on the device of their adjacent nodes
2024-11-02 09:41:40 +04:00
bfc3f9e185
split input as a separate subgraph
2024-10-30 23:40:13 +04:00
64291ad70c
fix next_sub_gf
2024-10-28 11:03:46 +04:00
925ea39382
finetune prefetch
2024-10-28 10:58:48 +04:00
76a7fc7527
support different window sizes
2024-10-26 12:34:14 +04:00
5685cb87ed
delete .github
2024-10-25 17:32:13 +04:00
c97ea10617
add mmap prefetch and unloading
2024-10-25 16:33:56 +04:00
ba5117581e
init
2024-10-23 14:29:14 +04:00
76d67387c0
init
2024-10-23 09:51:30 +04:00
2a01ff5fb1
init
2024-10-23 09:42:32 +04:00
6374743747
ggml : add backend registry / device interfaces to BLAS backend ( #9752 )
...
* ggml : add backend registry / device interfaces to BLAS backend
* fix mmap usage when using host buffers
2024-10-07 21:55:08 +02:00
f1af42fa8c
Update building for Android ( #9672 )
...
* docs : clarify building Android on Termux
* docs : update building Android on Termux
* docs : add cross-compiling for Android
* cmake : link dl explicitly for Android
2024-10-07 09:37:31 -07:00
Lizonghang