-
75a86e4eb1
use internal_mem - purgeable_mem to calculate the available_mem
Lizonghang
2024-12-04 10:24:54 +04:00
-
bd5e034746
set fio engine to sync if posixaio not loadable
Lizonghang
2024-12-04 00:55:50 +04:00
-
508c8ae83c
set fio engine to sync if posixaio not loadable
Lizonghang
2024-12-04 00:52:01 +04:00
-
c3d708862c
use disk read_rnd_bw by default
Lizonghang
2024-12-04 00:37:59 +04:00
-
7cd4936d58
rename to CPU RAM Read BW
Lizonghang
2024-12-04 00:04:22 +04:00
-
dc03b6216f
set default page size to 4KB if not available from system
Lizonghang
2024-12-03 23:25:47 +04:00
-
74dbec5086
set default readahead size to 128KB if permission is needed
Lizonghang
2024-12-03 22:55:16 +04:00
-
1a7237636e
add cgroup version detect
Zonghang Li
2024-11-30 09:48:19 +04:00
-
3074763ed4
fix memory detect in docker container
Zonghang Li
2024-11-30 09:36:01 +04:00
-
eb25858e87
remove cpus_allowed_policy in fio
Zonghang Li
2024-11-29 22:16:32 +04:00
-
7e4bb65eab
fix bugs on cpu
Zonghang Li
2024-11-29 22:06:02 +04:00
-
81fd77033e
add gpu support in device_memory_access_delay
Zonghang Li
2024-11-29 21:56:01 +04:00
-
6f54a12c7d
add gpu support in llama_model_kvcache_size and llama_model_compute_buf_size
Lizonghang
2024-11-29 21:06:32 +04:00
-
f8e9dc2713
add GPU support in device_compute_delay and device_disk_access_delay
Lizonghang
2024-11-29 20:21:22 +04:00
-
f8bcf38bfe
fix var name overwrite
Lizonghang
2024-11-29 20:00:17 +04:00
-
1123c00e45
recover comments in llama_profile_device
Lizonghang
2024-11-29 19:07:07 +04:00
-
68ecabc8c3
add cpu_read_ram_bw, metal_read_vram_bw, cuda_read_vram_bw
Lizonghang
2024-11-29 19:03:01 +04:00
-
0a6ffe68e0
add kernel kernel_read_vram
Lizonghang
2024-11-29 17:15:36 +04:00
-
639a33dfa5
add op GGML_OP_READ and GGML_METAL_KERNEL_TYPE_READ_VRAM
Lizonghang
2024-11-29 17:15:19 +04:00
-
6934d52d8a
graph->nodes[i] => node
Lizonghang
2024-11-29 17:14:25 +04:00
-
ce45587ea9
correct GB to GiB
Lizonghang
2024-11-29 11:20:19 +04:00
-
0f73d12247
decrease compute buf from available memory
Lizonghang
2024-11-29 11:15:54 +04:00
-
329d084061
add illustration for memory allocation of activations
Lizonghang
2024-11-29 10:34:21 +04:00
-
45a1e55eec
reduce kv cache from available memory
Lizonghang
2024-11-28 20:21:21 +04:00
-
9858d90ce4
get system readahead size automatically
Lizonghang
2024-11-28 16:18:41 +04:00
-
9a7bbce7ad
fix t_load_us
Lizonghang
2024-11-28 15:55:21 +04:00
-
740f7f0b95
use multithread disk r/w test
Lizonghang
2024-11-27 22:14:17 +04:00
-
f7507ec20b
fix disk r/w test, add disk access latency, and correct units (GB, GiB)
Lizonghang
2024-11-27 21:36:12 +04:00
-
9cd22177d0
remove arg test_file
Lizonghang
2024-11-27 21:34:45 +04:00
-
0a91ad3edc
fix cuda compatibility errors
Lizonghang
2024-11-26 22:35:58 +04:00
-
3f008f2ad9
add op GGML_OP_READ
Zonghang Li
2024-11-26 22:28:34 +04:00
-
f78c437172
add device_inp_embd_delay test, device_memory_bw test, device_cuda_memory_bw test,
Zonghang Li
2024-11-26 22:28:02 +04:00
-
a7a95b53fe
add q80xf32 and count_n_params
Lizonghang
2024-11-24 23:11:12 +04:00
-
3fe00a16a0
count model flops for f32xf32, f16xf32, q4kxf32, q6kxf32
Lizonghang
2024-11-24 13:13:32 +04:00
-
a5ba34169a
add f32, f16, q4k_f32, q6k_f32 flops test and fix duplicate inp_embd in subgraphs
Lizonghang
2024-11-23 21:36:34 +04:00
-
7ee1423006
add model_flops
Zonghang Li
2024-11-21 20:06:16 +04:00
-
80f6b72e71
remove device_flops from profiler api
Zonghang Li
2024-11-21 08:37:57 +04:00
-
477ecf2084
add llama_model_n_flops
Lizonghang
2024-11-20 19:40:27 +04:00
-
10f6f92c7e
add f32, f16, q8, q4k speed test for cuda
Lizonghang
2024-11-10 23:41:13 +04:00
-
f4260bb346
add device_flops() for cpu, metal, and cuda
Lizonghang
2024-11-10 23:11:05 +04:00
-
5fae6ac36f
add cpu flops test
Lizonghang
2024-11-09 20:53:42 +04:00
-
2bd4d03aa8
add automatic layer window size assignment workflow
Lizonghang
2024-11-08 18:21:03 +04:00
-
53cb3a6069
synchronize device info
Lizonghang
2024-11-07 22:02:01 +04:00
-
ef7fdf70cc
add LLAMA_API llama_profile_device
Lizonghang
2024-11-07 09:30:39 +04:00
-
b922418cca
convert MB to GB
Zonghang Li
2024-11-06 20:47:17 +04:00
-
407c71ae52
add cpu and gpu profile
Lizonghang
2024-11-06 20:42:28 +04:00
-
4e1be1065d
add memory speed test
Lizonghang
2024-11-06 10:57:30 +04:00
-
9a03b52785
fix device get name on linux
Zonghang Li
2024-11-05 22:07:09 +04:00
-
a7f3d917a1
add device get name
Lizonghang
2024-11-05 22:04:14 +04:00
-
6657885816
fix swap detect on linux
Zonghang Li
2024-11-05 21:57:09 +04:00
-
2d447266e9
add swap capacity test
Lizonghang
2024-11-05 21:42:45 +04:00
-
9eed6b14bf
add disk read speed test
Lizonghang
2024-11-05 21:12:02 +04:00
-
9cd66f2145
add profiler
Lizonghang
2024-11-05 20:29:09 +04:00
-
766ec7862b
test
Lizonghang
2024-11-05 17:22:24 +04:00
-
976a4c3534
fix cuda support
Lizonghang
2024-11-04 11:00:01 +04:00
-
9d6a6845ac
fix cuda support
Lizonghang
2024-11-03 12:02:30 +04:00
-
684b2ac05b
fix cuda support
Lizonghang
2024-11-03 11:19:50 +04:00
-
a83f577c63
reset backend_embd and out_embd as input tensors
Lizonghang
2024-11-02 20:40:21 +04:00
-
7aa771e701
set backend_embd and out_embd on the device of their adjacent nodes
Lizonghang
2024-11-02 09:41:40 +04:00
-
bfc3f9e185
split input as a separate subgraph
Lizonghang
2024-10-30 23:40:13 +04:00
-
64291ad70c
fix next_sub_gf
Lizonghang
2024-10-28 11:03:46 +04:00
-
925ea39382
finetune prefetch
Lizonghang
2024-10-28 10:58:48 +04:00
-
76a7fc7527
support different window sizes
Lizonghang
2024-10-26 12:34:14 +04:00
-
5685cb87ed
delete .github
Lizonghang
2024-10-25 17:32:13 +04:00
-
c97ea10617
add mmap prefetch and unloading
Lizonghang
2024-10-25 16:33:56 +04:00
-
ba5117581e
init
Lizonghang
2024-10-23 14:29:14 +04:00
-
76d67387c0
init
Lizonghang
2024-10-23 09:51:30 +04:00
-
2a01ff5fb1
init
Lizonghang
2024-10-23 09:42:32 +04:00
-
6374743747
ggml : add backend registry / device interfaces to BLAS backend (#9752)
Diego Devesa
2024-10-07 21:55:08 +02:00
-
f1af42fa8c
Update building for Android (#9672)
Andrew Minh Nguyen
2024-10-07 09:37:31 -07:00
-
6279dac039
flake.lock: Update (#9753)
Georgi Gerganov
2024-10-07 19:35:42 +03:00
-
d5ac8cf2f2
ggml : add metal backend registry / device (#9713)
Georgi Gerganov
2024-10-07 18:27:51 +03:00
-
96b6912103
metal : single allocation of encode_async block (#9747)
Paul Tsochantaris
2024-10-07 13:26:31 +01:00
-
d5cb86844f
contrib : simplify + minor edits [no ci]
Georgi Gerganov
2024-10-06 14:15:27 +03:00
-
f4b2dcdf49
readme : fix typo [no ci]
Georgi Gerganov
2024-10-06 13:49:41 +03:00
-
b6d6c5289f
sync : llama.cpp
Georgi Gerganov
2024-10-06 12:53:28 +03:00
-
b0915d5b51
vulkan : retry allocation with fallback flags (whisper/2451)
SRHMorris
2024-10-06 08:34:20 +01:00
-
8c475b97b8
rerank : use [SEP] token instead of [BOS] (#9737)
Georgi Gerganov
2024-10-05 15:55:04 +03:00
-
58b16695e1
sync : ggml
Georgi Gerganov
2024-10-05 15:53:49 +03:00
-
905f5485b2
metal : zero-init buffer contexts (whisper/0)
Georgi Gerganov
2024-10-05 14:33:54 +03:00
-
71967c2a6d
Add Llama Assistant (#9744)
Viet-Anh NGUYEN (Andrew)
2024-10-05 01:29:35 +07:00
-
17880771ad
sync : ggml
Georgi Gerganov
2024-10-04 18:50:25 +03:00
-
55951c018d
ggml : fix typo in example usage ggml_gallocr_new (ggml/984)
Daniel Bevenius
2024-10-04 15:46:18 +02:00
-
ff565769f2
ggml : fixes after sync (ggml/983)
Diego Devesa
2024-10-04 08:41:40 +02:00
-
f3fdcfaa79
ci : fine-grant permission (#9710)
Xuan Son Nguyen
2024-10-04 11:47:19 +02:00
-
133c7b46b3
Fixed RNG seed docs (#9723)
Daniel Kleine
2024-10-04 10:54:44 +02:00
-
d5ed2b929d
metal : remove abort (skip) (ggml/0)
Georgi Gerganov
2024-10-03 21:18:19 +03:00
-
1bb8a64ebf
sync : ggml
Georgi Gerganov
2024-10-03 21:17:49 +03:00
-
fabdc3bda3
ggml/ex: calculate accuracy in graph, adapt MNIST (ggml/980)
Johannes Gäßler
2024-10-03 17:29:59 +02:00
-
eee39bdc96
ggml: refactor cross entropy loss CPU impl. (ggml/976)
Johannes Gäßler
2024-10-02 15:32:39 +02:00
-
5d5ab1e5cc
metal : fix compute pass descriptor autorelease crash (#9718)
Jack Mousseau
2024-10-03 11:01:46 -07:00
-
a7ad553513
ggml-backend : add device description to CPU backend (#9720)
Diego Devesa
2024-10-03 17:39:18 +02:00
-
d6fe7abf04
ggml: unify backend logging mechanism (#9709)
bandoti
2024-10-03 12:39:03 -03:00
-
e3c355ba65
convert : handle tokenizer merges format from transformers 4.45 (#9696)
compilade
2024-10-03 10:22:15 -04:00
-
841713e1e4
rpc : enable vulkan (#9714)
Radoslav Gerganov
2024-10-03 13:00:52 +03:00
-
5639971466
Fixed dequant precision issues in Q4_1 and Q5_1 (#9711)
Ouadie EL FAROUKI
2024-10-03 07:50:44 +01:00
-
c83ad6d01e
ggml-backend : add device and backend reg interfaces (#9707)
Diego Devesa
2024-10-03 01:49:47 +02:00
-
a39ab216aa
llama : reduce compile time and binary size (#9712)
Xuan Son Nguyen
2024-10-02 15:49:55 +02:00
-
f536f4c439
[SYCL] Initial cmake support of SYCL for AMD GPUs (#9658)
Alberto Cabrera Pérez
2024-10-02 13:57:18 +01:00
-
00b7317e63
vulkan : do not use tensor->extra (#9407)
Radoslav Gerganov
2024-10-02 13:49:16 +03:00