This website requires JavaScript.
2ac95c9d56
SimpleChat: Simple histogram/repeatMatching driven garbageTrimming, Settings UI, Streaming mode, OpenAi Compat (Model, Authorization Bearer), Save/Restore session, Auto Settings UI (#7548 )
2024-06-01 21:50:18 +05:30
750f60c03e
CUDA: fix Pascal FA, deq. KV to FP16 for batch > 8 (#7681 )
2024-06-01 15:47:04 +02:00
9b596417af
CUDA: quantized KV support for FA vec (#7527 )
2024-06-01 08:44:14 +02:00
a323ec60af
server : update js (#7670 )
2024-05-31 22:23:04 +03:00
0515ad93f4
convert-hf : Handle NotImplementedError in convert-hf-to-gguf (#7660 )
2024-05-31 17:42:33 +02:00
c8047d538f
scripts: update compare_llama_bench.py [no ci] (#7673 )
2024-05-31 16:26:21 +02:00
30e238b246
Improve HIP compatibility (#7672 )
2024-05-31 14:00:29 +00:00
16926dff92
readme : link homebrew discussion
2024-05-31 15:04:58 +03:00
0c27e6f62e
ggml : fix loongson compile warnings (#7537 )
2024-05-31 14:17:10 +03:00
2e32f874e6
Somehow '**' got lost (#7663 )
2024-05-31 10:24:41 +02:00
1af511fc22
Add convert.py removal to hot topics (#7662 )
2024-05-31 10:09:20 +02:00
0541f06296
[no ci] docs: add aikit to readme (#7650 )
2024-05-30 16:57:16 -07:00
9022c33646
Fixed painfully slow single process builds. (#7326 )
2024-05-30 21:32:38 +01:00
5921b8f089
llama : cache llama_token_to_piece (#7587 )
2024-05-30 19:01:41 +03:00
5dcdf94676
Fix conan badge display [no ci] (#7645 )
2024-05-30 17:07:39 +02:00
2e2340de17
Add brew installation instruction to README [no ci] (#7616 )
2024-05-30 16:58:15 +02:00
7846540bd2
readme : add Conan badge (#7638 )
2024-05-30 14:52:50 +02:00
e6157f94c8
github: add contact links to issues and convert question into research [no ci] (#7612 )
2024-05-30 21:55:36 +10:00
9c4c9cc83f
Move convert.py to examples/convert-legacy-llama.py (#7430 )
2024-05-30 13:40:00 +02:00
59b0d07766
faster avx512 exp implementation (#7551 )
2024-05-30 07:32:55 -04:00
d5c05821f3
ggml : fix loongarch build (O2 issue) (#7636 )
2024-05-30 17:30:10 +08:00
972b555ab9
README: explain parallel build [no ci] (#7618 )
2024-05-30 09:52:39 +02:00
3854c9d07f
[SYCL] fix intel docker (#7630 )
2024-05-30 14:19:08 +08:00
eb57fee51f
gguf-py : Add tokenizer.ggml.pre to gguf-new-metadata.py (#7627 )
2024-05-30 02:10:40 +02:00
55d62262a9
metal : remove invalid asserts (#7617 )
2024-05-29 22:20:40 +03:00
975ec63ff2
metal : add missing asserts (#7617 )
2024-05-29 20:45:25 +03:00
fb76ec31a9
ggml : fix YARN + add tests + add asserts (#7617 )
2024-05-29 20:17:31 +03:00
cce3dcffc5
cuda : non-cont concat support (#7610 )
2024-05-29 15:38:26 +03:00
210d99173d
llama-bench : add support for the RPC backend (#7435 )
2024-05-29 14:45:44 +03:00
87bdf2a199
ggml : use atomic_flag for critical section (#7598 )
2024-05-29 13:36:39 +02:00
00281b7be3
scripts : remove mpi remnants
2024-05-29 14:31:18 +03:00
2ab977282b
sync : ggml
2024-05-29 14:29:52 +03:00
72de268bec
ggml : restore ggml_rope_xpos_inplace (ggml/0)
2024-05-26 18:35:23 +03:00
0e8d8bfd6c
Add Arc A750 and Arch linux to readme-sycl.md as verified GPU model and Linux distro (#7605 )
2024-05-29 12:23:47 +05:30
504f0c340f
ggml : fix typo in ggml.c (#7603 )
2024-05-29 10:09:31 +08:00
b864b50ce5
[SYCL] Align GEMM dispatch (#7566 )
2024-05-29 07:00:24 +08:00
02c1ecad07
Tokenizer WPM fixes (#7500 )
2024-05-28 21:46:34 +02:00
6bd12ce409
sycl : fix assert (#7563 )
2024-05-28 22:22:50 +03:00
5442939fcc
llama : support small Granite models (#7481 )
2024-05-28 20:49:49 +02:00
56411a950f
vulkan: properly initialize vulkan devices for LLAMA_SPLIT_MODE_NONE (#7552 )
2024-05-29 01:25:08 +08:00
2b737caae1
rpc : resource management rework (#7562 )
2024-05-28 18:13:36 +03:00
ee3dff6b8e
Add support for DeepseekV2ForCausalLM (#7519 )
2024-05-28 17:07:05 +02:00
edc29433fa
tests : fix test-tokenizer-0.sh
2024-05-28 15:04:09 +03:00
8b99e2aa66
llama : handle unknown utf8 bytes (#7588 )
2024-05-28 13:55:35 +03:00
271ff3fc44
github: add refactor to issue template (#7561 )
2024-05-28 20:27:27 +10:00
e2b065071c
[SYCL]fix ggml_sycl_mul_mat_id() to match the change of api (#7436 )
2024-05-28 17:53:37 +08:00
0548a4187f
ggml : generalize GGML_OP_CONCAT (#7563 )
2024-05-28 11:04:19 +03:00
9335b969e8
server: do not remove whitespace at the start of a completion chunk (#7524 )
2024-05-28 06:55:51 +02:00
c41767154e
Markdownish code block fix (#7571 )
2024-05-28 00:41:14 -04:00
74b239b3d5
llava : update clip.h (#7580 )
2024-05-28 11:48:16 +09:00
852aafb163
update HIP_UMA #7399 (#7414 )
2024-05-28 01:40:47 +02:00
0136966daf
adding in x64 targets to cmake presets (#7574 )
2024-05-27 18:40:12 -05:00
10b1e45876
make: add --device-debug to NVCC debug flags (#7542 )
2024-05-27 19:34:40 +02:00
197c00681b
Allow multiple copy function pointers for CUDA graph kernel param updates (#7565 )
2024-05-27 18:33:42 +01:00
95f84d5ce8
Fix q_xxs using mul_mat_q (#7459 )
2024-05-27 17:34:51 +01:00
5487593bc7
Add freq factors (#7495 )
2024-05-27 13:34:09 +01:00
1d8fca72ae
metal : add GGML_OP_REPEAT kernels (#7557 )
2024-05-27 12:10:19 +03:00
62bfef5194
metal : disable FA kernel for HS=256 (#7556 )
2024-05-27 10:38:39 +03:00
eaf6e03174
llama : add comments about experimental flags (#7544 )
2024-05-27 09:24:13 +03:00
d6ef0e77dd
github: add self sorted issue ticket forms (#7543 )
2024-05-27 10:54:30 +10:00
dff451cfa1
flake.lock: Update (#7540 )
2024-05-26 18:54:56 +03:00
d298382ad9
main: replace --no-special with --special (#7534 )
2024-05-27 00:10:17 +10:00
32a28217f4
Fix aya-23 conversion scripts (#7539 )
2024-05-26 16:02:34 +02:00
c429b33beb
llama : add Smaug 70B support (#7402 )
2024-05-26 08:28:35 -04:00
9146d36fe7
Readme: add akx/ggify to tools (#1484 )
2024-05-26 15:09:42 +03:00
b9adcbbf92
SimpleChat Completion Mode flexibility and cleanup, Settings gMe, Optional sliding window (#7480 )
2024-05-26 06:26:34 +05:30
9588f196b1
train : change default FA argument (#7528 )
2024-05-25 15:21:30 +03:00
3cbd23ed88
labeler: added Apple Metal detector (+Kompute) (#7529 )
2024-05-25 19:30:42 +10:00
00c6390793
main : don't print special tokens with --grammar (#6923 )
2024-05-25 05:04:03 -04:00
faa0e6979a
ggml: aarch64: SVE kernels for q8_0_q8_0, q4_0_q8_0 vector dot (#7433 )
2024-05-25 17:42:31 +09:00
9791f40258
android : module (#7502 )
2024-05-25 04:11:33 -04:00
902184dd3a
fix missing slash in fs_get_cache_directory() (#7503 )
2024-05-25 05:30:59 +02:00
57684331fc
Make tokenize CLI tool have nicer command line arguments. (#6188 )
2024-05-24 18:14:42 -07:00
b83bab15a5
gguf-py : fix and simplify quantized shape round-trip (#7483 )
2024-05-24 21:11:48 -04:00
d041d2ceaa
flake.lock: Update (#7232 )
2024-05-24 18:59:06 +03:00
27891f6db0
docker.yml: disable light-intel and server-intel test (#7515 )
2024-05-24 23:47:56 +10:00
fbca2f27fc
Add support for ArcticForCausalLM (#7020 )
2024-05-24 14:31:13 +02:00
0df0aa8e43
add build shared lib in win release package (#7438 )
2024-05-24 10:06:56 +08:00
74f33adf5f
readme : remove trailing space (#7469 )
2024-05-23 17:43:18 +03:00
1debe72737
ggml : silence UB sanitizer error during iq2_xxs quantization (#0 )
2024-05-23 17:17:43 +03:00
007489e895
Fix phi3 chat template confusion with zephyr (#7449 )
2024-05-23 16:15:15 +02:00
8b94e799df
readme : add Bunny in supported models [no ci] (#7469 )
2024-05-23 18:00:13 +05:30
3015851c5a
llama : add getters for n_threads/n_threads_batch (#7464 )
2024-05-23 14:29:26 +02:00
55ac3b7aea
ci : use Pythia models instead of OpenLlama (#7470 )
2024-05-23 15:28:14 +03:00
dacfcebd60
readme : add GPT-NeoX + Pythia to the list of supported models (#7491 )
2024-05-23 15:12:43 +03:00
9b82476ee9
Add missing inference support for GPTNeoXForCausalLM (Pythia and GPT-NeoX base models) (#7461 )
2024-05-23 11:49:53 +02:00
a61a94e543
llama : rename n_ctx -> cache.size, less confusing (#0 )
2024-05-23 12:38:18 +03:00
152da28ae5
labeler.yml: add embedding label detector [no ci] (#7482 )
2024-05-23 17:40:43 +10:00
d48c88cbd5
ggml : remove ggml_flash_attn and ggml_flash_ff (#7463 )
2024-05-23 10:00:44 +03:00
e84b71c2c6
ggml : drop support for QK_K=64 (#7473 )
2024-05-23 10:00:21 +03:00
1b1e27cb49
Update vulkan rope implementation to support frequency factors (#7475 )
2024-05-23 08:59:59 +02:00
fbf777d2b9
main : minor (#7462 )
2024-05-23 09:43:24 +03:00
cd93a28cb1
CUDA: fix FA out-of-bounds reads (#7479 )
2024-05-23 00:31:20 +02:00
1e374365d1
SimpleChat: a simple and dumb web front end for testing /chat/completions and /completions end points and try chat (#7350 )
2024-05-22 23:23:21 +05:30
197ff91462
build : remove zig (#7471 )
2024-05-22 20:05:38 +03:00
6ff13987ad
common : normalize naming style (#7462 )
2024-05-22 20:04:20 +03:00
38c03478a3
CUDA: fix FA out-of-bounds writes (#7465 )
2024-05-22 17:58:25 +02:00
b18532a4ef
phi3 : duplicate rope factors in each layer (#7447 )
2024-05-22 16:10:46 +02:00
fcda1128bc
vulkan: add workaround for iterator boundary check to fix clang-cl debug build (#7426 )
2024-05-22 20:53:21 +08:00
03d8900ebe
llama : add missing model type names (#7445 )
2024-05-22 07:08:18 -04:00
HanishKVC