This website requires JavaScript.
e4caa8da59
ci : run when changing only the CUDA sources (#1800 )
2023-06-12 19:12:47 +02:00
58970a4c39
Leverage mmap for offloading tensors to GPU (#1597 )
2023-06-12 20:44:16 +08:00
8c0a10e64d
metal : fix failure to load model (#1817 )
2023-06-12 14:31:36 +03:00
fa84c4b3e8
Fix issue where interactive mode crashes when input exceeds ctx size (#1789 )
2023-06-11 08:19:17 -06:00
12b063f0ec
Fixed WSL cuda's OOM error (#1594 )
2023-06-11 21:20:52 +08:00
31d2b5f4a4
Update SHA256SUMS with current hashes for models quantized using q4_0 (#1798 )
2023-06-11 17:38:53 +08:00
4de0334f5c
cmake : fix Metal build (close #1791 )
2023-06-10 22:56:53 +03:00
3f1223155a
k-quants : GCC12 compilation fix (#1792 )
2023-06-10 22:51:36 +03:00
303f5809f1
metal : fix issue with ggml-metal.metal path. Closes #1769 (#1782 )
2023-06-10 10:47:34 -04:00
059e99066d
doc : fix wrong address of BLIS.md (#1772 )
2023-06-11 00:08:11 +10:00
17c10acfb4
ggml : force no_alloc == false when creating opt tensors (close #1699 )
2023-06-10 12:06:45 +03:00
e9b66ee982
metal : add Q4_1 implementation (#1785 )
2023-06-10 11:28:11 +03:00
4f0154b0ba
llama : support requantizing models instead of only allowing quantization from 16/32bit (#1691 )
2023-06-10 01:59:17 -06:00
ef3171d162
ggml : workaround for missing _mm256_setr_m128i in GCC < 8 (#1638 )
2023-06-10 15:49:40 +08:00
555275a693
make : add SSSE3 compilation use case (#1659 )
2023-06-10 14:41:59 +08:00
98ed165574
OpenCL: Add release memory (#1741 )
2023-06-10 01:24:40 +09:00
ae9663f188
Windows nvcc workaround (#1753 )
2023-06-09 13:58:15 +02:00
b33dee282f
metal : fix build "tanhf" -> "tanh"
2023-06-09 11:11:04 +03:00
92f44ff7f7
metal : add GELU implementation (#1770 )
2023-06-09 04:00:51 -04:00
245fc3c37d
metal : faster q4_0 (#1775 )
2023-06-09 10:39:59 +03:00
72ff5282bf
metal : add Q2_K implementation (#1762 )
2023-06-08 22:28:21 +03:00
0bf7cf1b29
Revert "ggml : load data into int8x16x4_t using vld4q_s8 on arm64 (#1738 )"
2023-06-08 20:48:14 +03:00
8432d4d9f7
ggml : load data into int8x16x4_t using vld4q_s8 on arm64 (#1738 )
2023-06-09 00:47:56 +08:00
0f291e1f65
metal : Q6_K implementation (#1752 )
2023-06-08 19:46:22 +03:00
8fc8179919
Add llama.cpp docker support for non-latin languages (#1673 )
2023-06-08 15:58:53 +08:00
b50b570ed9
ggml : fix fprintf warnings (#1720 )
2023-06-08 00:12:28 -07:00
53aba3f393
clang-tidy : restore dot file from accidental deletion
2023-06-08 10:09:08 +03:00
4161bdc04d
metal : add Q4_K implementation (#1733 )
2023-06-08 10:08:23 +03:00
0035858273
k-quants : add missing compile definition to CMakeLists (#1748 )
2023-06-08 08:02:48 +01:00
5c64a0952e
k-quants : allow to optionally disable at compile time (#1734 )
2023-06-07 10:59:52 +03:00
5b57a5b726
flake : update to support metal on m1/m2 (#1724 )
2023-06-07 00:15:31 -04:00
4dc62c545d
readme : add June roadmap
2023-06-07 07:15:08 +03:00
35a84916fb
main: add the possibility to open the prompt cache read-only (#1640 )
2023-06-07 04:10:17 +02:00
2d7bf110ed
llama : fix vram_scratch var
2023-06-06 22:54:39 +03:00
2a4e41a086
llama : fix compile warnings
2023-06-06 22:41:53 +03:00
17366df842
Multi GPU support, CUDA refactor, CUDA scratch buffer (#1703 )
2023-06-06 21:33:23 +02:00
44f906e853
metal : add f16 support
2023-06-06 20:16:57 +03:00
d5b111f53d
Clblast fixes + enhancements to save VRAM and offload more layers (#1675 )
2023-06-07 01:00:01 +08:00
2d43387daf
ggml : fix builds, add ggml-quants-k.o (close #1712 , close #1710 )
2023-06-06 10:18:03 +03:00
7ad7750c5c
gitignore : add .clang-tidy
2023-06-06 09:55:10 +03:00
7a74dee6b4
llama : temporary disable Q6_K output quantization (#1711 )
2023-06-06 09:39:38 +03:00
590250f7a9
metal : add checks for buffer size (#1706 )
2023-06-05 23:28:17 -04:00
f4c55d3bd7
docs : add performance troubleshoot + example benchmark documentation (#1674 )
2023-06-05 23:32:36 +03:00
f1465624c2
readme : fix typo (#1700 )
2023-06-05 22:28:37 +02:00
c2df36d60d
llama : consistently catch and throw only exceptions deriving from std::exception (#1599 )
2023-06-05 22:24:29 +02:00
9d0693bce3
metal : use shared buffers between CPU and GPU (#1696 )
2023-06-05 13:24:04 -07:00
efe0507632
ggml : fix internal overflow in ggml_time_us on Windows (#1702 )
2023-06-05 22:11:49 +02:00
e7fe66e670
ci : disable auto tidy (#1705 )
2023-06-05 23:05:05 +03:00
99009e72f8
ggml : add SOTA 2,3,4,5,6 bit k-quantizations (#1684 )
2023-06-05 22:56:18 +03:00
5220a991a5
Increase 3B scratch buffers. (#1698 )
2023-06-05 13:43:08 +03:00
d1f563a743
llama : fix Metal KV cache sync (close #1695 )
2023-06-05 10:19:03 +03:00
827f5eda91
readme : update hot topics
2023-06-04 23:38:19 +03:00
ecb217db4f
llama : Metal inference (#1642 )
2023-06-04 23:34:30 +03:00
dcb2ed4826
OpenCL: Fix duplication of layers in VRAM and RAM, add GPU mul kernel (#1653 )
2023-06-04 08:12:05 +02:00
d8bd0013e8
Add info about CUDA_VISIBLE_DEVICES (#1682 )
2023-06-03 16:35:20 +03:00
b5c85468a3
Docker: change to calling convert.py (#1641 )
2023-06-03 14:11:53 +02:00
136476e898
Fix prompt cache saving and chat-persistent rollover (#1678 )
2023-06-03 07:28:45 -04:00
ffb06a345e
OpenLLaMA 3B support (#1588 )
2023-05-30 21:24:22 +03:00
7552ac5863
ggml : sync cgraph import / export API
2023-05-29 19:31:44 +03:00
5d1830b99d
ggml : fix bug in ggml_alibi
2023-05-29 19:30:49 +03:00
248367605e
Work around for recalculating logits in cached prompts (Fixes #1585 ) (#1609 )
2023-05-29 05:13:40 -07:00
0e730dd23b
Adding git in container package dependencies (#1621 )
2023-05-29 06:45:50 +02:00
3b126f654f
LLAMA_DEBUG adds debug symbols (#1617 )
2023-05-28 21:01:02 +02:00
1b78ed2081
Only show -ngl option when relevant + other doc/arg handling updates (#1625 )
2023-05-28 11:48:57 -06:00
337aea1139
examples : add --alias option to gpt_params to set use friendly model name (#1614 )
2023-05-28 20:14:24 +03:00
bb051d9723
opencl : no need to allocate cl_mem on heap (#1612 )
2023-05-29 01:13:36 +08:00
ca74884f66
opencl : use strstr to check if fp16 supported (#1611 )
2023-05-29 01:09:56 +08:00
a6704643b6
ggml : add support for the RISCV architecture (#1616 )
2023-05-27 21:03:25 +01:00
0df7d63e5b
Include server in releases + other build system cleanups (#1610 )
2023-05-27 11:04:14 -06:00
97c9b77c4f
Add documentation about CLBlast (#1604 )
2023-05-27 18:47:55 +03:00
0ecb1bbbeb
[CI] Fix openblas (#1613 )
2023-05-27 17:24:06 +03:00
93618031c7
ggml : add ggml_tensor_overhead()
2023-05-27 16:19:56 +03:00
83c54e6da5
[CI] CLBlast: Fix directory name (#1606 )
2023-05-27 15:18:25 +03:00
bdbda1b17a
ggml : sync ggml core (minor additions, e.g. ggml_get_tensor_by_name())
2023-05-27 12:22:05 +03:00
66874d4fbc
Some improvements to loading the session with --prompt-cache (#1550 )
2023-05-25 20:18:01 -06:00
1fcdcc28b1
cuda : performance optimizations (#1530 )
2023-05-25 23:07:29 +02:00
ac7876ac20
Update CLBlast to 1.6.0 (#1580 )
2023-05-24 10:30:09 +03:00
c31bbe934b
readme : add docs for chat-persistent.sh (#1568 )
2023-05-24 02:24:01 -04:00
1359b6aba5
chat-persistent.sh : use bracket expressions in grep (#1564 )
2023-05-24 06:16:22 +00:00
7d873811f3
Fix handling of "invalid property" when creating OpenCL command queue (#1565 )
2023-05-23 18:01:15 +02:00
2e6cd4b025
OpenCL Token Generation Acceleration (#1459 )
2023-05-22 23:33:24 +02:00
7e4ea5beff
examples : add server example with REST API (#1443 )
2023-05-21 11:51:18 -06:00
7780e4f479
make : .PHONY clean (#1553 )
2023-05-21 16:03:44 +02:00
265db9834e
ggml : output 3d sizes in ggml_graph_dump_dot()
2023-05-21 11:56:23 +03:00
fab49c685e
ggml : update WASM SIMD
2023-05-20 20:00:41 +03:00
b8ee340abe
feature : support blis and other blas implementation (#1536 )
2023-05-20 23:58:31 +09:00
9ecb30f959
OpenCL: Fixes for older devices. (#1435 )
2023-05-20 17:57:39 +03:00
29cf5596fe
llama : define magic numbers as integer constants (#1518 ) (#1520 )
2023-05-20 15:58:15 +03:00
3de84b2606
ggml : add ggml_clamp() (#1539 )
2023-05-20 15:34:45 +03:00
affc76edfd
cuda : loading models directly into VRAM, norm calculation on GPU, broadcasting for ggml_mul (#1483 )
2023-05-20 14:19:28 +02:00
ea600071cb
Revert "feature : add blis and other BLAS implementation support (#1502 )"
2023-05-20 12:03:48 +03:00
07e9ace0f9
feature : add blis and other BLAS implementation support (#1502 )
2023-05-20 18:02:48 +09:00
ec2e10c444
llama : add llama_init_backend() API (close #1527 )
2023-05-20 11:06:11 +03:00
d2c59b8ba4
Fix for mingw (#1462 )
2023-05-20 00:40:02 -07:00
503db28849
llama : fix name shadowing and C4146 (#1526 )
2023-05-20 09:22:37 +02:00
8a203f9fa1
llama : fix compile warnings in llama_set_state_data()
2023-05-20 10:14:31 +03:00
4fd3e29297
ggml : fix scalar implementation of Q4_1 dot
2023-05-20 10:13:19 +03:00
2d5db48371
ggml : use F16 instead of F32 in Q4_0, Q4_1, Q8_0 (#1508 )
2023-05-19 22:17:18 +03:00
6986c7835a
tests : add missing header
2023-05-19 21:17:28 +03:00
943e6081cc
examples : add persistent chat (#1495 )
2023-05-19 13:39:51 -04:00
slaren