electron-rare/prima-cpp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,264 Commits 1 Branch 0 Tags
main
Commit Graph

428 Commits

Author SHA1 Message Date
DandinPower 73969b44b1 feat: use bitsqueeze as communication compression library to replace the previous quantization/k_quantization/sparsity. Also add a new argument call comm_compression_threshold to allow user can set a threshold for compression (good for only compress prefilling tokens) 2025-12-06 21:08:10 +08:00
DandinPower 7480ffbd30 Merge remote-tracking branch 'upstream/main' into merge/merge_86ca21e_from_upstream 2025-08-07 20:19:18 +08:00
DaninPower c949e531d5 feat: make llama-perplexity can support with multi node feature 2025-08-02 11:48:26 +00:00
Li, Zonghang 86ca21e49c server: fix bugs when running speculative decoding 2025-07-13 21:52:59 +08:00
Zonghang Li 1ea2d61a97 speedup: add arg --keep-out-in-cuda to run the output layer on CUDA 2025-06-28 10:58:18 +04:00
Li, Zonghang e8d3e5a631 update README 2025-06-27 20:16:30 +04:00
Li, Zonghang 3a03549fed update README 2025-06-26 22:37:08 +04:00
Li, Zonghang ba59a1a07a update README 2025-06-26 22:33:28 +04:00
DandinPower b51eb86676 doc: update documentation for multi splits feature 2025-06-22 22:58:07 +08:00
Joseph Liaw 6a898ef9d8 docs: clarify split loading usage 2025-06-19 19:55:56 +08:00
Zonghang Li dd589561b4 improve the computing buffer estimate 2025-06-19 08:02:43 +00:00
Li, Zonghang dfb1feb54e update README 2025-06-16 12:09:07 +04:00
Li, Zonghang fbf853341b add endpoint /v1/cancel 2025-06-07 11:34:38 +04:00
Lizonghang ef1e10101e add test for IQ1 and doc for device selection 2025-06-04 15:12:00 +04:00
Lizonghang c54a6a0132 fix context shifting 2025-05-19 16:58:35 +04:00
Lizonghang 2cc01483fd support server mode 2025-05-14 18:28:46 +04:00
Lizonghang ebd09fc83c Merge branch 'dev' 2025-05-14 14:19:53 +04:00
Lizonghang 258fb2d06b add QA: How to manually profile a device 2025-05-14 14:19:20 +04:00
Li, Zonghang e2de4511c5 Update README.md 2025-05-11 18:15:39 +08:00
leeetao b212d74dc3 update Readme.md 2025-04-17 09:17:11 +00:00
Zonghang Li f9702ec4c0 update README.md 2025-04-16 15:55:43 +04:00
Li, Zonghang b59d6d9cdf Update README.md 2025-04-15 09:59:08 +08:00
Li, Zonghang 6d13836c44 Update README.md 2025-04-11 01:41:43 +08:00
Li, Zonghang 4845abf25e Update README.md 2025-04-11 01:20:36 +08:00
Lizonghang e48b804730 update README.md 2025-04-09 13:55:30 +04:00
Li, Zonghang 55f8dc588f Update README.md 2025-04-09 10:56:25 +08:00
Lizonghang e421d788d3 update README 2025-04-08 23:15:43 +04:00
Lizonghang 03ff9a7654 update README 2025-04-07 23:28:01 +04:00
Li, Zonghang a3a1f4499b Update README.md 2025-04-07 22:14:44 +08:00
Li, Zonghang 98d73778a6 Update README.md 2025-04-07 22:13:31 +08:00
Li, Zonghang 5984b1b75f Update README.md 2025-04-07 22:13:12 +08:00
Li, Zonghang ebd15b4112 Update README.md 2025-04-07 22:12:11 +08:00
Li, Zonghang 35adc76337 Update README.md 2025-04-07 22:08:14 +08:00
Lizonghang 87eb1aa7ec update README 2025-04-07 18:06:57 +04:00
Lizonghang fffefb9259 update README 2025-04-07 17:57:57 +04:00
Lizonghang 3b264352e7 update README 2025-03-30 23:39:36 +04:00
Lizonghang 2a01ff5fb1 init 2024-10-23 09:42:32 +04:00
Viet-Anh NGUYEN (Andrew) 71967c2a6d Add Llama Assistant (#9744) 2024-10-04 20:29:35 +02:00
Paweł Wodnicki 3f1ae2e32c Update README.md (#9591) 
Add Bielik model.
 2024-10-01 19:18:46 +02:00
Georgi Gerganov 589b48d41e contrib : add Resources section (#9675) 2024-09-29 14:38:18 +03:00
Aarni Koskela 43bcdd9703 readme : add tool (#9655) 2024-09-28 15:07:14 +03:00
Georgi Gerganov b5de3b74a5 readme : update hot topics 2024-09-27 20:57:51 +03:00
Riceball LEE 1d48e98e4f readme : add programmable prompt engine language CLI (#9599) 2024-09-23 18:58:17 +03:00
Shane A 0aadac10c7 llama : support OLMoE (#9462) 2024-09-16 09:47:37 +03:00
OSecret d6b37c881f readme : update tools list (#9475) 
* Added link to proprietary wrapper for Unity3d into README.md

Wrapper has prebuild library and was tested on iOS, Android, WebGL, PC, Mac platforms, has online demos like [this](https://d23myu0xfn2ttc.cloudfront.net/rich/index.html) and [that](https://d23myu0xfn2ttc.cloudfront.net/).

* Update README.md

Fixes upon review
 2024-09-15 10:36:53 +03:00
Faisal Zaghloul 449ccfb6f5 Add Jais to list of supported models (#9439) 
Co-authored-by: fmz <quic_fzaghlou@quic.com>
 2024-09-12 02:29:53 +02:00
Georgi Gerganov 38ca6f644b readme : update hot topics 2024-09-09 15:51:37 +03:00
Antonis Makropoulos 5ed087573e readme : add LLMUnity to UI projects (#9381) 
* add LLMUnity to UI projects

* add newline to examples/rpc/README.md to fix editorconfig-checker unit test
 2024-09-09 14:21:38 +03:00
Georgi Gerganov b69a480af4 readme : refactor API section + remove old hot topics 2024-09-03 10:00:36 +03:00
Younes Belkada b40eb84895 llama : support for falcon-mamba architecture (#9074) 
* feat: initial support for llama.cpp

* fix: lint

* refactor: better refactor

* Update src/llama.cpp

Co-authored-by: compilade <git@compilade.net>

* Update src/llama.cpp

Co-authored-by: compilade <git@compilade.net>

* fix: address comments

* Update convert_hf_to_gguf.py

Co-authored-by: compilade <git@compilade.net>

* fix: add more cleanup and harmonization

* fix: lint

* Update gguf-py/gguf/gguf_writer.py

Co-authored-by: compilade <git@compilade.net>

* fix: change name

* Apply suggestions from code review

Co-authored-by: compilade <git@compilade.net>

* add in operator

* fix: add `dt_b_c_rms` in `llm_load_print_meta`

* fix: correct printf format for bool

* fix: correct print format

* Update src/llama.cpp

Co-authored-by: compilade <git@compilade.net>

* llama : quantize more Mamba tensors

* llama : use f16 as the fallback of fallback quant types

---------

Co-authored-by: compilade <git@compilade.net>
 2024-08-21 11:06:36 +03:00