electron-rare/mlx-lm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 11ebc98ada server: support chat_template_kwargs and top_logprobs (#829) Ryan Goulden 2026-02-02 19:51:38 -08:00
  • 1630f9bf16 Add Step 3.5 Flash (#836) Tarjei Mandt 2026-02-03 13:48:46 +11:00
  • b7cc3aa5e5 allow creation of BatchRotatingKVCache instead of BatchKVCache when empty cache(s) are passed to BatchGenerator (#834) christian-lms 2026-02-02 10:43:09 -05:00
  • 7afcfac51a enable loading custom models (#830) Awni Hannun 2026-01-30 19:17:14 -08:00
  • 1ecd27a31a fix cli (#827) gaurav 2026-01-31 01:06:43 +05:30
  • 1fe1b3c901 Support distributed inference in the server (#741) Angelos Katharopoulos 2026-01-30 11:03:55 -08:00
  • 8a0f3781e9 fix mixed quant (#825) Awni Hannun 2026-01-29 16:54:47 -08:00
  • 56b8c0f383 Add LongCat Flash Lite (#819) Tarjei Mandt 2026-01-30 08:31:18 +11:00
  • ad9434bef0 actually add cli (#823) Awni Hannun 2026-01-29 13:30:54 -08:00
  • 04fd7ccb97 Fix Kimi K2.5 tool call handling (#821) Tarjei Mandt 2026-01-30 07:35:51 +11:00
  • 7f1b7fe6bc Fix for Exception - MultiLinear.to_quantized() missing 'mode' (#809) Inferencer 2026-01-30 01:23:35 +10:00
  • c645a55582 Fix NemotronH config compatibility with HuggingFace format (#820) Luqman 2026-01-29 23:23:27 +08:00
  • 1bdc8afca3 Bump mlx version and version (#816) Awni Hannun 2026-01-28 17:10:49 -08:00
  • 96699e6dad Add Kimi-K2.5 (#813) Tarjei Mandt 2026-01-28 05:23:39 +11:00
  • b012e1a1e9 Add LongCat Flash tool parser (#810) Tarjei Mandt 2026-01-27 06:51:44 +11:00
  • f53a9b0689 Transformers v5 (#811) Awni Hannun 2026-01-26 11:51:30 -08:00
  • beceb5c16d Fix ArraysCache.from_state not initializing left_padding and lengths (#807) v0.30.5 lpalbou 2026-01-25 15:46:35 +01:00
  • 12073b1db4 Sync random seed across ranks in distributed chat (#801) Tarjei Mandt 2026-01-24 02:56:20 +11:00
  • d9846d37cb fix: use correct variable for logprobs in batch generation (#800) Luqman 2026-01-23 23:24:39 +08:00
  • 1b76e3d580 Allow qq ops with activation quantization (#749) Awni Hannun 2026-01-22 08:59:58 -08:00
  • e5ddaff99b add kimi tool parser (#791) Evan Quiney 2026-01-22 15:46:07 +00:00
  • 14b9faf1bc Adding TeleChat3 (#773) Gökdeniz Gülmez 2026-01-22 16:35:58 +01:00
  • 1423702019 Update glm4_moe_lite To Store KV Latent in Cache (#780) n8programs 2026-01-21 16:55:14 -05:00
  • 5ed1e48a3c bump transformers (#746) Awni Hannun 2026-01-21 13:45:15 -08:00
  • 4100703f1a move Xielu Activation in Apertus to activations.py (#772) Gökdeniz Gülmez 2026-01-20 15:51:16 +01:00
  • 8c08a46da4 server: use OpenAI compatible finish_reason (#782) Ryan Goulden 2026-01-20 06:50:37 -08:00
  • bc891dca4c import logging as it throws no logging error in place of actual error (#778) Maanas Verma 2026-01-20 19:53:24 +05:30
  • 02228601cd Add glm4 moe lite model (#776) v0.30.4 Ivan Fioravanti 2026-01-19 17:13:13 +01:00
  • 8daabcc7c1 Shard LongCat Flash (#771) Tarjei Mandt 2026-01-18 10:21:36 -05:00
  • cd7d9a536e Add minimax tensor sharding (#760) Evan Quiney 2026-01-17 16:45:42 +00:00
  • 25246632cf Fix Longcat Flash extended context support (#768) Tarjei Mandt 2026-01-17 10:42:22 -05:00
  • 6651d2e0bf Add gpt-oss sharding (#761) Evan Quiney 2026-01-17 15:42:11 +00:00
  • 43dcf2f0c0 fix: unused batch_size parameter for mlx_lm.evaluate (#762) AndrewTan517 2026-01-17 23:29:32 +08:00
  • 769069d66b Fix CacheList batching (#769) Tarjei Mandt 2026-01-17 10:29:21 -05:00
  • 5261ab85ee Fix swiglu parameter order (#767) Tarjei Mandt 2026-01-17 09:13:23 -05:00
  • c27c94a0ff fix(falcon_h1): support tied embeddings and correct muP scaling (#764) gaurav 2026-01-16 20:41:18 +05:30
  • fd80ac89fb Adds support for Nemotron Super 49b v1.5 (#756) Thomas Lazarus 2026-01-16 09:03:05 -06:00
  • edbf61dd8b Use compiled Swiglu (#753) Awni Hannun 2026-01-14 11:58:41 -08:00
  • c2a716c871 Update for latest mlx (#759) Awni Hannun 2026-01-14 11:57:44 -08:00
  • 63c9873617 Handle empty caches during batch merge (#755) Ivan Fioravanti 2026-01-14 17:47:47 +01:00
  • 7585c142a6 fix: Fix type hint and pydoc for batch_generate (#745) Julian Tibble 2026-01-12 15:36:48 +00:00
  • 44d12e5d6f Fix batch generation for IQuestLoopCoder model (#748) Tarjei Mandt 2026-01-09 19:50:51 -05:00
  • 7a86c1289e Make cache list batchable (#743) Awni Hannun 2026-01-09 10:47:05 -08:00
  • a20eefd7c2 Refactor tokenizer error handling to use warnings instead of exceptio… (#744) Gia Huy Vuong 2026-01-09 22:19:33 +07:00
  • 3eb6ecf2b6 Fix tools parameter in apply_chat_template call (#747) Tarjei Mandt 2026-01-09 10:08:01 -05:00
  • 39a96ab18b Add a server benchmark for continuous batching (#728) Awni Hannun 2026-01-08 14:35:40 -08:00
  • 43082feafa Make MambaCache compatible with batch generation for nemotron-h (#690) Nikhil Mitra 2026-01-08 16:42:36 -05:00
  • 5cce1495e0 Add extract method to ArraysCache for item retrieval (#740) Gökdeniz Gülmez 2026-01-08 15:30:20 +01:00
  • 509f5aef89 Fix sliding window batching (#738) Awni Hannun 2026-01-07 10:07:56 -08:00
  • 0f76343ea4 Add IQuest Coder V1 Loop variant (#716) Tarjei Mandt 2026-01-07 10:39:24 -05:00
  • 298b67c755 Add AWQ/GPTQ weight transformation utilities (#730) Eric Curtin 2026-01-06 16:50:58 +00:00
  • 94497d5255 fix release (#733) v0.30.2 Awni Hannun 2026-01-05 18:31:29 -08:00
  • 4c80c68ea6 patch (#731) v0.30.1 Awni Hannun 2026-01-05 16:54:13 -08:00
  • ac8ae2c05a Improve reasoning and tool call parsing in server (#711) Awni Hannun 2026-01-05 14:18:52 -08:00
  • 7a4d137df6 Add K-EXAONE MoE (#719) Tarjei Mandt 2026-01-05 11:50:33 -05:00
  • 90db1e6266 Add Solar Open (#721) Tarjei Mandt 2026-01-05 11:38:53 -05:00
  • d3dc2e3f33 Add logits_processors support to batch_generate (#635) Thomas Lazarus 2026-01-04 17:19:00 -06:00
  • 7423bf6752 Add YoutuLLM (#720) John Mai 2026-01-04 01:12:55 +08:00
  • 5dec49a12f Add IQuest Coder V1 (#714) Tarjei Mandt 2026-01-01 09:17:14 -05:00
  • 3727e01cd7 Fix empty /v1/models response for locally loaded models (#713) cxl-git-hub 2026-01-01 06:22:00 +08:00
  • 09579644ac Add RWKV7 (#580) Molly Sophia 2026-01-01 06:21:42 +08:00
  • 0081085a91 chore: add model-path param flag for convert API for better clarity (#702) jaycoolslm 2026-01-01 04:30:38 +08:00
  • fed582eede Fix chat template detection for models with custom tokenizers (#712) Tarjei Mandt 2025-12-31 09:43:34 -05:00
  • 7973b8cfe8 allow mxfp8 and nvfp4 (#709) Awni Hannun 2025-12-30 09:19:36 -08:00
  • 7096618d50 Ignore generation_config decode errors (#708) will-lms 2025-12-29 17:03:29 -05:00
  • 1e0c0f3985 Fix GIL starvation in _generate thread when batch is idle (#706) Sebastian Jug 2025-12-29 14:52:58 -05:00
  • 68f18bae14 batch_generate fails with Phi3 (LongRoPE) when prompts have different lengths (#707) Viacheslav Ivanov 2025-12-29 20:52:43 +01:00
  • f5ae09a807 Enhance load_config function to check for config file existence and i… (#701) Gia Huy Vuong 2025-12-27 21:59:38 +07:00
  • 08c8c0a5ea support minimax m2 (#700) Awni Hannun 2025-12-26 14:02:37 -08:00
  • a9311cca23 shard glm (#698) Awni Hannun 2025-12-24 12:05:22 -08:00
  • 9fe5f43abf custom dsv32 chat template (#693) Awni Hannun 2025-12-22 13:52:58 -08:00
  • 1b2d11b5c7 Bump the version (#692) v0.30.0 Angelos Katharopoulos 2025-12-18 13:43:49 -08:00
  • 657a66c5c4 revert return dict and wrap apply_chat_template (#691) Awni Hannun 2025-12-18 13:16:44 -08:00
  • 595fb4bdbf bump to transformer v5 (#689) Awni Hannun 2025-12-17 16:34:51 -08:00
  • 79a0721c9a Model parallel generation (#676) Angelos Katharopoulos 2025-12-17 13:35:28 -08:00
  • cc3264c22e More useful error message for unsupported batching (#687) Awni Hannun 2025-12-17 12:30:53 -08:00
  • a227a9e9f3 Add mimo v2 flash (#685) Awni Hannun 2025-12-17 06:50:16 -08:00
  • cd9ca9f068 fixes for transformers v5 (#684) Awni Hannun 2025-12-17 06:08:08 -08:00
  • 7744d0f40b fix: server busy-waiting in request queue polling (#674) Jinhyeok Lee 2025-12-17 07:16:34 +09:00
  • f3ed856610 support nemotron 3 (#678) v0.29.0 Awni Hannun 2025-12-16 08:50:44 -08:00
  • ede65a1484 Fix for Devstral-2 (#671) Inferencer 2025-12-12 02:40:12 +10:00
  • 3d3e0751a3 fix (#669) Awni Hannun 2025-12-09 17:11:51 -08:00
  • 085e36e6ab Fix SuScaledRoPE (#660) Anthony 2025-12-09 16:59:28 +01:00
  • eea2e5f5de Fix server batching condition for SSMs (#655) Angelos Katharopoulos 2025-12-08 23:18:53 -08:00
  • cb763947ee Fix fusion and test (#668) Awni Hannun 2025-12-08 16:39:46 -08:00
  • b343a0556f fix dsv32 and gemma3 (#664) Awni Hannun 2025-12-08 16:14:10 -08:00
  • 82dfd39ef2 default repetition penalty to 0.0 in the server (#658) Awni Hannun 2025-12-08 16:14:00 -08:00
  • 84996808a2 Use test data zipfile in CI (#662) Awni Hannun 2025-12-08 16:13:34 -08:00
  • 99f8fd6cc8 fix: calling correct dequantize function (#666) Hritik Kumar 2025-12-09 03:04:42 +05:30
  • 7d042c6124 fix for rnj-1 (#657) test_data Awni Hannun 2025-12-08 07:16:57 -08:00
  • 0fbff353db Fix slow batch generation in server by setting wired_limit (#652) otarkhan 2025-12-05 21:28:17 +02:00
  • 0ad37e2bbf version bump (#651) Angelos Katharopoulos 2025-12-03 15:30:36 -08:00
  • 454bf9a22b Fix the release action and revert version (#650) v0.28.4 Angelos Katharopoulos 2025-12-03 14:23:06 -08:00
  • 133b5d3bd7 version bump (#649) Awni Hannun 2025-12-03 14:04:00 -08:00
  • abc52a0a48 Add deepseek v32 (#512) Awni Hannun 2025-12-03 14:03:21 -08:00
  • 6b42901468 Batching in the server (#626) Angelos Katharopoulos 2025-12-03 13:14:18 -08:00
  • f353e0178b fix lora fusion for non affine quantization (#647) Awni Hannun 2025-12-03 10:33:01 -08:00
  • f940cf3a95 fix flaky test (#643) Awni Hannun 2025-12-02 13:33:11 -08:00
  • 34cbb8b51a Add a prompt cache that can hold multiple prompts (#625) Angelos Katharopoulos 2025-12-02 13:29:55 -08:00
  • 4bc21cc17b Ministral3 (#642) Awni Hannun 2025-12-02 10:59:45 -08:00