828 Commits

Author SHA1 Message Date
Angelos Katharopoulos fc7d84448b Bump the version (#954) 2026-03-06 13:41:47 -08:00
spicyneuron 47be7150a6 fix: convert() uses incorrect defaults for quantization mode (#935) 2026-03-05 17:02:34 -08:00
Yongyue Sun 35fa620279 Add --prefill-step-size as cmd line argument (#943) 2026-03-04 17:40:01 -08:00
Noah Lyons 8162aaad56 step3p5: use rotating cache for sliding attention layers (#949) 2026-03-04 17:17:29 -08:00
Awni Hannun 834fac934c fix qwen3.5 sanitize (#928) 2026-02-24 17:04:43 -08:00
Awni Hannun 179da774b1 Clear the cache during batch generation (#926) 2026-02-23 19:50:35 -08:00
Awni Hannun 720f2369ba add tokens to eval to avoid large graphs when they are not used (#924) 2026-02-23 14:38:08 -08:00
Flynn 65725dcec2 Add filter guard to list comprehension (#918) 2026-02-23 14:22:54 -08:00
n8programs d4701ba513 clear cache on prompt ingestion in server (#917)
Co-authored-by: N8 <n8@n8programs.com>
2026-02-23 12:13:25 -08:00
Angelos Katharopoulos 321e764e0a Make the cache limits more friendly (#910) 2026-02-19 13:52:09 -08:00
Angelos Katharopoulos 83ff9c96d5 Improve the cache size limits (#906) 2026-02-19 10:13:48 -08:00
Yuri Khrustalev 9c113f7019 Allow reading LFM2 models nested rope params (#908)
Co-authored-by: yuri <yuri@liquid-macstudio-2.local>
2026-02-18 16:25:54 -08:00
Gökdeniz Gülmez 7d6c5e4af7 Add tie_word_embeddings modulars in mistral and qwen3 moe (#889)
* Add tie_word_embeddings option and update model call logic in Mixtral and Qwen3 models

* Update copyright year to 2026 and modify input handling in Mixtral and Qwen3 models
2026-02-18 11:23:40 -08:00
Awni Hannun ad067ea627 bump for next version (#904) 2026-02-17 07:39:58 -08:00
Angelos Katharopoulos d7b91e80f0 Fix sharded rms norm in MiniMax M2.5 (#898) 2026-02-16 17:20:07 -08:00
Awni Hannun 1fd521c3c7 fix qwen3.5 casting to fp32 (#902) 2026-02-16 10:23:31 -08:00
Ryan Goulden 572ada278c server: add usage.prompt_tokens_details.cached_tokens to json response (#849) 2026-02-16 08:37:35 -08:00
Ivan Fioravanti fb47f8fb99 Add the trust remote code option to mlx_lm perplexity (#896) 2026-02-15 20:43:23 -08:00
Tarjei Mandt 7a720882a7 Add JoyAI LLM Flash (#894) 2026-02-15 08:06:44 -08:00
spicyneuron 014ebc6a46 Fix mixed quant predicates for MLA models (#892) 2026-02-15 02:44:01 -08:00
Angelos Katharopoulos c6d9d3c9f5 Share model (#871) 2026-02-13 15:48:37 -08:00
Angelos Katharopoulos bcf630614f Fix save/load of CacheList (#886) 2026-02-12 18:41:48 -08:00
Gökdeniz Gülmez 1974376d70 Add GLM5 (#867)
* Add GLM4 MoE DSA model implementation with configurable parameters

* Update Acknowledgments to include GLM4 MoE DSA support

* format

* update ackn.

* Fixes

* Update acknowledgments to include contributions for GLM MoE DSA and additional architectures

* use dsv32 for glm5

* fix

* Fix rope theta

---------

Co-authored-by: Tarjei Mandt <kernelpool@gmail.com>
Co-authored-by: Awni Hannun <awni@apple.com>
v0.30.7
2026-02-12 10:11:16 -08:00
Tarjei Mandt 7e67225e1d Faster DSV32 generation (#885) 2026-02-12 07:35:20 -08:00
JJJYmmm 0fd3126496 [MODEL] support qwen3.5 series w/o vision (#869)
* support text-only qwen3.5 series

Co-authored-by: johnmai-dev <johnmai-dev@users.noreply.github.com>

* add test

* fix sanitize and add test

* make it more readable

* fix lint

---------

Co-authored-by: johnmai-dev <johnmai-dev@users.noreply.github.com>
2026-02-12 07:23:51 -08:00
Tarjei Mandt ca0d1c9630 LongCat MLA (#868)
* LongCat MLA

* Fix comment

* Remove workaround
2026-02-12 06:54:44 -08:00
Awni Hannun 82edd51a1e Devstral tool parser (#874) 2026-02-11 09:41:23 -08:00
Gökdeniz Gülmez aca4c149a1 Make validation set optional in training process (#857) 2026-02-10 15:24:44 -08:00
Tarjei Mandt 8f1c56ec83 Fix DeepSeek V3.2 indexer and weight loading (#866) 2026-02-10 12:15:12 -08:00
viktike 84ae19e675 Pythonic tool calling for LFM2 models (#864)
* Fix tool calling 404 Error: content with non-thinking (instruct) models.

* Add pythonic style tool call parser for LFM2

* test + format

---------

Co-authored-by: Awni Hannun <awni@apple.com>
2026-02-09 15:23:07 -08:00
Awni Hannun 645a326a2e Bump version for next release (#865) 2026-02-09 14:40:57 -08:00
Tarjei Mandt fd6959dca7 Fix Kimi Linear (#853)
* Fix Kimi Linear

* Avoid concat/split

* Use fused rms_norm
2026-02-06 17:31:32 -08:00
Awni Hannun f18526f8d6 DSV3 MLA (#839)
* mla

* try to speed up prefill

* update dsv32 as well
v0.30.6
2026-02-04 12:06:42 -08:00
Tarjei Mandt 25a4c8369e Fix sliding window mask during generation (#843)
* Fix sliding window mask during generation

* make window mask for regular cache

---------

Co-authored-by: Awni Hannun <awni@apple.com>
2026-02-04 08:44:01 -08:00
Awni Hannun e08ec15b72 Fix batch mamba (#842)
* fix batch mamba

* remove mamba cache
2026-02-03 19:31:42 -08:00
Tarjei Mandt b77ec6b951 Fix Step 3.5 Flash model conversion (#840)
* Fix Step 3.5 Flash model conversion

* Detect converted norm weights

* Check layer names
2026-02-03 17:03:54 -08:00
Sebastian Jug ab050d1fac Deepseek V3.2 implementation fixes (#838)
* fix

* fix

---------

Co-authored-by: Awni Hannun <awni@apple.com>
2026-02-03 07:58:29 -08:00
Josh Lehman 942b3ed4b6 fix: handle GLM 4.7 tool call fallbacks (#792)
* fix: handle glm47 tool call fallbacks

- add JSON/plain-text fallback parsing for GLM 4.7 tool calls
- normalize fallback argument values using tool schema
- add tests covering JSON/plain-text fallback paths
- Refs: n/a (pebbles not initialized)

Regeneration-Prompt: |
  Fix GLM 4.7 tool parser crash when the <arg_key> regex does not match. Keep the
  existing arg_key/arg_value parsing path intact, but add defensive fallbacks:
  first try JSON tool-call shapes (name+arguments, function+arguments, or nested
  tool objects), then a plain-text form like "name {json}" or "name key=value".
  If none parse, return a safe unknown tool with raw text in arguments. Use tool
  schema types to preserve string arguments and deserialize non-string values.
  Add tests in tests/test_tool_parsing.py that reproduce the crash with JSON
  input and verify the plain-text fallback behavior. Avoid dependency changes.

* simplify test

* rebase

---------

Co-authored-by: Awni Hannun <awni@apple.com>
2026-02-03 06:48:22 -08:00
Ryan Goulden 11ebc98ada server: support chat_template_kwargs and top_logprobs (#829)
* server: support chat_template_kwargs and top_logprobs

* Adds support for clients sending "chat_template_kwargs",
  matching other open source LLM servers.
  This is gated behind `--trust-client-kwargs` because transformers
  does not provide any safe way to do this.

* changes the server's logprobs response to better match the OpenAI
  chat api & other open source servers.

* server: fix response when handling exceptions

* server: --client-chat-template-args whitelist

* simplify

* comment

---------

Co-authored-by: Awni Hannun <awni@apple.com>
2026-02-02 19:51:38 -08:00
Tarjei Mandt 1630f9bf16 Add Step 3.5 Flash (#836)
* Add Step 3.5 Flash

* Shard model

* Feedback
2026-02-02 18:48:46 -08:00
christian-lms b7cc3aa5e5 allow creation of BatchRotatingKVCache instead of BatchKVCache when empty cache(s) are passed to BatchGenerator (#834) 2026-02-02 07:43:09 -08:00
Awni Hannun 7afcfac51a enable loading custom models (#830) 2026-01-30 19:17:14 -08:00
gaurav 1ecd27a31a fix cli (#827)
* fix cli

* nit

* fix typo in subpackages

---------

Co-authored-by: Awni Hannun <awni@apple.com>
2026-01-30 11:36:43 -08:00
Angelos Katharopoulos 1fe1b3c901 Support distributed inference in the server (#741) 2026-01-30 11:03:55 -08:00
Awni Hannun 8a0f3781e9 fix mixed quant (#825) 2026-01-29 16:54:47 -08:00
Tarjei Mandt 56b8c0f383 Add LongCat Flash Lite (#819)
* Add LongCat Flash Lite

* Fix integer overflow

* Reuse LongCat Flash components

* proper cache

* use arrays cache

* add test for longcat

---------

Co-authored-by: Awni Hannun <awni@apple.com>
2026-01-29 13:31:18 -08:00
Awni Hannun ad9434bef0 actually add cli (#823) 2026-01-29 13:30:54 -08:00
Tarjei Mandt 04fd7ccb97 Fix Kimi K2.5 tool call handling (#821)
* Fix Kimi K2 tool call handling

* Add unit test
2026-01-29 12:35:51 -08:00
Inferencer 7f1b7fe6bc Fix for Exception - MultiLinear.to_quantized() missing 'mode' (#809)
* Fix for Exception - MultiLinear.to_quantized() missing 'mode'

Add mode parameter to mixed_quant_predicate_builder as MLX now requires mode to be specified for nn.quantize class_predicate

* nit

---------

Co-authored-by: Awni Hannun <awni@apple.com>
2026-01-29 07:23:35 -08:00
Luqman c645a55582 Fix NemotronH config compatibility with HuggingFace format (#820)
HuggingFace's NemotronH config uses separate `time_step_min` and
`time_step_max` fields, but mlx-lm expected a `time_step_limit` tuple.
This caused loading failures since `time_step_limit` was required but
never populated from the config.

- Make `time_step_limit` optional with default None
- Add `time_step_min` and `time_step_max` optional fields
- Add `__post_init__` to construct tuple from separate fields
2026-01-29 07:23:27 -08:00