* Fix ArraysCache.from_state not initializing left_padding and lengths
The base class from_state uses __new__ to bypass __init__, but
ArraysCache.state.setter only sets self.cache, not left_padding
or lengths. This causes AttributeError when using loaded caches
with models that use MambaCache (e.g., Qwen3-Next).
The fix overrides from_state in ArraysCache to properly initialize
these attributes before setting state.
* Add test
---------
Co-authored-by: Awni Hannun <awni@apple.com>
* in. com.
* Add Telechat3 MLP and Decoder Layer implementations
* Implement Telechat3 model and decoder layer with attention mechanism
* Add support for 'telechat3-yarn' in initialize_rope function
* Update Acknowledgments and add Telechat3 model tests
* format
* fix test
---------
Co-authored-by: Awni Hannun <awni@apple.com>
* import logging as it throws no logging error in place of actual error
* Update mlx_lm/utils.py
---------
Co-authored-by: Awni Hannun <awni.hannun@gmail.com>
* fix: handle tied embeddings and muP scaling mismatch
Support models with "tie_word_embeddings": true by conditionally initializing the lm_head and reusing embed_tokens weights.
Since falcon_h1 applies disparate muP multipliers to embeddings and the language head during sanitization, a scaling correction factor (lm_head_mult / embedding_mult) is applied in the forward pass when weights are tied. This ensures mathematical correctness without duplicating weights in memory.
Fixes ValueError: Missing 1 parameters: lm_head.weight
* simplify sanitize logic
Co-authored-by: Awni Hannun <awni.hannun@gmail.com>
---------
Co-authored-by: Awni Hannun <awni.hannun@gmail.com>
The 'prompts' argument to batch_generate has type List[List[int]].
However, previously the type hint was wrong, and the pydoc had the
correct type but the wrong argument name.
* Refactor tokenizer error handling to use warnings instead of exceptions for missing tool call tokens
* disable tool calling if not in vocab
* add tool call warning
---------
Co-authored-by: Awni Hannun <awni@apple.com>
* Make MambaCache compatible with batch generation
* fix: Support right-padding masking in ArraysCache, add tests
* almost working
* test pass
* update models + gated delta
* rebase + fix
* fix
* allow batching in server
---------
Co-authored-by: Awni Hannun <awni@apple.com>
* Add AWQ/GPTQ weight transformation utilities
Add functions to transform AutoAWQ and GPTQ packed weights to MLX
quantization format. This includes an unpacking function for AWQ/GPTQ
weights and a transformation function that converts the quantization
format from AutoAWQ/GPTQ to MLX. The transformation handles both
symmetric and asymmetric quantization by computing biases from zero
points. Also add a test case to verify the transformation works
correctly.
Signed-off-by: Eric Curtin <eric.curtin@docker.com>
* nits
* nits
---------
Signed-off-by: Eric Curtin <eric.curtin@docker.com>
Co-authored-by: Awni Hannun <awni@apple.com>
* Parse reasoning in server
* redesign and start to fix tool parsing
* add function gemma
* fix
* fix
* glm47 tools
* add minimax m2 tool parser
* tool_call finish reason
* Keep model wired in the server to reduce ttft
* infer tool parser
* Update generate.py
* add samplers and logits processors with per example option and server support
---------
Co-authored-by: Awni Hannun <awni@apple.com>
* server: expose local --model in /v1/models.
use full local path as model id for /v1/models
在加载本地模型时，/v1/models返回空列表，导致 open webui获取不到模型，没办法使用。所以这里把本地模型的全路径作为 model_id返回。
* nits
* fix test
---------
Co-authored-by: chenxilong <cxl750529174@163.com>
Co-authored-by: Awni Hannun <awni.hannun@gmail.com>
Awni Hannun