dmcc73 37ad1fb3ed Call warmup_speculative at startup to pre-compile LpB kernels ...

The warmup_speculative() function was defined but never called.
Custom Metal kernels (LpB) require first-call compilation (~200ms).
Without warmup, the first speculative cycle is slow, dragging down
average TPS by 10-20% on short generations.

In mlx_bench testing: cold 48 TPS → warm 60 TPS for DFlash,
cold 39 TPS → warm 44 TPS for MTP.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-01 23:15:05 +01:00

..

eval_configs

Implement continuous batching (#1642)

2026-03-09 15:04:45 +00:00

src/exo_bench

bench: make exo-bench available via nix run on all platforms (#1415)

2026-02-06 21:07:17 +00:00

vendor

Implement continuous batching (#1642)

2026-03-09 15:04:45 +00:00

bench.toml

fmt: add taplo TOML formatter to treefmt configuration

2026-02-27 17:19:48 +00:00

eval_tool_calls.py

bench: restore --danger-delete-downloads planning phase (#1542)

2026-02-19 15:42:02 +00:00

exo_bench.py

Call warmup_speculative at startup to pre-compile LpB kernels

2026-04-01 23:15:05 +01:00

exo_eval.py

Implement continuous batching (#1642)

2026-03-09 15:04:45 +00:00

harness.py

Implement continuous batching (#1642)

2026-03-09 15:04:45 +00:00

parallel_requests.py

Implement continuous batching (#1642)

2026-03-09 15:04:45 +00:00

pyproject.toml

Implement continuous batching (#1642)

2026-03-09 15:04:45 +00:00

run_lcb_tps_bench.sh

Add LCB TPS benchmark script

2026-03-18 17:37:36 +00:00

scenarios.toml

feat: add prefill progress bar for long prompts (#1181)

2026-02-19 03:18:25 +00:00

single-m3-ultra.toml

fmt: add taplo TOML formatter to treefmt configuration

2026-02-27 17:19:48 +00:00