mirror of
https://github.com/ollama/ollama
synced 2026-04-23 08:45:14 +00:00
bbbad97686
MLX runners (image generation and LLM) previously bypassed the scheduler's standard load path via a separate loadMLX method. This meant they skipped VRAM fitting checks and couldn't participate in model eviction. Now all model types flow through the same load function. Model eviction for MLX is based on weights as KV cache and compute graph are dynamic. This means that eviction does not take into account the worst case memory and models can still compete for memory but it is a significant improvement. |
||
|---|---|---|
| .. | ||
| cache | ||
| mlx | ||
| model | ||
| sample | ||
| cache.go | ||
| client.go | ||
| imports.go | ||
| pipeline.go | ||
| runner.go | ||
| server.go | ||
| utf8_buffer.go | ||
| utf8_buffer_test.go | ||