mirror of
https://github.com/ollama/ollama
synced 2026-04-23 08:45:14 +00:00
1044b0419a
* model: add MLA absorption for glm4moelite Split the combined KV_B tensor into separate K_B and V_B tensors during conversion, enabling MLA (Multi-head Latent Attention) absorption which compresses the KV cache for improved efficiency. * ggml: enable MLA flash attention for GLM-4.7-flash Add support for gqa_ratio 4 in MLA flash attention kernels. GLM-4.7-flash uses head size 576 with gqa_ratio 4, which was previously only supported for gqa_ratio 16 (DeepSeek). Metal changes: - Enable head size 576 for flash attention - Increase simdgroups to 8 for large heads (>=512) - Add case 8 kernel dispatch for 8 simdgroups CUDA changes: - Add gqa_ratio 4 support for head 576/512 - Add tile configs for (576, 512, 4) and (576, 512, 8) - Add MMA config cases for ncols 4 - Add template instances for ncols2=4 * model: add compatibility validation for glm4moelite architecture |
||
|---|---|---|
| .. | ||
| imageproc | ||
| input | ||
| models | ||
| parsers | ||
| renderers | ||
| testdata | ||
| bytepairencoding.go | ||
| bytepairencoding_test.go | ||
| model.go | ||
| model_test.go | ||
| sentencepiece.go | ||
| sentencepiece_test.go | ||
| textprocessor.go | ||
| vocabulary.go | ||
| vocabulary_test.go | ||
| wordpiece.go | ||
| wordpiece_test.go | ||