Local AI models on the Minisforum AI X1 Pro-370

The following is an informal benchmark of local AI models running on a Minisforum AI X1 Pro-370. I do not claim to have aimed for or achieved any kind of optimality for these particular models or for this hardware. I have used llama.cpp.

Performance

Prefill is the model reading your prompt – the wait before the first token, so I’ve given the wall-clock time as well as the rate. Decode is everything after that – thinking and writing the reply. Both slow down as context grows, and not at the same rate, so both are measured across a range of context lengths. All runs used llama-bench on an otherwise idle machine, two timed repetitions after a warm-up, everything on the iGPU.

I used the Unsloth dynamic quants (UD-Q4_K_XL) throughout. Gemma4 is gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf from unsloth/gemma-4-26B-A4B-it-qat-GGUF, 14.25 GB, SHA256 a7c5bc71…3d0f5891, repo revision 7b92b5b2. Qwen3.6 is Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf from unsloth/Qwen3.6-35B-A3B-GGUF, 22.36 GB, SHA256 707a55a8…1043f4450, repo revision a483e9e6. GLM 4.7 Flash is GLM-4.7-Flash-UD-Q4_K_XL.gguf from unsloth/GLM-4.7-Flash-GGUF, 17.52 GB, SHA256 b0d4fbc1…13f4ba7f, repo revision 0d32489e.

Gemma4

KV cache f16

llama-bench -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -ngl 99 -fa on \
            -ctk f16 -ctv f16 -p 512,2048,8192,16384,32768,65536 -n 0 -r 2
llama-bench -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -ngl 99 -fa on \
            -ctk f16 -ctv f16 -p 0 -n 128 -d 0,512,2048,8192,16384,32768,65536 -r 2
Context Prefill (t/s) Time to ingest Decode (t/s)
empty context N/A N/A 30.02
512 461.78 1.1 s 28.31
2048 457.13 4.5 s 26.79
8192 422.62 19.4 s 25.60
16384 378.54 43.3 s 24.29
32768 318.72 102.8 s 22.00
65536 240.46 272.5 s 18.75

Gemma4 with f16 KV cache. Twin-axis line chart against context length on a log scale. Prefill declines gently from 462 tokens/s at 512 tokens to 240 at 64K. Decode declines from 28.3 tokens/s at 512 tokens to 18.8 at 64K depth.

KV cache q8_0

llama-bench -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -ngl 99 -fa on \
            -ctk q8_0 -ctv q8_0 -p 512,2048,8192,16384,32768,65536 -n 0 -r 2
llama-bench -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -ngl 99 -fa on \
            -ctk q8_0 -ctv q8_0 -p 0 -n 128 -d 0,512,2048,8192,16384,32768,65536 -r 2
Context Prefill (t/s) Time to ingest Decode (t/s)
empty context N/A N/A 29.62
512 447.90 1.1 s 28.65
2048 427.72 4.8 s 27.51
8192 365.73 22.4 s 26.66
16384 302.26 54.2 s 25.30
32768 235.70 139.0 s 23.21
65536 162.11 404.3 s 19.99

Gemma4 with q8_0 KV cache. Same axes. Prefill declines from 448 tokens/s at 512 tokens to 162 at 64K, falling away more steeply beyond 16K than the f16 chart above. Decode declines from 28.7 tokens/s at 512 tokens to 20.0 at 64K depth, staying slightly above the f16 figures.

Qwen3.6

KV cache f16

llama-bench -m Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf -ngl 99 -fa on \
            -ctk f16 -ctv f16 -p 512,2048,8192,16384,32768,65536 -n 0 -r 2
llama-bench -m Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf -ngl 99 -fa on \
            -ctk f16 -ctv f16 -p 0 -n 128 -d 0,512,2048,8192,16384,32768,65536 -r 2
Context Prefill (t/s) Time to ingest Decode (t/s)
empty context N/A N/A 22.06
512 348.02 1.5 s 22.17
2048 343.24 6.0 s 22.00
8192 320.18 25.6 s 21.48
16384 289.72 56.6 s 20.32
32768 254.26 128.9 s 18.86
65536 205.86 318.4 s 16.40

Qwen3.6 with f16 KV cache. Twin-axis line chart against context length on a log scale. Prefill declines from 348 tokens/s at 512 tokens to 206 at 64K. Decode declines only gently, from 22.2 tokens/s at 512 tokens to 16.4 at 64K depth.

KV cache q8_0

llama-bench -m Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf -ngl 99 -fa on \
            -ctk q8_0 -ctv q8_0 -p 512,2048,8192,16384,32768,65536 -n 0 -r 2
llama-bench -m Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf -ngl 99 -fa on \
            -ctk q8_0 -ctv q8_0 -p 0 -n 128 -d 0,512,2048,8192,16384,32768,65536 -r 2
Context Prefill (t/s) Time to ingest Decode (t/s)
empty context N/A N/A 22.01
512 356.23 1.4 s 21.93
2048 350.57 5.8 s 21.74
8192 328.34 24.9 s 21.27
16384 293.27 55.9 s 20.92
32768 243.58 134.5 s 20.15
65536 183.34 357.5 s 17.83

Qwen3.6 with q8_0 KV cache. Same axes, and almost indistinguishable from the f16 chart above until 32K. Prefill declines from 356 tokens/s at 512 tokens to 183 at 64K. Decode declines from 21.9 tokens/s at 512 tokens to 17.8 at 64K depth.

GLM 4.7 Flash

KV cache f16

llama-bench -m GLM-4.7-Flash-UD-Q4_K_XL.gguf -ngl 99 -fa on \
            -ctk f16 -ctv f16 -p 512,2048,8192,16384 -n 0 -r 2
llama-bench -m GLM-4.7-Flash-UD-Q4_K_XL.gguf -ngl 99 -fa on \
            -ctk f16 -ctv f16 -p 0 -n 128 -d 0,512,2048,8192,16384 -r 2
Context Prefill (t/s) Time to ingest Decode (t/s)
empty context N/A N/A 28.78
512 395.38 1.3 s 26.87
2048 354.55 5.8 s 23.44
8192 234.93 34.9 s 16.98
16384 158.80 103.2 s 12.23

GLM 4.7 Flash with f16 KV cache, measured only to 16K. Twin-axis line chart against context length on a log scale. Both curves fall away far more steeply than the other two models: prefill from 395 tokens/s at 512 tokens to 159 at 16K, and decode from 26.9 tokens/s at 512 tokens to 12.2 at 16K depth.

I found that GLM 4.7 performed poorly except for smaller contexts at f16. My early trials with the other KV cache quants all came out slower than f16 on this model, so I abandoned them rather than sweeping them here.

Other Models

I have only included the results here for the models that I found useful/usable. Dense models ran at a fraction of the speed of the MoE models above on the same iGPU, so I left them out. I didn’t capture those runs with llama-bench, and in early testing with Ollama the CPU was slower again, as you’d expect.

Hardware/OS/Configuration Details

This is the stock configuration of the Minisforum AI X1 Pro-370, except I replaced the stock NVMe SSD with a Samsung 990 Pro due to a (currently undiagnosed) failure. My company bought it at the start of 2026 for £1,279.00 inclusive of VAT from Minisforum UK (Shenzhen Weigao Keji Youxiangongsi).

This is not a headless system so some of the iGPU memory is being used for graphics, even when almost idle. I didn’t record it, but I think it was approx 1 GB.

For a rough idea of what this machine does under sustained load: across nearly three hours of continuous inference the CPU package sat at 76–80 °C, the iGPU at 73–80 °C, and the memory modules at 45–49 °C. Not running inference, those settle to around 40 °C and 38 °C. The iGPU held 2896–2898 MHz of its 2900 MHz maximum throughout, so nothing was thermally throttling.

Note that this system has an NPU, which isn’t used in any of my testing. I briefly tried FastFlowLM with Lemonade, but found no benefit for the models of interest and decided not to explore further. The NPU may be more interesting for other AI/ML workloads.

It runs cool and quiet, and while it is a mini PC, the case leaves more room for air to circulate than the smaller mini PCs I’ve used, so even with the fans at maximum it stays quieter than my laptop.

I needed this PC for general development, not for local AI, and from that perspective I’m happy with it and the local AI is the cherry on top. The capable models that I’m using on it weren’t available when it was bought, so that’s been a happy surprise. It’s good for light AI workflows and the speed/capability drawbacks aren’t a problem for more structured/asynchronous workflows. If you’ve been occasionally hitting a remote API and want to take that local, you’ll probably be quite happy with it. But if you’re using it for heavier use like coding agents and expect interactivity, expect disappointment. There’s no good expansion path and while I’m hopeful we’ll see further model improvements that suit this machine, you should realise that it’s easy for models to fall outside that sweetspot and be bottlenecked by memory performance. The only upgrade option would be adding an eGPU and if the GPU is doing the work anyway, you could have economised on the PC itself.