glm 5.3 flash (321B) on a 3090 + a 3080 ti: from 26 to 31.7 tok/s, and why i'm still sticking with qwen

what i wanted to know

GLM 5.3 Flash is a very good model, but with 321 billion parameters it obviously doesn't fit in 36gb of VRAM: even with a very aggressive quant, its files weigh 89gb, which is more than my ram. Maya still manages to run it by keeping the most used experts in VRAM and computing the others on the CPU. i wanted to see what two new options that are supposed to speed it up were worth, and above all whether it became usable for my agents.

the numbers

decode in tok/s, on documents of 8K, 32K and 120K tokens, with 512 tokens generated each time:

the update alone changes nothing, the options are what bring the gain.

why it speeds up

on this machine, the bottleneck is the CPU computing the experts that aren't in VRAM. routing affinity gives a small bonus to the experts already sitting in VRAM, which takes their usage rate from about 76 to 82% and cuts CPU work per token by roughly 25%. CPU defer, on the other hand, brings almost nothing on its own (+1.5%, within the noise), and only adds about 1.7 tok/s once combined with affinity.

the price

these two options change the answers. comparing the output probabilities with a run without any option, the gap is 60 to 110 times bigger than the natural variation from one run to another, and CPU defer costs as much gap as affinity for a much smaller gain, which makes it a bad trade-off here. it doesn't prove the answers are worse, only that they're different.

and in real life, for an agent?

i gave both models the same task, a read-only review of a real TypeScript repo, cut off after 5 minutes. GLM with both options did 24 steps at 12.5 seconds each, while Qwen 3.8 Flash Next on Strata did 36 at 8 seconds, on the same machine.

what i keep

GLM 5.3 is still a good model, but almost nobody has the hardware to run it comfortably: here it tops out around 30 tok/s while Qwen goes past 100 on the same box. for my agents, i'm sticking with Qwen. if you still want to try it, the install is in the maya guide.