# running glm 5.3 flash (321B) on two consumer cards with maya how to install project maya, pick the right version of the model when your cards have 24gb or less, and the two options that got me 22% more speed, with what they cost - date: 2026-10-22 - type: guide (local ai) - hardware: 3090 + 3080 Ti, 78gb DDR4 - source: measurements made by the author on his own hardware - html page: https://pkillsleep.dev/en/guides/maya-glm-53/ ## what maya is for GLM 5.3 Flash is a model with 321 billion parameters, about 18 billion of which work on each token. even heavily compressed, it weighs around 90gb, more than my RAM. [Project Maya](https://github.com/mw00/project-maya) still runs it by storing its experts on three tiers: the ones your conversation uses the most in VRAM, the next ones in RAM, and the rest on the SSD, then it moves them around as the conversation goes. **you'll need**, according to the official docs: an nvidia RTX 20 card or newer (a single one or up to 16), at least 32gb of RAM, a fast NVMe SSD with ~100gb free (definitely not a hard drive), linux with the nvidia driver, CUDA 12, g++ and python 3.10 or newer. **be honest with yourself before you start**: on my machine it tops out around 30 tok/s, while Qwen 3.8 Flash Next on Strata goes past 100 on the same box. GLM is a very good model, but almost nobody has the hardware to run it comfortably. if you want a fast model for agents, look at the [strata guide](/en/guides/strata-deux-gpu/) instead. ## 1. install ```sh git clone https://github.com/mw00/project-maya.git && cd project-maya ./setup.sh ``` the script checks what's missing, shows the exact command to install it, and installs nothing outside its own folder. it shows you each download before starting it, compiles everything once (20 to 40 minutes, not counting the download), then starts the model. then open `http://127.0.0.1:8080` to chat with it. to start it again later, just run `./setup.sh` again: it doesn't download anything twice. ## 2. pick the right version of the model Maya offers its own compressed versions of GLM 5.3 Flash. for cards with 24gb or less, take **Maya-S24**: its attention is compressed to 4 bits, which frees about 1.5gb of VRAM for the experts, and the docs announce +14% speed compared to Maya-S. you can pick it without going through the questions: ```sh ./setup.sh --model Maya-S24 --gpus 0,1 --context 131072 ``` `--gpus 0,1` splits the model's layers across your two cards, and `--context` sets the context length (32K by default, up to 131K for GLM). **good to know**: at startup, Maya reserves almost all the free RAM (everything except ~6gb) for its experts. while it runs, the rest of the machine has little memory left, so don't run it next to another big model. ## 3. the two options that speed it up (and what they cost) on a machine like mine, the bottleneck isn't the graphics card: it's the CPU computing the experts that aren't in VRAM. two options, coming from a recent contribution and not yet described in the official docs, go after that: - `STRATA_GLM_ROUTE_AFFINITY=0.02` gives a small bonus to the experts already in VRAM when it picks which ones to use. on my machine, the share of experts served from VRAM goes from about 76 to 82%, and CPU work per token drops by about 25%; - `STRATA_GLM_DEFER_CPU=1` shifts part of the CPU work. on its own it brings almost nothing here, but it adds a little once combined with the first one. you add them with the installer's `--env` option, which keeps them in the config: ```sh ./setup.sh --env STRATA_GLM_ROUTE_AFFINITY=0.02 --env STRATA_GLM_DEFER_CPU=1 ``` together, they took me from 26.1 to 31.7 tok/s on average. **but they change the model's answers**: it no longer picks quite the same experts, so no longer quite the same words. i measured the gap against a reference output, it's real; it doesn't prove the answers are worse, only that they're different. if you want to keep the model as is, turn nothing on. and if you keep only one, keep affinity: the CPU defer costs as much gap for a lot less speed. the details of the measurements are in [the glm 5.3 experiment](/en/labo/glm-53-maya-3090-3080ti/). ## 4. measure your own machine Maya has a built-in bench, the same one behind the numbers on its home page: ```sh ./maya.sh --bench ``` compare your result to their table of real machines, it'll tell you whether your setup is in the right range. for reference, the docs quote 53 tok/s on two 3090s capped at 225 W with 128gb of RAM and a Threadripper: RAM and CPU matter a lot with this model. ## what can get stuck - **startup takes several minutes**: normal, it reserves and fills the RAM. - **other programs crawl**: Maya took almost all the free RAM, that's on purpose. - **speed is low at first**: the caches fill up with what you use, it goes up as the conversation goes on. - **updating**: `git pull` then `./setup.sh`, the model isn't downloaded again. on my machine, the update from v1.1.11 to v1.1.18 alone changed nothing in speed.